Under what circumstances does the scp command fail?

Question

I have two linux(redhat 5.6, 64 bit) servers(hp proliant 380p g8), server1's eth0,eth2 bonded to bond0.Similarly server 2's eth0,eth2 bonded to bond0. server1's eth0 connects to cisco(3560 catalyst) switch1's port3 and eth2 to switch2's port3. for server2 it's eth0:port15(sw1),eth2:port15(sw2)

All network elements are in the same

    network:10.2.3.32
    netmask-255.255.255.224
    gateway-10.2.3.33

Switches are interconnected (using cross cables) at 12-12 port and 24-24 port. All physical connections up, cables tested, LEDs of ports on. I pinged from server1 to switch1,switch2,server2 and it worked fine. It also worked fine other way server2-switch2-switch1-server1.

I have a customized product CD from which I have mounted the contents like this(on both servers as root):

    mount /dev/cdrom /mnt
    cp -rp /mnt/apps/auto_install /apps/.
    umount /mnt
    eject

Now I have a file /apps/auto_install/airwide/install_apps_env.sh in server1 which i want to scp to /apps/auto_install/airwide/ of server2 after making changes in the file. These are the details of the directories and the file:

    drwxrwxrwx 16 root root 4096 Mar 8 16:03 apps
    drwxrwxrwx 16 1015 200 4096 Mar 8 16:03 auto_install
    drwxrwxrwx 16 1015 200 4096 Mar 8 16:03 airwide
    -rwxr-xr-x 16 root root 3458 Mar 8 16:03 install_apps_env.sh

Before doing scp i did all ping tests for testing interconnections between the network elements and it was ok. Also when i try inter server ssh its working. I tried scp for other files from server1 to server2, and it worked. But when i try this as root :

    scp server1:/apps/auto_install/airwide/install_apps_env.sh server2:/apps/auto_install/airwide

the scp doesn't happen. So I kill it and when i recheck ping test from

    server1-switch1(doesn't happen)
    server1-switch2(doesn't happen)
    server1-server2(obviously doesn't happen)

But ping test between:

    server2-switch2-switch1(happen in any order)

i.e they are in network interconnectivity, but server1 is detached from network connectivity

Any help on why scp is not happening for only that particular file from server1 to 2, and why server1 connectivity is getting detached from swithces and server2?

Thanks in advance