After adding new subnet ip to new interface, the new ip was being used in previous interface em1 like so: ifcfg-em1:3 ... when I moved the IP to ifcfg-em2 and issued the following command ifdown ifcfg-em1:3
I lost my connection to ifcfg-em1 until i rebooted the machine. then I was able to get into the OS using ifcfg-em1 and ifcfg-em2 was useable also.
Just wondering why did I have to reboot?
Thanks