Goto 'Network Connections' right click on the connections not in use and click 'disable'.
That will turn off the hardware for the connection and make the taskbar icon disappear.
I typically turn off all of the connections that are not in use because it also has the added benefit of saving power.
Note: If you want to add 'Network Connections' to the 'Start Menu', right click on the 'Start' button, goto 'Properties', and find the option in the menu to add a link to the 'Network Connections' folder in the 'Start Menu'.
SideNote: You can also turn off your cd drive in Control Panel->System->Hardware find it in the menu, right click and disable. It's especially nice if you have a cd drive that is unnecessarily loud/obnoxious.
2Which version of windows are you using? – BloodPhilia – 2010-06-14T16:01:08.160
XP, but I imagine the answer should be relatively the same for anything from 2K3, and beyond. – BIBD – 2010-06-14T17:49:26.543