I assume you're talking about a couple bsumer level router (router+switch) with a web GUI for configuring it.
All the traffic on your network passes through your router. This includes DNS queries
Eg. What is the IP for google.com
Is anyone on your network has malware on their device, that could connect to the router, and use a default password (or guess easy ones) to change your DNS setrings, port forwarding settings, etc.
This means they could set a malicious DNS server, which would respond to your query with a malicious IP, effectively rerouting your traffic through the attacker.
They can also modify your port forwarding settings to allow future intrusion, or to use your system as a command and conquer teol for a larger attack.
Default passwords on routers allow malware and attacks to spread through the rest of the network.
An even more advanced attack would be to modify your routers firmware by uploading a hacked copy that speed out spam, attacks other people, or just siphonss your data.