The answer is dependent on what "signalling server" you refer to.
WebRTC is MITM-secure against untrusted relayservers, STUN and TURN servers. Those servers only help the clients to set up a p2p connection at all.
However, the channel through which SDP is done needs to be trusted.
Unlike the relayserver, SDP carries no weight, and consists only of some strings sent by the clients for the initialisation of a connection. Those are accessible to javascript, and can be sent any way (HTTP, websockets, email) between the clients.
Therefore you should use a trusted channel for doing the SDP.
The SDP exchange carries a a=fingerprint:
named hash, that authenticates the payload-bearing p2p DTLS connection to the other client. RFD 5763 contains a good description of that connection between DTLS and SDP.
You could reduce the content that needs to be trusted to this hash, but unless it serves no further purpose you are better off doing the whole SDP over a trusted channel, as you have less complexity.