The OSI layers are a model which was not meant for IP but for an older, competing protocol. IP does not fit well in that layer, especially when envisioning SSL (which must be both in layer 6 and layer 4, which is logically impossible). These layers are just confusing, so don't use them.
IP restrictions means that your server will refuse to pursue connections which are tagged with the wrong IP address. This does not protect the data against eavesdroppers.
Let me take an analogy: suppose that your connections are good old letters, with a destination address, and a sender address. The envelopes are transparent. The equivalent of IP restrictions is when the recipient refuses to even open, let alone respond to, envelopes which are not tagged with a "known sender" address. But nobody guarantees that the sender address written on the envelope is genuine; sending a letter with a fake sender address is easy. Of course, if I, an attacker, send a letter with your address as alleged sender, I won't see the response (unless I plunder your mailbox). But the recipient may have already have acted upon the letter contents, believing them to originate from you. Moreover, since the envelopes are transparent, every postman may read the contents of the letters.
For real protection, you need opaque letters, entrusted to armed conveyers in an armored vehicle. In the Internet world, this is called SSL (aka HTTPS in a Web context).