Presumably your "client-server communications" occur over a network, so a network snffer, like Wireshark, is probably a good start. Hopefully the particular protocol that your application is using is one that Wireshark can decode. If not, then you'll need to save the captured traffic and analyze it with a tool that can decode the protocols in the stack, preferrably up to layer 7. (If any of the protocol stack along the way is proprietary you're probably in trouble for anything other than traffic analysis.)
If your protocol is HTTP(S) based, Fiddler might also be up your alley, though it's very easy to encapsulate an ugly, proprietary protocol inside HTTP(S) such that it's as incomprehensible as a binary protocol (Sage "ACT!" HTTP-based replication, I'm talking to you).
It's difficult to respond to your "dummy proof" request. To use a tool like Wireshark effectively (especially some of the nice advanced features like following TCP streams or detecting anomalies) you really need to have an understand of the protocol layer stack and what's going on. Until computers can get to the point of being "intelligent" there's no good software substitute for human knowledge.
Edit:
I've done some analysis for Customers who were experiencing "slowness" issues with opaque, proprietary layer 7 protocols (hello, Bentley "AutoPlant", I'm talking to you) and I can say that there's some good value in even simple traffic analysis (who talks to who when, and how much). The insinuation was that "the network" was being a bottleneck, so a little sniffing of the traffic in combination with running a "Process Monitor" trace on a client PC produced some nice graphs that showed comparisons of client CPU load, server CPU load, network I/O, and back-end SAN I/O. In the end, this was all just traffic analysis of a sort, but it helped identify a nasty "looping" bug in the client software that had nothing to do with "the network".
As another example, there was a question on here yesterday about slow logons with roaming user profiles where the caching behaviour of roaming user profiles Windows XP came into question. Not having access to the Windows source code (as I so often don't), I opted to perform traffic analysis to determine what the caching behaviour was. The client/server protocol, SMB, isn't opqaue, but I treated it as such since what I was really looking for was a data transferred metric.
How deep down the rabbit-hole you go just depends on what you're looking for.