I'll assume you're talking about BitTorrent, rather than Gnutella or other P2P protocols.
There's no standard port for BitTorrent traffic, so you're going to have to do some digging.
First, BitTorrent talks to a set of trackers. This communication is done over HTTP, and will have (at least) the following headers:
info_hash
- a URL-encoded hash of the metadata (.torrent) file.
peer_id
- a URL-encoded random string.
port
- the port number that the client is listening on - useful for filtering later!
However, sometimes this is done over HTTPS, so it makes it a little more difficult to detect. You should look for any traffic occurring on ports 80 or 443.
Next, you're going to want to look for traffic between peers. The peer handshake nicely provides an easy-to-search header:
<pStrLen><pStr><reserved><infoHash><peerId>
The pStrLen
field tells us how long pStr
is in bytes, and pStr
is the protocol identifier. Normally, this is "BitTorrent protocol", so pStr
will be 19. You can therefore search for packets that start with 13 42 69 74 54 6f 72 72 65 6e 74 20 70 72 6f 74 6f 63 6f 6c
.
Some BitTorrent clients support encrypted client-to-client protocols, but I'd guess that they're not going to be that rough on you for a forensics challenge.
Source: http://wiki.theory.org/BitTorrentSpecification
It's possible to identify encrypted BitTorrent traffic using passive statistical analysis. Erik Hjelmvik and Wolfgang John showed techniques for analysing encrypted communications protocols, including BitTorrent. Table 5.4 in their paper shows the confidence level to be 0.965 for MSE (encrypted BitTorrent). Section 6.1 explains some details of the analysis, and mentions a few properties that can be identified with relative ease.