pfSense: Possible to traffic capture the actual WAN port?

Question

The pfSense router seems to have having problems connecting to the internet. My modem, and my ISP, confirm that i have sync, but at the same time pfSense sometimes just will not connect (using PPPoE).

i want to try to debug the problem, by watching the PPPoE connection trying to get built-up, which typically is of the form:

1. PADI

pfSense broadcasts a PPPoE Active Discovery Initiation (PADI) packet out to my ISP:

DESTINATION_ADDR: ff:ff:ff:ff:ff:ff     ;broadcast mac address
SOURCE_ADDR:      00:01:02:3d:71:85     ;pfSense WAN adapter mac address
ETHER_TYPE:       8863                  ;PPPoE Discovery stage
PAYLOAD:          11090000
                  1                     ;Version always 0x1
                   1                    ;Type always 0x1
                    09                  ;Code: 0x09 = PADI
                      0000              ;Session ID: 0x0000

2. PADO

The ISP then responds with a PPPoE Active Discovery Offer (PADO) packet:

DESTINATION_ADDR: 00:01:02:3d:71:85     ;pfSense WAN adapter mac address
SOURCE_ADDR:      00:90:1a:a0:a1:f4     ;ISP's PPPoE server mac address
ETHER_TYPE:       8863                  ;PPPoE Discovery stage
PAYLOAD:          11070000
                  1                     ;Version always 0x1
                   1                    ;Type always 0x1
                    07                  ;Code: 0x07 = PADO
                      0000              ;Session ID: 0x0000

3. PADR

pfSense then requests to start a PPPoE session with the guy who responded to its broadcast by sending a PPPoE Active Discovery Request (PADR) packet to them:

DESTINATION_ADDR: 00:90:1a:a0:a1:f4   ;ISP's PPPoE server mac address
SOURCE_ADDR:      00:01:02:3d:71:85   ;pfSense WAN adapter mac address
ETHER_TYPE:       8863                ;PPPoE Discovery stage
PAYLOAD:          11190000
                  1                   ;Version always 0x1
                   1                  ;Type always 0x1
                    19                ;Code: 0x19 = PADR
                      0000            ;Session ID: 0x0000

4. PADS

My ISP then responds with a session ID in a The PPPoE Active Discovery Session-confirmation (PADS) packet:

DESTINATION_ADDR: 00:01:02:3d:71:85   ;pfSense WAN adapter mac address
SOURCE_ADDR:      00:90:1a:a0:a1:f4   ;ISP's PPPoE server mac address
ETHER_TYPE:       8863                ;PPPoE Discovery stage
PAYLOAD:          11651234
                  1                   ;Version always 0x1
                   1                  ;Type always 0x1
                    65                ;Code: 0x65 = PADS
                      1234            ;Session ID: 0x1234

...and on it goes.

i want to monitor the PPPoE session creation. i want to see if/when pfSense is broadcasting a PADI initiation packet. i want to see if my ISP isn't responding with a PADO. i want to capture the packets on the WAN interface.

Those of you who are sharp will realize the problem. i want to capture packets on the actual WAN interface of the computer, not the PPPoE "WAN" interface, which is not up yet. It's not up yet because the PPPoE connection isn't up yet.

i want to capture packets on the wire, not packets going out the PPPoE virtual connection.

Is it possible to capture packets on the WAN port, with PPPoE as the method for connecting to the internet, with pfSense?

score 3 · Accepted Answer · answered Jul 25 '10 at 06:10

Yes, you have to do it via SSH and not the GUI Diagnostics>Packet capture screen though, as it'll capture on the ng0 PPPoE interface. Just tcpdump as normal on the actual Ethernet interface being used for the PPPoE connection. Send it to file and pull it across to a machine with Wireshark for easier analysis. Something like:

tcpdump -i em0 -s 0 -w /tmp/pppoe.pcap

to grab everything on em0 to the file /tmp/pppoe.pcap

Ian Boyd · Answer 2 · 2013-08-31T17:27:35.397

To expand upon Chris's answer, the actual command i used was:

>tcpdump -i xl0 -e 'not (pppoes and ip)'

Where

-i [interface]

specifies my WAN interface, which for me is xl0

-e

include the source and destination MAC addresses, so i can see who is sending what packets to who

'not (pppoes and ip)'

is an expression that says i want to exclude ip packets inside of a PPPoE Session packet.

Note: tcpdump has options to monitor just the PPPoE "discovery" packets (PADI, PADO, PADR, PADS):

>tcpdump -i xl0 -e pppoed

This would show just the PPPoE discovery packets. i preferred the other syntax, because once the PPPoE session is established, i then get to see login/password authentication with my ISP, as well as IP/DNS/route/MTU information.

This led me to discover that pfSense is sending a PPPoE Active Discovery Terminate (PADT) packet out of the blue. That is why my link is going down - pfSense is hanging up randomly.

The PADT tag indicates the end of the session, and nobody is allowed to send anymore packets on that session. From RFC2516 - A Method for Transmitting PPP Over Ethernet (PPPoE):

5.5 The PPPoE Active Discovery Terminate (PADT) packet

This packet may be sent anytime after a session is established to
indicate that a PPPoE session has been terminated. It may be sent by
either the Host or the Access Concentrator. The DESTINATION_ADDR
field is a unicast Ethernet address, the CODE field is set to 0xa7 and the SESSION_ID MUST be set to indicate which session is to be terminated. No TAGs are required.

When a PADT is received, no further PPP traffic is allowed to be sent
using that session. Even normal PPP termination packets MUST NOT be
sent after sending or receiving a PADT. A PPP peer SHOULD use the
PPP protocol itself to bring down a PPPoE session, but the PADT MAY be used when PPP can not be used.

My next question will, of course, be

Why is pfSense hanging up randomly?

The answer is that an ATM switch inside Bell Canada's concentrator is faulty. The card would randomly reset, losing any memory of my PPPoE session.

Meanwhile, my router doesn't know its PPPoE session is no longer valid, and continues to try to communicate over that same SessionID. The concentrator, not recognizing that session, ignores any packets coming from me.

pfSense, after sensing 10 seconds of no traffic, begins sending LCP Echo-Requests, at 10s intervals, to the remote network. After 40 seconds of not getting any responses, pfSense brings down the PPPoE session and starts a new one:

PADI

The concentrator, seeing a broadcast requesting to start a session, responds; and shortly thereafter the new PPPoE session is established.

Current (faulty) ATM switch MAC:

00:90:1a:a0:a1:f4 Unisphere Solutions (was: redstone communications)

New ATM switch MAC:

to be installed 8/6/2010

pfSense: Possible to traffic capture the actual WAN port?

1. PADI

2. PADO

3. PADR

4. PADS

2 Answers2

Linked