Why is ping so much faster than UDP sockets?

Question

I'm trying to benchmark how fast UDP sockets are on my local network.

By pinging my local IP address, I obtain a round trip time of ~80 microseconds.

I wrote a minimal UDP server and client to benchmark latency as follows. The client and server are running on the same host, using the same network interface. The client sends a timeval to the server via datagram sockets, and upon receipt, the server immediately takes the difference between the current timeval and the received timeval. Doing this, I obtain ~110 microseconds for a one-way trip,for a total of around ~220 microseconds RTT.

Why is ping so much faster than my program?

cc udpserver.c -o udpserver
cc udpclient.c -o udpclient
./udpserver 4321
./udpclient 127.0.0.1 4321

udpserver.c

#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>

    void error(char *msg)
    {
        perror(msg);
        exit(1);
    }
    int main(int argc, char *argv[])
    {
         int sockfd, newsockfd, portno;
         struct timeval tval_recv, tval_now;
         struct sockaddr_in serv_addr;
         int n;

     /* Check arguments */
     if (argc < 2) {
         fprintf(stderr,"ERROR, no port provided\n");
         exit(1);
     }

     /* Set up socket */
     sockfd = socket(AF_INET, SOCK_DGRAM, 0);
     if (sockfd < 0)
        error("ERROR opening socket");

     /* Setup socket address for server */
     memset((char *) &serv_addr, '\0', sizeof(serv_addr));
     serv_addr.sin_family = AF_INET;
     serv_addr.sin_addr.s_addr = INADDR_ANY;
     serv_addr.sin_port = htons(atoi(argv[1]));

     /* Bind socket to socket address */
     if(bind(sockfd, (struct sockaddr *) &serv_addr,
              sizeof(serv_addr)) < 0)
              error("ERROR on binding");

     while(1) {
       n = recvfrom(sockfd, &tval_recv, sizeof(tval_recv), 0, NULL, NULL);

       if(n < 0)
         error("ERROR in recvfrom");

       gettimeofday(&tval_now, NULL);

       printf("%ld.%06ld\n", tval_now.tv_sec - tval_recv.tv_sec,
           (long int)(tval_now.tv_usec - tval_recv.tv_usec));
     }

     return 0;
}

udpclient.c

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netdb.h>

void error(char *msg) {
    perror(msg);
    exit(0);
}

int main(int argc, char **argv) {
    in_addr_t s_addr;
    int sockfd, portno, n;
    struct sockaddr_in serveraddr;
    struct timeval tval;
//    struct timespec ts_current;

    /* Check arguments */
    if (argc != 3) {
       fprintf(stderr,"Usage: %s <hostname> <port>\n", argv[0]);
       exit(0);
    }

    /* Create the socket */
    sockfd = socket(AF_INET, SOCK_DGRAM, 0);
    if (sockfd < 0)
        error("Error opening socket");

    /* Set the socket address */
    s_addr = inet_addr(argv[1]);
    portno = atoi(argv[2]);
    memset((char *) &serveraddr, '\0', sizeof(serveraddr));
    memcpy(&serveraddr.sin_addr.s_addr, &s_addr, sizeof(s_addr));
    serveraddr.sin_family = AF_INET;
    serveraddr.sin_port = htons(atoi(argv[2]));

    /* Send packets */
    while(1) {
      gettimeofday(&tval, NULL);
//      clock_gettime(CLOCK_MONOTONIC, &ts_current);
      n = sendto(sockfd, &tval, sizeof(tval), 0, (struct sockaddr *)&serveraddr,
        sizeof(serveraddr));
      if(n < 0)
        error("ERROR in sendto");
      sleep(1);
    }

    return 0;
}

Answer 1

Wild guess:

ping is a tool using the icmp protocol. icmp sits on layer 3, the network layer, of the OSI model.

udpclient.c is a tool using the udp protocol. udp sits on layer 4, the transport layer, of the OSI model.

With each layer, additional data will be added to the raw data. As a basic example: an icmp packet contains a source IP address and a destination IP address. A udp datagram contains all that plus e. g. the UDP port information. This port information needs to be interpreted by a higher level in the network stack.

So UDP packets have to crawl up one level more in the network stack. One could imagine taking the stairs to the 4th floor instead of the 3rd floor. It will take longer, however much longer than 30µs.

I believe that those 30µs consist of

• data size icmp vs udp
• NIC / driver / CPU time for encapsulating and decapsulating
• CPU time which udpclient.c uses