xinetd read post data

Question

I'm trying to write a simple handler for a webhook using xinetd and bash. I have the trivial case working with this config for xinetd:

service github-hooks
{
    port            = 61000
    socket_type     = stream
    protocol        = tcp
    wait            = no
    user            = ubuntu
    server          = /home/ubuntu/github-hooks.sh
}

and this bash script:

#!/bin/bash
echo -e "HTTP/1.1 200 OK"

now, I want to read in the post data that is being sent in the webhook so I can do something more interesting that always returning 200.

How do I do read the post data from my bash script?

I've tried:

while read line; do
    echo "$line" >> /home/ubuntu/test
done < /dev/stdin

but that is not working for me.

Edit

Thanks to the suggestion below, I stopped xinetd and used nc to see what raw data was coming over the wire:

$nc -l 61000

and got this:

POST / HTTP/1.1
Host: <snip>:61000
Accept: */*
User-Agent: GitHub-Hookshot/375c44e
X-GitHub-Event: pull_request
X-GitHub-Delivery: 2dc1fb00-1c8e-11e6-9955-64afafb6ce32
content-type: application/json
X-Hub-Signature: sha1=45afd85b7d4312fa8ac4c56638e8e9699e2ddb36
Content-Length: 20558

{"action":"opened","number":116,"pull_request": <snip>

So, the data is being sent. Now, Knowing that there are exactly 11 lines being sent, I read exactly 11 lines:

for i in {0..10}
do
    read line
    echo "$line" >> /home/ubuntu/test
done

And, I get the same output (great success :)

POST / HTTP/1.1
Host: <snip>:61000
Accept: */*
User-Agent: GitHub-Hookshot/375c44e
X-GitHub-Event: pull_request
X-GitHub-Delivery: 2dc1fb00-1c8e-11e6-9955-64afafb6ce32
content-type: application/json
X-Hub-Signature: sha1=45afd85b7d4312fa8ac4c56638e8e9699e2ddb36
Content-Length: 20558

{"action":"opened","number":116,"pull_request": <snip>

Maybe I should just read 9 lines then use the Content-Length parameter to read the rest? I still don't really understand what's going on so any information would be very helpful.

Answer 1

OK, I never did figure out how to read while there is input but I did notice that there is a length in the header. So, I parse out the length and then read that number of characters:

#!/bin/bash
for i in {1..9}
do
  read line
done

array=(${line})
length=${array[1]}

read line
read -n ${length:0:-1} line

echo "$line" >> /home/ubuntu/test

echo -e "HTTP/1.1 200 OK"

And, now I've got the post data that github is sending over.

Answer 2

To read standard post data for an xinetd http request, a program just reads standard input.

I have written a bash script that reads and parses the http headers as passed in through standard input. Useful when called from xinetd. I created this as a starting point for someone that wants to create an xinetd service that can recognize HTTP REST-like calls. https://github.com/rglaue/xinetd_bash_http_service

But essentially, reading the HTTP input (Headers and POST data) in an xinetd service looks something like this:

# Read HTTP Headers, line by line
# We're looking for the Content-Length HTTP header
: ${HTTP_CONTENT_LENGTH:=0}
while read -t 0.01 line; do
    # If the line is empty, stop reading headers
    if [ -z "$line" ]; then break; fi
    # Read each HTTP Header
    if echo "${line}" | grep -qi "^some-header:"; then
      # do something here
    elif echo "${line}" | grep -qi "^Content-Length:"; then
      HTTP_CONTENT_LENGTH="$(echo "${line}"|cut -d" " -f 2-)"
    fi
done
# Next read from standard input into the HTTP_POST_DATA variable
# (This assumes the request is a POST)
while IFS= read -N $HTTP_CONTENT_LENGTH -r -t 0.01 post_buffer; do
    echo "Reading in the HTTP Post Data"
    HTTP_POST_DATA="${HTTP_POST_DATA}${post_buffer}"
    if [ ${#HTTP_POST_DATA} -ge ${HTTP_CONTENT_LENGTH} ]; then
      # Make sure we stop reading, since we have read enough.
      break;
    fi
done

Why use IFS=

• The -r flag already reads the data in without stopping at delimiters
• IFS is the Internal Field Separator used by bash to determine how words are split, and how lines are split.
• We empty IFS to make sure bash does not change our expected results from the raw read read -r.

For the read parameters:

• The -t 0.01 flag defines the read timeout in seconds (0.01 sec)
• The -r flag reads the data in raw, which ignores the delimiters \r\n
• The -N <num> flag reads <num> characters.

Answer 3

literally cat, or xargs, or grep . or anything else that reads from stdin.