1
I set up the following pipeline. This allows me to see the raw HTTP requests and responses that pass through the pipeline.
Note: I use BSD nc
below (this is what comes with macOS and AmazonLinux), but I've also tried nmap's ncat
, which can take the same arguments, and socat
1.7.3.2 The behavior is the same.
socat
takes different arguments, but the pipeline is the same. I'll add the socat
one-liner only, as the explanation would be the same.
One-liner:
mkfifo res; nc -kl 8888 < res | tee /dev/stderr | nc google.com 80 | tee res
OR
mkfifo res; socat tcp-listen:8888,reuseaddr,fork - < res | tee /dev/stderr | socat - tcp:google.com:80 | tee res
Explanation:
mkfifo res; \ <-- Make a named pipe file with the name 'res'.
nc -kl 8888 < res \ <-- Open a socket listening on all interfaces at TCP port 8888. Use the 'res' pipe as input.
| tee /dev/stderr \ <-- Copy the request to stderr, so it's emitted by the terminal. stdout will be used as input for the next netcat.
| nc google.com 80 \ <-- Resolve google.com, connect to it on TCP port 80, and send what was received on stdin. The response goes to stdout.
| tee res <-- Copy the response to the 'res' pipe.
I run this pipeline in one terminal (the pipline terminal), and use curl
to make the request in another terminal (the curl terminal):
curl --resolve google.com:8888:127.0.0.1 http://google.com:8888
On both macOS and Linux:
The curl terminal correctly receives and displays the response. The pipeline terminal displays the request AND response.
On macOS (macOS Mojave 10.14.3):
The pipeline continues running. I can send another request with curl
, and can see the request in the pipeline terminal and the response in both terminals.
On Linux (AmazonLinux 4.1.13):
The pipeline continues running. curl
hangs if I send another request, and I can't see the request or response anywhere.
Versions:
macOS
$ uname -a
Darwin ch007837.na.webmd.net 18.2.0 Darwin Kernel Version 18.2.0: Thu Dec 20 20:46:53 PST 2018; root:xnu-4903.241.1~1/RELEASE_X86_64 x86_64 i386 MacBookPro15,1 Darwin
$ curl --version
curl 7.64.1 (x86_64-apple-darwin18.2.0) libcurl/7.64.1 SecureTransport zlib/1.2.11
Release-Date: 2019-03-27
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile libz NTLM NTLM_WB SSL UnixSockets
$ bash --version | head -1
GNU bash, version 5.0.3(1)-release (x86_64-apple-darwin18.2.0)
$ ncat --version
Ncat: Version 7.70 ( https://nmap.org/ncat )
I'm not sure about nc
since it doesn't have a --version
flag, but it came with the OS. There's this string in the binary though:
$ strings "$(which nc)" | tail -1
@(#)PROGRAM:nc PROJECT:netcat-42.200.1
Linux
$ uname -a
Linux ip-10-200-38-72 4.1.13-19.31.amzn1.x86_64 #1 SMP Wed Jan 20 00:25:47 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
$ curl --version
curl 7.40.0 (x86_64-redhat-linux-gnu) libcurl/7.40.0 NSS/3.19.1 Basic ECC zlib/1.2.8 libidn/1.18 libssh2/1.4.2
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz UnixSockets
$ bash --version | head -1
GNU bash, version 4.2.46(1)-release (x86_64-redhat-linux-gnu)
$ ncat --version
Ncat: Version 5.51 ( http://nmap.org/ncat )
$ yum list nc | tail -1 | sed 's/ \+/ /g'
nc.x86_64 1.84-24.8.amzn1 @amzn-main
Note: on my Kubuntu 18.04.2 LTS the code seems to work like in your "On macOS" case. – Kamil Maciorowski – 2019-04-02T17:49:23.200
Thanks. I'm trimming down the list of variables. How are you running Kubuntu? AWS/Virtualbox/Bare metal? Do you know which
nc
? – Josh Cooley – 2019-04-03T02:01:40.253Kubuntu is directly installed on my laptop. Currently my
nc
is fromnetcat-openbsd 1.187-1ubuntu0.1
(it doesn't seem to support--version
or such). I get the same "On macOS" behavior withncat
instead ofnc
.ncat
is fromnmap 7.60-1ubuntu5
,ncat --version
yieldsNcat: Version 7.60 ( https://nmap.org/ncat )
. – Kamil Maciorowski – 2019-04-03T04:36:33.773