2

I created a simple python script to capture HTTP request. I captured HTTP Requests with my python script.

But I can't capture HTTPS requests. I just want to capture information about URLs.

I used scapy to sniffing. For example:

from scapy.all import IP, sniff
from scapy.layers import http

def tcp_ayikla(paket):
    if not paket.haslayer(http.HTTPRequest):
        return

    http_katmani = paket.getlayer(http.HTTPRequest)

    ip_katmani = paket.getlayer(IP)
    print '\n{0[src]} IP adresinden {1[Method]} {1[Host]}{1[Path]} sitesine ziyaret'.format(ip_katmani.fields, http_katmani.fields)

sniff(filter='tcp', prn=tcp_ayikla)

So, I have questions about sniffing.

1-) Why I can't get HTTPS requests?

If I use netstat like this:

netstat -ap | grep http

I can see HTTPS Requests

2-) Is there any way in another programming languages?

Because I am trying to log HTTP requests.

What did I do?

  • I tried parsing netstat output with python. I don't want to this. Because it's not pure Python.
  • I tried with scapy. I couldn't catch HTTPS requests.
  • I tried to read the URL informations from SQLite. I don't want to this. Because it's not about network.

What should I do?

Should I give up?

dr jimbob
  • 38,768
  • 8
  • 92
  • 161
Ali
  • 151
  • 1
  • 1
  • 6

2 Answers2

5

HTTPS runs on top of TLS. That is the entire HTTP request and response is encrypted, and an application eavesdropping on the network traffic will not be able to decrypt the traffic.

That said, you could log the IP addresses (especially those connected to servers on port 443 - HTTPS) as the IP layer is not encrypted.

This is what your netstat command does. It looks for TCP connections on your network card, and notes which ones are connecting to port 443 (well known protocol for https) and observes the IP address of the HTTPS servers you are connecting to.

I should note that if you capture encrypted network traffic, it is possible to decrypt such captured traffic if you save the SSL key from your browser. See here for instructions how to decrypt SSL eavesdropped network traffic with wireshark when you log your SSL keys; you should be able to do something similar with scapy.

Paolo Celati
  • 81
  • 7
dr jimbob
  • 38,768
  • 8
  • 92
  • 161
1

There are multiple techniques one can use to sniff HTTPS traffic. A few weeks ago I wrote a man-in-the-middle application with asyncio for easy viewing of your internet's traffic (either HTTP or HTTPS). You can find the project here; mitm: https://github.com/synchronizing/mitm

In essence, if you imagine a normal client connection looking like so:

client <-> server

mitm creates a man-in-the-middle proxy that acts as the server for the client, and as the client for the server:

client <-> mitm (server) <-> mitm (emulated client) <-> server

The mitm (server) and mitm (emulated client) can then spit out the requests happening between the client and the server.

Felipe Faria
  • 111
  • 2