I want to be able to do this thing where I can extract path andor hostname (preferably both) of all the embedded requests (HTTP
and HTTPS
) going within a webpage. I have tried using tshark
for this, but when I use this command:
sudo tshark -i etho0 -Y 'http.request.method == "GET"' -T fields -e http.request.method -e http.request.uri
it displays the right embedded requests paths for HTTP
, but it doesn't really display anything in the case of HTTPS
requests, only invisible blocks of lines between sequences of numbers. I understand that HTTPS
requests are encrypted and therefore, I may not be able to see the exact path. I looked into Wireshark
for this, and came across links where you can direct the environment variable SSLKEYLOGFILE
to a text file to which your browsers (Firefox/Chrome) will then subsequently log your private keys for each session. But that doesn't work in Firefox from version 48 onwards. For tshark as well, I cannot figure out how to actually obtain the private key of the server, and the entire process is not very clear to me, even after going through dozens of links. Could anyone please point in the right direction, especially with regards to how I can programmatically obtain (on the terminal) all the HTTPS request paths in a page similar to the tshark command for HTTP above (or the way we can clearly see the path in Firefox/Chrome's developer tools.)
Thanks!