How to login to website and auto-refresh page every X seconds from terminal?

Question

I am trying to login to a particular website, then proceed to a particular page on that website and auto-refresh every X seconds.

Here's what I have so I can login:

curl --data "Username=xx&Password=xx&Login=Login" [url that login form submits]

Here's what I have to auto-refresh the web page:

while true; do wget -qO- https://www.website > /dev/null; sleep 60; done

1. login to website
2. go to page to be auto-refreshed
3. auto-refresh every X seconds

How can I combine these codes (or similar functioning ones) so that they can carry out the task I need in a loop until cancelled with ie. CTRL + C?

Answer 1

The following wget code will make you appear to be online as long as you have your own cookie.txt file of when you were logged in. You can obtain a cookie.txt by pressing this extensions button while on the website you are trying to appear to be online. It will generate the cookie file to be used with the wget command.

https://chrome.google.com/webstore/detail/cookiestxt/njabckikapfpffapmjgojcnbfjonfjfg

This following code needs to be entered into bash terminal every 60 seconds, or however frequent you want the site to see that you have come online. I am currently running it within my crontab every 5 minutes. Most sites have a 5-10 minute timeframe of keeping you online within the system of the website, so this would only need to be ran once every 5-10 minutes, depending on the site and your personal preference.

wget -x --load-cookies ~/.config/cookies/website.txt "https://www.website.com/dashboard" >/dev/null

Answer 2

The issue you are having relates to (a) using 2 different programs to log in and keep getting updates, and more relevantly (b) you are not keeping track of the session (ie using cookies).

CURL is generally considered more flexible and less abstracted then wget, so it would make sense to use this. Try creating a file along the following lines

#! /bin/bash

curl -b cookiejar.txt --data "Username=xx&Password=xx&Login=Login" '[url that login form submits]'

while true
do
     curl -b cookiejar.txt 'https://www.website...'  >/dev/null
     sleep 60
done

The above code might require a bit of massaging depending on your site, but the parts I tested worked for me on the site I tested. The magic is the "-b cookiejar.txt" on each line which links the login and subsequent connections (using the standard cookie/session mechanism built into the protocol).

Note the ' characters surrounding the https requests. This may be required so that the command line is not misinterpreted.

You will of-course, need to mark the file as executable - using a command like chmod 755 filename - to get it to run.