how do I do ldap authentication with wget?

Question

I'm trying to crawl an old Reviewboard server I own so I can create a static copy and archive it, and I'm doing this using wget per recommendations from a few people, using the recommended way of crawling and downloading a server like so:

wget --recursive --no-clobber --page-requisites --html-extension --convert-links --domains $DOMAIN --http-user $USERNAME  --ask-password $THE_SERVER

However, the server requires an LDAP authenticated login, so all I ever get is the login page and some related files. The same issue occurs if I just do --user rather than --http-user.

My question is can I do LDAP authentication with wget, and if so, how?

Since I'm sure these comments will come up:

1. I've made a JSON archive using rb-tools already, but would strongly prefer a static web copy.
2. I have the option of allowing anonymous read-only access because I'm an admin on the server, but I'd like to avoid that if possible.

Answer 1

LDAP is not an authentication mechanism – it is merely an account storage backend; it's essentially the same as storing accounts in MySQL or such. Thus, there is no such thing as "LDAP authentication" as far as HTTP clients are concerned.

The actual mechanisms HTTP has are "Basic" or "Digest" for password-based logins, or "Negotiate" for Kerberos-based logins; the --http-user option would work if your website used one of those.

However, your website most likely uses a forms/cookies-based login page, which to HTTP clients is completely indistinguishable from a regular redirect to a regular webpage. Wget will not be able to automatically recognize and fill in those forms.

If that's the case, what you could do is log in via your regular web browser, then copy the necessary session cookies to wget's "cookie jar" file (or possibly pass them directly using --header). Then the website would recognize you as being logged in.

If you open "Developer Tools" → "Network" in Chrome or Firefox, each network request will have a "Copy as cURL" option which gives you a full curl command line. Find the specific --header … option which sends your session cookie, and give it to wget.