Wget download 403 Forbidden or 503 service unavaible

First of all I need to say that there are a lot of question about this error but I almost tried all but still I have.

url="https://cdn59.my.mail.ru/hv/55045204.mp4?sign=2cca46c09181dc952b3140f2ea3852d90cc2a19a&slave[]=s%3Ahttp%3A%2F%2F127.0.0.1%3A5010%2F55045204-hv.mp4&p=f&expire_at=1438858800&touch=1436155554&region=76"

When I use wget $url it gives me error 503 service Unavaible.

Then I added my user agent like wget -U 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.76 Safari/537.36' $url

NOW .It gives me 403 Forbidden error. I also tried header or --no-check-certificate and their combinations but still I have some annoying error. (the url has some metacharecters so I used either single quote or duble quote.)

It is interesting that I can download it from all browser like Firefox,chrome,etc.

What can be problem with my link ?

NOTE: I always refresh the url via getting json file from wget or browser. This is referer json url videoapi.my.mail.ru/videos/mail/pasha.44444/video/_myvideo/397.json

wget

makgun

Posted 2015-08-06T08:21:37.133

Reputation: 305

Answers

You need to login to the service and the session information is probably held in a cookie. When you use wget you are not supplying (or gaining and using) the cookie required to have the session information.

Something along of lines of the answer on stackoverflow may help.

albal

Posted 2015-08-06T08:21:37.133

Reputation: 1 155

There is no way to download it using wget or curl or axel and so on? – makgun – 2015-08-06T08:32:58.997

Yes you can use wget to save and load cookies. I updated my answer with a link to the first hit from my search. There is also the manual for wget if you need more details.

– albal – 2015-08-06T08:36:53.393

I wondered that I need a username and password for the site? I can download it via browser without any username or password. – makgun – 2015-08-06T08:45:42.233

1@albal You write an answer with no example. He even gave his URL.. Why don't you write an example, for his URL or any URL, with an example user/pass, else state that you're not sure – barlop – 2015-08-06T09:01:12.177

@albal Please give me an example what I need to write in command line or script? I don't have any username or password for this site but I need to download it via terminal not using browser because I am writing a small script to download video in one-click.(I know I am not professional but I write it for vk and dm) – makgun – 2015-08-06T09:24:59.747

@makgun maybe it only works in that browser where you once logged in. If I try your URL in my browser it doesn't work. And if a browser can't download it then wget can't.So maybe u need to log in at one link, then access your link. Like how when you log in at the yahoo mail screen, then u can access a link that goes straight to your email. But note that that is not an HTTP style user/pass screen. So I doubt wget can do it but we'll see what albal thinks since he wrote an answer. Maybe you can somehow get the cookies u have in your browser, but make them accessible to wget and then DL ur link. – barlop – 2015-08-06T10:14:14.010

@barlop Please first go to site and copy the new url and try again it on your browser.

– makgun – 2015-08-06T10:16:51.927

@makgun that works and in wget no user/pass http://pastebin.com/raw.php?i=7NjLFBZM

– barlop – 2015-08-06T10:18:57.603

I dont have any issue with getting json file THE PROBLEM is DOWNLOADING video – makgun – 2015-08-06T10:21:18.040

Let us continue this discussion in chat.

– barlop – 2015-08-06T10:22:58.243

1Still playing around with it. If you open a browser in private mode (incognito) and try to load the video URL you will get 403. If you then load the .json file a cookie will be set then you can download the video through the video URL. However, it would seem that wget or curl does not present itself as browser to the webserver. This may be because of Access-Control-Allow-Origin header. I think the only way to download this file is to use browser impersonation/simulation like mechanize in python. If you are happy to have a solution in python then I can attempt an example. – albal – 2015-08-06T10:26:33.233

1I was not successful in getting mechanize to impersonate a browser to be able to download the video file. I would think mail.ru don't want their content leeched and they seem to be doing a good job of preventing that from happening :-) – albal – 2015-08-06T13:35:12.563

@albal they prevents to download contents of video ? And in python it is not possible to crack the security or it is so hard work? – makgun – 2015-08-07T07:20:57.080

@albal see my answer – makgun – 2015-08-18T21:11:44.630

@barlop see my answer – makgun – 2015-08-18T21:11:48.800

I want to say that Finally,I successed download it from wget despite of Access-Control-Allow-Origin header.

#!/bin/bash
link="http://api.video.mail.ru/videos/mail/carpath1a/_myvideo/289.json"
name="BlaBlaBla"
mr480=$(printf "%s\n" "tmpvar=\$(wget -qO- --save-cookies=cookies \"\$link\" | perl -ne 's:.*\"key\"\:\"(.*?)\".*\"(http.*?)\".*:\$1\t\$2: && print') ; linktemp=\$(echo \"\$tmpvar\"|grep 480p|awk '{print \$2}') ; wget --retry-connrefused --waitretry=1 --read-timeout=20 --timeout=15 -t 10 --continue -nv --show-progress --load-cookies=cookies \"\$linktemp\" -O \$name-480 ; while [[ \$? = 8 ]];do wget --retry-connrefused --waitretry=1 --read-timeout=20 --timeout=15 -t 10 --continue -nv --show-progress --load-cookies=cookies \"\$linktemp\" -O \$name-480 ;done")
eval "$mr480" 
rm cookies

After 2-10 times repeat via while loop it starts download. I couldnt understand why it sends 503 or 403 but I know this two type errors return with code 8 so I used while loop. NOTE: Dont use && instead of ; because if you use && it wont run while loop so it will fail and finished the script.

makgun

Posted 2015-08-06T08:21:37.133

Reputation: 305