Googlebot blocked by robots.txt

Question

So recently I've tested my site with Google mobile-friendly test and the main loading issue was "Googlebot blocked by robots.txt" My robots.txtdoes allow Google bot I think?

What do you think guys? What's the problem here?

Answer 1

I am pretty sure your robots.txt file IS allowing everything on your domain, and everything is working correctly.

What appears to be happening is that you have resources from other sites imbedded in your page, and THESE RESOURCES ONLY are not being spidered by Google, because the robots.txt for those domains - which are outside your control, and probably should not be indexed anyway - have robots.txt files to disallow them.

If you have a look at the report and click on the highlighted "robots.txt" next to each line - it shows the robots.txt file for the URL - and these are outside your domain, and entirely different robots.txt