-1
We have setup one of web site on server and site build in php and symfony framework, As my requirement is to prevent Google from indexing and below of my robot.txt and Is prevent using .htaccess?
User-agent: *
Disallow:
So How to to prevent it and test?
Nullpointer
Posted 2016-06-29T11:12:41.267
Reputation: 152
4
Your file should be named: robots.txt
If you can see the content of the file on line : http://youwebsite/robots.txt, then it should be working.
Otherwise you can test it with this google tool: https://support.google.com/webmasters/answer/6062598?hl=en
hyde
Posted 2016-06-29T11:12:41.267
Reputation: 41
Problem is that, get 404 file not found but file is exists, not works; How can I prevent Google from indexing in my project, I'm using 1and1.com panel, So Where to change it? -i'm confuses. pls just give a way... – Nullpointer – 2016-06-29T13:30:22.017
What file shows a 404 error? The robots.txt one? Be carefull about the 's' because in your first post you named it robot.txt – hyde – 2016-06-30T12:27:43.570
It's my bit mistake on htaccess file, So that not working but Now I have two option for this 1) robots.txt and .htaccess ( http://www.htmlremix.com/seo/block-google-and-bots-using-htaccess-and-robots-txt )for prevent indexing ; So which one should better ?
– Nullpointer – 2016-06-30T12:33:20.693The robots.txt solution is the best beaucoup every search engine will check it. Whereas in the .htaccess you will need to specify every search engine user agent. And if one is changing you will have to change the name in your .htaccess – hyde – 2016-06-30T12:38:23.987
Last time I had to do something like this was after seeing them in my website stats. So if you don't see them in a while consider job done, maybe :| – pun – 2016-06-29T11:23:15.130