0

I have purchased a domain from godaddy( mentioning here as xyz.com ). I have hosted my website on aws using elasticbeanstalk which has generated a cname (mentioning here as abc.elasticbeanstalk.com).

Now in godaddy I have done 2 configurations: 1. Forward xyz.com to www.xyz.com 2. Mapped cname www to abc.elasticbeanstalk.com

xyz.com & www.xyz.com opens properly in browser and everything seems to be fine.

The problem is search engines are displaying abc.elasticbeanstalk.com in search results for keywords related to xyz.com.

I want to keep abc.elasticbeanstalk.com hidden from all these search engine crawlers. Has anyone faced this problem before?

Manish
  • 1
  • 1

4 Answers4

1

You can set a canonical URL for the site:

<link rel="canonical" href="https://example.com/current/request/uri" />

Another possible solution is setting a custom Apache configuration to redirect visitors hitting the EB domain. This requires a custom AMI for your EB installation.

ceejayoz
  • 32,469
  • 7
  • 81
  • 105
0

The only way to correctly stop search engines from indexing your site is to add a noindex meta.

<meta name="robots" content="noindex">

To prevent only Google web crawlers from indexing a page:

<meta name="googlebot" content="noindex">

Add this meta tag to any page you don't want indexed on Google.

Read more:https://support.google.com/webmasters/answer/93710?hl=en

RonanW.
  • 419
  • 2
  • 6
-1

yes the robots file example above will block the majority of search engines from crawling your site which will result in it possibly not being indexed. This does not necessarily mean it will be completely de-indexed though, there is still a small chance it might be indexed. The only way to be sure google will not index your site is to add the nofollow meta tag.

From Googles Support Docs

enter image description here

< meta name="robots" content="noindex">

RonanW.
  • 419
  • 2
  • 6
-2

Make a file http://abc.elasticbeanstalk.com/robots.txt with the following contents:

User-agent: *
Disallow: /

This tells all search engine crawlers that you don't want them to visit the site. This has been the standard way of solving this problem for years.

You can read more on the subject at http://www.robotstxt.org.

Tero Kilkanen
  • 34,499
  • 3
  • 38
  • 58
  • 1
    But this will also stop the search engine crawlers from crawling xyz.com rite? – Manish Jul 17 '15 at 09:44
  • Ah true, didn't realize that. http://stackoverflow.com/questions/25567575/hide-cname-from-search-engines shows a similar issue and an approach to handle it. – Tero Kilkanen Jul 17 '15 at 09:50