2

One of our competitors recently shut down. Instead of just taking their site down, they decided to redirect all their traffic to us with a 302 redirect. All of the traffic is redirected to us with the original path from their site, resulting in 404 pages. Most of the arriving traffic is directly from Google search results that still point to the old site.

Goolge now thinks we have 10s of thousands of 404 pages, as indicated by Google webmaster tools. It also provides a very bad user experience for those people arriving on our site's 404 page when they expected to get to another site, which makes us look like we hijacked the original site.

The incoming requests don't have any particular identifying detail what we can use to pinpoint they came from that redirect, as far as I can tell. The HTTP referer is from the originating source (typically Google).

Is there a way stop this from happening? should we try and find their hosting provider and get them to take it down? is there anything on our side we could do prevent this?

Eran Galperin
  • 629
  • 1
  • 5
  • 8
  • 2
    While this is on topic here, you may get better answers at our sister site [webmasters.se]. Personally I'd just redirect them all to a special landing page and see if I could gain a few new customers... – Michael Hampton May 16 '14 at 19:46
  • Take advantage of all the free traffic and redirect them to a special page for ex-competitor customers. They are basically giving you free business...why turn that away? – Grant May 16 '14 at 19:49
  • Because we can't identify that traffic specifically (it looks like it's arriving from search engines), we cannot do that. Also, that won't fix the massive increase in 404 pages detected by Google on our site, which is not a good indicator – Eran Galperin May 16 '14 at 20:34
  • Best idea I have is to contact their DNS provider, explain the situation, and ask them to shut down their DNS. Maybe you could sue them, although I'm not sure what the grounds would be. Contact your lawner. – Barmar May 16 '14 at 20:44
  • If the DNS provider is cooperative, they could point the name at your server. Then you would have a host header to go by. If you did not already do so, you should check if that is actually what they did already. But they not be cooperative. You could also try to buy the domain of them, but if you cannot agree on a price, then things start getting messy, and you might have been better off never asking. – kasperd May 16 '14 at 20:56
  • You can already identify that traffic reasonably well if the other site used a different scheme for constructing its URLs. – Michael Hampton May 16 '14 at 21:02
  • There are some schemes we recognize, but more are added every day. Is there no technical way to detect a 302 redirect into your site? – Eran Galperin May 16 '14 at 21:20
  • @EranGalperin There is no information in the HTTP request sent by the client, which will indicate, that it has followed a redirect to get to your site. The only way you could know that there was a redirect would be for you to download the page mentioned in the referer and see that there is no direct link to your site. That will of course not work if the referer header is absent or it is pointing to a dynamically generated page. – kasperd May 16 '14 at 21:30

2 Answers2

0

I don't think there are any technical means you can use to stop it. Instead I suggest you do the following:

  • Create a webpage explaining what happened to your competitor. Explain that it was your competitor's decision to redirect the traffic. And explain that you would be happy to welcome the visitor as a new customer.
  • On your own 404 page create a link with a wording such as "Looking for <name of competitor>" which points to the page with a full explanation.
  • Take the link of 404s that Google has generated for you and any that looks like it was due to a redirect from said competitor you create your own redirect to the explanation page as long as that redirect does not break your own site in any way.
kasperd
  • 29,894
  • 16
  • 72
  • 122
  • We don't want to put such a disclaimer on our general 404 page. We're are still looking for a technical solution to identify that traffic, not based on the HTTP referer, if possible. – Eran Galperin May 16 '14 at 20:38
  • The referer header will not tell you anything about the intermediate redirect, so it is not very useful. You could go and fetch that URL, but the content of that URL could depend on a POST parameter, so you may not find the direct or indirect link to your page. But the only two pieces of information you have is the referer and the nonexistent URL on your own page. And the referer might not always be there. – kasperd May 16 '14 at 20:49
  • You can still build a list of nonexistent URLs on your own site, where the users arrive. You don't need to modify your 404 page for that. The modification of the 404 page was only intended for those URLs you had not yet discovered. You can discover more URLs to include on the list, by looking through your logs. And you can manually inspect the referer URL, if you think that will help you know if it had a link to your competitor's site. – kasperd May 16 '14 at 20:52
  • The problem is that it would be an ongoing task as hundreds of URLs are added every day. As mentioned in the question, the referer is mostly Google, which doesn't really help. – Eran Galperin May 16 '14 at 21:19
  • You may be able to get a complete list through Google Webmaster Tools. You would need to prove to the signup flow, that you control their domain by putting a file in the root of the domain. Since they redirect to your domain, I guess it would work to simply put that file in the root of your own domain. – kasperd May 17 '14 at 07:23
0

Create the "welcome competitor's customers" page with a redirect that sends the Google bot (and only the Google bot) to that page if it's trying to access something that doesn't exist (be sure to use a 301 so that Google cleans up its index) . Log those redirects. Go through the log every few days and create redirects for regular users.

Give it a week or two. When the special log file stops growing, remove the Google bot redirect.

James Mason
  • 146
  • 4