Current situation is that we are getting thousands and thousands of 404 errors from bots looking for robots.txt in different places on our site due to domain redirects.
Our old website was a labyrinthine multisite powered by dotnetnuke with multiple domain names. We have changed over to a single site on Wordpress with one domain name. The remaining domain names now just redirect to categories on the site. This has meant that googlebot, bingbot and many others repeatedly try to index the domains which used to be full-fledged sites and get redirected.
www.EXAMPLE.co.uk redirects to www.EXAMPLE.co.uk/challenge/
and so /challenge/robots.txt has over a thousand 404s
the same with other redirects which end up at /walktoschool/robots.txt etc etc
Is there a smart way to redirect bots? Or a different way that this should have been handled or get the bots to stop? Our new website doesn't even use robots.txt, it uses htaccess in conjunction with Better WP Security. I have put in requests with Google and Bing to re-crawl the new website but this has been the result.
I am an amateur webmaster at a non-profit organization and I've really had to hit the ground running, any help would be gratefully received!