Gary Illyes from Google mentioned on LinkedIn that if you wish to you should utilize a single robots.txt file for all of your worldwide websites. He added that he’s not saying it is a good thought, however moderately, technically, you may implement your robots.txt this fashion.
Gary wrote, “Technically you could possibly have a single central robots.txt for all of your worldwide websites (de.instance, ch.instance, at.instance, and many others.) that is hosted in your CDN (cdn.instance/robots.txt).”
Though, Gary later added, “Not saying it is a good suggestion, or unhealthy for that matter, simply that it is attainable.”
Now, Ohgm wrote about this a number of years in the past over right here if you would like a extra detailed technical walkthrough.
However that is simply one other search engine optimization PSA from Gary.
Discussion board dialogue at LinkedIn.
