WordPress 5.3 to Use Robots Meta Tag to Better Discourage Search Engines from Listing Sites
WordPress is changing the method it uses to prevent search engines from indexing sites. Previously, if a user checked the “Discourage search engines from indexing this site” option in a site’s Settings > Reading screen, WordPress would add Disallow: / to the robots.txt file. This would prevent crawling but did not always prevent sites from…
Read More