Sitemap Submission Via RobotsTxt File

SEO Tip from Catanich Internet Marketing And SEO Consulting Services

Trick To Submit Your XML Sitemaps To All SEs By Using The RobotsTxt File

Sitemap Submission Via RobotsTxt File
Home   »   Webmaster Services   »  

Sitemap Submission Via RobotsTxt File

XML Sitemap Submission via the Robots.Txt File

In March 2008, the "Sitemaps Protocol" was expanded to include the "AutoDiscovery of Sitemaps" using robots.txt to let the SEs supporting the protocol know where to find your Sitemaps. We subsequently also announced the all client support for Sitemap cross-submissions using Google Webmaster Tools.

The days of paid submission has ended.

Today, as a standard service, we have married the two, allowing for Sitemap cross-submissions using robots.txt. This concept presents a common path for your site's submission to Google, Yahoo, Bing! (MSN) and Ask. Now, most of the second tier SEs are following suit as well.

This is how it's done!

Sitemap Submission Via Robots.Txt File

Add the following dark code to your robots.txt file to automatically submit your site to all participating SEs.

Robots.txt File Coding Example

# Robots.txt file created by 8/03/2018
# For domain:
# # All other robots will spider the domain
User-agent: *
Disallow: /_common/
Disallow: /_private/
Disallow: /_ScriptLibrary/
Disallow: /aspnet_client/
Disallow: /_client/

Design Notes

Any blank line in the robots.txt file will cause an error. No blank lines.