An XML Extensible Markup Language sitemap is a file that lists all or selected website pages, making sure search engines (like Google, Bing, DuckDuck and others) can find and crawl them all. XML Sitemaps are easy for web admins or website admins to inform search engines about pages on their sites available for crawling.
An XML sitemap can help speed up content discovery and indexing by search engines crawlers even when internal URL structure and navigation doesn’t point to them, making them hard to find.
Google was the first search engine that introduced Sitemaps 0.84 in June 2005. In November 2006, Google, Yahoo! and Microsoft announced joint support for the Sitemaps protocol. Sitemaps.org changed the schema version to Sitemap 0.90, and it is valid until today. Sitemap 0.90 protocol is offered under the Attribution-ShareAlike Creative Commons License and has wide adoption, including Google, Yahoo!, Bing, Baidu, Yandex, DuckDuckGo, Ask.com and others.
The XML Sitemaps protocol is based on ideas from “Crawler-friendly Web Servers”, with improvements including auto-discovery through robots.txt and the ability to specify the priority and change frequency of pages.
An XML sitemap can include information about pages, videos, and other files on your website, as well as the relationships between pages and other pieces of content. An XML sitemap can be divided into separate XML maps for different types of content, for example:
- posts (articles)
When a sitemap is divided into separate sitemaps, the best practice is creating a map index which links to particular maps.
The Sitemaps protocol has been created based on ideas from “Crawler-friendly Web Servers,” with a few improvements. One of the meaningful improvements was the auto-discovery of an XML sitemap using a robots.txt file. Another important improvement was the ability to specify the crawl priority and change the frequency of pages. You can find protocol details on the sitemaps.org page and learn more about sitemaps on the Google Developers website.
You can generate an XML sitemap manually using one of many tools (also online tools) and publish it to the main website folder via FTP. You can also prepare it manually in any text editor or spreadsheet, but you need to ensure that the file you created follows the rules published on sitemaps.org.
WordPress users can use several plugins to automatically generate an XML sitemap, set up priorities for selected types of content, include or exclude from indexing and other features.