Create Sitemaps

Picture: a server room containing sitemaps.

Sitemaps live in server rooms.

Why to create sitemaps? Sitemap is a way to tell search engines about changes on the web site or about the pages that might not be otherwise discovered.

Sitemap formats

Sitemap file must be UTF–8 encoded. You can name the sitemap file anything you wish. Any URLs must use entity escape codes for & ' " < >

Sitemaps explained sitemaps.org

XML sitemap example

<lastmod></lastmod> is optional tag in sitemaps. SEO folk uses it for sending messages to Google.

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
   <url>
      <loc>http://www.example.com/</loc>
      <lastmod>2011-06-08</lastmod>
   </url>
</urlset>

Sitemap index example

A single sitemap is limited to max 50,000 URLs or 10 MB in size. Larger number of URLs can be divided into multiple sitemaps and then grouped by using a sitemap index file.

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
   <sitemap>
      <loc>http://www.example.com/sitemap1.xml.gz</loc>
      <lastmod>2011-06-02</lastmod>
   </sitemap>
   <sitemap>
      <loc>http://www.example.com/sitemap2.xml.gz</loc>
      <lastmod>2011-06-08</lastmod>
   </sitemap>
</sitemapindex>

Creating mobile sitemaps

Creating sitemaps for mobile content is also simple, just add tag for mobile URLs in the sitemap.

<url>
    <loc>http://www.example.com/mobile-article.html</loc>
    <mobile:mobile/>
</url>

Sitemap txt format example

Site map can be a simple text file that contains one full (including the http) URL per line.

http://www.example.com/
http://www.example.com/about.html
http://www.example.com/catalog?item=11

Sitemap generator

Creating XML sitemaps for a website is simple with diverse online sitemap generators and sitemap generators for download.

Sitemap builders are different. After trying out various sitemap generators, Xenu's Link Sleuth (http://home.snafu.de/tilman/xenulink.html) seems to be the best tool for creating sitemaps manually. With the same you can check the site for broken links.

Creating sitemaps with Xenu

First check the website (on local machine or live) for broken links and other navigational issues with Xenu. Xenu's Link Sleuth is an amazingly fast, accurate and simple tool to find broken links on websites. Link Sleuth runs on Microsoft Windows and is proprietary software available at no charge. It's written by Tilman Hausherr (http://www.xenu.de/).

When convinced that everything is as supposed, create the sitemap. Under the File menu there is an option "Create Google sitemap file...". This option appears in the menu only after scanning the site.

Open the created sitemap in a text editor and strip off all lastmod, changefreq and priority tags. Leave one instance of lastmod under the main domain. That will save about 100 KB in a 600 KB sitemap.

Some spiders might prefer to deal with smaller sitemap files, 30 KB versus 500 KB. Compress sitemap.xml to sitemap.xml.gz with 7z from www.7-zip.org

Sitemap location

Sitemap has to be located above all files and folders it lists. Sitemap has to be located on the highest-level directory search engines should to crawl. Make sure that no URLs that reside in a higher-level directory are listed in the sitemap. Usually it's placed in web root directory.

A Sitemap file with location http://www.example.com/search-engine-optimisation/sitemap.xml can include any URLs starting with http://www.example.com/search-engine-optimisation/ but can not include URLs starting with http://www.example.com/web-development/

A common sitemap location is the root folder of website.

Sitemap validator

Google sitemap validator validome.org/google/

Submit sitemaps

Webmaster Tools

One way is to submit the sitemap under the webmaster sections of major search engines like Google, Yandeks, Bing and Yahoo.

Sitemap in robots.txt file

Easier way is to submit the created sitemap through a robots.txt file. Place a notification about the sitemaps and their locations in the robots.txt file.

Sitemap: http://www.example.com/sitemap.xml.gz
Sitemap: http://www.example.com/sitemap.xml
Sitemap: http://www.example.com/sitemap-2.xml
Sitemap: http://www.example.com/search-engine-optimisation/sitemap.xml

Search engine crawlers check for web site robots.txt files on daily basis and pick the sitemaps up quickly.

Informing search engine crawlers

Make sure that search engines pick your sitemaps up quick enough. Ping search engines to submit sitemaps. Enter into the browser address bar:

http://www.google.com/webmasters/sitemaps/ping?sitemap=http://www.example.com/sitemap.xml
http://www.bing.com/webmaster/ping.aspx?siteMap=http://www.example.com/sitemap.xml

Yahoo has shut down it's similar Site Explorer service in the beginning of 2012 and began outsource Yahoo! Search back-end functionality from Microsoft Bing. So, ping Bing!

The submissions.ask.com does not work since June 2012.


2011 2012

Sitemaps are useful tools in website SEO.

Google Reddit Twitter Facebook Pinterest StumbleUpon