Create Sitemaps

Create sitemaps as at some point in search engine optimisation it’s simply time for that. Sitemap is a way to tell search engines about changes on the web site or about the pages that might not otherwise discovered.

Sitemap formats

Sitemap file must be UTF–8 encoded. You can name the sitemap file anything you wish. Any URLs must use entity escape codes for & ’ " < >

Sitemaps explained sitemaps.org

XML sitemap example

<lastmod></lastmod> is optional tag in sitemaps.

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
   <url>
      <loc>http://www.example.com/</loc>
      <lastmod>2011-06-08</lastmod>
   </url>
</urlset>

Sitemap index example

A single sitemap is limited to max 50,000 URLs or 10 MB in size. Larger number of URLs can be divided into multiple sitemaps and then grouped by using a sitemaps index file.

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
   <sitemap>
      <loc>http://www.example.com/sitemap1.xml.gz</loc>
      <lastmod>2011-06-02</lastmod>
   </sitemap>
   <sitemap>
      <loc>http://www.example.com/sitemap2.xml.gz</loc>
      <lastmod>2011-06-08</lastmod>
   </sitemap>
</sitemapindex>

Creating mobile sitemaps

Creating sitemaps for mobile content is simple, just add tag for mobile URLs in the sitemap.

<url>
    <loc>http://www.example.com/mobile-article.html</loc>
    <mobile:mobile/>
</url>

Sitemap txt format example

Site map can be a simple text file that contains one full (including the http) URL per line.

http://www.example.com/
http://www.example.com/about.html
http://www.example.com/catalog?item=11

Sitemap generator

Creating sitemaps for website is made simple with diverse online sitemap generators and sitemap generators for download.

Sitemap builders are different. After trying out various sitemap generators, Xenu's Link Sleuth (http://home.snafu.de/tilman/xenulink.html) seems to be the best tool for creating sitemaps manually.

Creating sitemaps with Xenu.

First check the website (on local machine or live) for broken links and other navigational issues with Xenu. Xenu's Link Sleuth is an amazingly fast, very accurate and simple tool to find broken links on web sites. Link Sleuth runs on Microsoft Windows and is proprietary software available at no charge and is written by Tilman Hausherr (http://www.xenu.de/).

When convinced that everything is as supposed, create the sitemap. There is an option “Create Google sitemap file…” in the File menu on Xenu. However, this option appears only after checking the site.

Open the created sitemap in a text editor and strip off all lastmod, changefreq and priority tags. Only one instance of lastmod will remain, that’s under the main domain address. This will save about 100 KB at a 600 KB sitemap.

Some spiders might prefer to deal with smaller sitemap files, 30 KB versus 500 KB. Compress sitemap.xml to sitemap.xml.gz with 7z from www.7-zip.org

Sitemap location.

Sitemap has to be located above all files and directories it lists. Sitemap has to be located on the highest-level directory search engines should to crawl. Make sure that no URLs that reside in a higher-level directory are listed in the sitemap.

A Sitemap file with location http://www.example.com/search-engine-optimisation/sitemap.xml can include any URLs starting with http://www.example.com/search-engine-optimisation/ but can not include URLs starting with http://www.example.com/web-development/

Usual sitemap location is the root folder of website.

Sitemap validator

Google sitemap validator validome.org/google/

Pick your own:

Submit sitemap. Sitemap in robots.txt file

One way is to submit the sitemap under the webmaster sections of major search engines like Google, Bing and Yahoo.

Easier way is to submit the created sitemap through a robots.txt file. Place a notification about the sitemaps and their locations in the robots.txt file.

Sitemap: http://www.example.com/sitemap.xml.gz
Sitemap: http://www.example.com/sitemap.xml
Sitemap: http://www.example.com/sitemap-2.xml
Sitemap: http://www.example.com/search-engine-optimisation/sitemap.xml

Search engine crawlers check for web site robots.txt files on daily basis and pick the sitemaps up quickly.

Informing search engine crawlers

Make sure that search engines pick the sitemaps up quick enough. Ping search engines to submit sitemaps. Enter into the browser address bar -

http://www.google.com/webmasters/sitemaps/ping?sitemap=http://www.example.com/sitemap.xml
http://www.bing.com/webmaster/ping.aspx?siteMap=http://www.example.com/sitemap.xml

Yahoo has shut down it's similar Site Explorer service in the beginning of 2012 and began outsource Yahoo! Search back-end functionality from Microsoft Bing. So, ping Bing!

Since June 2012 submissions.ask.com does not work any more.

Pick your own:

2011 2012

To create sitemaps for a website is a very essential step in search engine optimisation.