Utilizing XML sitemaps and sitemap indices is an advanced tactic that we recommend to all of our clients at Distilled. In fact, Rob wrote an exceptional how to post on this topic earlier this year, looking atdiagnosing indexation issues using sitemap structure. We knew it was a good way to track indexation better than using site: but would you believe me if I said it can increase traffic to your site as well?

It can. And we have proof from a client that is willing to share their story. Thanks a bunch to Razoo, a site that helps others raise money for charity, for letting me use them for this post.

Added Clarification: Adding a multiple sitemap structure does not guarantee better indexation or more traffic, but it CAN have that affect. Razoo did have sitemaps implemented before, but they were not updated and did not reflect the site’s content. The results shown at the end of this post are after they implemented new sitemaps.

This tactic is not just for large sites, it can be used by sites of any size because we are always growing right? Right. And it is a tactic that should be employed from the beginning if possible, the earlier the better. The more data you have on indexation the better. But more than just data, the earlier you give the search engines a clear path to your content the better. So let’s start from the beginning and go over sitemaps and sitemap indices.

From One to Many

There are standard single XML sitemaps: one file of XML code explaining to the search engines what pages are important. This is a set of instructions to the search engines, and are more guidelines rather than rules. Here is an example from the sitemaps.org site of standard protocol.

From sitemaps.org

Whether you are starting fresh or taking one 50,000 URL sitemap to many, it may seem daunting. That code seems daunting even. Did you know that a sitemap can consist of a text file with just URLs? That can be it. And that can be submitted as a sitemap. #justsayin Complexity is not the issue here, this is all about organization.

The best way to break that out to many sitemaps is a matter of how your site is structured. Do you have a blog based system with categories and content in each category? Do you have sets of products? Or many locations for your business?

• Simple: Groups of 100 pages per sitemap (or 1000, or 10000, but try to keep it smaller)
• Better: Static Pages (homepage, about, etc.), Products, Blog
• Best: Static, Categories, Subcategories, Locations, Blog by Date, etc.

The key will be to structure your sitemaps by how deep your pages go and by section. For instance, if I were to take a site about Unique Doll Clothing and design a sitemap structure from that site I would do:

• *Main Sitemap
  • Static Pages
    • Home
    • About
    • Shipping
    • Gift Certificates
  • *Doll Footwear
    • Doll Shoes
      • Category Page
      • Backless Canvas Mules
      • Balloon Shoes
      • Etc.
    • Doll Boots
  • Doll Dresses
  • Doll Pajamas
  • *etc.

Sitemap Indices

In the example above, the asterisk next to the name is indicating that the sitemap file is an index, not a sitemap as mentioned above. Sitemap indices are sitemaps to point to other sitemaps. This what makes your life easier and more structured. If you loaded each section of your site as a separate sitemap, that’s fine, but rather annoying to page through on webmaster tools. If you were to use indices, you could drill down and see more detail in specific areas. Let me show you.

Level 1: Main Sitemap Look in Webmaster Tools

Level 2: Notice the “Format” in the upper left.

The second screenshot shows that Google knows that we are looking at a sitemap index, not a sitemap file. If this company went further, they’d be able to see even more detail. The more you parse the data, the more you know about what isn’t being indexed. Rob described this as a tree-structure. Note: you can’t put page listings in a sitemap index, they are just carriers of sitemaps. You can get all the gorey details of how to write a sitemap index on the sitemap.org site, so I won’t bore you with that.

Just wanted you to see why they are helpful. Sitemap indices help:

1. Indicate where indexation issues are.
2. Allow an overview look (the numbers for sitemap.xml) all the way down to specific areas. Great for reports!
3. Show the search engines what your site structure is supposed to be.
4. Identify possible duplicate content. (Have a section for doll shoes and doll boots? Those might cause duplicate content if they share products and their URLs are different)

Please note that an intricate structure is not necessary to get the results Razoo saw. This is just better for you and the things I mentioned above. There has been some confusion on this so I wanted to clarify.

Remember that case study I talked about with increased traffic? You wanted to see that right? Time is now, because they are the ones that used it to help with indexation and structure. They just happened to have the best metrics ever from implementing this. Have better metrics? I’d love to hear how this has worked for you!

Case Study: Razoo

Razoo is not my client, it’s someone else’s client although I was involved in the design and development of the site a while back and they recently saw an awesome spike in traffic. We checked it to be sure nothing else could be the cause and for sure, it was the use of XML sitemaps. See the chart below (smiley courtesy of Mike). The arrow is where the sitemaps were implemented.

I checked the stats again tonight, and sure enough, for over two weeks now, it’s still going up.

We looked into it and sure enough, the number of pages sending traffic shot up, more than doubled (486 to 1240). The same with keywords sending organic search traffic, more than doubled (548 to 1347). Nothing could be attributed to a fluke keyword or page, just everything went up. This is the power of good indexation and giving the search engines some help.

So take some time to think through your site and it’s structure. Talk to your IT team and see if they can break up the XML sitemaps into a tree structure. It’ll help you diagnose issues down the road (indexation and duplication) and may even bring an influx of traffic as the search engines find content they might not have found before. The power of backroads. *big smile*

This entry was posted on Monday, July 18th, 2011 at 11:33 pm and is filed under Uncategorized . You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.