Sitemaps generators (free)

Google, Yahoo, Live Search as well as other search engines agreed in 2006 to adhere to the same sitemap protocol, which made things simpler for webmasters. Basically you add a link to a sitemap.xml file in the robots.txt file (a simple line like Sitemap: <sitemap1_location>) and the search engines will download your sitemap and crawl (some of) the links mentioned in there. This is a better way to have your site indexed by search engines without relying only on them finding all the links of your website. You can see usage examples and read more about the Sitemap protocol here: sitemaps.org

 

Of course that the only problem is creating the sitemap.xml file according to that protocol. There are a lot of downloadable sitemaps generators out there, commercial and free ones, as well as online sitemap generators. A more complicated way is to use and install scripts that generate the sitemap directly from your website, the advantage of doing so being the fact that the sitemap will be updated dynamically. If you have Python knowledge, Google offers such a script itself: Google Sitemap Generator.

 

However, if you'd feel more comfortable using a program that installs on your computer (or if you don't have the knowledge to install such a script) you can choose one of the free sitemaps generators (for Windows) that are available:

• GSiteCrawler -  a free sitemap generator for Windows, is the one I use too. Offers several options, useful ones being: ability to filter out URLs based on specific keywords, it can upload the sitemap.xml file automatically via FTP to your website, runs multiple crawlers at once.
• Endsheet - Free Windows desktop web crawler that creates Google, Windows Live and Yahoo! Sitemaps. This ones requires .NET to be installed too.
• Diadematus - Free sitemap generator, requires .NET installed too.
• Devintelligence Sitemap Generator - Free sitemap generator, it will crawl for links and automatically remove the dead ones, as well as uploads the sitemap file via FTP when it finishes.
• G-Mapper - Google/Yahoo sitemap generator, which also supports crawling atom, rss and podcast links. You can customize which types of files to include in the sitemap and also uploads it via FTP when finished.
• Site Magellan - Site Magellan is a free sitemap generator and sitemap editor. This generator also finds broken links and allows you to visualize the structure of your website.
• Sorrowmans Google Sitemap Tools - free sitemap generator but outdated. It can filter files/links from your sitemap and upload it via FTP when it finishes.
• Gsitemap - Less featured sitemap generator and kind of outdated (last update in 2005), but should still work fine.

The disadvantage of installable sitemap generators is that it will take longer to create the list of links if you have a website with a large number of pages (say over 5000). However, most websites don't have that many links and the process of crawling and creating the sitemap shouldn't take that long.

 

There are a number of free online sitemaps generators too. While the more popular ones offer paid services too, the free sitemap generation could be a solution for websites with limited numbers of pages. I say limited because the online sitemap generators usually have a limit of URLs they can crawl, a clear disadvantage towards the installable generators:

• XML-Sitemaps Generator -  this is one of the free online sitemaps generators I used and recommend if you have less than 500 crawlable links (this is their maximum limit)
• Sitemapspal -  this one offers up to indexed 2000 URLs.
• PortalApp Generator - has a 300 URL limit.
• Pingoat Sitemap Creator - they don't mention the URL limit but I doubt it's over 300 URLS. Pingoat has a free blog ping service too that it's good.
• Tarrant.it Sitemap Generator - Italian online sitemap generator with a 500 URL limit.
• Hockeygods Online Sitemap Generator -  100url limit
• Peterz.de (German) -  Site is in German but it's pretty obvious where you have to put in the starting URL for the crawl (200 URL limit?).
• Autositemap -  this sitemap generator requires installing an image on every page of your website, has a paid version too.

Quite some generators to choose from. Important is to create and put the sitemap.xml file on your website, because this makes yours more friendly with the search engines crawlers.

Live Search gains in simplicity over Google Search

The Live Search design team has been playing around with a new interface for both the main Live Search page and the Live Search results page, and I must say that they're not doing a bad job. While Google always primed over MSN (now Live) and Yahoo in terms of a simple interface, I must say that Live Search is better - at least from this point of view. See below a comparison between the main Google Search interface and the Live Search interface:

I told you, while the Live Search page shows you only the search box and a link here and there way at the top (or bottom) of the main page, Google Search shows way too many links closer to the search box giving you the impression of a crowded place. As for the search results page, Live Search wins again:

Live Search results page

Google Search results page

While the Live Search page is still undergoing design changes (sometimes you can see a Change setting checkbox) I must say that I like the direction they're going. Unlike Google that seems to want to make you use iGoogle more than the main search page, Live Search chooses simplicity.

iPhone-mania reaches Google

Google announced several updates these days for iPhone users (oh, and iPod Touch), and if you're thinking why the answer is simple: next search engine battle field. Google, Yahoo and Live Search all offer services for mobile devices, because they all know that the next market with a huge increase is going to be the mobile search market (if not already huge). So it makes sense to expand as much as possible, even if this means customizing their services for particular devices (iPhone in this case): Google for iPhone, Yahoo for iPhone. Live Search does not offer a service specially for iPhones, mainly because it would be promoting the products of a competitor.

Ok, what's new from Google regarding the iPhone:

• Google News edition for the iPhone - to access it simply go to http://news.google.com using your iPhone. Using your iPhone you can see Google News sources and even watch associated YouTube content where available.
  
  
• Google for the iPhone goes internationally - The modified Google interface for the iPhone and iPod Touch is now available as a localized service for "33 new markets and 16 new languages, from Argentina to New Zealand, from Swedish to Chinese". I'm not that thrilled about Google's localization services, but I guess many others like it, since it was included. You can access it here: http://www.google.com/m/i (don't worry, Google will localize it automatically).
  
  
• Google for the iPhone can be used with an Apps domain too. To access it go to http://www.google.com/m/a/YOURDOMAIN (using the iPhone of course, and also YOURDOMAIN is your actual domain name).
• Gmail has a new look on the iPhone and offers new functionality: Auto-complete when composing, Automatic refreshing, Faster load times when viewing email.
  

Microsoft Live Search adds Sitelinks too

Google had the Sitelinks added since 2006, and 2 years later these are added into Microsoft Live Search too. Sitelinks (Microsoft calls them Deep links) are shortcuts to pages within a specific website that show below that website's listing. They are meant to help you navigate the website by providing links to popular destinations from that website. If Microsoft Live Search deeplinks works like Google sitelinks does then their system analyzes the link structure of a particular website to find shortcuts and shows them automatically.

As you can see the main difference between Live search deeplinks and Google Sitelinks is their number - Live Search shows only 4, while Google shows 8. Also, the link chosen to be displayed as sitelinks are a bit different between the two, which is somehow normal given that they both use different algorithms.

Also the Live Search deeplinks feature is announced as new, I assume they meant that it was added in its final state today given that there are reports this feature for Live Search goes back in 2007 - probably back then there were just tests.

As with any new Live Search feature this needs work too. One of the things they need to improve is the number of websites they recognize Deeplinks in Live Search, because Google shows them for many more websites. Another thing would be to include the ability to control their display through the Webmaster Live section like Google allows through Google Webmaster Center.

Live Search News gets updated

See, Microsoft's Live Search team is (somehow) productive too. They recently announced several improvements for the Live News Search service (or Live Search News?).
What I like in this update:

• Playing multimedia without leaving the news search results page. On the news results page they incorporate images and videos with "smart-motion thumbnails". What's nice about this is that you can watch the videos related to the news search you performed by hovering the video thumbnail and clicking the small Play icon.
• Related news searches. To the right of the news results (note this Google, to the right, not above or below the results!) you'll see a Related Searches list that shows up to 8 items based on the popularity of their searches (via the News interface?).
• They finally offer (some) news, as the headlines are updated "to the minute" if you use the Most Recent sort option - plus you can refine the news by the category they're part of (still shown on the right side).

You can give it a try too, say do a search for earthquake on Live Search and see how this compares to the same Google News earthquake query. I know, they say that the Live News Search updates faster now (even when using the Sort by Relevance filter), yet the newest result on Google News search is about the Illinois earthquake updated an hour ago, while the newest result on Live News earch shows as being updated 5 hours ago. There's some work to do here for the Live Search team, as availability is the most important feature for news - they'll get there soon as it seems they'll add news from blog sources too.