May 20, 2008

Sitemaps generators (free)

Google, Yahoo, Live Search as well as other search engines agreed in 2006 to adhere to the same sitemap protocol, which made things simpler for webmasters. Basically you add a link to a sitemap.xml file in the robots.txt file (a simple line like Sitemap: <sitemap1_location>) and the search engines will download your sitemap and crawl (some of) the links mentioned in there. This is a better way to have your site indexed by search engines without relying only on them finding all the links of your website. You can see usage examples and read more about the Sitemap protocol here: sitemaps.org

 

Of course that the only problem is creating the sitemap.xml file according to that protocol. There are a lot of downloadable sitemaps generators out there, commercial and free ones, as well as online sitemap generators. A more complicated way is to use and install scripts that generate the sitemap directly from your website, the advantage of doing so being the fact that the sitemap will be updated dynamically. If you have Python knowledge, Google offers such a script itself: Google Sitemap Generator.

 

However, if you'd feel more comfortable using a program that installs on your computer (or if you don't have the knowledge to install such a script) you can choose one of the free sitemaps generators (for Windows) that are available:

  • GSiteCrawler -  a free sitemap generator for Windows, is the one I use too. Offers several options, useful ones being: ability to filter out URLs based on specific keywords, it can upload the sitemap.xml file automatically via FTP to your website, runs multiple crawlers at once.
  • Endsheet - Free Windows desktop web crawler that creates Google, Windows Live and Yahoo! Sitemaps. This ones requires .NET to be installed too.
  • Diadematus - Free sitemap generator, requires .NET installed too.
  • Devintelligence Sitemap Generator - Free sitemap generator, it will crawl for links and automatically remove the dead ones, as well as uploads the sitemap file via FTP when it finishes.
  • G-Mapper - Google/Yahoo sitemap generator, which also supports crawling atom, rss and podcast links. You can customize which types of files to include in the sitemap and also uploads it via FTP when finished.
  • Site Magellan - Site Magellan is a free sitemap generator and sitemap editor. This generator also finds broken links and allows you to visualize the structure of your website.
  • Sorrowmans Google Sitemap Tools - free sitemap generator but outdated. It can filter files/links from your sitemap and upload it via FTP when it finishes.
  • Gsitemap - Less featured sitemap generator and kind of outdated (last update in 2005), but should still work fine.

The disadvantage of installable sitemap generators is that it will take longer to create the list of links if you have a website with a large number of pages (say over 5000). However, most websites don't have that many links and the process of crawling and creating the sitemap shouldn't take that long.

 

There are a number of free online sitemaps generators too. While the more popular ones offer paid services too, the free sitemap generation could be a solution for websites with limited numbers of pages. I say limited because the online sitemap generators usually have a limit of URLs they can crawl, a clear disadvantage towards the installable generators:

Quite some generators to choose from. Important is to create and put the sitemap.xml file on your website, because this makes yours more friendly with the search engines crawlers.

May 16, 2008

Freeware programs that increase your productivity

I do a thorough selection of programs I install (especially if they're free) mainly because you have to have a balance between the number of installed programs and your computers' resources. Every program consumes memory, cpu and this reflects in the performance of your overall system. I have a list of (mainly) free programs that I use and make me more productive, and thought it would be nice to share them in case you, my reader(s), didn't hear about them. So, here's my list of free programs:

  • Maxthon2 - this is a freeware browser which was named in its beginnings MyIE2 but because of the confusion with IE they had to rename it. After many (many) tests I ended up using it as my main browser because of several features that come installed by default and increase my online productivity. Such features: super drag-and-drop (drag a keyword and is searched through Google in a separate tab, drag a link and it opens in background), seamless integration with RoboForm (one of the best password managers), reduced memory footprint compared to FireFox (this is discussable, but one thing you'll notice about Maxthon2 is that when minimized it "flushes" the memory used - feature that FireFox doesn't have), the Paste and Go feature (copy a word/link from somewhere else, right-click on the URL field and select Paste and Go to open that link or search for that keyword - ok, it might not be that original since Opera had it first but still counts), uses the same layout engine as Internet Explorer (it's a plus in many cases, one being the smooth integration with RoboForm), handles feeds quite well (an RSS reader on the left hand side panel), offers a built-in screenshot utility, and has many more features that are however common among other browsers too (multi-tab browsing, mouse gestures). It's one of the most popular (if not the most popular) browsers in China (its origins).
  • AVG8 (free edition) - The best antivirus I ever tested (and used for a while) is NOD32 considering the ratio between detection rate and resources used. However, since they don't provide a free version, I had to choose one between avast! antivirus, AVG and Avira AntiVir. After some tests I decided AVG is the way to go and since then I never had a problem with any viruses (I use additional security programs though, mentioned below).
  • Spybot Search and Destroy - This is a free anti-spyware program I use from time to time to scan my system for "unwanted goodies". It does a good job, but when the problem is more acute, I also use HijackThis.
  • Autoruns and Process Explorer - Developed by Sysinternals (now part of Microsoft) these programs are a must if you want to have control over what programs run at startup, what services are enabled and other information about your system. This way you can get rid easier of the ever-present qtask autorun, or any adobe speed-up one which are consuming resources practically for nothing (merely 2 examples, many more other programs add small processes at startup without asking).
  • DeskPins - this small utility makes any window topmost, so you can have your own Always On Top option for any program window. This is very productive when you want to copy/paste things for example. I've covered it in the past too (an alternative to it would be the PowerMenu, this program adding the Always on Top option to the right-click menu of your window).
  • Ditto - This is an extension (free) for the windows clipboard. This lets you access any information that you store in your clipboard (text, images, html), not only the latest item.
  • Freenote - It's important to stay organized, but I don't like having to use a program that requires more time scheduling what you have to do than it actually saves you time (hint Outlook). So, I use Freenote to put sticky notes on my desktop, you simply double click the tray icon and a new freenote shows on your desktop where you can put any schedules you want. Just make sure when you install the program that you don't install the toolbar it comes with too (there's an option checked by default) - unless you want to (there are other ways to support the developer, a donation being one).
  • Unlocker - Ever wanted to delete a file and got the "Access is denied" error? Well this is where Unlocker comes in, it lets you "unlock" any file that is locked by other windows programs and you can delete it without headaches.
  • Total Commander - Best file manager, an excellent replacement for Windows Explorer. It's not free though, but one of the rare commercial programs that deserve being purchased. If you cannot pay for it, you'll have to stick with clicking on the "press the x button to start the program" nag screen.
  • WinRAR - again, not a free program, but an wonderful zip/rar compression/decompression tool. And, there is a way to get WinRAR version 3.51 for free here.
  • Foxit PDF reader - Adobe Reader is a bloated 20-something-MB download that lets you open PDF files, so if you are looking for a replacement Foxit PDF reader is the solution.
  • Windows Live Writer - If you publish a blog this is a must-have tool. Even if it's developed by Microsoft, this is an excellent free destkop blog-publishing program. It lets you publish blog entries without logging in to Blogger, Windows Live Spaces, LiveJournal, TypePad, Wordpress, Community Server and other blogging platforms. What's best is the support for image uploading too (via Picasa for platforms other than Windows Live Spaces). There are some plugins for it too, but the main program is good enough.
  • Skype - Everybody knows (or should know) about Skype, but I couldn't leave it behind because it's an excellent program that lets you make VOIP calls for free.
  • RoboForm - Not free (ok, there's a free basic version, but kind of limited) but RoboForm it's the best password manager. Integrated within Maxthon2 and you're worry free about losing your passwords. Lets you save filled forms too, add identities (saving you from entering every time your name, address, ...) and nevertheless save logins. Can be used successfully as a bookmark manager too for password protected websites. If you're considering paying for it, there are some roboform discounts available.
  • uTorrent with PeerGuardian 2 - uTorrent is the smallest BitTorrent Client, but I always recommend using PeerGuardian 2 with it too (it's an IP blocker, mainly it blocks incoming and outgoing connections based on an IP blocklist - that can keep out of your computer government institutions, anti peer-to-peer companies and many others).
  • ScreenshotCaptor - it's a free screenshot taking utility, a good alternative to commercial programs such as Snagit, that lets you capture images from your desktop.
  • Colorpix - This is a useful little utility (for designers mainly) that lets you grab the color code of any pixel from your screen.
  • FireFox with FireBug installed - I use Maxthon2 for most of my browsing, but for web development I used the combination of FireFox and FireBug. This lets me see in real-time how a change in the CSS file will modify the look of the website I'm working on - excellent tool that surpasses by much Internet Explorer's Developer Toolbar.
  • TaskbarShuffle - this is a program that sits in your tray and lets you organize the way the windows are shown in your Taskbar, if you want to move a window in front or after another one you can do it, as well as closing a window by double-clicking on it or clicking the middle button (just like in the browser).
  • CDBurnerXP - Why pay for expensive CD/DVD burning tools like Nero when this freeware will do the same thing (well, about 90% of what Nero does). It will let you burn CDs and DVDs (don't be fooled by its name, it burns dvds too), including Blu-Ray and HD-DVD discs. Plus, it lets you create ISO files and also burn them. An excellent free CD/DVD burner.
  • mPlayer - This is a free movie player that plays about everything and runs on both Windows and Linux. It plays mpeg/vob, avi, ogg/ogm, vivo, asf/wma/wmv, realmedia movies, videocd/svcd/dvd/3vix/divx movies and many more formats supported. The best part is that it comes with a set of pre-installed codecs that play every movie (almost).

If you have any freeware programs to share please do so.

May 15, 2008

Google, what happened to "Do No Evil"?

Anonymous is a group of people best known for their organized protests against the Church of Scientology. They also made an objective from exposing facts about Scientology, one such website created by them being Enturbulation.

Until recently they posted ads via the Google Adsense network on Enturbulation, but their Adsense account was suspended. Reason given:

While going through our records recently, we found that your AdSense account has posed a significant risk to our AdWords advertisers. Since keeping your account in our publisher network may financially damage our advertisers in the future, we've decided to disable your account.

 

Enturbulation is a website whose actions are geared towards exposing the Church of Scientology, and apparently the problem was that ads paid by Scientology appeared on the Enturbulation website. So the members of this anti-Scientology website were seeing ads that promoted Scientology, which kind of results in a bad experience mainly for the advertiser. This is why Google decided to close down the Anonymous Google Adsense account. However, even if Google acts like it would be neutral in this matter (pro or anti Scientology), there are some issues that let us believe Google took the part of Scientology:scientology

  • Scientology is currently investing enormous amounts of money via Google AdWords, you've probably seen their "Get the Facts about Scientology" image ads on Youtube and other Google Partners - so Google might have decided to protect their interest
  • Another thing is that AdWords publishers have the possibility to filter out websites that they don't want their ads to be displayed in, why didn't they simply filter the Enturbulation website (or other websites like this one) so that their interest would have been protected (maybe because they wanted ads to be displayed in such websites, after all it's all about conversion)

 

So, how about that "Do no Evil" slogan Google?

May 13, 2008

When Google's Hot Trends are really hot

htrends
There really must have been a surge in traffic for this particular keyword that made Hot Trends slip, as Google does its best job in keeping "unethical" searches off its suggested search terms, related searches, or in this case the Hot Trends. If it wouldn't do this we all know what would be the hottest searched keyword on Google (starts with "s" and ends with "ex").

May 7, 2008

Live Search gains in simplicity over Google Search

The Live Search design team has been playing around with a new interface for both the main Live Search page and the Live Search results page, and I must say that they're not doing a bad job. While Google always primed over MSN (now Live) and Yahoo in terms of a simple interface, I must say that Live Search is better - at least from this point of view. See below a comparison between the main Google Search interface and the Live Search interface:

I told you, while the Live Search page shows you only the search box and a link here and there way at the top (or bottom) of the main page, Google Search shows way too many links closer to the search box giving you the impression of a crowded place. As for the search results page, Live Search wins again:
Live Search results page


Google Search results page


While the Live Search page is still undergoing design changes (sometimes you can see a Change setting checkbox) I must say that I like the direction they're going. Unlike Google that seems to want to make you use iGoogle more than the main search page, Live Search chooses simplicity.

May 6, 2008

Google News offers related news searches

When searching for a keyword in Google News you'll see (if you scroll all the way down) a list of related news searches. This is a new feature that will suggest other search terms based on some automated algorithms, making sometimes connections between specific stories. Example, searching for guns will show the following related searches:

This related news searches feature also shows connections where you wouldn't think there would be any. For instance one of the suggested related searches for guns is OJ Simpson - because he was accused of robbery with a firearm - so Google News already made a connection between guns and the robbery. Plus, Las Vegas hotel is mentioned there too, because that's where the assumed robbery took place. Smart isn't it.

Doing another search for Obama, Google News shows these related searches:

The related searches seem somehow normal, however only 2 of the six related searches actually contain the keyword Obama. I know that people searching for Obama are interested in news about Hillary and McCain, but why do these related searches on the Google News page are so different from those displayed on the normal Google Search page (see image below)?

On the normal Google Search page the related searches are actually popular keywords but closely related to the original keyword you used - however on the Google News page the related searches are keywords that are somehow related with the original keyword, but popular in the Google News search.
Here's an example, when doing a Google Search for Obama you see Obama Muslim as a related search. When doing a Google News search for Obama you don't see Obama Muslim as a related news search, mainly because the other related news searches are more popular than this one. So on Google News related searches primes the popularity in the News section, while on Google Search primes the connection with your original keyword.

Yahoo Search adds SearchScan, a malware filter

Yahoo launched a beta service called SearchScan that will warn you when pages shown in the search results contain viruses, spyware or send unsolicited emails. SearchScan basically uses the reports from McAfee's SiteAdvisor, a service that gives you information about particular websites: if they contain spyware, adware, malware or other threats, if they send unsolicited emails, or if they link to other websites that contain these threats. The problem? If you're a webmaster you must be sure that the other websites you're linking to are "green", because if you are linking to sites that are marked as insecure (a big red X) your website will be marked accordingly.

Back to the SearchScan service, basically when you're doing a search on search.yahoo.com you'll see that some results have a warning right below the title (e.g. Warning: Dangerous Downloads) and when you hover over it you get more details about the threats that website contains:

SearchScan will be turned on by default for all users in the U.S., Canada, UK, France, Italy, Germany, Australia, New Zealand and Spain. Google Search also offers for a long time now a similar service in collaboration with StopBadware.org, which will mark websites that install browser exploits or other unwanted malware with a "This site may harm your computer" splash. Unlike Google's service, Yahoo SearchScan will also show a warning when the website you're about to visit sent multiple unsolicited emails, but, when you click on the search result of a such filtered website it will let you visit that site without any additional warnings (Google shows another page before letting you visit the website).
However, Yahoo SearchScan offers a setting that lets you "Never display websites indicated as potentially harmful" (by default it will "Alert you to websites indicated as potentially harmful") and you can change the setting here

One quick bug I've discovered using Yahoo SearchScan, whenever you hover a warning for a first result, the pop-up balloon with the information is stretched somewhere at the top of the browser and you only see the lower part of it

May 5, 2008

How Microsoft punished Yahoo

Microsoft officially announced (May 3, 2008) that it withdraws the proposal to purchase Yahoo. Because trading opened just Monday (announcement was made on Saturday) Yahoo feels now the wrath of its investors as the share price dropped by 22% in premarket. Even if it recovered a little it still currently faces a 15% drop compared to its share price on Friday, analysts expecting to see a 50% drop in share price (about how much it gained since Microsoft made public its intentions to buy it).

Microsoft warned about a possible hostile takeover a while ago, but that idea seems to have been dropped and one of the reasons Microsoft withdrew its bid is the partnership Yahoo made with Google to offer its search results. So what will happen next? Investors are still unsure if they should sell, buy, or keep their shares and trust Yahoo's decisions, but reading through the lines of the letter Microsoft CEO Steve Ballmer sent to to Yahoo's CEO Jerry Yang you can sense a possible legal battle, as through their search advertising partnership, Yahoo and Google will have more than 70% of the search advertising market making them targets for possible antitrust trials. Quoting the most important part of this letter:

We regard with particular concern your apparent planning to respond to a “hostile” bid by pursuing a new arrangement that would involve or lead to the outsourcing to Google of key paid Internet search terms offered by Yahoo! today. In our view, such an arrangement with the dominant search provider would make an acquisition of Yahoo! undesirable to us for a number of reasons:
• In addition, it would raise a host of regulatory and legal problems that no acquirer, including Microsoft, would want to inherit. Among other things, this would consolidate market share with the already-dominant paid search provider in a manner that would reduce competition and choice in the marketplace.
• This would also effectively enable Google to set the prices for key search terms on both their and your search platforms and, in the process, raise prices charged to advertisers on Yahoo. In addition to whatever resulting legal problems, this seems unwise from a business perspective unless in fact one simply wishes to use this as a vehicle to exit the paid search business in favor of Google.
• It could foreclose any chance of a combination with any other search provider that is not already relying on Google’s search services.

Good luck Yahoo, you'll need it.

YouTube simplifies the video player

All YouTube videos now have a "Try the New YouTube Player Beta!" link that lets you see how the future YouTube video player will look like.

Old Player

New Player

The main change targeted the lower toolbar. It is now smoother and offers only the option to switch to full screen, unlike the old player that also had a button to reduce the size of the video. The new player shows the vertical sound adjsting option that is displayed only when you hover the sound icon and at the end of the play there are minor adjustments on the way the share/replay options are shown (and the next/previous videos in the list).

It looks better, they've simplified the new player a little and that's good. Unfortunately for now if you have created Custom Video players for your website/blog, you cannot adjust their look based on the new YouTube player look. Probably that will change by default when the new video player will be the default one.

May 2, 2008

iPhone-mania reaches Google

Google announced several updates these days for iPhone users (oh, and iPod Touch), and if you're thinking why the answer is simple: next search engine battle field. Google, Yahoo and Live Search all offer services for mobile devices, because they all know that the next market with a huge increase is going to be the mobile search market (if not already huge). So it makes sense to expand as much as possible, even if this means customizing their services for particular devices (iPhone in this case): Google for iPhone, Yahoo for iPhone. Live Search does not offer a service specially for iPhones, mainly because it would be promoting the products of a competitor.

Ok, what's new from Google regarding the iPhone:
  • Google News edition for the iPhone - to access it simply go to http://news.google.com using your iPhone. Using your iPhone you can see Google News sources and even watch associated YouTube content where available.

  • Google for the iPhone goes internationally - The modified Google interface for the iPhone and iPod Touch is now available as a localized service for "33 new markets and 16 new languages, from Argentina to New Zealand, from Swedish to Chinese". I'm not that thrilled about Google's localization services, but I guess many others like it, since it was included. You can access it here: http://www.google.com/m/i (don't worry, Google will localize it automatically).

  • Google for the iPhone can be used with an Apps domain too. To access it go to http://www.google.com/m/a/YOURDOMAIN (using the iPhone of course, and also YOURDOMAIN is your actual domain name).
  • Gmail has a new look on the iPhone and offers new functionality: Auto-complete when composing, Automatic refreshing, Faster load times when viewing email.

May 1, 2008

"How to grow weed" - Googlers' top 8 questions

Google Suggest will offer search suggestions as you type letters into the search box, and while this is a good tool when you're trying to remember something (e.g. the name of a movie) it does make it fun sometimes.

The above results are shown when searching for How to, and they are ranked based on their popularity, as Google suggest:
"... uses data about the overall popularity of various searches to help rank the refinements it offers"
While some of these questions really make sense (tying a tie is really difficult thus its popularity), kind of makes you wonder why would people be interested in knowing how to grow weed, given it's illegal? Anyway, Google Suggest does its best in filtering out "bad suggestions" (otherwise porn would be the first suggestion for many searches given its popularity), but there are still slips like this - and why would they filter this out since the people searching might only be interested in understanding the process, not applying it. Dissecting a little the top 8 google suggest how to's, for our own knowledge:

  • How to tie a tie. What better instructions than video ones - Expert Instructions on How to Tie a Tie.

  • How to kiss. This is a tricky one, given it's not followed by "better/using tongue/a frog" - the first advice would be to close your eyes and let the partner show you. If she/he doesn't know, then rub your lips together and use your imagination. Joke aside, Youtube shows you how, again: How to kiss someone passionately

  • How to make out. Kind of the same as the previous question, though an updated version of "making out" includes more than kissing. Anyway, Wikihow gives good tips: How to Make Out for the First Time

  • How to play guitar. First step, buy one. Second step, learn it from Phoebe Buffet. Serious about learning? Then watch this video: How To Play Beginner Guitar Chords

  • How to get a passport. For US visitors only, you just need to find the nearest location to apply for a passport, and after that you'll receive the appropiate guidance. If you're outside US and interested in how to get an US passport, the best way is to marry an US citizen.

  • How to save money. WikiHow seems to give some advices (How to Save Money), but if you ask me the best way to save money resumes to two words: CONSUME LESS.

  • How to boil an egg. What's that difficult about doing this beats me: you put the egg into boiling water and you leave it for 3-5 minutes. Sometimes it will be a soft boiled one, other times it will be a hard boiled egg. The real question is for how long do you boil an egg?

  • How to grow weed. Best part, last. Weed stands for Marijuana (duh), and it's not that difficult. Toughest part is that if you'll get caught you'll share a room with a "nice" bald guy for a while. Your choice. Web is full with homemade tutorials on how to grow weed.


There's so many funny/strange/scary Google Suggest searches that I'm thinking of starting a weekly post on this.

Blogger statistics now in beta

Blogger tests a new Stats tab, that lets bloggers integrate Google Analytics statistics directly into the Blogger interface. This is great news for Blogger users since you won't need to open a new page and check the web analytics separatelly. In 2006 when I heard that Google bought Measure Map, a company that provided blog statistics, I subscribed to be announced when they will be giving away free accounts (private beta) but that never happened. It makes sense now, because Google let Measure Map die and instead integrated it into Blogger directly (using Google Analytics technology too).

This is an "early private beta" as they call it, so I wonder if it will take a long time until we can see this integrated. The Blogger team is also working on the new experimental features that are now available only in Blogger in draft, so I wonder if they'll somehow add the Blogger Statistics to this too and make all the features publicly available when ... well, when they'll finish. In the meantime we can all stick to using Google Analytics. It's interesting to see how this will be pushed to Blogger users, I mean will the analytics tracking code be installed by default and show you the statistics anyway, or will it be added as a blog element? Remains to be seen.

P.S. Hope you all had a Happy Easter and today, May 1st, you're enjoying a workless day.