With the recent introduction of the canonical link tag, search engines are starting to give us a pretty comprehensive set of tools to manage how a website is crawled and indexed.  These tools have been developing over time, and are a bit ad-hoc and overlap in confusing ways, but we now have some tools that solve some traditionally thorny SEO problems.

I thought it would be good to sit back and take inventory of these tools, and how we can use them.

First of all, here are some of the issues we’re trying to solve:

1. Keeping search engines from indexing pages we don’t want them to index.
2. Keeping search engines from crawling pages we don’t want them to crawl.
3. Keeping search engines from giving page rank to certain pages (whether on our site or on another site).
4. For pages that have variations in the URL due to parameters, capitalization issues, different pathways, etc, getting search engines to index just one version of that URL, and focus all page rank other URL formats get onto that one URL.
5. Removing pages from the index we’d like to get out.

To manage these issues, we now have some good tools:

(more…)

Google just announced a new feature that allows you to specify the canonical URL for a page.  Yahoo also supports this tag, and MSN intends to as well.

Many websites can show the same content under different URLs.  Often this is just how many ecommerce and content management systems work — they generate URLs with various tags, parameters, path names, etc as a way to allow the back end technology to keep track of the page being displayed, as well as the context in which the page was generated (e.g., the catalog category for an ecommerce site, where the product being displayed may be shown in several categories).

For example, the following all might be the same page:

• www.mysite.com/icecream/vanilla.html
• www.mysite.com/desserts/vanilla-icecream.html
• www.mysite.com/products.php?type=vanilla&category=icecream
• www.mysite.com/products.php?type=vanilla&category=desserts

You can also get variations based on inconsistent capitalization:

• www.mysite.com/IceCream/Vanilla.html
• www.mysite.com/icecream/vanilla.html

The problem with showing the same content under different URLs is that it can damage your SEO rankings.  (more…)

Online Ad Revenue YoY (via techcrunch)

Google, Yahoo, Microsoft and AOL have all finished reporting their earnings for Q4 of 2008. The result? Growth! After seeing their revenue growth decline every quarter in 2008, their ad revenues actually started to pick up in Q4 by 3 percent. Together they showed a 8 percent growth year over year. In Q4 of 2007, the search engines showed a 12.7 percent growth year over year. By Q3, growth rates was only 0.6 percent year over year.
(more…)

Several sources close to Yahoo and Microsoft has mentioned that a search partnership deal between Yahoo and Microsoft is likely to be worked out now that Yahoo has named it’s new CEO, former Autodesk CEO Carol Bartz.
(more…)

Yahoo has announced Carol Bartz, 60, to succeed Jerry Yang as the CEO of Yahoo. Bartz is a respected tech veteran who served in Autodesk as the CEO from 1992-2006. Autodesk is a software company which grew from $300 million to $1.5 billion during her 14 years of service. Autodesk’s stock price during that tenure grew ten-folds. She also comes with past experience as an executive at other major tech companies such as Digital Equipment Corp, Sun Microsystems, and currently serves on the board of Cisco Systems and Intel Corp.
(more…)