With the recent introduction of the canonical link tag, search engines are starting to give us a pretty comprehensive set of tools to manage how a website is crawled and indexed.  These tools have been developing over time, and are a bit ad-hoc and overlap in confusing ways, but we now have some tools that solve some traditionally thorny SEO problems.

I thought it would be good to sit back and take inventory of these tools, and how we can use them.

First of all, here are some of the issues we’re trying to solve:

1. Keeping search engines from indexing pages we don’t want them to index.
2. Keeping search engines from crawling pages we don’t want them to crawl.
3. Keeping search engines from giving page rank to certain pages (whether on our site or on another site).
4. For pages that have variations in the URL due to parameters, capitalization issues, different pathways, etc, getting search engines to index just one version of that URL, and focus all page rank other URL formats get onto that one URL.
5. Removing pages from the index we’d like to get out.

To manage these issues, we now have some good tools:

(more…)

Google just announced a new feature that allows you to specify the canonical URL for a page.  Yahoo also supports this tag, and MSN intends to as well.

Many websites can show the same content under different URLs.  Often this is just how many ecommerce and content management systems work — they generate URLs with various tags, parameters, path names, etc as a way to allow the back end technology to keep track of the page being displayed, as well as the context in which the page was generated (e.g., the catalog category for an ecommerce site, where the product being displayed may be shown in several categories).

For example, the following all might be the same page:

• www.mysite.com/icecream/vanilla.html
• www.mysite.com/desserts/vanilla-icecream.html
• www.mysite.com/products.php?type=vanilla&category=icecream
• www.mysite.com/products.php?type=vanilla&category=desserts

You can also get variations based on inconsistent capitalization:

• www.mysite.com/IceCream/Vanilla.html
• www.mysite.com/icecream/vanilla.html

The problem with showing the same content under different URLs is that it can damage your SEO rankings.  (more…)

Online Ad Revenue YoY (via techcrunch)

Google, Yahoo, Microsoft and AOL have all finished reporting their earnings for Q4 of 2008. The result? Growth! After seeing their revenue growth decline every quarter in 2008, their ad revenues actually started to pick up in Q4 by 3 percent. Together they showed a 8 percent growth year over year. In Q4 of 2007, the search engines showed a 12.7 percent growth year over year. By Q3, growth rates was only 0.6 percent year over year.
(more…)

I recently did a post on Flash and SEO, discussing how search engines (primarily Google so far) are now crawling and indexing Flash.  The bottom line was that while Flash is still not really SEO-friendly despite recent advancements, there are very good reasons to use Flash in many cases, and there are some excellent techniques to help sites that use Flash to get good SEO rankings.

One of the techniques for helping pages that use Flash to rank well is the use of “swfobject”.

swfobject is basically a JavaScript loader for Flash.  It provides some important benefits for users, such as detecting support for Flash, version compatibility checking, support for downloadingFlash updates, and graceful support for showing alternate content to users who don’t have Flash.  It is designed to work well with all major browsers. (more…)

There has been a lot of discussion and debate lately about the use of Flash on websites in the context of SEO.  The discussion was intensified last June when Google announced (again) that they had improved their support for crawling and indexing Flash content.

The conventional wisdom (and my personal opinion) is that if you are really focusing on SEO, and that is a high priority, Flash is still not way to go, for a variety of reasons discussed below.  However, there are many great reasons to use Flash, so you can’t just dismiss it. Flash is very effective for showing video, interactive applications and other rich media, and it just does things you can’t do with regular old HTML.

From a practical standpoint for many website, the question is not using Flash or not, but rather how to best use Flash on a website to meet user objectives, and still have that website work well for SEO.

Why is Flash an issue for SEO?

Flash presents some challenges for search engines, since most search engines have little or no ability to read and index content within Flash files as effectively as they can with HTML/CSS.  Flash files are not structured to present content directly, but are rather more like a program or script.  The information is structured for execution, not presentation.  This makes it much harder to process and interpret Flash from the perspective of a search engine. (more…)