With the recent introduction of the canonical link tag, search engines are starting to give us a pretty comprehensive set of tools to manage how a website is crawled and indexed.  These tools have been developing over time, and are a bit ad-hoc and overlap in confusing ways, but we now have some tools that solve some traditionally thorny SEO problems.

I thought it would be good to sit back and take inventory of these tools, and how we can use them.

First of all, here are some of the issues we’re trying to solve:

1. Keeping search engines from indexing pages we don’t want them to index.
2. Keeping search engines from crawling pages we don’t want them to crawl.
3. Keeping search engines from giving page rank to certain pages (whether on our site or on another site).
4. For pages that have variations in the URL due to parameters, capitalization issues, different pathways, etc, getting search engines to index just one version of that URL, and focus all page rank other URL formats get onto that one URL.
5. Removing pages from the index we’d like to get out.

To manage these issues, we now have some good tools:

(more…)

Google just announced a new feature that allows you to specify the canonical URL for a page.  Yahoo also supports this tag, and MSN intends to as well.

Many websites can show the same content under different URLs.  Often this is just how many ecommerce and content management systems work — they generate URLs with various tags, parameters, path names, etc as a way to allow the back end technology to keep track of the page being displayed, as well as the context in which the page was generated (e.g., the catalog category for an ecommerce site, where the product being displayed may be shown in several categories).

For example, the following all might be the same page:

• www.mysite.com/icecream/vanilla.html
• www.mysite.com/desserts/vanilla-icecream.html
• www.mysite.com/products.php?type=vanilla&category=icecream
• www.mysite.com/products.php?type=vanilla&category=desserts

You can also get variations based on inconsistent capitalization:

• www.mysite.com/IceCream/Vanilla.html
• www.mysite.com/icecream/vanilla.html

The problem with showing the same content under different URLs is that it can damage your SEO rankings.  (more…)

I recently did a post on Flash and SEO, discussing how search engines (primarily Google so far) are now crawling and indexing Flash.  The bottom line was that while Flash is still not really SEO-friendly despite recent advancements, there are very good reasons to use Flash in many cases, and there are some excellent techniques to help sites that use Flash to get good SEO rankings.

One of the techniques for helping pages that use Flash to rank well is the use of “swfobject”.

swfobject is basically a JavaScript loader for Flash.  It provides some important benefits for users, such as detecting support for Flash, version compatibility checking, support for downloadingFlash updates, and graceful support for showing alternate content to users who don’t have Flash.  It is designed to work well with all major browsers. (more…)

There has been a lot of discussion and debate lately about the use of Flash on websites in the context of SEO.  The discussion was intensified last June when Google announced (again) that they had improved their support for crawling and indexing Flash content.

The conventional wisdom (and my personal opinion) is that if you are really focusing on SEO, and that is a high priority, Flash is still not way to go, for a variety of reasons discussed below.  However, there are many great reasons to use Flash, so you can’t just dismiss it. Flash is very effective for showing video, interactive applications and other rich media, and it just does things you can’t do with regular old HTML.

From a practical standpoint for many website, the question is not using Flash or not, but rather how to best use Flash on a website to meet user objectives, and still have that website work well for SEO.

Why is Flash an issue for SEO?

Flash presents some challenges for search engines, since most search engines have little or no ability to read and index content within Flash files as effectively as they can with HTML/CSS.  Flash files are not structured to present content directly, but are rather more like a program or script.  The information is structured for execution, not presentation.  This makes it much harder to process and interpret Flash from the perspective of a search engine. (more…)

Google recently released an “SEO Starter Guide”.  You can download it as a PDF from here.

 This is an interesting development, mainly because historically Google has wanted to minimize SEO as a factor in website design and promotion, so that webmasters would focus on quality and users rather than gaming the system for SEO rankings.  For Google to give specific advice on SEO is a real shift in policy, though you could see it coming based on other information such as Matt Cutts’ statements over the last couple years.

The guide actually gives some pretty good advice, although it is quite limited in scope and avoids detail guidelines.  There are really no revelations here — it is all stuff you will find in most SEO resources.  Areas covered include:

• Title tags — how they are used in search results, making sure they are unique, make them short
•  Meta description tags — use them, make them unique, keep them succinct
• URL structure — make them descriptive, use keywords (!), keep directory structure simple
• Navigation — use text links, create a good internal link hierarchy, use XML site maps
• 404 Not Found page — use a 404 page to handle bad links
• Quality content — offer good, fresh, relevant, unique content of interest to users
• Anchor text — use keywords in link text
• Headings — use headings appropriately, including use of <h1>, <h2>, etc tags
• Image tagging — use alt tags on images, and use keywords in image file names
• Robots.txt — use robots.txt to manage where spiders crawl in your site
• Use nofollow — use rel=’nofollow’ tag on links to sites/pages you don’t trust, or links you don’t control
• Promote your site — use blogs, social media sites, etc to publicize your site
• Webmaster tools — use the webmaster tools from Google and other search engine for diagnositics and information

All in all, some good stuff.  It is interesting to see Google validate some very standard SEO techniques, including keywords in anchor text, keywords in URLs, use of title/meta tags, <Hn> tags for headings, etc.  It is also interesting how Google recommends simple, non-dynamic URLs with keywords (which to some extent is contrary to some recent advice they gave — see the post on this here).

Of course, there is a lot that Google does not cover here that are important SEO techniques.  Inbound linking is a big one — they talk vaguely about “promoting” your site and using social media, but no specifics.  They also give no advice on keyword targeting, optimal content writing, keyword density, etc, nor does it cover more complex topics like use of Flash, AJAX, CSS, JavaScript, etc. 

One interesting comment I hear regarding this is the idea that “SEO is dead”, since if Google is giving SEO advice, who needs SEO specialists?  I don’t agree with this view.  While there are some basic “best practices” that have emerged for SEO, and these are being blessed by Google, there is so much more to SEO if you want to compete.  This is especially true if lots of websites adopt these basic practices.  For sites that want to stand out, they need to go to the next level — beyond these basic practices.  For that, they will need expertise from SEO specialists, which means we are not (yet) an endangered species.

John Erickson
www.leadqual.com