Beginner's Guide to SEO
by Rand Fishkin of SEOmoz.org
Sections in this guide:
; Prologue: Who is SEOmoz and Why is this Guide Free? ; A: What is SEO?
o Why does my company need SEO?
o Why do the search engines need SEO?
o How much of this article do I need to read? ; B: How Search Engines Operate
o Speed Bumps and Walls
o Measuring Popularity and Relevance
o Information Search Engines Can Trust
o The Anatomy of a HyperLink
o Keywords & Queries
o Sorting the Wheat from the Chaff
o Paid Placement and Secondary Sources in the Results ; C: How to Conduct Keyword Research
o Wordtracker & Overture
o Targeting the Right Terms
o The "Long Tail" of Search
o Sample Keyword Research Chart
; D: Critical Components of Optimizing a Site
o URLs, Titles & Meta Data
o Search Friendly Text
o Information Architecture
o Canonical Issues & Duplicate Content ; E: Building a Traffic-Worthy Site
o Professional Design
o Authoring High Quality Content
o Link Bait
; F: Growing a Site's Popularity
o Community Building
o Press Releases and Public Relations
o Link Building Based on Competitive Analysis
o Building Personality & Reputation
o Highly Competitive Terms & Phrases ; G: Conclusion: Crafting an SEO Strategy
o Quality vs. Quantity
o Measuring Success: Website & Ranking Metrics to Watch
o Working with a Pro vs. Do-It-Yourself SEO
o Where to Get Questions Answered ; H: Links to More Information & Resources
Prologue: Who is SEOmoz and Why is this Guide Free?
SEOmoz is a Seattle-based Search Engine Optimization (SEO) firm and community resource for those seeking
knowledge in the SEO/M field. You can learn more about SEOmoz here. We provide a great variety of free
information via a daily blog, automated tools and advanced articles.
This article is offered as a resource to help individuals, organizations and companies inexperienced with search engine optimization learn the basics of how the service and process operates. It is our goal to improve your ability to drive search traffic to your site and debunk major myths about SEO. We share this knowledge to help businesses, government, educational and non-profit organizations benefit from being listed in the major search engines. SEOmoz provides advanced SEO services. If you are new to SEO, have read through this document, and require an
SEO firm's assistance, please contact us. Along with the optimization services we provide, we also recommend a
number of very effective SEO firms who follow the best practices described in this document.
What is SEO?
SEO is the active practice of optimizing a web site by improving internal and external aspects in order to increase the traffic the site receives from search engines. Firms that practice SEO can vary; some havea highly specialized focus while others take a more broad and general approach. Optimizing a web site for search engines can require looking at so many unique elements that many practitioners of SEO (SEOs) consider themselves to be in the broad field of website optimization (since so many of those elements intertwine).
This guide is designed to describe all areas of SEO - from discovery of the terms and phrases that will generate traffic, to making a site search engine friendly to building the links and marketing the unique value of the site/organization's offerings.
Why does my company/organization/website need SEO?
The majority of web traffic is driven by the major commercial search engines - Yahoo!, MSN, Google & AskJeeves
(although AOL gets nearly 10% of searches, their engine is powered by Google's results). If your site cannot be found
by search engines or your content cannot be put into their databases,
you miss out on the incredible opportunities available to websites
provided via search - people who want what you have visiting your
site. Whether your site provides content, services, products or
information, search engines are a primary method of navigation for
almost all Internet users.
Search queries, the words that users type into the search box which
contain terms and phrases best suited to your site carry
extraordinary value. Experience has shown that search engine traffic
can make (or break) an organization's success. Targeted visitors to
a website can provide publicity, revenue and exposure like no other.
Investing in SEO, whether through time or finances, can have an
exceptional rate of return.
Why can't the search engines figure out my site without SEO
Search engines are always working towards improving their technology to crawl the web more deeply and return increasingly relevant results to users. However, there is and will always be a limit to how search engines can operate. Whereas the right moves can net you thousands of visitors and attention, the wrong moves can hide or bury your site deep in the search results where visibility is minimal. In addition to making content available to search engines, SEO can also help boost rankings, so that content that has been found will be placed where searchers will more readily see it. The online environment is becoming increasingly competitive and those companies who perform SEO will have a decided advantage in visitors and customers.
How much of this article do I need to read?
If you are serious about improving search traffic and are unfamiliar with SEO, I recommend reading this guide front-to-back. There's a printable MS Word version for those who'd prefer, and dozens of linked-to resources on other sites and pages that are worthy of your attention. Although this guide is long, I've attempted to remain faithful to Mr. Strunk's famous quote:
"A sentence should contain no unnecessary words, a paragraph no unnecessary sentences, for the same reason that a drawing should have no unnecessary lines and a machine no unnecessary parts."
Every section and topic in this report is critical to understanding the best known and most effective practices of search engine optimization.
How Search Engines Operate
Search engines have a short list of critical operations that allows them to provide relevant web results when searchers use their system to find information.
1. Crawling the Web
Search engines run automated programs, called "bots" or "spiders" that use the hyperlink structure of the web
to "crawl" the pages and documents that make up the World Wide Web. Estimates are that of the
approximately 20 billion existing pages, search engines have crawled between 8 and 10 billion.
2. Indexing Documents
Once a page has been crawled, it's contents can be "indexed" - stored in a giant database of documents that
makes up a search engine's "index". This index needs to be tightly managed, so that requests which must
search and sort billions of documents can be completed in fractions of a second.
3. Processing Queries
When a request for information comes into the search engine (hundreds of millions do each day), the engine
retrieves from its index all the document that match the query. A match is determined if the terms or phrase is
found on the page in the manner specified by the user. For example, a search for car and driver magazine at
Google returns 8.25 million results, but a search for the same phrase in quotes ("car and driver magazine")
returns only 166 thousand results. In the first system, commonly called "Findall" mode, Google returned all
documents which had the terms "car" "driver" and "magazine" (they ignore the term "and" because it's not
useful to narrowing the results), while in the second search, only those pages with the exact phrase "car and
driver magazine" were returned. Other advanced operators (Google has a list of 11) can change which results
a search engine will consider a match for a given query.
4. Ranking Results
Once the search engine has determined which results are a match for the query, the engine's algorithm (a
mathematical equation commonly used for sorting) runs calculations on each of the results to determine
which is most relevant to the given query. They sort these on the results pages in order from most relevant to
least so that users can make a choice about which to select.
Although a search engine's operations are not particularly lengthy, systems like Google, Yahoo!, AskJeeves and MSN are among the most complex, processing-intensive computers in the world, managing millions of calculations each second and funneling demands for information to an enormous group of users.
Speed Bumps & Walls
Certain types of navigation may hinder or entirely prevent search engines from reaching your website's content. As search engine spiders crawl the web, they rely on the architecture of hyperlinks to find new documents and revisit those that may have changed. In the analogy of speed bumps and walls, complex links and deep site structures with little unique content may serve as "bumps." Data that cannot be accessed by spiderable links qualify as "walls." Possible "Speed Bumps" for SE Spiders:
; URLs with 2+ dynamic parameters; i.e. http://www.url.com/page.php?id=4&CK=34rr&User=%Tom% (spiders
may be reluctant to crawl complex URLs like this because they often result in errors with non-human visitors)
; Pages with more than 100 unique links to other pages on the site (spiders may not follow each one)
; Pages buried more than 3 clicks/links from the home page of a website (unless there are many other external
links pointing to the site, spiders will often ignore deep pages)
; Pages requiring a "Session ID" or Cookie to enable navigation (spiders may not be able to retain these
elements as a browser user can)
; Pages that are split into "frames" can hinder crawling and cause confusion about which pages to rank in the
Possible "Walls" for SE Spiders:
; Pages accessible only via a select form and submit button
; Pages requiring a drop down menu (HTML attribute) to access them
; Documents accessible only via a search box
; Documents blocked purposefully (via a robots meta tag or robots.txt file - see more on these here)
; Pages requiring a login
; Pages that re-direct before showing content (search engines call this cloaking or bait-and-switch and may
actually ban sites that use this tactic)
The key to ensuring that a site's contents are fully crawlable is to provide direct, HTML links to to each page you want the search engine spiders to index. Remember that if a page cannot be accessed from the home page (where most spiders are likely to start their crawl) it is likely that it will not be indexed by the search engines. A sitemap (which is discussed later in this guide) can be of tremendous help for this purpose.
Measuring Relevance and Popularity
Modern commercial search engines rely on the science of information retrieval (IR). That science has existed since the middle of the 20th century, when retrieval systems powered computers in libraries, research facilities and government labs. Early in the development of search systems, IR scientists realized that two critical components made up the majority of search functionality:
Relevance - the degree to which the content of the documents returned in a search matched the user's query intention and terms. The relevance of a document increases if the terms or phrase queried by the user occurs multiple times and shows up in the title of the work or in important headlines or subheaders.
Popularity - the relative importance, measured via citation (the act of one work referencing another, as often occurs in academic and business documents) of a given document that matches the user's query. The popularity of a given document increases with every other document that references it.
These two items were translated to web search 40 years later and manifest themselves in the form of document analysis and link analysis.
In document analysis, search engines look at whether the search terms are found in important areas of the document - the title, the meta data, the heading tags and the body of text content. They also attempt to automatically measure the quality of the document (through complex systems beyond the scope of this guide).
In link analysis, search engines measure not only who is linking to a site or page, but what they are saying about that page/site. They also have a good grasp on who is affiliated with whom (through historical link data, the site's registration records and other sources), who is worthy of being trusted (links from .edu and .gov pages are generally
more valuable for this reason) and contextual data about the site the page is hosted on (who links to that site, what they say about the site, etc.).
Link and document analysis combine and overlap hundreds of factors that can be individually measured and filtered through the search engine algorithms (the set of instructions that tell the engines what importance to assign to each factor). The algorithm then determines scoring for the documents and (ideally) lists results in decreasing order of importance (rankings).
Information Search Engines can Trust
As search engines index the web's link structure and page contents, they find two distinct kinds of information about a given site or page - attributes of the page/site itself and descriptives about that site/page from other pages. Since the web is such a commercial place, with so many parties interested in ranking well for particular searches, the engines have learned that they cannot always rely on websites to be honest about their importance. Thus, the days when artificially stuffed meta tags and keyword rich pages dominated search results (pre-1998) have vanished and given way to search engines that measure trust via links and content.
The theory goes that if hundreds or thousands of other websites link to you, your site must be popular, and thus, have value. If those links come from very popular and important (and thus, trustworthy) websites, their power is multiplied to even greater degrees. Links from sites like NYTimes.com, Yale.edu, Whitehouse.gov and others carry with them inherent trust that search engines then use to boost your ranking position. If, on the other hand, the links that point to you are from low-quality, interlinked sites or automated garbage domains (aka link farms), search engines have systems in place to discount the value of those links.
The most well-known system for ranking sites based on link data is the simplistic formula developed by Google's founders - PageRank. PageRank, which relies on log-based calculations, is described by Google in their technology
PageRank relies on the uniquely democratic nature of the web by using its vast link structure as an indicator
of an individual page's value. In essence, Google interprets a link from page A to page B as a vote, by page A,
for page B. But, Google looks at more than the sheer volume of votes, or links a page receives; it also
analyzes the page that casts the vote. Votes cast by pages that are themselves "important" weigh more
heavily and help to make other pages "important."
PageRank is derived (roughly speaking), by amalgamating all the links that point to a particular page, adding the value of the PageRank that they pass (based on their own PageRank) and applying calculations in the formula (see Ian Rogers' explanation for more details).
Google's toolbar (available here) includes an icon that shows a PageRank value from 0-10
PageRank, in essence, measures the brute link force of a site based on every other link that points to it without significant regard for quality, relevance or trust. Hence, in the modern era of SEO, the PageRank measurement in Google's toolbar, directory or through sites that query the service is of limited value. Pages with PR8 can be found ranked 20-30 positions below pages with a PR3 or PR4. In addition, the toolbar numbers are updated only every 3-6 months by Google, making the values even less useful. Rather than focusing on PageRank, it's important to think holistically about a link's worth.
Here's a small list of the most important factors search engines look at when attempting to value a link:
; The Anchor Text of Link - Anchor text describes the visible characters and words that hyperlink to another
document or location on the web. For example in the phrase, "CNN is a good source of news, but I actually
prefer the BBC's take on events," two unique pieces of anchor text exist - "CNN" is the anchor text pointing to
http://www.cnn.com, while "the BBC's take on events" points to http://news.bbc.co.uk. Search engines use
this text to help them determine the subject matter of the linked-to document. In the example above, the links
would tell the search engine that when users search for "CNN", SEOmoz.org thinks that http://www.cnn.com
is a relevant site for the term "CNN" and that http://news.bbc.co.uk is relevant to "the BBC's take on events". If
hundreds or thousands of sites think that a particular page is relevant for a given set of terms, that page can
manage to rank well even if the terms NEVER appear in the text itself (for example, see the BBC's
explanation of why Google ranks certain pages for the term "Miserable Failure").
; Global Popularity of the Site - More popular sites, as denoted by the number and power of the links pointing
to them, provide more powerful links. Thus, while a link from SEOmoz may be a valuable vote for a site, a link
from bbc.co.uk or cnn.com carries far more weight. This is one area where PageRank (assuming it was
accurate), could be a good measure, as it's designed to calculate global popularity.
; Popularity of Site in Relevant Communities - In the example above, the weight or power of a site's vote is
based on its raw popularity across the web. As search engines became more sophisticated and granular in
their approach to link data, they acknowledged the existence of "topical communities"; sites on the same
subject that often interlink with one another, referencing documents and providing unique data on a particular
topic. Sites in these communities provide more value when they link to a site/page on a relevant subject
rather than a site that is largely irrelevant to their topic.
; Text Directly Surrounding the Link - Search engines have been noted to weight the text directly
surrounding a link with greater important and relevant than the other text on the page. Thus, a link from inside
an on-topic paragraph may carry greater weight than a link in the sidebar or footer.
; Subject Matter of the Linking Page - The topical relationship between the subject of a given page and the
sites/pages linked to on it may also factor into the value a search engine assigns to that link. Thus, it will be
more valuable to have links from pages that are related to the site/pages subject matter than those that have
little to do with the topic.
These are only a few of the many factors search engines measure and weight when evaluating links. For a more complete list, see SEOmoz's search engine ranking factors article.
Link metrics are in place so that search engines can find information to trust. In the academic world greater citation meant greater importance, but in a commercial environment, manipulation and conflicting interests interfere with the purity of citation-based measurements. Thus, on the modern WWW, the source, style and context of those citations is vital to ensuring high quality results.
The Anatomy of a HyperLink
A standard hyperlink in HTML code looks like this:
In this example, the code simply indicates that the text "SEOmoz" (called the "anchor text" of the link) should be hyperlinked to the page http://www.seomoz.org. A search engine would interpret this code as a message that the page carrying this code believed the page http://www.seomoz.org to be relevant to the text on the page and particularly relevant to the term "SEOmoz".
A more complex piece of HTML code for a link may include additional attributes such as:
In this example, new elements such as the link title and rel attribute may influence how a search engine views the link, despite it's appearance on the page remaining unchanged. The title attribute may serve as an additional piece of information, telling the search engine that http://www.seomoz.org, in addition to being related to the term "SEOmoz", is also relevant to the phrase "Rand's Site". The rel attribute, originally designed to describe the relationship between
the linked-to page and the linking page, has, with the recent emergence of the "nofollow" descriptive, become more complex.
"Nofollow" is a tag designed specifically for search engines. When ascribed to a link in the rel attribute, it tells the engine's ranking system that the link should not be considered an editorially approved "vote" for the linked-to page. Currently, 3 major search engines (Yahoo!, MSN & Google) all support "nofollow". AskJeeves, due to its unique ranking system, does not support nofollow, and ignores its presence in link code. For more information about how this works, visit Danny Sullivan's description of nofollow's inception on the SEW blog.
Some links may be assigned to images, rather than text:
This example shows an image named "rand.jpg" linking to the page - http://www.seomoz.org/randfish.php. The alt attribute, designed originally to display in place of images that were slow to load or on voice-based browsers for the blind, reads "Rand Fishkin of SEOmoz" (in many browsers, you can see the alt text by hovering the mouse over the images). Search engines can use the information in an image based link, including the name of the image and the alt attribute to interpret what the linked-to page is about.
In this example, the redirect used scrambles the URL by writing it backwards, but unscrambles it later with a script and sends the visitor to the site. It can be assumed that this passes no search engine link value. SEOmoz
Keywords and Queries
Search engines rely on the terms queried by users to determine which results to put through their algorithms, order and return to the user. But, rather than simply recognizing and retrieving exact matches for query terms, search engines use their knowledge of semantics (the science of language) to construct intelligent matching for queries. An example might be a search for loan providers that also returned results that did not contain that specific phrase, but instead had the term lenders.
The engines collect data based on the frequency of use of terms and the co-occurrence of words and phrases throughout the web. If certain terms or phrases are often found together on pages or sites, search engines can construct intelligent theories about their relationships. Mining semantic data through the incredible corpus that is the Internet has given search engines some of the most accurate data about word ontologies and the connections between words ever assembled artificially. This immense knowledge of language and its usage gives them the ability
to determine which pages in a site are topically related, what the topic of a page or site is, how the link structure of the web divides into topical communties and much, much more.
Search engines' growing artificial intelligence on the subject of language means that queries will increasingly return more intelligent, evolved results. This heavy investment in the field of natural language processing (NLP) will help to achieve greater understanding of the meaning and intent behind their users' queries. Over the long term, users can expect the results of this work to produce increased relevancy in the SERPs (Search Engine Results Pages) and more accurate guesses from the engines as to the intent of a user's queries.
Sorting the Wheat from the Chaff
In the classic world of Information Retrieval, when no commercial interests existed in the databases, very simplistic algorithms could be used to return high quality results. On the world wide web, however, the opposite is true. Commercial interests in the SERPs are a constant issue for modern search engines. With every new focus on quality control and growth in relevance metrics, there are thousands of individuals (many in the field of SEO) dedicated to manipulating these metrics in order to control the SERPs, typically by aiming to list their sites/pages first. The worst kind of results are what the industry refers to as "search spam" - pages and sites with little real value that contain primarily re-directs to other pages, lists of links, scraped (copied) content, etc. These pages are so irrelevant and useless that search engines are highly focused on removing them from the index. Naturally, the monetary incentives are similar to email spam - although few visit and fewer click on the links (which are what provide the spam publisher with revenue), the sheer quantity is the decisive factor in producing income.
Other "spam" results range from sites that are of low quality or affiliate status that search engines would prefer not to list, to high quality sites and businesses that are using the link structure of the web to manipulate the results in their favor. Search engines are focused on clearing out all types of manipulation and hope to eventually achieve fully relevant and organic algorithms to determine ranking order. So-called "search engine spammers" engage in a constant battle against these tactics, seeking new loopholes and methods for manipulation, resulting in a never-ending struggle.
This guide is NOT about how to manipulate the search engines to achieve rankings, but rather how to create a website that search engines and users will be happy to have ranking permanently in the top positions, thanks to its relevance, quality and user friendliness.
Paid Placement and Secondary Sources in the Results
The search engine results pages contain not only listings of documents found to be relevant to the user's query, but other content, including paid advertisements and secondary source results. Google, for example, serves up ads from its well-known AdWords program (which currently fuels more than 99% of Google's revenues) as well as secondary
content from its local search, product search (called Froogle) and image search results.
The sites/pages ranking in the "organic" search results receive the lion's share of searcher eyeballs and clicks - between 60-70% depending on factors such as the prominence of ads, relevance of secondary content, etc. The practice of optimization for the paid search results is called SEM or Search Engine Marketing while optimizing to rank in the secondary results requires unique, advanced methods of targeting specific searches in arenas such as local search, product search, image search and others. While all of these practices are a valuable part of any online marketing campaign, they are beyond the scope of this guide. Our sole focus remains on the "organic" results, although links at the bottom of this paper can help direct you to resources on other subjects.
How to Conduct Keyword Research
Keyword research is critical to the process of SEO. Without this component, your efforts to rank well in the major search engines may be mis-directed to the wrong terms and phrases, resulting in rankings that no one will ever see. The process of keyword research involved several phases:
1. Brainstorming - Thinking of what your customers/potential visitors would be likely to type in to search
engines in an attempt to find the information/services your site offers (including alternate spellings, wordings,
2. Surveying Customers - Surveying past or potential customers is a great way to expand your keyword list to
include as many terms and phrases as possible. It can also give you a good idea of what's likely to be the
biggest traffic drivers and produce the highest conversion rates.
3. Applying Data from KW Research Tools - Several tools online (including Wordtracker & Overture - both
described below) offer information about the number of times users perform specific searches. Using these
tools can offer concrete data about trends in kw selection.
4. Term Selection - The next step is to create a matrix or chart that analyzes the terms you believe are valuable
and compares traffic, relevancy and the likelihood of conversions for each. This will allow you to make the
best informed decisions about which terms to target. SEOmoz's KW Difficulty Tool can also aid in choosing
terms that will be achievable for the site.
5. Performance Testing and Analytics - After keyword selection and implementation of targeting, analytics
programs (like Indextools and ClickTracks) that measure web traffic, activity and conversions can be used to
further refine keyword selection.
Wordtracker & Overture
Overture Keyword Selection Tool Wordtracker Simple Search Utility
Currently, the two most popular sources of keyword data are Wordtracker, whose statistics come primarily from use of
the meta-search engine Dogpile (which has ~1% of the share of searches performed online) and Overture (recently
re-branded as Yahoo! Search Marketing), which offers data collected from searches performed on Yahoo!'s engine (with a 22-28% share). While neither's data is flawless or entirely accurate, both provide good methods for measuring comparative numbers. For example, while Overture and Wordtracker may disagree on numbers and say that "red bicycles" gets 240 vs. 380 searches per day (across all engines), both will generally indicate that this is a more popular term than "scarlet bicycles", "maroon bicycles" or even "blue bicycles."
In Wordtracker, which provides more detail but has a considerably smaller share of data, terms and phrases are separated by capitalization, plurality and word ordering. In the Overture tool, multiple search phrases are combined.
For example, Wordtracker would independently show numbers for "car loans", "Car Loans", "car loan" and "cars Loan", whereas Overture would give a single number that encompasses all of these. The granularity of data can be more useful for analyzing searches that may result in unique results pages (plurals often do and different word orders almost always do), but capitalization is of less consequence as the search engines don't deliver different results based on capitalization.
Remember that Wordtracker and Overture are both useful tools for relative keyword data, but can be highly inaccurate when compared to the actual number of searches performed. In other words, use the tools to select which terms to target, but don't rely on them for predicting the amount of traffic you can achieve. If your goal is estimating traffic numbers, use programs like Google's Adwords and Yahoo! Search Marketing to test the number of impressions a
particular term/phrase gets.
Targeting the Right Terms
Targeting the best possible terms is of critical importance. This encompasses more than merely measuring traffic levels and choosing the highest trafficked terms. An intelligent process for keyword selection will measure each of the following:
; Conversion Rate - the percent of users searching with the term/phrase that convert (click an ad, buy a
product, complete a transaction, etc.)
; Predicted Traffic - An estimate of how many users will be searching for the given term/phrase each month
; Value per Customer - An average amount of revenue earned per customer using the term or phrase to
search - comparing big-ticket search terms vs. smaller ones.
; Keyword Competition - A rough measurement of the competitive environment and the level of difficulty for
the given term/phrase. This is typically measured by metrics that include the number of competitors, the
strength of those competitors' links and the financial motivation to be in the sector. SEOmoz's Keyword
Difficulty Tool can assist in this process.
Once you've analyzed each of these elements, you can make effective decisions about the terms and phrases to target. When starting a new site, it's highly recommended to target only one or possibly two unique phrases on a single page. Although it is possible to optimize for more phrases and terms, it's generally best to keep separate terms on separate pages, as you can provide individualized information for each in this manner. As websites grow and mature, gaining links and legitimacy with the engines, targeting multiple terms per page becomes more feasible. The Long Tail of Search
The "long tail" is a concept pioneered by Chris Anderson (the editor-in-chief of Wired magazine, who runs the Long
Tail blog). From Chris's description:
The theory of the Long Tail is that our culture and economy is increasingly shifting away from a focus on a relatively small number of "hits" (mainstream products and markets) at the head of the demand curve and toward a huge number of niches in the tail. As the costs of production and distribution fall, especially online, there is now less need to lump products and consumers into one-size-fits-all containers. In an era without the constraints of physical shelf space and other bottlenecks of distribution, narrowly-target goods and services can be as economically attractive as mainstream fare.
This concept relates exceptionally well to keyword search terms in the major engines. Although the largest traffic numbers are typically for broad terms at the "head" of the keyword curve, great value lies in the thousands of unique, rarely used, niche terms in the "tail." These terms can provide higher conversion rates and more interested and valuable visitors to a site, as these specific terms can relate to exactly the topics, products and services your site provides.