Why does the history of SEO matter?


As the old saying goes – history repeats itself and this is no different when looking at the history of SEO and the digital marketing field – although the tools might be new and evolving the one aspect of digital marketing that remains constant is change.

The history of Search Engine Optimisation (SEO) allows us to chart the main developments associated with it’s constant evolution – during which these four pillars of SEO stood the test of time –

Four Pillars of SEO

Generally the evolution of SEO so far has highlighted these four key areas when it comes to SEO key factors:

  1. High quality content – original and unique content that meets the needs of the search engine user
  2. On-page factors optimisation – accurate structuring of data on a page around a specific focus keyword, for example page title, page description, page URL and heading 1 – amongst others factors
  3. Off-page factors optimisation – relevant quality links and social media signals that endorse a page authority
  4. Technical factors optimisation – use of meta-data and set up of content and website architecture to communicate effectively with search engine robots to highlight what and how information should be indexed

With the wider adoption of search engines as one of the main tools to navigate the vast information resources available to us on the world wide web it is important to note that this subject area is still at an early stage of development.

So, when did search engines become a highway for commerce?

The 1990’s saw the emergence of mainstream search engines which inspired the need for content providers to make their digital profiles search engines friendly. The year 2000’s saw a major consolidation post the Dot Com bubble and now the big players are ultimately dominating the market – in the 2010’s.

In 2016 the majority of developed economies are using search engines as the highway for commerce in particular Europe, North America and Oceania / Australia. The following table highlights the top 10 developments in the early history of SEO and how these impact on the practices today.

Top 10 developments in the early SEO history and their consequences

Approximate time-line
Search engines features SEO consequence
1990 – 1993
  • First directory based internet search engines
  • Archie is considered to be the first search engine which updated its records on a monthly basis.
  • Most search engines are difficult to search some requiring programming knowledge to execute search commands
Slow updating and importance of being listed in a directory. Manual directory submission was needed to be listed and included in the list of search results. This was the start of Quality Content and Off-page Optimisation factors.
1994 – 1995
  • Natural language processing.
  • Innovations by AltaVista brought the ability to submit to search index and delete your own URL within 24 hours as well as use natural language queries for search.
  • Yahoo! Directory develops a man made search catalogue listing a page description and a URL
  • Lycos introduced the idea of relevance retrieval and included pre-fix matching and relevance retrieval
  • Robots.txt standard for web crawler indexing published
Still relying on the directory submission model, the importance of a well crafted page title and description are increasing. First On-page optimisation factor development.
The idea of relevance of the content in relation to the search results is becoming important.
First attempts on using robots.txt a website can tell a search bot which pages to visit and which to ignore from its search index. It is time of one of the first technical optimisation factors.
1996 – 1998
  • Importance of links and PPC
  • Google comes to the market with its PageRank algorithm which ranks helps to establish an web page authority based on votes they received from links on other pages.
  • Inktomi was one of the first engines to implement a commission based revenue for its online shopping listings and pay for referral – now known as Pay Per Click (PPC).
  • Open Directory project creates the largest volunteer lead human edited internet directory for search results. This categorises the web into themes and offers webmasters the option to submit their web pages for consideration for inclusion.
The importance of web links from authoritative web pages is becoming apparent. Link exchange schemes are used to increase the number of incoming links to a website.
Paid option of SERP advertising is becoming an option and PPC as a model for instant results is becoming a reality.
Open directory requires webmasters to be precise and focused in writing a title and description tag for their web pages.
1999 – 2006
  • Dot com bubble consolidation
  • Around the year 2000 a number of search engines were asked by their stakeholder to show the business models for financial viability and unless they had a good customer base they were taken over merged with others. For example Overture was purchased by Yahoo! Search to make both organisations sustainable.
  • Google buys YouTube in 2006 and creates the second largest search engine
Still relying on the organic search engines listings the importance of PPC as a long terms strategy is increasing since more organisations want to see results on their online investment fast.
Increasingly two way link exchange is becoming less popular. Directories submissions are still important.
  • Schema.org
  • One of the first collaboration between major search engines such as Google, Bing and Yahoo. The inclusion of additional HTML formatting of web pages allows search operators to see information for structured data display such as reviews, events etc.
Now, additional information can be encoded into a web page which gives your pages a potential to be displayed better in SERPS. For example a review rating could appear with stars associated with your link.
  • Legal challenges for search engines
  • Google is facing a number of anti competitive behaviour investigations by the European Union.
  • The right to be forgotten
    On the territoriality of European Union rules apply to all search engine operators operating in EU member states. Search Engines are defined as controllers of personal data. Individuals have the right to ask for removal of information from search results in the EU if it could be interpreted as inaccurate, inadequate, irrelevant or excessive.
SEO impact is twofold – for when conducting SEO for an individual and their personal data is indexed in SERPs which is inaccurate, inadequate, irrelevant or excessive.
On the other hand when developing content for SEO in the EU it is important to bear in mind that this content could be requested for removal by third parties.
This limitation does not apply to search engines outside the EU
2015 –
  • Merger of Search Engines and Social
  • Facebook is receiving an increasing number of search queries due to the vast numbers of users
Each social media platform with relevant target audience is increasing for SEO attention. Basic presence on relevant networks and consistent use of hashtags for the multi platform organic results in important.

One of the key lessons that we can learn from SEO history is that focusing attention on “gaming the search engine ranking system” by identifying key variables in the search engine algorithm is a short term strategy and is counter productive in the long run – unless black hat SEO is the only strategy in your area.

For example, the first search engines were mainly focused on the on- page factors – hence the first optimisers focused on SEO techniques that increased keyword density on a web page resulting in some cases with pages that included white text on white background so that this text was only read by a search engine bot. This is now easily identified by search engines as SPAM and can harm your website. Similarly buying high number of incoming links as well as comments spam was a technique used in the 2000’s.

In case of Google the names Penguin and Panda are some of the major SPAM algorithm updates which are regularly refined and MOZ keeps track of these developments by Google algorithm here.

Webmaster guidelines for search engine optimisation

Search Engines release and regularly update their search engine user webmaster guidelines for search engine optimisation –

How does the history of SEO impact on current and future practices?

When compared to the history of print communications, search engine optimisation (SEO) is still in its early years of developments. However, the four trends that emerged over the years are all focusing on improving customer experience – in this case it is the search engine users. If a search engine does not return the most relevant search results fast chances are it will be overtaken by a competitor.

Historically, the market leader for search engine results in the vast majority of markets is Google but it was designed for the English speaking audiences. However, a large population of the world speaks other languages hence their own preferred search engines – China has Baidu; Russia is dominated by Yandex, Japan is dominated by Yahoo! Japan and Naver in South Korea.

The use of search engine optimisation (SEO) and social media optimisation (SMO) are becoming the two “below the line” marketing communications channels where organisations engage in order to sustain their online profiles visibility. Compared to the traditional TV and Radio advertising the calculations of Return on Investment (ROI) and targeting of the specific audiences for example through Pay Per Click (PPC) are the two key reasons television advertising budgets are reduced in favour of SEO and SMO.

Increasingly cross platform content strategies are developed to incorporate the what would have been a TV commercial for online video optimisation and engagement on social networks such as YouTube, Vimeo and other platforms.

To keep up with the constantly evolving nature of search engines industry generic industry analysis reports from publishers such as Mintel and ComScore and IAB Europe are a good starting point. A number of tools such as the wayback machine also allow you to trace the historic development of a website as captured by this tool https://archive.org .

Why do we learning the history of SEO?

The patterns of innovation in search engines technology development tends to be constant over the years. The key search engine for the Western European market as well as the vast majority of the world in the last ten years is Google. There is a chance that this domination will not last since the European Union do not like monopolies but at present it is likely that Google will remain a search engine of choice for the majority of users. This means that optimisation of online profiles would have to focus in the first place on the user, then on Google and then other search engines such as Bing and Yahoo.

One of the key aspects of history of SEO is learning by doing and ethos of continuous improvement. This is done through regular monitoring of keywords being used on your digital profiles and on the search queries bringing up your digital profiles. Since marketing is about understanding the consumer needs and offering the solutions to these needs in the form of services and products SEO is a good market research tool and a successful SEO campaigns tend to be based on good historic analysis of how your consumers engage with your digital profiles in the past and how you were able to serve their needs with relevant information.

How do you learn from your recent SEO history?

History of SEO or any other digital strategies heavily relies on your own practices and reflection on your digital campaigns success and failures. One of the key tools for your website or social media platforms performance is your success of SMART (Specific, Measurable, Achievable, and Timely) objectives tracking. Your digital marketing efforts tend to have an overriding business objective – making it clear for you is always important. In the Digital Business Maturity Model we recognise that at the basic level it could be simply to get customer enquiries for your organisation which is an objective but to measure it we need to capture data which could be done electronically through the tracking of enquiry forms submissions or simply taking at every interaction with the customer the information on how they heard about you in the first place.

Setting and evaluating SMART objectives is not a simple process especially since the first time you are facing the task for your business you are unlikely to have a feeling of what is Achievable and Realistic. The specific part or an objective usually mean that it is focused on a particular aspect such as website visitors, compilations of your business goals. The level of achievable will very much depend on your resources (financial as well as time). For example if it takes to create at least two day to research and develop a blog post for one person and that individual has only half a day a week to spend on this task – chances are they are only going to be able to produce one blog post a month. Hence saying that a blog post a week will be published is unrealistic and not achievable. Whilst in SEO the past performances suggest that number 1 SERP ranking is likely to bring the highest number of visitors for that particular keyword chances of one organisation ranking for all its relevant terms are very small in particular in competitive industries.

See how search engine users view and interact with Google SERPS. Other tools that help in SEO are search trends as well as past search behaviour tracked by tools such as Google Analytics to help you with reflections on your own historic performance. A routine followed by successful digital agencies is to allocate specific time in a week – say Friday afternoon where latest developments in the industry are identified and implications are reviewed for subsequent activities for planning and prioritisation of lessons learned – Change is the only constant.

Why not join our free Massive Open Online Course in Digital and Social Media Marketing or if you are in Salford you can benefit from commercial training courses such as Digital Marketing Strategy and Search and Socail Media Marketing. All of these courses are developed in clsoe collaboations with industy and offer some insights into the ongoing change faced by organisations working in SEO.