The History of SEO and Search Engines

The history of search engines is inseparable from the history of the internet itself. Without the internet, there would be no Google, no Yahoo!, no Bing — and no SEO.

The idea started in the 1960s, when the U.S. Department of Defense began researching resilient communications networks. Secretary of Defense Neil McElroy founded the Advanced Research Projects Agency (ARPA) in 1958, which eventually led to ARPANET — a packet-switched network that went live in 1969 and became the precursor to the modern internet.

On January 1, 1983, ARPANET switched from its original NCP protocol to TCP/IP, the moment many historians mark as the “birth” of the internet. Tim Berners-Lee proposed the World Wide Web in March 1989, and the first HTTP-based pages went public in 1991. ARPANET itself was decommissioned in 1990. By the mid-1990s, commercial web browsers put the internet in ordinary homes — and search engines became essential.

The Early Ideas for Search Engines

Search tools existed before the web. In the 1960s at Cornell University, Gerard Salton and his colleagues created the SMART Information Retrieval System — the foundation for most modern search theory. Many concepts still used today (vector space models, term frequency, inverse document frequency) trace back to SMART.

In the early 1980s, the WHOIS protocol became the first widely used internet lookup tool. It was not a search engine in the modern sense — it queried databases to find information about domain registrations — but it set the pattern of “send a query, get structured results back.” WHOIS is still used today.

Archie (1990)

The first recognizable internet search engine was Archie, created in 1990 by Alan Emtage, a student at McGill University in Montreal. The name is a pun on “archive.” Archie crawled public FTP sites and built a searchable index of the files they hosted. It could not search web pages — the web did not exist yet — but it introduced the core idea: an automated crawler that builds a searchable index.

Gopher (1991)

A year later, Mark McCahill and his team at the University of Minnesota released Gopher, a text-based system for organizing documents across the internet. Gopher was quickly followed by search tools for its content: Veronica and Jughead. McCahill’s work directly influenced URL syntax, and the University of Minnesota later contributed heavily to what became POPmail.

Excite (1994)

The first commercial web search engine, Excite, was created in 1994 by six Stanford undergraduates and launched fully in 1995. It combined web search with weather, news, stock prices, email, and a personalized homepage — arguably the first “portal.” Excite was massively popular through the late 1990s, declined in the 2000s, and today operates as a Bing-powered search page.

Yahoo! (1994)

Yahoo! was founded in 1994 by Stanford PhD students Jerry Yang and David Filo. It started as “Jerry and David’s Guide to the World Wide Web” — a hand-curated directory of websites, organized by category. That directory model was Yahoo!’s original innovation: instead of algorithmic results, editors reviewed and classified sites.

Yahoo! used other search engines (including Inktomi and later Google) to power its web crawling until 2003, when it began generating its own index. At its peak, Yahoo! was the most visited website in the world. It acquired AlltheWeb, Overture, Inktomi, AltaVista, and others, and — for a time — dominated web advertising through its Overture acquisition.

Yahoo! sold Tumblr to Automattic in 2019 and has since been through multiple ownership changes (Verizon, then Apollo). It remains an active portal and search brand, though its search results are today powered by Bing.

WebCrawler (1994)

Also founded in 1994, WebCrawler by Brian Pinkerton at the University of Washington was the first search engine to index the full text of the web pages it crawled — a critical leap. Earlier crawlers only indexed page titles or URLs. WebCrawler let you search the contents of pages themselves. It was acquired by AOL and later InfoSpace, and still operates as a metasearch tool today.

Lycos (1994)

Lycos came out of a Carnegie Mellon research project led by Michael Loren Mauldin. By 1999 it was one of the most-visited sites in the world and offered search, email, web hosting, and social networking features. Like many 90s-era search engines, Lycos pivoted away from core search over time. It still exists as a minor portal site but is no longer a major destination.

Infoseek (1994)

Infoseek, founded by Steve Kirsch in 1994, was the first search engine to sell pay-per-click advertising — a revenue model that still dominates search ads today. Disney acquired Infoseek in 1998 and folded it into Go.com, its short-lived web portal. The Infoseek domain now redirects to Go.com.

AltaVista (1995)

Launched in 1995 by Digital Equipment Corporation, AltaVista was a major leap in speed and scale. Its crawler, Scooter, indexed an order of magnitude more pages than competitors, and its natural-language query support and Boolean operators made it the researcher’s favorite. AltaVista also launched Babel Fish, one of the first web translation services. Yahoo! acquired AltaVista in 2003, and the service was officially shut down in July 2013. Visiting altavista.com today redirects to Yahoo!.

Ask Jeeves (1996)

Ask Jeeves, founded in 1996 in Berkeley, California, was the first major natural-language search engine — users could ask a question in plain English instead of typing keywords. Its ExpertRank algorithm was an early attempt at using backlinks to evaluate page relevance (predating Google’s PageRank by a couple of years). In 2006, Ask Jeeves rebranded as Ask.com and shifted focus to a Q&A community model. It still exists today but has largely exited general web search.

Google (1998)

Google was founded on September 4, 1998 by Stanford PhD students Larry Page and Sergey Brin. (The google.com domain had been registered a year earlier, in September 1997.) Their breakthrough was an algorithm called PageRank, which evaluated a page’s authority based on the quantity and quality of links pointing to it. Instead of ranking by keyword density alone, Google ranked by the web’s own votes of confidence.

The results were dramatically more relevant than everything else on the market. By the early 2000s, Google had overtaken AltaVista, Yahoo!, and the rest to become the dominant search engine — a position it still holds in 2026. Google grew into an entire platform (Gmail, Maps, YouTube, Android, Chrome, Workspace, Cloud), and “to google” entered the dictionary as a verb in 2006.

The Birth of SEO (Late 1990s–Early 2000s)

The moment search engines became commercially important, people started trying to rank higher in them. That is the origin of search engine optimization.

Early SEO was primitive and easily gamed:

• Meta keyword stuffing — packing the <meta keywords> tag with every relevant term, including competitors’ names.
• Keyword density tricks — repeating keywords dozens of times in visible and invisible text (often in 1px white-on-white font).
• Directory and link farm submissions — getting listed everywhere possible to inflate perceived authority.
• Doorway and cloaked pages — serving different content to search engine crawlers than to users.

These tactics worked because early search engines relied heavily on on-page signals. Google’s PageRank upended that by weighting off-page signals (links) — which triggered the next wave of manipulation: link buying, link exchanges, and eventually link farms.

Google’s response came in November 2003 with the Florida Update, the first major algorithm refresh to target manipulative SEO. Thousands of sites lost rankings overnight, and the modern cycle of algorithm updates vs. SEO tactics began.

The Google Algorithm Era (2011–2022)

From 2011 onward, Google rolled out a series of named algorithm updates that each reshaped SEO practice. The major ones:

Panda (February 2011) — targeted thin and low-quality content. Affected about 12% of all search queries. Content farms with lots of shallow, ad-heavy pages lost enormous traffic. Panda was folded into Google’s core ranking system in 2015.

Penguin (April 2012) — targeted manipulative backlink profiles (link schemes, keyword-stuffed anchor text, low-quality links). Unlike Panda, Penguin ran in real time from 2016 onward. See our guide on nofollow link attributes for how modern SEO handles outbound and inbound link qualification.

Hummingbird (September 2013) — introduced natural-language processing and semantic search. Google began understanding the intent behind a query, not just its keywords. Affected roughly 90% of all searches.

Mobilegeddon (April 2015) — made mobile-friendliness a ranking factor on mobile searches.

RankBrain (October 2015) — Google’s first major machine-learning component in the core algorithm. Helped interpret ambiguous, novel, or conversational queries.

BERT (October 2019) — a neural-network-based natural-language model that dramatically improved understanding of context and word relationships in queries.

Mobile-first indexing (2019) — Google began using the mobile version of sites as the primary version for indexing and ranking.

Core Web Vitals (June 2021) — loading speed, interactivity, and visual stability became explicit ranking signals under the “page experience” umbrella.

Helpful Content Update (August 2022) — targeted content clearly written for search engines rather than humans. Integrated into Google’s core ranking system with the March 2024 core update.

Bing and the Rest of the Field

Bing launched on May 28, 2009, replacing Microsoft’s earlier Live Search (which had evolved from MSN Search). The Microsoft-Yahoo search alliance was announced in July 2009 and fully implemented by August 2010, with Bing powering Yahoo!’s organic results. That alliance was renegotiated in 2015 to be non-exclusive. Bing remains the #2 search engine in the U.S. by market share and holds smaller shares globally, where Baidu (China), Yandex (Russia), and Naver (Korea) dominate their home markets.

Privacy-focused alternatives gained traction in the 2010s: DuckDuckGo (launched 2008) blocks trackers and does not personalize results; Brave Search (2021) built its own independent index; Kagi (2022) is a paid ad-free search engine; Ecosia plants trees with its ad revenue. None threaten Google’s dominance, but they show the market for search alternatives is alive.

The AI Search Era (2023–2026)

The biggest shift in search since Google’s founding started in 2023. ChatGPT (launched November 2022) demonstrated that large language models could answer questions conversationally, and search engines scrambled to respond.

Microsoft Bing integrated GPT-4 (now Copilot) in February 2023, making Bing the first major search engine to embed conversational AI.

Google announced SGE (Search Generative Experience) in May 2023 as a Labs experiment. It rebranded and launched broadly as AI Overviews in May 2024, placing AI-generated summaries at the top of many result pages.

New AI-first search products emerged: Perplexity, ChatGPT Search, You.com. Each cites sources inline and answers questions directly, often making a traditional click unnecessary.

The impact on SEO has been significant. An Ahrefs analysis found sites hit by AI Overviews lose on average 24% of their organic traffic, with some losing up to 45%. As of late 2024, AI Overviews appeared on 4.5–12% of queries, heavily skewed toward informational topics (up to 67% in health-related queries). A new discipline — Generative Engine Optimization (GEO) or LLMO — is emerging to optimize for citations inside AI-generated answers.

The fundamentals of SEO have not disappeared. If anything, the AI era has amplified E-E-A-T (Experience, Expertise, Authoritativeness, Trustworthiness) because AI systems prefer to cite authoritative sources. For a practical take on how long it takes to see SEO results in this environment, see our guide on realistic SEO timelines. For analytics and tracking in the AI era, our Google Tag Manager guide covers modern measurement.

Frequently Asked Questions

What was the first search engine?
Archie, released in 1990 by Alan Emtage at McGill University, is generally credited as the first internet search engine. It indexed public FTP sites rather than the web (which did not yet exist), but it introduced the core crawler-and-index model that modern search engines still use.

When did SEO begin?
The term “search engine optimization” emerged in the mid-to-late 1990s as search engines became commercially important. The earliest SEO tactics focused on meta-keyword manipulation and keyword density. The discipline matured in the 2000s as Google’s PageRank made backlinks the central ranking signal, and professionalized through the 2010s in response to named algorithm updates like Panda and Penguin.

Why did Google become dominant?
Google’s PageRank algorithm produced dramatically more relevant results than competitors by analyzing the web’s own link structure — effectively using backlinks as votes of confidence. Combined with a clean, fast interface and a willingness to be patient on monetization, Google’s results quality won the market within a few years of launching in 1998.

How is SEO changing in the AI era?
AI Overviews and generative search are reducing clicks to organic results on informational queries (studies find an average 24% traffic loss on affected pages). The SEO response has shifted toward being cited by AI systems rather than ranking for blue-link clicks: stronger E-E-A-T signals, original research and data, structured content that AI models can extract cleanly, and brand authority in specific topics. Traditional ranking factors still matter — AI systems draw from the same web index — but the competition for a citation inside an AI answer is the new frontier.

Bottom Line

From Archie’s FTP indexing in 1990 to AI Overviews summarizing the web in 2026, search has undergone four major eras: the pre-web crawlers, the commercial search engine boom of the late 1990s, the Google algorithm era, and now the generative AI era. SEO has evolved in parallel — from keyword stuffing to link building to content quality to E-E-A-T to, increasingly, being cited by AI systems.

The one consistent thread: search engines reward content that genuinely helps users find what they need. The mechanics change. The principle has not.