Updated August 2026: this article replaces an obsolete Flash tutorial with current, accessible guidance.
Search systems generally work in three broad stages: crawling, indexing and serving results. Understanding the distinction makes technical SEO problems much easier to diagnose.
Crawling
Automated crawlers discover URLs through links, previous visits and sitemap files. They request a page much like a browser. A broken link, server error, robots rule or access restriction can prevent that request from succeeding.
Indexing
After fetching a page, a search engine analyses its text, images, video, canonical signals and relationship to other pages. It may store the page in its index, but successful crawling does not guarantee indexing. Duplicate, low-value or inaccessible content may be excluded.
Serving results
When somebody searches, the engine chooses results it considers relevant and useful for that query and context. Many systems and signals contribute; there is no single setting that determines the order.
Website owners should keep important pages linked from ordinary navigation, return correct HTTP status codes, maintain a clean XML sitemap and monitor Search Console. Investigate the affected stage instead of treating every visibility problem as a “ranking” problem.
Further reading: Google’s explanation of how Search works.