Every search engine does three jobs: it crawls the web, indexes what it finds, and ranks results for each query.
Understanding these three steps explains almost every SEO tactic. Each one is a stage your page has to pass to show up.
The three steps
- Crawl. Bots follow links and fetch pages across the web.
- Index. The engine stores and understands each page it keeps.
- Rank. For each search, it orders the best, most trusted matches.
What it means for you
- Not crawlable? Google cannot inspect the current content, though a blocked URL may still be indexed from links elsewhere.
- Not indexed? It cannot rank, no matter how good it is.
- Ranking rewards relevance plus trust. Answer well and be credible.
Note: AI search services can retrieve web sources, but their search, user-retrieval, and model-training systems are different. Check the relevant crawler instead of assuming they all work like Googlebot.
Frequently asked questions
What are the three steps a search engine takes?
Crawling (bots fetch pages), indexing (it stores and understands them), and ranking (it orders the best matches for each query).
Why is my page not showing in Google?
It may not be crawled or indexed yet, or it is blocked. Use URL Inspection to check its actual state; a robots-blocked URL can sometimes be indexed without its page content.
Do AI search tools work the same way?
Broadly yes. They crawl, store and then pick sources, so the fundamentals that help Google also help you get cited by the AIs.
Further reading: Google robots.txt limitations.