The Four Stages of Google Indexing in SOHO SEO
In my last article, I argued that search traffic hasn’t disappeared—it has simply changed form.
But saying that “the form has changed” isn’t enough. Unless you understand what has actually changed and the order in which Google processes a single article, you have no meaningful way to respond.
I see the process as four stages: crawling, rendering, indexing, and ranking. Trying to do SEO without understanding that sequence is like taking medicine without knowing which part of your body is sick.
Stage 1: Crawling — The moment Google discovers your article
Crawling is the stage at which Googlebot visits your site and maps out its pages.
This is where people make one of their biggest mistakes: assuming that “if I publish an article, Google will find it automatically.” That’s wrong.
Googlebot’s visit frequency and coverage are not unlimited. There is a concept called a crawl budget. Put simply, Google has a finite amount of time and resources to spend on any one site.
That is why I don’t see a sitemap as merely a “file listing your articles.”
I see it as a map designed to help Google use its crawl budget as efficiently as possible.
The core sitemap does not properly carry image information. I added the <image:image> tag so that, when Googlebot visits, it can discover not only the text but also the featured image.
Making more information available in a single crawl—that is the best thing I can do at the crawling stage.
A news sitemap follows the same logic.
It does, however, need to be handled somewhat differently.
The news sitemap specification requires that it include only articles published within the past 48 hours.
Why is the window so strict? Because Google uses this sitemap to check what is happening right now.
I think of it as a freshness signal. There is little point in forcing articles older than 48 hours into the feed. From a crawler’s perspective, they are more likely to be noise.
There is one more thing. Redirects and 404 management are surprisingly important at the crawling stage.
When the link structure changes but old URLs are left to die, Googlebot keeps following those dead links and wastes crawl budget. I’ve seen this happen on multiple sites.
Sites that overhaul their URL structure without setting up redirects often see crawling slow down—even after new articles are published.
That is why I built 301/302 redirect management and 404 logging into SOHO SEO.
This is not a glamorous feature. But it is basic maintenance—the essential housekeeping needed to keep the first gate, crawling, in good order.
Stage 2: Rendering — How Google “sees” your page
If crawling is about discovering that a page exists, rendering is the stage where Google actually executes the page like a browser and checks its final appearance.
This is another area where many people get confused. They assume that having the content in the HTML source is enough.
But Google actually renders the page and evaluates it as closely as possible to how it appears to a human visitor.
The mobile viewport was the most important consideration for me at this stage.
Google now uses mobile-first indexing. In other words, it judges a page based on the result rendered in a mobile environment.
A page with a missing or incorrectly configured viewport meta tag starts at a disadvantage before it even reaches the later stages.
Images get cropped, text spills off the screen, or clickable elements overlap.
I added a viewport-settings check to the SOHO SEO checklist.
It may seem minor, but if this is not set up correctly, doing everything else right will not make much difference.
Meta tags such as Open Graph and Twitter Cards are also important during rendering.
They affect not only how Google renders the page, but also how your article appears when it is shared on KakaoTalk, Facebook, or Twitter.
I cannot understand people who dismiss this as “not part of SEO.”
If the preview is broken when a link is shared, people are less likely to click it in the first place. When click-through rates fall, Google can treat that as a signal too.
That is why I built SOHO SEO to automatically output everything from og:title, og:description, og:image, og:type, and og:url to Twitter Cards.
It is a basic feature for sites that do not use Yoast or Rank Math. If another SEO plugin is already active, it quietly does nothing to avoid generating duplicate output.
The rendering stage is a battle over what users can see.
Stage 3: Indexing — The moment your page is added to Google’s database
A page is not automatically indexed just because it has been crawled and rendered.
Indexing is the stage at which Google decides whether to add a page to its database and consider it a candidate for search results.
The issue I paid the closest attention to here was noindex management.
By default, WordPress leaves date archives, tag archives, author archives, and even search-results pages open to indexing.
That is a problem.
These archive pages often do nothing more than list posts without offering any original content. If hundreds of them are indexed, they dilute the site’s overall content-quality signals.
So when I built the “Index Selection” feature, I established one absolute rule: individual posts and static pages must never be eligible for noindex, regardless of the combination of settings. This is enforced at the very top of the code with an early return.
The only targets are date, tag, and author archives; search-results pages; page-two-and-beyond home listings; and category archives that the user has explicitly selected.
The method matters too.
I did not use robots.txt to block crawling altogether.
Instead, I use a meta noindex tag. The reason is straightforward.
If you block a page with robots.txt, Googlebot cannot revisit it at all. As a result, a page that has already been indexed may remain in the index indefinitely.
You create the paradox of blocking a page because you want it gone, only to find that it is not actually removed.
A meta noindex tag is different. Google can continue visiting the page and confirm the instruction—”do not index this”—each time.
If a page is already causing problems and you want to remove it from the index, this is the right approach.
This is one area where my judgment differs from the approach taken by competing plugins such as Rank Math, even though I used them as a reference.
Structured data is another essential part of the indexing stage.
Outputting the headline, featured image, publication date, modified date, author, and publisher through Article JSON-LD is not merely decorative—something added to make the page look better.
It explicitly gives Google the information it needs to classify the page and assess its E-E-A-T—experience, expertise, authoritativeness, and trustworthiness.
The indexing stage is a contest over classification and inclusion.
Stage 4: Ranking — The final gate that determines visibility and entry into Discover
The final stage is ranking.
This is where Google orders the indexed pages it may actually show users and decides whether they are eligible to appear in recommendation feeds such as Discover.
I see this stage as the culmination of the “trust signals” discussed in my previous article.
Is the featured image at least 1,200 pixels wide? Is max-image-preview:large guaranteed in the robots meta tag? Is the headline free of clickbait? These are the reasons the SOHO SEO checklist scores and displays these items.
Even if a page passes the first three stages, it will not appear in Discover if it lacks sufficient trust signals at the ranking stage.
I also believe internal links affect this stage.
When related articles are linked to one another, Google can recognize that the site covers a subject in depth.
That, in turn, becomes a signal of authority.
So what actually matters?
Crawling, rendering, indexing, and ranking. The four stages are sequential, but they are also independent of one another.
If crawling fails, the next stage never begins. If rendering breaks, the page is at a disadvantage during indexing.
Without indexing, ranking is impossible. That is why I believe an SEO plugin needs to address all four stages. A plugin that handles only one stage is only doing half the job.
Many people reduce SEO to “writing an article and adding keywords.”
In practice, however, it means managing the entire four-stage pipeline.
In the next article, I will map out what an “SEO plugin” actually touches—from meta tags and structured data to sitemaps and robots directives.