How Should You Manage Noindex in WordPress?

Table of Contents

“Index Selection” — A Safety Feature That Applies Noindex Only to Archive Pages and Never Touches Posts or Pages

The previous two articles covered structured data and max-image-preview:large.

Both dealt with the question of how to make content appear more effectively in search. This article takes the opposite approach: what should be kept out of search results? In other words, it focuses on controlling what gets indexed.

Why Remove Anything at All?

WordPress is generous by default.

It leaves date archives, tag archives, author archives and even search-results pages open to indexing.

The problem is that most of these pages are not really content in their own right. Open a date-archive page for “June 2026,” for example, and all you see is a list of posts published that month.

There is no reason for a reader to arrive at this page through search. But when hundreds of pages like it are indexed, the ratio of “real content” to “listing pages” deteriorates across the site.

I see this as a dilution of the site’s content-quality signals. That is why pages that do not need to appear in search should be excluded explicitly.

I started by establishing absolute rules.

When designing this feature, I first decided what must never be touched.

Those are individual posts and pages. No combination of settings can ever apply noindex to either one.

This is not an option; it is a rule enforced at the very top of the code. If is_singular() returns true, the request proceeds exactly as it is, regardless of what any setting says. There is no code anywhere in the file that can run before or bypass this early return.

Why make it so strict? I consider this one of the most dangerous failure points an SEO plugin can have.

I have heard of multiple cases where someone misclicked in the settings or accidentally combined several checkboxes, causing every post on the site to be set to noindex. Search traffic can drop to almost zero overnight.

Recovery takes time, too. I wanted to make this kind of incident structurally impossible.

So rather than validating the option values, I designed the system so that posts and pages skip this filter altogether. It does not matter what the user selects. They are not part of the target set in the first place.

The feature applies to a clearly defined set of page types.

Date/monthly archives, tag archives, author archives, search-results pages, pages 2 and beyond of the home page (blog listing), and category archives explicitly selected by the user. They are all “listing” pages. They are not original works themselves, but pages that gather and display original works.

Why Use Meta Noindex Instead of robots.txt

I also chose the implementation carefully.

Instead of using robots.txt to block crawling, the plugin uses only the <meta name="robots" content="noindex"> tag.

The reason is counterintuitive. If robots.txt blocks the page, Google cannot revisit it at all.

But to remove a page that has already been indexed, Google has to revisit it and confirm, “This page says noindex.” If robots.txt blocks it, that check can never happen.

That creates a paradox: a problematic page that has already been indexed may remain in the index indefinitely.

You blocked it because you wanted it gone, but in practice it never gets removed. Meta noindex works differently.

Because Google can continue visiting the page and check the directive, this is the right approach for actually removing an already indexed page from search.

It took three rounds of trial and error to arrive at the current structure.

There is one thing I want to be candid about. The current “one of three options per type” structure was not how I designed it from the start. I revised it three times.

The first version bundled the home page, categories, tags, dates, authors and search into a single checkbox. After using it, I saw the problem: different users needed different scopes, but they could not configure each type separately.

The second version split the settings into six checkboxes, taking Rank Math’s “Archive Subpages” feature as a reference. That introduced a different problem.

For dates, authors, tags and search, “noindex everything” and “noindex only paginated pages” existed as entirely separate checkboxes. If both were enabled, the second checkbox became a dead switch with no effect.

If everything was already set to noindex, pages 2 and beyond were obviously included. From the user’s perspective, there was no way to know that one of the two selected checkboxes meant nothing.

In the third version, I reorganized the system around the principle of “one setting per type.”

Dates, authors, tags and search each have three radio-button choices: keep indexed, noindex everything, or noindex pages 2 and beyond. Because each type can have only one of three states, the structure cannot produce overlapping settings.

I changed the design so that the problem of one option becoming a dead switch among the four possible combinations of two checkboxes could not arise in the first place.

What I learned from this experience is that redesigning a system so a bug cannot occur in the first place is a much more fundamental solution than fixing bugs one by one.

The home page must never be excluded wholesale.

Categories are handled slightly differently. I kept the existing “select manually” approach and added a separate option beneath it: “Noindex pages 2 and beyond for categories that are not selected.”

For the home page—in other words, the blog post listing—I did not provide a “noindex everything” option at all.

The only available choice is to noindex pages 2 and beyond.

Removing the entire home page from search would mean blocking the site’s main gateway for organic traffic,
I judged this to be harmful in general. Neither Yoast nor Rank Math offers “noindex the entire home page” as a default option.

After confirming this, I decided to remove the option entirely. I believed making it impossible for users to click by mistake was more reliable than displaying a warning.

Automatically cover future categories as well

While building the feature, I found one more gap.

Category settings applied only to categories selected manually. That meant an administrator had to return to the settings screen and check the box every time a new category was created.

There was no automatic coverage. Custom taxonomies such as WooCommerce product categories and real-estate listing types were excluded from the scope entirely.

That was clearly a flaw, especially since the plugin was meant to be used across multiple sites with different themes, not just on one particular site.

So I added the same three-choice structure used for dates, authors, tags and search to category archives as well.

Choosing “everything” or “pagination” applies the setting automatically not only to existing categories, but also to any categories created in the future. There is no longer any need to check them one by one.

Custom taxonomy archives were added as a fifth type using the same approach.

WordPress core’s is_tax() returns true only for taxonomies other than categories and tags, so it never overlaps with the existing settings.

On sites without custom taxonomies, this setting simply has no effect.

So users who had saved earlier settings could carry them forward unchanged

Changing the structure three times meant that some users had already saved settings in earlier versions.

I did not want their settings to change unexpectedly with a single update.

So I made the new code read the old option keys and automatically convert them to the new format.

Users who had enabled both “everything” and “pagination”—the problematic configuration from the previous version—were migrated to full.

Because the live site was already serving everything as noindex, this ensured that the output would not suddenly change during migration.

Once the user clicks Save, the settings are stored again using the new schema, and the old keys are no longer used.

The Bottom Line on Noindex Management

Index selection is not a flashy new feature.

It is a basic but potentially critical function that precisely removes list-style pages that have no reason to appear in search.

While building it, I treated two principles as non-negotiable.

Posts and pages must never be touched under any circumstances, and the settings structure itself must not allow overlapping options or dead switches. I do not see the three redesigns as something to be embarrassed about; they were part of the process of continually closing gaps that emerged through real-world use.

In the next article, I will take a closer look at the difference between robots.txt and meta noindex, and explain the right way to remove pages that have already been indexed.

SOHO SEO
LUX Content Migration

This article is displayed with the DECOBLOCKS plugin applied.

DecoBlocks