WFL
← All articles

How to Check Whether a Site's Pages Are Actually Indexed by Google

A seller sells you 400 articles. Google has 160 of them. How to find the real indexed count, separate an index gap from index bloat, read the not-indexed reasons, and price what you are actually buying.

How to Check Whether a Site's Pages Are Actually Indexed by Google

Checklist item 005: Search the domain with 'site:domain.com' on Google — all pages must be indexed

Open checklist →

A listing says the site has 400 articles. The seller is not lying — there really are 400 posts in the WordPress admin. But Google has indexed 160 of them, and a page Google has not indexed cannot rank, cannot receive organic traffic, and cannot earn you anything.

You are not buying published pages. You are buying indexed pages that rank. Those are three different numbers, and the gap between them is one of the easiest places in due diligence to overpay.

Why the Indexed Count Is the Number That Matters

Content is usually the headline asset in a content site acquisition, and it is usually priced by volume — so many articles, so many words, at so much per article to replace. That arithmetic only holds for content Google has accepted.

Three things make this worth half an hour of your time:

  • It reprices the asset. If 60% of the content is not indexed, the replacement-cost argument the seller is making applies to 40% of what you are paying for.
  • It is a quality verdict you did not have to form yourself. Google crawled those pages and declined to include them. That judgement is free, and it is usually about thin, duplicated or templated content.
  • It predicts your post-acquisition workload. An index gap is not a switch you flip. Getting a few hundred pages indexed means fixing whatever caused Google to pass on them, which is normally a rewrite, not a technical tweak.

This check sits next to, but is not the same as, checking the site for penalties. There, indexing is a symptom — evidence that Google has formed a negative view of the site. Here you are doing an inventory: counting what you are buying and what state it is in.

Step 1: The site: Operator — Useful, but Not for Counting

Everyone starts here, and it is a reasonable place to start as long as you know what the tool is.

Run site:domain.com on Google. You get a list of indexed URLs and a result count.

The count is an estimate. Google says so, and it is frequently out by a wide margin in both directions. Never put the site: number in a spreadsheet and multiply it by anything.

What the operator is genuinely good for:

  • Spot checks. Pick ten article URLs from the site's sitemap at random and search site:domain.com/the-exact-slug. No result means that specific page is not indexed. That is a binary fact, not an estimate, and ten of them give you a rough hit rate in two minutes.
  • Discovery. site:domain.com with no other terms sometimes surfaces subdomains, staging environments, or sections the seller never mentioned. A site:dev.domain.com or site:domain.com/wp-content/ that returns results is worth a question.
  • Shape. Scroll a few pages deep. If the indexed URLs are overwhelmingly tag pages, paginated archives and author pages rather than articles, that tells you what Google thinks the site is made of.

Then stop using it for anything numerical and get the real data.

Step 2: Get the Real Number from the Page Indexing Report

Ask the seller for read-only Search Console access — the same restricted-user access the penalty check requires, so you only need to ask once.

In Search Console, go to Indexing → Pages. Google splits every URL it knows about into two buckets: indexed and not indexed.

Write down the indexed figure. That is the real inventory. Now compare it against two other numbers:

  1. The seller's claimed article count. Ask for it in writing before you look at Search Console, so the answer is not shaped by what you have already seen.
  2. The number of URLs in sitemap.xml. Fetch domain.com/sitemap.xml directly; it is public and needs no permission.

Three numbers, and the relationships between them tell the story:

PatternWhat it means
Claimed ≈ sitemap ≈ indexedHealthy. Proceed.
Claimed ≈ sitemap, indexed much lowerIndex gap. Google is rejecting content. Step 4 tells you why.
Indexed much higher than sitemapIndex bloat. Google has indexed things nobody intended.
Claimed much higher than sitemapThe seller is counting drafts, deleted posts, or pages excluded on purpose. Ask.

Step 3: Separate the Index Gap from Index Bloat

These are opposite diseases and they are priced differently, so do not lump them together.

An index gap is content you are paying for that Google has not accepted. It is a direct valuation problem: the asset is smaller than the listing says.

Index bloat is Google indexing material that was never meant to rank — tag archives, paginated series, internal search result pages, attachment pages, parameter variants of the same article. It is rarely a crisis, but at scale it dilutes the site's topical signal and wastes crawl budget on pages that will never earn anything. It also inflates the site: count, which is sometimes exactly why a seller points you at the site: count.

To tell them apart, look at which URLs are indexed. In the Pages report, click into Indexed and scan the URL list. Count roughly what proportion are real articles versus archive, tag, feed and parameter URLs. A content site where a third of indexed URLs are /tag/ pages has bloat, whatever its total looks like.

Step 4: Read the Not-Indexed Reasons

This is the substance of the check. Search Console groups every excluded URL under a named reason, and the names are precise. Some are completely benign; some are the seller's problem; one or two are yours to fix after the sale.

Google's own page indexing report documentation lists them. Here is what each means when you are the buyer rather than the owner:

Reason as Search Console names itWhat it means to a buyer
Crawled - currently not indexedGoogle fetched the page and judged it not worth including. Usually thin content, duplication, or almost no internal links. At scale this is a quality verdict and the single most important number on this screen.
Discovered - currently not indexedGoogle knows the URL exists and has not bothered to crawl it. At scale, a site-level trust and crawl-budget signal.
Duplicate without user-selected canonicalGoogle found several near-identical pages and picked one. Common on sites built from spun or templated content.
Duplicate, Google chose different canonical than userThe site declared a canonical and Google overruled it. Worth inspecting — it often means the "original" is itself a copy.
Alternate page with proper canonical tagNormal and intentional. Not a finding.
Page with redirectNormal. Expect a lot of these if the site has ever been restructured.
URL marked 'noindex'Intentional exclusion. Check what is excluded — if real articles are noindexed, ask why.
Blocked by robots.txt / Indexed, though blocked by robots.txtThe second is a misconfiguration: Google indexed a page it was told not to crawl, so it has no content to work with.
Soft 404The page returns 200 but looks empty or error-like to Google. Often thin category pages or broken templates.
Not found (404) / Server error (5xx) / Redirect errorTechnical debt you inherit. A handful is normal; hundreds means the site has not been maintained.
Page indexed without contentGoogle indexed the URL but could not read the content — frequently a rendering or JavaScript problem.
Blocked due to unauthorized request (401) / access forbidden (403)Usually staging or member areas leaking into the sitemap.

The number to focus on: Crawled - currently not indexed, as a share of real article URLs. A handful is normal on any site. If it is hundreds of articles on a site whose content is the thing you are buying, Google has already given you its opinion of that content, and it agrees with the content quality assessment you should be doing anyway.

Step 5: Reconcile Against the Sitemap

The Pages report has a filter most buyers never touch. At the top, switch from All known pages to the specific sitemap.

This matters because "all known pages" includes every URL Google has ever encountered — old deleted posts, parameter variants, URLs from scraped copies of the site. Filtering to the sitemap narrows it to the pages the site is actively asking Google to index, which is much closer to the asset definition.

Run the same comparison inside that filter. If the site submits 400 URLs and Google indexes 160 of them, that is unambiguous: the owner is asking for 400 and being refused 240.

Step 6: Ground-Truth a Sample with URL Inspection

Reports aggregate; aggregates hide things. Spend ten minutes on individual pages.

Take ten URLs the seller would consider valuable — the articles they named as traffic drivers, plus a few picked at random from the sitemap — and run each through the URL Inspection tool (the search bar at the top of Search Console).

For each one, note:

  • Whether it is on Google at all
  • The reported last crawl date — articles not crawled in over a year are not being refreshed in the index
  • Whether the user-declared canonical and the Google-selected canonical match; a mismatch means Google is attributing that content to a different URL
  • Whether it was discovered via the sitemap

A site where the named best articles come back as not indexed, or canonicalised elsewhere, is a different acquisition from the one being advertised.

Step 7: Price the Gap

Turn the finding into a number before you negotiate, because "some pages aren't indexed" is easy for a seller to wave away and "you are selling me 160 articles, not 400" is not.

  1. Establish the indexed article count — real articles only, excluding tag, archive and feed URLs.
  2. Express it as a percentage of the claimed count.
  3. Apply that percentage to whatever content-based valuation the seller is using.
  4. Add the cost of closing the gap: if 240 pages sit under Crawled - currently not indexed, assume a rewrite rather than a fix, and cost it at whatever you pay per article.
  5. Check whether the indexed pages are the ones earning. Cross-reference the indexed list against the top pages in the Performance report — a site can have a large index gap that costs nothing if every page that matters is indexed and ranking.

That last point stops you over-correcting. A 400-page site with 160 indexed pages carrying all the traffic may be perfectly fine to buy, as long as you pay for 160 pages and not 400.

What to Do With What You Find

FindingAction
Indexed ≈ sitemap ≈ claimed count, indexed URLs are real articles✅ Proceed
Small gap, reasons are benign (redirects, intentional noindex, canonicals)✅ Proceed
Moderate bloat from tag and archive pages, articles all indexed⚠️ Cheap to fix post-acquisition; mention it, don't over-weight it
20–40% of articles under Crawled - currently not indexed⚠️ Revalue on the indexed count and budget the rewrite
Hundreds of Discovered - currently not indexed⚠️ Site-level trust problem; check it against penalties and traffic history
Large Duplicate clusters across article URLs⚠️ Likely spun or templated content; verify originality before anything else
The seller's named best pages are not indexed, or canonicalised elsewhere⛔ The advertised asset is not the real one
Over half the claimed content not indexed⛔ You are buying a fraction of what is being sold
Seller refuses Search Console access so you cannot check⛔ Walk away

Quick Reference Checklist

Before signing off on this item:

  • Seller's claimed article count obtained in writing, before looking at any data
  • sitemap.xml fetched and its URL count recorded
  • Indexed count read from Indexing → Pages, not from a site: result count
  • Ten specific URLs spot-checked with site:domain.com/slug
  • Indexed URL list reviewed for bloat — proportion of real articles versus tag, archive and parameter URLs
  • Not-indexed reasons reviewed, with Crawled - currently not indexed counted separately
  • Pages report re-run filtered to the submitted sitemap, not all known pages
  • Ten valuable URLs inspected individually, with last-crawl dates and canonical matches recorded
  • Subdomains and staging environments checked for accidental indexing
  • Indexed article count applied to the seller's content-based valuation

This takes about thirty minutes and it is one of the few checks that produces a direct, defensible number to negotiate with. Work through it alongside the rest of the due diligence checklist — and remember that every page in the gap is a page you are being asked to pay for and cannot yet earn from.