Skip to content
pageinspection

The status code is the first thing anything learns about a page

A 404 on a URL nobody requests is housekeeping. A 404 reached from your own navigation is a link you published to nothing, and the useful half of this finding is not the broken page, it is the page doing the linking.

The short answer

A broken link is any internal link that resolves to something other than a working page: a 404, a 500, or a soft 404 that returns 200 with an error message in the body. The soft one is the most damaging, because nothing reports it. Every broken link is a page your own site pointed a crawler at and then failed to deliver.

CriticalAudit check · HTTP status codes

The codes that matter, and what each one promises

A status code is a commitment about what to do next, which is why the difference between two codes that both mean "not here" is worth knowing.

codes.txt
200  here it is                  the only code that carries content
301  moved, permanently          follow it, and update the link
302  moved, for now              follow it, keep asking the old URL
304  unchanged since you asked   nothing sent, and that is a good outcome
404  not here                    may come back, so it is retried for a while
410  gone, deliberately          stop asking. the honest code for a deletion
403  refused                     usually auth or a bot rule, not a mistake
429  too many requests           you are being rate limited, so slow down
500  we broke                    temporary, retried, and it keeps the index
                                 entry alive in the meantime
503  unavailable                 the correct code for planned maintenance

Two of these are under-used. 410 says a page was removed on purpose, and using it for deliberate deletions is a clearer signal than a 404, which is treated as possibly accidental. And 503 during maintenance is the difference between a crawler waiting and a crawler concluding your site broke.

The one to be careful with is 302. A temporary redirect left in place for two years tells every crawler to keep requesting the old address and to keep crediting it, which is the opposite of what a migration wanted. The redirect guide covers what happens when several of these stack up.

The soft 404 is the dangerous one

A soft 404 is a page that says "not found" in words and 200 in its status line. It is worse than a real 404 for one reason: nothing treats it as an error, so it is crawled, indexed, counted as a real page, and reported by your own tooling as working.

The common producers, all of which look reasonable in isolation:

  • A custom error page served by rewrite. Somebody wanted a branded 404, implemented it as a rewrite to /404, and the rewrite returns the status of the page it rendered.
  • A client-rendered app with no server route. The server returns the shell with a 200 and the router decides afterwards that nothing matches. Every unmatched URL on the site is now a successful empty page.
  • A redirect to the home page. The most damaging version, because it produces a 200 on a real page. A user who followed a dead link is dumped at the front door with no explanation, and a crawler learns that thousands of URLs all resolve to one page.
  • An empty result rendered as a page. A product that was discontinued, a category with nothing in it, a search with no matches. Technically correct and functionally a dead end.

A missing page should say 404 and offer a route onward.

Those two are not in tension. Return the correct status code in the header, and render a page with your navigation, a search box and links to the likely destinations. That serves the person who arrived and tells every machine the truth.

What to avoid is the redirect. Sending a dead URL to your home page converts an honest error into a false success, and it removes any chance of noticing the pattern later.

Check the status one URL returns

This reports the code that came back for the URL you enter, and interprets it. What it cannot do is the more valuable half: telling you which of your pages links to a failure, which is a property of your link graph rather than of the failing URL.

Check the status code one URL returns

No signup required. Each free search audits one page, paste any URL to see it in action.

A pass here on a URL that is broken for a crawler is possible, and the reason is usually a bot rule serving a different answer to a different agent. The crawlability guide has the test for that.

Why an error your own links reach is Critical

A 404 by itself is not a defect. Every site on the internet returns them, and a URL nobody should request returning one is the system working. What earns Critical is the qualifier in the check: reached from your own links. That is a page you published pointing at a page that does not exist, and both ends of it are yours.

The reason it outranks nearly everything else is that the cost is paid twice. A visitor who followed the link is gone, having received nothing. And a crawler following the same link spends a request learning nothing, which on a large site is a meaningful share of the attention you get.

The rating is also about what a cluster of these implies. One broken link is an editing slip. Forty in one section is a migration that did not finish, a renamed field, or a template building URLs out of data that changed. The individual 404s are symptoms, and the check earns its place by making the pattern visible before somebody notices the traffic.

Finding the page that links to the failure

The broken URL is the easy half. Here is the half that lets you fix it.

find-source.sh
BROKEN=/guides/sharpenning   # note the typo: this is the dead URL

# which of your pages links to it?
curl -s https://example.com/sitemap.xml \
  | grep -o '<loc>[^<]*</loc>' | sed 's/<[^>]*>//g' \
  | while read -r page; do
      curl -s "$page" | grep -q "href=\"$BROKEN\"" \
        && echo "linked from: $page"
    done

# and is it a soft 404? compare the code against the words on the page
curl -s -o /tmp/body.html -w 'status: %{http_code}\n' "https://example.com$BROKEN"
grep -ioE 'not found|does not exist|no longer available' /tmp/body.html | head -3

A status: 200 in the first command alongside hits in the second is a soft 404, and it is the finding this guide most wants you to catch, because nothing else will report it.

Then fix the link rather than the target. Recreating a page to satisfy a typo leaves the typo in your markup, and it will be copied the next time somebody duplicates that template. Where the old URL genuinely had value, links pointing at it or traffic arriving, redirect it once with a 301 and correct the internal link as well.

Questions this check raises

What is a soft 404 and why does it matter?
A soft 404 is a page that says "not found" to a reader while returning HTTP 200 to a machine. Nothing in a status report flags it, so the URL stays in the index, competes with real pages, and absorbs crawl budget indefinitely. Returning a real 404 or 410 is the fix, and it is usually a one-line change.
Do broken outbound links hurt as much as broken internal ones?
No. An internal 404 is a link you control pointing at a page you own that does not work, which is a direct signal about site quality and a wasted crawl. An outbound link that has rotted reflects the other site changing, and while it is worth fixing for readers, it carries far less weight.
Should a removed page return 404 or 410?
Either works and 410 is slightly more decisive: it states the page is intentionally gone rather than merely missing, and crawlers drop it a little faster. If the content moved, neither is right, and a 301 to the replacement is the correct answer.