Methodology and crawl policy
The Website Migration Check compares an old website or sitemap with its replacement and identifies missing pages, incorrect redirects, indexing mistakes, metadata regressions, and unexpected content changes. It reports observable technical differences — it does not, and cannot, promise that a migration preserved rankings, traffic, conversions, or search visibility.
Both addresses can be a domain, homepage URL, or an explicit sitemap URL. If the old site is already decommissioned and its sitemap can no longer be fetched, its content can be pasted directly instead — every old URL is still checked individually against the new site as normal.
Per-URL outcomes
- Preserved
- URL remains valid and expected signals are intact.
- Redirected
- A permanent redirect reaches a valid new-site page.
- Review
- Destination works but one or more migration signals are questionable.
- Missing
- No usable replacement was observed.
- Failed
- The tool could not complete validation.
What gets checked
- Redirect behavior
- Permanent vs. temporary redirects, chains, loops, redirects to the old host or an unrelated domain, malformed Location headers, and too-many-redirects.
- Homepage catch-all
- A large group of unrelated old URLs all redirecting to the new homepage is flagged for review, not automatically treated as incorrect.
- Sitemap coverage
- Unchanged paths, old URLs absent from the new sitemap, new URLs not in the old one, duplicates, and old-host entries left in the new sitemap.
- Indexability
- HTTP status, robots.txt access, X-Robots-Tag, meta robots tags, and canonical URL/host/path on reachable new pages.
- Metadata regression
- Title, description, heading, language, Open Graph, and structured-data changes — only compared when both old and new pages are independently reachable.
- Content equivalence
- A bounded, deterministic content fingerprint and token similarity between old and new pages, when both are available. Reported as "changed substantially," never as "lost," unless conclusive.
- Internal links
- A bounded sample of links on new pages that still point at the old hostname — not a complete internal-link audit.
- Site configuration
- HTTPS availability, robots.txt and sitemap discovery, and sitemap declarations referencing the wrong host.
Deterministic sampling
When an inventory exceeds the page limit, a stable sample is selected: the homepage, shallow top-level paths, the oldest and newest entries by last-modified date, and a representative, path-balanced sample of the rest. The same inputs and analyzer version always select the same set — re-running a check on the same pair reproduces the same sample.
Limits
- Maximum 50 old URLs validated per report.
- Maximum 50 corresponding new pages analyzed.
- Maximum 5 sitemap documents fetched per side, including sitemap indexes.
- Maximum 5,000 sitemap entries parsed per side before sampling.
- Maximum 5 redirects followed per old URL.
- Maximum 3 concurrent requests per target hostname.
- One recent scan per normalized old/new pair reused from cache for 12 hours.
- Pasted old sitemap content is capped at 2 MB and is never stored — only its parsed URL entries are kept.
Privacy and crawl etiquette
Reports are private and non-indexable by default; publishing a report is an explicit, reversible choice that omits private diagnostics. Running a check makes bounded requests to both submitted websites from an identifiable scanner that respects robots.txt, rejects private/internal network addresses, and never forwards cookies, authorization headers, or visitor-controlled headers.
