An XML sitemap is a machine-readable inventory of URLs a publisher wants discovery systems to know about. It is useful when important pages are new, numerous or not yet well connected through ordinary links. It also provides a concrete list that a site owner can compare with the website's intended public content. A sitemap should describe the publishing system accurately; it should not become a forgotten export that continues listing pages after they disappear.
Before generating a file, decide what belongs in the inventory. A public article, service page or product category may qualify. An authenticated account screen, a temporary preview and a tracking variant of an existing page usually raise different questions. The decision comes from page purpose and URL identity, not from the fact that a database happens to contain an address.
Start with a trustworthy publishing inventory
Identify the source that knows which pages are currently published. In a CMS this may be the publication records and their visibility settings. In a static project it may be the content files and route manifest. Generate sitemap entries from that source where possible. Manually maintaining a second independent list introduces opportunities for new pages to be omitted and retired pages to linger.
For a small fictional documentation website, the inventory could contain a homepage, six topic pages and twelve tutorials. A useful review compares those nineteen intended public URLs with the generated file. It does not assume every path discovered in server logs belongs there. Use the site crawler to gather another view of the site's linked pages, while remembering that a bounded crawl is not an authoritative content database.
Use the correct XML structure
A normal URL sitemap uses a urlset root containing URL entries. A sitemap index points to other sitemap files. Review the actual fetched document to determine which structure it uses. A successful HTTP response alone is insufficient: an application can return status 200 while serving an HTML error screen at a path ending in .xml.
The example below shows a minimal URL sitemap. It uses a placeholder origin and a single illustrative page. Replace these with the intended production URLs. Generate well-formed XML with a suitable serializer or generator rather than assembling arbitrary values through unsafe string concatenation.
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url><loc>https://example.com/tutorials/first-project</loc></url>
</urlset>Keep URL choices consistent with canonical decisions
If several addresses represent the same document, decide which one the sitemap should advertise. Compare that decision with internal links and canonical declarations. A file full of tracking URLs or obsolete hostname variants makes the inventory harder to interpret. It can also conceal inconsistencies in how the application constructs links and metadata.
Review the canonical URL explanation before removing variants indiscriminately. Some URLs differ because they represent distinct languages, products or genuinely different documents. The objective is a coherent list of intended public pages, not an artificially short list that loses meaningful destinations. Write down the inclusion rule so another editor can apply it to future content without guessing.
Check reachability and indexing policy separately
Take a sample of listed URLs and request them directly. Inspect their final destination, response status and page content. Note redirects and errors rather than assuming the sitemap has already validated its entries. Then inspect the relevant indexing and crawl directives. Listing a URL does not override an unintended block or prove that a search system will include it.
The sitemap validator performs its stated structural and URL-format inspections on the fetched resource. It does not establish the status of every listed page or certify search inclusion. Use the indexability checker for individual pages and follow important failures into the responsible template or publishing record. Keep each tool's coverage limits alongside the results when sharing a report.
Plan for multiple files as the inventory grows
A large site may divide entries by content type, language or another maintainable grouping. Choose a structure that helps operations: if one product feed fails, an owner should be able to identify which part of the inventory is affected. Track how the index references each child file and verify that all required files are publicly reachable after deployment.
Google's sitemap-building documentation describes supported sitemap formats and the publishing process. Consult the current protocol and search-system requirements when deciding file size and entry limits. Do not treat an old number copied into a project comment as a permanent operating specification. Build validation into the generator so a growing inventory fails visibly before release rather than silently producing an unusable file.
Publish discovery information where it belongs
After deploying a sitemap, confirm its public address and any robots.txt declaration. A local file, a build artifact and a deployed resource are three different stages. Hosting rules may rewrite the path, serve the wrong content type or expose a stale version. Fetch the production address independently and compare its entries with the publishing inventory used for that release.
Submission and discovery feedback can then be reviewed through the appropriate first-party tools. Preserve the submitted URL and date in your maintenance record. If a service reports an error, investigate the returned resource and its children instead of repeatedly resubmitting the same faulty address. The robots.txt guide explains how a declaration fits alongside crawl rules without replacing other configuration decisions.
Maintain the file through ordinary content changes
Test three common operations: publishing a new page, changing a public path and retiring content. The sitemap should reflect each operation according to the documented inclusion policy. When adding modification metadata, ensure it represents meaningful content changes rather than an arbitrary timestamp refreshed by every build. Misleading metadata makes the inventory less trustworthy to both operators and consumers.
For a migration, compare old and new inventories before release and preserve a mapping of important replaced URLs. After release, recheck the index, child files and a representative page sample. The migration checklist covers the broader release process. A reliable sitemap is a maintained description of the website you actually publish, supported by reachable pages and sensible internal navigation; it is not a shortcut around those responsibilities.
Compare an inventory with a discovery graph
| Approach | Evidence | Useful for |
|---|---|---|
| Sitemap inspection | Fetched XML structure and listed URL syntax | Reviewing the published inventory file |
| Crawler | What links and configured seeds reveal | Comparing discovered page relationships |
| Publishing records | What the owner intends to release | Deciding which public URLs belong in the inventory |
Using CrawlNFix for this task
The sitemap validator is a free CrawlNFix tool with no website account or paid-plan requirement. It needs a connection to inspect the public file being served now. Save the resulting evidence for local review if needed. Offline access to the saved result does not fetch child sitemaps or validate newly published pages, so run the online check again after changing the production generator.