How to Extract and Use URLs from a Sitemap

A sitemap is an XML file that lists the pages of a website in a machine-readable format. Every URL sits inside a `` tag, which is compulsory and must contain the absolute, canonical version of the address, beginning with http or https and staying under 2,048 characters. The surrounding structure uses a `` element as the container and individual `` elements to wrap each entry. Optional tags like `` (last modified date) and `` can accompany each URL, though Google's own documentation confirms it ignores `` and `` values entirely, making `` the only optional tag worth prioritizing. Most sitemaps declare the `http://www.sitemaps.org/schemas/sitemap/0.9` namespace, signaling conformance to the shared standard established at sitemaps.org.

Before extraction can begin, the sitemap needs to be located. The most common convention is appending `/sitemap.xml` to the root domain, for example `example.com/sitemap.xml`. If that returns nothing, check the site's `robots.txt` file at `example.com/robots.txt` and look for a line beginning with `Sitemap:`, which will point directly to the real file location. Large sites, particularly ecommerce stores and news publishers, often use a sitemap index file rather than a single sitemap. A sitemap index is essentially a directory of individual sitemap files, each capped at 50,000 URLs or 10 MB. Recognizing whether you are dealing with a single sitemap or an index changes how you approach extraction, since an index requires fetching and parsing multiple child files. For sites built on platforms that automate this, sitemap generation handles the structure automatically.

Online sitemap parsers are the fastest starting point. Paste the sitemap URL into the tool, and it fetches, parses, and returns every `` value as a clean list, usually with the option to copy or download as a CSV file. No code required. For larger or more complex extractions, a Python script using the `requests` and `xml.etree.ElementTree` libraries can fetch the XML, iterate over every `` element, and pull the text content of each `` tag into a list or dataframe. This approach handles sitemap index files well because the script can loop through child sitemaps automatically. Dedicated crawling software such as Screaming Frog can also ingest a sitemap URL directly and cross-reference its contents against what the crawler actually finds, surfacing discrepancies between what the sitemap declares and what is reachable.

A clean URL list is the starting point for several high-value workflows. Bulk link checking runs every URL through an HTTP status checker to identify pages returning error responses, redirects, or other unexpected status codes. Content audits use the list to map every page against its metadata, word count, or traffic data, making it easier to spot thin or duplicate content. Redirect mapping during a site migration depends on having a complete pre-migration URL inventory so that every old address can be matched to its new destination. Competitive research is another application: extracting a competitor's sitemap reveals their content structure, how many pages they have indexed, and which topic areas they prioritize. For teams working on local SEO landing pages, a sitemap export also confirms that every generated page is actually being declared to search engines.

Extraction is only as useful as the sitemap itself. A sitemap that lists pages returning error responses, or omits newly published content, misleads crawlers and auditors alike. The `` tag, when accurate, helps indexing platforms prioritize genuinely fresh content over pages that have not changed. Platforms that generate pages programmatically, such as Landing Creator's AI landing page generation, produce and update sitemaps automatically as new pages are created, so the declared URL set stays in sync with what is actually live. If you manage your sitemap manually, auditing it periodically against your live page inventory catches orphaned URLs and missing entries before they affect crawl efficiency or SEO automation workflows.