/v1/sitemap endpoint
Full URL list from sitemap.xml. Falls back to shallow crawl if no sitemap exists.
Language:
POST
/v1/sitemap
5 creditsAttempts to fetch and parse sitemap.xml (and sitemap index files). If no sitemap is found, performs a shallow crawl of the homepage to discover linked pages. Useful for bulk-scraping or building crawl queues.
Request body
| field | type | req | description |
|---|---|---|---|
url | string | yes | Root URL of the site. Must include protocol (https://). |
options.maxPages | number | no | Max URLs to return. Default: 500. |
options.timeout | number | no | Timeout in ms. Default: 30000. Max: 60000. |
Example
curl -X POST https://contextforai.ashutoshswamy.in/v1/sitemap \
-H "Authorization: Bearer cfai_your_key_here" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com"}'Response
200 OK
{
"url": "https://example.com",
"source": "sitemap.xml",
"count": 48,
"pages": [
{ "url": "https://example.com/", "lastmod": "2026-07-01", "priority": 1.0 },
{ "url": "https://example.com/about", "lastmod": "2026-06-15", "priority": 0.8 }
],
"extractedAt": "2026-07-01T10:00:00Z"
}