Check API.
Is any site ready for AI agents?
A free, keyless JSON API that scores any public URL for AI-agent readiness — the same checks as the web checker, scriptable.
How do I use the AI agent readiness API? GET /api/check?url=https://example.com for a single URL, or POST /api/check with {"urls": [...]} (up to 6) for bulk. No key required; rate limited to 30 requests per minute per IP. Each result scores 0–9 with per-check details.
GET — single URL
curl "https://toolpantry.app/api/check?url=https://example.com"
POST — bulk (up to 6 URLs)
curl -X POST https://toolpantry.app/api/check \
-H "Content-Type: application/json" \
-d '{"urls": ["https://example.com", "https://example.org"]}'
JavaScript
const res = await fetch('/api/check?url=' + encodeURIComponent('https://example.com'));
const { url, score, maxScore, checks } = await res.json();
Response shape
{
"url": "https://example.com/",
"score": 6,
"maxScore": 9,
"checks": [
{ "id": "robots", "label": "robots.txt welcomes AI crawlers",
"pass": true, "detail": "Search and AI crawlers are allowed", "weight": 1 },
...
]
}
Scoring
| Check | Weight |
|---|---|
| robots.txt welcomes AI crawlers | 1 |
| llms.txt exists | 2 |
| llms-full.txt exists | 1 |
| Markdown content negotiation | 2 |
| JSON-LD structured data | 1 |
| XML sitemap | 1 |
| rel=alternate Markdown links | 1 |
Bulk requests run URLs sequentially to respect Cloudflare subrequest limits — allow a few seconds for 6 URLs. Only public http(s) URLs are accepted; private addresses and non-web schemes are rejected. Fixing the gaps is documented in the open-source kit.
Is the API free?
Yes — no key, no signup. It is rate limited to 30 requests per minute per IP to keep it polite.
What are the limits?
Single checks via GET, bulk via POST with up to 6 URLs per request (Cloudflare subrequest limits). Loop over batches for more.
Can I use it commercially?
The data describes public files of public websites. Use it freely; attribution to toolpantry.app is appreciated.
Where is the source code?
The checker logic, worker, and skills are open source at github.com/wu736139669/markdown-for-agents-worker (MIT).