Skip to main content
ScrapeBadger is built to be called by AI agents (Claude Code, Codex, Cursor and others) as well as by code. Everything on this page also works from plain HTTP.

Set up an agent with one prompt

Give your agent this prompt:
SKILL.md tells the agent everything it needs:
  • how to get a key and connect MCP or the CLI;
  • which endpoint fits a task;
  • how to keep results small and what things cost.
The full endpoint catalog, with the MCP tool name for every endpoint, is at /SKILL/endpoints.md. With the CLI, one command installs the skill into Claude Code and Codex and registers the MCP server:

Get a key without copying one

There are two ways an agent can get a key without anyone pasting it.

Device sign-in

scrapebadger login prints a short code and opens https://scrapebadger.com/device. You sign in, check the request and approve it. That creates a new API key on your account, which the CLI receives directly, so no key passes through the conversation. Under the hood:
  1. POST /v1/public/agent/device/code returns a device_code, a user_code and a verification_uri.
  2. The client polls POST /v1/public/agent/device/token until it gets the key or one of authorization_pending, slow_down, access_denied or expired_token.

Trial keys (no account)

An agent with a shell can mint a free trial key before you sign up:
A trial key comes with these limits: To keep using the key, open its claim_url (scrapebadger connect prints it). Signing in there attaches the key to your account as a normal key. Unused trial credits are not transferred. Over HTTP:
  1. POST /v1/public/agent/trial/challenge returns a challenge and a difficulty.
  2. Find a decimal nonce such that sha256(f"{challenge}:{nonce}") has at least difficulty leading zero bits.
  3. POST /v1/public/agent/trial with {"challenge", "nonce", "label"} returns the key, its claim_url and a notice to show to the user.

Quote a call before running it

POST /v1/quote is free. It runs nothing and charges nothing.
The response contains:
  • credits: the min, max, base and per-item cost;
  • usd: USD per 1,000 calls at the list rate and the best plan rate;
  • valid and errors: your parameters, checked against the endpoint, with each bad field named;
  • latency: the endpoint’s p50, p95 and p99 over the last 30 days.

Keep large results small

These query parameters work on any read endpoint. They change what is sent back, never what is billed.
If a jq expression fails, the response is {"_view_error", "data": <full result>}, so the paid call is never wasted.

Results are not stored

ScrapeBadger does not cache or store responses. Each call fetches fresh data and is billed, so an identical repeat is billed again, and an earlier result can’t be read back. Decide what you need before you call, and narrow the call itself with the views above. GET /v1/results/{result_id} has been removed and answers 410 Gone with "error": "results_removed". Responses no longer carry an X-Result-ID header.

Run long calls asynchronously

Add async=true to the query string, or send Prefer: respond-async, wait=25. This works on almost every endpoint and with any HTTP method.
  • If the call finishes within the wait (0–120 s, default 25), you get the normal response.
  • Otherwise you get 202 with a request_id and next_action:
Poll GET /v1/requests/{request_id}. It is free. It returns 202 while the call runs, then the original status, headers and body. The finished result is held in memory only, never written to disk, for at most 15 minutes so you can collect it. The call is billed once, when it completes. A call that never completes is never billed. Don’t re-send the original request: that runs, and bills, a second call. To make a resend safe, give the submit an Idempotency-Key header: a resend with the same key returns the first call instead. To have the result POSTed to your server instead of polling, add callback_url=https://…. See Async requests and webhooks.

Report a problem

POST /v1/feedback is free.
  • Use "type": "bug" for an empty or malformed result on valid input, a misleading error, or a charge that doesn’t match the quote.
  • Use "type": "feedback" for an API or field you needed and couldn’t find.
Include the failing call’s X-Request-ID so we can find it.

MCP for agents

The hosted MCP server has two endpoints:
  • https://mcp.scrapebadger.com/mcp has all endpoint tools plus the helpers below.
  • https://mcp.scrapebadger.com/mcp/lean has only the helpers, for clients that load every tool definition into context:
    • search_endpoints and describe_endpoint
    • quote_call and call_endpoint (which accepts jq, fields, max_items and summary)
    • get_request
    • report_bug and send_feedback
    • get_account_info
A result too large for the context window comes back as a preview. To get the rest, call again with jq, fields or max_items to narrow it; the new call is billed. A call still running after about 45 seconds returns a request_id; use get_request to fetch the result. See MCP.

Out of credits

A 402 response body includes upgrade_url, free_credits_url and pricing_url. For a trial key, it includes claim_url instead.