One call, usable content
Scrape a URL into Markdown, links, the HTML, or JSON shaped by your schema; map a site from its links and sitemap; scrape up to 20 URLs as one batch. Every call is an ordinary gated job.
Built
Product / web data
Scrawl
Turn web pages into clean, versioned data you own and can cite.
Scrawl turns a URL, a site, or a cataloged crawl into Markdown, structured JSON, files, and versioned datasets, as owned jobs behind one egress and policy gate. Your own AI extracts to your schema and answers questions with citations to the records it used. Operators work from Telegram, software through ApexAPI and the SDKs, and agents through ApexMCP.
At a glance
- Who it is for
- Teams that need web data they can trust and keep: researchers, operators, and developers feeding software or AI agents.
- What it does
- Scrape, map, and crawl sources; extract files and structured data with your own AI; keep everything as owned jobs and versioned datasets you can query, diff, and cite.
- How it is reached
- Telegram for operators, ApexAPI and the SDKs for software, ApexMCP for agents, and the Scrawl Plugin for the deployment.
- Where it stands
- In developmentNot publicly hostedPricing coming soon
- Next step
- Ask about access Status and pricing
See it work
One URL in. Clean, typed data out.
Ask for Markdown and JSON in one call. Scrawl fetches the page once through its egress gate, converts it to Markdown by rule, and has your own AI fill your schema, then checks the answer against that schema before returning it.
Built ApexAPI, ApexMCP, Telegram, SDKs, and command line
Request
POST /v1/scrape
Authorization: Bearer <ApexAPI key>
Idempotency-Key: trail-runner-2
{
"url": "https://shop.example.com/products/trail-runner-2",
"formats": ["markdown", "json"],
"schema": {
"type": "object",
"properties": {
"name": { "type": "string" },
"weight_g": { "type": "number" },
"in_stock": { "type": "boolean" }
}
}
}
Response (abridged)
200 OK
{
"job_id": "a91c27e04b13",
"status": "completed",
"url": "https://shop.example.com/products/trail-runner-2",
"title": "Trail Runner 2",
"markdown": "# Trail Runner 2\n\nA light trail shoe with a 6 mm drop…",
"json": {
"name": "Trail Runner 2",
"weight_g": 268,
"in_stock": true
},
"truncated": false,
"untrusted_external_content": true
}
- Telegram
/scrape https://shop.example.com/products/trail-runner-2- ApexMCP
apex_scrapy_scrape_url- Python SDK
client.scrape(url, formats=["json"], schema=schema)- Command line
scrawl scrape https://shop.example.com/products/trail-runner-2
untrusted_external_content marks page text that software and agents must treat as data, never as instructions.What Scrawl covers
A web-data product, not a scraping script.
Collect web content reliably and keep what was collected: the files, the records, and the evidence of how they were produced.
Your AI, governed
Extraction to your schema, whole-dataset extraction, and answers with citations run on the AI provider you connect. Page text reaches the model as untrusted content, and every call is metered.
Built
Versioned datasets
Crawls publish immutable versions with stable record IDs and provenance. Compare versions, query by field, search, fetch a record, and quarantine one without rewriting history.
Built
Jobs that finish once
Every operation is an owned job: idempotent submits, admission control, a hard time limit, cancel from any process, and recovery after a restart.
Built
Artifacts and media
Images, video, audio, documents, data files, web assets, and archives are classified by their bytes, validated, stored, and listed in a manifest.
Built
Your proxies, explicit egress
Bring your own proxy pools with encrypted credentials, liveness checks, quarantine, rotation, and domain rules applied before a request leaves.
Built
Scrape, map, batch, and crawl
From one page to a whole site.
Start with the call that fits: one page, the map of a site, a batch of pages, or a cataloged crawl that publishes a dataset. Each runs as an ordinary job through the same gate, so a quick scrape and a long crawl obey the same rules.
POST /v1/scrape
Scrape
One URL in; Markdown, links, the HTML, or JSON shaped by your schema out, from a single gated fetch. The call waits up to a minute for the answer, then hands back the job to collect later.
POST /v1/map
Map
The same-host links on a start page plus its sitemap, deduplicated, up to 500 URLs: the list of what to scrape next.
POST /v1/scrape/batch
Batch
Up to 20 URLs as ordinary scrape jobs, four at a time, policed and billed as one request, with one status to follow.
Deterministic Markdown
HTML becomes Markdown by rule, not by a model: main content, absolute links, tables, and code. The same page gives the same Markdown.
Text only, by design
A URL that serves an executable, archive, or other non-text file is refused before it is read, and a mislabelled one is caught by its bytes.
Everywhere at once
The same scrape runs from Telegram, ApexAPI, ApexMCP, the SDKs, and the command line, through the same gate and job record.
| Job type | Use it to | Produces |
|---|---|---|
| Scrape | Turn one URL into Markdown, links, the HTML, or JSON shaped by your schema, in a single call. | The formats you asked for, kept with the job |
| Map | List a site's same-host pages from a start page and its sitemap before deciding what to collect. | Up to 500 deduplicated URLs |
| Crawl | Run a cataloged Scrapy spider across a site and collect structured items. | Items, crawl log, artifacts, dataset version |
| Fetch | Retrieve one URL exactly as it is served. | Response body and metadata |
| View | See a page the way the crawler receives it before writing extraction. | Captured page |
| Parse | Run a spider's parsing logic against one URL to test structured extraction. | Parsed items |
| Check | Confirm a cataloged spider still behaves as expected before a full crawl. | Check result |
| HTML extraction | Pull text, links, or CSS-selected content without a crawl. | Text, links, selections |
| Regex extraction | Test patterns against a page or a previous job's output, and save the ones that work. | Matches, saved patterns |
Crawl, fetch, view, parse, and check jobs run on the Scrapy framework; scrape and map are built on the same gated fetch.
Your AI, on your terms
Extraction and answers from the AI you bring.
Scrawl does the collecting; the model you connect does the reading. It works only on content you own, through the same jobs and ownership as everything else, and never gains permissions of its own.
Reads
Extract
Turn one page into a JSON object that matches your schema or instruction, validated against the schema and repaired once if it does not match.
Built
Publishes a version
Extract a dataset
Run the same extraction over every record of a version and publish the result as a new, derived version, each record citing the record it came from.
Built
Reads
Ask
Ask a question of your active datasets and get an answer with numbered citations to the exact records it used, or a plain statement that the data cannot answer it.
Built
Confirm to run
Plan
Describe an objective and the planner proposes the jobs. Nothing runs until you confirm, and a confirmed plan passes the same gate as any other job.
Built
Reads
Explain and debug
Ask what a finished job did, or why it returned less than expected, with the job's own evidence as context.
In development
Request
POST /v1/ask
Authorization: Bearer <ApexAPI key>
{
"question": "Which trail shoes under 300 g are in stock?",
"dataset_id": "ds_trail_shoes"
}
Response (abridged)
200 OK
{
"answerable": true,
"answer": "Two are in stock under 300 g: the Trail Runner 2 at 268 g [S1] and the Ridge Lite at 285 g [S3].",
"citations": [
{ "tag": "S1", "title": "Trail Runner 2", "url": "https://shop.example.com/products/trail-runner-2" },
{ "tag": "S3", "title": "Ridge Lite", "url": "https://shop.example.com/products/ridge-lite" }
],
"sources_considered": 8
}
Your provider, your key
Connect an OpenAI-compatible endpoint, including a self-hosted or local model, or Anthropic's API. Keys are stored encrypted for their owner, never shown again, and identified only by a fingerprint.
Pinned endpoints
A custom endpoint is pinned to the address checked when you saved it, so a later DNS change cannot send your pages somewhere else.
Page text is untrusted
Pages reach the model fenced as untrusted content. Instructions hidden in a page are data, not commands, and are tested against pages written to hijack the model.
Usage you can read
Every model call is recorded by provider, model, and feature: tokens, never content. Your usage is readable through the API and from Telegram.
Versioned datasets
From one-off results to data you can rely on.
A crawl's output becomes a dataset you can come back to: versioned, attributable, searchable, comparable, and correctable one record at a time. Datasets are reachable through ApexAPI, ApexMCP, and the SDKs.
- 01
Index
A successful crawl is indexed into a named, stable dataset.
- 02
Version
Each publication is an immutable version with a manifest, provenance, and content hashes.
- 03
Activate
One version is active; earlier versions remain intact for comparison and rollback.
- 04
Retrieve
Search, query by field, fetch a record by ID, and compare two versions record by record.
- 05
Control
Quarantine a single record without deleting it or rewriting the version.
See what changed
Compare two versions and get every record that was added, removed, or changed, matched by stable ID and content hash.
Query by field
Discover the fields a version holds and their types, then filter records with typed conditions instead of downloading everything.
Stable identities
A record keeps the same ID across versions, while each version gives it a version-specific document ID—so a reader can cite exactly what it used.
Provenance
Every version records the job and source it came from, with content hashes that make changes between versions visible.
Agent-ready retrieval
A search-and-fetch contract lets AI agents find records across active versions and read them in full, with resource updates when a new version publishes.
Request
GET /v1/datasets/ds_trail_shoes/diff?from=v_0912&to=v_0919
Response (abridged)
200 OK
{
"totals": { "added": 3, "removed": 1, "changed": 2 },
"entries": [
{ "record_id": "rec_ridge_lite", "change": "changed", "quarantined": false },
{ "record_id": "rec_summit_gtx", "change": "removed", "quarantined": false },
{ "record_id": "rec_trail_runner_3", "change": "added", "quarantined": false }
]
}
Files and media
Retrieve the files, not just a list of links.
Extraction finds candidate files on a page, classifies them by their bytes as well as their name, validates them, and stores what qualifies, with a manifest that accounts for everything it skipped. Today it runs from Telegram in the hosted profile.
Built
IMG
Images
PNG, JPEG, GIF, WebP, SVG, AVIF, and other image files found on or linked from the page.
VID
Video
Directly linked MP4, WebM, and QuickTime files. Streaming-platform ripping is blocked.
AUD
Audio
Directly linked MP3, WAV, OGG, FLAC, AAC, and Opus files.
DOC
Documents
PDF, Word, Excel, PowerPoint, OpenDocument, and RTF files.
DATA
Data files
JSON, CSV, TSV, XML, YAML, and text files, plus job exports as CSV, JSON Lines, or XML.
WEB
Web assets
HTML, stylesheets, scripts, fonts, and manifests, including the page source as served.
ARC
Archives
ZIP, tar, gzip, RAR, and 7z files, enabled per deployment.
MAN
Manifests
One manifest per extraction: what was found, kept, skipped, and why, with per-class CSV summaries.
EVD
Evidence
Job logs, access-barrier evidence, and ZIP bundles of each asset class.
- Every candidate is classified by extension and declared type, then validated before it is stored.
- Robots rules, domain reputation, and size limits apply to each file, not only to the page.
- When a site answers with a challenge or login wall, the job records evidence of the barrier instead of pretending it succeeded.
- Retention and cleanup remove expired artifacts on schedule.
Jobs and evidence
Every operation is an owned job that finishes once.
Whichever surface starts the work, it enters the same lifecycle: one request, one gate, one record of what happened. Retries, crashes, and restarts never run it twice or leave it hanging.
Built
- 01
Request
An operator, application, or agent submits a typed job and receives a job ID.
- 02
Gate
Identity, ownership, target catalog, proxy and egress, abuse, rate, and resource limits are resolved.
- 03
Execute
The dispatcher crawls, fetches, parses, extracts, or checks the approved target.
- 04
Record
Results, artifacts, manifests, and logs stay attached to the job and its owner.
- 05
Publish
A successful crawl can publish an immutable dataset version for retrieval.
Safe retries
Submits accept an Idempotency-Key: a retry replays the first answer instead of starting the work twice.
Admission control
Each caller or account has a bound on active jobs. Past it, a request is refused with a retry time instead of queueing without limit.
A hard time limit
A watchdog stops a job that runs past its limit, together with every process it started.
Cancel from anywhere
A cancel reaches the job whichever Scrawl process is running it.
Restart recovery
After a restart, jobs that were in flight are reconciled instead of left hanging.
Shared state
Jobs and idempotency keys can live in PostgreSQL, so several Scrawl instances share one view of the work.
- Submit
- Start any job type and get its ID back immediately.
- Inspect
- Read a job's state, parameters, and outcome.
- History
- List your recent jobs with their results.
- Cancel
- Stop queued or running work you own.
- Results
- Read the structured output of a finished job.
- Artifacts
- List a job's files and download them individually or as a bundle.
- Export
- Write results as CSV, JSON Lines, or XML.
Proxies and egress
Your proxies. Explicit egress.
Scrawl runs the proxy pools you bring and decides, per job, how a target may be reached. ApexAi does not operate or resell proxy inventory.
Built
Import your pool
Bring proxies you are entitled to use; add them one by one or import a list.
Encrypted and redacted
Proxy credentials are encrypted at rest and redacted everywhere they are displayed or logged.
Liveness and quarantine
Proxies are tested; failing ones are quarantined automatically and can be re-enabled.
Rotation
Healthy proxies rotate across jobs according to the pool's mode.
Domain rules
Per-domain policy decides how a target may be reached before a request leaves.
One pool, every request
The same pool serves fetches, crawls, and asset extraction.
Ways in
One product, four callers.
These are four ways into the same Scrawl, not four products. Each enters the same job lifecycle and policy.
Telegram operator
The daily control surface: scrape and crawl, follow jobs, pull artifacts, manage proxies, and work with your own AI.
Built
Explore Telegram operatorApexAPI
Typed HTTP for scrape, map, batch, crawls, jobs, artifacts, datasets, extraction, and cited answers, with Python and TypeScript SDKs built from the same contract.
Built
Explore ApexAPIApexMCP
Tools and resources for agents that scrape, map, and crawl, query and diff datasets, extract, and ask with citations.
Built
Explore ApexMCPScrawl Plugin
Health, readiness, smoke tests, evidence, approvals, and bounded maintenance for a Scrawl deployment.
In development
Explore Scrawl PluginTelegram for operators
An inline menu organizes the work by family and adapts to the deployment, so commands a deployment cannot back are never offered. Operator access is by invitation while Scrawl is in development.
Scrape and crawl
Scrape a URL to Markdown or JSON, run cataloged spiders, fetch a single URL, and check a spider before a full crawl.
Inspect
View and parse a page the way the crawler sees it before committing to extraction.
Extract
Pull images, documents, data files, media, and web assets from a page.
Text and patterns
Lightweight HTML extraction and regex tools, including patterns saved for reuse.
Jobs and artifacts
Follow your own work, download its files, export results, or cancel it.
Proxies
Add, import, test, and rotate the proxy pool you bring.
Your AI
Connect your own AI provider, extract to a schema, ask your datasets, and plan an operation you then confirm.
Account
Credit balance and payments, where the deployment is hosted.
For example
/scrapy_extract <url>- Find and keep a page's images, documents, data, media, and web assets.
/ai_ask <question>- Ask your datasets a question and get an answer with numbered sources.
The complete, current command list lives in the bot's /help and menu.
SDKs and tools for developers
Built from the same ApexAPI contract and tested against a real Scrawl, not a mock. They are not published yet: there is no package to install until Scrawl opens. Built
Python SDK
Synchronous and asynchronous clients for every public ApexAPI route, with typed results and errors, automatic idempotency keys, and bounded retries.
TypeScript SDK
A dependency-free client for Node and other fetch-based runtimes, with the same routes, error model, and retry rules.
Command line
Scrape, map, batch, extract, and work with jobs and datasets from a terminal or a CI job.
Claude Code integration
A skill and two commands that let Claude Code scrape and map through your Scrawl deployment, treating page text as untrusted.
Scrawl Plugin for the deployment In development
Keeps a Scrawl deployment observable and correctable: crawler, proxy, and job health, artifact and dataset checks, smoke tests, evidence, approvals, and bounded maintenance.
- observe
- Product and crawler health · Proxy pool readiness · Job and queue health · Alerts on degraded state
- verify
- Registered smoke tests · Artifact, storage, and dataset checks · Deployment and version identity
- control
- Approvals for operator actions · Bounded maintenance and cleanup
- record
- Redacted diagnostics · Evidence for every check and action
Hosted operation
Built to run as a service for many accounts.
The hosted profile adds account isolation, credits, payments, limits, and abuse controls on PostgreSQL and Redis. A trusted self-hosted profile keeps full Scrapy project tooling for the operator who owns the machine.
In development Not publicly hosted
Account isolation
Jobs, artifacts, datasets, and proxy pools belong to one account and are never visible across accounts.
Credits
Work reserves credits before it runs and settles them afterward; every movement is written to an immutable ledger.
Payments
Credit packs paid with Telegram Stars inside Telegram, each payment recorded before it is credited so none can be lost. Payments are not open yet.
Limits
Concurrency, size, rate, and domain limits apply per account before work is queued.
Abuse controls
An abuse gate, the target catalog, and an acceptable-use pledge stand in front of hosted execution.
Durability
PostgreSQL and Redis back jobs, ledgers, and queues.
Authority
What every surface has in common.
- Every surface—Telegram, ApexAPI, ApexMCP, and the Scrawl Plugin—passes the same identity, ownership, target, egress, abuse, and resource policy.
- Customers, applications, and agents never get an arbitrary shell, arbitrary Python, source editing, or unrestricted filesystem access. Scrapy project tooling on a self-hosted deployment stays with the operator who owns the machine.
- Access barriers are recorded as evidence, not bypassed; streaming-platform ripping is blocked.
- Scrawl runs your proxies; ApexAi is not a proxy provider.
Roadmap
Where Scrawl is going next.
Product direction, labelled by state. None of it is part of what Scrawl does today.
Roadmap
- JavaScript rendering for pages that only produce their content in a real browser, with the rendered page and screenshots kept as artifacts and rendering routed through the job's own proxy and egress policy.Planned
- Persistent browser sessions, login continuity for sites you are entitled to use, human-assisted sign-in and two-factor steps, and headful sessions an operator can watch and guide.Planned
- File and media extraction through ApexAPI, ApexMCP, and the SDKs, with a quote and a spending limit before anything is fetched.Planned
- Authenticated remote access to ApexMCP, with identity checked on every request.Planned
- The hosted service: accounts, credits, payments, and limits for many customers on shared infrastructure.In development
Foundation already in place: an isolated browser-worker service, with authenticated calls and navigation checks, exists in Scrawl's source. No job uses it yet.
Status and pricing
What exists, and who you will pay.
Built means implemented and verified in the product, not yet released. Plans are not published; hosted work is metered, and the proxies and AI provider you bring bill you directly.
- Product
- In development
- Telegram operator
- Built
- ApexAPI
- Built
- ApexMCP
- Built
- Product plugin
- In development
- Hosted service
- Not publicly hosted
Scrawl's Telegram operator surface, ApexAPI, and ApexMCP are built and verified; its Scrawl Plugin and hosted service are in development. Nothing is generally released yet: there is no public sign-up or publicly hosted endpoint.
- One-call scrape is built for Telegram, ApexAPI, and ApexMCP; map, batch, versioned datasets, and extraction and cited answers with your own AI are built for ApexAPI and ApexMCP. Python and TypeScript SDKs and a command-line tool are built but not published.
- ApexMCP runs as a local or loopback server for an authorized operator or account; authenticated remote access is planned.
- File and media extraction runs from Telegram in the hosted profile; the same through ApexAPI, ApexMCP, and the SDKs is planned.
- JavaScript rendering, persistent browser sessions, and human-assisted login are planned.
Pricing coming soon
Final ApexAi plans and usage prices are not published yet. The cost model is already built into the product, so here is where money will and will not flow.
| Paid to | Covers | How it works | Price |
|---|---|---|---|
| ApexAi | Hosted work | Hosted jobs are metered in credits: a job reserves credits before it runs and settles them afterward, with every movement written to a ledger. | Pricing coming soon |
| Your proxy provider | Proxy traffic | When you bring your own proxies, their cost stays between you and your provider. ApexAi does not sell proxy capacity. | Billed by your provider |
Final plans and public access details will be published before launch. Follow launch updates
Bring the sources you need data from.
Scrawl is not self-serve yet. Tell us what you collect, how often, and what you need back; these answers make the first job useful on day one.
Sources
List the sites and pages you need, and confirm you are allowed to collect from them.
Output
Define what you want back: fields, files, or a dataset you will query later.
Rendering
Note which sources only show their content in a real browser.
Proxies
If you will route through proxies, gather ones you are authorized to use.
Volume
Estimate how many pages, and how often: once, daily, or continuously.
What to keep
Decide between files and media, exports, and versioned datasets for software and agents.