Skip to content

DocsReading Dawnscan

Data sources

Every third-party source Dawnscan reads, what it is used for and what it is never used for.

Every third-party call in Dawnscan goes through a SourceAdapter in packages/sources. Adapters are the only place the product talks to an external provider, they are never called from a page render path, and each one ships with saved fixtures so CI never depends on a live API.

Adapters

Adapter Purpose Cache TTL Notes
dexscreener Primary market context 5 min Deepest-liquidity pair wins; figures are never merged across pairs
geckoterminal Fallback market context 5 min Same normalized shape as DEX Screener
blockscout Contract metadata, verification, deployment evidence 1 h Holder endpoints are deliberately not implemented
rpc-contract eth_getCode existence check 1 h Lightweight verification only; Dawnscan runs no node
github-repo Repository activity window 30 min Stars are display context, never a score input
github-releases Releases → GITHUB_RELEASE ShipEvents (M4) 30 min Drafts excluded; stable externalId for dedupe
website Page metadata + feed discovery 6 h SSRF guarded, 2 MB cap, HTML content-type only
feed RSS/Atom/changelog entries 2 h Max 50 entries; summaries tag-stripped
sourcify Verified contract source 24 h Optional — a miss is missing, never an error
coingecko-markets Price, market cap, FDV, volume for up to 250 CoinGecko ids per call 1 h Matched to tokens by (chain, address) from the registry; a 0 figure is absent; source coingecko
virtuals-market The launchpad's curve/pool valuation for a batch of 25 agents, in VIRTUAL 1 h Holder fields never leave the payload; converted with a same-run CoinGecko rate; source virtuals
robinhood-stock-assets / robinhood-stock-price Tokenized-equity assets, multipliers and raw underlying bid/ask 1 h / 60 s Off by default (HEY_STOCK_TOKEN_PRICES_ENABLED); price only, no market cap; source robinhood-stock-api
launchpad Interface + registry only No provider ships until its access is public, documented and permitted

Guarantees

Every adapter validates its payload with Zod, applies a timeout, retries transient failures with exponential backoff and jitter, honours Retry-After, sends conditional requests (If-None-Match / If-Modified-Since) and returns caching metadata.

Adapters never throw for upstream conditions. They return a SourceResult whose status is one of fresh, not_modified, missing, rate_limited or error, with a normalized errorCode. This is what makes degraded mode work: a provider outage is structurally distinct from "the project stopped shipping", and callers use shouldRetainPreviousData() to keep the last known value and mark the source stale.

Cost and safety

Conditional requests plus content hashing mean an unchanged source costs one 304 and does no downstream work. Response bodies are capped, content types are checked, and URLs supplied by builders or users are screened against private, loopback, link-local and IPv4-mapped-IPv6 destinations before any request is made.

Redirects are followed manually with a cap of 5 hops, and every hop is re-validated against the same rules. Letting the runtime follow redirects would hide the intermediate URLs, so a public address could redirect into a private one unchecked.

Known limitations

  • DNS rebinding is not covered. The guard validates the literal host; a hostname that resolves to a private address needs connection-time IP pinning. Tracked for M9.
  • robots.txt and terms-aware crawling are not implemented. PRD V4 section 27 requires this for website/docs ingestion, which is M4. The M2 adapters fetch only URLs a builder or admin has explicitly registered, one page per call, never crawling.
  • Feed entries are capped at 50 per fetch, newest-first as the provider orders them.
  • redirect: 'manual' relies on Node/undici semantics, where the 3xx response and its Location header are readable. Adapters run in the worker, not in a browser context.