Dataset coverage

What's actually in the dataset, measured rather than estimated. Every number on this page is a snapshot — the index changes hourly, so read /v1/meta rather than hardcoding any of it.

Scale#

One live count everywhere#

Field completeness#

Percentages are of all 603,335 rows.

Read these as filter costs. has_salary=true discards about two thirds of the dataset before any of your other filters apply, and taxonomies= discards a third — not because those jobs are irrelevant, but because nobody classified them. Filtering on a sparse field is a much stronger statement than it looks.

Two rows are worth reading twice:

• company resolves for all but two postings. The minimal fallback shape is genuinely rare — and still worth handling, because a null-pointer on a path that fires twice in 600,000 rows is a bug you'll find in production.
• location.remote is true for 14.1%, but 127,895 postings are classified work_arrangement=Remote. Different fields, different evidence. See the comparison.

Date range#

The 2009 end of that range is an archive, not a hiring signal. Old postings survive as rows because nothing forced them out, and the overwhelming majority are long since expired or filled — /v1/jobs excludes anything past its expires_at, but a board that quietly went dark without declaring expiry leaves postings behind.

Default to a time_frame unless you specifically want history:

# Almost always what you want. Without it, you're searching back to 2009.
curl -G https://api.hyperjobs.io/v1/jobs \
  -H "Authorization: Bearer $HYPERJOBS_KEY" \
  --data-urlencode "time_frame=7d"

newest from /v1/meta is the freshness check worth alerting on: if it drifts more than a couple of hours from now, ingestion is behind, and that tells you more than any count.

Refresh cadence#

Ingestion runs hourly. Each cycle works the stalest boards first — least recently scraped, in order — so no board is starved and freshness degrades evenly rather than in clumps.

Boards aren't all refreshed at the same rate, because they don't all cost the same to fetch:

The fast tier serves clean APIs and can be hit directly, so it runs every hour. The heavy and proxy tiers are slower or rate-limited at the source, so they run every third cycle in larger batches — the same throughput, spent less often. Between the tiers, the whole pool of 108,389 boards is covered every few hours.

Dead boards disable themselves#

A board that fails five consecutive times is disabled automatically and stops consuming cycles. Companies delete their boards, migrate ATS, and go out of business; without this the rotation would fill with corpses and starve the live boards.

A disabled board's existing postings don't vanish — they age out through expiry like anything else.

Expiry and reaping#

Postings are removed by board diffing. Every time a board is re-fetched, the postings that were there last time and aren't there now are marked expired. The employer taking a job down is the signal; we don't guess.

Two consequences:

• Expiry is as fresh as the board's tier. A removed Greenhouse role is reaped within the hour; a removed Workday role can linger for several. A posting is never expired earlier than the next time its board is checked.
• A dark board strands its postings. If a board stops responding entirely and is disabled, there's no diff to compute, so its postings sit until something else expires them. That's most of what the long tail back to 2009 is.

Some boards declare an expires_at directly, and that's honoured. Most don't, which is why expires_at is usually null on a perfectly live posting — a null expires_at doesn't mean the job runs forever.

/v1/jobs hides expired postings for you. /v1/jobs/{id} is more forgiving: an expired posting stays retrievable by id while the row is still in the pool, so check expires_at rather than assuming a 200 means live. To purge a mirror, read /v1/jobs/expired — the ids as they expire, built to be replayed as deletes. And if a cached id does return 404 (purged, or folded as a duplicate), treat it as "delete my copy".

Query performance#

Don't hardcode these numbers#

Everything above rots. The dataset grows every hour, and a "we track 603,335 jobs" line in your UI starts drifting the moment you ship it.

// One request, at build or request time. Never a constant in your source tree.
const { jobs, companies, newest } = await (
  await fetch("https://api.hyperjobs.io/v1/meta", {
    headers: { Authorization: `Bearer ${process.env.HYPERJOBS_KEY}` },
  })
).json();

Related#