# Dataset coverage

> How much data there is, how complete each field is, how often it refreshes, and how fast you can query it.

Source: https://hyperjobs.io/docs/reference/coverage

What's actually in the dataset, measured rather than estimated. Every number on
this page is a snapshot — the index changes hourly, so read
[`/v1/meta`](/docs/api-reference/meta) rather than [hardcoding any of
it](#dont-hardcode-these-numbers).

## Scale

| | Count |
| --- | --- |
| Postings, all rows | 603,335 |
| Postings, excluding duplicates | 596,768 |
| Marked as duplicates | 6,567 |
| Expired | 8,482 |
| [Companies](/docs/objects/company) | 158,545 |
| [ATS boards tracked](/docs/reference/sources#boards-tracked-per-provider) | 108,389 |

## One live count everywhere

<Note title="The counters agree">
  [`/v1/health`](/docs/api-reference/health) and
  [`/v1/meta`](/docs/api-reference/meta) report the **same** number: live
  postings — no [duplicates](/docs/guides/deduplication), no expired rows —
  which is exactly the population
  [`/v1/jobs`](/docs/api-reference/jobs-list) searches. An unfiltered
  `/v1/jobs?limit=1` returns the same figure as `total`.

  (In a pre-release draft, `/v1/health` counted every row including duplicates
  and expired postings, and `/v1/meta` excluded only duplicates — three subtly
  different totals. That asymmetry never shipped.) `/v1/health` is free,
  unauthenticated, and fast enough to poll.
</Note>

<Note>
  [`/v1/jobs/feed`](/docs/api-reference/jobs-feed) sees the same
  population — it excludes duplicates **and** expired postings.
  Expiries arrive on their own feed,
  [`/v1/jobs/expired`](/docs/api-reference/jobs-expired) — see
  [feed sync](/docs/guides/feed-sync).
</Note>

## Field completeness

<Tip title="`coverage_pct` is the live version of this table">
  [`/v1/meta`](/docs/api-reference/meta) now publishes `coverage_pct` — per-field
  fill rates for country, skills, work_arrangement, seniority, taxonomies,
  salary, education, and language, remeasured about every 15 minutes. That's
  the canonical live source. The table below is a snapshot, kept for reading
  the shape of the dataset and the fields `coverage_pct` doesn't cover.
</Tip>

Percentages are of all 603,335 rows.

| Field | Populated | Share |
| --- | --- | --- |
| [`description`](/docs/objects/job#content) | 600,321 | 99.5% |
| [`company`](/docs/objects/job#company) resolved to a full profile | 603,333 | 99.9997% |
| [`skills`](/docs/objects/job#classification) | 504,787 | 83.7% |
| [`taxonomies`](/docs/objects/job#classification) | 385,246 | 63.9% |
| [`salary`](/docs/objects/job#salary) | 217,221 | 36.0% |
| [`location.remote`](/docs/objects/job#location) is true | 85,069 | 14.1% |

Read these as filter costs. `has_salary=true` discards about two thirds of the
dataset before any of your other filters apply, and `taxonomies=` discards a
third — not because those jobs are irrelevant, but because nobody classified
them. Filtering on a sparse field is a much stronger statement than it looks.

Two rows are worth reading twice:

- **`company` resolves for all but two postings.** The [minimal fallback
  shape](/docs/objects/company#the-three-shapes-on-a-job) is genuinely rare —
  and still worth handling, because a null-pointer on a path that fires twice in
  600,000 rows is a bug you'll find in production.
- **`location.remote` is true for 14.1%, but 127,895 postings are classified
  `work_arrangement=Remote`.** Different fields, different evidence. See [the
  comparison](/docs/reference/enums#work_arrangement).

## Date range

| Bound | Date |
| --- | --- |
| Newest posting | `2026-07-08T20:12:11.000Z` |
| Oldest posting | `2009-12-05` |

The 2009 end of that range is an archive, not a hiring signal. Old postings
survive as rows because nothing forced them out, and the overwhelming majority
are long since expired or filled — [`/v1/jobs`](/docs/api-reference/jobs-list)
excludes anything past its `expires_at`, but a board that quietly went dark
without declaring expiry leaves postings behind.

Default to a [`time_frame`](/docs/guides/filtering#time) unless you specifically
want history:

```bash
# Almost always what you want. Without it, you're searching back to 2009.
curl -G https://api.hyperjobs.io/v1/jobs \
  -H "Authorization: Bearer $HYPERJOBS_KEY" \
  --data-urlencode "time_frame=7d"
```

`newest` from [`/v1/meta`](/docs/api-reference/meta) is the freshness check worth
alerting on: if it drifts more than a couple of hours from now, ingestion is
behind, and that tells you more than any count.

## Refresh cadence

Ingestion runs **hourly**. Each cycle works the stalest boards first — least
recently scraped, in order — so no board is starved and freshness degrades
evenly rather than in clumps.

Boards aren't all refreshed at the same rate, because they don't all cost the
same to fetch:

| Tier | Providers | Cadence | Boards per cycle |
| --- | --- | --- | --- |
| **Fast / direct** | `greenhouse`, `ashby`, `lever`, `smartrecruiters`, `personio`, `teamtailor`, `breezy`, `pinpoint`, `jazzhr`, `recruiterbox`, `rippling`, `phenom` | Every cycle | 9,000 |
| **Heavy** | `oracle`, `icims`, `successfactors`, `cornerstone`, plus sharded `workday` and `bamboohr` | Every 3rd cycle | 16,000 |
| **Proxy-only** | `workable`, `recruitee` | Every 3rd cycle | 8,000 |
| **LinkedIn** | `linkedin` | Every cycle — last-hour delta | — |

The fast tier serves clean APIs and can be hit directly, so it runs every hour.
The heavy and proxy tiers are slower or rate-limited at the source, so they run
every third cycle in larger batches — the same throughput, spent less often.
Between the tiers, the whole pool of 108,389 boards is covered **every few
hours**.

<Note title="What this means for freshness">
  A new posting on a Greenhouse or Ashby board typically appears within the hour.
  A posting on a Workday or iCIMS board can take several. LinkedIn is refreshed
  as an hourly delta of the previous hour.

  So [`posted_at`](/docs/objects/job#timestamps) is a good ordering key and a
  good "newer than X" filter, but the lag between a job going live and appearing
  here varies by [source](/docs/reference/sources). Don't build anything that
  assumes uniform latency across boards.
</Note>

### Dead boards disable themselves

A board that fails **five consecutive times** is disabled automatically and stops
consuming cycles. Companies delete their boards, migrate ATS, and go out of
business; without this the rotation would fill with corpses and starve the live
boards.

A disabled board's existing postings don't vanish — they age out through expiry
like anything else.

## Expiry and reaping

Postings are removed by **board diffing**. Every time a board is re-fetched, the
postings that were there last time and aren't there now are marked expired. The
employer taking a job down is the signal; we don't guess.

Two consequences:

- **Expiry is as fresh as the board's tier.** A removed Greenhouse role is
  reaped within the hour; a removed Workday role can linger for several. A
  posting is never expired earlier than the next time its board is checked.
- **A dark board strands its postings.** If a board stops responding entirely
  and is [disabled](#dead-boards-disable-themselves), there's no diff to compute,
  so its postings sit until something else expires them. That's most of what the
  long tail back to 2009 is.

Some boards declare an [`expires_at`](/docs/objects/job#timestamps) directly, and
that's honoured. Most don't, which is why `expires_at` is usually null on a
perfectly live posting — **a null `expires_at` doesn't mean the job runs
forever**.

[`/v1/jobs`](/docs/api-reference/jobs-list) hides expired postings for you.
[`/v1/jobs/{id}`](/docs/api-reference/jobs-retrieve) is more forgiving: an
expired posting stays retrievable by id while the row is still in the pool, so
check `expires_at` rather than assuming a `200` means live. To purge a mirror,
read [`/v1/jobs/expired`](/docs/api-reference/jobs-expired) — the ids as they
expire, built to be replayed as deletes. And if a cached id does return
[404](/docs/errors#404-not-found) (purged, or folded as a
[duplicate](/docs/guides/deduplication)), treat it as "delete my copy".

## Query performance

<Warning title="Every `/v1/jobs` query scans the full dataset">
  The structured-filter hot path is unindexed. A `/v1/jobs` request scans and
  sorts on the order of 600,000 rows regardless of how selective your filters
  are — no structured-filter combination gets you an index seek. (There is one
  index: the text-search params are FTS-backed.)

  Practically:

  - **Narrow filters don't run faster, but they return faster.** The scan costs
    what it costs; the response size is what you control.
  - **[`description=`](/docs/guides/filtering) is still the slowest filter.**
    It's a Google-style search over title + description (FTS-backed,
    not a raw substring scan), but pair it with something narrower anyway.
  - **Deep `offset` [pagination](/docs/pagination) gets worse the deeper you
    go.** A high `offset` still scans and sorts everything before discarding
    the rows it skipped. Use `cursor` instead — the keyset cursor from
    `next_cursor` is stable at any depth.
  - **Don't fetch descriptions in bulk.** `description_format` on a `limit=200`
    list is the usual reason a request feels slow. List without them, then pull
    individual postings from
    [`/v1/jobs/{id}`](/docs/api-reference/jobs-retrieve).

  This is a real constraint, not a warning to route around: requests do complete,
  and normal filtered queries are fine. It means you should build against the
  [feed](/docs/api-reference/jobs-feed) for bulk sync rather than paging
  `/v1/jobs` to the end.
</Warning>

<Tip>
  For anything resembling "give me the whole dataset", use
  [`/v1/jobs/feed`](/docs/api-reference/jobs-feed) and page it with
  `next_cursor`. It's built for exactly that, and it doesn't degrade as you get
  further in the way `offset` does. [Feed sync](/docs/guides/feed-sync) has the
  loop.
</Tip>

## Don't hardcode these numbers

Everything above rots. The dataset grows every hour, and a "we track 603,335
jobs" line in your UI starts drifting the moment you ship it.

| You want | Read |
| --- | --- |
| Total jobs, companies | [`/v1/meta`](/docs/api-reference/meta) |
| Freshness | `newest` from [`/v1/meta`](/docs/api-reference/meta) |
| Per-field fill rates | `coverage_pct` from [`/v1/meta`](/docs/api-reference/meta) |
| Valid `?source=` values, with counts | `sources` from [`/v1/meta`](/docs/api-reference/meta) |
| Count for any filter | `total` from `/v1/jobs?…&limit=1` |
| Liveness | [`/v1/health`](/docs/api-reference/health) — free, no key, no rate limit |

```js
// One request, at build or request time. Never a constant in your source tree.
const { jobs, companies, newest } = await (
  await fetch("https://api.hyperjobs.io/v1/meta", {
    headers: { Authorization: `Bearer ${process.env.HYPERJOBS_KEY}` },
  })
).json();
```

## Related

<CardGroup cols={2}>
  <Card title="Dataset metadata" icon="Activity" href="/docs/api-reference/meta">
    The live counts these numbers approximate.
  </Card>
  <Card title="Syncing the feed" icon="RefreshCw" href="/docs/guides/feed-sync">
    The right way to pull the dataset in bulk and keep it current.
  </Card>
  <Card title="Sources & ATS coverage" icon="Plug" href="/docs/reference/sources">
    Which boards feed the index, and which tier each one is in.
  </Card>
  <Card title="Duplicates & collapsing" icon="Copy" href="/docs/guides/deduplication">
    What the 6,567 duplicate rows are, and why they're hidden.
  </Card>
</CardGroup>