# Where the data comes from > Which boards we scrape, how often postings refresh, and what it means when a job expires or disappears. Source: https://hyperjobs.io/help/data-sources Postings are scraped directly from **employers' own ATS boards** — Greenhouse, Lever, Ashby, Workday, SmartRecruiters, Workable, Personio and others — not bought from an aggregator and not resold from a third-party feed. That sourcing decision explains most of the dataset's behaviour: `apply_url` points at the employer's real application form, postings appear when the employer publishes them, and they disappear when the employer takes them down. Each posting is enriched with skills, seniority, salary and location, and joined to firmographics for the hiring company. [Coverage](/docs/reference/coverage) has the field-by-field completeness numbers — enrichment is derived by classifiers and isn't perfect, and that page is honest about where it's thin. ## Which boards The `source` field on every posting names the board it came from, and it's also a filter — the same public slug in both places (`lever`, `greenhouse`, …), so a value you read in a response feeds straight back as a query. [Sources & ATS coverage](/docs/reference/sources) lists every valid value. The live source breakdown, with counts, is on [`/v1/meta`](/docs/api-reference/meta). Read it from there rather than hardcoding a list — the list in your source tree is the one that drifts. ## How often it refreshes Ingestion runs **hourly**, working the stalest boards first. Boards aren't all refreshed at the same rate, because they don't all cost the same to fetch: the fast tier runs every cycle, the heavy and proxy-only tiers every third cycle in larger batches. Across the tiers the whole pool is covered every few hours. In practice: - A new posting on a Greenhouse or Ashby board typically appears **within the hour**. - A posting on a Workday or iCIMS board can take **several**. Don't build anything that assumes uniform latency across boards. `posted_at` is a good ordering key and a good "newer than X" filter, but the lag between a job going live and appearing here varies by source. The full cadence table is in [coverage](/docs/reference/coverage). If you want a freshness signal to alert on, use `newest` from `/v1/meta`. If it drifts more than a couple of hours from now, ingestion is behind — that tells you more than any count does. ## What "expired" means A posting is marked expired when it **stops appearing on the employer's board**. We diff each board every time we re-fetch it: postings that were there last time and aren't there now are reaped. The employer taking the job down is the signal — we don't guess, and we don't expire postings on a timer. Two consequences worth knowing: - **Expiry is only as fresh as the board's tier.** A removed Greenhouse role is reaped within the hour; a removed Workday role can linger for several. A posting is never expired earlier than the next time its board is checked. - **A board that goes dark strands its postings.** If a board stops responding entirely, there's no diff to compute. A board that fails five consecutive times is disabled automatically so it stops consuming crawl cycles; its existing postings age out through expiry rather than vanishing at once. `/v1/jobs` excludes anything expired, so expired postings simply stop appearing in your results. That's expected, not a bug. The posting was either taken down by the employer or collapsed into an equivalent posting from another source. If you keep a local mirror, treat a `404` on a previously-seen id as a delete — see [syncing the feed](/docs/guides/feed-sync) and [duplicates](/docs/guides/deduplication). ## Duplicates The same role often sits on several boards at once. Equivalent postings are collapsed, which is why `/v1/health` and `/v1/meta` report different counts — `/v1/health` counts every row including duplicates and expired postings, `/v1/meta` counts only live, non-duplicate ones. `/v1/meta` is the number you want. See [deduplication](/docs/guides/deduplication). ## Found something wrong? Send us the job id — see [reporting incorrect data](/help/data-corrections). ## Next How complete each field is, and how fast queries run. The commercial side.