# Pagination

> Cursor and offset paging, their limits, and how to walk a large result set without missing rows.

Source: https://hyperjobs.io/docs/pagination

Every list endpoint pages with a **cursor**: the response carries a
`next_cursor`, and you pass it back as `?cursor=` to get the next page.
`offset` is still there for shallow, random-access paging. There are no `Link`
headers.

```bash
curl -H "Authorization: Bearer $HYPERJOBS_KEY" \
  "https://api.hyperjobs.io/v1/jobs?skill=python&limit=200"
```

```json
{
  "data": [ … ],
  "total": 21497,
  "limit": 200,
  "offset": 0,
  "next_cursor": "eyJwIjoiMjAyNi0wNy0wOCAxNDowMjoxMSIsImlkIjoiYXNoYnk6N2MxZjJhOTQuLi4ifQ"
}
```

`total` is the count of **all** matching rows, not the page. The page itself is
`data.length`, which is `limit` until the last page.

## Cursors

`next_cursor` is an opaque base64url keyset cursor — don't parse it, just send
it back:

```bash
curl -H "Authorization: Bearer $HYPERJOBS_KEY" \
  "https://api.hyperjobs.io/v1/jobs?skill=python&limit=200&cursor=eyJwIjoiMjAyNi0wNy0wOCAx…"
```

- **`null` means done.** The last page carries `"next_cursor": null`.
- **Stable at any depth.** A cursor pins your position by key, not by row
  count, so page 500 costs the same as page 2 and rows ingested mid-walk can't
  shift under you. See [deep paging](#deep-paging-drifts) below.
- **Cursor wins over offset.** On `/v1/jobs`, passing `cursor` ignores
  `offset`, and the response omits the `offset` key. Pick one style per walk.
- A malformed or hand-edited cursor is a
  [`400`](/docs/errors#400-bad-request), not an empty page.
- The feeds — [`/v1/jobs/feed`](/docs/api-reference/jobs-feed),
  [`/v1/jobs/expired`](/docs/api-reference/jobs-expired),
  [`/v1/jobs/modified`](/docs/api-reference/jobs-modified) — return rows in
  **ascending** index order and page the same way: follow `next_cursor` until
  `null` and you've drained the window.

<Note>
  A `collapse` response has no `next_cursor` — collapsed groups have no stable
  keyset to cursor over. Page those with `offset`, which collapse supports as
  usual.
</Note>

## Offset

`/v1/jobs` and `/v1/companies` also take `offset` (min 0, **max 100,000**) for
shallow or random-access paging — jumping straight to page 12 of a UI, say.
Beyond 100,000, or for any full walk, use the cursor.

## Limits

| Endpoint | Default `limit` | Max `limit` | Cursor? | Offset? |
| --- | --- | --- | --- | --- |
| `/v1/jobs` | 50 | 200 | Yes | Yes (max 100,000) |
| `/v1/jobs/feed` | 200 | 1,000 | Yes | No |
| `/v1/jobs/expired` | 1,000 | 5,000 | Yes | No |
| `/v1/jobs/modified` | 200 | 1,000 | Yes | No |
| `/v1/companies` | 50 | 200 | No | Yes (max 100,000) |

Out-of-range values are rejected, not clamped: `limit=5000` on `/v1/jobs` is a
`400` with `"hint": "maximum is 200"`, and the minimum everywhere is 1 —
`limit=0` and `limit=-1` are `400`s too. (An early draft of this API had a
negative-limit-returns-everything backdoor; it's gone.)

## Walking a full result set

```js
async function* allJobs(params, key) {
  const PAGE = 200; // the max — fewer, bigger pages cost less quota
  let cursor = null;

  for (;;) {
    const qs = new URLSearchParams({ ...params, limit: PAGE });
    if (cursor) qs.set("cursor", cursor);
    const res = await fetch(`https://api.hyperjobs.io/v1/jobs?${qs}`, {
      headers: { Authorization: `Bearer ${key}` },
    });
    if (!res.ok) throw new Error(`${res.status}`);

    const { data, next_cursor } = await res.json();
    yield* data;

    if (!next_cursor) return;
    cursor = next_cursor;
  }
}
```

No `total` bookkeeping, no short-page heuristics — the cursor is the loop
condition.

## Deep paging drifts

Results are sorted newest-first, and the dataset is continuously ingested.
Both facts together mean **offset paging over a live index is not a consistent
snapshot**: if 30 new jobs are indexed while you're on page 3, the rows that
were on page 3 shift to page 4, and you'll see 30 rows twice — while rows that
fall off the far end are missed entirely.

This is inherent to offset paging, not specific to this API — and it's what
the cursor fixes. A keyset cursor pins your position to the last row you saw,
so new rows arriving above you can't shift the pages below. In order of
preference:

<Steps>
  <Step title="Use the cursor">
    For any walk longer than a page or two, follow `next_cursor`. No drift, no
    depth limit, no duplicates from ingestion racing your walk.
  </Step>

  <Step title="If you must offset-page, deduplicate by id">
    Collapsed results have no cursor, so offset paging is sometimes
    unavoidable. Collect into a `Map` keyed by `job.id` rather than an array —
    costs nothing, fixes double-counting, doesn't fix misses.
  </Step>

  <Step title="Sync with the feed instead">
    For anything that needs completeness — keeping a local mirror, driving
    alerts — don't walk the index at all. Take one full snapshot, then poll
    [`/v1/jobs/feed`](/docs/api-reference/jobs-feed) with a cutoff.
    [Syncing the feed](/docs/guides/feed-sync) has the pattern.
  </Step>
</Steps>

## Counting without fetching

Counting has its own endpoint now:
[`/v1/jobs/count`](/docs/api-reference/jobs-count) takes the same filters as
`/v1/jobs` and returns just the number.

```bash
curl -H "Authorization: Bearer $HYPERJOBS_KEY" \
  "https://api.hyperjobs.io/v1/jobs/count?skill=rust&remote=1"
```

```json
{ "count": 1284 }
```

<Note>
  Counts are an exact `COUNT(*)` over the same filters, not an estimate —
  which also means a very broad query pays to count rows it will never return.
  If you only need "are there any", `/v1/jobs?limit=1` and check `data.length`.
</Note>

## Sorting

Not configurable. `/v1/jobs` is always newest-first by `posted_at` (falling
back to `created_at` when a posting carries no publish date), with `id` as the
tiebreak. The feeds (`/feed`, `/expired`, `/modified`) are the opposite —
**ascending** by index, expiry, and modification time respectively, so a
cursor walk drains them oldest-first. `/v1/companies` is always ordered by
data tier, then follower count.

If you need a different order, fetch the filtered set and sort client-side —
with `limit=200` and a tight filter that's usually one or two requests.