Every /v1/jobs query is a full scan

A /v1/jobs request with structured filters — enums, geo, salary, company — scans and sorts the whole index. Not the rows matching your filters: all of them. skill=rust&remote=true&time_frame=24h does the same amount of work underneath as a request with no parameters at all.

One family of filters escapes the scan. title, description, and their *_advanced variants run against a full-text index (check the full_text_search flag on /v1/meta): the index hands back matching row ids, and only those rows get tested against your other predicates. A text query is the one narrowing that genuinely narrows.

Everything else — seniority, country, salary_gte, the company axis — walks the table. Adding one of those filters is not an optimisation. It is a request for less data.

"Faster" and "smaller" are two different wins#

A narrow structured filter doesn't run faster. It returns faster. The distinction is not pedantry — it tells you which knobs are connected to anything.

Three of those four are yours. The first one moves only when an index can carry the predicate — which today means text. So the mental model is: the scan is a flat tax, text search is the one rebate, and everything after the scan is priced by the row.

Which is why normal filtered queries are fine. The flat tax is tens of milliseconds, not seconds — a constraint on how you architect against the API, not a reason to be timid with it.

description= got demoted from villain to workhorse#

The original version of this post called description= the expensive one — a substring crawl over every multi-kilobyte text blob in the index, "no shortcut available to it and none coming". The shortcut came. description= is Google-style search over the text index now (words AND, "quoted phrases", OR, -exclusions), and description_advanced= adds boolean grouping. The syntax is validated — malformed input is a 400, not a scan that returns everything.

The advice that survives: it is still a combined title-or-description search, a hit doesn't tell you where the phrase appeared, and if you are reaching for description=react when skill=react exists, you are asking a text index a question a classifier already answered.

total is a real count#

Not an estimate, not sampled, not cached. An exact count over the same filters as your query — and /v1/jobs/count now gives you the number without carrying a page of rows alongside it.

The cost has a direction worth knowing: a very broad query pays to count rows it will never return. If the question is only "are there any", fetch limit=1 and check data.length instead.

Offset is discarded after the work is done#

offset is not a seek — page 200 costs everything page 1 costs, plus the sorting of the 10,000 rows it throws in the bin. That is why offset caps at 100,000 and why the deep-paging answer is the cursor: pass back next_cursor and the keyset condition (older than the last row you saw) replaces the discard. The scan doesn't disappear, but the walk stops getting more expensive as it goes, and — the part that matters more in practice — it stops drifting.

While you are here: don't turn descriptions on in bulk. description_format on a limit=200 list is the most common reason a request feels slow, and that one is not the scan at all — it is multi-kilobyte payloads, 200 at a time. List without them, then pull the postings you care about from /v1/jobs/{id}.

The index moves while you walk it — unless you cursor#

Results are newest-first and ingestion never stops. Offset paging over that is not a consistent snapshot: 30 new jobs landing while you're on page 3 shift every row down, so you see 30 rows twice and miss rows falling off the far end.

The draft-era version of this section ended in a three-step mitigation ritual (dedupe by id, bound the window, give up and use the feed). The cursor deletes the ritual: next_cursor pins your position to a row rather than a row number, so inserts above you shift nothing. Page until it comes back null — that is the loop, whole. (Pagination has the mechanics; the old advice to stop on a short page rather than trusting offset >= total is obsolete with it.)

Offset still exists for what it's good at: shallow, human-shaped paging where page 2 means "show me a few more".

The arithmetic that should settle it#

Walking the whole index is no longer fragile — cursors made it safe. It is still the wrong shape of work for staying current.

At limit=200, one full cursor walk of the searchable set is a few thousand requests. Fine — that is a bootstrap, you do it once. Repeat it hourly and it is over 2.1 million requests a month: more than four times a Growth plan's entire quota, spent re-downloading a dataset that changed by a fraction of a percent.

Polling /v1/jobs/feed, /v1/jobs/expired and /v1/jobs/modified every 15 minutes — births, deaths, and edits — costs 8,640 requests a month. It fits inside a Starter plan, and its cost scales with what changed rather than with what exists. Re-scanning the whole index to discover that 400 rows moved is the wrong shape of work, and the request count is just where the wrongness shows up. Feed sync has the loop.

The honest version#

This is a real constraint, and here is exactly where it sits.

The draft-era complaint list read: description= should be a text index, offset should be a cursor, the hot path should be indexed. Two of the three shipped. What remains unindexed is the structured hot path: nothing seeks seniority=, country=, or salary_gte= — they are tested per row, which is why they cost the same whether they match everything or nothing.

Why that's a trade rather than an oversight: the tax is tens of milliseconds, and — the part that actually matters — it's flat. It doesn't grow with how complicated your query is, only with the dataset. A predictable cost you can design around beats a clever one you can't, and designing around it is what the rest of this post is for.

How that's implemented underneath is our problem, and it's meant to stay that way: the compatibility policy is the promise that the parameters and the semantics are the contract, whatever is behind them.

Which leaves the guidance, unchanged: /v1/jobs is for search — an intent, a tight filter, one page. The feeds are for completeness. Ask the first one for bulk and you get the scan, once per page, for as many pages as it takes.

The cheapest thing you can do#

There is no default time_frame. Leave it off and your query searches back to the oldest posting in the index — 2009-12-05. That end of the range is an archive, not a hiring signal.

# Almost always what you want. Without it, you're searching back to 2009.
curl -G https://api.hyperjobs.io/v1/jobs \
  -H "Authorization: Bearer $HYPERJOBS_KEY" \
  --data-urlencode "skill=rust" \
  --data-urlencode "time_frame=7d" \
  --data-urlencode "limit=50"

It won't shrink the scan — only text search shrinks the scan. It shrinks the count, the sort's survivors, the payload and your parse, and it makes the rows you get back mean something. One parameter.

The full performance picture is in dataset coverage; the paging mechanics are in pagination.