Processing and coverage
Uploads are processed asynchronously. The coverage report says when data has landed, and what did not.
An upload answers as soon as the payload is safe in the raw store. Normalizing it happens in the background, so "did my data land?" has its own answer: the coverage report.
curl -s "$OHS/v1/users/$USER_ID/coverage" -H "Authorization: Bearer $KEY"Prop
Type
The rule of thumb
After an upload, poll the coverage report until pending_jobs is 0.
Then read. A non-zero rejected_records means some lines were not stored; an operator finds
each line and its reason in ohs_data.failed_records.
A non-zero parked_jobs is for the operator: those jobs failed 5 times.
Retries and crashes
- A failed job is retried with exponential backoff (30 seconds, doubling) and parked after 5 attempts.
- A job interrupted by a crash is claimed again after its 10 minute visibility timeout.
- A provider's 429 defers that connection's job until
Retry-After, without counting a failed attempt. - Records that failed for a recoverable reason are replayed once per process start, so a fixed release picks them up after a restart.
What to do after a long outage, and how to re-process a user from the raw files, is in Operating notes.