Releasesv0.1.0

Configuration

Every environment variable, the database pool size and provider keys per role.

Environment variables only, documented line by line in .env.example.

VariableDefaultMeaning
OHS_DATABASE_URLset by composePostgres connection URL; size its pool with pool_max_conns (see below)
OHS_ENCRYPTION_KEYnone, required32 random bytes, base64. Seals stored OAuth tokens and webhook secrets. openssl rand -base64 32 makes one, or ohs doctor prints one
OHS_PREVIOUS_ENCRYPTION_KEYSnoneComma separated keys OHS_ENCRYPTION_KEY replaced: they still open what they sealed, never seal. Set only while rotating the key
OHS_PUBLIC_URLhttp://localhost:8080Public base URL, used for OAuth callbacks and notification URLs. Use your https address
OHS_RAW_STOREfile:///var/lib/ohs/rawfile:///path or s3://bucket?endpoint=&region=&path_style=true
OHS_ROLESapi,collect,etlRoles this process runs
OHS_BOOTSTRAP_TENANTnoneCreates this tenant at first start and logs its API key once
OHS_LISTEN:8080Listen address
OHS_LOG_LEVELinfodebug, info, warn or error; logs are JSON on stderr
OHS_CONSOLEfalsetrue serves the read-only admin console on /console. It asks for a tenant API key and reads the API with it; off, the route is not mounted (404)
OHS_SETUP_TOKENnone16 characters or more (openssl rand -hex 32). Turns on the browser first-run setup: with OHS_CONSOLE=true, /console/setup creates the first tenant and shows its API key once, for whoever types this token, while the instance has no tenant. Unset, /v1/setup is not mounted
OHS_METRICStrueServe Prometheus metrics on /metrics (see Metrics)
OHS_METRICS_TOKENnoneBearer token a scrape of /metrics presents, 16 characters or more (openssl rand -hex 32). Without it the API listener refuses every scrape
OHS_METRICS_LISTENnoneServe /metrics on this address instead of the API listener, for example 127.0.0.1:9464; the token is then asked for only when set
OHS_CONSENT_REQUIREDfalseStore nothing from a source until the user's consent for it is recorded; by default a source without consent stores everything
OHS_AUDIT_RETENTION_DAYS365Delete audit events older than this many days, once a day; 0 keeps every event
OHS_RETENTION_DAYS0Delete raw files and normalized data older than this many days; 0 keeps everything
OHS_PULLS_PER_PROVIDER4Pulls of one provider running at once, across every collect instance. See provider rate limits
OHS_BACKFILLS_PER_PROVIDER2Of those, backfills of new connections
OHS_INGEST_CONCURRENCY8Device uploads one api process handles at once; more answer 503 with Retry-After. The Pi overlay sets 2
OHS_WEBHOOK_ALLOW_PRIVATEfalsetrue lets webhook endpoints use plain http and private, loopback or link-local addresses (a receiver on the same host or network). Leave it off when tenants are not trusted
OTEL_EXPORTER_OTLP_ENDPOINTnoneTurns on OpenTelemetry traces, sent over OTLP to this address. The other standard OTEL_ variables apply; see observability
OHS_RATE_LIMIT_INBOUND30/120Notification requests (/inbound/{provider}) per client address: rate/burst in requests per second, 0 disables. See rate limits
OHS_RATE_LIMIT_INBOUND_PROVIDER300/600Notification requests per provider, whatever the address; 0 disables
OHS_RATE_LIMIT_OAUTH2/20OAuth callback requests per client address, and POST /v1/setup requests in a bucket of their own; 0 disables
OHS_TRUSTED_PROXIESnoneComma-separated CIDRs (or addresses) of your reverse proxies. Only a request from one of them has its client address read from X-Forwarded-For
OHS_<PROVIDER>_CLIENT_ID, _CLIENT_SECRETnoneKeys of a provider; its connector is on when they are set. Every process that runs collect or etl needs them (see below)

Database pool size

Each process opens one Postgres pool. The compose file sets pool_max_conns=20 (all roles in one process). Set it in the URL form of OHS_DATABASE_URL (postgres://...?pool_max_conns=20): the keyword/value form (host=db pool_max_conns=20) is not supported for it, because the migrations and ohs doctor drop the pool_ parameters from the URL form only. Elsewhere, without a setting pgx sizes it to the number of CPUs (at least 4), which is too small once several roles share it: the ETL alone runs 4 jobs per tier on two tiers, and the uploads, pulls, deletion worker and background jobs each hold a connection while they work. Set the size with pool_max_conns in the URL:

OHS_DATABASE_URL=postgres://ohs:secret@db:5432/ohs?sslmode=disable&pool_max_conns=20
Roles in the processRecommended pool_max_conns
api,collect,etl (the default, and the Pi)20
api only12
collect only8
etl only12

Keep the sum over all processes under Postgres max_connections (100 by default, 30 in the Pi overlay), with a few to spare for psql and migrations.

Provider keys in every role

The collect role pulls and subscribes; the etl role carries out user deletions, which revoke grants and unsubscribe notifications at the provider. Give the provider keys to every process that runs either role. A process without them cannot call that provider: it logs a warning, and a deletion it carries out still deletes the data but leaves the grant valid at the provider.

Put a reverse proxy with TLS in front of port 8080; the bearer secrets travel in headers. hardening.md is the full production checklist, and security-model.md the threat model behind it.

Rate limits

The provider-facing routes take no credential: POST and GET /inbound/{provider} (notifications and the URL check) and GET /oauth/{provider}/callback. Neither does POST /v1/setup (the first-run setup), which checks a token. Each notification may verify a signature, write rows and queue pulls, and each callback reads the database, so an in-memory token bucket bounds them, per api process. A request over a limit is answered 429 with Retry-After (seconds) before any work, and counted in /metrics as ohs_rate_limited_total{route_group="inbound"}, {route_group="oauth_callback"} or {route_group="setup"}. The authenticated API is not limited.

BucketDefaultVariable
Notifications, per client address30 requests/s, bursts of 120OHS_RATE_LIMIT_INBOUND
Notifications, per provider (all addresses)300 requests/s, bursts of 600OHS_RATE_LIMIT_INBOUND_PROVIDER
OAuth callbacks, per client address2 requests/s, bursts of 20OHS_RATE_LIMIT_OAUTH
First-run setup (POST /v1/setup), per client addresssame as OAuth callbacks, its own bucketOHS_RATE_LIMIT_OAUTH

The values are rate/burst (30/120; the rate may be a decimal, 0.5/10), and 0 turns a bucket off. The defaults sit well above what a provider sends, batched notifications and a large backfill included; providers retry a 429. The per provider bucket keeps a flood from many spoofed addresses from queueing unlimited work. An IPv6 client is counted by its /64. Buckets idle for 10 minutes are dropped, and at most 100,000 addresses are tracked per route group: beyond that, new addresses share one bucket until idle ones are dropped.

The client address is the TCP peer. Behind a reverse proxy every request comes from the proxy, so list the proxy in OHS_TRUSTED_PROXIES: for a request from one of those addresses, the client is the rightmost X-Forwarded-For entry that is not itself a trusted proxy. The header is ignored from any other peer, so a client cannot pick its own address. With the bundled compose file and a proxy on the host in front of the published port 8080, requests reach the container from the Docker bridge gateway. Make sure the proxy appends the address it received the request from to X-Forwarded-For (most do by default), then trust the bridge networks:

# .env
OHS_TRUSTED_PROXIES=172.16.0.0/12

Name only addresses your proxies use: a trusted range that also holds clients lets them forge the header. Several api processes each keep their own buckets, so the limits add up across them.

Edit on GitHub

On this page