Security

Reporting a vulnerability

Please report security problems privately, not in a public issue:

• use GitHub’s “Report a vulnerability” (private security advisory) on this repository, or
• email the maintainer at the address on the GitHub profile, with “jobwatch security” in the subject.

Include what you found, how to reproduce it and which version or image digest you ran (cosign verify output from docs/verify.md is ideal). You get an acknowledgement within 3 working days. Fixes are released as new signed images and agent binaries; the advisory names the first fixed version. Please give us 90 days before publishing, less if the issue is already being exploited.

Supported versions: the latest release of the agent (ghcr.io/<owner>/jobwatch-agent, jobwatch-host) and the server (ghcr.io/<owner>/jobwatch-server). There are no long-term branches yet.

Threat model (summary)

jobwatch has three parts: agents in your clusters and on your hosts (open source, Apache-2.0), a server you or a provider runs, and a web app served by that server. The design goal is that the server and its operators learn when your jobs run and whether they fail, but never read their logs, and that an agent can be trusted with read-only access to your cluster.

What is signed

What is end-to-end encrypted

Job logs, when the org’s log mode is sealed (proto/phase4.md section A):

• The org has an X25519 key pair created in the Owner’s browser. The server stores only the public key.
• Agents seal each failed run’s masked log tail to that public key (jwseal1: X25519 + HKDF-SHA256 + AES-256-GCM, with a header binding org, jobKey and runId as AAD). The server stores the ciphertext and can’t open it.
• The org private key is wrapped to each approved device: a non-extractable WebCrypto key in that browser profile. It is also wrapped to a recovery key shown once to the Owner.
• Device approval is done by an Owner/Admin who compares the device fingerprint out of band. That comparison is what stops a malicious server from adding its own device.
• Agents can pin the org key fingerprint (encryption.orgKeyFingerprint). With a pin, a swapped key or a downgrade to plaintext makes the agent send no logs at all and raise key_mismatch on its own channels.
• metadata mode sends no log lines anywhere, not even sealed.

What is not encrypted (the server sees it)

• Metadata: cluster, namespace and job names, schedules, labels, run times and durations, exit codes and reason codes, image names, node names, incident history, who acknowledged what.
• Logs in plaintext mode (the phase-1/2 default). Agents still mask secrets in log lines before sending (rules in proto/README.md), but masking is best effort.
• Ping bodies in plaintext mode (dropped in sealed/metadata mode).
• Notification text sent by the agent to your own channels (Telegram, Slack, Discord, webhook). Those platforms see what you let the agent send; includeLogs defaults to false in sealed/metadata mode.
• Channel secrets (bot tokens, webhook URLs, SMTP passwords) are stored in Postgres (channels.secrets) so the server can send. They are write-only in the API and never logged, but anyone with database or backup access can read them. Protect database backups accordingly.

Trusting the web app

The browser decrypts logs, but the browser code itself comes from the server. A malicious or compromised server could serve JavaScript that exfiltrates decrypted logs or device keys the next time you open the app. End-to-end encryption protects against a passive or curious operator, database leaks and backups. It does not protect against an operator who actively changes the web app. If that matters to you:

• run the server yourself from a verified image (docs/verify.md), or
• open logs only from a deployment whose image digest you checked.

Everyone holding the org key can decrypt every cluster’s logs. Cluster scoping limits which sealed logs the server hands out, not who could decrypt them.

Access control

• Roles are Owner > Admin > Operator > Viewer. Any membership can be limited to clusters.
• Enforcement happens twice: application filters, and Postgres row-level security under an API database role without BYPASSRLS. Out-of-scope objects are 404.
• Org-wide settings (channels, routes, members, agent tokens) need Admin with all clusters. Encryption settings and retention changes need Owner.
• Ping URLs carry a 128-bit ping key (phase 4). Checks created before phase 4 keep their check-id URL until the key is rotated; the UI flags them, because their check id (visible to Viewers in the job key) is still a credential.
• The agent’s RBAC is read-only (get/list/watch on CronJobs, Jobs, Pods, pod logs, and opt-in discovery CRDs), plus write access to its own key Secret. Reruns need the opt-in runner.enabled role, limited to listed namespaces.
• Slack and Telegram buttons act as the linked jobwatch user, with the same role and cluster checks as the web app.

API keys and reports (phase 5)

proto/phase5.md. Org-managed API keys carry a role (incl. report-only Reporter) and an optional cluster list; a key can never exceed its creator’s role or clusters, keys can’t create keys (/v1/api-keys and /v1/usage are for signed-in users only, even a legacy Owner key gets 403), and revocation/expiry applies within 10 s on every instance (lookup cache). Secrets are jwk_ + 256 random bits, shown once, stored as SHA-256; the database role of the API cannot read the hash column; only the 10-character prefix is shown or logged; they never appear in metrics labels. The Report API is authenticated by the key over TLS but, unlike agent batches, not signed: a leaked Reporter key lets someone send fake results (and, in a plaintext org, log lines) for its clusters until it is revoked, limited by per-key rate limits, the per-org check cap and eventId dedupe/at window. In sealed and metadata orgs no plaintext line is stored from a report, and the server never decrypts a report’s envelope. Agents and the jobwatch-report CLI mask jwk_/jwa_ strings in log lines; the CLI reads its key only from JOBWATCH_API_KEY, fails closed on a key mismatch and reports the program name only, never its arguments. The CLI’s optional spool directory holds events (mode 0600).

Secrets handling

• The server stores tokens, API keys, session ids, CSRF values, invite tokens and link codes only as SHA-256 hashes.
• Sessions are server-side; the browser holds an HttpOnly cookie, plus a CSRF cookie bound to the session.
• Nothing secret goes into logs or alert text; errors from Telegram/Slack/Discord clients are scrubbed of tokens.
• Agent secrets come from Kubernetes Secrets or environment variables (tokenEnv, webhookUrlEnv, …), never the ConfigMap.
• Images are distroless/chiseled and non-root.