Run long jobs from GitHub Actions without paying for the wait
A two-hour integration suite, a model evaluation, a firmware build on real hardware: a hosted runner bills every minute it sits waiting. Start the work as a detached job on a machine you own, let the workflow finish in seconds, and look at the result later — from a delayed check, or when your own receiver gets a signed "job ended" webhook.
- Daily reactivation. By default an account API key (and OAuth connector) is refused with
403 reactivation_requiredwhen the account owner has not opened the dashboard in the last 24 hours. Unattended CI needs either a daily dashboard visit or the per-account opt-out in the dashboard (a deliberate, risk-increasing setting — see Security). - What you actually save. The start and the check each still bill about one runner minute (rounded up per job). What you stop paying for is the waiting.
- Use a separate API key for CI. Do not reuse your main key: create one only for this workflow so it
can be revoked on its own. A key reaches every machine on its account, so keep the account that CI uses as small as
you can. Store it as a repository secret and expose it only to trusted triggers (push to
main, tags, approved environments) — never to pull requests from forks orpull_request_target.
Two patterns
- Pattern A — delayed check (no webhook). A
startjob launches the work; acheckjob, held back by an environment wait timer, asks once whether it finished. - Pattern B — completion webhook.
on_complete.urlon the start request; AI Commander sends one signed POST to a receiver you own, and that receiver starts a GitHub workflow with its own token.
Pattern A: start now, check after a wait timer
start calls POST /api/v1/jobs and saves the jobId as a job output.
check runs in an environment with a wait timer,
then makes one GET /api/v1/jobs/{id}: exited with exit code 0 is green; running,
unknown or a non-zero code is red, and the last 50 log lines are printed.
GITHUB_TOKEN may be possible but is not something we have verified,
so we do not recommend relying on it. A schedule: trigger is the usual alternative, but it only runs on the
default branch, at most every 5 minutes, can be delayed or dropped under load, is disabled after 60 days without
activity in public repositories, and bills at least one minute per run on private repositories.
Copy-paste workflow (also in the repository as examples/github-actions/long-job-check.yml):
# Run a long job on your own machine from GitHub Actions without paying for the wait.
# Pattern A from https://aicommander.dev/use-cases/github-actions-long-jobs/
#
# start - POST /api/v1/jobs, records the jobId as a job output, finishes in seconds.
# check - runs in the `aic-wait` environment, whose WAIT TIMER delays it (the wait is not
# billed), then does ONE GET /api/v1/jobs/{id}:
# status "exited" and exitCode 0 -> green
# "running", "unknown", non-zero -> red (last 50 log lines are printed)
# A red "still running" just means the timer was too short: re-run the check later.
#
# Before you use it:
# * Environment wait timers are available for public repos on Free/Pro/Team and for private
# repos only on GitHub Enterprise. Create the environment (Settings -> Environments ->
# `aic-wait`) and set "Wait timer" to how long the job usually takes.
# * Secret AIC_API_KEY: a SEPARATE AI Commander API key used only for CI, so you can revoke
# it on its own. Variable AIC_MACHINE: the machine alias (or AIC- code).
# * Only run this from trusted triggers (push to main, tags, approved environments) - never
# from pull requests from forks or `pull_request_target`, which could leak the key.
# * By default an API key stops working (403 reactivation_required) when the account owner
# has not opened the dashboard in the last 24 h, unless that requirement is turned off for
# the account in the dashboard. Unattended CI needs one or the other.
# * Billing: each of the two jobs still bills about one runner minute; what you save is the
# time the job runs on your machine.
name: Long job on my own machine
on:
push:
branches: [main]
workflow_dispatch:
permissions: {}
jobs:
start:
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
job_id: ${{ steps.start.outputs.job_id }}
steps:
- id: start
env:
AIC_API_KEY: ${{ secrets.AIC_API_KEY }}
AIC_MACHINE: ${{ vars.AIC_MACHINE }}
# The command runs on YOUR machine via /bin/sh -c (cmd.exe on Windows).
JOB_COMMAND: /home/ci/run-long-job.sh
run: |
set -euo pipefail
body=$(jq -n --arg code "$AIC_MACHINE" --arg cmd "$JOB_COMMAND" --arg sha "$GITHUB_SHA" \
'{code: $code, command: $cmd, name: "gha-long-job", env: {GIT_SHA: $sha}}')
resp=$(curl -sS --fail-with-body -X POST https://aicommander.dev/api/v1/jobs \
-H "Authorization: Bearer $AIC_API_KEY" \
-H 'Content-Type: application/json' \
-d "$body")
job_id=$(jq -er '.job.jobId' <<<"$resp")
echo "Started job $job_id"
echo "job_id=$job_id" >> "$GITHUB_OUTPUT"
check:
needs: start
runs-on: ubuntu-latest
timeout-minutes: 5
environment: aic-wait # the wait timer lives on this environment
steps:
- env:
AIC_API_KEY: ${{ secrets.AIC_API_KEY }}
AIC_MACHINE: ${{ vars.AIC_MACHINE }}
JOB_ID: ${{ needs.start.outputs.job_id }}
run: |
set -euo pipefail
[[ "$JOB_ID" =~ ^[0-9a-f]{16}$ ]] || { echo "::error::bad job id"; exit 1; }
base="https://aicommander.dev/api/v1/jobs/$JOB_ID"
resp=$(curl -sS --fail-with-body -G "$base" --data-urlencode "code=$AIC_MACHINE" \
-H "Authorization: Bearer $AIC_API_KEY")
status=$(jq -r '.job.status' <<<"$resp")
exit_code=$(jq -r '.job.exitCode' <<<"$resp")
echo "status=$status exitCode=$exit_code"
if [ "$status" = "exited" ] && [ "$exit_code" = "0" ]; then
exit 0
fi
case "$status" in
running) echo "::error::Job $JOB_ID is still running - re-run this check later." ;;
unknown) echo "::error::Job $JOB_ID ended with no recorded exit code (outcome unknown)." ;;
*) echo "::error::Job $JOB_ID exited with code $exit_code." ;;
esac
echo "--- last 50 log lines ---"
curl -sS -G "$base/logs" --data-urlencode "code=$AIC_MACHINE" --data-urlencode "tail_lines=50" \
-H "Authorization: Bearer $AIC_API_KEY" | jq -r '.logs.chunk // empty' | base64 -d || true
exit 1
Pattern B: a completion webhook to your own receiver
Add on_complete to the start request. When the job ends, AI Commander sends one small, signed POST to
the URL; the workflow that started the job has long since finished.
curl -sS --fail-with-body -X POST https://aicommander.dev/api/v1/jobs \
-H "Authorization: Bearer $AIC_API_KEY" -H 'Content-Type: application/json' \
-d '{"code":"build-box","command":"/home/ci/run-long-job.sh","name":"gha-long-job",
"on_complete":{"url":"https://aic-hook.example.com/hook/k7Qx2mVb9RtLw4sYpN8eHc3JdZ6aUf1G","deadline_seconds":21600}}'
# → {"ok":true,"job":{"jobId":"9f2c1b7ad4e05386","status":"running",…},
# "webhook":{"state":"armed","host":"aic-hook.example.com","deadlineAt":1760300000000}}
- Best effort. At most 3 attempts — at 0, 1 and 5 minutes after the event — then it stops (an attempt more than about 15 minutes late is dropped, and nothing is sent for an unlinked or blocked machine); if nothing
arrives, poll
GET /api/v1/jobs/{id}yourself. A permanent 4xx answer ends the attempts; redirects are not followed. Delivery is best effort — at most 3 attempts (0, 1 and 5 minutes after the event), then it stops; duplicates are possible (rarely, when relay storage fails), so de-duplicate byAIC-Delivery-Id; pollGET /api/v1/jobs/{id}if you must be sure. - Deadline.
deadline_secondsdefaults to 24 h (60 s to 7 days). If no end event arrived by then, ajob.deadlineevent is sent instead — it says nothing about the job's state. If the relay could not record the job id (rare: a storage failure right after the job started) and the job has still not finished at the deadline, nojob.deadlineis sent — pollGET /api/v1/jobs/{id}. - Minimal payload.
job.endedcarriesv,event,deliveryId,jobId,status,exitCode,endedAt,endedAtApproximate,attempt;job.deadlinecarriesv,event,deliveryId,jobId,attempt. Never the command, logs, job name or machine alias. - The URL is a secret. It may carry one in its path (e.g. n8n). It is stored encrypted until delivery
finishes, then erased, and the API only ever shows its host. It must be
httpson port 443, in canonical form (exactly equal tonew URL(url).href), with no credentials; there are no custom headers or body templates. - Requirements. An account credential (API key or OAuth) — anonymous sessions cannot use it — and
agent 1.5.0 or newer (job events support) on the machine; otherwise the start
is refused (
403 forbidden) and the job is not started. Other refusals:400for an invalidon_complete,429 rate_limitedat 100 webhooks in flight per account, 500 per account and target host per day. A502 errorreading “Could not record the job webhook; the job was not started.” is safe to retry; any other502may be an uncertain start, so checkGET /api/v1/jobsbefore retrying.
Actions: write permission. Never use a classic token or a token with
broader access. Prefer workflow_dispatch: repository_dispatch needs Contents: write,
which is write access to your code. You run the receiver and its token at your own risk.
Example receiver: a Cloudflare Worker
It does the four things every receiver must do: verifies the Ed25519 signature with the
published public key (selected by AIC-Key-Id), binds the
signature to its own URL, rejects timestamps more than 5 minutes off, and de-duplicates by deliveryId with an atomic claim in D1
(INSERT … ON CONFLICT DO NOTHING) before any side effect. A KV get-then-put is not enough:
KV is eventually consistent, so concurrent replays of one valid signed delivery would all dispatch.
Then it checks that the job is yours, and only then does it call GitHub's workflow_dispatch endpoint. The signed string is
v1.<AIC-Timestamp>.<deliveryId>.<b64url(sha256(url))>.<raw body>.
// Cloudflare Worker: verifies AI Commander job webhooks, then dispatches a GitHub workflow.
// Bindings:
// DB D1 database (atomic de-duplication by deliveryId, schema below)
// GITHUB_TOKEN secret: fine-grained PAT, ONE repository, ONLY "Actions: write"
// RECEIVER_URL var: the exact canonical URL you passed as on_complete.url
// GH_REPO var: "owner/repo"
// GH_WORKFLOW var: "aic-job-ended.yml"
// GH_REF var: "main"
// AIC_API_KEY secret: a DEDICATED AI Commander API key for this receiver only
// AIC_MACHINE var: the machine alias your jobs run on, e.g. "build-box"
const KEYS_URL = "https://aicommander.dev/.well-known/aic-webhook-key.json";
const MAX_JOB_AGE_MS = 7 * 24 * 3600 * 1000; // optional: ignore jobs started longer ago than this
const enc = new TextEncoder();
const toB64url = (buf) =>
btoa(String.fromCharCode(...new Uint8Array(buf)))
.replace(/\+/g, "-").replace(/\//g, "_").replace(/=+$/, "");
const fromB64url = (s) =>
Uint8Array.from(atob(s.replace(/-/g, "+").replace(/_/g, "/") + "===".slice((s.length + 3) % 4)),
(c) => c.charCodeAt(0));
export default {
async fetch(request, env) {
if (request.method !== "POST") return new Response(null, { status: 405 });
// 1. Bind to OUR OWN configured URL, never to anything the request claims.
if (request.url !== env.RECEIVER_URL) return new Response(null, { status: 404 });
const ts = request.headers.get("AIC-Timestamp") ?? "";
const deliveryId = request.headers.get("AIC-Delivery-Id") ?? "";
const signature = request.headers.get("AIC-Signature") ?? "";
const kid = request.headers.get("AIC-Key-Id") ?? "";
// 2. Five-minute window against our own clock.
if (!/^\d+$/.test(ts) || Math.abs(Date.now() / 1000 - Number(ts)) > 300) {
return new Response("stale", { status: 401 });
}
// 3. Ed25519 over v1.<timestamp>.<deliveryId>.<b64url(sha256(url))>.<raw body>
const body = await request.text();
const urlHash = toB64url(await crypto.subtle.digest("SHA-256", enc.encode(env.RECEIVER_URL)));
const signed = enc.encode(`v1.${ts}.${deliveryId}.${urlHash}.${body}`);
const keySet = await (await fetch(KEYS_URL, { cf: { cacheTtl: 300 } })).json();
const jwk = (keySet.keys ?? []).find((k) => k.kid === kid);
if (!jwk) return new Response("unknown key", { status: 401 });
let valid = false;
try {
const key = await crypto.subtle.importKey(
"jwk", { kty: "OKP", crv: "Ed25519", x: jwk.x }, { name: "Ed25519" }, false, ["verify"]);
valid = await crypto.subtle.verify({ name: "Ed25519" }, key, fromB64url(signature), signed);
} catch { valid = false; }
if (!valid) return new Response("bad signature", { status: 401 });
const msg = JSON.parse(body);
if (msg.deliveryId !== deliveryId || !/^[0-9a-f]{16}$/.test(msg.jobId)) {
return new Response("bad payload", { status: 400 });
}
// 4. De-duplicate ATOMICALLY before any side effect: up to 3 attempts (and any
// replay) carry the same deliveryId; only the request that inserts the row proceeds.
const claim = await env.DB.prepare(
"INSERT INTO deliveries (id, at) VALUES (?, ?) ON CONFLICT(id) DO NOTHING")
.bind(deliveryId, Date.now()).run();
if (claim.meta.changes !== 1) return new Response(null, { status: 204 });
// 5. OWNERSHIP: the signature proves AI Commander sent this, NOT that the job is ours.
// Anyone who learns this URL can put it in on_complete of their OWN jobs. Ask the
// relay, with OUR key, whether this job exists on OUR machine.
let owned;
try {
const r = await fetch(
`https://aicommander.dev/api/v1/jobs/${msg.jobId}?code=${encodeURIComponent(env.AIC_MACHINE)}`,
{ headers: { Authorization: `Bearer ${env.AIC_API_KEY}` }, signal: AbortSignal.timeout(10_000) });
// Only 404 means "no such job for this key/machine". Everything else that is not 2xx
// (403 incl. reactivation_required, 401, 429, 5xx) is "could not decide" -> retry.
if (r.status === 404) {
owned = false;
} else if (!r.ok) {
throw new Error(`ownership check: HTTP ${r.status}`);
} else {
const job = (await r.json()).job ?? {};
owned = job.jobId === msg.jobId &&
(typeof job.startedAt !== "number" || Date.now() - job.startedAt <= MAX_JOB_AGE_MS);
}
} catch {
// Could not decide (network, timeout, 403, 401, 429, 5xx): release, let the next attempt retry.
// A dormant account's key gets 403 reactivation_required, so we answer 502 until the
// account is active again; the 3 attempts may run out, so after long dormancy poll instead.
await env.DB.prepare("DELETE FROM deliveries WHERE id = ?").bind(deliveryId).run();
return new Response("ownership check failed", { status: 502 });
}
// Not ours: drop it and KEEP the claim, so a replay of this delivery is never re-evaluated.
if (!owned) return new Response(null, { status: 204 });
// The webhook is a HINT: the dispatched workflow confirms with GET /api/v1/jobs/{id}.
// ANY failure (network error, timeout, non-2xx) releases the claim and answers 502,
// so AI Commander's next attempt (at 0 / 1 / 5 min only) can dispatch again.
let dispatched = false;
try {
const gh = await fetch(
`https://api.github.com/repos/${env.GH_REPO}/actions/workflows/${env.GH_WORKFLOW}/dispatches`,
{
method: "POST",
headers: {
Authorization: `Bearer ${env.GITHUB_TOKEN}`,
Accept: "application/vnd.github+json",
"X-GitHub-Api-Version": "2022-11-28",
"User-Agent": "aic-webhook-receiver",
"Content-Type": "application/json",
},
body: JSON.stringify({ ref: env.GH_REF, inputs: { job_id: msg.jobId, event: msg.event } }),
signal: AbortSignal.timeout(10_000),
});
dispatched = gh.ok;
} catch { dispatched = false; }
if (!dispatched) {
await env.DB.prepare("DELETE FROM deliveries WHERE id = ?").bind(deliveryId).run();
return new Response("dispatch failed", { status: 502 });
}
return new Response(null, { status: 204 });
},
};
on_complete on jobs in their own
account (up to 500 a day per account and host) and receive validly signed deliveries that would trigger your workflow
and burn your runner minutes. A check inside the dispatched workflow comes too late. Step 5 above is what stops
cross-account triggering: before dispatching, the receiver calls GET /api/v1/jobs/{jobId}?code=<your machine
alias> with its own key and dispatches only if the job exists for that key (and, optionally, started
recently); only a 404 is dropped with 204 and the claim is kept, while any other
failure (403, 401, 429, 5xx, network error) releases the claim and answers
502 so the next attempt can retry. It costs one
authenticated GET per delivery. Use a dedicated, minimal-reach API key for it, not your CI key; the
dormancy gate applies to it too, so if the account goes dormant the check fails with 403 reactivation_required
and the receiver answers 502 until it is active again (the 3 attempts may run out; after a long dormancy,
poll GET /api/v1/jobs instead).
The D1 binding and table it needs (prune rows older than a day with a cron if you like), plus its vars and secrets:
# wrangler.toml
[[d1_databases]]
binding = "DB"
database_name = "aic-hook"
database_id = "<from: wrangler d1 create aic-hook>"
-- schema.sql (wrangler d1 execute aic-hook --remote --file=schema.sql)
CREATE TABLE IF NOT EXISTS deliveries (id TEXT PRIMARY KEY, at INTEGER NOT NULL);
# wrangler.toml - plain vars
[vars]
RECEIVER_URL = "https://aic-hook.example.com/hook/<long-random-path>"
GH_REPO = "owner/repo"
GH_WORKFLOW = "aic-job-ended.yml"
GH_REF = "main"
AIC_MACHINE = "build-box"
# secrets (never in wrangler.toml)
wrangler secret put GITHUB_TOKEN # fine-grained PAT, one repository, Actions: write only
wrangler secret put AIC_API_KEY # dedicated AI Commander key for the ownership check
The dispatched workflow treats the webhook as a hint and confirms the result itself with an authenticated
GET /api/v1/jobs/{id}, so even a duplicated or replayed dispatch can never produce a wrong result:
# .github/workflows/aic-job-ended.yml - dispatched by your receiver
name: AI Commander job ended
on:
workflow_dispatch:
inputs:
job_id: { required: true, type: string }
event: { required: true, type: string }
permissions: {}
jobs:
check:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- env:
AIC_API_KEY: ${{ secrets.AIC_API_KEY }}
AIC_MACHINE: ${{ vars.AIC_MACHINE }}
JOB_ID: ${{ inputs.job_id }} # passed via env, never interpolated into the script
run: |
set -euo pipefail
[[ "$JOB_ID" =~ ^[0-9a-f]{16}$ ]] || { echo "::error::bad job id"; exit 1; }
# Same check script as Pattern A: confirm with an authenticated GET.
resp=$(curl -sS --fail-with-body -G "https://aicommander.dev/api/v1/jobs/$JOB_ID" \
--data-urlencode "code=$AIC_MACHINE" -H "Authorization: Bearer $AIC_API_KEY")
status=$(jq -r '.job.status' <<<"$resp"); exit_code=$(jq -r '.job.exitCode' <<<"$resp")
echo "status=$status exitCode=$exit_code"
[ "$status" = "exited" ] && [ "$exit_code" = "0" ]
Any receiver that accepts an arbitrary JSON body works the same way — an n8n Webhook node or your own endpoint — as
long as it is reachable over https on port 443 (a self-hosted n8n on port 5678 is refused), verifies the
signature and de-duplicates atomically before acting. Do not point on_complete straight at
a Slack incoming webhook URL: it cannot verify the signature, and it rejects a body without text or
blocks with a 400, which ends delivery as a permanent failure. To notify Slack, put a small receiver like
the one above in between and have it post the message.
Planned: a built-in GitHub preset
Planned, not available. We intend to add an option that dispatches a GitHub workflow directly, without a receiver
of your own, using a fine-grained token limited to one repository and Actions: write. Until it ships, use
Pattern A or Pattern B.
FAQ
Let the runner go home early
Install the agent on the machine that should do the work, then start jobs on it from your workflow.