Ask Claude to start the training. Come back tomorrow and ask how it went.

A fine-tune runs for ten hours. The chat you started it from will not last ten hours, and neither will your laptop's battery. AI Commander starts the run as a detached job on your own GPU box, hands back a job id, and gets out of the way — the run keeps going after the chat ends, the laptop sleeps, or the network drops.

Nothing to keep open

You already know the ritual. Open a terminal, ssh into the box, start tmux so the session survives the connection dropping, launch the run under nohup with output redirected somewhere you will remember, detach, and then hope you can find your way back to the same pane tomorrow. Half of that is ceremony for keeping a session alive that nobody wanted to keep alive.

A detached job removes the ceremony. The agent spawns the process on the machine, writes its stdout and stderr to a file there, records the exit code to disk, and returns a jobId to the caller immediately. Nothing on your side has to stay connected for the run to continue:

• Close the chat — the job does not belong to the conversation, it belongs to the machine.
• Shut the laptop, get on a train, lose Wi-Fi. The run is not on your laptop.
• The machine drops offline for an hour. The process is untouched; you simply cannot query it until the machine is back.
• Pick it up tomorrow from a different client — the phone, another laptop, a script — with the jobId.

The job also survives the AI Commander agent itself restarting, on all three platforms. On Linux — the platform most GPU boxes run — that is because each job started by an agent that already has this feature is launched into its own transient systemd scope, outside the agent service's control group, so stopping, restarting or upgrading the agent leaves it running. The one condition: the box runs systemd and the agent runs as root. Where it does not (a non-systemd NAS, a non-root agent) jobs stay in the service's control group and an upgrade still takes them down — the agent says which of the two it got in one line at startup, and again only if a later re-probe changes that answer. A run that was already going when the agent was upgraded to that version is in no scope either, and the restart the upgrade performs is what ends it: an upgrade is safe for the runs you start afterwards, not for the one in flight. So check remote_job_list and upgrade between runs, not during one.

No tmux, no screen, no SSH session to keep alive. Ask for the run, then ask later how it went.

Pick the machine, then reserve the card

Before anything starts, the agent needs to know where to put it. list_machines reports every machine on your account, and every NVIDIA card on each one — model, total and used VRAM in MiB, and current utilization. session_status reports the same for a single machine. That is enough to choose a box from data the client already has, instead of blind-probing each one with nvidia-smi:

No GPU section at all means no NVIDIA card or no driver on that machine — the agent should say so rather than try. Where a card is busy, the fix is not to hope: remember the card's index and pass it as gpu_index when starting the job. That takes an exclusive lock on the card and sets CUDA_VISIBLE_DEVICES for the job, so a second job asking for the same card is refused with gpu_busy, naming the job that holds it, instead of both runs hitting CUDA OOM four hours in. The lock is released when the job ends.

The figures are a reading the machine pushes about once a minute. While a machine is offline they are its last known values and may be stale.

The five tools, and their REST twins

Five MCP tools cover the whole life of a job, and each one has an HTTP equivalent for clients that do not speak MCP. code names the machine exactly as it does for /api/v1/exec — in the JSON body for the POST, and as a query parameter everywhere else.

cwd must be an absolute path that already exists, and env takes string values only. There is no elevated option for jobs — a job runs with exactly the rights remote_exec has on that machine. Full shapes and error codes are in the tool reference and the machine-readable OpenAPI spec.

What it looks like

A PyTorch fine-tune, in a uv workspace under the user's home, on the free card:

Nine hours later, from the phone, in a brand new conversation:

The offset_bytes round-trip is the whole trick for following a long run: each log reply carries a nextOffsetBytes, you hand it back on the next call, and you get only what was written since. Note that nothing polls on its own: there is no scheduler and no callback, so a job is looked at exactly when someone asks about it — which is why the jobId matters more than the conversation it was started in. See the GPU jobs skill for the full procedure an agent follows — workspace layout with uv, the CUDA sanity check, and where model weights should land.

What this is not

Your own box is a real alternative to Modal or RunPod for this work — it is your hardware, your disk, and a real shell. What it is not is a managed platform, and it is worth being blunt about the difference:

• Nothing schedules. There is no queue. A job starts when you ask for it, or it is refused — gpu_busy if the card is taken, too_many_jobs at 32 running jobs on a machine. Poll and retry, or pick another card.
• Nothing retries. A crashed run stays crashed until someone starts it again.
• Nothing is kept for you. Finished jobs — metadata, log and workspace — are retained for about a week and then removed automatically: the agent sweeps expired jobs in the background, roughly hourly, driven by job start and list activity. Starting or listing a job can therefore be the thing that deletes an older expired one. Copy out anything you want to keep well before the week is up.
• unknown means unknown. If the agent restarted and the process is gone with no recorded exit code, the outcome genuinely cannot be determined. It is not a quiet success; read the log and check your checkpoints.
• A job outlives you. Nothing stops one started by mistake — it burns GPU, CPU and disk until it finishes or is cancelled. Expensive work deserves an explicit confirmation before it starts.
• Pro file transfer is a courier, not a drive. With Pro, a checkpoint can come back: built-in remote file transfer with remote_pull(code, path) reads an absolute path off the machine into a temporary blob on the relay and returns a download link. But the stored copy is unreadable 24 hours after it is created — fetched or not — and an hourly, retrying sweep removes the inaccessible bytes afterward. The link itself stops working after an hour, and a single file must be under 100 MiB. That comfortably covers a LoRA adapter, a metrics dump or a sample render; it does not cover a multi-GB full-weights checkpoint, and for those the older recipe is still the right one — have the job copy the artifact to storage you already control (aws s3 cp, rclone, scp) as its final step. Going the other way, remote_push(code, blob_id, dest_path) writes a stored blob onto the machine. Free and anonymous callers cannot initiate transfer. Transfers are also metered: a transfer call answers in about 55 seconds, an account gets 60 transfers an hour and 5 GiB of relay storage per rolling day — a hard ceiling on any rolling 24 hours, spent by pulls and uploads but not by pushes, freed in hourly steps over up to 25 hours, and a pull holds the full 100 MiB against it until the transfer settles — one more reason a real checkpoint belongs in the job's own upload step. Details in Docs → File transfer.

Set it up

On the GPU machine (Linux), install the agent:

Follow the signed Linux installer steps and verify the installer before sudo.

Connect your AI client, save the machine under an alias like gpu-box, and describe the run you want started.

FAQ