Driving div from a script

div is the surface an agent drives, so its output is split by whose it is. A shell script, a CI job and an agent can all branch on what it does without reading a word of what it says.

stdout is the run’s

The run’s own output goes to stdout and stderr exactly as the container wrote them — stdout on stdout, stderr on stderr — and nothing of ours goes in between. Everything div says itself goes to stderr.

div logs run-7 -f | grep loss
div run --gpu h100 eval.py > eval.log   # eval.log is the run's output and nothing else
div storage get training-volume-1 log/training.jsonl - | tail -5

Exit codes are the run’s

A run that exits 3 makes div run exit 3. The same holds for div logs -f once the run has finished. A command typed wrong exits 2 and names the flag; the API refusing exits 1. Exit codes and errors has the whole table.

if ! div run --gpu h100 train.py::run --max-spend 20; then
  echo "training failed with $?" >&2
fi

A run can answer, not just exit

An exit code says whether the run crashed. What it measured goes in its result: a JSON object written to DIVERGENT_RESULT_PATH before it exits, which div result prints and a check is judged against.

json.dump({"metrics": {"accuracy": acc}}, open(os.environ["DIVERGENT_RESULT_PATH"], "w"))

div run --json eval.py --check 'metrics.accuracy>=0.8' > run.json   # exits with the run's own code
jq -e 'all(.checks[]; .verdict == "passed")' run.json > /dev/null || echo "the experiment did not pass" >&2
div result "$(jq -r .name run.json)" metrics.accuracy

The exit code says whether the run worked; the checks say whether the experiment did. They are kept apart on purpose, so a script can tell a crash from a result it did not like.

JSON when you want a structure

Command	What --json prints
div run --json	the run as it ended, as the API sent it
div run --detach --json	the run as it was placed, then returns
div logs --json	the log lines with their sequence numbers
div runs --json	the runs, newest first, as the API sent them
div stop --json	the run as it stopped
div catalog --json	GET /catalog, as the API sent it
div whoami --json	GET /me, as the API sent it
div limits --json	GET /billing/summary, as the API sent it

The CLI keeps every field it does not name, so --json prints what the API sent — including fields added after this build of div.

Placing and walking away

--detach prints the run’s name on stdout and returns as soon as the run is placed:

name=$(div run --detach --gpu h200 --gpus 8 finetune.py::lora --max-spend 400)
div logs "$name" -f

Ctrl-C does the same thing to a run you are attached to — it detaches, leaving the run going to its ceiling, and prints on the way out:

detached. run-k4t9 is still running at $4.95/h and stops at its $5.00 ceiling.
watch it again with  div logs run-k4t9 -f
stop it with         div stop run-k4t9

Finding and stopping what is running

div runs prints one run per line with the name first and no header, so it pipes. div stop exits 0 whether it stopped the run or the run had already finished, so a script does not have to check first:

div runs --status active                                        # what is holding hardware now
div runs --status active | cut -d' ' -f1 | xargs -n1 div stop   # stop all of it

Colour

Colour is switched off when stdout is not a terminal. When it is on it follows the product’s rule: green for a run that is running, oxblood for one that failed, mustard for a ceiling that caught up, and grey for a clean finish.

A key without a login

On a machine nobody sits at, skip div login and set the key in the environment. A key made in the dashboard under Settings works the same as one div login collected.

export DIVERGENT_API_KEY=dv_live_...

See Configuration for everything else that can be set this way.