div telemetry

Synopsis

div telemetry <run> [key...] [flags]

Prints the metrics a run appended to its telemetry file.

Arguments

Argument	Description
<run>	Run name or ID.
[key...]	Keys to print as columns, such as loss eval_loss. Without keys, prints a summary of every key.

Flags

Flag	Description	Default
--x <key>	Key for the first column. seq is the point’s position in the run.	step if the run records it, otherwise seq
--csv	Print CSV instead of tab-separated columns. Without keys, prints a column for every key.
--json	Print every point as one JSON object per line: {"seq": 1, "data": {...}}.
-f, --follow	Keep printing new points until the run ends.
--token <key>	API key to use.	see Configuration
--api-url <url>	API URL to use.	see Configuration

Output

Without keys, one row per key with its point count and its last, minimum and maximum value:

key        count    last     min    max
step         412   41200     100  41200
loss         412  0.0412  0.0398   2.31
eval_loss     41  0.0267  0.0261   1.88

With keys, tab-separated columns on stdout: the x key first, then one column per key. A point without a value for a key has an empty cell, not zero.

If any lines were dropped, the counts per reason are printed to stderr.

The telemetry file

The run appends one JSON object per line to the path in DIVERGENT_TELEMETRY_PATH (/tmp/divergent/telemetry.jsonl).

• Each number is kept under its key. Nested objects become dotted keys: {"eval": {"loss": 0.4}} is eval.loss.
• Strings, booleans, nulls, NaN and Infinity are ignored. The rest of the line is kept.
• Values of the org’s secrets are masked.

Limit	Value
Line size	4 KiB. Larger lines are dropped.
Numbers per line	64. Lines with more are dropped.
Keys per run	256
Points per run	200,000
Key format	letters, digits, _ and -, separated by dots. Keys such as train/loss are dropped from the point.

Lines that are not JSON objects are dropped. The run’s log reports the first drop for each reason.

telemetry.toml

Lays out the charts on the run’s Telemetry tab. div run uses telemetry.toml at the root of the uploaded directory, or the file passed with --telemetry. The layout is saved when the run starts. An invalid file rejects the run.

x = "step"                       # x axis for every panel

[[panel]]
title = "Loss"
y = ["loss", "eval_loss"]        # up to 4 keys
scale = "log"

[[panel]]
title = "F1"
y = ["eval_strict_f1", "eval_typed_f1"]
min = 0.9
max = 1.0
rule = 0.95                      # horizontal reference line

[[panel]]
y = "eval_leak_recall"
wide = true                      # spans both columns

[[stat]]
title = "Samples/s"
y = "eval_samples_per_second"
reduce = "last"

Key	Applies to	Description	Default
x	file, [[panel]]	Key to plot against.	the point’s position in the run
title	[[panel]], [[stat]]	Display name.	the keys
y	[[panel]], [[stat]]	Key, or up to 4 keys for a panel.	required
scale	[[panel]]	linear or log.	linear
min, max	[[panel]]	Y-axis bounds. min must be below max.	fitted to the data
rule	[[panel]]	Y value of a horizontal reference line.	none
wide	[[panel]]	Span both columns.	false
reduce	[[stat]]	last, min or max.	last

Without a telemetry.toml, the tab shows one panel per key, in the order the keys first appeared, plotted against step if the run records it. A point without a panel’s x key is not drawn on that panel.

Exit codes

Code	Meaning
0	Printed.
1	The API rejected the request.
2	Invalid flags or arguments.

Examples

div telemetry run-7
div telemetry run-7 loss eval_loss > curve.tsv
div telemetry run-7 eval_loss --x epoch | tail -1
div telemetry run-7 loss -f
div telemetry run-7 --csv > run-7.csv

See Chart training metrics.