d10 production baseline (batch32+tok1B+14shards)

Funded Job Failed

UUID:58f7ecf1-727e-49d1-b40e-cd8dc4d15141

Tao Linvia🐎 pentiumCreated 213 days agoUpdated 213 days ago

FAILED — Spot preemption (NOT OOM)

Exit code 137 (SIGKILL) but reason=Error, not OOMKilled. Training was running fine at step 270 when killed. Peak mem 21 GB is well within 40 GB.

GCP reclaimed the spot VM. Need to retry.

Funding

Done

Fundings at snapshot$18.64

Individual funding and transaction records are not included in the public archive.

Repository: https://github.com/mlpatron/mlpatron-nanochat
Version: experiment/baseline-search → e39ceb9a3a373755eb29e04fe8c88563830ba089
Image: ghcr.io/nblintao/mlpatron-nanochat:latest
→ ghcr.io/nblintao/mlpatron-nanochat@sha256:28c5544c206237e407241b0a72a8707e63e52750f487b751675b51ff1bce1520
Entrypoint: a100_preset
Resources: NVIDIA A100 40GB × 1·a2-highgpu-1g (12 vCPU / 85 GiB)
Parameters: depth=10·eval_every=300·max_seq_len=2048·tok_max_chars=1000000000·num_iterations=1500(dryrun: 20)·num_data_shards=14·device_batch_size=32