# Autoresearch On-demand GPU nodes and asynchronous GPU jobs for AI agents, over MCP at https://mcp.autoresearch.sfcompute.com. A node is a container on an H100 with a persistent, encrypted disk, billed per minute; stop it and it parks: files intact, billing stopped. The next command wakes it. Every node type has a name and the name is the chip you ask for: "h100-1" is one GPU, "h100-8" is a full 8-GPU machine. Both also answer to their original spellings, "h100" and "8xh100", which is what responses echo. CPU-only nodes exist too, in two sizes: chip "cpu-2" (2 reserved physical cores, 8 GiB RAM) for downloads, ingest, and preprocessing, and chip "cpu-8" (8 reserved physical cores on one NUMA node, 32 GiB RAM) for parallel CPU work - same node semantics, no GPU ever, each priced as one unit. ## When to use this Use Autoresearch when a task needs a real GPU machine and there is no cluster to put it on. The jobs it is built for: - **Train or fine-tune a model** on 1 or 8 H100s. create_node, run the training command detached, poll it, export the weights. Nothing to provision, and stop_node parks the disk between sessions so the next run starts where the last one ended. - **Sweep hyperparameters.** submit_jobs takes a shared spec plus the variants and queues the whole grid in one call; each variant reports its own result and artifact. - **Reinforcement learning rollouts.** create_sandbox_env bakes an environment once, fork_sandbox clones it in about 10 ms, and sandbox_exec drives many at once - the loop shape, on its own request budget so it never spends an agent's tool calls. - **Benchmark or evaluate on real hardware,** where a number from a CPU or an emulator would not be the number. - **Preprocess, download, or convert a dataset** on a CPU-only node (chip "cpu-2" or "cpu-8") so no GPU bills while bytes move, then hand the result to a GPU node - snapshot the CPU node and restore onto the GPU shape, or go through org storage (export_data there, import_data here - no egress fee on either leg) when the data should stay reusable. - **Run something long and unattended** while a human watches: open a mission first and every node, command, job, artifact, and dollar rolls up on one page you hand them the URL for. When it is NOT the right tool: - You want a hosted model to call. This is compute you run your own code on, not an inference endpoint. - You want an interactive shell to sit in. Exec runs through run_command over the API and logs are the interface; if the work cannot be expressed as commands, this is the wrong platform. - The work has no GPU in it and no deadline. A laptop is cheaper. How to call it: an agent connects over MCP (below) and uses the tools; a script or CLI uses the HTTP API with a bearer token. Both doors reach the same capabilities - one route per tool, enforced by a test, with switch_active_org the single exception (see HTTP API below) - and both are self-serve - `gman login` mints a token from a browser or a device code, and authorizing the MCP server mints one through the consent screen. No sales call, no waiting list, no form. ## MCP quickstart https://mcp.autoresearch.sfcompute.com Streamable HTTP with OAuth 2.1; MCP clients discover auth automatically (RFC 9728); a human approves once. Example: claude mcp add --transport http givemeanode https://mcp.autoresearch.sfcompute.com On a remote or cloud machine, that approval redirects to a localhost which is a different machine from the one the client runs on, so it never lands. Install the CLI there and run `gman mcp install` instead. It signs in with a device code, finds every coding agent on that machine, and points each at `gman mcp serve`, the same door over stdio, with no redirect anywhere. For a machine rebuilt per task, set GMN_TOKEN to an org service token and pass it as an Authorization header instead. The agent acts as the signed-in customer. Exec runs over MCP: if you can run commands at all, you use run_command; logs are the interface. This page is also served over the MCP connection itself, as the resource gman://docs/llms.txt (resources/read) - the fetch that works from inside any sandbox, whatever its egress policy. The server instructions and the tool descriptions are deliberately short (MCP clients clip long text); this page is the full contract they point at. ## The CLI: gman `gman` is this platform from a terminal. It is what a script, a CI job, or an agent with a shell and no MCP client uses. It installs in one line: curl -fsSL https://autoresearch.sfcompute.com/cli.sh | bash gman login # or: export GMN_TOKEN= gman node create --name tulip --chip h100 --wait gman run tulip -- nvidia-smi gman node stop tulip A command reads `gman [--flags] [arguments]`. `--` ends the flags, and everything after it is the remote command. Most nouns take an `ls` verb that lists and a `get` verb that reads one thing. Where a capability on this page has no verb yet, `gman api -d ` calls the endpoint directly. Read the syntax rather than guessing at it. One page carries every command, every flag and every exit code. The page is generated from the CLI's own argument definitions, so it is exactly what the binary accepts. Three ways to read it: - https://autoresearch.sfcompute.com/cli.txt serves it, with no install and no auth. Nothing links that URL, so this line is where you learn it. - `gman reference` prints the same page, offline. - `gman --help` prints one command's entry. Two facts about the CLI that no tool description carries. `--json` prints the API response body without changing it, and that body is the object the matching tool returns as structured content, so a shell pipeline and an MCP call read the same fields. `--json --quiet` prints the body and nothing else. The exit code answers "did it work": 0 success, 1 refusal, 2 usage, 3 auth, 4 not found, 5 transport. `gman run` is different, because it passes the REMOTE command's exit code through. Transfers use a second hostname. Every presigned URL the platform mints - create_context uploads, get_context / export_file downloads, job artifact / preview / sample links - points at the object store, currently https://fly.storage.tigris.dev, and never at the API host. A sandbox egress allowlist must permit that host as well as the API's, or every tool call succeeds while every transfer 403s at the proxy (the credential is fine; the host is blocked). Share links (share_artifact) are the one exception: they serve from https://s.givemeanode.io with a real filename and content type, made to be pasted in front of humans - Slack unfurls them, renders them. ## What these tools can touch Every customer tool acts inside ONE workspace, on compute the account rents by the minute: its own containers, its own encrypted disks, its own jobs, its own forked microVMs. No end user's data, no other tenant, and no production system is reachable from any of them. A node is a rental with a meter, and the meter runs until somebody ends it. That is worth stating because the teardown tools are named like acts of destruction, and an agent that hesitates over the name gets the expensive outcome: nodes idling, jobs running past their answer, forks billing RAM minutes for work that finished. Ending your own rented compute is the routine close of a piece of work and the cheap direction, and an agent is expected to make the call itself: - stop_node parks the disk with files intact and the next run_command wakes it. Reversible; the only thing it ends is the bill. - kill_command ends one process group inside your own container. The node, its files, and its other commands keep running. - cancel_job stops one of your own batch jobs. The job's record and logs stay readable through get_job for their 30 days. - delete_sandbox frees a fork, and the fork re-forks from its snapshot at any time. delete_sandbox_snapshot frees a baked image and leaves the forks already running alone. - delete_node and delete_snapshot are the two that really are irreversible: they crypto-erase your own disk or your own snapshot, and nothing outside it. snapshot_node before a delete_node keeps whatever you might still want. The guardrails are in the platform, not in your hesitation. A teardown of a node another investigation created refuses unless you pass force: true and the refusal names who created it; deleting a context a live job still needs refuses and names the jobs; removing the last admin of an org refuses; and every write is recorded in the org's audit log (list_audit_events). Where a call is genuinely not yours to make, it is refused and the refusal says why - so the honest default is to make the call. Moving bytes is the same story, told for the transfer verbs. write_file, export_file, copy_file, import_data, export_data, the context uploads (create_context / finalize_context), and snapshot_node move YOUR OWN workspace's bytes between your own node, the platform's artifact store, and storage your org connected. The egress and connection verbs - export_file, copy_file, import_data, export_data, snapshot_node - are each recorded in the org's audit log (what moved and where, never the bytes), and finalize_context records the committed upload (context.finalize); write_file and create_context are deliberately not audited per call (agents write files by the dozen; the volume - or the finalize - is the record). A connection's credential is exercised on the host - it never enters the node or this chat - and nothing becomes public: the one verb that publishes is share_artifact, its description says so, and unshare revokes it. The platform itself never asks a human to approve a transfer: a routine upload or download inside your own workspace is yours to make, exactly like the teardown calls above. The additive transfers (export_file, create_context, finalize_context, snapshot_node) declare destructiveHint: false, so a client that keys per-call consent prompts on the MCP default has what it needs to stop asking every time; the verbs that can overwrite what is already there (write_file replaces a file, copy_file and import_data merge over a tree, export_data can replace an object at the key you name) honestly keep the default, and a client may keep prompting for those. ## Missions The unit your work ships in. A mission is a named investigation ("serve llama-70b under $X") grouping the nodes, jobs, commands, artifacts, and spend that answer one question, with a page a human watches while the agent works. The arc: open_mission(name) once - FIRST, before the first create_node or submit_job - then pass mission: "" on create_node / run_command / submit_job / submit_jobs / export_file. Work and cost roll up on the mission's page; metrics, logs, and traces arrive labeled by mission (query_metrics / query_logs / search_traces filter on it); starter charts draw automatically and save_chart adds the views worth keeping; post_update(title, body) is how you SAY something on that page, and it is what the page leads with; finish_mission(result) declares the verdict. Every mission response carries the page URL (https://autoresearch.sfcompute.com/missions/) - hand it to the human early so they watch the work come together instead of waiting for a report. Missions never cap or gate anything (workspaces are policy; missions are reporting), but ungrouped work is invisible on /missions - prefer a mission. Who sees the page: only the org, never the public internet. It asks for sign-in; members see missions in workspaces they can act in, org admins see all, and the billing role sees every mission's receipt (costs, results) but not its updates. Posting results or job ids with post_update is not public disclosure. The one public thing is a share link, minted by share_artifact or an ![alt](art-...) embed, revocable with unshare. ## Capacity: queue depth starts more machines The single most expensive mistake an agent makes here is reading a busy fleet as a reason to hold work back. It is the opposite. Queued demand is an INPUT to procurement, not a symptom of it: a queued node's GPUs are a term in the fleet's own machine target, counted on the next pass, and a job backlog that persists for a shape raises the same target for it. That number is what makes another machine start. So the work you queue is what makes the fleet grow, and the work you hold back is what keeps it small - and an agent that surveys a full fleet, concludes "no capacity", and quietly shrinks its plan has retracted the only signal that would have answered it. What follows from that: - **Queue it.** A full fleet is a reason to submit, not to wait for a better moment. There is no better moment that does not go through the queue. - **Queued time is free**, for nodes and for jobs alike, and your place is held - to `expires_at` for a node (extendable, ceiling 12h, and a lapse re-queued within 15 minutes keeps the place you waited for), for `queue_ttl_minutes` for a job (default 3 days). - **Do not cancel because the wait is long.** Cancel when you no longer want the result. Cancelling a queued job or deleting a queued node withdraws the demand that was going to start another machine, so the next wait is longer, not shorter. - **Submit the whole shape of the work.** Eight jobs queued is a truer demand signal than one job queued eight times in series, and it is the one the fleet can actually size itself against. - **It is a direction, not a promise.** Spend ceilings, a frozen market, and a treasury whose own target says the fleet already covers what it wants can each mean no machine is coming right now. On a queued NODE, `capacity_inbound` is the field that answers this for your specific entry: true means the wait you are quoted is bounded by a machine being added rather than by other tenants finishing. False is "not right now", and it can turn true on a later poll. What is unconditional is the direction: a queue nobody joins starts nothing. - **The jobs term is SUSTAINED.** It reads a trailing average of the backlog rather than the instant, and stays silent until the backlog has persisted, so expect the fleet to answer a real backlog over an hour - not a single submit over a minute. Pace your expectations accordingly and leave the work queued. Every queued answer carries this as `demand_note`, beside the position and the estimate, because that is where the doubt arrives. Access is scoped. Authorizing the server is a consent checklist (OAuth scopes); the tool list a connection sees is exactly its grant, and every account operation lives in some scope; only the ceremonies (signup, card entry, ToS) need the browser. Each scope comes in two strengths: the bare word grants read AND write, a `:read` suffix grants only the read-only slice (write implies read). A token that names no scopes acts with the customer's full authority (infra + org). The scopes: - infra: your infrastructure. Nodes (lifecycle, exec, files) and batch jobs - org: your organization. Spend and billing, members, invites, roles, caps, referrals - operator: Autoresearch staff only (fleet operations) A refused call names the scope word to request; to add one, the human re-authorizes the server with that word included. Ask for the narrowest scope the task needs (org:read over org when reading is enough) - never more. Tools, by scope. infra (nodes): - list_nodes(): your nodes: name, chip, state - get_node(name): status, rate, queue position; the poll target while a node is queued, provisioning, or waking. A ready node with no session reports "running (idle)" plus idle_stops_at - the moment it auto-stops unless a run_command lands first. A node that is WORKING does not auto-stop: sustained GPU or CPU utilization pushes idle_stops_at forward for as long as the work lasts, so a trainer started under nohup, setsid or tmux keeps running after the command that launched it returned. `utilization.busy` (below) is that verdict. A process that is alive but doing nothing reads as idle, because from outside it is; hold_node keeps such a node awake. A placed node whose host has gone silent carries `host` {responding: false, last_heard_at, silent_seconds, note}: exec tools fail with "host isn't responding" while it lasts (the status label alone stays "running" - it is a state fact, not a reachability fact), most silences are a brief platform deploy, and the note names both endings (host reconnects, or the node is marked lost with billing closed back to the last heartbeat and automatic credits) plus the exit (stop_node ends the meter; a stop of a silent host is replayed on reconnect). No `host` block = the host answers - a monitoring loop should alarm on its presence, not wait for status to change. A stopped node says why it stopped: `stopped` carries reason (idle_grace_expired, wake_hold_lapsed, stop_requested, a cap or policy stop, container_exited, ...), at, a note naming the next move, and - when that session billed - ran_seconds, its billed length. Beside it, wake_estimate quotes what a wake requested right now would face: queued_wakes (the line it would join; wakes are served before creates) plus the measured estimated_ready_seconds and its p90 - weigh it before committing to a wake, or before letting a parked disk stay the only copy of files you will need soon. A reading of the queue as it stands, never a reservation. While placed it also carries `resources` - the container's cgroup budget (cpu_limit cores, memory_limit_bytes). Size parallelism and memory to THOSE numbers: a GPU node's CPU limit is a CFS quota, which nothing inside the container can see, so `os.cpu_count()`, `/proc/cpuinfo`, `psutil` and `free` all report the HOST's ~200 cores / ~1 TiB and autotuning off them overcommits ~8x and gets OOM-killed. The same numbers ride inside the container as $GMN_CPU_LIMIT and $GMN_MEMORY_LIMIT_BYTES, with OMP_NUM_THREADS, RAYON_NUM_THREADS, MAX_JOBS and PYTHON_CPU_COUNT preset to the budget, so the OpenMP/BLAS/torch, Rust (rayon) and torch-extension-build (ninja) pools size to your slice without being told. These are set on the CONTAINER, so they also outrank an `ENV` baked into your own image (run_command's `env` is where you override them; a Dockerfile `ENV` loses). On a catalog image `nproc` answers the budget too, because GNU nproc reads OMP_NUM_THREADS - on your own image check first, since busybox/Alpine nproc reports the host and distroless has none. The one that still reports the host everywhere is `os.cpu_count()` - PYTHON_CPU_COUNT only reaches it on Python 3.13+, and the default image is on 3.12 - so anywhere you would write `os.cpu_count()` (a DataLoader's num_workers, multiprocessing.Pool()), read $GMN_CPU_LIMIT instead. `sched_getaffinity` reports the host too, and that one is the truth: the quota buys burst across the whole machine rather than a slice of it, so it is the right answer to "where may I run" and the wrong one to "how wide a pool". Processes and threads have a ceiling too: /sys/fs/cgroup/pids.max inside the container, 4096 by default - enough for almost everything, but many-engine serving stacks (several inference engines side by side) can cross it, and past it every new thread fails EAGAIN ("Resource temporarily unavailable"). The ceiling is the node_pids_max limit (list_limits shows it; support raises it per org), applied at the node's next start or wake. A live node with recent samples also carries `utilization` - what the host sees this node doing, sampled every minute from the worker's heartbeat. `busy` is the platform's verdict and is load-bearing: while it is true the idle timer will not stop this node, and idle_stops_at keeps moving instead of firing. Beside it, on a GPU node, the busiest GPU's samples (gpu_pct_max, gpu_pct_latest, sampled_minutes/window_minutes, latest_sample_at), and on any node cpu_pct - percent of ONE core, so 300 means three cores busy - with observed_at and the shorter busy_window_minutes the verdict is taken over. This is the cheap answer to "is my run still doing anything": busy false with a max near zero across a fully sampled window is a finished or stalled run that is still being paid for. sampled_minutes short of window_minutes means the rest of the window carries no sample and is UNKNOWN, never idle; an absent field was never sampled, and the whole block is absent when nothing was sampled at all - never read a missing block or field as 0%. query_metrics (gmn_gpu_utilization) has the full per-GPU series. Beside it, `gpu_memory` answers the other half - not "is anything running" but "will another thing FIT": per GPU, used_mib and free_mib from the worker's newest heartbeat (as_of), plus free_mib_min, the tightest card on the node. Size a co-located workload against free_mib_min, never against a node-wide total: two trainers on one box OOM on a single device, not on the sum. The UUIDs are the ones nvidia-smi -L lists inside the node. A card that went unsampled on that beat simply has no numbers - unknown, never zero - and the whole block is absent when the newest reading is stale, so a missing block is never "plenty of room". query_metrics (gmn_gpu_memory_used_bytes) has the full series. get_node also carries `environment`: the image's toolkit (image_nvcc) and the torch the image ships (image_torch; null = none, so pip resolves it from the index), plus - once placed - driver_version, cuda_major_max (the highest CUDA major this driver runs) and compute_capability (the TORCH_CUDA_ARCH_LIST value). Building a CUDA extension (flash-attn, DeepSpeed, apex, xformers) requires torch's CUDA major to EQUAL nvcc's major: check that before a long compile, or let node_doctor check it. Machines retire on a schedule, and once one is scheduled this call carries machine_retires_at with a machine_retirement_note saying what happens to the node at that moment and how to keep your files. It shows up hours ahead, on stopped nodes as well as running ones, so a long unattended run should poll for it - node_doctor(node): preflight a running node from inside its container: torch's CUDA major vs the image's nvcc, the cgroup budget vs the host numbers, virtualenvs that no longer load under this image, and a one-line GPU matmul. Answers {ok, findings, passed, skipped, facts}, each finding carrying a problem and a fix. A check that does not apply is skipped with a reason, never reported as a problem. It changes nothing and never wakes a stopped node (it refuses instead: a preflight should not start billing to answer); on a running node it is an exec, so it re-arms the idle grace - create_node(name?, chip, image?, max_wait?, mission?): never blocks: returns "provisioning" (with the locked rate) or "queued" (with a live position; the scheduler provisions it the moment a slot frees, and the place lapses at the shape's expires_at; a re-call with max_wait extends it forward-only, and a shorter re-ask never cuts a hold (ceiling 12h, so a multi-hour wait is one hold rather than an hourly re-ask); after a lapse, re-call create_node with the same name to re-queue - within 15m of the lapse it keeps the place it already waited for). A queued answer prices BOTH ways the node can arrive - the queue draining, and us adding a machine for it - and quotes whichever is sooner; capacity_inbound is true when the wait is bounded by capacity being added rather than by other tenants finishing, so read it before redesigning a job around a long number (false is "not right now", and it can turn true on a later poll). Either way the queued answer carries demand_note: the entry itself is a term in the fleet's machine target, so holding your place is what starts the machine - see "Capacity: queue depth starts more machines" above. When a scheduler FILTER holds the node rather than the line ahead - disk, a driver floor, scratch, profiling - the queued answer carries blocked_on {resource, needs_bytes, note}: what this node needs, roughly how far short the closest machine is, and the move that changes it. Act on it rather than polling, because waiting does not clear a filter; where the constraint also makes the estimate meaningless it is null with unknown_because naming the same resource, which is an answer and not a long wait. The remedy only ever names moves your own request affords, so a note that does not mention your scratch or a snapshot means you asked for neither and neither is what is holding you. One reason reads differently: resource "capacity" means no machine is eligible for reasons that are ours, not yours, and it KEEPS the estimate - a busy fleet is exactly the wait a queue measures - so there is nothing to act on, your place is held, and the number beside it is real. A queued answer carries both expires_at and estimated_ready_seconds, and when the hold lapses before the estimate says the node could arrive, hold_note says so and names the extend call - a hold that runs out first means the node never arrives at all. When the estimate itself exceeds the 12h ceiling no single hold can cover the wait, and hold_note teaches the renewal chain instead: re-call create_node before each expires_at (a lapse re-queued within 15m keeps the place), or move work that can run unattended to submit_job, whose queue holds a place for 3 days. When the node arrives it idles on its billed grace window (rate.minimum_minutes: currently 15 min on 1x, 20 min on 8x) and then auto-stops, and that clock starts at provision time - so poll get_node at any cadence comfortably inside the window and land one run_command to keep the node; no 30-second loop needed. chip "cpu-2" and "cpu-8" are the CPU-only shapes, each billed as one unit, no GPU ever: cpu-2 (2 reserved physical cores, 8 GiB RAM) for downloads, ingest, and preprocessing whose output you hand onward through org storage - that work is bounded by the origin and the disk, so two cores are the honest size; cpu-8 (8 reserved physical cores on one NUMA node, low inter-core latency INSIDE the node, 32 GiB RAM) for parallel CPU work. A CPU shape promises nothing about proximity to GPU capacity: it may be placed far from your GPU nodes - in a different region - so move data with storage and contexts, not by assuming a LAN; a private network reaches it (mesh = reachability, never latency). get_node/list_nodes report each placed node's region. A volume snapshot DOES cross the CPU/GPU boundary, both directions: a snapshot taken on a CPU shape restores onto a GPU shape and the reverse, so an ingest on cpu-2 can be handed to a GPU node as a snapshot. A snapshot that shows a region is stored in that region, and a node created from it runs there. A snapshot without a region restores anywhere. A node created from a snapshot downloads it before it starts: a snapshot of 128 MiB or more through eight parallel connections, a smaller one through one. While it does, get_node reports the bytes done and the connections in use (lanes) in restore_progress and the measured rate in provisioning.download_bytes_per_second - plan the wait from that rate. In production eight connections moved a 35 GB snapshot at 118 MB/s from central storage. From in-region storage the same eight connections moved it at 390 MB/s. One connection moved 17 to 30 MB/s (September 2026). A snapshot UPLOAD runs through eight parallel connections and moved snapshots of 32 to 35 GB at 107 to 110 MB/s to central storage. To in-region storage it moved a 35 GB snapshot at 157 MB/s. clock_lock and profiling are GPU features and refuse on CPU shapes. image omitted = the catalog default, a bare CUDA toolkit that ships NO torch - PyTorch work should name a pytorch-* entry, whose torch is preinstalled and built for its toolkit (the create answer says so when a GPU node lands on the torchless default, and naming any unknown image is refused with the current list). image also takes YOUR OWN container image as a registry ref - anything with a '/', ':' or '@digest' ("nvcr.io/nvidia/pytorch:24.01-py3", "ubuntu:24.04") - run under the platform's supervision: a tag resolves ONCE at create to a manifest digest and every start runs exactly that build; the platform supplies the entrypoint (your image's own never runs); your persistent disk mounts at the `home` parameter (an absolute path; default the image's WORKDIR, else /root), and ONLY that path survives stops; run_command needs /bin/sh in the image; a private ref pulls through the org registry connection whose scope pin admits it. get_node's custom_image block reports what the image carries (python3, sudo, nvcc) after first start. Images are never scanned - they run in the same hardened isolation as every node. First start on a host pulls the image (the create answer quotes the size); the catalog stays the recommended path - pre-pulled everywhere, CUDA contract CI-gated - hold_node(name, minutes): keep a READY node awake while you work on it across several turns, at a quoted price you see before it starts. This is the answer to "how do I stop my node disappearing between calls" - never a sleep keepalive, which costs the same node time, occupies a command slot, and files a sleep on your mission page as if it were work. A hold defers the two stops that fire because nothing LOOKS like it is happening: the idle timer, and the workspace's idle-burn auto-stop if somebody armed one (GPU utilization cannot see a file transfer, a dependency install, or a dataset download). Everything else still stops a held node - a spend cap, an unpaid invoice, an org admin, a host retirement. Ceilinged by node_hold_ceiling_hours (list_limits), and a running detached command defers the same two stops on its own - release_node(name): end a hold early, back to the normal idle timer; you stop paying for the held window you did not use - stop_node(name): stop paying now; the disk parks with files intact, and the next run_command wakes it. The answer carries wake_estimate - what a wake requested right now would be quoted - so the cost of getting the node back is visible as the disk parks, not first at the wake. Billing ends at once; the disk parks after the node saves its last writes to durable storage, minutes after a large write. Until then status reads "stopping" (the stop_node answer too, past a 20 s wait). snapshot_node needs the parked disk, so poll get_node for "stopped" before you call it; run_command needs no wait and wakes the node at once - delete_node(name): destroy the disk (crypto-erase, irreversible); stopped/queued/lost nodes only; stop_node first - run_command(node, command, env?, timeout?, detach?, mission?, max_wait?): run a shell command in the node's container; sync returns {exit_code, stdout, stderr, truncated}; detach:true returns {command_id, log_path} immediately for long-running work; secrets go in env, never the command string (command lines are recorded); mission attributes the work and the node time it opens. An env HOME reaches your command but not the command's markers: the log, the exit marker and a declared result stay under the image's home (~/.givemeanode/commands), where get_command and the result harvest read them. Commands run as the dev user with passwordless sudo, so `sudo apt-get update && sudo apt-get install -y ` is the supported way to add distro packages - the update matters because images ship with empty package lists, the -y because commands run with no stdin (a prompt aborts), and plain apt-get fails with a dpkg lock error asking whether you are root (the fix is the sudo prefix). Packages install outside /home/dev, so a stop resets them: re-run the install after a wake, and keep anything that must survive under /home/dev. On a stopped node it wakes the node, and max_wait sizes the queue hold that wake claims (default 1h, ceiling 12h) - size it to the queued answer's estimated_ready_seconds, since a hold that lapses first means the wake never lands. A sync run is budgeted by `timeout`, which defaults to 60 seconds: the ceiling is 300 on the HTTP API door, and 60 over MCP, where your client's own per-call deadline (commonly 60s) lands first and nothing streams back mid-call, so a longer budget is refused at call time rather than lost in your client. Anything that needs more than the ceiling belongs in detach:true, whose command runs unbounded. Omit timeout when you detach: on a detached launch it budgets only the wake of a stopped node, so an over-ceiling value is refused there too. A sync command killed at its timeout reports the output captured before the kill, and that tail is only as good as the program's flushing: stdout is block-buffered when not on a tty (Python holds ~8 KiB back), so a script that prints progress but runs past the deadline can die with an EMPTY tail - run it with python -u / PYTHONUNBUFFERED=1 or flush=True, and put anything that outlives the budget in detach:true, whose log survives - get_command(node, command_id, offset?): status + incremental output of a detached command; never wakes a stopped node. `oom_kills` plus a note means the kernel's OOM killer fired during the run: a bare "Killed" in the log with exit 137 is the container's memory ceiling, not a bug in your script - reduce parallelism/batch to the get_node resources budget - list_commands(node): a node's detached commands, newest first - kill_command(node, command_id): TERM the command's process group, KILL after 10s; idempotent - write_file(node, path, content, encoding?, executable?): write a small file onto a node (configs, entry scripts) without a command line; not for secrets (run_command's env) or datasets (pull from inside the node, or ride create_context/get_context for sandbox-only bytes). Answers {path, bytes_written, sha256, sha256_verified}: the file is hashed on the node and must match before it lands, so hash your copy locally and compare one string to know a base64 payload arrived intact (sha256_verified:false means the image has no sha256sum and the check was length-only). On a STOPPED node it wakes the node and the write does NOT happen: you get the wake with written:false, nothing is stored server-side waiting to land, and you re-send the content once the node is running - to push several files, wake, then hold_node, then write. From a shell this is `gman cp ./local.py :~/local.py` - export_file(node, path, mission?): export a file or directory to a download URL (on the object store hostname above); the way to get results out. The URL is private and machine-shaped; share_artifact is the human-shaped follow-up. From a shell, `gman cp :~/out/x ./` does the export and the fetch in one step (a directory arrives as a tar) - get_artifact(artifact_id): the record and a fresh download URL for an earlier export (an art-... id from export_file, get_job, or a mission page); exported bytes are kept about 30 days - share_artifact(artifact_id, expires_after?): make ONE exported artifact public at a clean, embeddable URL (https://s.givemeanode.io/shr-.../cat1.jpg - real filename, real content type, no query string). Paste it in Slack and an image unfurls inline; works; browsers render rather than download. The bytes are frozen at share time, so the link outlives the export's 30-day object; it lives 365 days by default (expires_after: "48h".."365d" chooses; re-sharing refreshes the same URL) and unshare revokes it any time. ANYONE holding the URL can fetch the file - sharing is a deliberate, audited decision, never a default. Images, PDFs, video, audio, and plain text render inline; HTML and SVG never do (they download); directories share as a .tar - unshare(share_id): revoke a share link - the URL stops answering - list_shares(): everything your org currently has public, with serve counts (org-visible on purpose; any member can unshare) - copy_file(src_node, src_path, dst_node, dst_path, mission?): copy a file or directory between two RUNNING nodes of your workspace in one call (no shared network needed) - a file lands at dst_path, a directory's contents land extracted under it, merged per file; poll get_import(dst_node, import_id) until complete infra (asynchronous jobs): - submit_job(chip, idempotency_key, mission?, ...): an asynchronous GPU job: builds your image, queues fairly, runs your command, bills only the running attempt. The container's env carries its cgroup budget ($GMN_CPU_LIMIT, $GMN_MEMORY_LIMIT_BYTES) with OMP_NUM_THREADS, RAYON_NUM_THREADS, MAX_JOBS and PYTHON_CPU_COUNT preset to the CPU budget, so the OpenMP/BLAS/torch, Rust (polars, tokenizers) and torch-extension-build (flash-attn, vLLM, DeepSpeed - all read MAX_JOBS) pools size to your slice instead of the host's full core count (a value in `env` wins - a latency-bound phase pays dearly for a pool sized to the host). Three limits on that. They are the RUN container's env and do NOT reach your Dockerfile's RUN steps, so a source build in the image (flash-attn, vLLM, DeepSpeed) needs its own `ENV MAX_JOBS=...` line. `nproc` answers the budget only where GNU coreutils supplies it - your image is yours, and busybox/Alpine nproc reports the host. And `os.cpu_count()` reports the host everywhere: size Python pools off $GMN_CPU_LIMIT. To land the output capture in your OWN storage instead of a download link, pass output_to: {connection, dest} - a write-enabled s3-keys or storage/ connection and a prefix (e.g. "s3://acme-ml/synth/"); the capture lands at -.tar with no credential ever reaching your container. Declare what the run is doing WHILE it runs: one curl to POST $GMN_METADATA_URL/v1/preview {path} captures the job's one replaceable peek at a file, and POST /v1/sample {path, kind, label?} appends to a series that accumulates (a metric image or a json row per step). Both are free, and both are what turns "I polled, and later learned the job was broken" into seeing the pattern at minute 8 - a first job should declare one of them. get_job carries their state; the timeline itself is list_samples. To gate this job on an earlier one, pass depends_on: {job, require?}. The job then does not build, does not queue and never places until that job SUCCEEDS and `require` holds over the JSON it declared at $GMN_RESULT_PATH, so a cheap probe in front of an expensive arm costs nothing when the probe says no. `require` is one comparison, not an expression language: `result` plus an optional dotted path, then == != < <= > or >=, then a literal - e.g. "result.ok == true" or "result.metrics.peak_gib < 72". If the gate closes (the parent failed, declared no result, or the predicate is false) this job ends `canceled` carrying the reason, having built nothing and run nothing; get_job.depends_on says which side it is on while it waits, and its own queue_ttl_minutes still bounds the wait. The parent must be built from the SAME context as this job unless you pass depends_on.allow_different_context: true - a probe that validated other code is the mistake the gate exists to catch. shared.depends_on on submit_jobs gates a whole sweep the same way. A context build that must wait for a build slot (your org at its concurrent-build limit, or every slot in the fleet busy) says so in the submit response's build_note, and get_job repeats it while the wait lasts; identical contexts share one build, and a prebuilt image skips the build. Checkpoints: if $GMN_CHECKPOINT_DIR is non-empty at start, resume from it; checkpoint by writing there and POSTing /v1/checkpoint on the metadata service - an interrupted attempt then bills only to its last capture and the rerun restores your slot. Captures have a rate floor: one accepted POST per job per 300s (checkpoint_min_interval_secs on list_limits) - pace your checkpoint cadence to it. A faster declare answers 429 with Retry-After naming the seconds until the floor opens (also retry_after_secs in the body): back off and redeclare - nothing failed, and the last committed slot stands. A declare while a capture is in flight coalesces onto it. The preview, sample, and yield doors carry their own floors (their list_limits rows) and the same Retry-After on their 429s. resume: "checkpoint" also unlocks placement in shorter capacity windows (a window's end becomes a planned stop-and-resume, never a failure). Once one job has checkpointed in your org, later jobs with the same image and the same command inherit resume: "checkpoint" (get_job shows resume_source: "inferred" and the job it learned from); pass resume: "none" to run one job as a single attempt. A job that has checkpointed may POST /v1/yield to end its own attempt and resume from the slot - the supported way to qualify restoration; the yielded attempt bills its full span (a restore, never a discount) and the rerun re-queues like any requeue, possibly on a different machine CHECKS ARE REQUIRED HERE. Declare what the job must do, and the platform applies it: checks: {success: {check: {type: file_exists, path: /output/rows.jsonl}}} runs once when the job stops, and the job reaches `succeeded` only if it passes (a failure lands `check_failed`, a verifier that could not RUN lands `check_errored`). checks: {health: {check: {type: heartbeat, path: /output/heartbeat}, interval_s: 600, on_fail: kill}} runs while the job is live, and on_fail (required) is record, kill, or reschedule. The types: file_exists, file_grows, file_size, jsonl_field, jsonl_rate, heartbeat, gpu_util, gemm_sane, http, tensor_changed, exec - combined with all_of / any_of. The platform evaluates them from OUTSIDE your container, so you install nothing and a check works when your job's environment is the broken part; use exec only where no built-in states your assertion (a command that can only pass is refused, and a failed exec check records the tail of what your command printed - your own output, so an org with command_output off gets the verdict without it). Before a kill or a reschedule the platform captures your checkpoint slot, so declare resume: "checkpoint" and write $GMN_CHECKPOINT_DIR to make a check cost a resume instead of the run. A health kill lands `self_terminated`, which is its own terminal state - a check that fires is the contract working - submit_jobs(shared, variants, label, idempotency_key): a sweep - up to 256 variants of one job in one call; vary env/command for one build + N runs; all-or-nothing validation; shared.mission attributes every variant; shared.checks (required on MCP) covers every variant - validate_job(chip, command?, env?, max_duration_minutes?, checks?, model?): run PREFLIGHT and validate a checks block without submitting anything. Free, ~2s, changes nothing: use it in a loop while you get a spec right. Preflight also runs on every submit and has two checks - the job's worst-case cost against your REMAINING spend cap (a BLOCK, with the arithmetic; force: true overrides for one submission), and the node's disk against the size of the model the job downloads (a WARNING; declare model: "owner/name@revision" to make it exact). Both stay SILENT when the data is not sufficient, and no_preflight: true skips them - get_job(job_id, stream?, offset?, tail?, attempt?): job status (attempts, preempted_count, failure reason and exit code once terminal) + build/run logs; the poll target after submit_job. The queue estimate is recomputed on every scheduler pass - seconds apart - so an unchanged estimated_wait_seconds across polls means the inputs (queue ahead, drain rate) haven't changed, not that the estimate went stale; never cancel-and-resubmit over a steady number, since the resubmit forfeits the queue place the job was holding - and the backlog the job was contributing to the fleet's procurement target, which is what starts the machines that drain the queue (queue.demand_note, and "Capacity: queue depth starts more machines" above). A run-log slice carries attempt_state (not_started | live | complete | never_ran) and eof, so a poll loop knows whether to read again instead of guessing from next_offset; not_started means no run log YET (still building/ queued/starting - keep polling the job), never_ran means none ever; an attempt that never ran has no log and never resolves to another attempt's bytes. Mid-run, get_job carries the telemetry the tool description only points at, field by field: - build ({phase, bytes_downloaded, bytes_total, layers_done, layers_total, started_at, updated_at}) is the build's live progress. An import's phase walks downloading -> extracting -> pushing; a Dockerfile build's walks fetching -> validating -> extracting -> starting (the build envelope coming up) -> building (its steps) -> pushing. Past fetching a context build's byte integrals hold still by design - those legs move no countable bytes - so read the PHASE for where it is, updated_at only for whether the worker is still supervising it (every leg that can run for minutes heartbeats it every ~10s), and stream: "build" for what the build itself is doing: a Dockerfile step that hangs, hangs in the log. While starting, startup is the same shape for the run's own image pull (a cache-hit job never builds, so this is its one long stage; resolving -> downloading -> extracting -> creating); absent fields mean no signal yet, never stalled - checkpoint_capture, after a POST /v1/checkpoint, is that capture's telemetry ({phase: spooling | uploading | committing | committed | failed, in_flight, bytes_done, bytes_total (an estimate while spooling, exact while uploading), started_at, updated_at, elapsed_seconds, billed, error?}). A multi-GB slot legitimately takes minutes; updated_at advancing means it is moving, and rate/ETA are bytes_done over elapsed_seconds. billed: true is the honest cost answer: capture runs BESIDE your workload on the machine holding the bytes, so the attempt's clock keeps running - overlap it with training rather than idling, and POST /v1/yield once checkpointed_at advances to end the billed attempt on that boundary. phase: failed with error means the slot did NOT advance: stop waiting on checkpointed_at and declare another capture - preview, after a POST /v1/preview {path} (inside $GMN_OUTPUT_DIR), is the job's one mid-run peek: {captured_at, attempt, size_bytes, download: {url, expires_at}, error?} - a tar of the declared path, REPLACED on each capture, fetched with plain curl (the link is minutes-scoped, every poll mints a fresh one, the download lives ~30 days after the capture). attempt is the run that took the peek - after a preemption or yield it may predate the current attempt, so compare it before judging the current run by it. Declaring the SAME path while a capture runs joins it; a DIFFERENT path is refused naming the one in flight - the path is the request, never silently swapped. A preview is a peek (a sample grid, a metrics file), not a deliverable: look at it and decide - keep going, resubmit, or cancel_job now and pay only for the minutes that taught you something; attempt-end capture and output_to remain the real output path. preview.error names the last capture failure; the previous peek still stands and still downloads - host_gpu_health, while an attempt is live, if and only if something is wrong with the machine under it: {status: failing | degraded, detail, hint, since?}. Absent is the normal case, not silence. Every GPU host proves its idle devices against a sustained answer-checked bf16 GEMM on a cadence, so a device that answers but cannot compute is caught and stops taking work; an attempt cut short by one is a PROVIDER fault - severed, never billed, drawn from the provider-fault restart budget, re-placed on a healthy host. Read this before blaming your own code for a CUDA or cuBLAS failure - receipt, on a TERMINAL job: what ran and what it cost, in one place. image_digest is the manifest digest we resolved and pinned for the run - the platform-owned half of the identity, so a result can be tied to the exact bytes that produced it. billed_seconds, charged_usd_micros, billed_attempts, and attempts[] (each with gpu_count, billed_seconds and the effective rate_usd_micros_per_gpu_minute) are the billing journal's own rows, the ones your invoice totals to - never a re-derivation. Billed seconds are PER GPU: charge = seconds x gpu_count x rate / 60. An attempt WE ended (a preemption, a host we cut) bills only the progress that survived: nothing at all unless a checkpoint capture or a task report landed inside that attempt's own window, and then only up to that mark. An attempt with no row here was free, so a shorter list is the free attempts, not a gap - samples, after POST /v1/sample {path, kind, label?} declares a series, carries the series' state: {count, latest_captured_at, error?} - the timeline itself is list_samples, and samples.error names why the last capture landed nothing (a spent max_samples_per_job budget, an oversize file); earlier samples stand untouched - list_jobs(limit?, label?, summary?): your jobs, newest first; label filters one sweep, summary:true is the rollup with every terminal job's declared result - submit_job(..., tasks, tasks_open?, task_lease_minutes?, task_max_attempts?, task_max_runners?): a TASK SET - one job carrying a list of units instead of doing one thing. The command runs once per machine and leases units from inside the container with POST $GMN_METADATA_URL/v1/tasks/lease (no credential; identity is placement), so one container start and one image pull cover thousands of units and losing a machine costs the units in flight rather than the whole run. Upload each unit's result to the presigned output_put_url the lease hands you and report done only once that PUT returns: a unit can run twice, so a shared output path is the one way to get a wrong answer here. The lease answers 204 when the set is drained (exit) and 200-with-empty when the pool is momentarily dry but open (wait). The platform runs the set on several machines as the backlog warrants, up to task_max_runners (default 4, max 32), and stops them as it drains; get_job reports total/done/failed/leased/pending plus the live runner count - add_tasks(job_id, tasks) / seal_tasks(job_id): append to a set submitted with tasks_open: true, then close it; the job finishes once a sealed pool drains - cancel_job(job_id): immediate; a queued or building job stops for free. Cancelling a task set's job stops every machine working the set - create_context(sha256, size_bytes) / finalize_context(context_id): presigned upload for large bytes (a build context, or a dataset that exists only in your sandbox) over the 4 MiB inline limit, up to 10 GiB; then submit_job with context_id, or get_context for a download URL; the upload PUTs at the object store hostname above - get_context(context_id): status plus a presigned download URL for a finalized upload; landing it on a node is one run_command curl infra (sandboxes - fork-per-sample execution for RL): - create_sandbox_env(image?, from_image?, size?, ram_gib?, warm?): bake ONCE. Boots a small isolated microVM from a curated image ("sbx-base", the default: python3, node 24, git, curl, a compiler; "sbx-min": bash + coreutils; "sbx-task": python3 + numpy), runs your `warm` setup, and snapshots the fully initialized machine - memory, processes, page cache, disk. Returns a snapshot id to fork forever. 2 GiB RAM by default; a workspace starts with an 8 GiB ceiling and support raises `sandbox_ram_gib` as far as the 64 GiB platform cap, and list_limits reports your own. from_image takes a container image of YOUR own instead of a curated name - an RL task image, a framework's official image, a SWE-bench instance - so an environment that already exists as a Docker image is imported rather than re-expressed as a warm script. Digest-pinned ("ghcr.io/acme/task@sha256:<64 hex>"), because a tag can be moved to point at different bytes and the conversion is cached under the image's identity; `docker buildx imagetools inspect` or `crane digest` prints it. linux/amd64. Converted on first use (minutes for a large image) and cached after that, so a second bake of the same digest skips the pull. A private registry needs an org registry connection covering the ref; a public one needs nothing. The image's Env and WorkingDir are honoured; its Entrypoint and User are not, because commands run as root through your own shell - so put anything the entrypoint would have started into `warm`. sandbox_converted_images_per_hour bounds DISTINCT images per workspace (50 by default); re-using one you already converted is free and uncounted - SIZES (size:): the default sandbox is ONE vCPU, which is the right shape for an agent taking one tool call at a time and the wrong one for a parallel build. `size` names a bigger machine instead: "sandbox-sm" (1 vCPU / 2 GiB, the default, bills as 2 GiB), "sandbox-md" (4 vCPU / 8 GiB, bills as 16), "sandbox-lg" (8 vCPU / 32 GiB, bills as 32), "sandbox-xl" (16 vCPU / 64 GiB, bills as 64). Each size also raises the NETWORK ceiling - 1 Gbps at the default, then 2, 5 and 10 - which is a cap and not a reservation: sandboxes share their host's uplink and the cap is what stops one taking it. It matters most for a cold dependency install, which is usually wire-bound rather than CPU-bound. On "sandbox-lg" and "sandbox-xl" /tmp is MEMORY rather than disk - half the shape's RAM, so 16 GiB and 32 GiB, at least what the disk gave you - which makes scratch writes and especially deleting them much faster. The trade: /tmp competes with your process for that memory, so a job filling both can be OOM-killed where it used to get ENOSPC; write tens of GiB of scratch under your working directory instead and it stays on disk. "sandbox-sm" and "sandbox-md" keep a disk-backed /tmp and are unchanged. `package_cache: true` on create_sandbox or fork_sandbox serves that install from an npm PULL-THROUGH CACHE on the machine your sandbox runs on, shared by everything on that host, so a package pulled recently by anyone arrives without crossing the internet - it is still a real cold download of every package your lockfile names, nothing is pre-seeded. OFF BY DEFAULT, AND IT REWRITES YOUR LOCKFILE: the cache serves tarballs from a loopback address and bun records THAT address for every package, so a bun.lock produced under it differs from the one you committed on every dependency and any --frozen-lockfile or `git diff` check on it afterwards FAILS. Ask for it on throwaway installs where only speed matters; leave it off for anything whose lockfile you keep. A sandbox carrying its own `.npmrc` is left alone and resolves exactly where that file says (scoped registries always win too); if the cache is unavailable the sandbox resolves registry.npmjs.org directly, which is what it would have done anyway. A core comes with 4 GiB, so above the default what you are billed and what you get are the same number and the memory carries no premium; a workload wanting cores and little memory still pays for the cores, because cores are what it takes off the machine. `size` and `ram_gib` are alternatives, not both; your ceiling is the `sandbox_vcpus` limit, which starts at 4 - "sandbox-md" needs no permission, and "sandbox-lg"/"sandbox-xl" are a raise away. The size is fixed AT THE BAKE and every fork inherits it, so bake the shape you mean to fork. The cores are weighted to you rather than reserved outright, so a busy host can still slow a large sandbox down - the trade that keeps unpark unrefusable; while it is parked you pay for no cores at all - fork_sandbox(from, count?): clone 1..=256 running sandboxes from a snapshot, each resuming its exact state in a few tens of milliseconds. Forks are FREE (copy-on-write); you pay each fork's own RAM-minutes while it lives. N samples = 1 bake + 1 fork call, never N bakes - SANDBOX IDS ARE OPAQUE: pass back the exact string you were given. Neither length nor shape is contract - they have been 16 characters and now run to about 80 - so size any column you store them in for growth, and never parse one - sandbox_exec(execs: [{sandbox, cmd, deadline_ms?}] | sandbox + cmd): run shell commands across many sandboxes in ONE call - the rollout shape. Results in request order with exit_code, stdout, stderr, duration_ms, timed_out (1 MiB caps per stream; 60s default deadline, 600s max). A result carrying outcome "void" is OUR infrastructure failure, excluded from billable time in the hour it is recorded - retry it on a fresh fork rather than scoring it. A result carrying cancelled: true means your own delete_sandbox won a race against the exec: the verdict is unknowable, the time is billed, nothing to retry. oom_kills > 0 (with a note) means the guest kernel's OOM killer fired during the run: a bare "Killed" with exit 137 is the sandbox's memory budget biting, billed, and it will die the same way on retry - bake the env with more ram_gib or use less memory, and score it as a real failure, never as a void - snapshot_sandbox(sandbox): capture a RUNNING sandbox's current state as a new forkable snapshot, without stopping it - the branch point for exploring K continuations from one identical state - create_sandbox(image?, size?, ram_gib?, setup?): boot ONE fresh sandbox and hand it over running. With no setup, the first call for an image boots it and later calls for the same image and size fork the snapshot the first one left, in tens of milliseconds (`reused_snapshot` says which happened). With a setup, every call boots and pays the setup again. Either way N sandboxes want create_sandbox_env + fork_sandbox(count: N), which is one call rather than N. The response carries this sandbox's `snapshot` plus a `prefix_note` when your workspace just booted the same recipe - except when it says `shared_base: true`, meaning it forked a base we keep warm, or `pooled: true`, meaning Autoresearch had already started this sandbox and handed it over (no boot, no fork, billed from the handover): those bases are ours, so there is no `snapshot` to fork and another create_sandbox is just as fast - delete_sandbox(sandboxes: [...] | sandbox) / delete_sandbox_snapshot( snapshot): end the RAM-minute meter, end the stored-bytes meter. Sandboxes bill while they EXIST, so delete each fork as its work is scored - and delete them ARRAY-SHAPED, one call for the batch, the same way you forked and exec'd them. Results in request order, one per entry, each carrying deleted: true|false. A FALSE entry is still running and still billing (not your id, or the host refused the teardown) and the call still succeeds, so check the entries: a `deleted` count below what you sent means read them - get_sandbox_stats(): live sandboxes, snapshots, and loop-shape counters (bakes, boots, forks, execs, voids, oom_execs, deletes) - the audit that says whether the loop forks or re-bakes, and whether the samples keep hitting their memory budget - A sandbox CAN reach the internet: git clone, pip install, DNS and outbound HTTPS all work, up to 1 Gbps per sandbox. It cannot reach anything private - not your nodes, not your networks, not this API, not another sandbox - so code your model wrote can fetch what it needs without reaching your infrastructure. Loopback works too: a server one command starts is reachable at 127.0.0.1 from the next command in the same sandbox. Baking still beats downloading for anything every sample needs: one bake pays for it once, N forks inherit it. A model or a dataset per sample belongs on a node or in a job, with the sandbox grading what comes back - expose_sandbox_port(sandbox, port, auth?) / unexpose_sandbox_port( sandbox, port): a PUBLIC HTTPS URL for a server running inside a sandbox - the preview link for a dev server, so a page your code just wrote can be looked at (or curled, or driven by a browser) from outside. Ports 1024-65535, and the server must be listening on the sandbox's OWN 127.0.0.1, which is what a dev server binds by default. THE SERVER HAS TO SURVIVE THE EXEC THAT STARTED IT: sandbox_exec waits for its command's output to end, so `npm run dev &` alone hangs the exec until its deadline and then dies with it - redirect, `(npm run dev > /tmp/dev.log 2>&1 &)`, and the exec returns at once. Exposing a port nothing listens on succeeds and the URL then answers 502 naming the port, so start the server first. The URL IS the secret (an unguessable hostname): treat it like a password, or pass auth: "bearer" for a token on top, returned ONCE in that response. Idempotent per (sandbox, port) - the same call returns the same URL with a refreshed expiry, so ask whenever you need it rather than caching. It dies with the sandbox and a FORK DOES NOT INHERIT IT (a fork is a new sandbox: expose its own port); an inbound request wakes a parked sandbox, so traffic puts it back on the active RAM tier. Streaming, SSE and WebSocket all pass through. Works on an egress: "none" sandbox too, because nothing inside has to reach out for the URL to work. list_endpoints(sandbox: "sbx-...") lists them - For a HERMETIC sandbox, pass egress: "none" to create_sandbox_env / create_sandbox: no network device at all, so nothing inside can reach anything. Everything it needs must be baked in. The choice is made at bake time and every fork inherits it - fork_sandbox cannot change it, so bake a second env for the other posture - From a terminal or CI the same capabilities are `gman sandbox` verbs: `bake` (--image, --ram-gib, --warm or --warm-file), `create`, `fork --count N`, `exec ... -- ` (or `exec --batch file.jsonl` for a command per sandbox), `snapshot`, `rm ...`, `rm-snapshot ...`, `expose `, `unexpose `, and `ls`. One exec passes the remote exit code through; a fan-out exits 0 only when every result exited 0, and `rm` exits non-zero if any fork came back undeleted infra (missions, and the telemetry labeled by them): - open_mission(name?, title?): create or attach the named mission; the response carries the page URL and the attach contract - get_mission(name): the receipt, one call - cost so far, member nodes / jobs / artifacts, charts, the declared result - list_missions(workspace?, active_only?, limit?): newest first - finish_mission(name, result?): declare the verdict (inline JSON); it renders on the page and returns verbatim on get_mission - post_update(name, title, body?, kind?): a titled markdown note on the mission's front page - post when something a human would want to know changed (a sweep started, an arm ruled out, the answer found, you are stuck), never on a timer. Write ![what it shows](art-...) in the body to put an exported image, video, or audio clip inline: the artifact id becomes a public link, so the same text renders in Slack too. A marker inside a code fence stays literal and publishes nothing. kind: "alert" only when a person needs to act. Append-only: a correction is a new update - list_updates(name, limit?, before?, at?): the stream back, newest first - the cheap way for a context-fresh session to re-orient on the PLOT rather than the numbers (get_mission says what it cost, this says what was tried, ruled out, and found) - query_metrics(query, mission?, run?, ...) / query_logs(...) / search_traces(...) / get_trace(trace_id): PromQL / LogQL / TraceQL over your org's telemetry, mission- and run-scoped; GPU, system, and spend series arrive with zero instrumentation - register_scrape(node, port, path?): the platform scrapes your own Prometheus endpoint (a trainer's reward series, vLLM's /metrics) into the same queryable store - one call replaces the scrape-loop scaffolding - save_chart(mission, name, spec, default?) / delete_chart(mission, name): agent-designed charts on the mission's page infra (limits): - list_limits(org?): every limit's effective value for you in this org - the deciding layer (default, or an org/customer override), whether support can set it per-org (settable), and what it bounds; the numbers the teaching errors quote, before the wall instead of at it org (money, read-only; with list_team and list_referrals, this is the org:read slice): - get_usage(month?): spend in USD ("YYYY-MM", default current) - per node, per batch (jobs by sweep label, with the priciest named), and the builds/storage lines, summing to the month's total. That total is what has BILLED; a node running right now has accrued minutes the meter has not journaled yet, so the current month also carries `in_progress_usd` and `spend_so_far_usd` - the latter is the figure get_billing quotes and spend caps enforce on - get_billing(org?): dunning state, MTD spend (includes usage in progress), caps; admin/billing roles also see payment status, credits, per-member spend, the prepaid balance with its expiry, and cash received org (administration; role-gated exactly like /team): - list_team(org?): memberships (with the ACTIVE org), roster, live invites, org nodes; looking also accepts invites addressed to you - list_audit_events(events?, actor?, since?, after?, org?): the org's audit log as WorkOS-shaped events - every decision, who, from where, when; admin/billing see the whole org, members their own decisions. data.actor.type answers "my key or the platform?": customer/ service_token is one of your people, system is us ending a node nobody asked us to end (grace lapsed, the container died, the host went away, a queued wake's hold expired). There is no wake tool - run_command, write_file and node_doctor wake a stopped node as a side effect, and node.wake's details.trigger names which one - invite_member(email, role?, org?) / revoke_invite(invite_id, org?) - set_member_role(member, role, org?) / set_workspace_cap(workspace, cap_usd?, org?) - remove_member(member, org?) - switch_active_org(org): where your next node bills - stop_org_node(node_id, org?): the runaway-student-node stop - list_referrals(org?) / claim_referral_code(code, org?) operator (Autoresearch staff: a verified @sfcompute.com login, or a gmn_ admin token): the `givemeanode admin` surface as tools. Status, workers, nodes, jobs, builds, market, pins, doctor, audit search; mutations only when the server has writes enabled AND the identity is write-scoped. ## HTTP API (PREVIEW) The same capabilities, REST-shaped, for CLIs and scripts - the second door. PREVIEW: the contract may move until launch, when the routes are promoted verbatim to /v1 (GMAN-31); the OpenAPI document is the source of truth, at either of these (the short one is the conventional path, the long one is the door's own): https://autoresearch.sfcompute.com/openapi.json https://autoresearch.sfcompute.com/preview/openapi.json Same bearer tokens as MCP, same scopes, same wording in refusals. Every operation carries a unique operationId, a description, typed parameters and a typed request body generated from the same schema the MCP tool takes, and a typed response - so a function-calling client can build a call from the document without guessing. Each also names its capability in x-capability; path and query parameters overlay the JSON body. List endpoints page with ?cursor=&limit= returning {items, next_cursor?}. Logs are offset-polled (?offset=), never streamed. Workspace binding is per request (?workspace=), never ambient. Example: curl -s -H "Authorization: Bearer $TOKEN" https://autoresearch.sfcompute.com/preview/nodes The sandbox plane rides this door too, one route per tool (POST /preview/sandboxes/envs, /forks, /execs, /deletes, /{id}/snapshot, POST /preview/sandboxes to boot one, DELETE /preview/sandboxes/{id} and /preview/sandboxes/snapshots/{id}, POST /preview/sandboxes/snapshots/{id}/expiry, GET /preview/sandboxes for stats), on its own request budget separate from the rest of the API - which is how a job or a trainer drives sandboxes without an MCP session. COVERAGE: every MCP tool listed above has a route here, checked on every build, so a capability you found over MCP is a capability you can script. The single exception is switch_active_org, which has nothing to switch on a door that carries the org and the workspace per request. Volume snapshots are POST /preview/nodes//snapshots, GET /preview/snapshots, PATCH and DELETE /preview/snapshots/; holds are POST and DELETE /preview/nodes//hold; the observability reads are POST /preview/logs/queries, /preview/metrics/queries and /preview/traces/queries, GET /preview/traces/ and /preview/samples. This door is how a plain shell polls instead of an agent: a queued node's wait, and the grace window once it arrives, are a curl loop away - curl -s -H "Authorization: Bearer $TOKEN" https://autoresearch.sfcompute.com/preview/nodes/tulip answers with the same status/idle_stops_at as get_node (the doors have separate rate limiters, so a script's loop never spends the agent's tool budget), and `gman node create --chip h100 --name tulip --wait` is that loop prepackaged: it blocks until the node is ready and exits 0. ### Errors, versions, and rate limits **Errors** are one shape, on every 4xx and 5xx: {"error": {"code": "refused", "message": "..."}} `code` is machine-readable and the vocabulary is OPEN - treat a code you do not know as a generic failure of its HTTP status, never as a parse error. Today's codes: unauthenticated (401), forbidden (403, and the message names the scope word to request), invalid (400), not_found (404), refused (422, a well-formed request the platform declined - the message says what to do instead), rate_limited (429), internal (500), settling (503, the node changed state under every pass the call made - nothing ran, nothing billed, and the same call works shortly; `Retry-After` names the wait). `message` is the same teaching text the MCP door serves and is safe to show a human verbatim. Every response also carries a `request-id` header; quote it in a ticket and the logs line up. The typed schema is `components/schemas/Error` in the OpenAPI document. **Versions** live in the path. `/preview` is the pre-launch contract; at launch these routes are promoted verbatim to `/v1` and `/preview` becomes an alias. Once a path is versioned, breaking changes take a new version segment - an existing one only gains optional fields and optional parameters. A route on its way out carries RFC 8594 headers for at least 30 days before it stops answering: Deprecation: true Sunset: Sat, 29 Aug 2026 00:00:00 GMT Check for those two on every response if you are writing something that must keep working unattended. The machine-readable policy is `x-versioning` in the OpenAPI document. **Rate limits** ride every response, not just the refusal, so you can pace yourself instead of discovering the wall: RateLimit-Limit: 60 RateLimit-Remaining: 41 RateLimit-Reset: 19 RateLimit-Policy: 60;w=60 A 429 adds `Retry-After` in seconds; wait that long, do not spin. There are two budgets and they are separate on purpose: the general door (`mcp_calls_per_minute`, a 60-second bucket) and the sandbox door (`sandbox_api_rps`, a one-second window), so a training loop driving sandboxes never spends the agent's tool calls. GET https://autoresearch.sfcompute.com/preview/limits reports your own ceilings. Batch where the API offers it - one array-shaped sandbox_exec beats one request per sandbox. A dropped MCP connection (every tool vanishing mid-session, then returning) is a fact about the connection, never about your infrastructure: nodes, queue holds, grace clocks, and detached commands live server-side and carry on while a client is offline. On reconnect, read before acting - get_node shows where the node landed, and list_commands shows whether a DETACHED command that never answered actually started - so retry the read rather than re-create the node. A synchronous run_command that never answered leaves no command row and its process may still be running: treat it as unknown and check its effects before re-running (one more reason to detach anything you cannot afford to lose track of). A wait that must survive a client blip (a queued node's grace window, say) belongs in a shell loop on this door, which does not ride the agent's MCP session. No token yet? The gman CLI mints one: install (`curl -fsSL https://autoresearch.sfcompute.com/cli.sh | bash`), then `gman login` - a browser flow where a browser can open, or a device code (RFC 8628; auto-selected on headless and remote machines, `--device-code` to force) approved from any other device: nothing listens locally, no ports are forwarded, and no token is ever copied by hand. ## Pricing Per-minute rates. A node bills while running or idling in its grace window; stopped nodes cost nothing. A node that goes idle keeps running for its grace window, currently 15 min on 1x shapes and 20 min on 8x ($0.90 and $9.60 at the H100 list rate if the window runs out), quoted on every rate as minimum_minutes / session_minimum_usd. There is no minimum charge: stop_node (or the in-container POST /v1/stop) ends the window early and billing stops at the stop. The current effective rate (with any active discount) rides every create_node/get_node response as `rate`: the price in the response is the price you pay. The week has a shape. Fridays are the cheapest day of the week and weekends are cheap, on the US Pacific calendar. The discount applies to the minutes that actually run on those days. A node you create on Thursday still gets Friday's price on Friday, and a node you create on Friday pays the ordinary rate again on Monday. A calendar day only ever lowers your rate. The rate quoted when your session opened is the most you will ever pay for it. Every rate-carrying response reports where the week stands in `rate.calendar`, which gives you today's percent off, the schedule, and the next cheaper day with its rate, so work that can wait a day can see what waiting is worth. Batch jobs are already discounted below the interactive rate, and a calendar day reaches them only when that day is cheaper still. At today's numbers Friday moves a job and the weekend does not. The `calendar.moves_jobs_today` field on submit_job says which. Sandboxes are priced apart from nodes: a sandbox bills per GiB of its RAM per minute while it is RUNNING (forking is free, baking bills as ordinary sandbox time, and a void's time is excluded from billable time in the hour it is recorded), and a stored snapshot bills per GiB-month of its deduplicated bytes - the resident rate while its bytes sit on a sandbox host for instant forks, or the cheaper cold rate once idle snapshots move to the object store (automatic; the first fork afterwards pays a one-time rehydrate and the response says how long it took). Every snapshot gets a durable off-box copy in the background, so a host failure cannot lose one, and snapshots live until you delete them unless you set your own expiry (expires_after at capture, or set_sandbox_snapshot_expiry later). A sandbox with no command in flight PARKS itself after ~1.5s idle: it hands back the memory its guest can spare, so you stop paying the RAM rate on what was handed back and pay the far cheaper parked GiB-month rate on the disk it holds. Processes you left running keep running - a park only ever takes spare memory - and your next exec wakes it in tens of ms. Nothing to call. How much you save depends on how much the sandbox was holding: one idle between tool calls with nothing resident gives back nearly all of it, one holding a loaded model keeps that memory and keeps paying for it, because only what is actually handed back is discounted. The live figures are in the pricing table at https://autoresearch.sfcompute.com and the month to date is get_usage's `sandboxes` line. Billing portal: https://autoresearch.sfcompute.com/billing. ## Machine payments (an agent buying credit) An agent with delegated spend authority buys prepaid credit itself over MPP, Stripe's HTTP 402 Machine Payments Protocol. POST https://autoresearch.sfcompute.com/pay/v1/topups with an org service token (gmnt_..., org write scope) and body {"amount": "50.00"} (dollars as a string, whole cents, $10 to $10,000). The 402 carries a signed challenge in WWW-Authenticate; an MPP client (Stripe ships them for Node and Python) mints a shared payment token for the challenged amount and retries the same request with the payment credential. Success answers {"ok": true, "credited", "balance"} plus a Payment-Receipt header naming the Stripe payment; a 202 means the payment is still settling and the credits mint when it does - never pay again with a new token before checking. Each credential pays exactly once; a replay is refused with no second charge. On an org that pays by card but has none saved yet, the first settled machine payment converts billing to prepaid: work runs against the credit balance and stops at zero, another top-up resumes it, and adding a card on https://autoresearch.sfcompute.com/billing returns the org to monthly invoicing. Out-of-credit and no-payment-method refusals name this door whenever it is live. ## Private data & object storage One small file off your own machine does not need any of this: `gman cp ./train.py :~/train.py` puts it there (and `gman cp :~/out/metrics.json ./` brings one back, `gman cp a:~/x b:~/x` moves one between nodes). It rides the request body, so it is capped at max_write_file_bytes (`gman limits`) - configs and entry scripts, not datasets. Everything below is the path for the big ones. Private sources (a gated HF repo, a private bucket) ride stored org connections: a human sets one up once (`gman connection create` - the secret never enters a chat), then agents call list_connections and import_data(node, connection, source, dest_path) and the bytes appear on the node; poll get_import. A connection belongs to ONE org: a session opened in an org acts there, an unbound session acts in the caller's active org, and every connection and secret answer names the org it acted in - a miss says when the name exists in another of the caller's orgs. Until a human has set a connection up, the run does not wait: run_command's env carries the cloud credentials (never the command string) and `aws s3 sync` inside the node moves the bytes. Object storage is the no-vendor path: an org admin runs `gman storage bucket create `, uploads with `gman storage cp`, and the bucket appears to agents as connection `storage/` - no credential exists anywhere. A directory becomes one object per file, uploaded in parallel (`--concurrency`, default 32), so a tree of thousands of small files lands in minutes. Coming back down is the same command. `gman storage cp corpus/train/ ./train` downloads everything under a prefix (a bare `corpus/` is the whole bucket) in parallel, and a destination ending in `.zip` writes one archive instead of a directory. Without the trailing slash the source is a single object. An object deleted mid-download is named and the command exits nonzero, so an incomplete corpus is never reported as complete. An agent explores a bucket it did not fill with list_storage_objects(bucket, prefix, delimiter): `delimiter: "/"` folds each next segment into `prefixes` so a million-object bucket browses a directory at a time, and `truncated`/`next_after` page the rest. It reads the index of the bucket rather than any bytes, so no node runs and no egress is billed. Then import_data pulls a key, a "/"-terminated prefix for the whole subtree, or a non-slash key prefix (when no exact object has that key) to take a flat prefix in chunks: .../items/a, then .../items/b, and so on. On a storage/ connection the bucket is already named, so a source or dest may be written bucket-relative: `train/` and `s3://corpus/train/` mean the same thing. An s3-keys connection reaches a bucket we do not own, so there the full url is required. One import takes up to 20,000 objects (`max_import_files`, on list_limits and operator-raiseable); the chunk form is how you get under it. You also start at most `imports_per_hour` imports an hour (default 30). That counts YOUR calls across every node and workspace you use, in this org and any other, so a teammate never spends your budget and your own second node always does, and the window slides, so a slot frees an hour after the call that took it. The refusal names the count, the reset time and the wait. An import of a whole prefix is one call however many objects it carries, so a chunked bootstrap is usually a prefix import that never touches the wall. A private Docker image for a JOB works the same way and needs nothing at submit time: an org admin stores a `registry` connection (`gman connection create --kind registry --scope 'ghcr.io/acme/*' --username --material-stdin`), and submit_job(image: "ghcr.io/acme/trainer:v4") authenticates the pull server-side. GHCR, Docker Hub, Quay, Harbor, Artifact Registry, ACR; not ECR, whose tokens expire after 12 hours. A private base image inside a Dockerfile FROM is not covered. Storage bills per GB-month, prorated daily, marginal tiers: first 100 GB free, $0.10/GB to 10 TB, $0.07 to 100 TB, $0.05 to 1 PB, $0.035 beyond; deletes stop accruing the next day. get_billing and `gman storage bucket ls` show the footprint and month-to-date dollars. ## Data retention The org decides what we keep about its workloads, at https://autoresearch.sfcompute.com/settings/data. Six captures, each a switch, and OFF MEANS NEVER WRITTEN - not written and deleted later: command output (the 8 KiB tail get_command answers with after a node stops), command lines (sealed at rest, never echoed to you; feeds the mission Commands tab), declared results (your GMN_RESULT_PATH JSON - mission receipts are built on it), logs (query_logs, 31 days by default, 1-365 settable), traces (14 days, same range), and job logs/artifacts (30 days, or purged when the job ends). The log and trace windows are applied by those stores rather than at capture, so a change lands within minutes, not instantly, and shortening one deletes what is already past it. Command output and stored command lines age off their records after 90 days for every account (1-365, settable); the record itself survives, because it is what proves the work ran and what it cost. get_retention() answers what is on for this org and what is kept regardless. READ IT before assuming an output will still be there: when a capture is off, get_command says so in a `retention` field rather than leaving you to poll a field that will never appear. Only an org admin changes it - on /settings/data, with `set_retention`, or with `gman retention set --logs off`, which is the form that belongs in a repo. Turning a capture off also purges what it already holds (an hourly sweep drains history), and turning it back on recovers nothing: declined content was never written, and purged content is gone. Kept regardless of every switch: the audit log (24 months - decisions and money, never workload bytes), billing records, routing metadata (which node ran, where, how long, what it cost), support tickets, and your volume's contents until you delete the volume. Sandboxes need no setting: sandbox_exec returns {exit_code, stdout, stderr} and writes nothing anywhere. Nor do sync execs unattached to a mission, write_file content, or env values (encrypted at rest, echoed by name only, redacted from anything we ship). ## Research agents Research is in private preview. It is off by default for an organization (file_ticket to ask). Research attaches a researcher to a mission (https://autoresearch.sfcompute.com/docs#research). You give it an objective, your uploaded source, a check set, and a budget. The managed researcher measures a baseline, changes one factor per hypothesis, runs each experiment as a job from your source, and reads the evaluations the platform computes over the results files. When an experiment beats its control and passes every check, the platform runs it again on fresh executions; if those pass, the mission completes with a confirmed result. A negative or inconclusive result is a valid outcome. - start_research(name, objective, source_context?, checks?, benchmark?, budget_usd, planner?, intelligence_tier?, plan?, deadline_hours?): the mission is created if new. Upload the source first with create_context + finalize_context and pass the context id (`gman research start --source .` does both). planner "managed" (default) lets our researcher choose the experiments at tier i1, i2, or i3; planner "customer" runs the research.plan/v1 plan you pass and waits for the next one. The budget is a hard cap: every experiment job and every researcher turn reserves its maximum charge before it starts, and a fifth is held for confirming the selected result. - get_research(name): state (ready, running, waiting, needs_input, verifying, stopping, stopped, completed, failed), version and control_version, budget (spent, reserved, protected, available), the open question, the result, every experiment with its verdict and measurements, and the record (plans, observations, evaluations, decisions). Calls no model; poll freely. - steer_research(name, expected_version, ...): one action per call. The action is a plan, an instruction to the researcher, a question_id + answer, a planner handoff (needs expected_control_version and accepted_work "continue" or "cancel"), or complete: true. A stale expected_version is refused. Read again and retry. - stop_research(name): no new work, running experiments cancelled where possible, the record kept. list_research() lists the workspace's. Each experiment runs `sh -c ` from /workspace/source and must write a JSON object to its results file (default results.json). A check set (research.checks/v1) names a dotted metric path, a comparison (>=, <=, >, <, ==, !=), and either a threshold or a relative_to_control ratio; the evaluator takes the median over the experiment's repetitions. A check is passed, failed, or unknown. A missing metric is unknown, never a pass. Experiment jobs bill as jobs at the published job rates; the managed researcher bills intelligence units at $0.01 (i1), $0.03 (i2), or $0.06 (i3) per unit, shown as `research` on get_usage and the mission receipt. The HTTP routes are POST /preview/research, GET /preview/research, GET /preview/research/, POST /preview/research//steer, and POST /preview/research//stop; the CLI is `gman research start|get|steer|stop|ls`. ## Post-training and RL (recipes) Known-good starting points, not requirements - full walkthroughs at https://autoresearch.sfcompute.com/docs#post-training. All run on one 8x node or as jobs; multi-node training is not offered yet (a node is one machine - no cross-node fabric, and tenant isolation forbids node-to-node traffic; scale out with sweeps, not distributed training). - LoRA SFT on one node: submit_job(chip: "h100", chip_count: 8, command: "python sft.py", context_id: "ctx_...", hf_cache: true, resume: "checkpoint", idempotency_key: "sft-run-1"). chip is the model ("h100") and chip_count (1-8) the count - "h100-8" (or its original spelling "8xh100") names a create_node node type, not a submit_job chip, and chip_count defaults to 1 if omitted. The context carries a Dockerfile (FROM your training base, pip install trl peft) plus code. Write trainer state to $GMN_CHECKPOINT_DIR each epoch (preemption then costs minutes, not the run) and the final adapter + eval verdict to $GMN_OUTPUT_DIR / $GMN_RESULT_PATH so results ride get_job. hf_cache is a per-MACHINE shared cache at HF_HUB_CACHE (best-effort, LRU-evicted): a sweep's later jobs skip the base-model download only when they land on a machine that already pulled it, not fleet-wide. - GRPO on one node: three cooperating processes colocate on one 8x - a vLLM server holding the policy, the TRL GRPOTrainer, rollouts looping between them. Needs an image carrying vLLM + TRL: until trl-0.21-cuda12.9 records a build, create_node(chip: "8xh100", image: "vllm-0.26-cuda12.9"), then run_command("pip install trl peft datasets accelerate"), then two detached run_command calls (serve with restart: "on-failure", then train). A reward curve is not free telemetry: query_metrics returns GPU/system/spend automatically, but a reward series exists only if the trainer exposes it on a Prometheus port and you register_scrape(node, port) (otherwise tail the rewards file). Group the run under a mission and finish_mission(result) so the receipt is the run report. - Rollouts and rewards: the rollout envelope (the Rollouts section below) puts identity, memoization and provenance-stamped verdicts over the two primitives named here, and is the shape for evaluations and GRPO groups. Raw: run environment rollout batches as submit_jobs sweeps (verdicts through $GMN_RESULT_PATH). Untrusted tool/code execution runs in sandboxes (the infra tool block above): create_sandbox_env once, fork_sandbox(count: N) per sample, one array-shaped sandbox_exec, one array-shaped delete_sandbox. A job drives the same plane over /preview/sandboxes/* with a service token in run_command's env, never the command string. A judge/reward model is a second serving process (vLLM or SGLang). Colocate it on a slice of the same 8x when you can: there is no node-to-node path by default, so a judge on a SEPARATE node is reachable only via expose_port (a public HTTPS URL, which stop kills and a wake does not resurrect - re-expose after any restart; vanity: "