Runtime Protocol v1

The canonical protobuf schema is maintained in the Devsy Runtime SDK. The module path is github.com/devsy-org/devsy-runtime-sdk; the logical plugin name is devsy-runtime. HashiCorp application protocol 1 and Info API major 1/minor 3 are separate version checks. Same-major newer minor versions are accepted. Unknown mount/recreate enum values are rejected because they control host behavior.

Runtime boundaryCopy Anchor Link

Info, Preflight, ProvisioningPreflight, ReusePreflight, Find, TargetArchitecture, RunImage, Start, Stop, Delete, Exec, and Logs are the v1 RPC surface. Runtime state persists in the backend across plugin processes. Plugins do not own image build/tag/push, registry credentials, Compose, IDE configuration, snapshots, provider machine lifecycle, or updates.

RunImage receives resolved intent. Its empty response acknowledges completion; Find queries state. image_built_locally is an image-origin hint, not permission to build. Optional privileged/init flags distinguish absent from explicit false. Environment and mounts may contain secrets and must not appear in diagnostic logs.

remote_user carries the developer identity used for workspace ownership, separately from the container process user. The host forwards both values without substituting one for the other. Runtimes use remote_user when set, otherwise user, otherwise root for workspace ownership. dockerless indicates that the host will build the developer filesystem after the image starts, so that identity may not yet exist in the image. A runtime that resolves mount ownership from image contents must validate this case before changing workspace resources. These fields describe provisioning intent and do not authorize replacement of an existing workspace.

Runtime name, driver name/version, and capabilities are required in Info. Runtime version may be empty when a backend cannot report it without expensive setup. An empty mount list means no supported mount types. ProvisioningPreflight may be a no-op when its capability is false. Logs may return Unimplemented when its capability is false. Reprovision means RunImage can update an existing workspace with complete resolved intent; it does not imply that an empty request is safe. The Devsy host does not enable in-place reprovisioning; workspace changes follow the negotiated recreation policy.

TargetArchitecture returns canonical amd64 or arm64. A runtime may require an existing workspace to answer; hosts must not require pre-start architecture discovery from such runtimes.

LifecycleCopy Anchor Link

Find returns found=false for ordinary absence, without a NotFound RPC error. A found response includes container details with normalized state running or stopped. Transport, permission, and backend errors remain errors.

Start on an already running workspace succeeds; missing returns NotFound. Stop on an already stopped workspace succeeds; missing may return NotFound. Delete normalizes missing state to success for cleanup. Provisioning compatibility checks must precede destructive teardown.

API 1.2 adds optional capabilities.reuse_preflight. Before reusing a workspace, the host calls ReusePreflight with its workspace ID and current resolved developer identity. The runtime validates its own creation-time contract, such as mount policy or ownership, without starting, stopping, deleting, or modifying the workspace. An incompatible contract returns structured FailedPrecondition with explicit --recreate guidance. The host propagates that error without scheduling replacement, preserving the existing VM. Missing workspaces return NotFound; backend, permission, and context failures remain errors. When the capability is absent, hosts skip the RPC and retain their existing identity-resolution behavior. Explicit recreation follows separate provisioning checks and the negotiated recreation policy.

API 1.3 adds negotiated recreation during RunImage. A host opts in with InfoRequest.supports_recreate_on_run; a supporting runtime responds with RECREATE_MODE_ON_RUN. Devsy then leaves the existing VM in place and forwards the current operation's explicit recreation permission as RunImageRequest.allow_recreate. Negotiation alone grants no replacement permission, and a later request with false remains false. Existing explicit reset and rebuild operations also grant recreation permission. Automatic identity or secret migrations must request consent instead of silently enabling replacement.

Older hosts and providers retain their negotiated DELETE or STOP behavior. ON_RUN does not enable in-place reprovisioning: it delegates replacement to the runtime so final effective configuration can be validated before teardown.

For MicroSandbox, explicit replacement requires v0.7.7 or newer even with workspace permission policy off. The built-in and supporting external driver pass --replace to that operation instead of stopping and removing the old VM first. The runtime validates the effective configuration, including managed overrides, before removing the target. This preserves the old VM when configuration is rejected; it does not roll back failures after replacement begins. Image preparation, cache imports or new volumes can precede final validation. Root preparation, launch failures, cancellation or crashes after teardown can still lose the old VM.

Exec and LogsCopy Anchor Link

The first client frame is exactly one ExecStart containing argv. Later frames contain stdin bytes or exactly one CloseStdin; the client then closes its send side. Data after CloseStdin, repeated Start, unset payloads and empty stdin data frames, and unexpected EOF before CloseStdin are InvalidArgument. v1 Devsy callers use tty=false; runtimes reject unsupported TTY requests.

Each side uses one send pump and one receive loop. Data chunks should be at most 32 KiB. Empty input is represented by CloseStdin with no preceding data frames. Stdout/stderr are separate byte streams, with no text decoding or PTY. The receiver drains output before exactly one terminal ExecExit. Command success requires exit_code == 0 and an empty signal. A nonempty signal means command failure regardless of exit_code, including its protobuf default of zero. Ordinary nonzero or signal-terminated command exit is carried in ExecExit and the RPC succeeds. Setup/transport/backend failures are RPC errors; stream EOF without an exit is not command success. Context cancellation/deadlines terminate the operation and release its resources. Plugins that launch children must ensure child cleanup; server bootstrap and host-supervised process ownership are separate responsibilities.

Logs uses merged binary OutputChunk frames. Output buffering must remain bounded. Do not call Send concurrently from stdout and stderr copiers.

Errors and trustCopy Anchor Link

Use canonical gRPC status and attach RuntimeError details for stable categories, actionable messages, optional backend diagnostics, retryability, and structured context. Raw backend diagnostics must be redacted before display. Unknown detail fields remain forward-compatible; callers must not parse messages to classify errors.

The plugin binary is trusted provider code. Devsy resolves the selected external runtime from its Agent.Binaries declaration and requires a SHA-256 checksum before launch, including cached and local absolute paths. The resolved executable must be a regular file; Unix hosts also check effective execute access. The runtime supervisor is the running Devsy executable's internal runtime-supervisor entry point, not another provider-distributed binary. The SDK transport runner does not download executables or establish checksum trust. A generic provider download or an Agent.Binaries path alone does not establish this trust, and the magic cookie is not authentication or sandboxing.

Plugin process environmentCopy Anchor Link

Devsy inherits the agent host's environment, then appends resolved provider/workspace values. This external-driver path does not supply a separate extra-environment map. This is the environment of the trusted plugin and its runtime CLI children, separate from the guest environment carried in RunImage and ExecStart. The external driver preserves this behavior; it does not introduce an environment allowlist.

PATH and home directories can select runtime CLIs and configuration. Proxy and CA settings, Docker configuration and credential helpers, and temporary-directory settings can affect the provider's image clients. Preserving those inputs does not prove every proxy topology, credential store or host profile works. The native consumer cases below cover their controlled registry/TLS/proxy/credential paths. Never log the full environment or secret-bearing runtime arguments; redact backend diagnostics before display.

The host owns each supervised plugin session. The SDK server bootstrap does not itself own backend children; runtimes remain responsible for independently managed services and children outside the supervisor's process ownership. Cancellation does not promise rollback of backend effects that already committed.

See the rollout decision for the tested migration path, platform applicability and compatibility boundary.

For SDK usage and development commands, see the SDK README.

MicroSandbox parity gateCopy Anchor Link

MicroSandbox integration suites share a provider matrix and test-session setup. The dedicated integration-tests-microsandbox job defines the suite catalogue and common runtime installation once, while keeping each suite on an independent Linux runner. A CI-only Go utility built from the tested checkout installs and checks the pinned MicroSandbox bundle, executes one labelled E2E suite and retains its log and Ginkgo JSON report; it is not a Devsy release artifact. The utility's verify-report command checks the selected label in that report.

Each native run must report the exact expected count, with every selected case passed once and no selected skips or retries. The default provider-mode: parity requires equal built-in and external provider counts. An explicitly external-only catalogue row uses provider-mode: external and requires every selected case to carry only the external provider label. Dry-run and programmatic-focus reports cannot satisfy this gate. Prior reports prove only the labels and cases they contain; the earlier 74-case parity baseline does not prove the added three-case consumer contract or the expanded 77-case catalogue. See the test architecture guide for fixture ownership and extension rules.

The up-provider-microsandbox E2E label runs the same lifecycle and ownership scenario against the built-in provider and the external v0.1.8 release, each with isolated Devsy configuration. CI pins MicroSandbox v0.7.7 by checksum and requires access to KVM; unavailable virtualization fails this job instead of producing a passing skipped test. Each provider scenario has a ten-minute deadline and the CI job has a 25-minute deadline.

The shared scenario exercises agent delivery, SSH, a 1 MiB binary stdin/stdout round trip with separate stderr and a nonzero guest exit, root workload versus developer identity, bind-mount ownership and mode mirroring, stop/start, recreation, rejection of an identity change without recreation while preserving VM-local data, and deletion of the VM.

Run it on Linux with KVM or Apple silicon after installing MicroSandbox v0.7.7 or newer:

DEVSY_REQUIRE_MICROSANDBOX=true task cli:test:e2e:suite -- up-provider-microsandbox

The separate up-provider-microsandbox-images label runs three image cases against each provider: registry fallback with an image absent from Docker, loading a Docker-only image, and a Dockerfile build with a local Dev Container Feature. Guest checks verify the image's filesystem and environment markers and the Feature's installed file and environment. The registry case uses an isolated loopback registry and verifies manifest reads; no external test registry credentials are required. CI gives this matrix a 20-minute test deadline and a 25-minute job deadline.

The image suite also requires the Docker CLI and a running Docker daemon, in addition to the MicroSandbox and virtualization prerequisites above.

DEVSY_REQUIRE_MICROSANDBOX=true task cli:test:e2e:suite -- up-provider-microsandbox-images

The up-provider-microsandbox-mounts label runs five mount scenarios against each provider. It checks bidirectional bind access, read-only write rejection, named-volume persistence through stop/start and recreation, tmpfs reset, and strict/relaxed/off stat virtualization with private host permissions. Permission checks use host-created files after workspace setup so recursive chown cannot mask the guest ownership fallback. Host mode changes under off must become visible after MicroSandbox's five-second guest attribute cache expires. Each scenario has a ten-minute deadline; CI gives the matrix a 20-minute test deadline and a 25-minute job deadline. Test-owned named volumes are removed after workspace cleanup.

This suite needs the same MicroSandbox and virtualization prerequisites as the lifecycle suite:

DEVSY_REQUIRE_MICROSANDBOX=true task cli:test:e2e:suite -- up-provider-microsandbox-mounts

The up-provider-microsandbox-resources label runs six scenarios against each provider: default CPU/memory sizing, explicit provider options overriding hostRequirements, zero provider values falling back to hostRequirements, effective resources below larger boot ceilings, and rejection of CPU or memory ceilings below the initial allocation. It checks the running VM's active configuration and the guest's online CPU count and MemTotal, allowing 128 MiB for kernel reservations rather than treating persisted configuration as proof of guest capacity.

The ceiling scenario uses msb modify on the test-owned VM to grow CPU and memory to their boot ceilings, then checks guest convergence within 60 seconds. Requests beyond either ceiling must fail with a restart-required plan while preserving active resources, VM creation time, guest boot ID, and a filesystem marker. This exercises the capacity provisioned by each driver; Devsy does not expose a live-resize API. Each scenario has a ten-minute deadline, with a 20-minute CI test deadline and a 25-minute job deadline. The suite has the same runtime and virtualization prerequisites as lifecycle testing:

DEVSY_REQUIRE_MICROSANDBOX=true task cli:test:e2e:suite -- up-provider-microsandbox-resources

The up-provider-microsandbox-storage label runs three persistent-root scenarios against each provider: the runtime's default capacity, an explicit provider storage option overriding hostRequirements.storage, and a zero provider option falling back to a fractional-GiB host requirement rounded up to whole GiB. It checks active root-disk configuration and guest filesystem capacity, allowing for filesystem metadata instead of comparing free space with the raw disk size.

Each scenario verifies that a VM-local file survives stop/start and disappears after recreation, while a host workspace file persists through both operations. Each scenario has a ten-minute deadline, with a 20-minute CI test deadline and a 25-minute job deadline. This suite uses the same runtime and virtualization prerequisites as the lifecycle suite:

DEVSY_REQUIRE_MICROSANDBOX=true task cli:test:e2e:suite -- up-provider-microsandbox-storage

The up-provider-microsandbox-ephemeral label runs three scenarios against each provider. An unsized ephemeral root delegates to the runtime default (half the initial memory on the pinned runtime); an explicit storage size overrides that default. The suite checks the active tmpfs backing and guest capacity, root-file loss after stop/start, retained sandbox identity, and persistent host workspace content. The default-size scenario also checks named-volume persistence and recreation. A fresh creation with a root larger than initial memory must fail with the runtime's size diagnostic and leave no test-labeled VM or source-file changes.

Ephemeral roots use a RAM-backed upper layer beneath the guest's overlay filesystem. They do not enable the runtime's separate sandbox-record removal flag. These tests do not establish preservation of an existing VM when an invalid recreation is requested; that requires separate validation before teardown. Each scenario has a ten-minute deadline, with a 20-minute CI test deadline and a 25-minute job deadline.

DEVSY_REQUIRE_MICROSANDBOX=true task cli:test:e2e:suite -- up-provider-microsandbox-ephemeral

The up-provider-microsandbox-egress label checks default public TCP access and explicit egress denial against each provider, at creation, after stop/start, and after recreation. A test-owned Linux interface exposes a controlled destination that the pinned runtime classifies as public. Positive host and default-policy VM probes bracket each denial check; the blocked VM must retain SSH access, report connection refusal or reset, and send no request to the destination. Active network configuration and VM identity checks accompany the probes. The fixture requires Linux, root access with CAP_NET_ADMIN, and iproute2; CI fails when these prerequisites are unavailable.

DEVSY_REQUIRE_MICROSANDBOX=true task cli:test:e2e:suite -- up-provider-microsandbox-egress

The up-provider-microsandbox-prebuilds label publishes a Dockerfile with a local Feature to an isolated registry, then checks matching reuse and changed-Dockerfile invalidation against each provider. Matching reuse must read the published hash without invoking a Docker build or entering build-strategy selection. Changed input must miss a different hash, build the new image, and retain the old published manifest. Guest file and environment markers verify the selected image and Feature contents. The fixture removes only its own image tags and requires a Linux host with a local Docker daemon, buildx, and loopback HTTP registry access.

DEVSY_REQUIRE_MICROSANDBOX=true task cli:test:e2e:suite -- up-provider-microsandbox-prebuilds

The host starts its generated bootstrap SSH helper with literal arguments when the runtime supports argv execution, so setup can begin before the Dockerless filesystem provides /bin/sh. Runtimes without that capability retain command-string execution.

The up-provider-microsandbox-dockerless label runs four scenarios against each provider with host Docker unavailable. It explicitly selects the existing Dockerless path; it does not test automatic fallback. Both the Docker CLI and Go client are directed to test-owned failures, while a real guest build must install a Dockerfile and local Feature. Image PATH, credentials-directory configuration, Feature environment and a bare-name executable are checked through setup hooks and SSH. The root scenario repeats up, stop/start and recreation, checking build reuse, VM identity, root-file persistence and preserved host data. A fixed non-root developer must succeed with off stat virtualization and private host permissions; fresh creation under strict or relaxed must fail before VM or build-volume creation, preserving host data and permissions.

This suite requires Linux with KVM and public access to the base and Dockerless builder images. It does not require a usable host Docker daemon. Each scenario has a ten-minute deadline, with a 20-minute CI test deadline and a 25-minute job deadline. Cleanup removes only the fixture's workspace, VM, build volume and provider.

DEVSY_REQUIRE_MICROSANDBOX=true task cli:test:e2e:suite -- up-provider-microsandbox-dockerless

The up-provider-microsandbox-recreate label tests rejected explicit replacement for running and stopped VMs against both providers. Each case checks the unchanged VM generation and active configuration before resuming a stopped original, then checks persistent root, bind and named-volume data together with the workspace and SSH records. A subsequent valid replacement must change the VM generation, reset the root marker and preserve the external data. These request-invalid cases do not establish preservation for managed-policy conflicts or failures after replacement begins. Each scenario has a ten-minute deadline, with a 20-minute CI test deadline and a 25-minute job deadline.

DEVSY_REQUIRE_MICROSANDBOX=true task cli:test:e2e:suite -- up-provider-microsandbox-recreate

The Linux-only up-provider-microsandbox-managed-recreate label checks a valid 2048 MiB replacement with an explicit 2048 MiB boot ceiling. A managed policy raises initial memory to 3072 MiB, making the effective configuration invalid. The workspace retains its original provider options throughout; changing global provider defaults would not override those saved workspace values. Running and stopped originals must retain their VM generation, root and persistent mount data, workspace record and SSH files for both providers. Resetting the policy must allow the same replacement request. The CI wrapper overlays /etc in a private mount namespace before creating any policy file; its scratch remains in the disposable runner for job teardown. These checks require that isolated wrapper and do not establish rollback after replacement has begun. Each scenario and the suite retain the request-invalid suite's time bounds.

On a disposable Linux host with the same prerequisites, prepare the E2E binaries and run the managed suite through its isolated wrapper. The scratch directory is retained for host teardown; running the label without the wrapper does not exercise these cases.

task cli:test:e2e:build
(cd e2e && go test -c -o e2e.test .)
(cd e2e && sudo env PATH="$PATH" RUNNER_TEMP="$(mktemp -d)" \
  DEVSY_REQUIRE_MICROSANDBOX=true DEVSY_REQUIRE_MSB_MANAGED_POLICY_FIXTURE=true \
  ../hack/run-managed-policy-e2e.sh ./e2e.test -test.v -ginkgo.v \
  -test.timeout 1200s -ginkgo.fail-on-empty -ginkgo.flake-attempts=1 \
  -ginkgo.label-filter=up-provider-microsandbox-managed-recreate)

The up-provider-microsandbox-logs-cancel label checks completed stdout/stderr workload logs while running and stopped, direct runtime Exec cancellation of a ready parent and child, and cancellation with blocked owned stdin against each provider. A separate SSH observer verifies process identity using PID, kernel start time, and a unique command marker. Caller return and process death must both occur within ten seconds of cancellation; the VM's creation time, boot ID, and source files must remain intact. Logs do not start a stopped VM. These checks do not establish log ordering, binary log fidelity, TTY cancellation, or cancellation of arbitrary nonclosable input.

DEVSY_REQUIRE_MICROSANDBOX=true task cli:test:e2e:suite -- up-provider-microsandbox-logs-cancel

The up-provider-microsandbox-compatibility label checks preservation of a running workspace when a test-owned shim returns old, malformed, or failing version responses. It then restores the supported CLI and requires explicit recreation to succeed. The shim injects version-command faults while retaining a real supported VM; it does not emulate an actual historical MicroSandbox release. External-provider discovery may report a version-command failure earlier than the built-in runtime. These cases run on native Linux and macOS, not Windows.

DEVSY_REQUIRE_MICROSANDBOX=true task cli:test:e2e:suite -- up-provider-microsandbox-compatibility

For built-in Exec, cancellation closes non-file io.ReadCloser input whose Close unblocks Read; ordinary completion and caller-owned file input retain their ownership. Agent binary delivery owns its acquired source and closes it when the individual transfer attempt is canceled, before retrying with a fresh source. Cleanup joins cancellation callbacks before returning.

The Linux-only up-provider-microsandbox-registry-consumer label selects exactly three cases from the published external provider's Runtime Protocol RunImage path. Its positive case uses an owned TLS registry, fixture CA, restricted CONNECT proxy and private Docker credential helper, verifies the provider executable digest and exact authenticated manifest/blob response bytes, then checks a guest marker. Independent missing-CA and missing-credential cases require concrete refusal and no created VM. Each case owns fresh image and runtime state. This suite does not prove ordinary up image metadata resolution, remote agent binary preparation, or general egress denial. Reports from the earlier 74-case parity baseline remain useful for those existing cases, but do not count as runs of this external-only row.

DEVSY_REQUIRE_MICROSANDBOX=true task cli:test:e2e:suite -- up-provider-microsandbox-registry-consumer

The Linux-only up-provider-microsandbox-remote-agent-binaries label uses the published external provider with agent.local: false and a test-owned remote agent container. It separately counts the Devsy agent executable transfer and the runtime binary download through Agent.Binaries. After the first successful up, it removes only the agent's provider-download cache, makes the runtime source return 404, and requires a later up to reuse the retained workspace binary with unchanged checksum and inode. A guest command verifies the workspace remains usable. The Devsy agent endpoint remains available; this does not promise offline reuse of that separate executable or ordinary up image metadata resolution. This added one-case external-only row brings the configured catalogue to 78 cases; earlier 77-case reports do not prove it. Acceptance requires current-head native Linux/KVM reports, not documentation or registration alone.

DEVSY_REQUIRE_MICROSANDBOX=true task cli:test:e2e:suite -- up-provider-microsandbox-remote-agent-binaries

For the existing two-case lifecycle label, the external-provider case can capture ordinary fresh and reused up wall time and filtered runtime startup events when CI supplies all five explicit inputs: DEVSY_TEST_RUNTIME_SPAWN_PROBE, DEVSY_TEST_RUNTIME_SUPERVISOR, DEVSY_TEST_RUNTIME_PERFORMANCE_DIR, DEVSY_TEST_RUNTIME_REVISION, and DEVSY_TEST_RUNTIME_SDK_VERSION. The built-in case keeps its existing path. After both ordinary commands pass, the helper stages a checksum-verified copy of the published runtime and runs direct/supervised/supervised/direct Info/Find comparisons, each with a first launch and 100 warm samples. The JSON artifact records tool/runtime provenance and reaped processes. Startup events measure launch, handshake and dispense before RPC; they exclude binary resolution, RPC and reaping. These measurements use warm host caches, add no latency threshold or VM case, and do not claim cold-cache performance or total plugin overhead.

These scenarios do not establish complete parity. Compatibility shims verify preservation under injected version faults but do not establish behavior against an actual historical MicroSandbox release. Green lifecycle, image, mount, resource, persistent-storage, ephemeral-root, TCP-egress, prebuild, Dockerless, request-invalid and managed-policy recreation, logs, and cancellation checks alone do not authorize that cutover. Dockerless refusal checks cover fresh creation, not preservation after an invalid recreation request. Egress coverage does not establish UDP, DNS, TLS, ingress, or live-policy modification behavior.

Prebuild invalidation coverage changes the Dockerfile; it does not independently establish hashing of local Feature script contents.