/_tiny/health.
This makes the VM — not a Linux process, not a Docker container — the tenant
isolation boundary.
Host and guest boundary
The Linux worker agent performs the privileged operations:1
Create a deployment directory
Readable only by the worker agent.
2
Build an ext4 data image
Containing the immutable artifact and the runtime host.
3
Create a TAP /30 and an nftables table
Unique to the deployment.
4
Start Firecracker through jailer
Under a dedicated UID/GID and cgroup.
5
Configure the VM
Kernel, read-only rootfs, writable artifact drive, MMDS, and network interface through the Unix API socket.
6
Send configuration through MMDS, boot, and wait
For the private health endpoint before returning the VM to the gateway.
What the guest contains
Node.js 22,wget, iptables, setpriv, a UID/GID 1000 worker account, a POSIX
shell, and /sbin/tinycloud-init from packages/provider-firecracker/guest.
The init:
mounts the data drive
reads MMDS into the worker’s stdin
blocks all subsequent MMDS access
drops permanently to UID/GID 1000
starts Node with filesystem permissions
Network policy
Every VM receives a deterministic private /30. The gateway talks directly to the guest’s port 8080. Guest-originated connections are accepted only when addressed to the host’s control-plane port — which serves both the capability broker and the data service. Established response traffic is allowed; all other host input and forwarding from that TAP is dropped. Vendor calls go through named broker operations whose workload token, grants, constraints, rate limits, response filter, and audit trail remain in the control plane.Direct egress, when enabled
Direct manifest egress is off unless the operator setsTINY_FIRECRACKER_ALLOW_EGRESS=true, and capabilities() reports that honestly
so a plan refuses rather than a deploy failing halfway.
When it is enabled, the host resolves each declared hostname once, at deploy
time, and writes the resulting addresses into that VM’s forward chain along with
a single permitted DNS resolver and a masquerade rule for its guest address.
Resolving on the host rather than trusting the guest’s own DNS is the point: a
name-based rule can be widened later by whoever controls the answer.A hostname that resolves into a private range is refused outright, because an
egress rule that reaches the host’s own network is lateral movement, not egress.
Durable resources
The VM is replaceable and never receives host filesystem mounts. Attaching a host SQLite file to two VMs during a health-checked rollout is unsafe, and copying it into an ephemeral VM breaks durability — so the provider does neither. Instead the data lives on the control-plane side of the private link, and the guest sends SQL across it. The guest configuration carries the database’s name and engine and the data-service URL; it carries no path, no host, and no credential, so there is nothing in a compromised VM to steal or to print. The control plane resolves the deployment’s workload token to exactly one environment’s binding and dispatches to either a SQLite file it owns or a per-environment Postgres role on the operator’s cluster.Because the binding is per environment and not per VM, a rollout, a rollback,
and a sleep/wake cycle all see the same data — and a superseded deployment’s token
stops working the moment the route moves.
Build VMs
A deploy whose manifest declaresbuild.command boots a second, disposable VM
first, from a separate image carrying the toolchain the serving image
deliberately lacks.
It gets the same jailer, KVM, seccomp, and cgroup treatment, plus one difference:
a firewall hole to the package registry addresses the host resolved. It has no
broker, no data service, no private network, and no neighbours.
The build runs as UID 1000 with --no-new-privs, and MMDS is firewalled off
before any build code executes.
The extracted tree is then re-packed through the ordinary artifact pipeline,
which re-applies the upload limits and strips credential-shaped files. The VM is
destroyed after every build, successful or not.
Host requirements
Linux with KVM exposed at
/dev/kvmcgroups v2,
ip, nft, mkfs.ext4, and truncatedebugfs (from e2fsprogs) and tar, to read build results out of an image without mounting itpostgresql-client, if Postgres snapshots are wantedFirecracker and its matching jailer binaries
a Linux kernel compatible with Firecracker
a read-only rootfs containing the guest prerequisites above
the worker agent running with permission to use KVM, create TAP devices, update nftables, and configure jailer cgroups
Building images
Kernel and base-rootfs patching remain an operator image-release process. They
are not performed during an app deployment.
npm, tar, and git:
TINY_FIRECRACKER_BUILD_ROOTFS the platform reports
executesBuildCommands: false. It never falls back to building on the host.
Hosted configuration
FirecrackerProvider; it has no local runtime fallback and fails
startup on non-Linux hosts.
Lifecycle and failure behavior
Reconciliation is idempotent under the deployment operation key and adopts a live jailer process after an agent restart. Desired state is encrypted on the host.sleep pauses the VM and wake resumes it without changing its provider
identity. destroy terminates the VMM, removes its TAP/nftables policy, deletes
the jail, and reports any residual paths for the leak scanner.