Xylem

1.The thesis & the inversion

Today's industry stack is OS → container runtime → orchestrator, where the orchestrator (Kubernetes, Nomad, Borg, OpenStack) is an external control plane doing scheduling, replication, health-checking, failover, and autoscaling against an OS that knows nothing about the cluster. Xylem inverts that: it folds the control plane into the kernel.

A "service" becomes a first-class kernel object carrying a desired replica count and a supervision policy. Redundancy, scaling, and failover become kernel verbs — not YAML reconciled by an outside controller.

The arborist vs. the tissue

The name is the plant tissue. Xylem is a distributed transport and support network spanning the whole organism, redundant by construction: when a vessel cavitates (a local failure), flow reroutes around the dead vessel — built-in failover as a structural property. It regrows each season (self-healing / self-scaling) and it is load-bearing (structural).

2.Why JefeOS, why now

JefeOS needs an identity that differentiates it from "yet another hobby Unix clone." The owner's decision is to be a natively-clustered OS — self-redundant, self-scaling, failure-resistant — chosen because it is more fun and less saturated than re-treading POSIX. The practical secondary reason it fits: failure-resistance is already JefeOS's de-facto through-line. The recent engineering culture has been reliability-first without anyone calling it "the cluster story," and that work is exactly the substrate Xylem needs.

Xylem did not wait for the future JSL-2 isolation cell. The first C++ supervision increment operates on a plain kernel task or task-group, which was enough to prove the local reconcile loop and stateless cross-vessel placement. JSL-2 remains a separate future container track: namespaces, cgroups/resource isolation, and OCI lifecycle on JefeOS's own kernel. When that boundary lands, Xylem can supervise it without changing the clustering thesis.

3.JSL ⊥ Xylem — two orthogonal axes that compose at the cell

The single most important conceptual guard-rail in this document: JSL is not Xylem, and calling Xylem a "JSL tier" would be wrong. They are orthogonal axes that compose at exactly one point — the cell.

JSL is the horizontal axis (run more kinds of software on one vessel): JSL-1 is WSL1-style syscall translation, near-done through Alpine; JSL-2 is native containers, a single-vessel isolation cell. Xylem is the vertical axis (run the same software on more vessels, self-healingly). They meet at the cell:

The intended payoff of keeping them separate is payload independence: once an isolated Linux workload can be registered as a Xylem cell, lifecycle properties belong to JefeOS rather than to Linux. Today the C++ supervisor has demonstrated restart and deterministic-primary failover with its current task recipes; it has not demonstrated migration, N-of-M replication, or a production Linux workload moving between vessels.

4.The Linux-host boundary: translation today, isolation later

JefeOS's Linux compatibility work remains the ecosystem on-ramp, not a competing identity. The current roadmap draws a clear boundary: JSL-1 translates the Linux syscall ABI on the JefeOS kernel; JSL-2 adds native container isolation around that environment. A real upstream Linux kernel in a VM is not the JSL-2 plan.

The existing JSL and Alpine work is therefore reused, not discarded. JSL-2 makes the translated environment isolatable; Xylem supplies a cross-vessel lifecycle above it. That composition is still proposed. The Xylem work that has landed so far deliberately supervises plain tasks and does not claim the future cell boundary.

5.Prior art as gold standards, NOT clone targets

Cluster orchestrators — lessons to inherit, surfaces NOT to clone

The OS-native resilience lineage — what Xylem inherits

Folding resilience into the kernel is one of the most repeatedly-attempted ideas in systems history, and most attempts died — almost never because the idea was wrong, but because they were beautiful islands with no software: technically superb systems stranded outside the ecosystem gravity well.

6.The architecture, at a high level

The cell — the unit Xylem manages

In the full design, a cell is the atom of supervision: a named, isolated, relocatable unit of execution with a declared identity and a supervision contract. Xylem never schedules "a process" or "a container" directly — it schedules cells. That complete isolation object is proposed, not landed. The C++ implementation deliberately started with a plain task/task-group recipe so supervision and placement could advance without inventing the JSL-2 boundary.

Cluster membership — kernels watching each other

Before anything can be redundant, vessels must track who appears alive. The membership table moves peers through alive, suspect, dead, and left states in-kernel rather than relying on an external watcher. Phase 1 uses plaintext, unauthenticated SWIM-style gossip on UDP 7641. That is proven liveness machinery for a trusted lab fleet, not proof that a peer is authorized to place a workload. Production authority remains a separate CP/quorum problem.

Service as a first-class kernel object

A service is the declared thing; its cell is a runtime instance. On C++ this is now a first-class kernel object with a name, compiled-in CellEntry, restart policy, desired replica count, and placement policy. A dedicated xylem_super task runs the level-triggered reconcile loop. The desired count is stored, but only the singleton value 1 is honored today; scaling rules and N-of-M placement remain proposed.

Placement authority — partial safety machinery

The C++ kernel can configure a trusted Ed25519 authority key and install short-lived signed placement grants through a privileged, fault-survivable syscall. Mirrored checksummed NTFS records preserve roster floors, per-service placement floors, and revoke tombstones across reboot. Reconcile fails closed when configuration, durable state, signature, scope, time, or grant validity is missing, and a monotonic deadline demotes an expired running service.

Proposed: self-redundancy, self-scaling, live migration

• Self-redundancy is RAID for compute. The target is for desired_replicas = N to become an invariant the kernel maintains. When a vessel is marked dead, every cell it hosted would be a deficit, and the reconcile loop schedules replacements onto survivors — honoring anti-affinity so it doesn't recreate the single point of failure. Re-replication must be single-writer through a quorum-backed authority and resource fence. Today only a stateless deterministic singleton is implemented.
• Self-scaling could reuse the reconcile model. The proposed design moves desired_replicas between min/max from in-kernel load and memory signals. No autoscaling policy or multi-replica placement exists today.
• Live migration could build on per-process page tables. Private address spaces make memory enumeration possible, but the checkpoint format, fd and connection transfer, storage contract, and migration envelope are all absent.

7.The hard problems and current safety boundary

The thesis is seductive; the discipline this section imposes is the price of the differentiator. Every hard problem k8s has, Xylem also has — now inside ring 0, where bugs are panics instead of crash-looped pods.

8.The proof-of-thesis MVP — two-vessel failover

The first end-to-end demonstration of the thesis is now landed on the C++ kernel:

• Landed evidence: an in-kernel non-hollow failover self-test and tests/xylem-failover-conformance.ps1 assert primary-hosts/secondary-standby, power off the primary, and verify survivor promotion and local placement within the membership death bound.
• Cross-kernel evidence stops at membership: the reusable Phase-1 harness is green for C++↔C++ and C++↔Rust, but JefeRust has no Service, reconcile, or placement layer, so the failover demonstration itself is C++↔C++.
• Scope: this is a compiled-in demo cell, not a migrated process, N-of-M replacement, or production workload. There is no client front-door, CP quorum, authenticated gossip, or end-to-end stale-writer fence.

9.Phased roadmap (reliability-first sequencing)

Xylem remains a long arc, but its first control loop is now real. Each phase is gated and testable; the dev loop stays reliability-first throughout (a wedge or regression always preempts Xylem work). Effort figures are deliberately omitted — this is a direction, not a schedule, and distributed systems resist estimation.

1. Phase 0 — Isolated cell Proposed JSL-2 namespaces, cgroups/resource isolation, and OCI lifecycle remain a separate future track. They are not a prerequisite for the current plain-task supervisor.
2. Phase 1 — Kernel membership ✓ Landed SWIM-style membership and failure detection over UDP run on both kernels. Repeatable two-vessel C++↔C++ and C++↔Rust conformance is green. Traffic remains plaintext and unauthenticated, so this is trusted-fleet liveness, not placement authority.
3. Phase 2 — C++ service supervision + singleton failover ✓ Landed First-class Service, dedicated local reconcile loop, restart policy, crash-loop guard, and deterministic-primary ClusterSingleton placement have landed on C++. The cross-vessel demo/self-test/harness is green. Rust parity, OnFailure exit-code semantics, N-of-M placement, and production workload/panic-recovery proof remain open.
4. Phase 3 — CP authority + end-to-end fencing Partial C++ signed-grant verification/install, mirrored durable epoch floors, fail-closed admission, and deadline demotion have landed. Production quorum consensus, authenticated gossip, resource-side epoch rejection, and the two-boot/two-vessel witness proof have not.
5. Phase 4 — Service addressing + fleet observability Proposed A stable front-door so a client reaches a moved service, plus a fleet-wide operator view. Local status/service commands exist; VIP/DNS-SD and fleet aggregation do not.
6. Phase 5 — Live migration Proposed Checkpoint / ship / resume a stateless, connection-light cell — drain a vessel without killing the workload. Gate: connection/fd migration needs absent per-netns + shared storage.
7. Phase 6 — N-of-M placement + autoscale Proposed Honor desired_replicas > 1 with anti-affinity, then move it between min/max on in-kernel load/memory signals with hysteresis.
8. Phase 7 — Stateful cells (last, hardest) Proposed Durable replicated state. Gate: likely needs an external or replicated storage substrate; explicitly the longest and highest-risk phase.

The current boundary is a stateless singleton on a trusted lab fleet. Signed-grant admission advances the CP story, but only quorum-backed authority plus resource-side epoch enforcement can close split-brain for authoritative work. Stateful cells remain the last and hardest phase.

10.Design decisions / direction

These are the current implementation boundaries and design directions. The table distinguishes direction from shipped mechanisms:

11.Open & breakout items

The remaining work is concrete and separable: