Hugging Face GitHub Download Weights
ModernBERT MPS/CUDA Engine • 143 MB Checkpoint

The Automated Epistemic Gatekeeper for Edge AI

“Small local models have a 15% to 25% stochastic error rate, whether from CoT loops, hallucinated syntax, or formatting quirks. If you run them raw, they break in production. System 2's job is not to magically make a broken model write code; it is to catch those failures in 500ms on Apple Silicon MPS and say: Certainty is 41.8%, do not run this code. That is the automated gatekeeper that makes edge AI safe.”
Get system2_code.pt (143 MB) View on GitHub

How Dual-Process Inference Works

Pairing fast 4-bit small generators with a non-autoregressive consensus filter on consumer laptops.

01

Parallel Candidate Draw

Fast 4-bit 7B to 12B local generator (e.g. Gemma 4 12B) emits 4 zero-knowledge candidate completions simultaneously at 50 to 70 tok/s in under 3 seconds.

02

5-Head Perspective Deliberation

A 143 MB ModernBERT network evaluates candidates across Holistic, Context-Anchored, Choice-Forward, Symmetric Balance, and Extreme Salience perspectives in ~500ms on Apple Silicon MPS.

03

Cosine Cluster Consensus

Calculates pairwise cosine similarity (0.88 threshold). If consensus ≥ 88%, execute locally for free. If candidates diverge, Epistemic Drift triggers smart cloud escalation.

Swarm Benchmark: Apple Silicon M4 Max (128 GB Unified RAM)

Empirical multi-agent test constructing an in-memory thread-safe LRUTTLCache.

Tier / Model VRAM Footprint Module 1 (Node) Module 2 (Cache) Module 3 (Test Suite) Deliberation Time Pytest Status
Qwen 27B Smoll
2.0-bit GGUF		 6.77 GB 99.7% Conf 94.2% Conf 99.6% Conf 580 ms CoT Loop Truncation
Qwen 27B IQ3_XXS
3.0-bit GGUF		 10.18 GB 81.6% Conf 75.2% Conf 67.8% Conf 982 ms 2,500 Tok Ceiling
Google Gemma 4 12B
4.0-bit MLX		 6.77 GB 76.6% Conf 98.7% Conf 41.8% Conf 1,107 ms Drift Caught Live

Developer Quickstart

Run locally or deploy as a drop-in OpenAI-compatible proxy for Cursor, Windsurf, or Claude Code.

# 1. Install the System 2 core SDK pip install system2 # 2. Launch the local OpenAI-compatible bridge proxy python3 -m sys2.server --port 8000
from sys2 import System2Engine # Initialize on Apple Silicon GPU (mps) or NVIDIA (cuda) engine = System2Engine(device="mps") # Adjudicate competing hypotheses result = engine.adjudicate( question="Select the thread-safe implementation of an LRU eviction node:", choices=[candidate_a, candidate_b, candidate_c, candidate_d], domain="code" ) print(f"Winner: #{result.selected_index + 1} | Confidence: {result.confidence * 100:.1f}%")