HackYeah 2026 ReviewCouncil score
Median of 3 models, weighted by the task's official criteriaA genuinely in-the-path control layer with real hybrid guardrails, signed feeds, hash-chained auditing and a runnable test suite, weakened by self-authored evaluation, an unexplained 174-versus-68 test claim, undisclosed heavy AI assistance, and a live demo that resists independent inspection.
Deterministic contract, limit and binding controls combine with a real model cascade and a quote-verified judge that can only tighten, all fail-closed. The 9 rather than A's 9.2 reflects that the 0-harm evidence is self-authored, and the empty live injection probe is inconclusive because the JavaScript demo exposes little to automated checks, so Member A's and B's reading is better supported than C's claim of a superficial detector.
Members disagree here: scores range by 2.2 points.A true reference monitor where the agent holds no keys, with policy, contract and feed hot reload implemented in code and committed benchmark artifacts showing p50 0.17 ms and about 3,000 decisions per second. The numbers come from the team's own committed files rather than an independent re-run, and shared SQLite WAL state is the acknowledged scaling ceiling.
Members disagree here: scores range by 2.0 points.The audit log is hash-chained with a verify endpoint, CSV export carries the deciding control, judge quote and policy hash, and Prometheus metrics plus the evidence dashboard give investigation-grade context. The depth is demonstrated through code and screenshots rather than an inspected live incident.
Members disagree here: scores range by 2.0 points.68 measured cases across 10 files, explicitly positive and negative, runnable by judges without AI, including tampered-feed and fail-closed tests. The claimed 174 control tests are not supported by the measured count and the gap is unexplained.
Members disagree here: scores range by 3.5 points.Two commands to run, Dockerfile and a live Railway deployment, Ollama-first with a hosted no-Ollama mode, YAML-only contracts for adding tools, and a small StateStore interface giving a credible path to Postgres or Redis.
Members disagree here: scores range by 2.0 points.Strengths
- The gateway is genuinely in the path: the agent holds only the Gate object with connectors and credentials inside it, and an MCP gateway applies the same checks to external clients.
- The semantic controls are real model calls: a rules-to-qwen3:0.6b-to-qwen3:8b injection cascade and a judge blind to documents that must quote the user verbatim, with the AI only able to tighten decisions.
- Policy, contract and feed hot reload plus a signed attack feed with tamper refusal are implemented in code, with invalid edits rejected and the last good version kept.
- A hash-chained audit log with verification and CSV export, a runnable 68-case test suite, and committed before/after result files for both Gemini and local qwen3:8b.
Weaknesses
- The headline 0-harm results were measured on 23 scenarios the team wrote themselves against controls they also wrote, on their own synthetic ERP, so generality is asserted rather than shown.
- The form claims 174 control tests plus 18 live tests against 68 measured test functions, an unexplained gap even if parametrization is the likely cause.
- Heavy AI coding assistance is evident from about 10,800 lines and 160 files committed by a single author in 4 commits inside roughly five hours plus a skills-lock.json in the repo, but the submission never discloses it.
- The live demo is a JavaScript app that exposes little to automated inspection; a two-string injection probe registered no hits, so live effectiveness of the semantic controls is undemonstrated rather than disproven.
Red flags
- Quantitative claims in the submission exceed the measured facts: 174 claimed control tests versus 68 measured, with no explanation offered in the pack.
claude:glm-5.3-flash89.990% agreedots-studio/dots-3-note-preview:free81.5100% agreeinclusionai/ling-3.0-flash-sante:free67.060% agreeclaude:glm-5.3–judgeThe council, the evidence pack, the prompts and the queue are all on GitHub. If a review helped you, a star helps other teams find it.