sc-compose bounded adversarial-fuzzing campaign -- verification re-run against the fixed report-generation pipeline (PR #165, baseline 749a3e0)
DRIFT
| Fuzz run description | Iterations | Pass | Result |
|---|---|---|---|
| shape-probe: var-file shapes, recursion-depth latency, and numeric edge cases | 20 | 16/20 | FAIL |
| template-probe: Jinja loops, conditionals, includes, delimiters, and loop.* under --strict | 20 | 15/20 | FAIL |
| boundary-probe: CLI negative-contract cases, malformed input, path confinement, flag precedence | 29 | 29/29 | PASS |
| differential-probe: baseline-vs-head parity across the full rendering pipeline | 20 | 20/20 | PASS |
Overall campaign result: DRIFT -- 1 new confirmed_bug (GH #169, shape-probe quadratic-recursion latency), 1 new inconclusive finding (GH #170, shape-probe silent bignum precision loss), 1 confirmed_bug reconfirmation of an already-filed, already-triaged issue (GH #167 via template-probe, out of scope for PR #165), and 1 non-blocking documentation-gap note (boundary-probe, --var-file/--var precedence, not filed as a GH issue). The differential-probe and boundary-probe results confirm PR #165 itself is behavior-neutral for the rendering engine and introduces no new CLI regressions.
16/20 passed. GH #169 confirmed_bug (quadratic-time recursion-depth latency). GH #170 inconclusive (silent lossy-f64 bignum precision loss).
Target: var-file · Seed 261 · Iterations 20 · Passed 16 · Failed/flagged 4 · Result FAIL.
Two depth-cap-latency cases (depth 8000 = 0.697s, depth 16000 = 2.653s, scaling to ~25.6s at depth 50000) show O(depth^2) wall-time growth before sc-compose rejects deeply-nested var-file input via ERR_CONFIG_PARSE: recursion limit exceeded, in crates/sc-compose/src/var_file.rs::parse_var_file_contents's serde_yaml fallback path. Reproduced 3/3 deterministic. Filed as GH #169 (confirmed_bug).
Two bignum cases show a 45-digit JSON integer literal (beyond u64::MAX) silently accepted and downgraded to a lossy f64 (999999999999999999999999999999999999999999 → 999999999999999900000000000000000000000000.0) with no diagnostic. serde_json falls back to an f64 visit for out-of-range integers; no magnitude/precision guard exists on the JSON path (unlike the existing YAML NaN/Infinity guard). Filed as GH #170 (inconclusive, needs a documented numeric-fidelity decision).
The other 16 cases (nested/jagged JSON+YAML at moderate depth, empty values, numeric edges, mixed-type arrays) all passed or were correct intentional boundaries.
Full evidence, exact minimized template/input, and root-cause/recommended-fix detail: see the companion panel.
Companion detail panel: 20260729-2-fuzz-report/20260729-2-fuzz-report-shape-probe.xhtml
15/20 passed. GH #167 confirmed_bug reconfirmed (loop.* misclassified as undeclared under --strict) -- not new, out of scope for PR #165. 1 not_a_bug note (minijinja has no .items() method).
Target: renderer · Seed 261 · Iterations 20 · Passed 15 · Failed 5 · Result FAIL.
4 of the 5 failed cases were intentional-boundary negative-contract rejections (include-cycle, include path-traversal escape, malformed custom-delimiter template, oversized nesting depth) and 1 was a self-corrected test-authoring error -- not product findings.
loop.index/loop.last are misclassified as undeclared under --strict, reproducing in the simplest single-loop template, 3/3 deterministic. This reconfirms already-filed GH #167 from a prior campaign; it is out of scope for PR #165 and was not re-filed.
A dict-.items()-method case is not_a_bug: minijinja has no Python-style .items() method; the |items filter and bare for k in dict both verified working correctly with Unicode keys.
Nested loops (3 levels, jagged/empty arrays), conditionals/elif chains, includes (single/multi-level), custom delimiters, whitespace control, Unicode values/keys, and default()/missing-field handling all passed correctly.
Full evidence, exact minimized template/input, and root-cause/recommended-fix detail: see the companion panel.
Companion detail panel: 20260729-2-fuzz-report/20260729-2-fuzz-report-template-probe.xhtml
29/29 passed. 0 unexpected failures. Two non-blocking notes recorded in report text (not filed as new GH issues): an undocumented --var-file/--var precedence rule, and a reconfirmation that GH #168 is still present.
Target: cli · Seed 261 · Iterations 29 · Passed 29 · Failed 0 · Result PASS -- all negative-contract cases produced stable diagnostics as designed.
FUZZ-B01 (non-blocking documentation-gap note, not filed as a GH issue this round): --var-file silently overrides a same-key --var flag (last-loaded-wins), matching deterministic code behavior in crates/sc-compose/src/render_request.rs:267-274, undocumented in docs/requirements.md FR-2. Not a bug, but worth documenting.
FUZZ-B02 (not_a_bug): GH #168 (var-file path confinement gap) reconfirmed still present at baseline 749a3e0; not re-filed, already tracked.
All other cases (non-object top-level values, malformed-but-YAML-valid JSON, duplicate keys, invalid var names, missing files, YAML anchors/aliases, dangerous-but-inert YAML tags, multi-var-file precedence) produced correct stable diagnostics (ERR_CONFIG_VARFILE/ERR_CONFIG_PARSE, exit 3), no panics/hangs.
Full evidence and detail: see the companion panel.
Companion detail panel: 20260729-2-fuzz-report/20260729-2-fuzz-report-boundary-probe.xhtml
20/20 passed. No findings. Confirms PR #165 is behavior-neutral for the rendering engine.
Target: full · Seed 261 · Iterations 20 · Passed 20 · Failed 0 · Result PASS.
Compared merge-base b763783 (baseline) vs head 749a3e0: confirmed the PR #165 diff touches only docs/examples/skill-template files, zero changes under crates/ or bindings/ -- the engine is byte-identical between the two refs for this diff.
20/20 exit-code and stdout matches across JSON/YAML var-file parity, jagged/nested/empty structures, numeric/boolean/null leaves, nested loops+conditionals, Unicode, includes, strict/unknown-var error paths, custom delimiters, and --all/--json flags. Fully deterministic across 3 reruns per case.
Confirms PR #165 is behavior-neutral for the rendering engine.
Full evidence and detail: see the companion panel.
Companion detail panel: 20260729-2-fuzz-report/20260729-2-fuzz-report-differential-probe.xhtml
crates/sc-compose/src/var_file.rs::parse_var_file_contents) as a genuine performance-NFR gap.