Adversarial Fuzz Session qa-fuzz-20260729-0001

sc-compose bounded adversarial-fuzzing campaign -- QA validation of PR #165 report-generation wiring (step 10)

Generated: 2026-07-29

Source: adversarial-fuzzing skill v1.1.0 / quality-mgr QA execution

Overall status: DRIFT

Summary

Fuzz run descriptionIterationsPassResult
Rendered a purpose-built template against 20 bounded JSON/YAML var-files (recursive trees, mixed-type arrays, empty values, numeric edge cases, deep nesting, unicode, YAML anchors, metamorphic extra-field check) via `sc-compose render --var-file`.2019/20FAIL
Jinja rendering probe across example templates and synthetic .j2 fixtures covering nested loops/conditionals, @<path> includes (cycle/escape/recursive), custom brace-count and variable-delimiter modes, whitespace control, Unicode content, optional/missing fields, and the loop.* Jinja builtin under default and --strict validation.1816/18FAIL
Negative-contract probe of sc-compose render/validate/resolve CLI: malformed template/var-file syntax, non-object top-level var-files, invalid YAML keys, invalid profile/agent names, and --root path-confinement attempts via --file/--var-file traversal, absolute escapes, and traversal in --root itself.2221/22FAIL
JSON/YAML parity, 3x determinism, and metamorphic (extra-field, key-reorder) checks against jagged-array-values.md.j2, changelog-categories.md.j2, and frontmatter-demo.md.j2, both in default and --json render modes.2020/20PASS

Campaign qa-fuzz-20260729-0001: 4 workers, seed 157, worktree ./feature/fuzz-run-report-template at commit 111ac8a (branch feature/fuzz-run-report-template, PR #165). 2 confirmed bug(s), 1 inconclusive finding(s), 0 unhandled worker failures.

Worker: shape-probe DRIFT

DRIFT · confirmed_bug

19/20 cases rendered without panic/hang/timeout and with values matching expectations, including a metamorphic pass (unused extra top-level field produced byte-identical output). 1 case (YAML merge-key `<<: *anchor` inside an array-of-objects var-file) silently rendered empty id/value fields with zero diagnostics instead of the merged values or an explicit error, reproduced 3/3 times on a minimized template+input.

Fuzz run descriptionIterationsPassResult
Rendered a purpose-built template against 20 bounded JSON/YAML var-files (recursive trees, mixed-type arrays, empty values, numeric edge cases, deep nesting, unicode, YAML anchors, metamorphic extra-field check) via `sc-compose render --var-file`.2019/20FAIL

Inputs exercised

CaseTemplate / inputOutcome
shape-01case00-baseline.json: flat baseline (strings, tags array, deep dotted meta, jagged matrix, empties, mixed-sign floats/ints, items[])PASS
shape-02case01-huge-int.json: i64::MAX/MIN and u64::MAX-class integersPASS
shape-03case02-sci-notation.json: 1e300, -1e-300, 6.022e23, 1.0E10, -0.0/0.0PASS
shape-04case03-deep-nesting.json: 8-level-deep nested objectPASS
shape-05case04-mixed-types.json: arrays mixing int, string, bool, null, float, numeric-string, nested list/objectPASS
shape-06case05-all-empty.json: every field is its type's empty valuePASS
shape-07case06-jagged-empty-rows.json: matrix rows of varying length including zero-length rowsPASS
shape-08case07-yaml-parity.yaml: YAML equivalent of baseline JSON, diffed for parityPASS
shape-09case08-yaml-anchors.yaml: YAML anchor + merge key (<<: *defaults) inside items[]FAIL
shape-10case09-negatives.json: -0.0, i64::MIN, -1e-300, f64::MINPASS
shape-11case10-unicode.json: emoji, CJK, Cyrillic, Arabic, combining diacriticsPASS
shape-12case11-nulls-in-arrays.json: null interspersed inside arraysPASS
shape-13case12-booleans.json: true/false interspersed inside arraysPASS
shape-14case13-objects-in-matrix.json: matrix rows containing objects/nested arraysPASS
shape-15case14-tags-as-object.json: top-level tags is an object, iterated in a for-loopPASS
shape-16case15-large-array.json: tags array with 500 string elementsPASS
shape-17case16-deep-lists.json: unused 20-level-deep nested list fieldPASS
shape-18case17-yaml-flow-deep.yaml: YAML flow-style deep nested mappingPASS
shape-19case18-meta-level3-is-array.json: nested field is an array containing a nested objectPASS
shape-20case19-metamorphic-extra-field.json: baseline plus one unused top-level field, diffed byte-for-bytePASS

Findings

FUZZ-SHAPE-001

Minimal template
--- required_variables: - items --- {% for it in items +%} {{ it.id }}: {{ it.value }} {% endfor %}
Input
base: &base id: 1 value: anchored items: - <<: *base
Expected
Either (a) YAML 1.1 merge-key semantics resolve <<: *base into id: 1, value: anchored before the value reaches the template, or (b) sc-compose emits an explicit diagnostic flagging the unsupported << construct instead of silently producing an object with missing fields.
Observed
Renders ': ' (both it.id and it.value empty) with exit code 0, zero diagnostics in --json output, and no warning/error logged. The literal << key and its anchor-merged fields are silently dropped instead of being merged or rejected.
Requirement / ADR
No requirement or ADR currently covers this behavior. docs/requirements.md and the CLI help only state that --var-file loads input variables from a JSON or YAML object file without scoping which YAML 1.1 features (anchors, aliases, merge keys) are in or out of contract.
Requirement / ADR follow-up
create/update: add an explicit statement to docs/requirements.md (var-file ingestion) on YAML feature support, and either implement merge-key resolution or add a validation diagnostic when a literal << mapping key is observed post-parse, since serde_yaml 0.9 is documented upstream as not implementing YAML merge keys.
Root cause
serde_yaml 0.9 (pinned in Cargo.toml, used by parse_var_file_contents in crates/sc-compose/src/var_file.rs) does not implement YAML 1.1 merge-key (<<) resolution -- a known, unfixed upstream limitation. sc-compose passes the resulting mapping straight through with the literal << key intact and no validation warning.
Recommended fix
Either implement merge-key expansion as a var-file post-processing step before validation/rendering, or add a structural check in var_file.rs that flags any mapping containing a literal << key and emits a warning/error diagnostic (e.g. ERR_VAL_UNSUPPORTED_YAML_MERGE_KEY).

XHTML panel validation (xmllint --noout): FAILED (invalid XML: unescaped literal '<', '&' characters from fuzz command/YAML content, e.g. '<<: *anchor')

Worker: template-probe DRIFT

DRIFT · confirmed_bug

Nested loops/conditionals, recursive @<path> includes, include-cycle/escape boundary rejection, custom --brace-count and --variable-delimiters modes, whitespace control, Unicode variable values/content, and optional-field defaulting all behaved correctly against the documented contract. Two cases (same root cause) showed validation falsely flagging the standard Jinja `loop` context object (loop.last, loop.index, etc.) as an undeclared referenced token inside for-loops: a warning by default, but a hard validation/render failure under --strict.

Fuzz run descriptionIterationsPassResult
Jinja rendering probe across example templates and synthetic .j2 fixtures covering nested loops/conditionals, @<path> includes (cycle/escape/recursive), custom brace-count and variable-delimiter modes, whitespace control, Unicode content, optional/missing fields, and the loop.* Jinja builtin under default and --strict validation.1816/18FAIL

Inputs exercised

CaseTemplate / inputOutcome
tmpl-01render examples/jagged-array-values.md.j2 --var-file examples/jagged-array-values.sample-vars.jsonPASS
tmpl-02render examples/changelog-categories.md.j2 --var-file examples/changelog-categories.sample-vars.jsonPASS
tmpl-05synthetic nested.md.j2: nested for-loop over groups[].items[] with nested if/else and an empty inner array groupPASS
tmpl-06synthetic include-root.md.j2 -> @<parts/body.md> -> @<inner.md>, two-level recursive includePASS
tmpl-07cycle-a.md.j2 @<cycle-b.md> @<cycle-a.md.j2>: include cyclePASS (correctly rejected, ERR_INCLUDE_CYCLE)
tmpl-08escape.md.j2 @<../outside-escape.md>: include path escaping confinement rootPASS (correctly rejected, ERR_INCLUDE_ESCAPE)
tmpl-09render brace3.md.j2 --brace-count 3 --var name=Rand, body 'Hello {{{ name }}}!'PASS
tmpl-10render customdelim.md.j2 --variable-delimiters '<<' '>>' --var name=RandPASS
tmpl-11ws.md.j2 whitespace-control combination, cross-checked against FR-7c semanticsPASS
tmpl-lstriplstrip.md.j2: indented if/endif tags on their own line, verifying lstrip_blocksPASS
tmpl-12unicode.md.j2 with Omega/CJK/Arabic RTL/emoji mixed contentPASS
tmpl-13examples/agent-task-branching.xml.j2 mode=plan with plan_notes omitted (frontmatter default)PASS
tmpl-14examples/agent-task-branching.xml.j2 mode=bogus (unhandled value falls to else branch)PASS
tmpl-17real-missing.md.j2 referencing {{ role }}, never declared, --strict --jsonPASS (correctly errors ERR_VAL_UNDECLARED_TOKEN)
tmpl-15minimal-loop.md.j2: {% for x in items %}{{ loop.last }}{% endfor %}, run 3x with --strict --jsonFAIL
tmpl-16loop-index.md.j2: {% for x in items %}{{ loop.index }}:{{ x }} {% endfor %}, default (non-strict) --jsonFAIL
tmpl-18loop-customdelim2.md.j2 with --variable-delimiters still using default statement tags and loop.last inside ifFAIL (same false-positive diagnostic reproduces independent of expression-delimiter mode)
tmpl-strict-contrastsame minimal-loop.md.j2 run without --strictPASS for rendered bytes, but still emits the false warning

Findings

FUZZ-TP-01

Minimal template
--- required_variables: - items --- {% for x in items %}{{ loop.last }}{% endfor %}
Input
{"items": ["a", "b", "c"]}
Expected
docs/requirements.md FR-1035 requires full Jinja for-loop support; the Jinja `loop` context object (loop.last, loop.first, loop.index, loop.index0, loop.length, loop.revindex) is a language-level construct auto-injected by the for-loop, not a caller-supplied or frontmatter-declared token, so it should never be classified as an undeclared referenced token under FR-2a/FR-2c.
Observed
Default mode: render succeeds but emits {"code":"ERR_VAL_UNDECLARED_TOKEN","severity":"warning","message":"undeclared referenced token: loop.last"} (also reproduced for loop.index). Under --strict: same diagnostic promoted to severity error, process exits 2, no output rendered -- reproduced 3/3, and again independently with custom --variable-delimiters, confirming the bug is in identifier discovery, not delimiter-mode specific.
Requirement / ADR
docs/requirements.md FR-1035 (Jinja for-loop field access) and FR-2c (built-in render-context variable list, from which `loop` is absent because it is not a render-context variable, it is a Jinja loop-scope construct). FR-2a governs undeclared referenced tokens, which by its own framing means caller-facing variable names, not engine-internal loop state.
Requirement / ADR follow-up
Update FR-2a to explicitly state that the Jinja `loop` object and its standard attributes are excluded from undeclared-token discovery in every for-loop scope, mirroring how validation.rs::parse_for_loop_scope already excludes the loop's own bound iteration variable(s). No new ADR needed -- this is a bug-fix-sized correction to an existing requirement's implementation.
Root cause
In crates/sc-composer/src/validation.rs, parse_for_loop_scope (~line 904) only adds the for-loop's bound iteration variable name(s) to LoopScope.bound_names; it never adds the implicit `loop` identifier Jinja injects into every for-loop body. collect_identifiers (~line 930) checks bound_names and a fixed KEYWORDS list (~line 935) for exclusion, and `loop` appears in neither, so loop.last/loop.index get treated as ordinary undeclared caller tokens.
Recommended fix
In parse_for_loop_scope, unconditionally push "loop" into the returned LoopScope.bound_names set alongside the iteration binding name(s), so collect_identifiers treats any loop.<attr> reference inside that for-loop's lexical scope as bound/declared rather than an undeclared caller token. Add a regression test asserting discover_tokens_with_brace_count returns an empty set for a template body using loop.last/loop.index, plus a CLI-level test asserting render --strict succeeds for the same template.

XHTML panel validation (xmllint --noout): FAILED (invalid XML: unescaped literal '<', '@<path>' include syntax and '<<' delimiter examples in test-input descriptions)

Worker: boundary-probe INFO

INFO · inconclusive

Malformed template syntax, non-object/invalid var-files, invalid YAML keys, invalid profile/agent names, non-UTF-8 templates, and nonexistent roots all produced stable non-zero exits with structured ERR_* diagnostics and no panics or hangs -- correct boundary behavior. --file path traversal and absolute-path escapes outside --root were correctly rejected with ERR_RESOLVE_NOT_FOUND. One asymmetry was found and reproduced 3/3 times: --var-file performs no root confinement at all -- both relative traversal and absolute paths outside --root are read and rendered successfully (exit 0), unlike --file, which is strictly confined.

Fuzz run descriptionIterationsPassResult
Negative-contract probe of sc-compose render/validate/resolve CLI: malformed template/var-file syntax, non-object top-level var-files, invalid YAML keys, invalid profile/agent names, and --root path-confinement attempts via --file/--var-file traversal, absolute escapes, and traversal in --root itself.2221/22FAIL

Inputs exercised

CaseTemplate / inputOutcome
bound-01render --file bad_syntax.md.j2 (unclosed {{ tag) --var-file vars_ok.yaml --jsonPASS
bound-02render --file tpl.md.j2 --var-file vars_array.json (JSON array top-level) --jsonPASS
bound-03render --file tpl.md.j2 --var-file vars_scalar.json (JSON scalar top-level) --jsonPASS
bound-04render --file tpl.md.j2 --var-file vars_array.yaml (YAML array top-level) --jsonPASS
bound-05render --file tpl.md.j2 --var-file vars_scalar.yaml (YAML scalar top-level) --jsonPASS
bound-06render --file tpl.md.j2 --var-file vars_bad.yaml (unclosed flow seq) --jsonPASS
bound-07render --file tpl.md.j2 --var-file vars_bad.json (trailing comma, parsed leniently as YAML) --jsonPASS
bound-08render --file tpl.md.j2 --var-file vars_intkeys.yaml (non-string YAML keys) --jsonPASS
bound-09render --file tpl.md.j2 --var '123bad=value' --jsonPASS
bound-10render --file tpl.md.j2 --var 'novalue' --json (missing =)PASS
bound-11render --file tpl.md.j2 --var '=value' --json (empty key)PASS
bound-12resolve --mode profile --kind agent --agent '../../etc/passwd' --root <root> --jsonPASS
bound-13resolve --mode profile --kind agent --agent 'a/b;rm -rf' --root <root> --jsonPASS
bound-14render --root <root> --file ../outside/secret.md.j2 --json (relative traversal)PASS
bound-15render --root <root> --file <abs-outside>/secret.md.j2 --json (absolute escape)PASS
bound-16bcd <root> && render --root . --file tpl.md.j2 --var-file ../outside/secret_vars.yaml --jsonFAIL
bound-17brender --root <root> --file tpl.md.j2 --var-file <abs-outside>/secret_vars.yamlFAIL
bound-18validate --mode profile --kind bogus --agent foo --root <root> --jsonPASS
bound-19render --file tpl.md.j2 --var-file vars_empty.json --json (empty file)PASS
bound-20render --file binary.md.j2 --var-file vars_ok.yaml --json (non-UTF-8 bytes)PASS
bound-21render --root <root>/sub/.. --file tpl.md.j2 --var-file vars_ok.yaml --jsonPASS
bound-22render --root <root>/does_not_exist --file tpl.md.j2 --json (nonexistent root)PASS

Findings

FUZZ-BOUND-001

Minimal template
hello {{ name }}
Input
name: LEAKED-OUTSIDE-VALUE (located outside ROOT, at ROOT/../outside/secret_vars.yaml)
Expected
Ambiguous. The CLI help text for --root reads 'Workspace root for resolution and confinement' (unqualified), and docs/architecture.md states the include module enforces path confinement, but neither docs/requirements.md FR-7a nor the CLI option list states that --var-file paths are, or are not, subject to --root confinement. No test asserts var-file confinement either way, so no oracle currently exists to call this a defect versus an intentional caller-supplied-path exemption (analogous to --output, --guidance-file, --prompt-file).
Observed
Exit 0. Output: 'hello LEAKED-OUTSIDE-VALUE'. The var-file located outside --root, via both relative traversal and an absolute path outside root, is read and its values rendered without any path-confinement check. Reproduced 3/3 identical runs. crates/sc-compose/src/var_file.rs::load_var_file calls std::fs::read_to_string(path) directly with no confinement/root check, in contrast to --file (crates/sc-composer/src/resolver.rs) and @include expansion (crates/sc-composer/src/include.rs), both of which explicitly reject paths escaping the confinement root.
Requirement / ADR
No requirement or ADR currently covers this behavior. docs/requirements.md FR-7a describes --var-file's accepted value shapes but is silent on path confinement; docs/architecture.md's confinement language is scoped explicitly to the include module and --file/template resolution.
Requirement / ADR follow-up
Recommend the requirement owner decide and document explicitly whether --var-file (and likewise --guidance-file/--prompt-file, not probed here) is intended to be confined to --root like --file, or is an intentionally unconfined caller-supplied auxiliary input path. Until that decision is recorded, treat the --root help text as potentially misleading to users who assume it confines all file-path options.
Root cause
crates/sc-compose/src/var_file.rs::load_var_file reads the --var-file path via std::fs::read_to_string with no call into the ConfinementRoot / resolver confinement logic used for --file and @include (crates/sc-composer/src/types.rs ConfinementRoot, crates/sc-composer/src/resolver.rs, crates/sc-composer/src/include.rs). This is evidence-backed by direct source read, not inferred.
Recommended fix
Not recommending a code change during this fuzz probe (out of scope). Next action is a requirement/ADR decision; if var-file confinement is deemed in-scope, add a ConfinementRoot check in load_var_file mirroring resolver.rs's escape handling, plus CLI/boundary tests.

XHTML panel validation (xmllint --noout): FAILED (invalid XML: unescaped literal '<root>', '<abs-outside>' placeholders and '&&' in command strings)

Worker: differential-probe PASS

PASS · pass

20 bounded cases across three real example templates: JSON-vs-YAML var-file parity (plain and --json output mode), 3x repeated identical renders for determinism, and metamorphic checks (unused top-level/nested field addition, top-level and item-object key reordering, explicit vs default --unknown-var-mode ignore, --var CLI overrides vs equivalent var-file). All 20 cases matched the expected oracle byte-for-byte; a control case (reordering array items, where order is semantically load-bearing) correctly produced different output, confirming the oracle discriminates real changes from noise. No parity, determinism, or metamorphic violations found.

Fuzz run descriptionIterationsPassResult
JSON/YAML parity, 3x determinism, and metamorphic (extra-field, key-reorder) checks against jagged-array-values.md.j2, changelog-categories.md.j2, and frontmatter-demo.md.j2, both in default and --json render modes.2020/20PASS

Inputs exercised

CaseTemplate / inputOutcome
diff-01render jagged-array-values.md.j2 with sample JSON var-file vs equivalent YAML var-filePASS
diff-02render changelog-categories.md.j2 with sample JSON var-file vs equivalent YAML var-filePASS
diff-03render frontmatter-demo.md.j2 with JSON var-file vs equivalent YAML var-filePASS
diff-04render jagged-array-values.md.j2 3x identically, diff all pairsPASS
diff-05render changelog-categories.md.j2 3x identically, diff all pairsPASS
diff-06render frontmatter-demo.md.j2 3x identically, diff all pairsPASS
diff-07jagged var-file + unused top-level field vs baseline (default --unknown-var-mode ignore)PASS
diff-08changelog var-file + unused top-level field vs baselinePASS
diff-09frontmatter var-file + unused top-level field vs baselinePASS
diff-10changelog var-file with top-level keys reordered vs baseline order, JSONPASS
diff-11changelog var-file with top-level keys reordered, YAML, vs baseline JSON outputPASS
diff-12frontmatter var-file with top-level keys reordered vs baselinePASS
diff-13jagged var-file with rows/note key order swapped (two variants) vs each otherPASS
diff-14changelog var-file with unused field nested inside an items[] object vs baselinePASS
diff-15control case: changelog var-file with items[] array order swapped (semantically load-bearing) vs baseline -- expected to differPASS
diff-16explicit --unknown-var-mode ignore vs default (no flag) on same var-filePASS
diff-17three --var key=value CLI overrides vs equivalent single --var-file for frontmatter-demo.md.j2PASS
diff-18render changelog-categories.md.j2 --json 3x identically, diff all pairsPASS
diff-19jagged JSON vs YAML var-file parity under --json output modePASS
diff-20render frontmatter-demo.md.j2 --json 3x identically, diff all pairsPASS

XHTML panel validation (xmllint --noout): PASSED (xmllint --noout clean; this worker's content happened not to contain '<' or '&')

Recommendations

Session Metadata

campaign_idqa-fuzz-20260729-0001
worktree_path./feature/fuzz-run-report-template
commit111ac8a
pr165
seed157
max_workers4
cases_per_worker20
per_worker_timeout_s120