#!/usr/bin/env bash # Counts SSRF-guard regressions in PRODUCTION code. HARD GATE — a nonzero count # fails. # # WHAT THIS IS FOR # # The shared guard protects attacker-influenced egress across the application; # docs/SSRF_SECURITY.md describes its maintained security contract. This audit # is the mechanism that stops a new fetch site appearing unguarded, and it exists # because the ordinary test suite cannot identify every client construction. # # `.env.ci:140` sets IS_DEV_ENV=true, so `make ci` — the hermetic merge gate, # running T0+T1+T2 — takes the PERMISSIVE branch at every call site that holds an # allow-private boolean. A green merge gate is therefore compatible with every # guarded site being unguarded in production. The suite cannot see this; only a # grep over the source can. # # It is also a grep because human enumeration was tried and measured: the # remediation document's own list of call sites undercounted four separate times. # # THIS IS A SIBLING OF scripts/test-audit.sh, NOT A CATEGORY IN IT. That script # audits TEST code and says so in its scope note ("Production sources are out of # scope for every category except testing.Short()"). This one audits production # sources exclusively. The exemption mechanism, output format and failure # semantics are deliberately identical, so there is one convention to learn. # # THE FLOOR IS ZERO — with exemptions that are declared, not assumed. # # Every check here is a grep over text, and production code has legitimate # reasons to build an HTTP client: a fixed vendor API, the AppView's own # configured PDS, the guarded client's own construction. Those are exempted # individually, in the source, with a reason — so each one is a decision someone # made and can be found by grepping for the marker: # # client := &http.Client{...} // coves:allow-bare-client: # opts := PrivateHostOptions(true) // coves:allow-ssrf-hatch: # # and, where per-line markers would be noise because the whole file's subject # matter IS the thing being counted (the guard's own transport), a file-scope # form declared once near the top of the file: # # // coves:allow-bare-client-file: # # File-scope exemptions are printed on EVERY run with their reason and hit count, # so a broad exemption cannot go quiet. # # WHAT THE MARKER ACTUALLY REQUIRES, stated because each of these was once looser # than the paragraph above implied: # # * IT MUST BE IN A COMMENT, quote-aware. A marker inside a string literal used # to silence the rule, which is what happens when a log message or an error # string quotes the convention while explaining it. # * IT MUST CARRY A NON-EMPTY REASON. The bare token used to be enough. A # reason nobody has to write is a reason nobody writes, and the reason is the # only part of an entry a reviewer can disagree with. # * IT APPLIES ON ITS OWN LINE OR THE LINE BELOW, and only to its own category: # a justification about a bare client cannot license a resolver override. # * IT MUST STILL HAVE SOMETHING TO EXEMPT. An exemption outliving its # violation is a standing grant that no rule reports, and the next unguarded # client written under it inherits a reason composed about different code. # Rule 5 counts those. # # ONE CAVEAT THE NAMES HIDE: `coves:allow-bare-client` is a marker FAMILY, not a # per-rule marker. Rules 1, 1b, 1c, 1d, 1e, 1f and 1g all name it, so one marker # — and especially one `-file:` marker — exempts a file from every bare-client # rule at once, not from the one that fired. That is a real breadth, it is why # the file-scope table prints on every run, and it is why splitting a file is # usually the better fix than broadening its exemption. # # IT IS ALSO WHY 1c, 1e AND 1f REFUTE THE DEFAULT BY NAME. Those three fire on a # literal's OPENING line while the value they object to sits on a later line, so # a marker written about the bare client on that later line exempts rule 1 or 1b # and leaves the literal reading as remediated — one reason, written about one of # the two rules, silencing both. The refuted list is what keeps the second # question separate from the first. # # AN ALLOWLIST ENTRY IS A DOCUMENTED DECISION TO BE UNGUARDED. That is the whole # weight of the marker, and it is why an unlisted occurrence fails the build # rather than being snapshotted into a baseline: a wrong entry is worse than a # noisy audit, because it converts an oversight into a recorded choice nobody # revisits. # # THESE ARE TRIPWIRES, NOT PROOFS. A grep is bypassable by construction — a # client built by a helper in another package, a transport assigned after # construction. They catch drift cheaply. The actual guarantee is that each # converted site's own guard test refuses a private address. # # WHAT IS KNOWINGLY NOT COVERED, because a false claim of coverage is worse than # a documented hole — the hole gets read, the claim gets trusted: # # * PRODUCTION GO OUTSIDE cmd/ AND internal/. The corpus is those two roots and # nothing else. Today that leaves scripts/setup_dev_aggregator.go unscanned — # `go list` reports it as a real package, and it reaches the network twice # with http.Post — so this is a live hole and not a theoretical one. # # Widening the corpus is the obvious fix and cannot be done from this file: # the widened set immediately contains violations, and the markers that would # settle them have to be written in the files themselves. What IS enforced is # that the roots exist and hold production Go (rule 0), because the failure # that hides everything is not a file outside the corpus, it is a corpus of # no files reporting green over the whole tree. # # * A METHOD CALL ON AN ALREADY-BUILT CLIENT. `c.Do(req)` and `c.Get(url)` are # how every CONVERTED site uses its guarded client; today the tree holds 26 # `.Do(` and 141 `.Get(` calls in production code, almost all of them # correct. There is no text distinguishing a call on a guarded client from a # call on an unguarded one, so a rule here would be a permanent floor of ~167 # over exactly the code the remediation fixed — the count reviewers learn to # read past. # # THE SHAPES ARE CAUGHT AT CONSTRUCTION INSTEAD — but only the constructions # named here, and the claim is worth no more than its list. Rule 1 covers the # struct literal, new(), DefaultClient, the package-level helpers, and the # zero-value client however it is declared (`var c http.Client`, a struct # field, a parameter). Rule 1b covers the transport underneath and the # single-host proxy constructor, rule 1c the xrpc.Client that names no client # at all, rule 1e the reverse proxy assembled by hand, rule 1f the identity # directory whose client field is a VALUE (so omitting it compiles and yields # a zero-value client rather than a nil-pointer panic), rule 1g the four # library constructors that install http.DefaultClient for you, and rule 1d # the aliased import that renames past all of them. # # NOT COVERED AT CONSTRUCTION: any OTHER library type with an optional # `*http.Client`, `http.Client` or `Transport` field. The list in rules # 1c/1e/1f/1g is an ENUMERATION OF THIS TREE, not a rule that generalises — # it grew from two entries to seven because four more were already here when # someone looked, which is the honest measure of how well the enumeration # tracks the code. A type this list does not name is invisible. # # WHAT MAKES THAT SURVIVABLE IS THAT THE HATCH IS NOT ENUMERATED. Rule 3 asks # about the field assignment every hatch performs, so a new subsystem's guard # can be spelled however it likes and its OFF switch is still counted. The # enumerated rules are the tripwires; that one is the mechanism. # # * A CLIENT BUILT BY A HELPER IN ANOTHER PACKAGE and returned. Nothing at the # call site says what it is. Rule 1 catches the helper's own construction, # which is inside this tree; a client from a VENDORED dependency is not # visible to any rule here. # # * A TRANSPORT ASSIGNED AFTER CONSTRUCTION — `c.Transport = tr` where tr came # from elsewhere. # # The aliased-import case IS covered, by rule 1d, and not at the call site: the # call cannot be greppable (`nh.Get(url)` has no distinguishing text) but the # import that creates the alias must exist, and there is no reason in this tree # to rename net/http. # # Usage: # scripts/ssrf-audit.sh summary table; exits nonzero on any violation # scripts/ssrf-audit.sh -v table plus every offending file:line # # Environment: # SSRF_AUDIT_ROOT directory to scan instead of the repository root. # # This is a test seam, and it is the reason this script has tests at all. # An audit rule that cannot fail is the most useless artifact in the # repository, and asserting that a rule bites is not the same as watching it # bite; tests/audit plants a violating fixture under a temporary root and # requires a nonzero exit. It cannot weaken any rule — whatever root it # names is scanned by the identical pass the repository root goes through. set -uo pipefail if [[ -n ${SSRF_AUDIT_ROOT:-} ]]; then REPO_ROOT=$SSRF_AUDIT_ROOT else REPO_ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd) fi cd "$REPO_ROOT" || exit 2 VERBOSE=0 if [[ ${1:-} == "-v" || ${1:-} == "--verbose" ]]; then VERBOSE=1 fi CYAN='\033[36m' YELLOW='\033[33m' GREEN='\033[32m' RED='\033[31m' RESET='\033[0m' TOTAL=0 ALL_PATTERNS="" ALL_MARKERS="" declare -a ROWS=() declare -a DETAILS=() declare -a EXEMPT_FILES=() echo printf "${CYAN}═══════════════════════════════════════════════════════════════════════${RESET}\n" printf "${CYAN} SSRF GUARD REGRESSION AUDIT${RESET} (hard gate — any violation fails)\n" printf "${CYAN}═══════════════════════════════════════════════════════════════════════${RESET}\n" echo # --------------------------------------------------------------------------- # File scope # --------------------------------------------------------------------------- # Production code: every Go file under cmd/ and internal/ that is not a test. # # TEST FILES ARE OUT OF SCOPE, and not as a convenience. A guard test has to # stand up an httptest server and dial it, and httptest listens on LOOPBACK — # exactly the address class the guard refuses. Auditing test files here would # make the guard's own tests unwritable, and tests/ is where the hatch and the # resolver seam are SUPPOSED to appear. # # tests/ is excluded for the same reason even though tests/testkit holds # non-_test.go files: it is test support, and scripts/test-audit.sh already owns # that tree. PROD_ROOTS=(cmd internal) prod_code_files() { find "${PROD_ROOTS[@]}" -type f -name '*.go' 2>/dev/null | grep -v '_test\.go$' | sort -u } # --------------------------------------------------------------------------- # Reading a line the way Go reads it # --------------------------------------------------------------------------- # split_go_line code|comment # # Filters stdin, printing for each line either the code before its `//` comment # or the comment itself. Line count is preserved, so a caller can still address # lines by number after filtering. # # THE QUOTE TRACKING IS THE POINT, and it is why this is not `sed 's|//.*||'`. # Three separate checks in this script ask "does this line's TEXT contain X", # and text is not the same question as code: # # * `Client:` written in a COMMENT inside an xrpc.Client literal satisfied the # remediation check for rule 1c. An audit satisfied by a note about the fix # is measuring its own documentation — the same defect rule 4 shipped with, # where the Dockerfile's explanation of CGO_ENABLED=0 satisfied the check for # CGO_ENABLED=0. # * An exemption marker written inside a STRING silenced a rule, which is what # happens when a log line or an error message quotes the convention. # * `"https://example.com"` is in nearly every literal this script reads, so a # naive comment strip would cut a line in half at the URL's slashes and take # any brace after it with it — miscounting the literal. split_go_line() { awk -v mode="$1" ' function comment_at(s, i, n, c, q, inq, inraw) { n = length(s) for (i = 1; i <= n; i++) { c = substr(s, i, 1) if (inraw) { if (c == "`") { inraw = 0 }; continue } if (inq) { if (c == "\\") { i++; continue } if (c == q) { inq = 0 } continue } if (c == "`") { inraw = 1; continue } if (c == "\"" || c == "\047") { inq = 1; q = c; continue } if (c == "/" && substr(s, i + 1, 1) == "/") { return i } } return 0 } { at = comment_at($0) if (mode == "code") { print (at ? substr($0, 1, at - 1) : $0) } else { print (at ? substr($0, at) : "") } } ' } # marker_on_line # # True when carries IN A COMMENT and with a non-empty reason # after the colon. # # Both halves are load-bearing, and neither was checked. The old test was a # fixed-string grep for the marker without its trailing colon, anywhere on the # line — so `// coves:allow-bare-client` with nothing after it silenced the rule # as well as a justification would, and so did the same text inside a string. # # The reason is not decoration: the header stakes the whole allowlist on an entry # being a DOCUMENTED DECISION, and the reason is the only part of one a reviewer # can disagree with. A reason nobody has to write is a reason nobody writes. marker_on_line() { local file=$1 line=$2 marker=$3 [[ $line -ge 1 ]] || return 1 sed -n "${line}p" "$file" 2>/dev/null | split_go_line comment | grep -qE "${marker}:[[:space:]]*[^[:space:]]" } # --------------------------------------------------------------------------- # Scanning # --------------------------------------------------------------------------- # composite_literal_body # # Prints the composite literal that OPENS on , from that line through the # line where its braces balance again. Used by the fifth argument to scan, so a # rule can ask about the literal's contents rather than about one line's text. # # Brace counting rather than a fixed window, and rather than stopping at the # first closing brace, because these literals nest: every xrpc.Client in this # tree holds an &xrpc.AuthInfo{...} whose `},` closes several lines before the # outer one does. A window that stopped there would miss any field written after # it and report the remediated form as a violation — which is the failure that # teaches people the audit is wrong. # # COMMENTS ARE STRIPPED BEFORE THE LITERAL IS READ, which is the difference # between asking what a literal DOES and asking what it SAYS. `Client:` written # in a comment inside an xrpc.Client literal used to satisfy rule 1c's # remediation check, so the literal that merely mentions the fix passed as though # it had applied it. Stripping also keeps a brace inside a comment out of the # depth count. composite_literal_body() { local file=$1 start=$2 split_go_line code < "$file" 2>/dev/null | awk -v start="$start" ' NR < start { next } { body = $0 # Everything before the opening brace belongs to the enclosing # expression; counting it would start the depth in the wrong place. if (NR == start) { sub(/^[^{]*/, "", body) } print $0 depth += gsub(/\{/, "{", body) - gsub(/\}/, "}", body) if (depth <= 0) { exit } } ' } # scan