Fastly Next-Gen WAF (Signal Sciences) agent fails open before rules load in Kubernetes sidecar deployments (vendor disputes; CVE pending)
Full Disclosuremailing list archivesFrom: Zac Lohrenz <zaclohrenz () gmail com> 2026-11-19 04:19:37 Author: seclists.org(查看原文) 阅读量:12 收藏

fulldisclosure logo

Full Disclosure mailing list archives


From: Zac Lohrenz <zaclohrenz () gmail com>
Date: Tue, 17 Nov 2026 17:29:12 -0600

Fastly Next-Gen WAF (Signal Sciences) agent: fail-open before rules
load in Kubernetes sidecar deployments

Product:     Fastly Next-Gen WAF agent (sigsci-agent) with the NGINX
             module, Kubernetes sidecar deployment
Vendor:      Fastly, Inc.
Affected:    All agent versions in default configuration, through at
             least 4.83.2 (tested 2026-11-12). Before 4.52.0 no
             mitigation existed. 4.52.0 and later are affected unless
             the startup probe is configured.
Weakness:    CWE-636 Not Failing Securely ('Failing Open');
             CWE-1188 Initialization of a Resource with an Insecure
             Default; CWE-696 Incorrect Behavior Order
CVSS 4.0:    6.3 Medium
             CVSS:4.0/AV:N/AC:L/AT:P/PR:N/UI:N/VC:N/VI:L/VA:N/SC:N/SI:L/SA:N
CVE:         Requested from MITRE CNA-LR 2026-09-29; pending
Vendor
position:    Not a vulnerability; intended fail-open design. Will not
             change the default.
Mitigation:  Configure the agent startup probe (4.52.0+)
Researcher:  Zac Lohrenz
Writeup:     https://zaclohrenz.com/posts/cve/fastly-nextgen-waf/
Evidence:    https://github.com/zaclohrenz/fastly-ngwaf-poc


1. Summary
----------

In the Kubernetes sidecar deployment of Fastly's Next-Gen WAF (NGINX
ingress controller with the sigsci module, plus a sigsci-agent sidecar
sharing a Unix socket), the WAF passes traffic uninspected whenever
the agent has not loaded a ruleset, while the pod reports Ready.

  - Until the agent creates its RPC socket, the module cannot connect
    (errno=2), logs an error, and passes the request through.
  - The agent creates the socket when its first configuration
    download attempt ends, whether or not it succeeded. With no
    ruleset loaded, it answers every RPC.PreRequest with
    WAFResponse 200 and the header "X-SigSci-Agent: disabled", and
    keeps doing so until a later download succeeds.

Nothing in the default deployment ties pod readiness to rule load.
Every rollout, restart, eviction and scale-out creates a window, and
any startup failure to fetch configuration extends it for as long as
the failure lasts.


2. Lab testing
--------------

Tested 2026-11-12: sigsci-agent 4.83.2 (index digest
sha256:4aa2f5296b5f2eeb1a71e79643a93afb19ce85a635a21895ff35b2d0701578d2)
and sigsci-nginx-ingress-controller 1.15.1-fastly1.4.0 (index digest
sha256:a4c66bfe5c0b06c6507ce89881ba891711ab3009b2d5d934db192a91e0f452ba),
deployed with Fastly's published sigsci-values.yaml (commit 9794dfb) in
a local kind cluster. Generator at ~20 req/s including SQLi, XSS and
path traversal payloads; echo backend logging all request headers.
Black-box: logs, documented flags, the agent's documented RPC debug
dump, and HTTP headers. No Fastly credentials were available, so the
agent could never load a ruleset.

Phases observed:

  Phase                       Behavior                Default signal
  --------------------------  ----------------------  -----------------
  Socket not yet created      Module fails open       NGINX [error]
                                                      "cannot connect
                                                      uds socket" per
                                                      request
  Socket created, no ruleset  Agent answers           None in NGINX;
                              WAFResponse 200 +       agent logs only
                              X-SigSci-Agent:         "Config update
                              disabled                failed" ~every 30s

Config-download failure types tested, 3 runs each: connection drop,
HTTP 403 (rejected credentials), DNS NXDOMAIN, TCP reset. All four
produced the same state: 20,304 decoded agent responses, one distinct
value (allow, marked disabled). The state persisted with retries every
~30 s for as long as configuration could not be loaded.

With slow failures (dropped connections) the socket appeared after
~60 s and ~935 requests per run passed during the no-socket phase.
With fast failures it appeared within ~1.7 s. The 60 s figure is an
artifact of how egress was blocked, not a typical duration.

Mitigations tested:

  - Startup probe (SIGSCI_STARTUP_PROBE_LISTENER + startupProbe): pod
    held out of Service endpoints, zero requests to the upstream,
    3 of 3 runs.
  - Readiness probe initialDelaySeconds 10/30/60 (vendor-suggested
    alternative): traffic, including attack payloads, reached the
    upstream after each delay expired.
  - No documented module or agent option made either component fail
    closed. The experimental debug-standalone=4 removed the
    no-socket phase and the "disabled" marker but still allowed every
    request.

For contrast, freezing the agent after its socket existed (the
documented timeout path) made the module fail open after ~320 ms per
request, also with no log line at default level. The vendor documents
the timeout behavior; it documents nothing about an agent that answers
with no ruleset loaded.

Per-request evidence of the disabled state appears only with
non-default options, e.g. SIGSCI_STATSD_ADDRESS (counter
sigsci.agent.rpc.tx.dropped.engine_disabled) or
SIGSCI_DEBUG_LOG_UPLOADS (AgentEnabled:0).

Limits: without working credentials, a healthy start was not observed
(whether a valid agent ever opens its socket before its first ruleset
loads, the probe's 503 -> 200 transition, and how these requests appear
in the vendor console). The finding is about failure at startup: in
every failure tested, an agent that could not load its configuration
allowed everything until it could, and the default deployment gave
Kubernetes no readiness signal for it.

Full report, manifests, scripts and raw evidence with SHA-256
checksums: https://github.com/zaclohrenz/fastly-ngwaf-poc


3. Detection
------------

  - Agent container spec: look for SIGSCI_STARTUP_PROBE_LISTENER or
    SIGSCI_STARTUP_PROBE_FILEPATH AND a startupProbe on the agent
    container that checks it. If either is missing, the deployment is
    exposed. The env var alone does nothing.
  - Backend logs: search request headers for "X-SigSci-Agent:
    disabled". Requests that passed while the socket was missing carry
    no X-SigSci-* header and will not show up this way.
  - Agent logs: "Config update failed" near pod starts.
  - Ingress controller logs: "cannot connect uds socket" from the
    sigsci module.


4. Mitigation
-------------

On agent 4.52.0 or later, configure the startup probe:

  containers:
    - name: sigsci-agent
      ports:
        - name: "startup-port"
          containerPort: 2024
      env:
        - name: SIGSCI_STARTUP_PROBE_LISTENER
          value: "0.0.0.0:2024"
      startupProbe:
        httpGet:
          path: /startup
          port: startup-port
        failureThreshold: 90
        periodSeconds: 2

The endpoint returns 503 until rules are loaded. Size failureThreshold
x periodSeconds for the slowest realistic rule fetch; a pod that cannot
load rules within that time is restarted by the kubelet.

Fastly's published Helm values (sigsci-values.yaml and
sigsci-nginxinc-values.yaml, commit 9794dfb, 2026-11-10) deploy the
agent with no startup probe or readiness gate, and no version in that
repository's history has included one. The vendor's docs-site
Kubernetes guides were updated in October 2026 to include the probe.


5. Vendor position
------------------

Fastly's replies, quoted:

  2026-10-14: Fastly "does not classify this behavior as a
  vulnerability"; fail-open is "a deliberate, foundational design
  choice in the Fastly NGWAF architecture designed to prioritize
  application availability and continuous traffic flow in the event of
  component unavailability, timeouts, or uninitialized states." The
  same reply: "we entirely agree with your assessment of the security
  implications in a dynamically scaling Kubernetes environment."

  2026-10-22: "our install guides and suggested configs don't include"
  the startup probe; a CVE is not the right fit "since the right
  default varies a lot by deployment type (fail-closed at startup is
  ideal for some and an outage for others)."

  2026-10-29: not a vulnerability under CNA Operational Rules v4.2.6,
  citing 4.1.3.4 ("CNAs MAY determine consistently well-documented and
  widely known default configurations to not be Vulnerabilities");
  "We won't change the default fail-open behavior in situations where
  the agent is not available yet or is experiencing other issues."

  2026-11-03: discussion moved to the CNA-LR; Fastly reserves the right
  to dispute there. Agent versions 4.52.0 and earlier are past its
  two-year support cycle.

  2026-11-17: case closed; Fastly will provide its analysis directly
  to the CNA-LR if the CVE request proceeds.

Researcher position, in brief: CNA rule 4.1.3.2 is a SHOULD for
defaults that introduce under-documented or unexpected security
behavior, and 4.1.3.3 a SHOULD for defaults that enable direct
exploitation; 4.1.3.4 is a MAY. The vendor's reliability documentation
covers module fail-open on agent timeout but does not describe an agent
answering allow with no ruleset loaded. The vendor stated the agent
"still performs preliminary request handling" while loading its initial
ruleset, but did not answer two written questions (2026-10-30) about
what the agent returns in that state and how operators can detect it.
The lab results above answer them as far as possible without
credentials.


6. Timeline
-----------

  Early 2024   Reported to Fastly support (prior employer's account)
  2024-03-14   Agent 4.52.0: startup probe options GA; default
               unchanged
  2026-04-12   Initial public writeup
  2026-09-24   Emailed Fastly security requesting classification and
               CVE coordination; no acknowledgment
  2026-09-29   CVE requested from MITRE CNA-LR
  2026-10-06   Report forwarded to Fastly support; case opened
  2026-10-14   Fastly: not a vulnerability; agrees with security
               implications
  2026-10-22   Fastly: guides omit probe; CVE not the right fit
  2026-10-29   Fastly final position: not a vulnerability; default
               will not change
  2026-11-03   Fastly moves discussion to CNA-LR; questions unanswered
  2026-11-12   Lab testing on agent 4.83.2
  2026-11-17   Fastly closes case; details published
  Pending      CVE assignment


7. References
-------------

  Writeup:
  https://zaclohrenz.com/posts/cve/fastly-nextgen-waf/
  Evidence:
  https://github.com/zaclohrenz/fastly-ngwaf-poc
  Agent 4.52.0 changelog:
  https://www.fastly.com/documentation/reference/changes/2024/03/ngwaf-agent-4.52.0/
  Kubernetes startup probe guide:
  https://www.fastly.com/documentation/guides/next-gen-waf/setup-and-configuration/kubernetes/kubernetes-startup-probe/
  Agent reliability documentation:
  
https://www.fastly.com/documentation/guides/next-gen-waf/setup-and-configuration/agent-management/getting-started-with-the-agent/
  sigsci-values.yaml at 9794dfb:
  
https://github.com/signalsciences/sigsci-nginx-ingress-controller/blob/9794dfb38747fb7ec68e4e9286e3164debf1e665/sigsci-values.yaml
  CNA Operational Rules v4.2.6:
  https://www.cve.org/Resources/Roles/Cnas/CNA_Rules_v4.2.6.pdf

--
Zac Lohrenz
_______________________________________________
Sent through the Full Disclosure mailing list
https://nmap.org/mailman/listinfo/fulldisclosure
Web Archives & RSS: https://seclists.org/fulldisclosure/


Current thread:

  • Fastly Next-Gen WAF (Signal Sciences) agent fails open before rules load in Kubernetes sidecar deployments (vendor disputes; CVE pending) Zac Lohrenz (Nov 18)

文章来源: https://seclists.org/fulldisclosure/2026/Nov/40
如有侵权请联系:admin#unsafe.sh