* fix(relay): bound the wait for upstream response headers (fixes unbounded heap growth)
The relay transport sets a dial timeout, a TLS handshake timeout and an expect-continue
timeout, but nothing bounds how long it waits for the upstream *response headers* after
the request has been written. An upstream that accepts the connection and then never
answers -- without sending FIN/RST, which is what happens when a NAT/firewall silently
drops the flow or the provider hangs -- parks the goroutine in
net/http.(*persistConn).roundTrip forever.
That goroutine keeps the whole request alive, which in practice means three copies of the
request body stay reachable for the lifetime of the process: the raw bytes from
io.ReadAll in CreateBodyStorageFromReader, the decoded messages held as json.RawMessage,
and the re-marshalled upstream body from common.Marshal. BodyStorageCleanup cannot help
here: it runs after c.Next() returns, and for these requests c.Next() never returns.
Measured on v1.0.0-rc.23 in production (see #6947 for the full evidence):
- 23 goroutines stuck in persistConn.roundTrip on a single 40h-old instance,
blocked between 353 and 1894 minutes (5.9h to 31.5h)
- 96.9% of the live heap, sampled after a forced GC, attributable to those three
body copies (HeapAlloc 892 MiB surviving three GC cycles; HeapObjects dropping
30x while bytes dropped only 25%)
- the live floor grows with uptime: 33.7 MiB at 0.1h, 89.2 at 13.8h, 510.0 at 40.1h,
955.2 at 146.8h, OOMKilled at 172.9h -- same image, same config, same load
Doubling the memory limit and adding GOMEMLIMIT only moved the OOM from 132h to 172.9h.
RELAY_TIMEOUT (http.Client.Timeout) cannot be used for this: it covers the whole response
read and would cut legitimate long streaming calls, which is why it defaults to 0.
ResponseHeaderTimeout only bounds the wait for the headers; streaming after they arrive is
unaffected.
The default is deliberately generous. Non-streaming upstreams usually send the response
headers only once generation has finished, so the value has to leave room for a long
completion. 1800s is 12x shorter than the shortest hang observed here while leaving
several times the headroom a normal non-streaming request needs; 0 restores the previous
unbounded behaviour.
The assignment goes next to the other transport.* lines rather than inside the else
branch: newRelayHTTPTransport() normally takes the http.DefaultTransport.Clone() path,
and DefaultTransport does not set ResponseHeaderTimeout either.
This repo already sets ResponseHeaderTimeout on its other outbound transports
(controller/model_sync.go, controller/ratio_sync.go); the relay path appears to have
been missed.
Refs #6947. Likely also the root cause of #6731, which reported the same symptom
(production OOM on /v1/responses after ~64h) but was closed for template reasons.
* review: clamp overflowing timeout values and switch the test to testify
Addresses the two CodeRabbit findings on this PR.
Overflow (common/init.go:113): a RELAY_RESPONSE_HEADER_TIMEOUT beyond ~9.2e9 seconds
overflows time.Duration and can wrap into a *tiny positive* timeout, which would cut
every relay request instead of only the stuck ones. The value is now clamped before the
conversion, with regression tests for both the negative and the overflowing input.
I did not add fail-on-startup validation for negative values, for two reasons: the
existing `if seconds > 0` guard already treats them as "disabled", and the neighbouring
env-driven timeouts in this file are less strict still -- RelayIdleConnTimeout is
converted with no guard at all. Failing startup on a bad value would be a behaviour
change out of step with the rest of the file; happy to add it if you'd prefer that
direction repo-wide.
Test style: switched to testify (require.Equal / require.Zero / require.Positive), which
is what every other test under service/ uses.
go build, go vet and go test ./common/... ./service/... pass.
(`go build ./...` fails on the `web/dist` embed both with and without this change -- the
frontend bundle is not checked in.)
- add SESSION_COOKIE_SECURE / SESSION_COOKIE_TRUSTED_URL env vars with
startup validation: enabling Secure requires at least one trusted
HTTPS entry URL
- wire common.SessionCookieSecure into the session cookie store instead
of a hardcoded Secure=false
- print a startup warning when Secure session cookies are disabled
- document the new settings in .env.example and docker-compose files
Secure stays off by default because many deployments front new-api with
plain-HTTP reverse proxies, where a hardcoded Secure default would break
logins entirely; enabling it safely depends on the deployment's TLS
setup, so it ships as an opt-in deployment-hardening flag.
* fix: support SMTP STARTTLS mode and NTLM auth
Add explicit SMTP STARTTLS configuration for 587-style connections and keep SSL/TLS as the implicit TLS mode.
Prefer PLAIN when advertised, keep LOGIN compatibility, and add NTLM as a fallback for Exchange SMTP servers that require it after STARTTLS.
* fix: respect explicit SMTP encryption mode
* fix: preserve SMTP TLS compatibility
Fall back to the OS hostname when NODE_NAME is not configured, so node
identity in audit/usage logs is populated automatically. In container and
Kubernetes deployments the hostname equals the container ID or Pod name,
which stays unique under autoscaling without any manual per-instance config.
Route OpenAI image streaming through shared stream handling, split image/realtime/usage helpers for maintainability, and include the related image request and rate limit updates.
Introduce NODE_NAME environment variable to identify node identity in top-up
audit logs, improving readability over auto-detected container internal IPs
in Docker/K8s deployments. Surface node_name in admin expanded log rows and
add it as a commented example to docker-compose.yml.
- Added TaskTimeoutMinutes constant to configure the timeout duration for asynchronous tasks.
- Implemented sweepTimedOutTasks function to identify and handle unfinished tasks that exceed the timeout limit, marking them as failed and processing refunds if applicable.
- Enhanced task polling loop to include the new timeout handling logic, ensuring timely cleanup of stale tasks.
Clamp request body size (including post-decompression) to avoid memory exhaustion caused by huge payloads/zip bombs, especially with large-context Claude requests. Add a configurable `MAX_REQUEST_BODY_MB` (default `32`) and document it.
- Enforce max request body size after gzip/br decompression via `http.MaxBytesReader`
- Add a secondary size guard in `common.GetRequestBody` and cache-safe handling
- Return **413 Request Entity Too Large** on oversized bodies in relay entry
- Avoid building large `TokenCountMeta.CombineText` when both token counting and sensitive check are disabled (use lightweight meta for pricing)
- Update READMEs (CN/EN/FR/JA) with `MAX_REQUEST_BODY_MB`
- Fix a handful of vet/formatting issues encountered during the change
- `go test ./...` passes
txgo