* fix(relay): bound the wait for upstream response headers (fixes unbounded heap growth)
The relay transport sets a dial timeout, a TLS handshake timeout and an expect-continue
timeout, but nothing bounds how long it waits for the upstream *response headers* after
the request has been written. An upstream that accepts the connection and then never
answers -- without sending FIN/RST, which is what happens when a NAT/firewall silently
drops the flow or the provider hangs -- parks the goroutine in
net/http.(*persistConn).roundTrip forever.
That goroutine keeps the whole request alive, which in practice means three copies of the
request body stay reachable for the lifetime of the process: the raw bytes from
io.ReadAll in CreateBodyStorageFromReader, the decoded messages held as json.RawMessage,
and the re-marshalled upstream body from common.Marshal. BodyStorageCleanup cannot help
here: it runs after c.Next() returns, and for these requests c.Next() never returns.
Measured on v1.0.0-rc.23 in production (see #6947 for the full evidence):
- 23 goroutines stuck in persistConn.roundTrip on a single 40h-old instance,
blocked between 353 and 1894 minutes (5.9h to 31.5h)
- 96.9% of the live heap, sampled after a forced GC, attributable to those three
body copies (HeapAlloc 892 MiB surviving three GC cycles; HeapObjects dropping
30x while bytes dropped only 25%)
- the live floor grows with uptime: 33.7 MiB at 0.1h, 89.2 at 13.8h, 510.0 at 40.1h,
955.2 at 146.8h, OOMKilled at 172.9h -- same image, same config, same load
Doubling the memory limit and adding GOMEMLIMIT only moved the OOM from 132h to 172.9h.
RELAY_TIMEOUT (http.Client.Timeout) cannot be used for this: it covers the whole response
read and would cut legitimate long streaming calls, which is why it defaults to 0.
ResponseHeaderTimeout only bounds the wait for the headers; streaming after they arrive is
unaffected.
The default is deliberately generous. Non-streaming upstreams usually send the response
headers only once generation has finished, so the value has to leave room for a long
completion. 1800s is 12x shorter than the shortest hang observed here while leaving
several times the headroom a normal non-streaming request needs; 0 restores the previous
unbounded behaviour.
The assignment goes next to the other transport.* lines rather than inside the else
branch: newRelayHTTPTransport() normally takes the http.DefaultTransport.Clone() path,
and DefaultTransport does not set ResponseHeaderTimeout either.
This repo already sets ResponseHeaderTimeout on its other outbound transports
(controller/model_sync.go, controller/ratio_sync.go); the relay path appears to have
been missed.
Refs #6947. Likely also the root cause of #6731, which reported the same symptom
(production OOM on /v1/responses after ~64h) but was closed for template reasons.
* review: clamp overflowing timeout values and switch the test to testify
Addresses the two CodeRabbit findings on this PR.
Overflow (common/init.go:113): a RELAY_RESPONSE_HEADER_TIMEOUT beyond ~9.2e9 seconds
overflows time.Duration and can wrap into a *tiny positive* timeout, which would cut
every relay request instead of only the stuck ones. The value is now clamped before the
conversion, with regression tests for both the negative and the overflowing input.
I did not add fail-on-startup validation for negative values, for two reasons: the
existing `if seconds > 0` guard already treats them as "disabled", and the neighbouring
env-driven timeouts in this file are less strict still -- RelayIdleConnTimeout is
converted with no guard at all. Failing startup on a bad value would be a behaviour
change out of step with the rest of the file; happy to add it if you'd prefer that
direction repo-wide.
Test style: switched to testify (require.Equal / require.Zero / require.Positive), which
is what every other test under service/ uses.
go build, go vet and go test ./common/... ./service/... pass.
(`go build ./...` fails on the `web/dist` embed both with and without this change -- the
frontend bundle is not checked in.)
* feat(localization): added zh_TW
* fixed based on @coderabbitai
* updated false translation for zh_TW
* new workflow
* revert
* fixed a lot of translations
* turned most zh to zh-CN
* fallbacklang
* bruh
* eliminate ALL _
* fix: paths and other miscs thanks @Calcium-Ion
* fixed translation and temp fix for preferencessettings.js
* fixed translation error
* fixed issue about legacy support
* reverted stupid coderabbit's suggestion
Keep new-site links (/{lang}/docs/...) where matching pages exist in the current docs repo
Revert links that have no equivalent in the new docs to the legacy paths on doc.newapi.pro:
Google Gemini Chat
Midjourney-Proxy image docs
Suno music docs
Apply the same rule consistently across all README translations (zh/en/ja/fr)
- Replace legacy `docs.newapi.pro` paths with the new `/{lang}/docs/...` structure across all README translations
- Point key sections (installation, env vars, API, support, features) to their new locations
- Ensure language-specific links use the correct locale prefix (zh/en/ja) and keep FR aligned with English routes
Clamp request body size (including post-decompression) to avoid memory exhaustion caused by huge payloads/zip bombs, especially with large-context Claude requests. Add a configurable `MAX_REQUEST_BODY_MB` (default `32`) and document it.
- Enforce max request body size after gzip/br decompression via `http.MaxBytesReader`
- Add a secondary size guard in `common.GetRequestBody` and cache-safe handling
- Return **413 Request Entity Too Large** on oversized bodies in relay entry
- Avoid building large `TokenCountMeta.CombineText` when both token counting and sensitive check are disabled (use lightweight meta for pricing)
- Update READMEs (CN/EN/FR/JA) with `MAX_REQUEST_BODY_MB`
- Fix a handful of vet/formatting issues encountered during the change
- `go test ./...` passes
- Add IO.NET to trusted partners section in README.md
- Add IO.NET to trusted partners section in README.en.md
- Use io-net.png logo and https://io.net/ as website link
- Adjust layout to display 3 partners in the second row for better balance
- IO.NET provides decentralized GPU computing services for AI workloads
- Add Alibaba Cloud to trusted partners section in README.md
- Add Alibaba Cloud to trusted partners section in README.en.md
- Use aliyun.svg logo and https://www.aliyun.com/ as website link
- Maintain consistent formatting with existing partners
- Replace fixed table layout with flexible paragraph layout
- Fix display truncation issues on mobile and small screens
- Increase partner logo height from 60px to 80px for better visibility
- Enable automatic line wrapping for partner logos
- Maintain all clickable links and functionality:
* Cherry Studio → https://www.cherry-ai.com/
* Peking University → https://bda.pku.edu.cn/
* UCloud → https://www.compshare.cn/?ytag=GPU_yy_gh_newapi
* Alibaba Cloud → https://bailian.console.aliyun.com/
- Keep center alignment and "no particular order" disclaimer
- Apply responsive improvements to both README versions
- Ensure consistent rendering across different screen sizes
The partners section now adapts gracefully to various viewport widths,
providing optimal viewing experience on desktop and mobile devices.
逺坂凛