mirror of
https://github.com/QuantumNous/new-api.git
synced 2026-08-31 02:41:34 +00:00
b518d0033b
* fix(relay): bound the wait for upstream response headers (fixes unbounded heap growth) The relay transport sets a dial timeout, a TLS handshake timeout and an expect-continue timeout, but nothing bounds how long it waits for the upstream *response headers* after the request has been written. An upstream that accepts the connection and then never answers -- without sending FIN/RST, which is what happens when a NAT/firewall silently drops the flow or the provider hangs -- parks the goroutine in net/http.(*persistConn).roundTrip forever. That goroutine keeps the whole request alive, which in practice means three copies of the request body stay reachable for the lifetime of the process: the raw bytes from io.ReadAll in CreateBodyStorageFromReader, the decoded messages held as json.RawMessage, and the re-marshalled upstream body from common.Marshal. BodyStorageCleanup cannot help here: it runs after c.Next() returns, and for these requests c.Next() never returns. Measured on v1.0.0-rc.23 in production (see #6947 for the full evidence): - 23 goroutines stuck in persistConn.roundTrip on a single 40h-old instance, blocked between 353 and 1894 minutes (5.9h to 31.5h) - 96.9% of the live heap, sampled after a forced GC, attributable to those three body copies (HeapAlloc 892 MiB surviving three GC cycles; HeapObjects dropping 30x while bytes dropped only 25%) - the live floor grows with uptime: 33.7 MiB at 0.1h, 89.2 at 13.8h, 510.0 at 40.1h, 955.2 at 146.8h, OOMKilled at 172.9h -- same image, same config, same load Doubling the memory limit and adding GOMEMLIMIT only moved the OOM from 132h to 172.9h. RELAY_TIMEOUT (http.Client.Timeout) cannot be used for this: it covers the whole response read and would cut legitimate long streaming calls, which is why it defaults to 0. ResponseHeaderTimeout only bounds the wait for the headers; streaming after they arrive is unaffected. The default is deliberately generous. Non-streaming upstreams usually send the response headers only once generation has finished, so the value has to leave room for a long completion. 1800s is 12x shorter than the shortest hang observed here while leaving several times the headroom a normal non-streaming request needs; 0 restores the previous unbounded behaviour. The assignment goes next to the other transport.* lines rather than inside the else branch: newRelayHTTPTransport() normally takes the http.DefaultTransport.Clone() path, and DefaultTransport does not set ResponseHeaderTimeout either. This repo already sets ResponseHeaderTimeout on its other outbound transports (controller/model_sync.go, controller/ratio_sync.go); the relay path appears to have been missed. Refs #6947. Likely also the root cause of #6731, which reported the same symptom (production OOM on /v1/responses after ~64h) but was closed for template reasons. * review: clamp overflowing timeout values and switch the test to testify Addresses the two CodeRabbit findings on this PR. Overflow (common/init.go:113): a RELAY_RESPONSE_HEADER_TIMEOUT beyond ~9.2e9 seconds overflows time.Duration and can wrap into a *tiny positive* timeout, which would cut every relay request instead of only the stuck ones. The value is now clamped before the conversion, with regression tests for both the negative and the overflowing input. I did not add fail-on-startup validation for negative values, for two reasons: the existing `if seconds > 0` guard already treats them as "disabled", and the neighbouring env-driven timeouts in this file are less strict still -- RelayIdleConnTimeout is converted with no guard at all. Failing startup on a bad value would be a behaviour change out of step with the rest of the file; happy to add it if you'd prefer that direction repo-wide. Test style: switched to testify (require.Equal / require.Zero / require.Positive), which is what every other test under service/ uses. go build, go vet and go test ./common/... ./service/... pass. (`go build ./...` fails on the `web/dist` embed both with and without this change -- the frontend bundle is not checked in.)
129 lines
4.8 KiB
Plaintext
129 lines
4.8 KiB
Plaintext
# 端口号
|
||
# PORT=3000
|
||
# 前端基础URL
|
||
# FRONTEND_BASE_URL=https://your-frontend-url.com
|
||
|
||
|
||
# 调试相关配置
|
||
# 启用pprof
|
||
# ENABLE_PPROF=true
|
||
# 启用调试模式
|
||
# DEBUG=true
|
||
# Pyroscope 配置
|
||
# PYROSCOPE_URL=http://localhost:4040
|
||
# PYROSCOPE_APP_NAME=new-api
|
||
# PYROSCOPE_BASIC_AUTH_USER=your-user
|
||
# PYROSCOPE_BASIC_AUTH_PASSWORD=your-password
|
||
# PYROSCOPE_MUTEX_RATE=5
|
||
# PYROSCOPE_BLOCK_RATE=5
|
||
# HOSTNAME=your-hostname
|
||
|
||
# 数据库相关配置
|
||
# 启用错误日志记录
|
||
# ERROR_LOG_ENABLED=true
|
||
# 数据库连接字符串
|
||
# SQL_DSN=user:password@tcp(127.0.0.1:3306)/dbname?parseTime=true
|
||
# 日志数据库连接字符串
|
||
# LOG_SQL_DSN=user:password@tcp(127.0.0.1:3306)/logdb?parseTime=true
|
||
# SQLite数据库路径
|
||
# SQLITE_PATH=/path/to/sqlite.db
|
||
# 数据库最大空闲连接数
|
||
# SQL_MAX_IDLE_CONNS=100
|
||
# 数据库最大打开连接数
|
||
# SQL_MAX_OPEN_CONNS=1000
|
||
# 数据库连接最大生命周期（秒）
|
||
# SQL_MAX_LIFETIME=60
|
||
# 慢查询日志阈值（毫秒），0 表示关闭慢查询日志，超出 0-3600000 范围回退默认值 200
|
||
# SQL_SLOW_THRESHOLD_MS=200
|
||
# 跳过用户额度列 64 位 schema 检查（仅在已确认数据库列可容纳 64 位时启用）
|
||
# SKIP_64BIT_QUOTA_SCHEMA_CHECK=true
|
||
|
||
|
||
# 缓存相关配置
|
||
# Redis连接字符串
|
||
# REDIS_CONN_STRING=redis://user:password@localhost:6379/0
|
||
# 同步频率（单位：秒）
|
||
# SYNC_FREQUENCY=60
|
||
# 内存缓存启用
|
||
# MEMORY_CACHE_ENABLED=true
|
||
# 渠道更新频率（单位：秒）
|
||
# CHANNEL_UPDATE_FREQUENCY=30
|
||
# 批量更新启用
|
||
# BATCH_UPDATE_ENABLED=true
|
||
# 批量更新间隔（单位：秒）
|
||
# BATCH_UPDATE_INTERVAL=5
|
||
|
||
# 任务和功能配置
|
||
# 更新任务启用
|
||
# UPDATE_TASK=true
|
||
|
||
# 对话超时设置
|
||
# 所有请求超时时间，单位秒，默认为0，表示不限制
|
||
# RELAY_TIMEOUT=0
|
||
# Relay HTTP 客户端空闲连接超时时间，单位秒，默认跟随 Go 标准库，设置为0表示不限制
|
||
# RELAY_IDLE_CONN_TIMEOUT=90
|
||
# 等待上游返回响应头的超时时间，单位秒，默认 1800，设置为 0 表示不限制。
|
||
# 仅约束「等待响应头」这一段；响应头返回之后的流式传输不受影响。
|
||
# 注意：非流式请求通常要等上游生成完毕才会返回响应头，因此该值需留足余量。
|
||
# RELAY_RESPONSE_HEADER_TIMEOUT=1800
|
||
# 流模式无响应超时时间，单位秒，如果出现空补全可以尝试改为更大值
|
||
# STREAMING_TIMEOUT=300
|
||
|
||
# TLS / HTTP 跳过验证设置
|
||
# TLS_INSECURE_SKIP_VERIFY=false
|
||
|
||
# Gin 可信反向代理（逗号分隔的 IP/CIDR）
|
||
# 未配置/留空：默认信任 127.0.0.0/8、::1、RFC1918 私网和 fc00::/7，并打印启动告警。
|
||
# none：严格模式，不信任任何代理且必须单独使用；显式列表完全替代默认值，应填写代理自身地址。
|
||
# TRUSTED_PROXIES=none
|
||
# TRUSTED_PROXIES=127.0.0.1,172.20.0.0/16
|
||
|
||
# Gemini 识别图片 最大图片数量
|
||
# GEMINI_VISION_MAX_IMAGE_NUM=16
|
||
|
||
# 会话密钥
|
||
# SESSION_SECRET=random_string
|
||
# 登录密码请求体 RSA-OAEP 加密；默认关闭，且不能替代 HTTPS
|
||
# PASSWORD_LOGIN_ENCRYPTION_ENABLED=true
|
||
# false/未配置：本地 HTTP 模式，关闭 refresh/logout OriginGuard，且不得设置 TRUSTED_URL；兼容本地开发代理。
|
||
# true：启用 Secure Refresh Cookie 和严格 OriginGuard，必须同时列出全部可信 HTTPS Origin。
|
||
# SESSION_COOKIE_TRUSTED_URL 多项用英文逗号分隔；不支持通配符、路径或域名后缀匹配。
|
||
# 这些设置不修改 relay CORS。
|
||
# SESSION_COOKIE_SECURE=false
|
||
# SESSION_COOKIE_TRUSTED_URL=https://example.com,https://admin.example.com
|
||
# 每用户最多保留的活跃登录 Session
|
||
# USER_SESSION_ACTIVE_LIMIT=50
|
||
# 单用户在签发窗口内允许创建的 Session 总数（包含已撤销）
|
||
# USER_SESSION_ISSUANCE_LIMIT=100
|
||
# Session 签发计数窗口（秒）；不得大于 revoked 保留期，超出时会自动钳制
|
||
# USER_SESSION_ISSUANCE_WINDOW_SECONDS=86400
|
||
# revoked Session 审计保留天数
|
||
# USER_SESSION_REVOKED_RETENTION_DAYS=7
|
||
# 最近一小时全局 Session 签发量超过此值时记录告警，不会拒绝登录
|
||
# USER_SESSION_HOURLY_ALERT_THRESHOLD=5000
|
||
|
||
# 其他配置
|
||
# 生成默认token
|
||
# GENERATE_DEFAULT_TOKEN=false
|
||
# Cohere 安全设置
|
||
# COHERE_SAFETY_SETTING=NONE
|
||
# 是否统计图片token
|
||
# GET_MEDIA_TOKEN=true
|
||
# 是否在非流（stream=false）情况下统计图片token
|
||
# GET_MEDIA_TOKEN_NOT_STREAM=false
|
||
# 设置 Dify 渠道是否输出工作流和节点信息到客户端
|
||
# DIFY_DEBUG=true
|
||
|
||
# LinuxDo相关配置
|
||
LINUX_DO_TOKEN_ENDPOINT=https://connect.linux.do/oauth2/token
|
||
LINUX_DO_USER_ENDPOINT=https://connect.linux.do/api/user
|
||
|
||
# 节点类型
|
||
# 如果是主节点则为master
|
||
# NODE_TYPE=master
|
||
|
||
# 可信任重定向域名列表（逗号分隔，支持子域名匹配）
|
||
# 用于验证支付成功/取消回调URL的域名安全性
|
||
# 示例: example.com,myapp.io 将允许 example.com, sub.example.com, myapp.io 等
|
||
# TRUSTED_REDIRECT_DOMAINS=example.com,myapp.io
|