mirror of
https://github.com/QuantumNous/new-api.git
synced 2026-09-01 19:41:57 +00:00
* fix(relay): bound the wait for upstream response headers (fixes unbounded heap growth) The relay transport sets a dial timeout, a TLS handshake timeout and an expect-continue timeout, but nothing bounds how long it waits for the upstream *response headers* after the request has been written. An upstream that accepts the connection and then never answers -- without sending FIN/RST, which is what happens when a NAT/firewall silently drops the flow or the provider hangs -- parks the goroutine in net/http.(*persistConn).roundTrip forever. That goroutine keeps the whole request alive, which in practice means three copies of the request body stay reachable for the lifetime of the process: the raw bytes from io.ReadAll in CreateBodyStorageFromReader, the decoded messages held as json.RawMessage, and the re-marshalled upstream body from common.Marshal. BodyStorageCleanup cannot help here: it runs after c.Next() returns, and for these requests c.Next() never returns. Measured on v1.0.0-rc.23 in production (see #6947 for the full evidence): - 23 goroutines stuck in persistConn.roundTrip on a single 40h-old instance, blocked between 353 and 1894 minutes (5.9h to 31.5h) - 96.9% of the live heap, sampled after a forced GC, attributable to those three body copies (HeapAlloc 892 MiB surviving three GC cycles; HeapObjects dropping 30x while bytes dropped only 25%) - the live floor grows with uptime: 33.7 MiB at 0.1h, 89.2 at 13.8h, 510.0 at 40.1h, 955.2 at 146.8h, OOMKilled at 172.9h -- same image, same config, same load Doubling the memory limit and adding GOMEMLIMIT only moved the OOM from 132h to 172.9h. RELAY_TIMEOUT (http.Client.Timeout) cannot be used for this: it covers the whole response read and would cut legitimate long streaming calls, which is why it defaults to 0. ResponseHeaderTimeout only bounds the wait for the headers; streaming after they arrive is unaffected. The default is deliberately generous. Non-streaming upstreams usually send the response headers only once generation has finished, so the value has to leave room for a long completion. 1800s is 12x shorter than the shortest hang observed here while leaving several times the headroom a normal non-streaming request needs; 0 restores the previous unbounded behaviour. The assignment goes next to the other transport.* lines rather than inside the else branch: newRelayHTTPTransport() normally takes the http.DefaultTransport.Clone() path, and DefaultTransport does not set ResponseHeaderTimeout either. This repo already sets ResponseHeaderTimeout on its other outbound transports (controller/model_sync.go, controller/ratio_sync.go); the relay path appears to have been missed. Refs #6947. Likely also the root cause of #6731, which reported the same symptom (production OOM on /v1/responses after ~64h) but was closed for template reasons. * review: clamp overflowing timeout values and switch the test to testify Addresses the two CodeRabbit findings on this PR. Overflow (common/init.go:113): a RELAY_RESPONSE_HEADER_TIMEOUT beyond ~9.2e9 seconds overflows time.Duration and can wrap into a *tiny positive* timeout, which would cut every relay request instead of only the stuck ones. The value is now clamped before the conversion, with regression tests for both the negative and the overflowing input. I did not add fail-on-startup validation for negative values, for two reasons: the existing `if seconds > 0` guard already treats them as "disabled", and the neighbouring env-driven timeouts in this file are less strict still -- RelayIdleConnTimeout is converted with no guard at all. Failing startup on a bad value would be a behaviour change out of step with the rest of the file; happy to add it if you'd prefer that direction repo-wide. Test style: switched to testify (require.Equal / require.Zero / require.Positive), which is what every other test under service/ uses. go build, go vet and go test ./common/... ./service/... pass. (`go build ./...` fails on the `web/dist` embed both with and without this change -- the frontend bundle is not checked in.)
267 lines
7.2 KiB
Go
267 lines
7.2 KiB
Go
package common
|
|
|
|
import (
|
|
"crypto/tls"
|
|
//"os"
|
|
//"strconv"
|
|
"sync"
|
|
"time"
|
|
|
|
"github.com/google/uuid"
|
|
)
|
|
|
|
var StartTime = time.Now().Unix() // unit: second
|
|
var Version = "v0.0.0" // this hard coding will be replaced automatically when building, no need to manually change
|
|
var SystemName = "New API"
|
|
var Footer = ""
|
|
var Logo = ""
|
|
var TopUpLink = ""
|
|
|
|
// var ChatLink = ""
|
|
// var ChatLink2 = ""
|
|
var QuotaPerUnit = 500 * 1000.0 // $0.002 / 1K tokens
|
|
// 保留旧变量以兼容历史逻辑,实际展示由 general_setting.quota_display_type 控制
|
|
var DisplayInCurrencyEnabled = true
|
|
var DisplayTokenStatEnabled = true
|
|
var DrawingEnabled = true
|
|
var TaskEnabled = true
|
|
var DataExportEnabled = true
|
|
var DataExportInterval = 5 // unit: minute
|
|
var DataExportDefaultTime = "hour" // unit: minute
|
|
var DefaultCollapseSidebar = false // default value of collapse sidebar
|
|
|
|
// Any options with "Secret", "Token" in its key won't be return by GetOptions
|
|
|
|
var SessionSecret = uuid.New().String()
|
|
var CryptoSecret = uuid.New().String()
|
|
var SessionCookieSecure = false
|
|
var SessionCookieTrustedURLs []string
|
|
|
|
const (
|
|
DefaultUserSessionActiveLimit = 50
|
|
DefaultUserSessionIssuanceLimit = 100
|
|
DefaultUserSessionIssuanceWindowSeconds = 24 * 60 * 60
|
|
DefaultUserSessionRevokedRetentionDays = 7
|
|
DefaultUserSessionHourlyAlertThreshold = 5000
|
|
)
|
|
|
|
var (
|
|
UserSessionActiveLimit = DefaultUserSessionActiveLimit
|
|
UserSessionIssuanceLimit = DefaultUserSessionIssuanceLimit
|
|
UserSessionIssuanceWindowSeconds = int64(DefaultUserSessionIssuanceWindowSeconds)
|
|
UserSessionRevokedRetentionDays = DefaultUserSessionRevokedRetentionDays
|
|
UserSessionHourlyAlertThreshold = DefaultUserSessionHourlyAlertThreshold
|
|
)
|
|
|
|
var OptionMap map[string]string
|
|
var OptionMapRWMutex sync.RWMutex
|
|
|
|
var ItemsPerPage = 10
|
|
var MaxRecentItems = 1000
|
|
|
|
var PasswordLoginEnabled = true
|
|
var PasswordLoginEncryptionEnabled = false
|
|
var PasswordRegisterEnabled = true
|
|
var EmailVerificationEnabled = false
|
|
var GitHubOAuthEnabled = false
|
|
var LinuxDOOAuthEnabled = false
|
|
var WeChatAuthEnabled = false
|
|
var TelegramOAuthEnabled = false
|
|
var TurnstileCheckEnabled = false
|
|
var RegisterEnabled = true
|
|
|
|
var EmailDomainRestrictionEnabled = false // 是否启用邮箱域名限制
|
|
var EmailAliasRestrictionEnabled = false // 是否启用邮箱别名限制
|
|
var EmailDomainWhitelist = []string{
|
|
"gmail.com",
|
|
"163.com",
|
|
"126.com",
|
|
"qq.com",
|
|
"outlook.com",
|
|
"hotmail.com",
|
|
"icloud.com",
|
|
"yahoo.com",
|
|
"foxmail.com",
|
|
}
|
|
var EmailLoginAuthServerList = []string{
|
|
"smtp.sendcloud.net",
|
|
"smtp.azurecomm.net",
|
|
}
|
|
|
|
var DebugEnabled bool
|
|
var MemoryCacheEnabled bool
|
|
|
|
var LogConsumeEnabled = true
|
|
|
|
var TLSInsecureSkipVerify bool
|
|
var InsecureTLSConfig = &tls.Config{InsecureSkipVerify: true}
|
|
|
|
var SMTPServer = ""
|
|
var SMTPPort = 587
|
|
var SMTPSSLEnabled = false
|
|
var SMTPStartTLSEnabled = false
|
|
var SMTPInsecureSkipVerify = false
|
|
var SMTPForceAuthLogin = false
|
|
var SMTPAccount = ""
|
|
var SMTPFrom = ""
|
|
var SMTPToken = ""
|
|
|
|
var GitHubClientId = ""
|
|
var GitHubClientSecret = ""
|
|
var LinuxDOClientId = ""
|
|
var LinuxDOClientSecret = ""
|
|
var LinuxDOMinimumTrustLevel = 0
|
|
|
|
var WeChatServerAddress = ""
|
|
var WeChatServerToken = ""
|
|
var WeChatAccountQRCodeImageURL = ""
|
|
|
|
var TurnstileSiteKey = ""
|
|
var TurnstileSecretKey = ""
|
|
|
|
var TelegramBotToken = ""
|
|
var TelegramBotName = ""
|
|
|
|
var QuotaForNewUser = 0
|
|
var QuotaForInviter = 0
|
|
var QuotaForInvitee = 0
|
|
var ChannelDisableThreshold = 5.0
|
|
var AutomaticDisableChannelEnabled = false
|
|
var AutomaticEnableChannelEnabled = false
|
|
var QuotaRemindThreshold = 1000
|
|
var PreConsumedQuota = 500
|
|
|
|
var RetryTimes = 0
|
|
|
|
//var RootUserEmail = ""
|
|
|
|
var IsMasterNode bool
|
|
|
|
const (
|
|
NodeNameSourceManual = "manual"
|
|
NodeNameSourceHostname = "hostname"
|
|
)
|
|
|
|
// NodeName 节点名称,优先从 NODE_NAME 环境变量读取,未配置时回退主机名。
|
|
// 用于审计日志和后台任务中标识节点身份;多实例部署时建议显式配置稳定 NODE_NAME。
|
|
var NodeName = ""
|
|
|
|
// NodeNameSource records how NodeName was chosen so future instance-management
|
|
// reporting can distinguish operator-configured names from automatic fallback.
|
|
var NodeNameSource = NodeNameSourceHostname
|
|
|
|
var NodeNameManuallyConfigured bool
|
|
|
|
var requestInterval int
|
|
var RequestInterval time.Duration
|
|
|
|
var SyncFrequency int // unit is second
|
|
|
|
var BatchUpdateEnabled = false
|
|
var BatchUpdateInterval int
|
|
|
|
var RelayTimeout int // unit is second
|
|
|
|
var RelayIdleConnTimeout int // unit is second
|
|
|
|
// RelayResponseHeaderTimeout limits how long the relay transport waits for the
|
|
// upstream response headers after the request has been fully written.
|
|
// 0 disables it (previous behaviour: wait forever).
|
|
//
|
|
// Note this is NOT the same as RelayTimeout (http.Client.Timeout), which covers
|
|
// the whole response read and therefore breaks legitimate long streaming calls.
|
|
// ResponseHeaderTimeout only bounds the wait for the response headers; once the
|
|
// headers arrive, streaming is unaffected.
|
|
var RelayResponseHeaderTimeout int // unit is second
|
|
var RelayMaxIdleConns int
|
|
var RelayMaxIdleConnsPerHost int
|
|
|
|
var GeminiSafetySetting string
|
|
|
|
// https://docs.cohere.com/docs/safety-modes Type; NONE/CONTEXTUAL/STRICT
|
|
var CohereSafetySetting string
|
|
|
|
const (
|
|
RequestIdKey = "X-Oneapi-Request-Id"
|
|
UpstreamRequestIdKey = "X-Upstream-Request-Id"
|
|
)
|
|
|
|
const (
|
|
RoleGuestUser = 0
|
|
RoleCommonUser = 1
|
|
RoleAdminUser = 10
|
|
RoleRootUser = 100
|
|
)
|
|
|
|
func IsValidateRole(role int) bool {
|
|
return role == RoleGuestUser || role == RoleCommonUser || role == RoleAdminUser || role == RoleRootUser
|
|
}
|
|
|
|
var (
|
|
FileUploadPermission = RoleGuestUser
|
|
FileDownloadPermission = RoleGuestUser
|
|
ImageUploadPermission = RoleGuestUser
|
|
ImageDownloadPermission = RoleGuestUser
|
|
)
|
|
|
|
// All duration's unit is seconds
|
|
// Shouldn't larger then RateLimitKeyExpirationDuration
|
|
var (
|
|
GlobalApiRateLimitEnable bool
|
|
GlobalApiRateLimitNum int
|
|
GlobalApiRateLimitDuration int64
|
|
|
|
GlobalWebRateLimitEnable bool
|
|
GlobalWebRateLimitNum int
|
|
GlobalWebRateLimitDuration int64
|
|
|
|
CriticalRateLimitEnable bool
|
|
CriticalRateLimitNum = 20
|
|
CriticalRateLimitDuration int64 = 20 * 60
|
|
|
|
UploadRateLimitNum = 10
|
|
UploadRateLimitDuration int64 = 60
|
|
|
|
DownloadRateLimitNum = 10
|
|
DownloadRateLimitDuration int64 = 60
|
|
|
|
// Per-user search rate limit (applies after authentication, keyed by user ID)
|
|
SearchRateLimitEnable = true
|
|
SearchRateLimitNum = 10
|
|
SearchRateLimitDuration int64 = 60
|
|
)
|
|
|
|
var RateLimitKeyExpirationDuration = 20 * time.Minute
|
|
|
|
const (
|
|
UserStatusEnabled = 1 // don't use 0, 0 is the default value!
|
|
UserStatusDisabled = 2 // also don't use 0
|
|
)
|
|
|
|
const (
|
|
TokenStatusEnabled = 1 // don't use 0, 0 is the default value!
|
|
TokenStatusDisabled = 2 // also don't use 0
|
|
TokenStatusExpired = 3
|
|
TokenStatusExhausted = 4
|
|
)
|
|
|
|
const (
|
|
RedemptionCodeStatusEnabled = 1 // don't use 0, 0 is the default value!
|
|
RedemptionCodeStatusDisabled = 2 // also don't use 0
|
|
RedemptionCodeStatusUsed = 3 // also don't use 0
|
|
)
|
|
|
|
const (
|
|
ChannelStatusUnknown = 0
|
|
ChannelStatusEnabled = 1 // don't use 0, 0 is the default value!
|
|
ChannelStatusManuallyDisabled = 2 // also don't use 0
|
|
ChannelStatusAutoDisabled = 3
|
|
)
|
|
|
|
const (
|
|
TopUpStatusPending = "pending"
|
|
TopUpStatusSuccess = "success"
|
|
TopUpStatusFailed = "failed"
|
|
TopUpStatusExpired = "expired"
|
|
)
|