1230 Commits
Author SHA1 Message Date
Calcium-Ion 0ed497f066 feat(relay): hosted-tool conversion fidelity, reasoning normalization, and billing usage integrity (#7137)
* feat(relaykit): preserve hosted tools across conversions

- add protocol-neutral hosted-tool DTOs, conversion metadata, and loss policies
- bridge citations, grounding metadata, and hosted-tool stream lifecycles
- document the public conversion behavior and channel policy controls

* refactor(relaykit): normalize reasoning and thinking intent

- centralize provider-neutral reasoning intent, effort, and budget mappings
- parse model suffixes at the host entry boundary while preserving provider-owned tails
- keep adaptive Claude thinking and explicit zero-token compatibility consistent

* fix(billing): preserve authoritative usage across relay hops

- carry native BillingUsage sidecars through direct and streamed protocol bridges
- merge partial and terminal usage monotonically with safe fallback settlement
- retain cache metadata, penultimate usage, and per-call Gemini tool surcharges

* feat(relay): bridge Responses with Claude and Gemini protocols

- add direct request, response, and stream converters across supported relay formats
- expose Claude count_tokens and Chat-to-Responses compatibility endpoints
- carry conversion diagnostics through the host while retaining the curated public goldens

* fix(relay): wire relaykit conversions into host channels

- connect handlers, adaptors, and channel settings to the standalone conversion layer
- keep model mapping, pricing identity, retries, and provider-specific suffix behavior aligned
- ignore local audit artifacts and retain focused public regression coverage
2026-09-01 21:53:35 +08:00
CaIon 6c22550ea3 feat(task): resolve channel-mapped aliases and case variants for plugin models
Channel model_mapping keys exposed in a channel's model list now act as
first-class aliases for task-plugin models across the whole line:

- Derived alias view (model/task_model_alias.go): built from enabled
  channels' model_mapping, chain-following with cycle detection, declared
  names always win, cross-plugin conflicts dropped. Rebuilt on channel
  cache refresh, registry generation change, and a 60s TTL.
- Request path: PinTaskPluginEndpoint resolves declared-name case folds
  and mapping aliases before endpoint lookup (never rewriting the body
  until the endpoint is claimed), pins with MappedModel, and the decode
  contract accepts alias echoes without loosening model ownership for
  normal pins. Legacy /v1/tasks submit folds case variants the same way.
  Fixes aliases on POST /v1/responses silently falling through to the
  main relay against task channels.
- Mapping order: ModelMappedHelper now runs before the plugin submit
  hook builds and caches the upstream body, so channel model_mapping
  actually reaches the upstream request. Plugins receive the mapped
  name as ctx.upstreamModel in both decode and submit contexts.
- Billing: identity stays the origin name; when the alias has no tiered
  expression, the selected channel's mapping tail expression applies.
  Pricing page and billing-expr smoke tests resolve aliases to the
  owning plugin's usage schema.
- Case folding: ASCII-only fold with exact-match priority; same-plugin
  and cross-plugin fold collisions rejected at registration.
- Plugins: model-keyed rate tables, req_key derivation, and combo
  validation in doubao/kling/jimeng/hailuo/vidu/sunoapi now key on
  ctx.upstreamModel || ctx.model; render/echo paths keep ctx.model.
2026-08-30 19:13:51 +08:00
CaIon 918427d8ab feat(auth): make password encryption opt-in #6743 2026-08-29 20:54:07 +08:00
CaIon b80d633cf5 feat(auth): encrypt password login transport
Closes #6743
2026-08-29 20:11:36 +08:00
Alex Xiang 0f2a2075ab fix(relay): 请求参数校验错误返回 HTTP 400 (#6774)
* fix(relay): return 400 for invalid request parameters
2026-08-29 19:21:09 +08:00
Calcium-Ion eb48396d5f feat(task): replace built-in task adaptors with a sandboxed JS plugin system (#7076) 2026-08-29 18:51:57 +08:00
Seefs a073f74b38 refactor: deprecate int32 (#7025)
* refactor: deprecate int32

* fix(db): reject legacy user quota schemas at startup

* fix(quota): enforce wallet bounds and saturating billing conversions

* fix(rate-limit): keep count*duration from wrapping int64

* fix: error message
2026-08-26 20:57:54 +08:00
Seefs 4add708ebe feat: channel test (#6917)
* feat: channel test

* fix: code smell
2026-08-18 18:03:59 +08:00
Seefs 2b0efd8484 refactor: advanced custom channel route editor (#6865)
* refactor: advanced custom channel route editor

* fix(channels): show raw balance response from balance cell
2026-08-18 17:31:21 +08:00
CaIon 47ba9d2c63 fix(topup): guard wallet quota during recharge 2026-08-14 17:34:57 +08:00
Shawn Wang 2a0ce3475c fix(topup): reject uncreditable orders before payment (#6845)
* fix(topup): reject uncreditable orders before payment

* fix(topup): align zero-ratio Stripe validation

* fix(topup): mirror settlement conversions in validation
2026-08-14 17:01:32 +08:00
wans10andCaIon 58d4e9bd3b fix(billing): 异步任务退款时同步减少 used_quota (#6795)
* fix(billing): 异步任务退款时同步减少 used_quota
退款时仅恢复了 quota(剩余额度),但未同步减少 used_quota(已用额度),
导致"总额度"(quota + used_quota)随退款次数持续虚增,超出用户实际充值金额。

修复三处退款路径:
- RefundTaskQuota:任务失败完整退款
- RecalculateTaskQuota:差额结算退款分支
- controller/midjourney.go:Midjourney 任务失败退款

新增 model.UpdateUserUsedQuota 公开函数,仅调整 used_quota 不影响 request_count。

* fix(billing): 任务退款时同步扣减渠道 used_quota

* fix(billing): complete async task refund accounting

* style(model): group internal Midjourney fields

---------

Co-authored-by: CaIon <i@caion.me>
2026-08-13 22:06:40 +08:00
CaIon ccd535ef8e fix: harden concurrent quota and status updates 2026-08-11 22:03:47 +08:00
CaIon 50e5377ea5 fix(topup): settle recharge orders atomically 2026-08-11 22:03:47 +08:00
CaIon d7992672a6 fix(oauth): avoid overwriting user state when binding 2026-08-11 22:03:45 +08:00
Seefs bb234ff418 refactor(responses): remove compact model suffix handling (#6770) 2026-08-11 13:42:32 +08:00
Seefs 4cf9107f04 feat(billing): highlight matched conditional multipliers in logs (#6561)
* feat(billing): highlight matched conditional multipliers in usage logs

* fix(billing): make request rule tracing stable and type-safe
2026-08-10 12:50:27 +08:00
Seefs 5d3423bec1 feat(channels): add auto-disable-only channel test mode (#6728) 2026-08-10 12:47:44 +08:00
Seefs b941253aea fix: test Claude/Gemini endpoints with native request format (#6698) 2026-08-07 13:35:46 +08:00
RedwindA 0cd9dc85e3 Merge commit from fork 2026-08-06 17:53:55 +08:00
Calcium-Ion 0ab0202060 Feat/auto group (#6590)
* feat(token): support custom auto group order

* feat(keys): enhance auto group presentation

* fix(keys): rework Auto flow border and compact inherited order

The Auto group highlight previously tinted the whole control surface
with a gradient and animated only a 1px top sweep, which read as a
background color rather than a flowing border. Replace it with a
border-only effect: an aria-hidden, pointer-events-none overlay whose
conic gradient is masked down to a thin ring hugging the rounded
perimeter, so the highlight travels around all four edges and corners
every 3.2s. The interior stays neutral with a restrained static
primary border and glow; prefers-reduced-motion hides the moving
layer while keeping the static emphasis.

The inherited global Auto order also rendered as spacious two-line
rows with circular sequence markers, wasting drawer space. Render it
as a compact wrapping strip of one-line chips (index, name, ratio
badge) with descriptions kept accessible via title and sr-only text,
scrolling only past a much smaller max height.

Custom add/remove/reorder editing, empty-array inheritance semantics,
and the submit payload are unchanged.

* fix(keys): preserve Auto inheritance and unify effects

* refactor(keys): temporarily disable AutoGroupBadge in api-key-group-cell
2026-08-01 23:19:01 +08:00
Calcium-Ion cfaba1dd67 fix(billing): harden tiered retry group-switch billing (#6570)
Follow-up to #6518 (issue #6480) addressing three review findings:

- Document and lock in arrears semantics for the wallet Reserve top-up:
  when an auto-group retry lands on a more expensive group, the full
  reservation delta is deducted unconditionally (balance may go
  negative), mirroring settlement, so the logged pre-consumed quota
  always reconciles with the actual balance movement. Genuine DB
  errors still fail the attempt with update_data_error. Subscription
  funding keeps its insufficient-quota behavior: subscriptions enforce
  a hard used<=total cap and do not support arrears.
- PriceData.FreeModel is cleared when a retry switches from a free
  group to a paid one, keeping it consistent with the billing session
  created at that point.
- getChannel refreshes GroupRatioInfo only after channel selection
  succeeds, and the retry loop records the channel in use_channel
  before PrepareTieredBillingForSelectedGroup can fail.
2026-08-01 09:35:51 +08:00
吴天一 df43f80153 fix(billing): settle tiered retries with final group (#6518) 2026-07-31 19:16:46 +08:00
June Chi cb4c8c02f8 feat(oidc): 支持自定义 OIDC 登录显示名称 (#6012)
* docs: add design spec for OIDC custom display name

Mirrors the existing Custom OAuth Provider name pattern so admins can
show a meaningful label instead of the hardcoded "OIDC" on the login
page and in related copy.

* feat(oidc): add configurable display name with OIDC fallback

* feat(oidc): use configured display name in provider name and status API

* feat(oidc): add display name field to default-theme OIDC settings

Claude-Session: https://claude.ai/code/session_01FDkWJqigJi9yE3HG5pjZP5

* feat(oidc): show configured display name on default-theme login button

* feat(oidc): add display name field to classic-theme OIDC settings

* feat(oidc): show configured display name on classic-theme login button

* fix(oidc): trim whitespace before applying display name fallback

* chore: remove internal design doc from PR

Design/planning docs are working artifacts for this session and
shouldn't be submitted to the upstream project.

* fix(oidc): lead with example in classic-theme display name placeholder

Reorders the combined placeholder to show the example first, then
the fallback note, matching the Custom OAuth Provider Name field's
placeholder convention (example-only) that this feature mirrors.

* fix(i18n): improve Russian grammar in OIDC display-name placeholder translation

Leads each clause with its condition/subject and adds the missing
verb, per PR review feedback.

* test(web): remove redundant OIDC harness tests
2026-07-29 16:27:15 +08:00
CaIon b8bb3f40ac refactor: update import paths to use new types package 2026-07-27 16:45:02 +08:00
Calcium-Ion 86ac0f7745 refactor: extract protocol conversion layer into standalone relaykit module (#6369)
* test(relayconvert): add golden snapshot matrix and relaykit boundary guard

Phase 0 of the relaykit extraction plan: pin byte-level output of every
registered (from,to) request/response/stream conversion route, and
forbid kit-bound packages from growing host-only imports.

* wip(relayconvert): drop gin.Context from converter signatures; add convmeta draft

Phase 1 in progress: relayconvert now takes context.Context; host media
resolver adapts gin.Context back at the service boundary.

* refactor(relayconvert): decouple converters from RelayInfo, gin, and settings

Phase 1 of the relaykit extraction plan:
- converters now depend on convmeta.Meta (implemented by RelayInfo) instead
  of *relaycommon.RelayInfo; ClaudeConvertInfo and the format guesser move
  to convmeta with aliases left behind
- host settings reach converters via a convmeta.Options snapshot built in
  RelayInfo.ConvOptions; no more model_setting/reasoning global reads inside
  the conversion layer
- effort-suffix helpers move to service/relayconvert/reasoning (old package
  forwards); chat-to-responses upgrade policy moves to service (host routing
  logic, not conversion)
- golden conversion matrix unchanged

* test(relayconvert): tighten boundary — kit packages now free of gin/setting imports

* refactor(dto): drop gin and logger dependencies

Phase 2 (part 1): dto.Request.IsStream now takes *http.Request instead of
*gin.Context (Gemini's impl reads query/path off the std request); dto's
three logger calls become common.SysError. Boundary test allowlist is now
empty — kit-bound packages import no gin/setting/logger/model.

* refactor(kit): extract dependency-free kitutil; dto/types/relayconvert stop importing common

Phase 2 of the relaykit extraction plan:
- new service/relayconvert/kitutil holds the pure helpers the kit needs
  (JSON wrappers, pointer/string/uuid/timestamp utils, MaskSensitiveInfo,
  pluggable LogInfo/LogError hooks, Debug flag)
- dto, types, and all relayconvert packages now use kitutil; their only
  remaining internal deps are dto/types/constant
- common keeps every original symbol (MaskSensitiveInfo delegates to
  kitutil) so host code is untouched; main.go routes kit logging into
  common.SysLog/SysError and mirrors DebugEnabled
- golden conversion matrix unchanged

* refactor(kit): move EndpointType/FinishReason to types; OpenRouter dialect via Options

Kit packages (dto/types/relayconvert/reasonmap) no longer import constant:
- EndpointType and finish-reason values live in types; constant re-exports
- the OpenRouter special-case in claude->openai request conversion reads
  Options.OpenRouterDialect, set by the host from the channel type;
  InitChannelMeta invalidates the cached snapshot on channel switch

* refactor: extract relaykit submodule (dto/types/relayconvert/reasonmap)

Phase 3 of the relaykit extraction plan:
- new go module github.com/QuantumNous/new-api/relaykit containing dto
  (minus task family), types, relayconvert (with convmeta/kitutil/reasoning),
  and reasonmap; host consumes it via require + replace, go.work for dev
- task-family dto (task/suno/midjourney/video) stays in the host dto
  package; dual-consumer host files alias it as taskdto
- relaykit builds and tests standalone (GOWORK=off): no host imports,
  no gin, no DB, no settings
- golden conversion matrix unchanged

* build(docker): copy relaykit/go.mod before go mod download

The local-replace submodule's go.mod must exist inside the build context
for the main module graph to resolve.

* fix: address relaykit extraction regressions

* fix: address relaykit review regressions

* docs: document Meta nil receiver contract

* fix(relaykit): fail OpenAI→Claude conversion without max_tokens; reject negative default_max_tokens

The Claude Messages API requires max_tokens (omitting it is a 400
"Field required"), but with a nil Options.Claude.DefaultMaxTokens hook
the converters silently emitted a request the upstream is guaranteed to
reject. Both OpenAI Chat and Responses → Claude conversions now return
sharedclaude.ErrMissingMaxTokens when no path (client value, default
hook, thinking-adapter floor) supplied one. Unreachable in the host,
which always configures the hook.

Host side, claude.default_max_tokens now rejects negative values at the
option API before persisting — they would wrap into huge unsigned values
during conversion. Zero stays allowed: the current API treats
max_tokens: 0 as cache pre-warming.

* fix: make Gemini safety settings read path race-free
2026-07-27 15:56:21 +08:00
CaIon f51dd4d808 fix: allow Advanced Custom Responses Compact
close #6461
2026-07-27 15:21:30 +08:00
CaIon 398cdafecf feat: add New API channel support 2026-07-27 15:20:19 +08:00
CaIon bc14c18f60 refactor: update task refund logic and remove legacy handling 2026-07-26 20:47:52 +08:00
CaIon 2d23cdf291 feat: configurable tool pricing, Sub2API channel, and alpha search billing
Add admin-configurable tool-call prices with cross-provider surcharge
settlement, Sub2API channel support, /v1/alpha/search relay, and usage-log
surcharge UI.
2026-07-26 20:05:15 +08:00
CaIon 1721144221 fix(auth): keep login state on rate-limited or failing token refresh
When the dashboard token refresh endpoint returned 429 (shared
critical rate limit) the frontend classified it as out_of_sync,
cleared local auth state, and redirected to /sign-in. The rate limit
itself is working as intended; the bug is that a temporary rejection
was treated as a terminal auth failure.

- Treat 429 refresh responses as transient errors on the frontend,
  keeping the session retryable instead of clearing it. Only explicit
  401 or confirmed session mismatch/race exhaustion clears auth state.
- Return Retry-After on all rate-limited responses (remaining TTL on
  Redis, window duration on the in-memory limiter) so clients can
  back off.
- Log the underlying error with request context when auth session
  errors map to 500 AUTH_INTERNAL_ERROR, and replace fmt.Println with
  request-scoped logging in the Redis rate limiter error paths.

Fixes #6361
2026-07-21 12:39:29 +08:00
feitianbubuandCaIon e0d5156115 fix: prevent duplicate suno task refunds via cas status update (#6074)
* fix: prevent duplicate suno task refunds via cas status update

* fix: reconcile failed task refunds

---------

Co-authored-by: CaIon <i@caion.me>
2026-07-20 22:03:13 +08:00
QuentinHsu 4aa08f917e fix(playground): resolve auto group model listing (#6163)
* fix(playground): resolve auto group model listing

- merge and deduplicate available models in configured auto group order.
- reuse special usable group rules and add model filtering regression coverage.

* refactor: extract GetGroupsEnabledModels to dedupe group model expansion
2026-07-20 18:21:24 +08:00
yyhhyyyyyy e13d4033e5 fix(channel): improve proxy client compatibility and cache lifecycle (#6157)
* fix(channel): improve proxy client compatibility and cache lifecycle

* test(controller): use non-fatal assertions for channel tests
2026-07-20 18:11:22 +08:00
Calcium-Ion 31d70fca39 refactor(auth): replace dashboard sessions with stateless tokens and session control (#6329)
* refactor(auth): replace dashboard sessions with stateless tokens

* feat(auth): harden session issuance and distributed enforcement

* fix(proxy): preserve trusted proxy compatibility defaults

* refactor: address dashboard auth review feedback

* refactor: remove classic frontend and flatten web app
2026-07-20 16:48:43 +08:00
SeefsandCaIon a6cf42c0f1 feat: support upstream model fetch for advanced custom channels (#5971)
* feat: support upstream model fetch for advanced custom channels

* fix: add advanced custom routes as separate groups

* fix: select advanced custom route entry before adding

---------

Co-authored-by: CaIon <i@caion.me>
2026-07-18 13:39:53 +08:00
Seefs 57746fc972 feat(channel): support Codex upstream model discovery (#6184)
* fix(i18n): clarify Go regex and field passthrough copy

* feat(channel): support Codex upstream model discovery

* Revert "fix(i18n): clarify Go regex and field passthrough copy"

This reverts commit d63d7975db.
2026-07-18 13:04:16 +08:00
DawnMoon1542 506e41c116 fix: add server-side sorting to user list, prevent client-side sort on paged data (#6194)
* fix: add server-side sorting to user list, prevent client-side sort on paged data

The user management table applied client-side sorting to the current
page slice while pagination was handled server-side, causing ID-asc
views to show pages out of order (e.g. 23-42, 3-22, 1-2).

Backend: add sort_by/sort_order query params to GetAllUsers and
SearchUsers with a column whitelist for safe ORDER BY generation.
Frontend: pass sorting state to the API and reset to page 1 on sort
change. Generic useDataTable hook now disables client-side sorted row
model and sort UI for tables with manual pagination but no server-side
sort handler.

* fix: add id tie-breaker to non-unique sort columns, remove side effect from state updater

Append secondary ORDER BY id DESC when sorting by non-unique columns
(quota, created_at, etc.) to prevent row duplication/skipping across
OFFSET pages.

Move onPaginationChange out of setSorting updater to avoid side effects
inside a pure function (React Strict Mode double-invocation safety).
2026-07-17 19:36:44 +08:00
Seefs b6b97a66e3 fix: purge authentication data on hard user deletion (#6168)
* fix: purge authentication data on hard user deletion

* fix: fail closed when 2FA status lookup fails

* fix: reject stale Telegram login callbacks

* fix(twofa): prevent concurrent backup code and lockout bypasses

* fix(auth): harden user deletion and Telegram verification
2026-07-14 14:25:54 +08:00
CaIon 7a2b9d86e8 feat: enhance model search functionality with status and sync filters 2026-07-11 22:36:09 +08:00
Calcium-Ion c36418c863 feat: enhance text protocol conversion and advanced custom routing (#5825)
* refactor: consolidate relay protocol converters

* refactor relayconvert text converters

* feat: refine relay converters and advanced custom routing

* refactor: enhance logging and add thought signature handling for Gemini requests

* refactor: enhance channel cache and pricing endpoint handling for advanced custom models

* feat: preserve billing usage semantics

* feat: add protocol-aware billing usage

* Delete useless files

* chore: update action versions in workflow files

* chore: update Docker action versions in workflow files

* fix: harden billing usage settlement and hot-path route matching

- estimate Gemini completion tokens locally when billable usageMetadata is
  prompt-only but output content was received (e.g. client aborts the stream
  before the final chunk), and rebuild the attached billing_usage as estimated
  so settlement does not bill zero output tokens
- guard NewClaudeMessagesBillingUsage against all-zero ClaudeUsage, matching
  the OpenAI/Gemini constructors, so a zero billing_usage cannot override a
  non-zero top-level usage during settlement
- cache compiled advanced-custom route model regexes; they run on the request
  hot path and were recompiled per request
- move the effectiveBillingUsage remap to PostTextConsumeQuota only, and
  document that calculateTextQuotaSummary expects remapped usage
- document the updatePricingLock -> channelSyncLock lock ordering that
  InitChannelCache/CacheUpdateChannel rely on, and the aux-struct pitfall in
  GeminiChatResponse.UnmarshalJSON
2026-07-11 20:44:12 +08:00
CaIon 337169e0a4 revert: undo t0ng7u UI design-system refactor
Revert the following commits:
- 308e3e347 feat(web): polish themed data views and add task log details
- b2a890e75 fix: Fontsource asset resolution across workspace layouts
- 9d1ca545e refactor(web): refine data-table cards and pricing page layout
- 0918bdb49 refactor(web): consolidate design-system primitives and responsive data views
- 262ab9312 style(web): unify design system across default frontend
2026-07-11 14:51:54 +08:00
t0ng7u 0918bdb49a ♻️ refactor(web): consolidate design-system primitives and responsive data views 2026-07-11 05:13:16 +08:00
feitianbubu 246d62aa5e chore: remove dead files resurrected by v1.0 launch commit (#6041)
controller/task_video.go and service/pre_consume_quota.go were already
removed in ba25ba88f and 116004fd4 (logic lives in service/task_polling.go
and BillingSession now), then brought back as stale copies by a42b39760.
Both have zero callers.

controller/swag_video.go is a leftover swag stub: the swaggo pipeline is
gone and docs/openapi/relay.json already documents these routes.
2026-07-09 22:03:45 +08:00
Seefs a72e5082e9 feat(system-info): add stale instance cleanup actions (#5953) 2026-07-07 21:22:49 +08:00
CaIon fc1259f583 refactor(price): improve handling of other ratios in PriceData 2026-07-07 21:22:19 +08:00
Seefs 9b93d61b7f feat(subscription): add admin quota reset actions (#5952)
* feat(subscription): add admin quota reset actions

* fix(subscription): keep quota reset in plan row actions

* refactor(subscription): move user subscription actions into menu
2026-07-07 12:47:41 +08:00
CaIon bae799ccb1 fix(billing): surface quota saturation events for admin auditing
Thread int32 saturation clamps from tiered settlement and video task
recompute into the consume/task logs under admin_info, so oversized or
malformed billing inputs stay auditable. Clamp negative audio duration
before token conversion and gate the saturation UI markers on admin.
2026-07-07 12:20:07 +08:00
CaIon d0bd8aac74 fix(billing): validate quantity parameters and harden quota calculations
Bound user-supplied count/duration parameters at request validation,
route ratio multipliers through guarded setters, and use saturating
int conversions in all quota math paths.
2026-07-07 00:21:06 +08:00
CaIon df087b022d feat(ssrf): implement SSRF protection in HTTP clients and validation functions 2026-07-06 14:52:01 +08:00