Commit Graph
386 Commits
Author SHA1 Message Date
Vance Ingalls 05af338655 merge: pull in the fallow-audit fixes 2026-08-12 10:49:16 -07:00
Vance Ingalls 7c410fd075 fix(core,studio): clear the remaining Fallow audit findings on the FX panel
- Split FxSection's per-node row into FxNodeRow + FxNodeControls so the
  CRAP score (31.6, threshold 30) splits across two smaller units instead
  of moving wholesale with one extraction.
- Dedupe the repeated "open the add menu, read its items" block in
  propertyPanelFxSection.test.tsx into openAddMenuItems().
- Merge build-audio-fx-runtime.ts and build-position-edits-render.ts into
  one build-inline-artifact.ts, config-selected by CLI arg — the two
  scripts were a byte-for-byte clone save for names.
- Exempt canary.test.ts's rawFnv (a deliberate independent
  reimplementation used to cross-check canaryBucket, per its own
  docstring) and the property-panel test files' shared renderInto/mount
  scaffolding (pre-existing across 9 files, 2 outside this stack) in
  .fallowrc.jsonc, consistent with this file's existing exemptions for
  the same class of intentional/pre-existing duplication.
2026-08-12 10:45:31 -07:00
Vance IngallsandClaude Opus 5 cbf77bcada feat(core): automation envelope model for audio tracks
Adds the data model behind Ableton-style automation lanes: breakpoint
envelopes over track volume or one knob of one effect in the track's FX
chain, stored on the element as `data-automation`.

Times are clip-local, so an envelope travels with the clip when it moves —
the clip-envelope model rather than arrangement automation.

`sampleAutomationLane` is the single interpolator. The lane drawing, the
preview scheduler and the render bake all call it, so the picture and the
sound cannot disagree about the curve. Log-scaled parameters interpolate in
log space, matching what their own knob already promises.

FX nodes gain a stable `id`, minted by count rather than randomly so the
document is the same on every machine. Lanes address nodes by id, so
reordering a chain never re-points a lane at a different effect, and a lane
whose effect was deleted is dropped rather than left to reattach.

Also warns when a track carries both a volume lane and a GSAP volume tween,
since only the lane is heard and the tween silently does nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 08:40:24 -07:00
Vance Ingalls e275c04c46 feat(core): voiceover carve analysis
Finds the bands a voice occupies so a music bed can be dipped there, letting
the voice sit in front without ducking the whole track.

Carve is a relationship between two tracks rather than an effect on one, so it
stays out of the FX chain. What it emits is an ordinary chain of peaking
filters, so a carve composes with whatever else is on the track and needs no
separate rendering path.

Selection is weighted toward intelligibility rather than raw voice energy.
Ranking purely by power lands on the fundamental almost every time, because
that is where a voice is loudest — but the masking that actually hurts a
voiceover happens higher up, and dipping 160 Hz mostly just thins the bed. The
bias is a control, not a constant: at 0 it follows raw energy, at 1 it weights
toward 1-3 kHz.

Ranking happens in dB, which matters more than it looks. Speech spreads 20-30 dB
across these bands — it falls off roughly 6 dB per octave above the fundamental
— so a weighting has to be on that scale to move anything at all. A
multiplicative weight of `1 - bias + bias * shaped` is bounded below by
`1 - bias`, capping its influence at 10*log10(1/(1 - bias)): 5.2 dB at the 0.7
default, 3 dB at 0.5. That is no influence against a real voice — every bias
short of ~0.95 would rank exactly like bias 0 and carve the fundamental, the
outcome the bias exists to prevent, while looking decisive against a fixture
whose bands sit 2 dB apart. So the bias is a dB penalty, zero at 2 kHz and worth
up to 30 dB at full strength, and relative cut depths come from a dB difference
rather than a ratio of weighted linear powers.

The bias reweights ranking without overriding the spectrum — a band the voice
has no energy in is not worth carving, and scores -Infinity rather than
competing — so a strongly low-pitched voice can still select low at full bias.
What the tests hold is that biasing never selects lower than the unbiased
ranking, that the DEFAULT bias reaches the presence region on a voice with a
realistic tilt, and that bias 0 still follows raw power exactly.

Includes a radix-2 FFT rather than a dependency; one Welch-style averaged
spectrum over third-octave bands does not justify pulling in a DSP library.
2026-08-12 02:35:08 -07:00
Vance Ingalls bb9dd61a6a feat(engine): render audio FX in an OfflineAudioContext
Reads `data-fx-chain` off an audio element and runs the chain over the trimmed
WAV before volume automation is baked in — effects should see the raw signal,
and the envelope belongs on their output.

The processing happens in an OfflineAudioContext inside the headless browser
the engine already drives, running the same graph builders the studio previews
with. That is the point of the approach: one implementation per effect, so the
render agreeing with the preview is a property of the architecture rather than
a tolerance to police. Reimplementing each effect as an FFmpeg filter would
mean two implementations to keep in step, and for the dynamics processors and
modulated delays there is no filter that behaves the same way.

`build:audio-fx-runtime` bundles the graph builders into an injectable IIFE,
following the same pattern as the existing runtime artifacts, so the browser
runs exactly the code the studio does.

The page loads from a file:// URL rather than about:blank because AudioWorklet
is only exposed in a secure context — the compressor, limiter, gate and
bitcrush processors would otherwise fail to register with an opaque error.
file:// qualifies and needs no listening socket.

The chain is serialised into the attribute the way colour grading carries its
config, so there is no side-car file to resolve or lose.

An FX failure is fatal for the whole mix rather than a per-track soft failure.
Every other audio failure mode degrades gracefully — the track drops, siblings
continue — but substituting the dry signal for a processed one ships a render
that sounds plausible and is not what the author set up. Since the per-element
work races under Promise.all, an internal AbortController chained off the
caller's signal aborts in-flight siblings before workDir is removed.
2026-08-12 02:35:08 -07:00
Vance IngallsandClaude Opus 5 5752d22492 feat(core): the audio FX registry (#3019)
* feat(core): audio FX registry

One declarative description of every effect that can be applied to an audio
track: fourteen across filters, dynamics, non-linear and time, each exposing
its full parameter surface rather than a curated subset.

Parameters carry the range, step, unit and scale a control needs, so a panel
can generate its UI from this rather than hard-coding a form per effect, and a
value that survives `normalizeAudioFxParams` is always safe to realise.
Everything is declared in the units a person thinks in — dB, ms, Hz.

Parsing rejects an unknown effect id rather than skipping the node. A chain
that quietly loses an effect renders something other than what was authored,
which is worse than refusing to load it.

Data only: no audio is produced here. The graph that realises each effect is
referenced by the `web` id and lands in the next change, which keeps this
module free of browser globals so the engine and the linter can import it.

* fix(core): stop declaring knobs that move nothing

Three parameters were declared with ranges, defaults and hints, and read by no
builder — dials an author could turn with no audible result.

- `chorus.decay` and `bitcrush.aa`: removed. FFmpeg's chorus feeds a decay back
  into its delay line and a bitcrusher's anti-alias needs a real filter; adding
  either is new DSP, not a fix, so the honest move is to stop advertising them.
- `lowshelf.q` / `highshelf.q`: removed. The Web Audio spec leaves Q unused for
  shelving filters, so the control moved nothing — and because the shared Q
  helper marks it automatable, an author could draw an envelope on it and hear
  nothing at all.

`phaser.decay` and `gate.knee` stay: the first drives the sweep depth, and the
second is now read by the gate's processor.

A test asserts each of these directly, since the existing exposure invariant only
checks that a flagged parameter reaches an AudioParam — a parameter the node then
ignores passes it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 00:09:59 -07:00
Miguel Ángel e0ba41c024 chore: release v0.7.107 (#3228) 2026-08-11 16:19:20 -04:00
Miguel Ángel 636dc042a7 feat(core): sanitize rich text on the way into a composition (#3141)
* feat(core): sanitize rich text on the way into a composition

Studio's patch vocabulary was inline-style, attribute, html-attribute and
text-content. text-content assigns textContent, and the text-field model
escapes markup on the way out and refuses a change in child structure, so a
styled span had no route into a composition file.

Adds a rich-text operation with one, guarded by a single sanitizer called on
both ends of the trip: in the browser so the preview shows what will be
saved, and on the server because that is where the file is written. Tags and
style properties are a small allowlist, and an unexpected tag loses its
formatting rather than its words. Spans an edit adds get their ids in the
same write, so a follow-up write cannot race it.

No UI yet — this is the persistence contract the editor is built on.

* fix(core): document and test the sanitizer boundary

* fix(core): harden rich text sanitizer traversal
2026-08-11 02:03:47 -04:00
Miguel Ángel c9dd8413c3 chore: release v0.7.106 (#3197) 2026-08-10 22:53:34 -04:00
Vance Ingalls bd1c1af291 chore: release v0.7.105 (#3152) 2026-08-10 08:08:16 -07:00
Miguel Ángel c96b30c717 chore: release v0.7.104 (#3147) 2026-08-09 17:19:31 -07:00
Vance Ingalls adb13ce125 chore: release v0.7.103 (#3127) 2026-08-09 03:14:27 -07:00
Miguel Ángel b1f7d8881b chore: release v0.7.102 (#3119) 2026-08-08 13:13:50 -07:00
Vance Ingalls eba96feda7 chore: release v0.7.101 2026-08-07 19:50:48 -07:00
Vance IngallsandMiguel Ángel d9b00e57eb chore: release v0.7.100 (#3093)
Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
2026-08-07 16:12:25 -07:00
Miguel Ángel 9aa90f6e3e chore: release v0.7.99 2026-08-07 15:31:57 +00:00
Vance Ingalls 6114749d8e chore: release v0.7.98 2026-08-06 23:37:23 -07:00
Miguel Ángel c4b41072c7 chore: release v0.7.97 2026-08-07 05:16:27 +00:00
Vance Ingalls 026e6941ac chore: release v0.7.96 2026-08-06 14:53:26 -07:00
Miguel Ángel fc0298de4a chore: release v0.7.95 (#3068) 2026-08-06 10:12:20 -07:00
Miguel Ángel 29f004cfc0 chore: release v0.7.94 (#3045) 2026-08-04 20:42:17 -07:00
Miguel Ángel a4eb602ee2 chore: release v0.7.93 (#3044) 2026-08-04 20:27:07 -07:00
Vance Ingalls df8b9604fe chore: release v0.7.92 2026-08-04 01:49:05 -07:00
Vance Ingalls 8c6cf90ff9 chore: release v0.7.91 2026-08-03 22:48:00 -07:00
Vance Ingalls 71fd96bbf1 Merge pull request #2854 from heygen-com/feat/canary-rollouts
feat(core): percentage-based canary rollouts + calibration experiment
2026-08-03 22:35:37 -07:00
James Russo 1e51eaec2c chore: release v0.7.90 (#2958) 2026-08-02 21:57:19 -07:00
Miguel Ángel 411ada0d90 chore: release v0.7.89 (#2953) 2026-08-02 20:25:14 +02:00
Vance Ingalls 74fadf69c4 chore: release v0.7.88 2026-08-01 16:53:08 -07:00
Miguel Ángel 89e970fbd7 chore: release v0.7.87 (#2935) 2026-07-31 22:36:46 +02:00
Vance IngallsandClaude Opus 5 71ee156dac feat(studio): browser canary binding + leaf subpath imports
Adds the Studio (browser) binding so a canary can span the CLI and the editor,
and fixes a bundling mistake the studio test suite caught.

## The binding

Same public API as the CLI — `isCanaryEnabled("name")` — so a call site reads
identically whether it runs in Node or the browser. Three inputs differ:

- UNIT ID: `resolveStudioDistinctId()`, which already adopts
  `window.__HF_CLI_DISTINCT_ID` when the CLI launched Studio. A CLI-launched
  Studio therefore lands in the SAME cohort as the CLI: a rollout spanning
  render and editor is coherent for that user instead of enrolling their
  terminal but not their editor. A test pins that the id is passed through
  unmodified — prefixing or re-hashing it would silently break that parity.

- OVERRIDE: no `process.env` in a page, so `?hf_canary_<name>=on` mirrored
  into sessionStorage. Session scope is deliberate. A URL is the right carrier
  (shareable — "support: open this link"), but persisting a URL-borne override
  to localStorage would let one click silently pin a browser into a cohort
  forever, long after anyone remembers why. Closing the tab is the reset;
  `=reset` clears it explicitly.

- EXCLUSION: `navigator.webdriver` stands in for the CLI's `is_ci`. Automated
  browsers mint a fresh localStorage id per run, so they would hop cohorts
  between runs — noise in the signal, nothing learned about real users. An
  override still reaches them, which is how you test a canary under Playwright.

Studio's `trackEvent` now attaches `canaries` to every event, mirroring the CLI.

## The bundling fix

Importing the `@hyperframes/core` barrel into studio browser code broke two
unrelated hook test files with an esbuild TextEncoder invariant violation. The
barrel re-exports the whole core surface (parsers, lint, studio-server), so it
drags a Node-oriented dependency graph into a browser bundle — the test
failure was the symptom, the bundle bloat was the bug.

`@hyperframes/core` now exposes `./canary` and `./canary-registry`, declared in
packages/core/package-subpaths.json (the generated source of truth for exports —
hand-editing package.json is reverted by the sync script) and marked
`environments: [browser, bun, node]`. Both the studio AND cli bindings import
the leaf modules; the CLI gets the same benefit for a different reason, since
this resolves on the startup path — the reason the producer is lazily loaded.

Verified: the two hook files pass again; 269 studio files / 2982 tests, 98
core / 1433, 166 cli / 2194 green, `bun run lint` clean including the subpath
check. Fault-injection confirms both design decisions are pinned — swapping
session for local storage fails the scope test, prefixing the unit id fails the
CLI/Studio cohort-parity test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 15:14:35 -07:00
Vance Ingalls 12ee861123 chore: release v0.7.85 2026-07-30 10:38:10 -07:00
Vance Ingalls 065293ecf3 chore: release v0.7.84 2026-07-30 01:39:54 -07:00
Vance Ingalls 5244dde5f1 chore: release v0.7.83 2026-07-29 18:37:58 -07:00
Miguel Ángel b904343949 chore: release v0.7.82 2026-07-29 11:25:04 +00:00
James 0d42d65525 chore: release v0.7.81 2026-07-29 06:11:11 +00:00
Miguel Ángel 5829515932 chore: release v0.7.80 2026-07-29 02:22:54 +00:00
Miguel Angel Simon Sierra 659e22656e fix(studio): switch keyframe ease modes optimistically 2026-07-29 03:44:31 +02:00
Miguel Ángel a68729bcf1 chore: release v0.7.79 2026-07-28 23:06:45 +00:00
Vance Ingalls c691869e22 chore: release v0.7.78 2026-07-28 05:27:29 -07:00
Miguel Ángel 3c857d768b chore: release v0.7.77 2026-07-28 02:12:41 +00:00
James 5cad2bc312 chore: release v0.7.76 2026-07-27 05:08:40 +00:00
James 45b458c007 chore: release v0.7.75 2026-07-27 03:38:16 +00:00
James Russo 0cc78c1d42 chore: release v0.7.74 (#2818) 2026-07-26 21:57:54 -04:00
James 527751c871 chore: release v0.7.73 2026-07-26 21:25:03 +00:00
Ular Kimsanov e2e61b0767 Merge pull request #2798 from heygen-com/feat/agent-native-color-grading
feat(cli): expose agent-native color grading
2026-07-26 03:23:29 -07:00
ukimsanov c1dde28980 fix(cli): address media treatment review findings 2026-07-26 01:25:42 -07:00
James Russo 72ae39e3d0 chore: release v0.7.72 (#2800) 2026-07-26 02:01:10 -04:00
Miguel Angel Simon Sierra 5acbf240cb feat(core): add deterministic keyframe ease runtime 2026-07-25 14:12:16 +02:00
James d07d1874b6 chore: release v0.7.71 2026-07-24 19:03:16 +00:00
Vance Ingalls b9c3e3a4b6 chore: release v0.7.70 2026-07-23 20:59:30 -07:00