mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-08-31 02:41:44 +00:00
sync/hyperframes-codegen-3ff80b22
18
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Miguel Ángel
|
0285a711e9 |
feat(core): expose pretext text measurement on window.__hyperframes (#3302)
## What
Exposes the `pretext` text-measurement API on `window.__hyperframes`, so the API our agent-facing docs already describe actually exists.
Adds `pretext.prepare`, `.layout`, `.prepareWithSegments`, `.measureLineStats`, `.measureNaturalWidth`.
## Why
`skills/hyperframes-core/references/determinism-rules.md` is required reading for any agent authoring a composition. Line 59 tells them to call `window.__hyperframes.pretext.prepare(text, font)` then `pretext.layout(prepared, maxWidth, lineHeight)` for text measurement without a DOM reflow.
That object did not exist. The runtime exposed exactly `fitTextFontSize` and `getVariables`. Any composition following the documented recipe threw at runtime.
Deleting the doc line was the smaller change, but reflow-free measurement is genuinely the right tool for sizing text per frame, and `fitTextFontSize` is already built on it. Making the docs true is the better fix.
## How
- New `packages/core/src/text/pretext.ts` assembles the exposed surface in one place, with the include/exclude rationale next to it.
- `entry.ts` attaches it alongside the existing helpers. Sub-compositions inherit it for free: the scoping shim builds its scoped variant with `Object.assign({}, base, { getVariables })`, so anything added to the base object is carried through.
Two deliberate decisions:
**Wider than the doc named.** `layout()` returns only `{ lineCount, height }`. The doc's own "shrinkwrap containers" use case needs a width, which is impossible with just `prepare` + `layout`. `measureNaturalWidth` and `measureLineStats` make that claim achievable; `prepareWithSegments` is their required input.
**`clearCache` and `setLocale` withheld.** Both mutate state shared across compositions. Exposing them would let one composition change how a later one measures, making a render depend on what ran before it.
**Doc correction.** The reference called this "pure arithmetic, ~0.0002 ms per call". Not quite: `prepare` measures fonts through a canvas and throws outside a browser. Only the steps after a prepared string are arithmetic. Reworded, and documented the width helpers and the omissions.
## Trade-off
The runtime bundle grows **4,903 bytes (+1.30%)**, from 377,865 to 382,768. That ships inline in every composition. Measured by building the artifact with and without the change.
## Test plan
- [x] Unit tests added/updated
- [x] Manual testing performed
- [x] Documentation updated (if applicable)
`packages/core/src/text/pretext.test.ts` guards the shape of the published surface: the two documented names exist, the width helpers exist, and the two stateful functions are absent. Behaviour is deliberately not asserted there. `prepare` needs a canvas, and mocking it (as `fitTextFontSize.test.ts` must) would assert nothing real.
Real behaviour was verified by rendering a composition that calls the documented API:
```
lines=2 height=216 naturalWidth=1785
```
Self-consistent: natural width 1785 exceeds the 1600 container so it wraps to 2 lines, and 2 x 108 line-height is exactly the reported 216. The frame was inspected visually.
Also run:
- `packages/core` full suite from the package root: **903 passed, 46 files**
- `tsc --noEmit` and `tsc --noEmit -p tsconfig.runtime.json`: clean
- `oxlint` / `oxfmt`: clean
## Follow-ups (not in this PR)
An audit of the wider attribute surface found several more doc/runtime mismatches, including `data-gpu-mode` documented as an HTML attribute when it is a config field, and `data-no-timeline` being real, load-bearing, and absent from the table agents read. Those are separate changes.
|
||
|
Vance IngallsandClaude Opus 5
|
b839dbd2cc |
fix(core): give the chorus and phaser LFOs a phase, and unwire them on dispose (#3183)
* fix(studio): drop the unused DomEditSelection import in PropertyPanelFlat CI caught it on PR #3026 (wa-12-panel-params); a later refactor in the stack removed the last use of the type here without removing the import. * fix(studio): close the typecheck and fallow gaps wa-18b-reschedule opened useAutomationLanes.ts's write() assumed gesture-scoped coalescing and a preview-only commit that useDomEditAttributeCommits.ts never grew — backported that option support from its own later commit so the two sides of the API agree. The paste path and its tests were missing the box selection's v0/v1 bounds a sibling commit added to AutomationSelection. The FX panel's carve controls still edited the six mechanism numbers (maxCutDb, bands, intelligibilityBias) after carveProfile() collapsed authoring to one Strength knob, so those fields no longer existed on HfCarveSettings; UI now edits strength, and analyseCarveBands is called with carveProfile(strength). Also closes fallow's complexity, dead-code and duplication findings on this PR's diff: extracted automationLaneDragMath.ts (pure group/point-move math) and useAutomationRangeDrag.ts (the marquee-select gesture) out of useAutomationLaneGestures.ts, pulled a couple of render-loop ternaries and a resolver into named functions, dropped an export nothing outside its file used, and shared a step-simplifier between audioCarve's two envelope builders. The edge-stretch vs. box-select priority test in TimelineAutomationLane.test was still pinning the pre-box-select rule (edge wins over a point sitting on it) that a sibling commit deliberately reversed — a point inside the box is now selected content, so grabbing it drags the group instead. Updated the test to the shipped rule instead of the old one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(core): cap the via conic's weight so an edge-clamped via point can't NaN A via point pulled out past the segment (viaX: 5, viaY: -3) clamps to (0.999, 0.001) — exactly on the steady region's edge, where edge - viaX is 0. viaConic divided by that zero to get an infinite weight, and shapeVia turned Infinity into NaN a few steps later (Infinity - Infinity in the quadratic coefficient). NaN reaching setValueCurveAtTime silences the automated parameter for the rest of the render. Capped the weight at 1e6 instead of leaving it unbounded — past that point the arc already reads as touching the via point, so nothing visible is lost. Also hardened shapeVia's existing denominator guard (`<= 0`) to `!(> 0)`, since NaN fails the original comparison and fell through it. Review by Miga (PR #3208). * fix(studio-server): fingerprint the proactive waveform cache key too The route already keys the waveform cache on the asset's size and mtime as well as its path, so a rebuilt-in-place file gets fresh peaks instead of stale ones. generateWaveformCache — the proactive path that runs on upload — still called buildWaveformCacheKey with the path alone, so it wrote to a different key than the route reads from (making the pre-generated cache never found) and kept the exact collision bug this fingerprint exists to fix on its own path. Review by Miga (PR #3211). * style(docs): run oxfmt on the /hyperframes-audio skill docs Table column widths had drifted out of alignment with oxfmt's own rules, failing format:check and blocking the Preflight gate every downstream branch inherits. Whitespace only, no content change. * fix(core): stop \b from missing underscore-separated names, guard clipsOverlap's negative duration \b treats `_` as a word character, so \bbed\b never matched bed_01, music_bed_loop, or theme_song, and \bvo\b/\bvox\b/\btts\b had the same gap — an underscore-separated bed classified as "unknown" and could end up offered as its own carve source. Replaced the short hints with a boundary that actually excludes letters and digits on both sides. clipsOverlap computed end = start + duration without guarding sign, so a negative duration put end before start — an interval that does not describe anything, and one specific case showed it silently dropping a real overlap (a shorter, earlier broken end rejected a clip that genuinely contained the point). Duration clamps to zero instead: a clip cannot un-play time, and a zero-length clip at its start is the sane reading of "duration nobody wrote down as positive." Review by Miga (PR #3212). * fix(studio): widen PropertyPanel's resetModules render timeout again The 20s margin (already once widened for the same reason) is timing out in CI's full-monorepo Test run — the resetModules()+fresh-import render this test needs is uncached and competes with every other package's test suite for the same worker pool, and the same test passes in well under 2s standalone. Went to 45s rather than re-tuning to whatever number happens to clear the current CI load, since that number moves every time CI gains a package. * fix(studio): stop the single-candidate auto-apply carve firing twice Two auto-apply effects both fire when sourceOptions.length === 1: the multi-candidate effect only guards length === 0, so a single candidate passes it too, and the single-candidate effect passes its own guard right after — both compute the same sources list and both call setCarve, so the common case (one narrator, one bed) triggered two decodes, two FFT runs, and two concurrent attribute writes for one decision. The multi-candidate effect now defers to its sibling for exactly one candidate, which already has its own detailed handling for that case. Review by Miga (PR #3213). * feat(core): carve against every voice over a bed, always (#3212) * feat(core): carve against every voice over a bed, always dynamically A bed usually runs under a whole sequence — a narrator, an interview answer, a second presenter — and carving against one of them left the others fighting it. `source` becomes `sources`, and `mixCarveSources` sums every voice onto the BED's clock before anything is measured. That is what keeps one analysis sufficient: the chain is fixed, so there is no per-voice filter to switch between, and bands drawn from all the speech there is with envelopes that rise wherever any of it happens answer the actual question — where and when is speech masking this bed. Summed rather than averaged: two people talking at once mask more than either alone. Audio before the bed starts is dropped rather than folded in at zero, since it plays over nothing and shifting it would put a cut where there is no voice. `dynamic` is gone. A fixed depth thins the bed through every pause, and once both have been heard there is no reason to want it, so every carve follows the speech. Two helpers the panel and the headless script now share instead of each carrying a copy — two definitions of "what does this name suggest" drift, and then the two disagree about which track is the voice: - `classifyAudioName` reads a track's kind from its id and filename together. `unknown` is deliberately common: treating an unrecognised name as "not a voice" would hide the one track somebody needs to pick. - `clipsOverlap` keeps out a voice that never plays while the bed does. An unwritten duration counts as unbounded, not zero — refusing a clip whose length the composition leaves to the media would drop the commonest case there is. Files written before this still load: a single `source` reads as a one-voice list, a stored `dynamic` is ignored, and an absent attribute means the defaults whole. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(core): stop \b from missing underscore-separated names, guard clipsOverlap's negative duration \b treats `_` as a word character, so \bbed\b never matched bed_01, music_bed_loop, or theme_song, and \bvo\b/\bvox\b/\btts\b had the same gap — an underscore-separated bed classified as "unknown" and could end up offered as its own carve source. Replaced the short hints with a boundary that actually excludes letters and digits on both sides. clipsOverlap computed end = start + duration without guarding sign, so a negative duration put end before start — an interval that does not describe anything, and one specific case showed it silently dropping a real overlap (a shorter, earlier broken end rejected a clip that genuinely contained the point). Duration clamps to zero instead: a clip cannot un-play time, and a zero-length clip at its start is the sane reading of "duration nobody wrote down as positive." Review by Miga (PR #3212). --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(studio): port the carve UI off the removed source/dynamic fields #3212 (accidentally squash-merged into this branch instead of main) changed HfCarveSettings from a single `source` + `dynamic` toggle to a `sources` list with dynamic mode removed outright — the multi-voice UI consumer that goes with that shape lands in the very next PR, so this branch was left with a type that no longer matched its own code. Minimal port, not the multi-voice redesign that PR does properly: the "Listen to" picker and analyse() treat sources[0] as the one voice this UI still understands, and every dynamic-mode branch (the automated envelope lanes, the toggle, the checkbox) is gone along with the field — a carve is now always the static value the analysis computes, matching what the type change made permanent. Test suite trimmed the same way: the automation-lane and toggle tests covered behavior that no longer exists. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Opus 5
|
c2996c8626 |
feat(studio): the FX panel, generated from the registry (#3022)
* feat(engine): render audio FX in an OfflineAudioContext Reads `data-fx-chain` off an audio element and runs the chain over the trimmed WAV before volume automation is baked in — effects should see the raw signal, and the envelope belongs on their output. The processing happens in an OfflineAudioContext inside the headless browser the engine already drives, running the same graph builders the studio previews with. That is the point of the approach: one implementation per effect, so the render agreeing with the preview is a property of the architecture rather than a tolerance to police. Reimplementing each effect as an FFmpeg filter would mean two implementations to keep in step, and for the dynamics processors and modulated delays there is no filter that behaves the same way. `build:audio-fx-runtime` bundles the graph builders into an injectable IIFE, following the same pattern as the existing runtime artifacts, so the browser runs exactly the code the studio does. The page loads from a file:// URL rather than about:blank because AudioWorklet is only exposed in a secure context — the compressor, limiter, gate and bitcrush processors would otherwise fail to register with an opaque error. file:// qualifies and needs no listening socket. The chain is serialised into the attribute the way colour grading carries its config, so there is no side-car file to resolve or lose. An FX failure is fatal for the whole mix rather than a per-track soft failure. Every other audio failure mode degrades gracefully — the track drops, siblings continue — but substituting the dry signal for a processed one ships a render that sounds plausible and is not what the author set up. Since the per-element work races under Promise.all, an internal AbortController chained off the caller's signal aborts in-flight siblings before workDir is removed. * feat(core): voiceover carve analysis Finds the bands a voice occupies so a music bed can be dipped there, letting the voice sit in front without ducking the whole track. Carve is a relationship between two tracks rather than an effect on one, so it stays out of the FX chain. What it emits is an ordinary chain of peaking filters, so a carve composes with whatever else is on the track and needs no separate rendering path. Selection is weighted toward intelligibility rather than raw voice energy. Ranking purely by power lands on the fundamental almost every time, because that is where a voice is loudest — but the masking that actually hurts a voiceover happens higher up, and dipping 160 Hz mostly just thins the bed. The bias is a control, not a constant: at 0 it follows raw energy, at 1 it weights toward 1-3 kHz. Ranking happens in dB, which matters more than it looks. Speech spreads 20-30 dB across these bands — it falls off roughly 6 dB per octave above the fundamental — so a weighting has to be on that scale to move anything at all. A multiplicative weight of `1 - bias + bias * shaped` is bounded below by `1 - bias`, capping its influence at 10*log10(1/(1 - bias)): 5.2 dB at the 0.7 default, 3 dB at 0.5. That is no influence against a real voice — every bias short of ~0.95 would rank exactly like bias 0 and carve the fundamental, the outcome the bias exists to prevent, while looking decisive against a fixture whose bands sit 2 dB apart. So the bias is a dB penalty, zero at 2 kHz and worth up to 30 dB at full strength, and relative cut depths come from a dB difference rather than a ratio of weighted linear powers. The bias reweights ranking without overriding the spectrum — a band the voice has no energy in is not worth carving, and scores -Infinity rather than competing — so a strongly low-pitched voice can still select low at full bias. What the tests hold is that biasing never selects lower than the unbiased ranking, that the DEFAULT bias reaches the presence region on a voice with a realistic tilt, and that bias 0 still follows raw power exactly. Includes a radix-2 FFT rather than a dependency; one Welch-style averaged spectrum over third-octave bands does not justify pulling in a DSP library. * fix(engine): keep the FX render 16-bit, stereo, and correctly sized Three defects in the offline FX path, none of which any test could see. **Float output silently disabled sample-accurate volume automation.** The writer emitted 32-bit IEEE float; the very next mixer step bakes the volume envelope into the samples and accepts only 16-bit PCM, returning null otherwise. So enabling any effect downgraded that track to the ffmpeg expression path — capped at 32 straight segments, quantising a curved envelope, and on a dense one falling back to base volume. It now writes 16-bit PCM, clamped rather than wrapped so a limiter at 0 dB or a resonant filter cannot turn overshoot into a click. A test asserts the baker accepts the writer's own output and actually fades it. **Everything was folded to mono.** `prepareAudioTrack` goes out of its way to emit stereo — its pan filter exists to dodge ffmpeg's 3 dB mono-to-stereo rematrix — and this folded it, then wrote one channel. So adding a single peaking EQ collapsed a bed's width and cost ~3 dB in the render, while preview stayed stereo. Channels now travel as one plane each, through an OfflineAudioContext of the same width, and come back interleaved. **Small results decoded the wrong length.** `new Float32Array(buf.buffer)` discards byteOffset and byteLength, and Node pools small allocations: a 400-byte payload sits at offset 8 inside an 8 KiB pool, so a clip under ~1024 samples decoded as 2048 samples of unrelated memory — and the empty-result guard could not see it. The reader has the mirror-image fix: a float data chunk on an odd boundary (ffmpeg's pcm_f32le writes fmt(18) + fact, landing `data` at 58) now copies instead of throwing RangeError on an unaligned view. The tail limitation is now stated rather than mis-stated: the context is exactly as long as the input, so a reverb or delay still ringing is cut there. The old comment claimed the opposite. How far a tail may run past a clip's end changes the clip's length in the mix, so it is a product decision, not one to make here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(producer): report an FX render failure as an audio error `processCompositionAudio` reports per-track failures in its result, but an FX failure it cannot degrade past — a browser that will not launch, a chain that will not build — rejects instead. `runAudioStage` had no try, so that rejection escaped to the orchestrator as an unclassified pipeline exception, losing the stage/owner/retryable classification this stage exists to attach, and skipping its abort check on the way out. It now lands in `audioError` alongside every other cause, while an abort still keeps its own shape rather than being reported as an audio problem. Not done here: committing the generated `audio-fx-runtime-inline.ts` so a fresh clone typechecks packages/engine without building first. The bundle is built from the stub, and the stub changes three times across this stack — so the artifact differs per branch and would conflict on every restack. Its model, position-edits-render-inline.ts, is committed only because it is stable. Building before testing is this monorepo's existing contract (studio's tests need core's dist too), so the gap is not specific to audio FX and is better closed by a build ordering gate than by committing a per-branch artifact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(engine): skip the browser FX render cases when there is no browser CI's `Test` job was red on this PR with four failures, all the same cause: Failed to launch the browser process: spawn /home/runner/.cache/hyperframes/chrome/chrome-headless-shell The job installs ffmpeg and no browser, deliberately — every other suite that needs an external binary already guards on it (`describe.skipIf(!HAS_FFMPEG)`). These cases were the only ones assuming a Chrome, so they failed on an absent dependency rather than on anything about the code. Guards on `resolveHeadlessShellPath()` — the same resolver `acquireBrowser` launches through, so the check cannot drift from the thing it guards the way a hard-coded cache path would. A configured path that does not exist throws; that is caught and read as "cannot run here". Checked both directions rather than just the green one: with a browser all 11 cases run and pass, and with `HYPERFRAMES_BROWSER_PATH` pointed at a missing binary exactly 3 skip and the other 8 still run. A guard that silently skipped everything would have looked identical in CI. They keep their value where it exists — every developer machine, and any job that has run `hyperframes browser ensure`. Not touched: the CodeQL failure on this PR is a run from 2026-08-07, five days and several force-pushes stale. None of the 17 open repo alerts are in files this PR changes; it re-runs on this push. * chore(engine): suppress the temp-file alert with the reason it is safe CodeQL flags `writeWav`'s `writeFileSync` as js/insecure-temporary-file (high) — the one new alert on #3021, and the reason its CodeQL check is red. It is a false positive, and the comment says why rather than just silencing it: `path` is always inside a directory made by `mkdtempSync`, never a name assembled directly under `tmpdir()`. Both callers are covered — the browser host page writes into `mkdtempSync(join(tmpdir(), "hf-fx-host-"))`, and the render output goes to the producer work dir, itself `mkdtempSync(join(tempRoot, "producer-project-"))`. mkdtemp picks the random suffix and creates the directory 0700 in one syscall, so the predictable filename inside it cannot be pre-created or symlinked by another user, which is the attack the rule is about. The analyzer sees the dataflow reach `tmpdir()` and not the mkdtemp in between. Suppressed inline rather than dismissed in the UI, so the justification lives next to the code and the rule stays live for anything added later in this file. Matches the repo's existing convention — `planV2.ts:222` carries an `lgtm[js/insecure-temporary-file]` for a different reason on the same rule. Correcting myself: I first reported this alert as not real, having intersected the PR's files against the default-branch alert list, which does not contain PR-ref alerts. Querying ?ref=refs/pull/3021/merge returns it straight away. * test(engine): probe ffmpeg and Chrome instead of assuming them Two failures on #3021's Test job, both about the environment rather than the code under test. **Bare `ffmpeg` is not on PATH in CI.** The 16-bit fixture shelled out to `execFileSync("ffmpeg", ...)` and died with ENOENT. The job does provide ffmpeg, through `prepare-ffmpeg-bin`, which is what `getFfmpegBinary()` resolves — every other ffmpeg-dependent suite in this package already goes through it. Now this one does too, and the case is `skipIf(!HAS_FFMPEG)` so a contributor without ffmpeg skips rather than fails. **The browser guard trusted the wrong thing.** It asked `resolveHeadlessShellPath()` and treated a returned path as "a browser is here". CI's cache holds a chrome-headless-shell that resolves and then fails to spawn — a partial download is indistinguishable from a working one by `existsSync`, which is all that resolver checks. So the three browser cases ran anyway and failed on the launch. It now runs `--version` and requires exit 0, which is the same probe the ffmpeg suites use: ask the binary, do not infer from the filesystem. Checked both directions rather than just the green one. With a working browser all 11 cases run and pass; with `HYPERFRAMES_BROWSER_PATH` pointed at a binary that exits non-zero — CI's exact situation — exactly 3 skip and the other 8 still run. A guard that quietly skipped everything would have looked identical on the CI summary. * feat(core): register the audio-fx-rack canary at 0% Lands the rollout switch dark, per the registry's own procedure: "Start at percentage: 0 and merge that — a canary at 0 is dead code you can land safely and ramp without a code review." Declared at the bottom of the stack so every branch above can read it. The gate itself goes in at wa-4-fx-panel, where the rack first appears. Scope is deliberate and stated in the description: it gates the AUTHORING surface only. A composition that already carries `data-fx-chain` still plays and renders it. A canary should stage who can REACH a feature, not make an attribute somebody already wrote silently inert — an agent that writes a chain through the skill would otherwise produce a file whose audio processing vanishes with no error. * feat(studio): audio FX panel generated from the registry Controls for the whole chain: add, remove, reorder, bypass, and every knob each effect declares. Nothing in the panel knows what a compressor is. The registry supplies each parameter's range, step, unit and scale and the panel renders what it finds, so adding an effect or a knob upstream needs no change here, and the panel cannot offer a value the renderer would reject — a typed-in figure is clamped into the declared range on the way through. Frequency and time controls span three or four decades, so those declare a log scale and the slider maps exponentially; a linear slider would spend most of its travel somewhere useless. Reorder is a first-class control because chain order changes the sound: a reverb before a compressor is not the same as after. Carve gets its own block rather than an entry in the add menu, with a picker for the voice track to listen to. It processes this track based on another one, which is how a sidechain control works — it lives on the track that changes, and names the source. * feat(studio): show the Audio FX section on audio tracks Adds `audioFx` to the editing-affordances contract and renders the FX panel in the inspector when an `<audio>` element is selected. The section is audio-only. A `<video>` carries its sound on a separate `<audio>` element, so an FX chain on the video would have nothing to process. Chain and carve settings are written straight back onto the element as serialised attributes, the way colour grading carries its config, so persistence is an ordinary attribute write and needs no new server route. A chain that cannot be parsed renders as empty rather than breaking the panel, and the attribute is left untouched until the user changes something. The collapsed group summarises what is on the track ("2 effects + carve") so the state is visible without expanding it. Wired into PropertyPanelFlat rather than PropertyPanel: STUDIO_FLAT_INSPECTOR_ENABLED defaults to true, so the flat inspector is what actually renders. * refactor(studio): lift audioFxSummary out of PropertyPanelFlat `PropertyPanelFlat.tsx` is 612 lines here against the repo's 600-line cap, so the required File size check is red — the sole reason this PR is blocked. The review says as much: "mechanical fix (~5 min), not a design problem. Code itself is LGTM." Moves `audioFxSummary` to `audioFxSummary.ts`, the same file a later branch creates for it. Deliberately the smallest cut that clears the cap rather than the whole `AudioFxGroup` extraction: every later commit in the stack edits AudioFxGroup, so moving it here would collide with each of them, while almost nothing touches this function. 595 lines. * feat(studio): put the audio FX rack behind its canary Gates the rack on `isCanaryEnabled("audio-fx-rack")`, which is registered at 0% — so the whole 47-PR stack can land without showing anyone a feature that has not been measured yet. The gate sits on the AUTHORING surface and nowhere else. The runtime and the render still honour a `data-fx-chain` already on an element, so a composition written through the skill or by `carve.mjs` keeps its processing rather than going silently dry for anyone outside the cohort. A canary should stage who can REACH a feature, not make an attribute somebody already wrote stop working with no error. Gated at the panel rather than in `resolveEditingSections`: the affordance resolver is a pure function in core describing what an element CAN support, and rollout state is not a property of an `<audio>` tag. Pinned the 0% with a test, and checked it fails at 25 — a ramp should have to break something that says "this ships dark" out loud. One gap, stated rather than papered over: the gate itself has no unit test. I wrote one and deleted it, because `PropertyPanel.test.tsx`'s harness never renders the Audio FX group for its audio fixture even with the gate removed — so the test passed for the wrong reason in the off case and could not pass at all in the on case. A test that cannot fail for the right reason is worse than none. Verifying the gate needs the panel harness to mount that section first, which is its own change. * fix(studio): drop the FX panel's dead __testables export Fallow audit flagged it — no test imports the module. * fix(core,studio): clear the remaining Fallow audit findings on the FX panel - Split FxSection's per-node row into FxNodeRow + FxNodeControls so the CRAP score (31.6, threshold 30) splits across two smaller units instead of moving wholesale with one extraction. - Dedupe the repeated "open the add menu, read its items" block in propertyPanelFxSection.test.tsx into openAddMenuItems(). - Merge build-audio-fx-runtime.ts and build-position-edits-render.ts into one build-inline-artifact.ts, config-selected by CLI arg — the two scripts were a byte-for-byte clone save for names. - Exempt canary.test.ts's rawFnv (a deliberate independent reimplementation used to cross-check canaryBucket, per its own docstring) and the property-panel test files' shared renderInto/mount scaffolding (pre-existing across 9 files, 2 outside this stack) in .fallowrc.jsonc, consistent with this file's existing exemptions for the same class of intentional/pre-existing duplication. * fix(ci): allowlist the build-script consolidation in the no-main-deletions guard build-audio-fx-runtime.ts and build-position-edits-render.ts were merged into build-inline-artifact.ts to kill a fallow duplication finding; the deletion guard flagged that as an accidental loss since main still has both originals. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Vance IngallsandClaude Opus 5
|
cc40e35aa0 |
feat(engine): render the FX chain offline, and the carve analysis behind it (#3021)
* feat(engine): render audio FX in an OfflineAudioContext Reads `data-fx-chain` off an audio element and runs the chain over the trimmed WAV before volume automation is baked in — effects should see the raw signal, and the envelope belongs on their output. The processing happens in an OfflineAudioContext inside the headless browser the engine already drives, running the same graph builders the studio previews with. That is the point of the approach: one implementation per effect, so the render agreeing with the preview is a property of the architecture rather than a tolerance to police. Reimplementing each effect as an FFmpeg filter would mean two implementations to keep in step, and for the dynamics processors and modulated delays there is no filter that behaves the same way. `build:audio-fx-runtime` bundles the graph builders into an injectable IIFE, following the same pattern as the existing runtime artifacts, so the browser runs exactly the code the studio does. The page loads from a file:// URL rather than about:blank because AudioWorklet is only exposed in a secure context — the compressor, limiter, gate and bitcrush processors would otherwise fail to register with an opaque error. file:// qualifies and needs no listening socket. The chain is serialised into the attribute the way colour grading carries its config, so there is no side-car file to resolve or lose. An FX failure is fatal for the whole mix rather than a per-track soft failure. Every other audio failure mode degrades gracefully — the track drops, siblings continue — but substituting the dry signal for a processed one ships a render that sounds plausible and is not what the author set up. Since the per-element work races under Promise.all, an internal AbortController chained off the caller's signal aborts in-flight siblings before workDir is removed. * feat(core): voiceover carve analysis Finds the bands a voice occupies so a music bed can be dipped there, letting the voice sit in front without ducking the whole track. Carve is a relationship between two tracks rather than an effect on one, so it stays out of the FX chain. What it emits is an ordinary chain of peaking filters, so a carve composes with whatever else is on the track and needs no separate rendering path. Selection is weighted toward intelligibility rather than raw voice energy. Ranking purely by power lands on the fundamental almost every time, because that is where a voice is loudest — but the masking that actually hurts a voiceover happens higher up, and dipping 160 Hz mostly just thins the bed. The bias is a control, not a constant: at 0 it follows raw energy, at 1 it weights toward 1-3 kHz. Ranking happens in dB, which matters more than it looks. Speech spreads 20-30 dB across these bands — it falls off roughly 6 dB per octave above the fundamental — so a weighting has to be on that scale to move anything at all. A multiplicative weight of `1 - bias + bias * shaped` is bounded below by `1 - bias`, capping its influence at 10*log10(1/(1 - bias)): 5.2 dB at the 0.7 default, 3 dB at 0.5. That is no influence against a real voice — every bias short of ~0.95 would rank exactly like bias 0 and carve the fundamental, the outcome the bias exists to prevent, while looking decisive against a fixture whose bands sit 2 dB apart. So the bias is a dB penalty, zero at 2 kHz and worth up to 30 dB at full strength, and relative cut depths come from a dB difference rather than a ratio of weighted linear powers. The bias reweights ranking without overriding the spectrum — a band the voice has no energy in is not worth carving, and scores -Infinity rather than competing — so a strongly low-pitched voice can still select low at full bias. What the tests hold is that biasing never selects lower than the unbiased ranking, that the DEFAULT bias reaches the presence region on a voice with a realistic tilt, and that bias 0 still follows raw power exactly. Includes a radix-2 FFT rather than a dependency; one Welch-style averaged spectrum over third-octave bands does not justify pulling in a DSP library. * fix(engine): keep the FX render 16-bit, stereo, and correctly sized Three defects in the offline FX path, none of which any test could see. **Float output silently disabled sample-accurate volume automation.** The writer emitted 32-bit IEEE float; the very next mixer step bakes the volume envelope into the samples and accepts only 16-bit PCM, returning null otherwise. So enabling any effect downgraded that track to the ffmpeg expression path — capped at 32 straight segments, quantising a curved envelope, and on a dense one falling back to base volume. It now writes 16-bit PCM, clamped rather than wrapped so a limiter at 0 dB or a resonant filter cannot turn overshoot into a click. A test asserts the baker accepts the writer's own output and actually fades it. **Everything was folded to mono.** `prepareAudioTrack` goes out of its way to emit stereo — its pan filter exists to dodge ffmpeg's 3 dB mono-to-stereo rematrix — and this folded it, then wrote one channel. So adding a single peaking EQ collapsed a bed's width and cost ~3 dB in the render, while preview stayed stereo. Channels now travel as one plane each, through an OfflineAudioContext of the same width, and come back interleaved. **Small results decoded the wrong length.** `new Float32Array(buf.buffer)` discards byteOffset and byteLength, and Node pools small allocations: a 400-byte payload sits at offset 8 inside an 8 KiB pool, so a clip under ~1024 samples decoded as 2048 samples of unrelated memory — and the empty-result guard could not see it. The reader has the mirror-image fix: a float data chunk on an odd boundary (ffmpeg's pcm_f32le writes fmt(18) + fact, landing `data` at 58) now copies instead of throwing RangeError on an unaligned view. The tail limitation is now stated rather than mis-stated: the context is exactly as long as the input, so a reverb or delay still ringing is cut there. The old comment claimed the opposite. How far a tail may run past a clip's end changes the clip's length in the mix, so it is a product decision, not one to make here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(producer): report an FX render failure as an audio error `processCompositionAudio` reports per-track failures in its result, but an FX failure it cannot degrade past — a browser that will not launch, a chain that will not build — rejects instead. `runAudioStage` had no try, so that rejection escaped to the orchestrator as an unclassified pipeline exception, losing the stage/owner/retryable classification this stage exists to attach, and skipping its abort check on the way out. It now lands in `audioError` alongside every other cause, while an abort still keeps its own shape rather than being reported as an audio problem. Not done here: committing the generated `audio-fx-runtime-inline.ts` so a fresh clone typechecks packages/engine without building first. The bundle is built from the stub, and the stub changes three times across this stack — so the artifact differs per branch and would conflict on every restack. Its model, position-edits-render-inline.ts, is committed only because it is stable. Building before testing is this monorepo's existing contract (studio's tests need core's dist too), so the gap is not specific to audio FX and is better closed by a build ordering gate than by committing a per-branch artifact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(engine): skip the browser FX render cases when there is no browser CI's `Test` job was red on this PR with four failures, all the same cause: Failed to launch the browser process: spawn /home/runner/.cache/hyperframes/chrome/chrome-headless-shell The job installs ffmpeg and no browser, deliberately — every other suite that needs an external binary already guards on it (`describe.skipIf(!HAS_FFMPEG)`). These cases were the only ones assuming a Chrome, so they failed on an absent dependency rather than on anything about the code. Guards on `resolveHeadlessShellPath()` — the same resolver `acquireBrowser` launches through, so the check cannot drift from the thing it guards the way a hard-coded cache path would. A configured path that does not exist throws; that is caught and read as "cannot run here". Checked both directions rather than just the green one: with a browser all 11 cases run and pass, and with `HYPERFRAMES_BROWSER_PATH` pointed at a missing binary exactly 3 skip and the other 8 still run. A guard that silently skipped everything would have looked identical in CI. They keep their value where it exists — every developer machine, and any job that has run `hyperframes browser ensure`. Not touched: the CodeQL failure on this PR is a run from 2026-08-07, five days and several force-pushes stale. None of the 17 open repo alerts are in files this PR changes; it re-runs on this push. * chore(engine): suppress the temp-file alert with the reason it is safe CodeQL flags `writeWav`'s `writeFileSync` as js/insecure-temporary-file (high) — the one new alert on #3021, and the reason its CodeQL check is red. It is a false positive, and the comment says why rather than just silencing it: `path` is always inside a directory made by `mkdtempSync`, never a name assembled directly under `tmpdir()`. Both callers are covered — the browser host page writes into `mkdtempSync(join(tmpdir(), "hf-fx-host-"))`, and the render output goes to the producer work dir, itself `mkdtempSync(join(tempRoot, "producer-project-"))`. mkdtemp picks the random suffix and creates the directory 0700 in one syscall, so the predictable filename inside it cannot be pre-created or symlinked by another user, which is the attack the rule is about. The analyzer sees the dataflow reach `tmpdir()` and not the mkdtemp in between. Suppressed inline rather than dismissed in the UI, so the justification lives next to the code and the rule stays live for anything added later in this file. Matches the repo's existing convention — `planV2.ts:222` carries an `lgtm[js/insecure-temporary-file]` for a different reason on the same rule. Correcting myself: I first reported this alert as not real, having intersected the PR's files against the default-branch alert list, which does not contain PR-ref alerts. Querying ?ref=refs/pull/3021/merge returns it straight away. * test(engine): probe ffmpeg and Chrome instead of assuming them Two failures on #3021's Test job, both about the environment rather than the code under test. **Bare `ffmpeg` is not on PATH in CI.** The 16-bit fixture shelled out to `execFileSync("ffmpeg", ...)` and died with ENOENT. The job does provide ffmpeg, through `prepare-ffmpeg-bin`, which is what `getFfmpegBinary()` resolves — every other ffmpeg-dependent suite in this package already goes through it. Now this one does too, and the case is `skipIf(!HAS_FFMPEG)` so a contributor without ffmpeg skips rather than fails. **The browser guard trusted the wrong thing.** It asked `resolveHeadlessShellPath()` and treated a returned path as "a browser is here". CI's cache holds a chrome-headless-shell that resolves and then fails to spawn — a partial download is indistinguishable from a working one by `existsSync`, which is all that resolver checks. So the three browser cases ran anyway and failed on the launch. It now runs `--version` and requires exit 0, which is the same probe the ffmpeg suites use: ask the binary, do not infer from the filesystem. Checked both directions rather than just the green one. With a working browser all 11 cases run and pass; with `HYPERFRAMES_BROWSER_PATH` pointed at a binary that exits non-zero — CI's exact situation — exactly 3 skip and the other 8 still run. A guard that quietly skipped everything would have looked identical on the CI summary. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
James
|
0af07a07c5 | fix(core): enforce strict runtime safety | ||
|
Vance Ingalls
|
70ac80b08b | fix: clear fallow and windows artifact checks | ||
|
Vance Ingalls
|
960c31f668 | fix(core): make generated artifact formatting deterministic | ||
|
Vance IngallsandClaude Fable 5
|
67a4fb6b70 |
fix(core): use execFileSync for generated-file format step
CodeQL: shell command built from environment values — the oxfmt invocation interpolated a filesystem-derived absolute path into a shell string. execFileSync with array args avoids the shell entirely. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Vance Ingalls
|
85bab88afb | fix(core,producer): render SDK position edits in producer pipeline | ||
|
Miguel Ángel
|
2612b9bdfe | fix: publish Node-compatible package entrypoints (#1622) | ||
|
Miguel Ángel
|
20894ab9a3 |
fix: respect user timeouts on low-memory systems (#1221)
Closes #1219 ## Problem On 8GB RAM machines, renders time out at 5% with `Runtime.callFunctionOn timed out` during the duration probe. User-set timeout env vars (`PRODUCER_PUPPETEER_PROTOCOL_TIMEOUT_MS`) are silently ignored by the calibration path, and there are no CLI flags to control timeouts directly. ## Root causes 1. **Calibration timeout cap overrides user settings** — `createCaptureCalibrationConfig` used `Math.min(cfg.protocolTimeout, 30_000)`, meaning even if the user set 300s, calibration still capped at 30s. On slow hardware this causes unnecessary timeouts. 2. **8GB systems get no low-memory treatment** — `getLowMemoryFlags()`, `getGpuMemBudgetMb()`, `memoryAdaptiveCacheLimit()`, and `memoryAdaptiveCacheBytesMb()` all used `< 8192` as the threshold. Systems reporting exactly 8192 MB (common for 8GB machines) fell through to the "plenty of memory" path, getting no Chrome heap reduction or cache limits. 3. **No CLI flags for key timeouts** — Users had to discover the correct env var names (`PRODUCER_PUPPETEER_PROTOCOL_TIMEOUT_MS`, `PRODUCER_PLAYER_READY_TIMEOUT_MS`) by reading source. The non-existent `PUPPETEER_PROTOCOL_TIMEOUT` and `--browser-timeout` were common guesses that did nothing. ## Changes - `captureCost.ts`: `Math.min` → `Math.max` so the 30s calibration default is a floor, not a ceiling. User-set higher timeouts are now respected. - `browserManager.ts`: `>= 8192` → `> 8192` in `getLowMemoryFlags()` and `<= 8192` in `getGpuMemBudgetMb()` so 8GB systems get reduced Chrome heap and GPU memory budget. - `config.ts`: `< 8192` → `<= 8192` in `memoryAdaptiveCacheLimit()` and `memoryAdaptiveCacheBytesMb()` so 8GB systems get reduced frame cache limits. - `render.ts`: Added `--protocol-timeout <ms>` and `--player-ready-timeout <ms>` CLI flags, wired through `resolveConfig` overrides. - Updated calibration tests to match the new floor-not-ceiling behavior. - Added fallow suppressions for pre-existing unused exports in `captureCost.ts`. ## Test plan - [x] Engine config tests pass (`vitest run src/config.test.ts`) - [x] Browser manager tests pass (`vitest run src/services/browserManager.test.ts`) - [x] Calibration safeguard tests pass (4/4 in `renderOrchestrator.test.ts`) - [x] TypeScript compiles cleanly for engine and cli packages - [ ] CI pipeline |
||
|
Miguel Ángel
|
fb2e21090f |
feat(studio): GSAP tween editing in Design panel (#1102)
* feat(studio): GSAP tween editing in Design panel
Add a GSAP animation editor to the studio Design panel: select an element,
view and edit its tweens (properties, easing, timing), add/delete animations,
and drag custom bezier speed curves — all persisted back to the composition
HTML. Gated behind VITE_STUDIO_ENABLE_GSAP_PANEL.
Parsing of existing GSAP source now uses a recast + Babel AST parser instead of
regex, giving scope resolution, stable tween IDs, and round-trip preservation of
extras and unresolved raw values.
recast compiles to CommonJS that calls require("fs"), which breaks browser and
Vite SSR bundles. To contain it, @hyperframes/core is split into an isomorphic
layer and a Node-only AST layer:
- gsapSerialize.ts holds the recast-free helpers (serialization, keyframe
conversion, validation, shared types). htmlParser.ts is now fully isomorphic.
- parseGsapScript and the script-mutation helpers live in gsapParser.ts,
reachable only via the @hyperframes/core/gsap-parser subpath, loaded
server-side by the studio-api mutation routes and the linter via dynamic
import (recast stays external under SSR).
- The barrel and the gsap-constants subpath are recast-free, so studio browser
bundles never trace recast.
Adds AST parser unit + stress coverage and e2e helpers for the panel.
* fix(lint): await async lintHyperframeHtml in all callers
lintHyperframeHtml became async (gsap rules use dynamic import)
but lintProject and check-hyperframe-static weren't awaiting it,
causing typecheck failures and runtime crashes in CI.
Also wire LintRule type in gsap rules to fix fallow unused-type
finding, and suppress render.ts exported-for-tests symbols.
|
||
|
Miguel Ángel
|
072814e65c |
fix(core,cli,ci): harden runtime resolution + inline constant + smoke test (#458)
Guard buildHyperframesRuntimeScript() against missing entry.ts so it returns null instead of crashing with esbuild stderr output. Add getHyperframeRuntimeScript() that returns the pre-built IIFE as a baked-in string constant — no esbuild, no file I/O, no import.meta.url. Consolidate CLI runtime source resolution into a single module with a clear priority chain: esbuild from source (dev) → inlined constant (production) → pre-built artifact file (fallback). Add CI smoke test that npm-packs the CLI, installs globally, runs hyperframes preview, and asserts no stderr errors + runtime endpoint returns JS. Bump version to 0.4.16. |
||
|
|
ef6225da1d |
feat(core): add fitTextFontSize utility for pixel-accurate text measurement (#152)
Add @chenglou/pretext dependency and fitTextFontSize() utility that uses canvas measureText to compute the largest font size that fits text within a given width. Replaces character-count heuristics with actual font-aware measurement. - New fitTextFontSize() in @hyperframes/core/text, exposed on window.__hyperframes - Generalized for all text elements (captions, titles, etc.), not just captions - Unit tests (mocked pretext) + browser integration test (real Chromium canvas) - Updated captions skill docs with usage, exit guarantee, and self-lint patterns Co-authored-by: James <james.russo@heygen.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Miguel Ángel
|
1aca29a414 |
fix(core,cli): improve lint output - JSON flag, info/warning counts, severity display (#134)
## Summary - Respect `--json` flag on all lint exit paths so agents always get machine-readable output - Separate `infoCount` from `warningCount` in linter results (was conflated) - Display `info` vs `warning` severity distinctly in lint output |
||
|
Vance IngallsandClaude Opus 4.6
|
61c5257402 |
fix(ci): update publish workflow to use bun install (#36)
* fix(ci): update publish workflow to use bun install pnpm-lock.yaml was removed in the bun migration but publish.yml still referenced it. Use bun for install/build, keep pnpm for publish (publishConfig overrides + --provenance). * docs: update stale pnpm references to bun across docs and scripts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Vance Ingalls
|
20be2ea1c2 |
style: apply oxfmt baseline formatting across all source files (#25)
## Summary - Run `oxfmt .` across the entire codebase to establish formatted baseline - 299 files changed — mechanical formatting only, no logic changes - Double quotes, semicolons, 2-space indent, trailing commas, 100 print width Part 3/4 of [VA-851](https://linear.app/heygen/issue/VA-851/pre-migration-configure-eslint-prettier-and-conventional-commits) ## Test plan - [x] `pnpm format:check` — all 426 files pass - [x] `pnpm -r typecheck` — all packages pass - [x] `pnpm build` — all packages build - [x] All 348 tests pass |
||
|
Vance IngallsandClaude Opus 4.6
|
9f8e5ba5a1 |
initial code (#2)
* feat: initial code port from hyperframes-internal Port all OSS-ready packages from the internal monorepo: - @hyperframes/core — shared types, HTML generation, GSAP utilities, runtime - @hyperframes/cli — CLI for creating, previewing, and rendering compositions - @hyperframes/engine — framework-agnostic rendering engine (BeginFrame + FFmpeg) - @hyperframes/producer — video rendering pipeline (Puppeteer + FFmpeg) - @hyperframes/ui-player — browser-based video player component - @hyperframes/studio — composition editor (React frontend + Hono backend) Includes regression test suite with Docker-based test harness. All HeyGen-internal references, deployment infrastructure, and proprietary assets have been removed. Package names migrated from @app/* to @hyperframes/*. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: scrub internal codenames and stale references from OSS port - Replace static.heygen.ai runtime URLs in test fixtures - Remove internal CDN publish script (publish-hyperframe-runtime.ts) - Replace sandbox-studio, sandbox-interceptor, __magicEditRuntime with neutral names (studio, hyperframe-runtime, __hyperframeRuntime) - Fix stale Vault API / localhost references in docs - Remove broken deprecated_studio link Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove remaining internal codenames and stale references - Delete stale producer README.md and PIPELINE.md (referenced nonexistent files) - Replace "Cerberus" codename with "HyperFrames" in test design reviews - Replace magic-edit postMessage identifiers with hf-preview/hf-parent - Rename debug-magic-edit-timeline.ts to debug-timeline.ts - Replace "Motion Cut" with "HyperFrames" in Timeline comments - Fix studio/CLI references to nonexistent archive package (use local data/projects/ dir, stub render proxy) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |