`muxVideoWithAudio` passed `-avoid_negative_ts make_zero` unless the caller
set `preserveAudioPrimingEditList`. In practice the dominant path is an AAC
sidecar copied into mp4, where that flag is actively harmful: ffmpeg's
default is `auto`, which the mp4/mov muxers (AVFMT_TS_NEGATIVE) already
resolve to `disabled`. Forcing `make_zero` overrides the correct default,
discards the priming edit list the sidecar encode created, shifts the video
start_time forward by one AAC frame and writes an empty video edit at t=0 —
which edit-list-honoring players (QuickTime/Safari) render as a black first
frame.
Verified with ffprobe on a copy mux of a 30fps h264 mp4 and an AAC sidecar:
with `make_zero` video start_time 0.066000, elst: [media time -1,
dur 5940] + [media time 6000, dur 180000]
audio start_time 0.042993, elst: [media time -1, ...]
without (this fix) video start_time 0.000000, elst: [media time 6000,
dur 180000]
audio start_time 0.000000, elst: [media time 1024, ...]
The empty leading edit and the offset both disappear, and the audio keeps
its 1024-sample priming edit.
The flag is now never passed for a mux, in any mode. `preserveAudioPrimingEditList`
is part of the exported engine API, so it stays on `MuxVideoWithAudioOptions`
as `@deprecated` and no-op rather than being removed; the two internal callers
that set it (`assembleStage`, distributed `assemble`) drop it.
`buildEncoderArgs` and `streamingEncoder` still pass the flag for video-only
output and are deliberately left alone — those chunks are consumed as
intermediates, not as a delivered mp4/mov.
Fixes#3487
Co-authored-by: Alexandru Mincu <alex@mountsoftware.ro>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(cli): keep phrase-level CJK and Thai transcripts as separate cues
wordsToCues inferred whether entries were already grouped into phrases
by testing for internal whitespace. Chinese, Japanese, Thai and the
other scripts written without inter-word spaces never satisfy that
test, so their phrase-level transcripts were treated as word-level and
re-grouped into a single cue covering the whole clip.
A three-phrase Chinese transcript produced one cue; the same transcript
in English produced three. The failure was silent: the export
succeeded, and the user found out by watching the captions.
For entries with no whitespace at all, fall back to entry length when
they are in a spaceless script. Whisper emits word-level tokens for
those scripts one or two characters at a time, while a phrase-level cue
runs to several times that. The median is used so one long token cannot
declare word-level input pre-grouped, and a couple of short cues cannot
declare a real transcript word-level.
--preserve-cues still forces the same thing, and behaviour for
space-separated scripts is unchanged.
Fixes#3353
* test(cli): pin the spaceless phrase length threshold
Parent src-only scans skipped multi-format <video>/<audio> markup, so those
elements were never extracted, downloaded, or mixed and rendered blank/silent.
Lint now accepts a child <source src> as a resolvable media src.
lookupCacheEntry reported a hit purely on the presence of the
.hf-complete sentinel. The sentinel records that extraction finished,
not that the frames survived, so any per-file cleanup that empties the
directory leaves an entry that rehydrates with zero frames.
rehydrateCacheEntry then returns totalFrames: 0, the clip reaches the
coverage gate with nothing, and the render aborts with a message about
capture coverage. Because the poison is on disk rather than in the
composition, every later render of the project fails the same way with
nothing the user can change to fix it.
An entry now counts as a hit only when it carries the sentinel AND
still holds at least one frame file, so an emptied entry re-extracts.
The check is format-agnostic: a hit must be usable whatever extension
the frames carry.
Addresses the cache half of #3372.
compileTimingAttrs/injectDurations used parseFloat, so data-start="intro"
wrote a NaN data-end and extract preferred that over duration; parseNumeric
now skips the id-ref (parseVideoElements already resolves it).
collectRenderMedia's resolveHostWindow likewise read host data-start with
parseFloat, so chained sub-composition slots (data-start="hook") stacked at
0-2s and every scene after the first rendered black. It now resolves host
starts through the shared resolveReferencedStart, matching the media parsers.
Fixes#3361.
* fix(producer): fall back to screenshot capture on drawElement canvas-not-initialized
The fast-capture drawElement path only special-cased the "No cached
paint record" error to trigger a per-frame screenshot fallback; every
other error (including "drawElement canvas not initialized", seen at
frame 0 on some macOS/Chrome combinations) was rethrown, hard-failing
the whole render even though the docs promise automatic fallback on
incompatible compositions.
Extend the existing fallback branch (in both captureFrameCore and
captureFrameToBufferPipelined) to also catch canvas-not-initialized
errors via a shared isRecoverableDrawElementError predicate, with a
diagnostic message identifying which case triggered the fallback.
Closes#3423
Co-Authored-By: Miga <noreply@anthropic.com>
* fix(producer): address review — tighten error matching, audit batch path, add fallback-ratio guard
* fix(engine): add prepareFrameForCapture to batch screenshot fallback loop
* fix(engine): split canvas-not-initialized from composition-root-missing errors
drawElementService threw the same HF_DE_CANVAS_NOT_INITIALIZED error for
both !canvas and !root. Missing composition root (navigated/broken page)
was classified recoverable and fell back to pageScreenshotCapture, which
captured blank or wrong content silently.
Now:
- !root → HF_DE_COMPOSITION_ROOT_MISSING (not recoverable, hard fail)
- !canvas → HF_DE_CANVAS_NOT_INITIALIZED (recoverable, screenshot fallback)
Split applied at all 3 emit sites (serial, pipelined, batch).
Co-Authored-By: miga-heygen <miguel.sierra_miga@heygen.com>
---------
Co-authored-by: Miguel Ángel <miguel.sierra@heygen.com>
Co-authored-by: Miga <noreply@anthropic.com>
Root cause: the per-worker capture calls in captureFrameRange
(parallelCoordinator.ts) take no abort signal of their own, and only
checked `signal.aborted` BEFORE starting each frame — a no-op once a
worker is already awaiting an in-flight call. On WSL2, the native
drawElement/BeginFrame capture call can hang indefinitely at frame 0
with no error. The DE parallel-router's existing stall watchdog
(captureStreamingStage.ts) correctly fires `stallController.abort()`
after HF_DE_STALL_MS, but that abort had no way to reach a
worker already wedged inside a hung capture call — so
executeParallelCapture's Promise.all waited forever, the render hung
indefinitely, and the CLI's circuit breaker (which only runs after
executeRenderJob settles) never got a chance to trip.
Fix: race each per-frame capture call against the signal actually
firing (raceAgainstAbort), the same "can't cancel, only race" pattern
already used by the sequential capture path. Once the watchdog's abort
is observed, the wedged worker rejects, executeParallelCapture settles,
and the existing pinned-fallback retry / "reverted" outcome / circuit
breaker machinery (already correct) runs end to end.
Also widen the CLI breaker's trip condition from the literal string
"reverted" to "not a clean routed success", so any future non-success
outcome the observability layer records also latches the breaker
instead of silently falling through.
Closes#3441
Co-authored-by: Miga <noreply@anthropic.com>
Classify each <audio> element before Web Audio capture: same-origin,
CORS-opted-in, or a non-http(s) scheme stays on the primary
createMediaElementSource() path; cross-origin media without a
crossorigin opt-in withholds that call (the Web Audio spec makes such a
node output silence without throwing) and falls back to fetch +
decodeAudioData, preserving the FX graph whenever the server allows
CORS. Recheck the route at the transport's irreversible capture
boundary, and account for currentSrc, src, and <source> candidates the
same way the HTML resource-selection algorithm does.
Emit a stable preview diagnostic (`runtime_web_audio_bypass`) at media
discovery time, not only from playback scheduling, so `hyperframes
check` — which seeks but never plays — can surface it as a
`web_audio_bypass` finding. Diagnostics are suppressed during export
rendering, where the producer mixes audio offline and already applies
the FX chain. The existing non-unit-rate fail-closed rule stays scoped
to fx-chain/automation so this fix does not newly mute grouped or
above-unity tracks.
Takes over #3459 with the data-native-audio escape hatch removed per
review feedback: the automatic cross-origin detection already covers
the cases that mattered, so the extra per-element opt-in attribute,
its route-classifier branch, and its diagnostic path are dropped in
favor of a single automatic behavior.
Fixes#3458
Original-Author: desenmeng
Co-Authored-By: desenmeng <desenmeng@users.noreply.github.com>
Co-Authored-By: Miga <noreply@anthropic.com>
`registry_item_added` fires when a catalog block is installed and
`render_complete` fires when a video is produced, but nothing joined them, so
"did this video use the catalog?" had no answer.
`hyperframes add` now records each installed item in `hyperframes.json`
(installed files are plain composition HTML with no provenance marker, so this
manifest is the only record that a file came from the registry), and
`render_complete` reports both the items the project installed and the blocks
the rendered composition actually reaches. An item installed and then never
mounted was tried and dropped, which no add-time event can express.
The scan answering "which sub-compositions does this file mount" now has one
owner, `collectSubCompositionSrcs` in `@hyperframes/parsers`, shared with
lint's `lintMissingOrEmptySubComposition`. It holds two invariants that were
previously restated per call site and got re-derived wrongly: it is a text scan
rather than a DOM query, because `<template>` content is inert and every
sub-composition except the render entry is wrapped in one; and references
resolve root-relative at every nesting level, matching `parseSubCompositions`.
It walks tag by tag rather than running open-ended spans across the whole file,
so a malformed composition cannot stall the render plan.
Also: `registryItems` is declared in the config schema, which closes with
`additionalProperties: false`, with an ajv-backed test pinning every key the CLI
writes; counts are never truncated by the name cap, and the reported used blocks
stay a subset of the reported installed ones, with `registry_items_truncated`
marking a windowed list; and an unreadable manifest reports itself rather than
posing as a project that never used the catalog.
The runtime absorbed a series of authoring mistakes over time and `runtime/init.ts`
says so in its own comments, but the skills kept teaching the old rules. Four of
them actively cost an agent a failing run: add `crossorigin` (lint rejects it
unconditionally), never build a timeline inside `async` (lint calls that the
documented contract), never `gsap.set` later-scene clips (two fixHints instruct
exactly that), and 12 copyable media snippets with no `id`, which render silent.
Corrected in every place each claim appeared, including `hyperframes-animation`,
three workflow scripts, the scaffolded project instructions, the CLI `docs`
command, and the public docs site: `data-track-index` is a Studio display lane
the render never reads, `class="clip"` is a layout convention rather than a
visibility requirement, timed elements may nest, the visibility window is
half-open, sub-composition host dimensions are backfilled, and the root-fill rule
applies only to the layered-composite path.
Behaviour changes, each backed by a render rather than by reading code:
- `timeline_registry_missing_init` deleted. The runtime creates the registry
before any inline script; a composition without the guard line renders and
animates correctly.
- `video_nested_in_timed_element` kept, message corrected. A rendered repro shows
the nested-with-local-start case really does break, so the rule guards a real
defect, but nothing is "FROZEN": the extractor ignores the wrapper's offset
while visibility uses it, so the clip shows wrong frames and then vanishes.
- `mediaRenderIds` now stamps media whose source is a `<source>` child, closing a
duplicate-id gap the old `[src]`-only selector left open.
- Stale messages fixed on `subcomposition_root_styled_by_class` and
`deprecated_data_layer`.
`coreSkillContent.test.ts` pinned the literal sentence that made root
`data-start` look required, so it is narrowed to structure plus the regression it
genuinely catches.
Not covered, and flagged in the PR: the media global-vs-local start heuristic in
`runtime/init.ts` is the root cause behind the nested-video defect. Removing it
changes the meaning of existing compositions and needs its own deprecation.
* feat(studio): drag automation segments
* fix(studio): clear clip selection for group effects
* fix(studio): replace clip selection with audio bus
* fix(studio): make audio bus selection authoritative
* fix(core): harden audio FX and group identity
* fix(core): address audio group review feedback
* fix(core): align preview transport with grouped audio
* test(core): pin audio group gain ceiling
* fix(core): preserve solo bridge through stack
* fix(engine): harden grouped audio rendering
* docs(engine): explain grouped mix fallback invariant
* test(engine): allow grouped mixes to finish on Windows
* feat(lint): validate audio group membership and timing
* test(lint): pin audio group membership guards
* fix(studio): unify audio IDs and group state
* fix(studio): make audio-group edits transactional
* fix(studio): keep preview state synchronized
* fix(studio): align audio rows, automation lanes and headers
* fix(studio): stabilize timeline audio derivations
* refactor(studio): simplify group metadata memoization
* style(studio): keep timeline layout within size gate
* fix(studio): keep timeline preset apply off auditions
* fix(studio): harden carve and FX rack behavior
* fix(studio): repeat audio FX reveal requests
* fix(studio): reconnect property-panel audio controls
* fix(studio): unify property panel audio detection
* fix(studio): satisfy panel and deletion gates
* feat(studio,core)!: remove solo and the group meter
* docs(audio): keep removal rationale current
* refactor(core): retire studio solo bridge
* docs(audio): document grouped audio and its guardrails
* docs(audio): point handoff at replacement stack
* fix(core): harden audio FX and group identity
* fix(core): address audio group review feedback
* fix(core): align preview transport with grouped audio
* test(core): pin audio group gain ceiling
* fix(core): preserve solo bridge through stack
* fix(engine): harden grouped audio rendering
* docs(engine): explain grouped mix fallback invariant
* test(engine): allow grouped mixes to finish on Windows
* feat(lint): validate audio group membership and timing
* test(lint): pin audio group membership guards
* fix(studio): unify audio IDs and group state
* fix(studio): make audio-group edits transactional
* fix(studio): keep preview state synchronized
* fix(core): harden audio FX and group identity
* fix(core): address audio group review feedback
* fix(core): align preview transport with grouped audio
* test(core): pin audio group gain ceiling
* fix(core): preserve solo bridge through stack
* fix(engine): harden grouped audio rendering
* docs(engine): explain grouped mix fallback invariant
* test(engine): allow grouped mixes to finish on Windows
* feat(lint): validate audio group membership and timing
* test(lint): pin audio group membership guards
* fix(studio): unify audio IDs and group state
* fix(studio): make audio-group edits transactional
* fix(core): harden audio FX and group identity
* fix(core): address audio group review feedback
* fix(core): align preview transport with grouped audio
* test(core): pin audio group gain ceiling
* fix(core): preserve solo bridge through stack
* fix(engine): harden grouped audio rendering
* docs(engine): explain grouped mix fallback invariant
* test(engine): allow grouped mixes to finish on Windows
* feat(lint): validate audio group membership and timing
* test(lint): pin audio group membership guards
* fix(studio): unify audio IDs and group state
* fix(core): harden audio FX and group identity
* fix(core): address audio group review feedback
* fix(core): align preview transport with grouped audio
* test(core): pin audio group gain ceiling
* fix(core): preserve solo bridge through stack
* fix(engine): harden grouped audio rendering
* docs(engine): explain grouped mix fallback invariant
* test(engine): allow grouped mixes to finish on Windows
* feat(lint): validate audio group membership and timing
* test(lint): pin audio group membership guards
* fix(core): harden audio FX and group identity
* fix(core): address audio group review feedback
* fix(core): align preview transport with grouped audio
* test(core): pin audio group gain ceiling
* fix(core): preserve solo bridge through stack
* fix(engine): harden grouped audio rendering
* docs(engine): explain grouped mix fallback invariant
* test(engine): allow grouped mixes to finish on Windows
* fix(core): harden audio FX and group identity
* fix(core): address audio group review feedback
* fix(core): align preview transport with grouped audio
* test(core): pin audio group gain ceiling
* fix(core): preserve solo bridge through stack
Reported as "the grouping button did nothing — I clicked it and nothing
happened". The dialog WAS opening. It positioned itself at
`anchorRect.bottom + 4` with no flip and no clamp, and this button lives in a
track header at the bottom of the studio window, so it opened past the viewport
edge. It was the last floating surface in the timeline with no viewport handling
at all.
It now goes through `resolveFloatingPanelPosition`, the helper the other body
portals already position with (`RenderQueue`, `propertyPanelColor`), so it flips
above the anchor when there is no room below and clamps so neither edge leaves
the viewport. `GROUP_DIALOG_SIZE` is a declared estimate in the same style as
`FORMAT_PANEL_SIZE` and `COLOR_PICKER_SIZE`: `w-56` is exact, only the flip
decision reads the height, and the clamp keeps the dialog on screen either way.
Two tests, at a realistic bottom-of-window anchor and hard against the right
edge. Both verified to fail against the raw positioning.
Worth noting why this shipped: the existing `group-pointer` test passes with or
without the fix. happy-dom reports an all-zero rect for an unlaid-out button, so
the dialog landed at top:4 — on screen, and nothing like the real app. A geometry
test that never sets a geometry proves nothing.
Deliberately NOT included: a toast for the grouping write's silent
`elements.length < 2` bail. That path is real in code but I could not reach it
from the UI — the button only renders on a track with 2+ ungrouped clips, and
sub-composition audio arrives as separate single-clip rows, so the offer never
appears there. Adding a message for an unreachable branch, plus the file split it
would force to stay under the 600-line studio cap, is not justified by evidence.
* test(sdk): render-faithfulness test for serialize() bake contract (WS-F)
Adds session.render-faithful.test.ts with 8 assertions covering the full
op batch (setStyle + setText + setTiming + addGsapTween + moveElement).
Confirms serialize() emits fully override-baked, render-ready HTML — this
is the SDK-side guarantee that the backend render input needs no separate
override-set field. Also asserts GSAP <script> edits and data-composition-
variables survive serialization unchanged.
Decision recorded: no SDK bake helper. session.serialize() IS the bake.
Content-address → zip → S3 upload → pointer-swap is host/backend (WS-P/WS-R).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(sdk): close the two false-positive slots in the bake-contract assertions
`toContain('data-end="5"')` in the setTiming case and `toContain('data-y="50"')`
in the full-batch case both match a DIFFERENT element in the fixture —
hf-title already ends at 5, hf-box already sits at y=50 — so either assertion
would still pass if its write regressed to a no-op. Paired each with the
disappearance of the target element's own pre-mutation value, which is the
pattern the setText case already uses.
* test(sdk): assert the canonical data-duration timing shape, not legacy data-end
The false-positive slot was hiding a real behaviour change. `setTiming` now
routes through the parsers' `writeClipTiming`, which canonicalizes timing onto
`data-start` + `data-duration` and REMOVES the legacy `data-end`. The
full-batch case asserted `data-end="4.5"` and failed once rebased onto main;
the standalone case asserted `data-end="5"` and passed only because hf-title
carries that exact value in the fixture — the very collision this pass set out
to close.
Both now assert the start/duration pair and the disappearance of the target's
own legacy end.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The render POST's catch took no binding, so the exception was discarded and
every transport failure produced one sentence: "Could not reach render server.
Use `hyperframes render` from the CLI instead."
A dead server, a DNS failure, an aborted request and a server that died
mid-render are all indistinguishable under that string — and it is not only a
UI message, it is what travels into the feedback report. Three separate field
reports carried it verbatim, one of them describing a render that fails every
single time. A guaranteed reproduction that tells us nothing is worse than an
intermittent one that does.
Bind the error and append it. The CLI guidance stays, since it is still the
right next step for the user; it just no longer stands alone.
Regression test asserts both halves: the cause appears, and the guidance
survives. It fails on the unfixed code with `expected 'Could not reach render
server. Use `h…' to contain 'Failed to fetch'`.
