ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-11-03 20:37:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
6,261 Commits 1,549 Branches 619 Tags
eb6b2c9de9ff3bc9fbfb94ce0bdfd62678128776
Commit Graph
23 Commits
Author SHA1 Message Date
Miguel Ángel 54d0af57eb feat(engine): render sets a keyframe interval and a peak bitrate cap (#6444) 
* feat(engine): encoders cap peak bitrate and place keyframes on a set interval

* style(engine): shorten the rate-cap comment

* style(engine): drop the doc lines the comment ratchet flagged

* feat(producer): mp4 renders honour a keyframe interval and a peak bitrate cap on every encoder path

* style(producer): shorten the delivery encoder file headers

* style(producer): drop a comment line from the segment manifest

* style(producer): shorten the integration test header

* test: delivery encoder tests follow the ffprobe argv contract and run in the integration lane

* feat: the render request, server and Lambda plan carry the keyframe interval and peak bitrate cap

* fix(studio): the Studio render route passes the keyframe interval and bitrate cap to the encoder

* chore(studio-server): drop a comment the helper's name already says

* feat(cli): render and lambda render take --keyframe-interval, --max-bitrate and --buffer-size
 2026-10-27 00:29:06 +00:00
Muhtasim Munif Fahim 85dcc193d5 fix(cli): list the examples init --example accepts in docs examples (#5446) 
The examples topic listed title-card and video-edit, which do not exist,
and none of the eight examples in registry/examples. List blank plus the
registry examples with their registry-item descriptions, and add a test
that fails when the topic and registry/examples disagree.

Fixes #5393
 2026-10-26 14:22:09 +00:00
Jerrai 7a34d6fbe4 docs: align render, fps and transcribe docs with the CLI (#6166) 
* docs: align render, fps and transcribe docs with the CLI

- docs/guides/rendering.mdx: default render quality is `looks` (CRF 16),
  not `standard`.
- docs/packages/cli.mdx, render --help (render.ts): default --output is the
  timestamped `renders/<project>_<YYYY-MM-DD>_<HH-MM-SS>.<ext>`.
- skills/hyperframes-cli/references/preview-render.md: local --fps accepts
  1-240 or a rational and defaults to the root data-fps, else 30; only
  lambda and cloudrun are limited to 24, 30, 60.
- skills/hyperframes-cli/references/lint-validate-inspect.md: snapshot
  --range is documented as START,END like every other reference.
- skills/media-use/references/audio.md, skills/media-use/audio/references/
  transcribe.md, skills/hyperframes/references/captions.md: name the entry
  point for each transcription default (transcribe.mjs prefers Parakeet;
  `hyperframes transcribe` uses Parakeet when installed, else the small.en
  whisper model).
- transcribe --model help text and docs/packages/cli.mdx: list `small` and
  `medium`, which the CLI already accepts.
- skills-manifest.json: regenerated hashes for the three skills touched.

— Jerrai

* docs: lambda and cloud run also accept 25 fps

Distributed renders accept 24, 25, 30 and 60 fps (DISTRIBUTED_FPS in
packages/core/src/distributedFps.ts).

- skills/hyperframes-cli/references/preview-render.md: the --fps note
  lists 24, 25, 30, 60 for lambda and cloudrun.
- skills/hyperframes-cli/references/lambda.md, cloudrun.md: --fps lists
  include 25.
- packages/cli/src/docs/rendering.md (`hyperframes docs rendering`): --fps
  said 24, 30 or 60 and --quality defaulted to standard; local render takes
  1-240 or a rational, default root data-fps else 30, and the default
  quality is looks.
- skills-manifest.json: regenerated hyperframes-cli hash.

— Jerrai

* chore(skills): regenerate skills manifest after formatting

The format hook rewrote skill markdown after the manifest hook had hashed
it, leaving the hyperframes-cli hash stale.

— Jerrai
 2026-10-24 00:29:56 +00:00
WaterrrForeverandClaude Opus 5.5 688229314f feat(skills)!: publish seven knowledge skills aligned with Studio and retire the workflow skills (#5651) 
* refactor(skills): fold six domain skills into hyperframes and media-use guides

hyperframes-core, -animation, -keyframes, -creative and -studio move to
skills/hyperframes/{core,animation,keyframes,creative,studio}/, and
hyperframes-audio moves to skills/media-use/mix/. Each moved SKILL.md becomes
a frontmatter-free GUIDE.md so installers that scan recursively for SKILL.md
no longer see them as skills; the published list goes from 21 to 15.

Relative links, code paths and skill-name mentions are rewritten to the new
locations, the catalog surfaces, marketplace core-skills list, fallback core
set, module-copy targets and skills-manifest.json follow, and
`hyperframes skills update` now removes the retired skills (RETIRED_SKILLS)
when they are still installed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat(skills): slim the core entries and give media-use a HeyGen and a free path

The four entries every run opens are rewritten to what an agent cannot infer:
hyperframes 21.8 KB to 7.7 KB, hyperframes-cli 24.8 KB to 6.9 KB,
hyperframes-registry 9.9 KB to 4.3 KB, media-use 9.7 KB to 4.9 KB. Detail moved
whole into references (routing close calls and creator edits, CLI pin upgrades,
catalog search tiers, CLI conventions and feedback reports).

- hyperframes: state table, routing, the render contract, a terminal work loop
  (check, snapshot, preview before render), the desktop-app hand-off, and a
  table of the folded guides.
- media-use: one table of HeyGen versus free local tools per need; HeyGen
  sign-in, credits and the CLI's video-agent, avatar, clone, lipsync, translate
  and clipping commands; every free path works with no account.
- scripts/check-core-skills.mjs holds entries and descriptions to budget and
  checks every taught `npx hyperframes` command and path; runs in lint.
- Core entries are exempt from the formatter: its table padding cost ~1.5 KB
  per entry on every run.
- Content tests follow the text to its new home.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(skills): plugin runs skip skills update; media-use explains HeyGen sign-in and TTS setup

The smoke eval showed a presenter request never reaching media-use and the
free voiceover failing on a missing kokoro-onnx; plugin runs also ran the
skills update the plugin rules forbid.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* chore(skills): drop a committed Python bytecode file and ignore bytecode

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat(skills): align the entry with the Studio skills and keep the workflows' measured facts

The entry drops routing and the intent interview and carries the contract, the
editable-project rules and the work loop. Studio's format, runtimes, motion,
look and captions references come over for terminal use, frame presets move to
assets/, media-use gains Studio's mixing and local-command references, and
film.md and overlays.md keep what the workflow skills measured.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat(skills)!: retire the workflow skills and folded guides; publish seven knowledge skills

The skill pack now publishes exactly seven skills - hyperframes,
hyperframes-cli, hyperframes-registry, media-use, figma, slideshow and
remotion-to-hyperframes - carrying knowledge, not process. The eight
workflow skills, the folded domain guides and the routing/interview
references are deleted; their measured facts already live in the
hyperframes references (format, runtimes, motion, look, captions, film,
overlays) and media-use's mixing reference.

- bake-basemap.mjs and locate.mjs move to skills/hyperframes/scripts/
  (bake-basemap gains --help); extract-audio-data.py moves to
  media-use/audio/scripts/ for the caption motion reference.
- skills update: the eight workflows join RETIRED_SKILLS so existing
  installs prune them (by lock attribution only, since their names are
  generic); naming a retired skill reports where its knowledge went and
  succeeds instead of failing as unknown.
- Capture prompts point at the hyperframes skill's references/film.md;
  the live-map warning names the new bake-basemap path.
- Content tests that pinned knowledge are retargeted to the kept
  references; tests of deleted workflows, scripts and process go.
- Catalog surfaces, templates and docs describe the seven skills.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(skills): resolve the local.md captions link and allow capture's output files in citations

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(skills): regenerate the skills manifest from tracked files only

The committed media-use hash included an ignored __pycache__ bytecode file present
in the local checkout, so CI's clean checkout hashed differently.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(docs): restore generated catalog pages and docs navigation to main

Catalog PRs commit item sources only; CI regenerates the pages from the edited
registry sources. docs/docs.json is catalog navigation, so its Workflows group
rename is left for a non-catalog PR.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* build(cli): raise the CLI size budget for the skill references it bundles

The CLI bundles skills/hyperframes, which now carries the folded domain guides and
frame presets (155 KB to 1.08 MB in git). Budgets are the CI-measured size plus 5%:
packed 7526634, unpacked 22593213.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* test: keep repo-sweep guards meaningful after the workflow skills left

The last shell ffprobe caller and most root index.html examples lived in retired
skills. The shell detector is now pinned on a literal command line, and the
structure-rule sweep needs more than 5 roots (11 remain) rather than 30.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat(skills): slim figma, slideshow and remotion to the Studio shape; restore the storyboard board

slideshow 33.5 KB to 4.9, figma 18.2 to 6.2, remotion-to-hyperframes 11.0 to
4.9, each keeping the facts only it holds (the figma motion helpers, verify gate,
storyboard decoding and usage beacons; present and hand-off; the Remotion lint,
SSIM check and API map). The board page format from the storyboard recipe comes
back as hyperframes/references/storyboard.md: the desktop app's Storyboard tab
reads boards/storyboard.html by its frame, title and time classes. The entry and
the init template now say to snapshot the moments an edit changed before
replying, which the terminal eval found skipped in 22 of 38 runs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(skills): manifest without local bytecode; Studio's edit prompt names kept files

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(skills): render when the person asks for the file or for no stops, not only after approval

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(skills): keep the voiceover carve in media-use; presets stage their own fonts

Review follow-ups on #5651:
- carve.mjs and its tests move to skills/media-use/audio/scripts; the audio
  docs, mixing.md and two Studio comments point there.
- code-editorial's FRAME.md says to copy its bundled WOFF2s and add the
  @font-face blocks, instead of promising the retired build-frame.mjs.
- media-use states the host rule before the HeyGen sign-in steps.
- storyboard-format marks transition_in as a note the agent builds itself.
- Two stale citations (creator-editing-recipes.md, sub-compositions.md).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* chore(skills): regenerate the manifest without local bytecode

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(skills): the CLI render step follows step 4's rule, not approval alone

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(ci): run the core-skills check from lint again after the main merge

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat(skills): films that tell a story start from three storylines; maps use real imagery

Opus 5.5 arm D read motion.md and applied it, but its launch film was a
feature list (pain, three feature cards, proof, logo): film.md left story
to the model. And the map case never opened film.md, so it drew Natural
Earth outlines instead of baking satellite tiles.

- references/story.md: the generic half of Studio's moment-film story and
  craft (one thing, three storylines with world and device, the hook,
  beats, worlds, copy, brand, tells of a generated film). The person picks
  a storyline, or the agent takes its recommendation when asked for the
  file or for no stops.
- The entry's work loop names the storyline step for films over ~15 s; the
  references table routes stories to story.md and maps to film.md.
- film.md points at story.md and gives maps and real places a section.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(skills): story.md follows the entry: no voiceover unless asked

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(skills): bake-basemap starts from a globe and stops waiting out its idle timeout

The Opus 5.5 map run spent 13 of its 38 minutes on parallel bakes that
stalled and 15 more recording 300 frames, after rewriting the script because
it could only start on a flat map.

- PROJECTION=globe with a seeded starfield and atmosphere, and FROM/PAN to
  turn the globe to CENTER before the dive.
- Each frame polls for loaded tiles instead of waiting for an "idle" event
  that had often already fired: 30 frames went from 198-219 s to 10-13 s
  locally with byte-identical frames.
- Progress prints seconds a frame and minutes left; film.md shows the globe
  call and says to run one bake at a time.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(skills): render in the foreground; wait on a detached render's log, never pgrep -f

Both Opus 5.5 explainer runs started a detached render, then waited with
`until ! pgrep -f "hyperframes render"`, which matches its own loop: each
render took 88-117 s and the wait sat for the tool's full 10 minutes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat(skills): route each kind of film; storylines carry look and sound; add review and sound

The entry routes by request (launch, explainer, music, data, vertical, 3D,
character) to a self-contained file under references/types/, and a film
over ~15 s now gets music and sounds on its visible actions unless the
person asked for silence. Each storyline names its look and its sound, so
picking one picks both; STORYBOARD.md frames carry `reads` and `sound`.

review.md is the whole-film pass: the check warnings, the review sheets,
seven questions to look for, P0/P1/P2 notes and REVIEW.md. motion.md gains
eases by role, six title entrances, a reading-time rule and nine measured
transitions; look.md a design plan before the first scene, with presets
kept for decks and named picks. story.md adds the second default that
generated design falls into, with what to do instead.

The blank template and format.md no longer teach the fade-and-slide-up
entrance on a dark canvas.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat(cli): check warns on unread lines, dead stretches, small type, empty frame 0 and silence

A coding agent cannot watch its film, so check now measures what a viewer
notices over time. A new `craft` section holds warnings only (exit 0
unless --strict), sampled on the dense 8 fps grid the overlap pass
already seeks, skipped for --range checks:

- line_unread: a reading-size line (>= 30 px tall at 1080) on screen
  under 0.4 s for 1-2 words, else under 3/4 of 0.5 s + 0.3 s a word;
  the film's last line needs 2 s. Short lines are re-measured between
  samples to ~0.01 s before judging. Captions-track and
  data-layout-ignore text are exempt.
- dead_stretch: more than 3 s where the keepsMoving liveness signature
  does not change, outside the final 3 s and any data-hold element;
  quiet when sweep_static fires or a whole-composition keepsMoving
  assertion owns the limit.
- text_below_floor: text under 2.6% of the short side (28 px on 1080)
  held 0.5 s, one finding per file listing every element.
- first_frame_empty: frame 0 paints only the background.
- film_silent: over 15 s with no unmuted audio or audible video.

Each message names the fix and the skill reference to read. keepsMoving
now also leaves the film's last 3 s (a still end card) out of its
static runs, so the two agree.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat(cli): snapshot --review captures a beat sheet, seam strips and a phone sheet

snapshot --review picks the frames that show a film over time instead of
evenly spaced ones, in the same single browser session:

- review-beats.jpg: one settled frame per scene at its start + 0.3 s,
  plus one per timed data-beat="name" element, labelled time and id.
  Scenes are the composition's top-level timed clips and sub-composition
  hosts, read like `timeline` reads rows (captions and audio excluded)
  and timed by the runtime's start resolver over the live page.
- review-seam-NN.jpg: frames at -0.2, -0.1, 0, +0.1, +0.2 s around
  every cut (each scene start, and any scene end no scene starts at).
- review-phone.jpg: the beat frames at 360 px wide.

--composition scopes the review to one sub-composition; with fewer than
two scenes the beats spread evenly and a note says so. snapshot also
gains --json: one envelope with every frame's path and time and, under
--review, the scenes, beats and seams behind the sheets; progress moves
to stderr.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(cli): craft warnings stay out of --strict; a hold gets a real next action, not a caret

Craft findings (unread lines, dead stretches, small type, an empty first
frame, silence) are judgment calls a film may break on purpose, so
--strict keeps gating defects only. The check reference no longer
suggests a blinking caret to keep a still frame moving.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(cli): review spreads beats when every scene starts together; map credits skip the type floor

A map film whose basemap and overlay both start at 0 s got one beat
frame; the even spread now keys on distinct scene starts. film.md tells
agents to mark the imagery credit data-layout-ignore and back it, since
the new type floor flagged it on the first real run.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(cli): review frames each scene at its middle too, not only 0.3 s in

On the first real explainer, 0.3 s into a scene showed a bare colour
field before its title entered. Each scene longer than about a second
and a half now adds a frame at its middle, so the sheet shows what the
scene builds.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* feat(skills): a film is built from a numeric SPEC.md, with defaults banned by name and a look test

Studied in full: flxrnc's Motion Lab (vague vs directed prompt), 0xMovez's
course, EianLu's riso skill, agentchengfeng's style samples and the two
Opus 5.5 guides. Their common point: adjectives are where the default look
lives, so the agent now writes SPEC.md before any scene (GOAL, OUTPUT,
TIME, LOOK, MOTION, SHOTS, BANNED, QA), with a number for every adjective.

spec.md bans the defaults by name, unless the brand or reference uses
them (a dark void with small things floating, glow and particles, the
cream-serif-mono-label look, fade-and-slide-up, corner brackets), and
grows that list after the first render. Each storyline gets a 3 s test of
its hero shot before the pick; a look is written as the rules of one
medium. Composition numbers: the hero fills 40% of the frame, one hero
move per shot, the shot's information in within 2.5 s then a held poster.

motion.md adds springs by damping ratio, shorter exits, overlap, stagger
and the beat grid; review.md reviews as a harsh director and grows BANNED;
vertical.md takes the riso skill's measured safe band; explainer.md the
shapes of information.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(skills): story before look; every object reads as itself; posters only for info cards

The first spec-driven C02 swapped the dark default for a flat Swiss
poster, and the story went with it: two black bars nobody read as mirrors,
slogans instead of cause and effect, a storyline picked for its picture.

The spec now serves the story: a storyline is picked for the story a
stranger would follow, the look test asks first whether a stranger
understands what they see, and SPEC.md gains an OBJECTS line giving each
thing the material and details that make it read without a label. The
2.5 s held-poster rule is limited to information cards; in a story film
every shot causes the next. Explainers start from a person or situation
the viewer knows. Review names every object as a stranger would and
treats one that needs explaining as a P0.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(skills): storylines are three films in words, each with a device and a form

The three storylines are pitched as text and the person picks from them;
nothing is built for a storyline before the pick, so the 3 s hero-shot
look test is gone. Ideas pair a device with a form, five or more first,
keeping the three that differ most, and the one stop states the rest in
an assumption line that names every paid item beside its free option.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* ci: list main's new embedded-captions hf-cli test among the retired workflow files

The merge with main removed it with the rest of the retired skill, and the
deletion guard rejects any removal it has not been told about.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(core): keep the audio preset comment within core's size budget

The comment's new citation to media-use's mixing reference made the published
package 14 bytes larger than its budget, which sits at main's exact size.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* fix(cli): craft keeps its own 8 fps grid; comments back within the checks

Review on #5651 found that the merge moved craft onto the overlap pass's
times, which main caps at 180, so a brief line in a long film fell between
samples and line_unread missed it. Craft now builds its own grid at 8 fps up to
600 samples, as before the merge, and a 60 s film with a line shown for 0.2 s
pins it.

The Comments check passes again: REVIEW.md and STORYLINES.md join the files a
skill writes into a project, the comment lines this branch added come back
under each file's share at the merge-base, and the audio preset comment drops
a word the check reads as history.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-21 19:19:31 +00:00
Vance IngallsClaude Opus 5.5miga-heygenkapelame
 		37f30b1d06 feat: keep a video's sound on the clip by default (1/8) (#4813) 
* feat(cli): one owner for init media placeholders, audio stays on the video

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(cli): init keeps a video's sound on the clip; silent files scaffold muted

Supersedes the patchVideoSrc changes in #3977 and #3942 and carries their tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: miga-heygen <miga-heygen@users.noreply.github.com>
Co-Authored-By: kapelame <kapelame@users.noreply.github.com>

* feat(registry): example templates keep the a-roll's sound on the video

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(lint): video audio rules recommend keeping sound on the video

Update video_missing_muted and video_audio_double_source message and
fixHint text to recommend the new default: keep audio on the video with
data-has-audio="true" (not muted). Tests verify the new wording is in place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(parsers): a video is either audible (data-has-audio) or muted, never both

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): follow the compiler's audio rule in the audio-track toggle

Confirm before background removal drops a video's sound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skills): hyperframes-core teaches sound on the video by default

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skills): workflows and scaffold guidance keep a video's sound on the clip

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: footage with sound keeps it on the video clip

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cli): init from-file a-roll is timed, muted strip is attribute-exact

The from-file a-roll video gets data-start="0" so a scaffold lints clean, and
the orphaned audio-only slot comment is dropped. setVideoAudio now walks the
tag attribute by attribute, so a class value containing the word muted
survives. The ffprobe-missing warning says the video is assumed to have sound
and how to switch to muted, and DEFAULT_META notes the failure surfaces at
render until lint support lands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): keep has-audio in step with the muted toggle, test cutout confirm

Muting a video now clears has-audio and unmuting sets it to true, in both the
Design and flat media panels, matching the Has audio track toggle. Audio
elements are unchanged. Adds coverage for both panels plus the cutout
window.confirm guard: declined leaves src alone, accepted swaps src and mutes,
a silent video never prompts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(parsers): updateElementInHtml hasAudio only applies to videos, true or false

An audio element or an undefined hasAudio no longer picks up a muted attribute;
only an explicit false on a video writes muted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: video_missing_muted rule text, silent-footage notes, consistent audible examples

lint.mdx and cli.mdx describe the current video_missing_muted rule instead of
the retired unmuted-video wording. The editing recipes say how to adapt for
silent footage and require the outgoing clip's sound to end at a J-cut. The
Remotion media example shows the audible mapping consistently, data-attributes
states that has-audio is required unless muted, and the hairline frame note no
longer claims the video is muted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cli): encode init media filenames and probe --audio duration

Function replacers stop dollar-sign pattern expansion in filenames; src is percent-encoded.
--audio now passes the probed track length as the composition duration in both init paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(lint): audio-state-aware media rules and time-aware double-source check

video_missing_muted honors any explicit data-has-audio.
video_audio_double_source only considers audible videos and only fires
when the same-src clips overlap in time.
duplicate_audio_track now covers audible videos.
audio_file_without_element no longer claims silence when an audible video
carries the sound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skills): audio-stream check for remotion ports, followable J-cut recipe

Remotion Video maps to data-has-audio only when the file has an audio stream.
The J/L-cut recipe now shows both shots with muted pictures and separate audio.
embedded-captions removes the audio.mp3 scratch file it extracts.
skills-manifest.json regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(lint): trim comments to satisfy the comment ratchet

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): atomic muted edits, cutout keeps sound on a linked audio

Muted and Has audio track commit one batched write (one persist, one
undo entry) and remove data-has-audio instead of writing an empty
marker. Unmuting a video probes its src: audible -> true, silent ->
false, probe failed -> absent. Remove background on an audible video
inserts a linked sibling <audio> with the original sound in the same
write, replacing the drop-the-sound confirm.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(producer): skip hidden videos when auto-adding video audio

A video hidden in Studio is dropped by the mixer, so its src never reached
existingAudioSrcs and the render re-added it at full volume. The video parser
now records the shared hidden predicate and the auto-detect step skips hidden
videos.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): keep useDomEditSession under the 600-line cap

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: miga-heygen <miga-heygen@users.noreply.github.com>
Co-authored-by: kapelame <kapelame@users.noreply.github.com>
 2026-10-01 23:56:36 +00:00
jmoran-heygenJake MoranClaude Fable 5.1Miguel Angel Simon Sierra
 		839ae426dd feat(studio): typed audio values, clip fades, audible video drops (#4250) 
* feat(studio): typed audio values, clip fades, audible video drops

Audio rows in the media inspector (Volume, Speed, Media start) take a
typed value: click the readout, type "-6 dB", "1.5x" or "2.5s", Enter.
Speed presets are hidden on <audio> clips.

Clip-edge fades: data-fade-in / data-fade-out in seconds, honoured by the
preview runtime (per-tick gain), the render mixer (afade after volume) and
Studio (Fade in / Fade out rows, and drag dots at the top corners of audio
clips and audible videos). Linear on both sides so preview and render
agree; fades that outrun the clip scale to meet inside it.

Dropped videos with an audio stream now land with data-has-audio="true"
instead of muted, so the mixer keeps their sound. The media metadata
endpoint reports hasAudio from the ffprobe call it already makes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(audio): one owner for fades, dB, and afade-after-volume

Illegal fade values become no-fade at fadeGain; typed inspector input
still returns null so the readout stays red. Master and group mixes
share one filter chain with volume then afade. Fade-out stays anchored
to the trimmed clip end.

* fix(studio): share audio helpers and bound typed media values

* fix(studio): reserve sibling fade duration when editing

---------

Co-authored-by: Jake Moran <jakemoran2011@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Miguel Angel Simon Sierra <miguel.sierra@heygen.com>
 2026-09-21 18:45:08 -04:00
Miguel Ángel 2db126d4c8 docs(skills): one owner doc for everyday edits and a hyperframes-studio skill (#4125) 
* docs(skills): one owner doc for everyday edits and a hyperframes-studio skill

Agents got contradictory advice on volume fades and were blocked from splits
and copies by the Ask-agent guardrails.

- creator-editing-recipes.md is the single owner: adds align-to-event,
  copy-a-group and swap-media recipes and the one volume rule (use the lane)
- variables-and-media, data-attributes (skill and CLI copy), the gsap adapter
  and tracks-and-clips now defer to it
- Ask-agent guardrails allow the neighbour edits a split, retime, copy or swap
  requires and point at the owner doc
- new hyperframes-studio skill: sub-compositions for every scene, one caption
  track, one element kind per track, safe zones
- drift test lints every html example in the owner doc with the real linter
  and fails on any doc teaching a volume tween

* refactor(skills): drop unused tween allowlist from recipes drift test

* docs(skills): name the track kind attribute in the studio skill

* docs(skills): stop claiming lint rules that do not exist yet and fix track numbering

Review fixes:

- studio skill: drop the severity claim and the preview-overlay claim,
  group tracks by data-track-kind instead of numeric ranges that clashed
  with the recipes
- recipes: audio data-start is root time only in the root composition;
  scene-local inside a sub-composition file
- guardrails: timeline edits are exempt from the element-only and timing
  lines; the pointer names the skill reference, not a repo path
- montage doc defers to the volume lane; skill count is 21 everywhere
- drift test also scans the montage doc

* fix(skills): the ducking generator emits a volume lane and captions defer to the studio safe zones

- audio-duck.mjs prints a data-automation lane (clip-local times from the
  target data-start) instead of a timeline volume tween; --json adds the lane
- operations.md shows the generated lane; the drift test scans it
- embedded-captions safe-zone lines point at the studio skill, which owns
  the values

* chore(skills): regenerate the skills manifest after rebase

* docs(skills): remotion translation docs point volume ramps at the automation lane

* docs(skills): studio safe zones use the action-safe 90 and title-safe 80 boxes

* docs(skills): caption docs defer to the studio safe boxes

* chore(skills): regenerate the skills manifest after rebase

* fix(skills): studio skill names the safe margins; guardrail test matches the timeline-edit wording
 2026-09-18 23:19:40 -04:00
Miguel Ángel f059a6eb53 feat(cli): default BeginFrame, agent blank, and looks quality (#3927) 
* feat(cli): render summary names the capture path, gpu mode and stage timings

* fix(lint): drop gsap_exit_missing_hard_kill, its hazard does not reproduce on cold workers

* fix(cli): render slow-path hint fires only when the gpu was auto-probed

* test(cli): isolate agent hint suite from ambient agent env keys

The suite runs inside agent sessions. Ambient env keys fill the 16-key
hint cap and fail assertions that a plain shell would pass.

* docs(music-to-video): drop leftover hard-kill mandate after lint rule removal

Frame workers still required a tl.set hard-kill that the dropped lint
rule no longer enforces. Align the self-check with montage.md.

* refactor(cli): share hint-key pattern and collapse render summary args

Export HINT_KEY_PATTERN for the test isolation strip. Read duration and
frame count from the perf summary instead of passing them beside it.

* test(cli): lock capture-mode preference and 16-key hint isolation

Session mode, including drawelement, must print over observability.
The hint suite now fails if ambient keys already fill the 16-key cap.

* feat(cli): default BeginFrame, agent blank, and looks quality

Local auto render opts out of the software-GPU screenshot clamp so
BeginFrame can run. init --agent scaffolds a centered Inter blank with
no prompts. --quality looks (now the default) is CRF 16; delivery is high.

* fix(cli): honor screenshot env and teach agent pitfalls on the happy path

Local auto still requests BeginFrame unless PRODUCER_FORCE_SCREENSHOT=true.
The agent blank fills its canvas so --resolution can resize it. Skills now
name the four pitfalls and tell agents to read the render summary line.

* feat(cli): default init to the centered blank

Bare init scaffolds the centered Inter stage. --agent is a hidden alias.

* fix(cli): pack from-file and use it for video init

TTY --video with no --example selects from-file. Pack copies that template.

* fix(cli): wire --video onto from-file even with --example blank

One resolver picks the scaffold. Spawned init --video asserts a-roll src.

* fix(cli): narrow looks CRF on the quality alias

Only looks carries crf. Typecheck failed on the union without a guard.

* style(cli): shorten render-pipeline summary comment
 2026-09-13 21:44:19 -04:00
Miguel Ángel b2fc18b2df fix(skills,lint): correct composition-contract claims the code contradicts (#3468) 
The runtime absorbed a series of authoring mistakes over time and `runtime/init.ts`
says so in its own comments, but the skills kept teaching the old rules. Four of
them actively cost an agent a failing run: add `crossorigin` (lint rejects it
unconditionally), never build a timeline inside `async` (lint calls that the
documented contract), never `gsap.set` later-scene clips (two fixHints instruct
exactly that), and 12 copyable media snippets with no `id`, which render silent.

Corrected in every place each claim appeared, including `hyperframes-animation`,
three workflow scripts, the scaffolded project instructions, the CLI `docs`
command, and the public docs site: `data-track-index` is a Studio display lane
the render never reads, `class="clip"` is a layout convention rather than a
visibility requirement, timed elements may nest, the visibility window is
half-open, sub-composition host dimensions are backfilled, and the root-fill rule
applies only to the layered-composite path.

Behaviour changes, each backed by a render rather than by reading code:

- `timeline_registry_missing_init` deleted. The runtime creates the registry
  before any inline script; a composition without the guard line renders and
  animates correctly.
- `video_nested_in_timed_element` kept, message corrected. A rendered repro shows
  the nested-with-local-start case really does break, so the rule guards a real
  defect, but nothing is "FROZEN": the extractor ignores the wrapper's offset
  while visibility uses it, so the clip shows wrong frames and then vanishes.
- `mediaRenderIds` now stamps media whose source is a `<source>` child, closing a
  duplicate-id gap the old `[src]`-only selector left open.
- Stale messages fixed on `subcomposition_root_styled_by_class` and
  `deprecated_data_layer`.

`coreSkillContent.test.ts` pinned the literal sentence that made root
`data-start` look required, so it is narrowed to structure plus the regression it
genuinely catches.

Not covered, and flagged in the PR: the media global-vs-local start heuristic in
`runtime/init.ts` is the root cause behind the nested-video defect. Removing it
changes the meaning of existing compositions and needs its own deprecation.
 2026-08-24 18:04:39 -04:00
Miguel Ángel 8408a44745 fix(engine): tune VP9 cpu-used across render paths (#1614) 
* fix(engine): tune VP9 cpu-used across render paths

* fix: address VP9 review feedback
 2026-06-20 16:36:59 -04:00
James RussoClaude Opus 4.8Xuelong Mu
 		36b24acf20 feat: add video frame format render option (#1481) 
* feat: add video frame format render option

* refactor: single source of truth for video-frame-format allow-list

Addresses PR review (Via) on #1481: the ["auto","jpg","png"] set was
declared three times — render.ts (VIDEO_FRAME_FORMATS), server.ts
(inline includes), and renderConfigValidation.ts
(ALLOWED_VIDEO_FRAME_FORMATS) — three boundaries to update when a new
extraction format lands.

Hoist the constant + a reusable `isVideoFrameFormat` type guard into
@hyperframes/engine (where VideoFrameFormat is defined) and route all
three call sites through them. Behavior unchanged; also drops two
`as RenderConfig[...]` casts in favor of the guard (narrowing over
assertion, per repo TS conventions).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Xuelong Mu <xuelongmu@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-15 22:05:20 -07:00
Miguel Ángel 0e052e42d2 fix(engine): support AMD AMF GPU encoding 2026-05-26 13:35:55 -04:00
James f635deb86a feat(engine): cache probe Promise + log resolved mode + sync docs 
Three follow-ups from Vai's staff-eng review:

1. Concurrent-probe race (real bug): the parallel coordinator runs N
   workers via Promise.all, so `--workers 4` on a no-GPU host fired 4
   simultaneous probe Chromes — each paying the same 240 ms launch cost.
   Cache the *Promise* (not the resolved value): first caller assigns
   the in-flight Promise, every other concurrent caller awaits the same
   one. Verified with a new test asserting all concurrent callers get
   the identical Promise reference.

2. Stale rendering.md (lines 23, 29): user-visible contract said
   "browser GPU enabled by default", which was wrong post-auto. Now
   describes the auto / hardware / software trichotomy explicitly.

3. Silent fallback: auto-mode produced no output, so a regression to
   "always falls back to software even with GPU present" would have
   been invisible in production logs. Added a single stderr line per
   process when the probe resolves: `[hyperframes] browserGpuMode auto
   → <mode> (<reason>)`. Cache hits don't re-log.

Verification:
- Engine 536/536 (incl. new concurrent-dedup test asserting Promise
  reference equality across simultaneous callers)
- CLI 256/256
- Format / lint / typecheck clean
 2026-05-06 17:33:55 +00:00
JamesandClaude Opus 4.7 21243b6957 docs: lead with shape contrast for variable attributes 
Follow-up on the PR #603 review. The previous fix named both attributes
but didn't make their distinct shapes / roles obvious; a reader could
still wonder "are these two views of the same data?". Now the doc
opens with the shape contrast (array of declarations vs object of
values) and the section closes with a numbered precedence layering so
the merge order is unambiguous.

- compositions.md: replaced the bullet list with a shape-first
  description ("JSON array of declarations" vs "JSON object keyed by
  variable id"), an explicit "they aren't redundant" line, and a
  numbered list of the three precedence layers (declared default →
  host data-variable-values → CLI --variables).
- skills/hyperframes-cli/SKILL.md: highlighted the same shape contrast
  inside the parametrized-renders paragraph (declarations array vs
  values object).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 20:26:20 +00:00
JamesandClaude Opus 4.7 22bcd7a18b docs: clarify declaration vs override attributes (PR #603 review) 
James pointed out (compositions.md:33, SKILL.md:121) that the prose
referenced `data-variable-values` while the example below showed
`data-composition-variables`, leaving readers to wonder if the two
names referred to the same thing. They don't: one declares, the other
overrides per-instance. Both are now named at first mention and the
declare-vs-override split is called out explicitly.

- packages/cli/src/docs/compositions.md: replaced the single intro
  sentence with a two-bullet list ("data-composition-variables
  declares, data-variable-values overrides per-instance") and a
  follow-up explaining where the CLI fits in.
- skills/hyperframes-cli/SKILL.md: rewrote the parametrized-renders
  paragraph so declaration (data-composition-variables) and override
  (--variables) are distinct sentences, with the per-instance attribute
  parenthetical for completeness.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 20:25:42 +00:00
JamesandClaude Opus 4.7 211a9214d0 docs(skills): teach agents the variables system across SKILL.md + docs 
Distribution PR for the variables feature stack: tells agents how to
declare, read, and override variables across the four authoring
surfaces.

skills/hyperframes/SKILL.md:
- Added data-variable-values + data-composition-variables to the
  data-attributes tables (host element + <html> root respectively).
- New "Variables (Parametrized Compositions)" section right after
  "Composition Structure". Three-step pattern (declare / read /
  override), full worked example with enum variable, sub-comp
  per-instance pattern with two hosts sharing a source, and rules
  of thumb (always provide defaults; read once, not in frame loops;
  use --strict-variables in CI; type validation behavior).

skills/hyperframes-cli/SKILL.md:
- Added --variables, --variables-file, --strict-variables to the
  render flag table.
- Short paragraph below the table explaining the parametrized-render
  pattern with a forward reference to the hyperframes skill.

docs/packages/core.mdx:
- Added a code snippet showing getVariables<T>() inside a composition
  and validateVariables/formatVariableValidationIssue for tooling.

packages/cli/src/docs/compositions.md (the in-CLI `npx hyperframes
docs compositions` content):
- Replaced the hand-rolled JSON.parse(host.dataset.variableValues)
  pattern with the modern getVariables() pattern.

This is PR 4 of the 4-PR stack. The openai/plugins mirror is a
separate follow-up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 20:25:42 +00:00
Miguel Ángel 395fb9c084 feat: add browser GPU render mode (#571) 
## Problem

HyperFrames already had `--gpu`, but that flag only controlled FFmpeg hardware encoding. The browser capture path still forced Chrome/WebGL through SwiftShader software GL via `--use-angle=swiftshader`, so WebGL-heavy local renders could leave the biggest bottleneck on the CPU path.

That made the existing flag naming easy to misread: `--gpu` sounded like it accelerated the whole render, but it did not change the browser frame-capture backend.

## What this fixes

- Enables host browser GPU acceleration automatically for local CLI renders.
- Adds `--no-browser-gpu` as the local opt-out for software Chrome/WebGL capture.
- Keeps `--browser-gpu` as an explicit local browser-GPU request.
- Adds `browserGpuMode: "software" | "hardware"` to engine config, with `PRODUCER_BROWSER_GPU_MODE` env support for lower-level producer users.
- Keeps Docker browser capture on the deterministic software path.
- Maps hardware browser GPU mode to platform-native Chrome backends:
  - macOS: Metal-backed ANGLE
  - Windows: D3D11-backed ANGLE
  - Linux: EGL
- Blocks explicit `--browser-gpu --docker` with a clear error because Docker browser GPU passthrough is not cross-platform.
- Clarifies docs so `--gpu` means FFmpeg encoder GPU and browser GPU means Chrome/WebGL capture GPU.
- Keeps encoder backend selection auto-detected from FFmpeg capabilities:
  - NVIDIA: NVENC
  - macOS: VideoToolbox
  - Linux: VAAPI
  - Intel: QSV

## Why two flags

There are two separate GPU surfaces in the render pipeline:

1. Browser GPU controls Chrome frame capture.
   - Affects WebGL, canvas, CSS rendering, compositing, and screenshot capture inside the browser.
   - This is enabled automatically for local CLI renders.
   - Use `--no-browser-gpu` when you want the software browser baseline.

2. `--gpu` controls FFmpeg video encoding.
   - Affects the final encode step after frames have already been captured.
   - The concrete encoder is auto-detected from the host FFmpeg build and hardware.
   - It can be faster for some machines/codecs, but it is not equivalent to browser rendering acceleration.

The controls stay independent because users may want:

- `hyperframes render` for the fast local default with browser GPU capture.
- `hyperframes render --no-browser-gpu` for the software-browser local baseline.
- `hyperframes render --gpu` for browser GPU capture plus hardware FFmpeg encoding.
- `hyperframes render --no-browser-gpu --gpu` for software browser capture plus hardware FFmpeg encoding.
- `hyperframes render --docker` for deterministic browser capture.

## Why `--gpu` does not imply browser GPU

Keeping `--gpu` scoped to FFmpeg encoding avoids a semantic break and keeps the risk profile explicit:

- `--gpu` already means encoder acceleration. Expanding it to also change Chrome capture would silently alter behavior for users who only wanted hardware encoding.
- Browser GPU and encoder GPU have different portability. Encoder GPU can work in Docker when the host exposes the right devices; browser GPU passthrough is not cross-platform, so this PR intentionally blocks explicit `--browser-gpu --docker`.
- The Apple presentation benchmark shows why the controls should stay separate: browser GPU capture was the useful improvement, while macOS VideoToolbox via `--gpu` was slower and produced larger output for this `standard` H.264 run.

If HyperFrames later wants a single umbrella acceleration control, it should be explicit, for example `--acceleration browser|encoder|all` or `--gpu=browser|encoder|all`, rather than changing the meaning of the existing boolean `--gpu`.

## Root cause

`buildChromeArgs()` always injected `--use-gl=angle --use-angle=swiftshader`. `disableGpu` only appended `--disable-gpu`; it did not provide a hardware-GPU mode. That made the public `--gpu` flag look broader than it was, because render capture stayed software-backed even when encoder GPU was requested.

## Verification

### Local checks

- `bun install`
- `bun run build:hyperframes-runtime`
- `bun run --filter @hyperframes/engine test src/config.test.ts src/services/browserManager.test.ts`
- `bun run --filter @hyperframes/cli test src/utils/dockerRunArgs.test.ts src/commands/render.test.ts`
- `bun run --filter @hyperframes/cli typecheck`
- `bun run --filter @hyperframes/engine typecheck`
- `bun run --filter @hyperframes/producer typecheck`
- `cd packages/producer && bunx vitest run src/services/renderOrchestrator.test.ts`
- `bunx oxlint packages/cli/src/commands/render.ts packages/cli/src/commands/render.test.ts packages/cli/src/utils/dockerRunArgs.ts packages/cli/src/utils/dockerRunArgs.test.ts packages/engine/src/config.ts packages/engine/src/config.test.ts packages/engine/src/services/browserManager.ts packages/engine/src/services/browserManager.test.ts packages/producer/src/services/renderOrchestrator.test.ts`
- `bunx oxfmt --check ...` on changed source/docs files
- `git diff --check`
- `bun packages/cli/src/cli.ts render --help | rg -n "browser-gpu|no-browser-gpu|GPU"`
- `bun packages/cli/src/cli.ts render packages/producer/tests/css-spinner-render-compat/src --output /tmp/hf-auto-browser-gpu-smoke.mp4 --workers 1 --quality draft --fps 24 --strict`
  - Render plan prints `GPU: browser GPU (auto)`.
- `bun packages/cli/src/cli.ts render packages/producer/tests/css-spinner-render-compat/src --no-browser-gpu --output /tmp/hf-software-browser-gpu-smoke.mp4 --workers 1 --quality draft --fps 24 --strict`
  - Render plan does not print browser GPU.
- `bun packages/cli/src/cli.ts render packages/producer/tests/css-spinner-render-compat/src --docker --browser-gpu --output /tmp/should-not-render.mp4`
  - Exits 1 with `Browser GPU is local-only`.
- `buildDockerRunArgs()` regression coverage asserts Docker container args include `--no-browser-gpu`, preventing nested container renders from re-enabling browser GPU through the local CLI default.
- `resolveBrowserGpuForCli()` regression coverage asserts `PRODUCER_BROWSER_GPU_MODE=software` opts out when no CLI browser-GPU flag is supplied, while explicit `--browser-gpu` / `--no-browser-gpu` still win.
- `ffmpeg -v error -i /tmp/hf-auto-browser-gpu-smoke.mp4 -f null -`
- `ffmpeg -v error -i /tmp/hf-software-browser-gpu-smoke.mp4 -f null -`
- `ffprobe -v error -show_entries format=duration:stream=codec_name,width,height,r_frame_rate -of json /tmp/hf-browser-gpu-smoke.mp4` -> H.264, 1920x1080, 24fps, 5.0s

### Apple presentation benchmark

Rendered `/Users/miguel07code/Downloads/apple-presentation.zip` as supplied after extracting to `/tmp/hf-apple-profile/apple-presentation`.

Fixed settings:

- 1920x1080
- 30fps
- `standard` quality
- 4240 frames
- 141.32s duration
- 8-worker cap; render auto-calibration used 6 capture workers
- macOS host detected FFmpeg GPU encoder: `videotoolbox`

| Mode | Equivalent flags after this PR | Wall time | vs software-browser baseline | Speed | Capture | Encode | Output |
| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: |
| Software browser + CPU encode | `--no-browser-gpu` | 120.77s | baseline | 1.17x | 97.87s | 10.04s | 8.38MB |
| Browser GPU + CPU encode | default local render | 70.10s | 42.0% faster | 2.02x | 50.72s | 9.91s | 8.39MB |
| Software browser + encoder GPU | `--no-browser-gpu --gpu` | 133.16s | 10.3% slower | 1.06x | 103.58s | 18.31s | 25.43MB |
| Browser GPU + encoder GPU | `--gpu` | 74.12s | 38.6% faster | 1.91x | 46.69s | 17.93s | 25.45MB |

Result: browser GPU capture is the meaningful improvement for this WebGL/browser-capture-heavy presentation. VideoToolbox encoding was slower and produced larger files for this current `standard` H.264 path, so `--gpu` should stay separate and opt-in.

Why `--gpu` plus browser GPU was slower than browser GPU alone: the combined run captured about 4.0s faster than browser GPU alone, but VideoToolbox encoding was about 8.0s slower than CPU x264 encoding, so the encode loss outweighed the capture gain.

### VideoToolbox flag check

I also isolated the encode stage against the already-captured Apple frames to check whether macOS GPU encoding only needed special flags.

`ffmpeg -h encoder=h264_videotoolbox` does not expose a CRF/CQ-style quality option like x264. It exposes bitrate-oriented and VideoToolbox-specific options such as `-b:v`, `-realtime`, `-profile`, `-coder`, `-prio_speed`, `-power_efficient`, and `-allow_sw`. That means our current `-q:v` mapping is not equivalent to x264 CRF and can produce very different bitrate/size behavior.

Measured full-frame encode variants on this host:

| VideoToolbox variant | Encode wall time | Output size | Bitrate |
| --- | ---: | ---: | ---: |
| Current `-q:v 64 -allow_sw 1` | 18.76s | 25.31MB | 1.43 Mbps |
| Current without `-allow_sw 1` | 18.21s | 25.31MB | 1.43 Mbps |
| `-b:v 500k -maxrate 750k -bufsize 1000k -profile high -coder cabac -realtime 1 -prio_speed 1 -power_efficient 0` | 20.58s | 7.42MB | 0.42 Mbps |
| Same with `-b:v 1500k` | 20.84s | 16.70MB | 0.95 Mbps |
| `-b:v 500k -profile baseline -coder cavlc -realtime 1 -prio_speed 1 -power_efficient 0` | 18.11s | 8.94MB | 0.51 Mbps |

Conclusion: VideoToolbox can be made size/bitrate-predictable with explicit `--video-bitrate`, but the tested speed-oriented flags did not make it faster than CPU x264 wall time for this render. That reinforces keeping `--gpu` encoder acceleration explicit and separate from browser GPU capture.

Artifacts from the local benchmark:

- `/tmp/hf-apple-profile/results/cpu.mp4`
- `/tmp/hf-apple-profile/results/browser-gpu.mp4`
- `/tmp/hf-apple-profile/results/encoder-gpu.mp4`
- `/tmp/hf-apple-profile/results/full-gpu.mp4`
- `/tmp/hf-apple-profile/results/summary.json`

All four benchmark MP4s completed `ffprobe` and full `ffmpeg -f null` decode checks.

### Pixel comparison

Compared decoded MP4 output between software-browser and browser-GPU renders:

- Apple presentation:
  - 4240 frames compared
  - 636 exact matching decoded frame hashes
  - 3604 different decoded frame hashes
  - Average PSNR: 57.79 dB
- `css-spinner-render-compat` clean fixture:
  - 120 frames compared
  - 0 exact matching decoded frame hashes
  - Average PSNR: 61.57 dB

Interpretation: browser GPU output is not strict hash/pixel-identical to the software-browser path after lossy H.264 encode, but the measured deltas are visually tiny. Above 50 dB PSNR is typically visually indistinguishable for normal video review. Use `--no-browser-gpu` or Docker when strict cross-run/cross-machine reproducibility matters more than local speed.

### Browser verification

- Started HyperFrames Studio preview for `packages/producer/tests/css-spinner-render-compat/src`.
- Used `agent-browser` to open `http://localhost:5191#project/src` and verify the composition loaded in Studio.
- Screenshots:
  - `/tmp/hf-gpu-browser-proof/preview-loaded.png`
  - `/tmp/hf-gpu-browser-proof/preview-playing.png`
  - `/tmp/hf-gpu-browser-proof/preview-frame-60.png`
- Agent-browser recordings:
  - `/tmp/hf-gpu-browser-proof/preview-playback.webm`
  - `/tmp/hf-gpu-browser-proof/preview-seek.webm`

## Notes

- Browser GPU is enabled automatically for local CLI renders and disabled in Docker.
- `--no-browser-gpu` is the opt-out for software Chrome/WebGL capture.
- `--gpu` remains encoder-only and opt-in.
- The Apple presentation zip has existing lint errors around unmanaged nested videos and imperative media `play()` calls. The benchmark still compares the same supplied source across modes, but it should not be treated as a clean deterministic-composition fixture.
 2026-04-30 06:46:14 +02:00
Miguel Ángel fc52d21c59 docs: clarify composition variable usage (#420) 
## Summary
- replace the unsupported `data-var-*` example with the current `data-variable-values` pattern
- document that variable values are carried through but still applied manually inside the nested composition
- add matching reference notes in the data-attributes, HTML schema, core package, and CLI docs

## Verification
- `npx mintlify dev --port 3100`
- browser verification with `agent-browser` on `/concepts/compositions` and `/reference/html-schema`
- proof artifacts saved locally under `tmp/issue-416-docs/`
 2026-04-22 20:41:35 +02:00
Vance Ingalls 00af29c169 fix(cli): forward --hdr through Docker render + HDR docs (#346) 
## Summary

This PR ended up covering the full HDR Docker/docs follow-through plus the producer/engine work needed to make HDR still images render and regress correctly in CI.

The branch now does four things:

- forwards `--hdr` through the Docker render path in the CLI
- adds and expands HDR documentation across the docs site
- adds first-class HDR still-image support to the engine/producer pipeline
- adds targeted HDR regression coverage, including a CI-safe fallback for PNG HDR metadata detection when `ffprobe` does not expose PNG color tags

## What changed

### CLI and docs

- `hyperframes render --docker --hdr` now preserves `--hdr` when invoking the in-container CLI
- added a dedicated HDR guide and linked it from CLI, producer, engine, rendering, and common-mistakes docs
- documented HDR constraints and verification flow: HDR source requirements, MP4/H.265 Main10 output, PQ/HLG handling, Docker usage, and common SDR fallback causes

### Engine and producer HDR image support

- added `ImageElement` support to the engine composition model and parsing path
- threaded image elements through producer compilation and orchestration
- probed image sources for HDR color spaces so image-only compositions can trigger HDR output without requiring an HDR video source
- included HDR image start times in stacking queries so the layered compositor can place images correctly in z-order
- integrated HDR image compositing into the layered HDR render loop alongside native HDR video layers and SDR DOM overlays
- forced screenshot mode for HDR layered compositing where required to keep DOM/HDR layer composition deterministic
- skipped readiness waiting for natively extracted HDR videos in the engine path where it was unnecessary and could block layered HDR flows

### HDR metadata robustness

- added a fallback in `extractVideoMetadata()` to read PNG `cICP` metadata directly when `ffprobe` omits color-space fields for PNGs
- this specifically fixes CI/Docker detection for the `hdr-image-only` fixture, where the render was falling back to SDR because the PNG was not being recognized as BT.2020 PQ

### Regression coverage and fixture cleanup

- added `hdr-image-only`, a regression fixture that validates HDR still-image rendering end to end
- added `hdr-pq`, a focused HDR PQ regression fixture for the video path
- updated regression CI to run an `hdr` shard with `--sequential hdr-pq hdr-image-only`
- removed the older larger `hdr-regression/*` fixture set in favor of the smaller targeted regressions used by CI
- added the necessary fixture generation/readme material and checked-in golden outputs for the new HDR tests

## Why

The original PR description only covered the CLI flag forwarding and docs work. Since then, the branch also picked up the missing runtime support needed for HDR still images and the regression coverage to keep that path from breaking.

The practical issue this closes is:

- local host runs could pass while CI failed `hdr-image-only`
- the failure was a full-frame visual mismatch caused by SDR fallback, not unstable rendering
- root cause was PNG HDR metadata not being surfaced by `ffprobe` in the CI Docker environment
- parsing the PNG `cICP` chunk directly makes HDR detection deterministic across environments

## Test plan

### Local targeted checks

```bash
bunx oxlint packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts
bunx oxfmt packages/engine/src/utils/ffprobe.ts packages/engine/src/utils/ffprobe.test.ts
bun --cwd packages/engine test src/utils/ffprobe.test.ts src/utils/hdr.test.ts
```

### Producer regression runs on host

```bash
bun run --cwd packages/core build:hyperframes-runtime:modular
bun --cwd packages/producer test -- --sequential --exclude-tags slow,render-compat,hdr
bun --cwd packages/producer test -- --sequential hdr-pq hdr-image-only
```

Observed result:
- `fast` shard: 7 passed, 0 failed
- `hdr` shard: 2 passed, 0 failed

### CI-equivalent Docker verification

```bash
docker build -f Dockerfile.test -t hyperframes-producer:test .

docker run --rm \
  --security-opt seccomp=unconfined \
  --shm-size=4g \
  -v "$PWD/packages/producer/tests:/app/packages/producer/tests" \
  hyperframes-producer:test \
  --sequential hdr-pq hdr-image-only
```

Observed result:
- `hdr-image-only`: passed
- `hdr-pq`: passed
- shard summary: 2 passed, 0 failed

### Specific regression fixed

Before the PNG `cICP` fallback, the Docker/CI run failed `hdr-image-only` with:

- missing `"[Render] HDR source detected — output: PQ ..."` log line
- full-frame visual mismatch across all 100 checkpoints
- PSNR ~17 on every frame, indicating a consistent SDR-vs-HDR pipeline mismatch

After the fallback, the same Docker path recognizes the PNG as HDR and the shard passes.
 2026-04-20 12:16:24 -07:00
James Russo 08fb1de61f feat(cli): add command + hyperframes.json (#256) 
## What

PR 5/17 of the catalog system rollout. Adds the `hyperframes add` verb for installing blocks and components from the registry into an existing project, plus the `hyperframes.json` project config that tells `add` which registry to use and where to drop files. Stacks on #255.

- **`packages/cli/src/commands/add.ts`** — new `hyperframes add <name>` command. Resolves an item, validates target paths, installs files in parallel, builds an include snippet, copies it to the clipboard. Exposes a testable `runAdd(opts)` function; the citty default wraps it with console output + exit handling
- **`packages/cli/src/utils/projectConfig.ts`** — read/write/normalize `hyperframes.json`. Tolerant to missing and partial configs
- **`packages/cli/src/utils/clipboard.ts`** — minimal cross-platform clipboard (pbcopy / clip.exe / wl-copy / xclip / xsel). Zero deps. Gracefully no-ops in headless environments
- **`packages/cli/src/commands/init.ts`** — write `hyperframes.json` during scaffold if not already present
- **`packages/cli/src/cli.ts`** + **`help.ts`** — register `add` under Getting Started (directly below `init`)

Design doc: [Hyperframes Catalog System](https://www.notion.so/heygen/Hyperframes-Catalog-System-Design-Plan-341449792c69813f899dcd53b4c0383a).

## UX

```bash
# Scaffold a project (now writes hyperframes.json too)
npx hyperframes init my-video --example blank
cd my-video

# Add a block — files land, snippet copied to clipboard
npx hyperframes add claude-code-window
#  ✓ Added claude-code-window (hyperframes:block)
#    compositions/claude-code-window.html
#
#  Include snippet:
#    <iframe src="compositions/claude-code-window.html" data-start="0" data-duration="6"></iframe>
#
#  Copied to clipboard — paste into your host composition.

# Add a component effect
npx hyperframes add shader-wipe

# Headless / CI — no clipboard, JSON output for tooling
npx hyperframes add shader-wipe --no-clipboard --json
```

Running `hyperframes add warm-grain` (an example) errors clearly pointing to `init --example`.

## Docs (bundled in this PR per the tracker principle)

- `docs/packages/cli.mdx` — new `add` subsection under Commands (flags, examples, trigger rules) + new `hyperframes.json` section describing the config file shape

## Tests

- **`packages/cli/src/commands/add.test.ts`** — 11 tests:
  - `remapTarget` / `buildSnippet` pure helpers (5 tests)
  - `runAdd` integration against a mocked `fetch` registry: block install lands files + returns snippet, component install respects `paths.components` remap, example-typed names throw `AddError` with code `example-type`, unknown names throw `AddError` with code `unknown-item` (4 tests plus 2 covering block default path and non-default path preservation)
- **`packages/cli/src/utils/projectConfig.test.ts`** — 9 tests:
  - Write/read round-trip, partial-config normalization, corrupt-file handling, absent-file fallback to defaults, custom paths preserved
- **CLI suite:** 92 passed (was 72 on #255, **+20**). Same 4 pre-existing failures unchanged

## Scope decisions

- **`init.ts` full port to new resolver deferred.** The original plan bundled a removal of the `packages/cli/src/templates/` compat shim. That's ~300 more lines and isn't required for `add` to work. The compat shim from #254 still functions; a separate cleanup PR handles it
- **No ajv runtime schema validation.** Manifests are trusted as schema-valid. Full validation lands when third-party registries arrive (PR 14/15). Path safety is still enforced by the installer's `assertSafeTarget` guard
- **Default project paths stay under `compositions/`.** Blocks → `compositions/<name>.html`; components → `compositions/components/<name>/<file>`. Users override via `hyperframes.json#paths`

## Breaking / migration

**None.** Pure additive — new command, new file types, no existing commands or flags change. `init.ts` now writes `hyperframes.json` but that's a new additional file, not a modification of existing output.

## Stacks on

#255 — base branch. When #255 merges, this rebases onto `main`.

## Next in stack

PR 6 — `feat(registry): seed block — claude-code-window`. First real registry item. Exercises the full `hyperframes add <name>` flow end-to-end against a committed item on `main`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-13 21:04:59 -07:00
James Russo c8acd8abd8 feat(cli)!: rename --template to --example (#255) 
## What

PR 4/17 of the catalog system rollout. **Single clean cut** — the old flag is gone, replaced by `--example`. Alias changes from `-t` to `-e`. Stacks on #254.

- Rename `--template` → `--example` (alias `-e`) on `hyperframes init`
- Accept `--template` as a recognized-but-errored flag so users get a clear rename hint instead of citty silently ignoring the flag and producing a blank project
- Update all user-visible strings that referenced "template" as a user-facing concept in the init flow (picker prompt, step comments, offline-fallback suggestion)
- New `init.test.ts` covering both the success case (`--example` scaffolds) and the error case (`--template` exits 1 with rename hint)

Design doc: [Hyperframes Catalog System](https://www.notion.so/heygen/Hyperframes-Catalog-System-Design-Plan-341449792c69813f899dcd53b4c0383a).

## ⚠️ Breaking change

`--template` is no longer accepted. Example:

```bash
# before
npx hyperframes init my-video --template warm-grain

# after
npx hyperframes init my-video --example warm-grain
```

Users who still type the old flag will see:

```
The --template flag was renamed to --example. Example:
  npx hyperframes init my-video --example warm-grain
```

and the command exits with code 1. This is **user guidance, not backwards compat** — the old flag's behavior is fully gone.

## Docs (bundled per the tracker principle)

- `docs/templates.mdx` — every `--template` reference
- `docs/quickstart.mdx` — agent-mode and video-mode examples
- `docs/packages/cli.mdx` — prose, `--help` flag table, `-e` alias
- `packages/cli/src/docs/templates.md` — CLI-embedded help topic
- `README.md` and `CONTRIBUTING.md` — not affected (no flag references)

User-facing renames of the `templates.mdx` page title, nav entry, and URL route are deferred to PR 11 (catalog discoverability UX) as planned.

## Why

1. **"examples"** matches shadcn + Remotion convention for full-project scaffolds and frees the word "template" for future parameterization work (string templating, placeholder substitution)
2. Once `hyperframes add` lands in PR 5, "template" vs "block" vs "component" would be three subtly different concepts sharing one word — renaming the old one to "example" makes the taxonomy self-explaining

## How

- **citty silently ignores unknown flags.** Naively removing `--template` would cause `hyperframes init my-video --template warm-grain` to silently fall through and scaffold a blank project. So `--template` stays declared in the args schema, but its run handler immediately errors with a rename hint and exits 1
- **Internal names unchanged** — `templateId` local variables, `getStaticTemplateDir` function, `BUNDLED_TEMPLATES` constant. They're implementation details; their rename is scheduled for PR 5 when the compat shims in `packages/cli/src/templates/` are fully removed alongside the `init` refactor

## Test plan

- [x] `bun run test` in `packages/cli`: **72 passed** (was 70 on #254, +2 new `init.test.ts` cases). Same 4 pre-existing failures unchanged
- [x] **New unit tests** in `init.test.ts`:
  - `--example blank` non-interactive: exits 0, writes `index.html` to the target dir
  - `--template blank` non-interactive: exits non-zero, stderr contains the rename hint + corrected command line, target dir is **not** created
- [x] **Manual smoke:**
  - `npx hyperframes init /tmp/x --example blank` → "Created /tmp/x/"
  - `npx hyperframes init /tmp/y --template blank` → "The --template flag was renamed to --example..." exit=1
- [x] `bunx oxfmt --check` + `bunx oxlint` on changed files: clean
- [x] Pre-commit typecheck (core + studio): clean

## Incidental fix

Resolver test regression from PR 3's simplify follow-up: `loadAllItems`' warning-path test was still spying on `console.warn` after the `onWarn` callback refactor. Now uses the callback directly.

## Stacks on

#254 — base branch. When #254 merges, this rebases onto `main`.

## Next in stack

PR 5 — `feat(cli): add command + hyperframes.json`. The big UX PR where:
- `init.ts` gets fully ported to the new registry resolver
- Compat shims in `packages/cli/src/templates/` are removed
- Users gain the `add` verb for installing blocks and components into existing projects
- `hyperframes.json` project-config file lands

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-04-13 20:44:23 -07:00
Vance Ingalls 20be2ea1c2 style: apply oxfmt baseline formatting across all source files (#25) 
## Summary
- Run `oxfmt .` across the entire codebase to establish formatted baseline
- 299 files changed — mechanical formatting only, no logic changes
- Double quotes, semicolons, 2-space indent, trailing commas, 100 print width

Part 3/4 of [VA-851](https://linear.app/heygen/issue/VA-851/pre-migration-configure-eslint-prettier-and-conventional-commits)

## Test plan
- [x] `pnpm format:check` — all 426 files pass
- [x] `pnpm -r typecheck` — all packages pass
- [x] `pnpm build` — all packages build
- [x] All 348 tests pass
 2026-03-23 17:15:14 -07:00
Vance IngallsandClaude Opus 4.6 9f8e5ba5a1 initial code (#2) 
* feat: initial code port from hyperframes-internal

Port all OSS-ready packages from the internal monorepo:
- @hyperframes/core — shared types, HTML generation, GSAP utilities, runtime
- @hyperframes/cli — CLI for creating, previewing, and rendering compositions
- @hyperframes/engine — framework-agnostic rendering engine (BeginFrame + FFmpeg)
- @hyperframes/producer — video rendering pipeline (Puppeteer + FFmpeg)
- @hyperframes/ui-player — browser-based video player component
- @hyperframes/studio — composition editor (React frontend + Hono backend)

Includes regression test suite with Docker-based test harness.

All HeyGen-internal references, deployment infrastructure, and
proprietary assets have been removed. Package names migrated
from @app/* to @hyperframes/*.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: scrub internal codenames and stale references from OSS port

- Replace static.heygen.ai runtime URLs in test fixtures
- Remove internal CDN publish script (publish-hyperframe-runtime.ts)
- Replace sandbox-studio, sandbox-interceptor, __magicEditRuntime
  with neutral names (studio, hyperframe-runtime, __hyperframeRuntime)
- Fix stale Vault API / localhost references in docs
- Remove broken deprecated_studio link

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove remaining internal codenames and stale references

- Delete stale producer README.md and PIPELINE.md (referenced nonexistent files)
- Replace "Cerberus" codename with "HyperFrames" in test design reviews
- Replace magic-edit postMessage identifiers with hf-preview/hf-parent
- Rename debug-magic-edit-timeline.ts to debug-timeline.ts
- Replace "Motion Cut" with "HyperFrames" in Timeline comments
- Fix studio/CLI references to nonexistent archive package
  (use local data/projects/ dir, stub render proxy)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-21 22:43:56 -07:00