Files
hyperframes/skills/hyperframes-core/references/variables-and-media.md
37f30b1d06 feat: keep a video's sound on the clip by default (1/8) (#4813)
* feat(cli): one owner for init media placeholders, audio stays on the video

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(cli): init keeps a video's sound on the clip; silent files scaffold muted

Supersedes the patchVideoSrc changes in #3977 and #3942 and carries their tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: miga-heygen <miga-heygen@users.noreply.github.com>
Co-Authored-By: kapelame <kapelame@users.noreply.github.com>

* feat(registry): example templates keep the a-roll's sound on the video

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(lint): video audio rules recommend keeping sound on the video

Update video_missing_muted and video_audio_double_source message and
fixHint text to recommend the new default: keep audio on the video with
data-has-audio="true" (not muted). Tests verify the new wording is in place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(parsers): a video is either audible (data-has-audio) or muted, never both

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): follow the compiler's audio rule in the audio-track toggle

Confirm before background removal drops a video's sound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skills): hyperframes-core teaches sound on the video by default

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skills): workflows and scaffold guidance keep a video's sound on the clip

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: footage with sound keeps it on the video clip

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cli): init from-file a-roll is timed, muted strip is attribute-exact

The from-file a-roll video gets data-start="0" so a scaffold lints clean, and
the orphaned audio-only slot comment is dropped. setVideoAudio now walks the
tag attribute by attribute, so a class value containing the word muted
survives. The ffprobe-missing warning says the video is assumed to have sound
and how to switch to muted, and DEFAULT_META notes the failure surfaces at
render until lint support lands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): keep has-audio in step with the muted toggle, test cutout confirm

Muting a video now clears has-audio and unmuting sets it to true, in both the
Design and flat media panels, matching the Has audio track toggle. Audio
elements are unchanged. Adds coverage for both panels plus the cutout
window.confirm guard: declined leaves src alone, accepted swaps src and mutes,
a silent video never prompts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(parsers): updateElementInHtml hasAudio only applies to videos, true or false

An audio element or an undefined hasAudio no longer picks up a muted attribute;
only an explicit false on a video writes muted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: video_missing_muted rule text, silent-footage notes, consistent audible examples

lint.mdx and cli.mdx describe the current video_missing_muted rule instead of
the retired unmuted-video wording. The editing recipes say how to adapt for
silent footage and require the outgoing clip's sound to end at a J-cut. The
Remotion media example shows the audible mapping consistently, data-attributes
states that has-audio is required unless muted, and the hairline frame note no
longer claims the video is muted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cli): encode init media filenames and probe --audio duration

Function replacers stop dollar-sign pattern expansion in filenames; src is percent-encoded.
--audio now passes the probed track length as the composition duration in both init paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(lint): audio-state-aware media rules and time-aware double-source check

video_missing_muted honors any explicit data-has-audio.
video_audio_double_source only considers audible videos and only fires
when the same-src clips overlap in time.
duplicate_audio_track now covers audible videos.
audio_file_without_element no longer claims silence when an audible video
carries the sound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(skills): audio-stream check for remotion ports, followable J-cut recipe

Remotion Video maps to data-has-audio only when the file has an audio stream.
The J/L-cut recipe now shows both shots with muted pictures and separate audio.
embedded-captions removes the audio.mp3 scratch file it extracts.
skills-manifest.json regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(lint): trim comments to satisfy the comment ratchet

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): atomic muted edits, cutout keeps sound on a linked audio

Muted and Has audio track commit one batched write (one persist, one
undo entry) and remove data-has-audio instead of writing an empty
marker. Unmuting a video probes its src: audible -> true, silent ->
false, probe failed -> absent. Remove background on an audible video
inserts a linked sibling <audio> with the original sound in the same
write, replacing the drop-the-sound confirm.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(producer): skip hidden videos when auto-adding video audio

A video hidden in Studio is dropped by the mixer, so its src never reached
existingAudioSrcs and the render re-added it at full volume. The video parser
now records the shared hidden predicate and the auto-detect step skips hidden
videos.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(studio): keep useDomEditSession under the 600-line cap

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: miga-heygen <miga-heygen@users.noreply.github.com>
Co-authored-by: kapelame <kapelame@users.noreply.github.com>
2026-10-01 23:56:36 +00:00

8.2 KiB

Variables and Media

Two separate concerns, grouped because both control "what flows in from outside the HTML": runtime parameters (variables) and external media files (video/audio).

Variables

Declare variables on the <html> element with data-composition-variables. Each declaration needs id, type, label, and default:

<html
  data-composition-variables='[
    {"id":"title","type":"string","label":"Title","default":"Hello"},
    {"id":"accent","type":"color","label":"Accent","default":"#66d9ef"}
  ]'
></html>

Prefer declarative bindings — no script needed for direct substitution:

<img class="clip" data-start="0" data-duration="5" data-var-src="heroImage" src="fallback.jpg" />
<h1 class="clip" data-start="0" data-duration="5" data-var-text="title">Fallback</h1>
<style>
  .card {
    color: var(--accent);
  }
</style>
  • data-var-src="id" substitutes the element's src (URL string or image {url}); the authored src is the fallback.
  • data-var-text="id" substitutes the element's own text; element children (nested clips, animated spans) are preserved.
  • Every scalar variable is applied automatically as a --{id} CSS custom property on the composition root, so var(--id) CSS responds to overrides — no setProperty boilerplate.
  • Bindings resolve identically in preview and render, and per-instance for sub-compositions.
  • Caveat: media with audio should keep a real fallback src — render audio extraction reads the authored attribute (lint: media_variable_src_no_fallback).

For logic beyond direct substitution (loops, conditionals, derived values), read values once during initialization:

const { title, accent } = window.__hyperframes.getVariables();
document.getElementById("title").textContent = title;

Variable Rules

  • Supported types and their extra options (consumed by Studio's editing UI):
    • string — optional placeholder, maxLength
    • number — optional min, max, step, unit
    • color — none
    • boolean — none
    • enum — required options: [{ "value": "...", "label": "..." }, ...]
  • Always provide useful default values so preview works without CLI overrides.
  • Use data-variable-values='{"title":"Pro"}' on sub-composition hosts for per-instance overrides.
  • Use npx hyperframes render --variables '{"title":"Q4 Report"}' or --variables-file for render-time overrides.
  • Add --strict-variables in CI: turns undeclared keys, type mismatches, and enum values not in options into errors instead of warnings.
  • Read values once during init, not on every animation tick — variables don't change mid-render.
  • Media color grading can use exact variable references inside data-color-grading JSON. Use $gradingPreset or ${gradingIntensity} as the whole field value; the runtime resolves it from the current composition's variables before applying shader adjustments, finishing details, blur/pixelate effects, and custom LUTs.

Two JSON Shapes (Easy to Confuse)

  • data-composition-variables is an array of declarations (the schema): [{id, type, label, default}, ...]
  • --variables and data-variable-values are objects keyed by id (the values): {"title":"Q4","accent":"#fff"}

Media

<video>/<audio> work at any nesting depth, including inside a sub-composition <template> or a wrapper <div>. The runtime discovers media with a flat document.querySelectorAll("video, audio"), resolves each element's host composition via element.closest("[data-composition-id]"), and rebases its local data-start by the accumulated absolute start of every ancestor composition. A host at 2 with child media at 2 therefore starts that media at root time 4, consistently in preview, snapshot, extraction, and render. Legacy projects that deliberately authored a media start in root time must mark that element with data-hf-media-start-basis="global"; never infer the basis from overlapping numbers. New compositions should always use scene-local data-start. If a panel renders blank after a render, capture a per-frame snapshot and treat it as render-blocking.

The one real constraint is about timelines, not media placement: a sub-composition timeline cannot reach or animate host elements — neither document.querySelector("#host-id") nor a gsap selector string (tl.to("#host-id", …)) resolves across the boundary; a sub-comp timeline only drives its own subtree. So if a media element lives at the host root, its per-scene motion (scale/opacity/morph/tilt/breathing) must be authored on the MAIN timeline in index.html, at GLOBAL time (scene-local time + the scene slot's data-start). Keeping the media inside the scene sub-comp instead lets that sub-comp's own timeline animate it with scene-local time. For 3D tilt without a perspective parent, use gsap transformPerspective on the element. See composition-patterns.md archetype B.

A video with sound keeps it on the <video> (data-has-audio="true", no muted). Use a separate <audio> for music, voiceover, replacement audio, J/L cuts, or audio detached in Studio. Silent footage and b-roll: muted.

<video
  id="a-roll"
  class="clip"
  src="assets/demo.mp4"
  playsinline
  data-has-audio="true"
  data-start="0"
  data-duration="12"
  data-track-index="0"
  data-volume="1"
></video>

<!-- Separate <audio> only for other sound: music, voiceover, replacement audio. -->
<audio
  id="music"
  src="assets/bed.mp3"
  data-start="0"
  data-duration="12"
  data-track-index="2"
  data-volume="0.4"
></audio>

Media Rules

  • Do not call video.play(), audio.play(), pause, or seek in composition code. HyperFrames owns playback.
  • Do not drive host-root media from a sub-comp timeline: a sub-comp timeline cannot reach elements outside its subtree, so it has no effect. Drive host-root media from the main timeline at global time (or keep the media inside the sub-comp whose timeline animates it).
  • Do not animate timed media element dimensions; animate a non-timed wrapper instead.
  • Do not nest video inside a timed wrapper. lint rejects a <video data-start> whose ancestor also carries data-start (video_nested_in_timed_element, error), and the failure is real: the frame extractor resolves the video's start from its own data-start without the wrapper's offset, while visibility uses the wrapper's window. The clip then shows the wrong source frames and disappears partway through its slot. Put the timing on the wrapper or on the media element, never both.
  • Sub-compositions are exempt and work. A <video>/<audio> inside a sub-composition renders identically to one at the host root, because a composition host propagates its offset. Only plain timed wrappers (a <section data-start> around a <video data-start>) break.
  • Never add crossorigin to <video>/<audio>. lint rejects it unconditionally (media_crossorigin_breaks_preview, error) because a media host without Access-Control-Allow-Origin then fails silently in preview while renders still work, hiding the bug. There is no suppression, so this holds even for the canvas/WebGL/WebAudio readback case.
  • Every <audio> needs an id. The mixer selects audio[id][src], so an id-less <audio> is never mixed and the render is silent. lint catches it as media_missing_id.
  • A video's own sound stays on the <video> (data-has-audio="true", no muted). Add a separate <audio> only for other sound (music, voiceover, replacement audio, J/L cuts) and mute the video it replaces.
  • For volume fades and ducking, use the data-automation volume lane; the exact form is in creator-editing-recipes.md. data-volume is the static baseline. A timeline volume tween is ignored when a lane is present.

For media duration: <video> and <audio> can omit data-duration if the media's intrinsic length is known and you want the full clip. Otherwise provide data-duration explicitly.

Input codecs: render decodes video via FFmpeg (frames are pre-extracted and injected), so HEVC/H.265 assets (8/10-bit) render correctly everywhere; live preview auto-proxies any browser-hostile asset (transcodes and caches an H.264 copy on first use, opt out with --no-proxy or media.autoProxy: false), and lint emits an info-level hevc_preview_codec note naming affected assets.