ext/hyperframes
2
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-08-31 02:41:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: document professional grading and media treatments

Browse Source
This commit is contained in:
ukimsanov
2026-07-27 18:27:20 -07:00
parent 07e63b3758
commit 1a2c30a799
4 changed files with 798 additions and 315 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/docs.json
+2
View File
@@ -79,6 +79,8 @@
"guides/authentication",
"guides/video-components",
"guides/color-grading",
"guides/media-effects",
"guides/media-overlays",
"guides/html-in-canvas",
"guides/website-to-video",
"guides/figma",
docs/guides/color-grading.mdx
+289 -315
View File
@@ -1,347 +1,223 @@
---
title: Color Grading
description: "Apply real-time color grading, presets, LUTs, finishing, and media effects to video and image media in Studio and final renders."
description: "Correct and creatively grade video or image media with presets, scopes, color wheels, curves, HSL selections, and custom LUTs."
---
HyperFrames Studio can color grade project-local `<video>` and `<img>` media directly in the preview. The same `data-color-grading` settings are used by the render pipeline, so the exported video should match the look you preview.
HyperFrames provides real-time, media-level color grading for project-local
`<video>` and `<img>` elements.
This is a lightweight media color tool for generated videos, uploaded footage, social variants, and agent-authored compositions. It is not a DaVinci Resolve, Premiere, ACES, or OCIO finishing pipeline.
Studio and AI agents use the same validated grading contract, and final
rendering runs the same shader pipeline used by the preview.
## What Is New
The current pipeline is designed for SDR/Rec.709-oriented work. It provides
professional primary and selective grading tools, but it is not an ACES, OCIO,
or native HDR finishing pipeline.
| Capability | Status | Notes |
## The Mental Model
| Stage | Purpose | HyperFrames tools |
| --- | --- | --- |
| Studio Color Grading panel | Supported | Appears on selected `<video>` and `<img>` elements. |
| Manual controls | Supported | Exposure, contrast, highlights, shadows, white point, black point, warmth, tint, vibrance, saturation. |
| Presets | Supported | Named HyperFrames presets backed by shader settings, not bundled third-party LUT packs. |
| Custom LUT upload | Supported | Project-local 3D `.cube` LUT files with strength control. |
| Finish | Supported | Vignette and grain, with advanced settings behind the settings icon. |
| Studio Effects panel | Supported | Essentials, Retro & Glitch, Print, and Art effects share the same media shader and persisted `effects` object. |
| Studio Overlays panel | Supported | Inserts selected Registry overlays at the media timing as ordinary timeline layers. |
| Before preview | Supported | Hold the compare button to temporarily show the ungraded media. |
| Render parity | Supported | The render pipeline redraws the color-grading shader after video-frame injection. |
| Correct | Fix exposure, contrast, tonal balance, and casts | Adjust controls and source analysis |
| Grade | Shape color and mood | Tonal wheels, RGB curves, hue curves, and HSL color selections |
| Apply a look | Start from a tested style or an existing external look | Presets and custom 3D `.cube` LUTs |
| Finish | Add restrained optical texture | Vignette and grain |
| Stylize | Transform the pixels beyond normal grading | [Media Effects](/guides/media-effects) |
| Dress | Add an authored HUD, flash, light leak, or freeze-frame layer | [Media Overlays](/guides/media-overlays) |
Studio labels `whites`, `blacks`, and `temperature` as White Point, Black Point, and Warmth. Use the JSON keys shown in the data shape when authoring `data-color-grading` by hand or through an agent.
## Support Matrix
| Source / workflow | Supported? | What to expect |
| --- | --- | --- |
| 1080p SDR video | Yes | Best default path. Good for most uploaded/generated MP4/WebM/MOV media that browsers can decode. |
| 4K SDR video | Yes | Works when the browser and machine can decode it. Preview/render cost is higher. A 4K source only produces 4K output when the composition/render is also 4K. |
| 1080p / 4K images | Yes | Works on normal project-local images. Output resolution follows the element/composition render size, not hidden extra detail beyond that size. |
| iPhone SDR video | Yes | Treat as normal SDR media when it is tagged/decoded as SDR. |
| iPhone HDR / HLG / Dolby Vision-style uploads | Partial | The media can be loaded if browser/FFmpeg support the file, and Studio warns when HDR metadata is detected. The live Color Grading shader is still an SDR preview path, not true HDR grading. Use the existing [HDR Rendering](/guides/hdr) pipeline for HDR delivery and verify the output. |
| HDR render output | Related, not new | HyperFrames already has HDR render support. Color Grading does not yet provide HDR-aware grading controls. |
| LOG camera footage | Partial | Sliders and LUTs can be applied, but HyperFrames does not auto-detect camera LOG profiles or apply ACES/OCIO input transforms. Use a matching conversion/look LUT if you know the source profile. |
| Rec.709 creative LUTs | Yes | Best LUT path today. Use project-local 3D `.cube` files. |
| Camera conversion LUTs | Partial | Technically accepted if they are supported 3D `.cube` files, but correctness depends on the source footage matching the LUT's expected input color space. |
| Full-scene grading including text/DOM | Not yet | Color Grading is media-only. Captions, text, SVG, and regular DOM overlays stay unchanged. |
| Face/region-only grading or privacy | Not yet | Realtime effects process the whole selected `<img>` or `<video>`. Isolate the region into a separate cropped/masked media layer or use an external segmentation/tracking tool first. |
| Remote media URLs | Partial | WebGL pixel processing requires compatible CORS headers. Project-local assets are the reliable path. |
| Professional ACES/OCIO/HDR finishing | Not yet | Future render/color-management work, not this Studio shader path. |
## How It Works
Color grading is stored on media elements as `data-color-grading`:
```html index.html
<video
id="hero-video"
src="assets/hero.mp4"
data-start="0"
data-duration="6"
muted
playsinline
data-color-grading='{
"preset":"clean-studio",
"intensity":0.85,
"adjust":{
"exposure":0.05,
"contrast":0.08,
"highlights":-0.08,
"shadows":0.06,
"vibrance":0.04,
"saturation":0.04
},
"details":{
"vignette":0.08,
"vignetteFeather":0.72,
"grain":0.12,
"grainSize":0.25,
"grainRoughness":0.55
},
"effects":{
"blur":0.08,
"chromaBleed":0,
"tapeDamage":0,
"tapeTracking":0,
"tapeNoise":1,
"tapeSpeed":0.5,
"filmArtifacts":0,
"halftone":0,
"halftoneSize":0,
"twoInkPrint":0,
"twoInkPrintSize":0,
"ascii":0,
"asciiSize":0.066,
"asciiInvert":0,
"dither":0,
"ditherSize":0.25
},
"palette":["#0b0d0d","#eee9db"],
"colorSpace":"rec709"
}'
></video>
```
The runtime creates a sibling WebGL canvas for the media element, samples the current video or image frame, applies shader uniforms, then hides the native media only after a shader frame is ready.
All pixel-level stages are stored together on the selected media element. An
overlay is different: it is an ordinary editable composition layer above the
media.
## Studio Workflow
Select a real `<img>` or `<video>` element in Studio to open the media tools:
Select a real `<img>` or `<video>` element to open Color Grading in the Design
panel.
1. **Presets** shows every built-in starting point in a responsive preview
grid. Hover to preview it on the selected media, click to apply it, or choose
**Original** to return to the neutral preset.
2. **Adjust** provides tonal and color correction.
3. **Effects** groups configurable shader controls under Essentials, Retro &
Glitch, Print, and Art.
4. **Finish** provides vignette and grain.
5. **Custom LUT** accepts a project-local 3D `.cube` file.
6. **Overlays** installs Camcorder HUD, Editorial Flash, Organic Light Leak,
or Freeze-Frame Cutout from the existing Registry.
<Steps>
<Step title="Inspect the source">
Open **Scopes** and choose Histogram, Waveform, RGB Parade, or Vectorscope.
Scopes refresh from the selected media frame and help identify clipped
highlights, crushed shadows, channel imbalance, and saturation before you
make creative changes.
</Step>
<Step title="Make the primary correction">
Use Exposure, Contrast, Highlights, Shadows, White Point, Black Point,
Warmth, Tint, Vibrance, and Saturation. Correct obvious source problems
before applying a stronger look.
</Step>
<Step title="Shape tonal color">
Use the Shadows, Midtones, and Highlights wheels. Each wheel provides hue,
color amount, and level, so color and brightness can be adjusted within the
same tonal zone.
</Step>
<Step title="Refine with curves">
Use the Master curve for overall luminance, Red/Green/Blue curves for
channel shaping, and Hue vs Hue, Hue vs Saturation, or Hue vs Luma for
hue-selective adjustments.
</Step>
<Step title="Isolate a color when needed">
Add a Color Selection, capture the current frame, and use the eyedropper to
initialize its HSL qualifier. Switch to **Selection matte** to inspect the
selected pixels, refine hue/saturation/luma range and softness, then apply
bounded hue, saturation, luma, temperature, or tint correction. Up to four
ordered selections are supported.
</Step>
<Step title="Choose a look and finish">
Preview a built-in preset on the selected media or load a project-local 3D
`.cube` LUT. Use vignette and grain only when they support the visual
direction.
</Step>
<Step title="Compare and render">
Hold the compare control to see the ungraded source. Scrub representative
frames, verify faces and highlights, then render. The final render redraws
the grading shader after exact video-frame injection.
</Step>
</Steps>
An overlay inserted from this panel starts with the selected media, uses its
duration, and is placed on the next visual track when one is known. It then
behaves like any other composition layer: edit or remove it through Layers and
Timeline. The left Catalog remains the browse-all Registry surface.
To reuse the complete authored grade, use **Copy grade to** and choose
**Current file media** or **All project media**, then click **Apply**. This
copies the validated payload to other real media elements; it does not alter
overlay, caption, or DOM layers.
<Note>
Color Grading is intentionally **media-only**. It applies to `<video>` and `<img>` sources. Captions, text, divs, SVG, and UI graphics remain ungraded unless you render them into media first.
Color Grading is media-only. It processes real `<video>` and `<img>` elements,
not captions, text, SVG, or arbitrary DOM. Use first-class media elements for
visual assets that should remain editable and gradeable.
</Note>
<Note>
Project-local media is the safest path. Remote media must be served with compatible CORS headers and should use `crossorigin="anonymous"` when pixel processing is needed.
</Note>
## What Each Professional Tool Does
## Data Shape
### Color Wheels
```json
{
"preset": "clean-studio",
"intensity": 1,
"adjust": {
"exposure": 0,
"contrast": 0,
"highlights": 0,
"shadows": 0,
"whites": 0,
"blacks": 0,
"temperature": 0,
"tint": 0,
"vibrance": 0,
"saturation": 0
},
"details": {
"vignette": 0,
"vignetteMidpoint": 0.5,
"vignetteRoundness": 0,
"vignetteFeather": 0.65,
"grain": 0,
"grainSize": 0.25,
"grainRoughness": 0.5
},
"effects": {
"blur": 0,
"pixelate": 0,
"chromaBleed": 0,
"tapeDamage": 0,
"tapeTracking": 0,
"tapeNoise": 1,
"tapeSpeed": 0.5,
"filmArtifacts": 0,
"halftone": 0,
"halftoneSize": 0,
"twoInkPrint": 0,
"twoInkPrintSize": 0,
"ascii": 0,
"asciiSize": 0.066,
"asciiInvert": 0,
"dither": 0,
"ditherSize": 0.25
},
"palette": ["#0b0d0d", "#eee9db"],
"lut": {
"src": "assets/luts/look.cube",
"intensity": 0.75
},
"colorSpace": "rec709"
}
```
The three wheels target broad tonal zones:
Omit `enabled`; the presence of `data-color-grading` implies that grading is active. All numeric controls are clamped by the runtime. The current color grading path is Rec.709/sRGB-oriented and assumes browser-decoded media frames.
- **Shadows** shapes dark regions.
- **Midtones** shapes most faces, products, and general scene color.
- **Highlights** shapes bright regions and specular areas.
`chromaBleed` is a bounded `0` to `1` treatment primitive that horizontally
softens chroma detail while preserving the center sample's luma. It is useful
inside restrained creator-camera/camcorder treatments; it is not VHS, CRT, RGB
split, tracking noise, or a complete camera emulation.
Use small amounts first. The **Level** control changes the brightness of the
same tonal zone, while hue and amount introduce color.
Most effect amounts and settings are normalized from `0` to `1`. Bloom supports
up to `3`, bloom radius uses pixels, and enum controls use the integer choices
shown in Studio:
### RGB Curves
- `tapeDamage` adds deterministic horizontal time-base instability, lower luma
bandwidth, restrained ghosting, noise, sparse dropouts, and bottom-edge head
switching. `tapeTracking` adds bounded moving horizontal tracking tears,
`tapeNoise` scales tape noise and row jitter, and `tapeSpeed` controls their
deterministic motion (`0.5` is normal speed). These three subordinate
controls do nothing without `tapeDamage`. A complete analog-tape treatment
may also use restrained `chromaBleed`, scanlines, RGB separation, and rare
row tears; none of these controls adds CRT curvature.
- `filmArtifacts` adds deterministic sparse dust and short scratches. Combine
it with existing grain, vignette, color, and seek-safe GSAP gate weave for an
8mm treatment. It does not change the frame by itself when set to `0`.
- `halftone` blends in a four-angle CMYK print screen. `halftoneSize` controls
its resolution-aware dot-cell size from fine to coarse. The channel angles
and edge response use fixed print-oriented defaults to keep authored output
consistent across agents.
- `twoInkPrint` maps warm midtones and deep/cool shadows to fixed original
vermilion and teal spot screens with a dark overprint on warm paper.
`twoInkPrintSize` controls its resolution-aware screen size. Do not combine
it with `halftone` or describe it as a named commercial print process.
- `ascii` converts the selected media to a procedural 5x7 glyph field.
`asciiSize` controls cell size and `asciiInvert` switches the light/dark ink
polarity. It uses the first and last colors from `palette`.
- `dither` applies a temporally stable ordered 4x4 Bayer dither.
`ditherSize` controls cell size and all `palette` colors participate in the
result. This is ordered dithering, not Floyd-Steinberg or another sequential
error-diffusion algorithm.
- `bloom` extracts bright pixels and runs a bounded half-resolution separable
blur. `bloomRadius` controls its radius.
- `monoScreen` provides configurable mono print shapes, angle, spread, invert,
and palette controls.
- `scanlines`, `crtCurvature`, and `chromaticAberration` provide display
geometry and channel treatments with their related settings.
- `digitalGlitch` provides deterministic line tears, blocks, displacement,
selective pixelation, channel split, opacity, and speed controls.
- `engraving`, `crosshatch`, and `kuwahara` provide stylized art treatments with
their calibrated settings. Kuwahara uses bounded half-float intermediate
targets when the browser supports them and otherwise reports unavailable.
- **Master** remaps overall luminance.
- **Red**, **Green**, and **Blue** remap individual channels.
`palette` accepts two to six exact `#RRGGBB` colors in authored order. Use
dark-to-light order for normal luminance mapping; intentionally reverse the
array for an inverted result. The runtime validates and lowercases colors but
does not reorder them. ASCII, dither, mono screen, engraving, and crosshatch use
it; a palette or subordinate setting alone does not allocate an effect. Omit it
for the family default.
Curve points are normalized `[input, output]` pairs. Endpoints are implicit,
and each curve supports up to 16 points. An S-curve increases contrast; lifting
the lower-left region raises shadows; channel curves can build split-tone or
cast-removal adjustments.
When effects are combined, HyperFrames evaluates them in one fixed,
deterministic order: source framing and multipass blur/Kuwahara preparation;
chromatic and digital glitch; primary color grading and LUT blended by global
intensity; grain and film
artifacts; mono/engraving/crosshatch/halftone/two-ink/dither/ASCII; bloom,
scanlines, vignette, and CRT display masking; then before/after comparison.
Effects cannot currently be reordered. The fixed
pipeline keeps Studio, playback, seeking, and final render behavior
predictable.
### Hue Curves
Studio exposes these controls in the selected media element's **Effects**
accordion. Agents should choose one primary intent through the `media-use`
skill, then either seed from a source-aware treatment recipe or inspect the
canonical toolbox and assemble a bespoke combination:
- **Hue vs Hue** moves a selected hue toward another hue.
- **Hue vs Saturation** changes saturation around a selected hue.
- **Hue vs Luma** changes brightness around a selected hue.
```bash
hyperframes media-treatment --capabilities --json
hyperframes media-treatment --capability kuwahara --json
```
Hue curve points are `[hueDegrees, delta]` pairs and wrap around the color
wheel.
The first command reports a concise overview of every capability family. The
focused query reports one family's or effect's controls, calibrated apply
payload, render lane, palette support, and seek-safe animation paths. Use
`--all` only for exhaustive tooling. Recipes are tested shortcuts, not the
complete allowed surface. Persist the final combined payload with the same
`hyperframes media-treatment` command; unknown keys are rejected before the
composition is changed.
### HSL Color Selections
## Custom LUTs
A color selection qualifies pixels by hue, saturation, and luma, then applies a
correction only inside that matte. This is useful for restrained tasks such as
reducing an overly saturated shirt, cooling a background color, or protecting
skin from a broad creative grade.
HyperFrames supports project-local 3D `.cube` LUT files:
HyperFrames supports up to four ordered selections. These are static
media-level qualifiers: they do not include object tracking, rotoscoping,
facial recognition, or spatial masks.
```html index.html
<img
src="assets/product.jpg"
data-color-grading='{
"lut":{"src":"assets/luts/product-pop.cube","intensity":0.7}
}'
/>
```
### Scopes
Use `.cube` LUTs when users already have a look from another editor or camera workflow.
- HyperFrames currently supports 3D `.cube` LUTs for this path.
- 3D cube LUTs up to `LUT_3D_SIZE 64` are supported.
- 1D `.cube` LUTs and mixed 1D+3D LUT files are not supported yet.
- Supported headers include common `DOMAIN_MIN` / `DOMAIN_MAX` and DaVinci/IRIDAS-style `LUT_3D_INPUT_RANGE`.
- LUTs are not universal. A LUT looks correct only when the source footage roughly matches the LUT's expected input color space.
- Rec.709 creative LUTs are the safest fit today.
- LOG/camera conversion LUTs can be used, but HyperFrames does not yet manage camera color profiles for you.
## Render Behavior
Color Grading is part of the media runtime, so render uses the same settings as Studio preview. During render, HyperFrames injects exact video frames and asks the color-grading runtime to redraw before capture.
For 4K output, use the existing [4K Rendering](/guides/4k-rendering) workflow. Color Grading can run at 4K when the composition/render surface is 4K, but a 1080p source video does not become sharper just because the final render is 4K.
Performance follows the total number of treated pixels and the selected render
lane, not only the number of media elements. Several tiled videos can be
cheaper than several overlapping full-frame videos. Blur, Bloom, and Kuwahara
use multipass rendering. When more than two full-frame multipass-treated media
elements are visible together, verify continuous playback on the target
machine and simplify or pre-render the stack if frames drop. HyperFrames does
not impose a universal hard cap because GPU and decoder capacity varies by
device.
For HDR output, use the existing [HDR Rendering](/guides/hdr) workflow. Color Grading currently warns on detected HDR media, but the grading controls themselves are not HDR-aware.
When grading a video, animate opacity on a wrapper element instead of directly on the `<video>` element. The runtime hides the native media and draws the graded result through a sibling canvas, so wrapper opacity preserves preview/render parity.
## What Belongs Where
| User wants | Use |
| Scope | Use it for |
| --- | --- |
| Make uploaded footage look cleaner | Color Grading preset + adjust controls |
| Use a look from another editor | Custom 3D `.cube` LUT |
| Add polish to a product shot | Vignette, subtle grain, contrast, vibrance |
| Blur or pixelate selected media | Effects panel |
| Add restrained camcorder chroma softness | Effects panel or `effects.chromaBleed` through an agent |
| Build an analog VHS treatment | `effects.tapeDamage` + bounded tracking/noise/speed + restrained chroma, scanline, row-tear, grain, and color settings |
| Build an 8mm home-movie treatment | `effects.filmArtifacts` + grain/vignette/color + seek-safe GSAP weave |
| Build a print/editorial treatment | `effects.halftone` + `effects.halftoneSize` |
| Build a two-spot editorial print | `effects.twoInkPrint` + `effects.twoInkPrintSize` |
| Build a terminal/editorial character treatment | `effects.ascii` + `effects.asciiSize` + an optional two-color `palette` |
| Build a restrained multi-color pixel treatment | `effects.dither` + `effects.ditherSize` + a two-to-six-color `palette` |
| Add an organic light leak | Install the finite `organic-light-leak-overlay` Registry block |
| Build a freeze-frame cutout | Existing background removal + the `freeze-frame-dressing` Registry overlay block + host GSAP |
| Make a presenter float over graphics | Existing [Remove Background](/guides/remove-background) workflow |
| Put text behind a presenter | Existing `remove-background --background-output` workflow |
| Render HDR delivery files | Existing [HDR Rendering](/guides/hdr) workflow |
| Render 4K | Existing [4K Rendering](/guides/4k-rendering) workflow |
| Remove a person and reconstruct the room | External video inpainting, not HyperFrames background removal |
| Green screen keying | Preprocess with FFmpeg `chromakey` or a future HyperFrames command |
| Grade every pixel in the full scene including captions/DOM | Future compositor or render post-process |
| Professional ACES/OCIO color pipeline | Future high-fidelity color-management pipeline |
| Histogram | Overall distribution from dark to bright |
| Waveform | Brightness by horizontal image position |
| RGB Parade | Channel balance and clipped individual channels |
| Vectorscope | Hue direction and saturation |
## Agent-Friendly Examples
Studio scopes analyze a captured selected-media frame. They are inspection
tools and do not change the grade.
For AI agents, keep the instruction declarative:
## Built-In Looks
```text
Apply a clean studio preset to assets/interview.mp4, reduce highlights slightly,
lift shadows, add subtle vignette, and keep captions ungraded.
```
HyperFrames ships tested shader-setting presets, not bundled LUT files:
Expected markup:
| Intent | Presets |
| --- | --- |
| Natural and corrective | Neutral, Warm Daylight, Clean Studio, Skin Soft, Food Pop, Night Lift |
| Editorial and tonal | Muted Editorial, Vintage Wash, Soft Boost, Bright Pop, Deep Contrast |
| Monochrome | Mono Clean, Mono Fade |
Use these as starting points and tune them for the actual source. Presets that
also activate a stylized shader treatment are documented under
[Media Effects](/guides/media-effects).
## Agent Workflow
Agents should use the CLI as the normal authoring surface. The
`data-color-grading` attribute is the persistence contract, not the first thing
an agent needs to memorize.
Users do not need to name a technical control. Requests such as _"this
interview feels too dark and cold"_ or _"polish the footage without making it
look filtered"_ route through the `media-use` skill, which inspects the source,
discovers the relevant contract, applies a deterministic payload, and verifies
the result.
<Steps>
<Step title="Discover the relevant surface">
```bash Terminal
hyperframes media-treatment --capabilities --json
hyperframes media-treatment --capability grading --json
```
The overview stays concise. Focused queries such as `wheels`, `curves`,
`hue-curves`, `secondary`, `scopes`, or `lut` return exact controls, bounds,
and examples.
</Step>
<Step title="Analyze the selected source">
```bash Terminal
hyperframes media-treatment \
--project . \
--file compositions/interview.html \
--selector '#interview' \
--analyze \
--json
```
Analysis measures a local media source and returns metadata, warnings,
diagnosis, and a bounded suggested primary-correction patch. It does not
replace visual judgment or automatically invent a creative grade.
</Step>
<Step title="Apply a validated grade">
```bash Terminal
hyperframes media-treatment \
--project . \
--file compositions/interview.html \
--selector '#interview' \
--grading '{"adjust":{"highlights":-0.08,"shadows":0.06},"wheels":{"midtones":{"hue":32,"amount":0.05}}}' \
--apply \
--json
```
Use `--dry-run` when target or scope is uncertain. Use `--clear` to remove
the complete treatment from the target.
</Step>
<Step title="Verify representative frames">
Inspect more than one frame for video, compare before and after, and render
a short draft when the grade is important to the final result.
</Step>
</Steps>
Agents should normally start from source analysis or a tested preset, then add
small, explainable adjustments. Directly inventing many unrelated curve and
secondary values is difficult to review and easy to overcook.
## Advanced Data Shape
The CLI and Studio persist the resolved grade in `data-color-grading`:
```html index.html
<video
@@ -353,26 +229,124 @@ Expected markup:
playsinline
data-color-grading='{
"preset":"clean-studio",
"intensity":0.8,
"adjust":{"highlights":-0.08,"shadows":0.08},
"details":{"vignette":0.08}
"intensity":0.85,
"adjust":{
"exposure":0.04,
"highlights":-0.08,
"shadows":0.06,
"temperature":0.03
},
"wheels":{
"shadows":{"hue":218,"amount":0.04,"level":-0.01},
"midtones":{"hue":32,"amount":0.05,"level":0.01},
"highlights":{"hue":42,"amount":0.03,"level":0}
},
"curves":{
"master":[[0,0],[0.28,0.25],[0.72,0.76],[1,1]]
},
"hueCurves":{
"hueVsSaturation":[[0,0],[28,-0.05],[52,0]]
},
"secondaries":[{
"enabled":true,
"key":{
"hue":{"center":28,"range":18,"softness":12},
"saturation":{"min":0.15,"max":0.9,"softness":0.08},
"luma":{"min":0.12,"max":0.95,"softness":0.08}
},
"correction":{"saturation":-0.04,"temperature":0.03}
}],
"details":{"vignette":0.05,"grain":0.03},
"colorSpace":"rec709"
}'
></video>
```
The contract rejects unknown keys and clamps numeric values to Core-owned
bounds. Query the current contract instead of copying bounds into agent
instructions:
```bash Terminal
hyperframes media-treatment --capability secondary --json
hyperframes media-treatment --all --json
```
Use `--all` only for exhaustive tooling or contract inspection.
## Custom LUTs
HyperFrames supports project-local 3D `.cube` LUTs:
```json
{
"lut": {
"src": "assets/luts/product-look.cube",
"intensity": 0.7
}
}
```
- 3D `.cube` LUTs up to `LUT_3D_SIZE 64` are supported.
- 1D and mixed 1D+3D `.cube` files are not supported.
- Common `DOMAIN_MIN`, `DOMAIN_MAX`, and DaVinci/IRIDAS-style
`LUT_3D_INPUT_RANGE` headers are supported.
- HyperFrames does not bundle third-party LUT packs.
- A LUT is only correct when its expected input color space matches the source.
- Rec.709 creative LUTs are the safest current workflow.
- Camera LOG conversion LUTs can be loaded, but HyperFrames does not identify
camera profiles or apply ACES/OCIO input transforms automatically.
Never paste a `.cube` body into an agent prompt. LUT files commonly contain
tens of thousands of numeric rows. Keep the file local and use metadata,
validation, and rendered comparisons to evaluate it.
## Support Matrix
| Source or workflow | Status | What to expect |
| --- | --- | --- |
| 1080p SDR video | Supported | Recommended default path |
| 4K SDR video or image | Supported | Higher preview/render cost; output resolution still follows the composition |
| iPhone SDR video | Supported | Treated as normal browser-decoded SDR media |
| iPhone HDR, HLG, or Dolby Vision-style upload | Partial | HDR metadata is detected and warned; the grading shader remains SDR |
| HDR delivery render | Separate workflow | Use [HDR Rendering](/guides/hdr); grading controls are not HDR-aware |
| LOG camera footage | Partial | Requires a known matching transform/LUT; no automatic camera profile management |
| Full-scene grade including DOM/text | Not supported | Current grading targets individual media elements |
| Face or region tracking | Not supported | Use a separate isolated media layer or external tracking/masking workflow |
| Remote media | Partial | Requires compatible CORS headers; project-local assets are reliable |
| ACES/OCIO finishing | Not supported | Outside the current browser shader pipeline |
## Render and Performance
The runtime creates a sibling WebGL canvas, uploads the current image/video
frame as a texture, applies the same grading contract used by Studio, and hides
the native source after a shader frame is ready. During final rendering,
HyperFrames injects exact video frames and waits for the grading runtime to
redraw before capture.
For 4K output, follow [4K Rendering](/guides/4k-rendering). A 1080p source does
not gain new detail merely because the composition is rendered at 4K.
When animating a graded media layer's opacity, animate a wrapper rather than the
`<video>` itself. The visible pixels are drawn by the sibling grading canvas,
so wrapper opacity keeps the media and canvas together.
Project-local media is the safest path. Remote media must provide compatible
CORS headers and should use `crossorigin="anonymous"` when pixel access is
required.
## Related Guides
<CardGroup cols={2}>
<Card title="Remove Background" icon="scissors" href="/guides/remove-background">
Create transparent video/image cutouts for presenter and product overlays.
<Card title="Media Effects" icon="wand-magic-sparkles" href="/guides/media-effects">
Apply and animate shader-based optical, retro, print, and art treatments.
</Card>
<Card title="Rendering" icon="film" href="/guides/rendering#transparent-background">
Render transparent overlays and final MP4/WebM/MOV outputs.
<Card title="Media Overlays" icon="layer-group" href="/guides/media-overlays">
Add editable HUD, flash, light-leak, and freeze-frame composition layers.
</Card>
<Card title="HDR Rendering" icon="sun" href="/guides/hdr">
Render HDR10 outputs when your project uses HDR video or image sources.
Render HDR10 outputs and understand the boundary with SDR grading.
</Card>
<Card title="4K Rendering" icon="up-right-and-down-left-from-center" href="/guides/4k-rendering">
Render at 4K and understand what supersampling does and does not improve.
Render at 4K and understand source-versus-output resolution.
</Card>
</CardGroup>
docs/guides/media-effects.mdx
+320
View File
@@ -0,0 +1,320 @@
---
title: Media Effects
description: "Apply deterministic shader effects to video and image media, combine them with color grading, and animate supported strengths with GSAP."
---
Media Effects transform the pixels of a selected `<video>` or `<img>` beyond
normal color correction.
Blur, pixelation, VHS damage, ASCII, dither, halftone, engraving, crosshatch,
and Kuwahara painting all run inside the same media shader used by
[Color Grading](/guides/color-grading). Effects can be combined with correction,
color grading, finishing, and a custom LUT in one validated payload.
[Media Overlays](/guides/media-overlays) are separate editable composition
layers.
## Studio Workflow
<Steps>
<Step title="Select real media">
Select a project-local `<img>` or `<video>` in the preview, Layers panel, or
Timeline. Effects do not appear for arbitrary divs or CSS background images.
</Step>
<Step title="Choose a starting point">
Preview an effect-bearing preset or open the Effects section. A preset is a
tested starting payload; it does not restrict further adjustment.
</Step>
<Step title="Tune the relevant controls">
Effects are grouped by intent: Essentials, Retro & Glitch, Print, and Art.
Enabling an effect uses a calibrated default, while its related controls
remain adjustable.
</Step>
<Step title="Choose a palette when supported">
ASCII, Ordered Dither, Mono Screen, Engraving, and Crosshatch can use an
authored two-to-six-color palette.
</Step>
<Step title="Verify motion and framing">
Play and scrub videos. Check at least the beginning, middle, and end when an
effect moves over time. Confirm that object-fit/object-position framing
remains correct.
</Step>
</Steps>
## Effect Families
### Essentials
| Effect | What it does |
| --- | --- |
| Blur | Defocuses or softens the complete media layer |
| Pixelate | Converts the layer into a block mosaic |
| Bloom | Adds thresholded glow around bright regions |
### Retro & Glitch
| Effect | What it does |
| --- | --- |
| Chroma Softening | Smears chroma while retaining central luma |
| Tape Damage | Adds deterministic tracking errors, noise, ghosting, and dropouts |
| Film Artifacts | Adds deterministic dust and short scratches |
| Scanlines | Adds configurable horizontal display lines |
| CRT Curvature | Warps media toward curved display geometry |
| Channel Separation | Offsets color channels along an angle |
| Digital Glitch | Combines line tears, blocks, displacement, pixelation, and channel split |
### Print
| Effect | What it does |
| --- | --- |
| Halftone | Renders source color through a print-dot raster |
| Two-Ink Print | Reduces media to the built-in two-ink treatment |
| Ordered Dither | Quantizes media into an ordered limited palette |
| Mono Screen | Builds monochrome dot, shape, or line artwork |
### Art
| Effect | What it does |
| --- | --- |
| ASCII | Renders media as configurable procedural glyph cells |
| Engraving | Translates luminance into directional engraved lines |
| Crosshatch | Translates media into layered hand-hatched lines |
| Kuwahara Paint | Applies edge-preserving painterly smoothing |
## Effect-Bearing Presets
`Creator Camcorder`, `VHS Playback`, `8mm Home Movie`, `Editorial Halftone`, and
`Two-Ink Print` provide tested combinations of effect, correction, and
finishing settings. Preview one as a starting point, then tune the underlying
controls. They are normal shader payloads, not baked media or bundled LUTs.
## Agent Workflow
Agents should discover only the part of the toolbox relevant to the user's
intent:
The user may simply ask for _"an old home-video feel"_, _"a useful privacy
reveal"_, or _"something more graphic for this poster."_ The `media-use` skill
classifies that intent, then queries the narrow effect or recipe details instead
of loading the exhaustive contract.
```bash Terminal
hyperframes media-treatment --capabilities --json
hyperframes media-treatment --capability retro-glitch --json
hyperframes media-treatment --capability digitalGlitch --json
```
The concise overview lists capability families. A focused query returns the
effect's calibrated apply payload, controls, render lane, palette support, and
seek-safe animation path. Use the canonical payload instead of guessing
sub-control defaults.
Apply a complete, validated combination in one mutation:
```bash Terminal
hyperframes media-treatment \
--project . \
--file compositions/scene.html \
--selector '#hero' \
--grading '{
"adjust":{"contrast":0.05,"highlights":-0.06},
"effects":{
"tapeDamage":0.65,
"tapeTracking":0.55,
"tapeNoise":0.25,
"tapeSpeed":0.5,
"scanlines":0.2
},
"details":{"grain":0.08,"vignette":0.06}
}' \
--apply \
--json
```
Recipes are tested shortcuts, not a closed list. An agent may assemble a custom
payload when the source and intent justify it, but it should:
1. Choose one primary visual intent.
2. Query the exact effect or family contract.
3. Add only controls that visibly support that intent.
4. Avoid stacking several dominant stylizations without a reason.
5. Verify representative frames and a short render.
## Palettes
`palette` accepts two to six exact `#RRGGBB` colors in authored order.
Dark-to-light order creates normal luminance mapping; reversing the array
intentionally inverts that mapping.
```json
{
"effects": {
"dither": 1,
"ditherSize": 0.5
},
"palette": ["#080717", "#3c185f", "#d9339f", "#ff6b66", "#aafae0"]
}
```
Compatible effects are ASCII, Ordered Dither, Mono Screen, Engraving, and
Crosshatch. A palette alone does not activate an effect.
Discover built-in palettes without copying their values into instructions:
```bash Terminal
hyperframes media-treatment --capability palettes --json
hyperframes media-treatment --capability electric-ink --json
```
## Animation and Keyframes
The following paths have seek-safe CSS custom properties and may be animated by
a paused, registered GSAP timeline:
- Global treatment intensity
- LUT intensity
- Exposure
- Blur
- Bloom
- Kuwahara Paint
- Pixelate
- ASCII
- Ordered Dither
Example blur-to-focus reveal:
```html index.html
<div
class="clip"
data-composition-id="media-effect-demo"
data-start="0"
data-duration="4"
>
<video
id="hero"
src="assets/hero.mp4"
muted
playsinline
style="--hf-color-grading-blur: 0.8"
data-color-grading='{"effects":{"blur":0.8}}'
></video>
</div>
<script>
const tl = gsap.timeline({ paused: true });
tl.to(
"#hero",
{
"--hf-color-grading-blur": 0,
duration: 1.2,
ease: "power2.out",
},
0,
);
window.__timelines = window.__timelines || {};
window.__timelines["media-effect-demo"] = tl;
</script>
```
Author the initial custom-property value inline. Do not use timers, unseeded
randomness, frame-zero `set()` calls, or `onUpdate` callbacks. Query an effect's
focused capability to get its exact animation property and range.
Tape damage, digital glitch, film artifacts, and similar effects may contain
deterministic internal motion even when their overall strength is not one of
the keyframeable paths above.
## Important Effect Behavior
- **Tape Damage** is the primary analog-tape amount. Tracking controls moving
tears, Noise controls row jitter/noise, and Speed controls their deterministic
motion. Those subordinate controls do not activate tape damage by themselves.
- **Film Artifacts** adds sparse deterministic dust and scratches. Grain,
vignette, color, and optional seek-safe gate weave are separate choices.
- **Halftone** uses fixed print-oriented channel angles and edge behavior.
Cell size remains adjustable.
- **Two-Ink Print** is HyperFrames' own fixed two-ink mapping. Do not describe
it as a named commercial print process or stack it with Halftone by default.
- **ASCII** uses procedural glyph cells. Size, style, ink behavior, rotation,
and a compatible palette remain configurable.
- **Ordered Dither** uses a temporally stable 4x4 Bayer matrix. It is not
Floyd-Steinberg or another sequential error-diffusion algorithm.
- **Bloom** extracts bright regions before blurring them; it is different from
raising highlights or whites.
- **Kuwahara Paint** smooths regions while preserving major edges. Radius,
sharpness, and saturation shape the result.
## Deterministic Pipeline Order
HyperFrames evaluates combined treatment stages in one fixed order:
1. Source framing and multipass Blur/Kuwahara preparation
2. Chromatic and digital-glitch transforms
3. Primary correction, color grading, and LUT blended by global intensity
4. Grain and film artifacts
5. Mono, engraving, crosshatch, halftone, two-ink, dither, and ASCII
6. Bloom, scanlines, vignette, and CRT display masking
7. Before/after comparison
The order cannot currently be rearranged. A fixed order keeps Studio,
playback, seeking, and final rendering deterministic and prevents agents from
inventing incompatible effect graphs.
## Performance
Performance follows the number of treated pixels and the selected render lane,
not only the number of elements.
- Blur, Bloom, and Kuwahara use multipass rendering.
- Several tiled media elements can be cheaper than several overlapping
full-frame elements.
- When more than two full-frame multipass-treated elements are visible
together, verify continuous playback on the target machine.
- Simplify or pre-render a stack when preview frames drop.
- Kuwahara uses bounded half-float intermediate targets when supported and
otherwise reports itself unavailable.
For 4K delivery, verify the effect at the final composition size. Resolution-
aware effects preserve their intended scale, but 4K still processes more pixels
than 1080p.
## Low-Level Data Shape
Media effects live in the same persisted contract as grading:
```html index.html
<img
id="poster"
src="assets/poster.jpg"
data-color-grading='{
"effects":{
"ascii":1,
"asciiSize":0.066,
"asciiStyle":0,
"asciiColor":1,
"asciiRotation":0
},
"palette":["#001100","#00ff00"],
"colorSpace":"rec709"
}'
/>
```
Unknown keys are rejected and values are normalized by the Core contract. The
CLI should remain the primary authoring surface for agents.
## Related Guides
<CardGroup cols={2}>
<Card title="Color Grading" icon="palette" href="/guides/color-grading">
Correct and grade media with scopes, wheels, curves, selections, and LUTs.
</Card>
<Card title="Media Overlays" icon="layer-group" href="/guides/media-overlays">
Add authored HUD, flash, light-leak, and freeze-frame layers.
</Card>
<Card title="Keyframes" icon="diamond" href="/guides/keyframes">
Edit and verify deterministic GSAP animation in Studio.
</Card>
<Card title="Performance" icon="gauge-high" href="/guides/performance">
Diagnose source resolution, browser, decoder, and composition cost.
</Card>
</CardGroup>
docs/guides/media-overlays.mdx
+187
View File
@@ -0,0 +1,187 @@
---
title: Media Overlays
description: "Add editable, timeline-driven HUD, flash, light-leak, and freeze-frame dressing above video or image media."
---
Media Overlays are authored HyperFrames composition blocks that sit above a
video or image.
They add visual dressing such as a recording HUD, camera flash, light leak, or
freeze-frame treatment without baking those pixels into the source media.
Unlike [Color Grading](/guides/color-grading) and
[Media Effects](/guides/media-effects), overlays are not shader properties.
They are real HTML, CSS, and paused GSAP timelines installed from the
HyperFrames Registry. Once inserted, they remain normal editable layers in the
composition.
## Shipped Overlays
| Registry block | Purpose |
| --- | --- |
| `camcorder-hud` | Responsive REC indicator, battery, editable date placeholder, and timeline-driven counter |
| `editorial-flash-overlay` | Finite neutral-warm light layers for a camera-flash cut or social reveal |
| `organic-light-leak-overlay` | Finite CSS light leak for memory beats and motivated transitions |
| `freeze-frame-dressing` | Timeline-driven paper, tape, and flash dressing for a freeze frame or background-removed subject |
These are first-party authored blocks. Their Catalog poster/video assets are
previews only; the installed result is editable composition source.
## Studio Workflow
<Steps>
<Step title="Select the media beat">
Select the `<video>` or `<img>` the overlay should accompany.
</Step>
<Step title="Open Overlays">
The Design panel shows responsive preview cards for media-treatment overlay
blocks from the Registry.
</Step>
<Step title="Insert a block">
Choose an overlay. Studio installs its composition source, aligns it to the
selected media's start and duration, and places it on the next known visual
track.
</Step>
<Step title="Edit it as a normal layer">
Use Layers, Timeline, Design, or Code to change text, timing, color,
opacity, animation, or placement. Removing the overlay does not alter the
underlying media treatment.
</Step>
<Step title="Verify the complete beat">
Play and scrub through the overlay's entrance, hold, and exit. Check the
final render because the overlay, source media, and shader treatment are
separate compositing layers.
</Step>
</Steps>
The left Catalog remains the browse-all Registry surface. The Overlays section
is the media-focused shortcut for the selected beat.
## Agent Workflow
An agent may combine grading, a shader effect, treatment animation, and an
overlay when they support one coherent intent. It should not treat them as
mutually exclusive.
For example, a restrained camcorder treatment can combine:
1. A small source correction.
2. Chroma softening and tape damage on the media.
3. The `camcorder-hud` Registry block above it.
4. A finite reveal or exit timed to the edit.
Discover overlays through the media-treatment overview:
```bash Terminal
hyperframes media-treatment --capabilities --json
```
Install one through the Registry:
```bash Terminal
hyperframes add camcorder-hud --no-clipboard
hyperframes add organic-light-leak-overlay --no-clipboard
```
The command installs composition source under the project. The agent then
embeds and times that sub-composition according to the host composition's
normal rules.
Agents should:
- Use an overlay when it adds information, motivated light, or useful visual
language.
- Keep text and HUD fields editable.
- Align finite overlays to a deliberate beat rather than leaving them active
for the entire video.
- Avoid adding decorative overlays automatically to accuracy-sensitive,
brand-sensitive, or deliberately clean footage.
- Verify that the overlay does not obscure captions, faces, product UI, or
essential content.
## Choosing the Right Overlay
### Camcorder HUD
Use for creator-camera, home-video, recording, or found-footage language. The
counter is timeline-driven and the REC behavior is deterministic. Edit the
date/label placeholders instead of rendering arbitrary timestamp text into the
source media.
### Editorial Flash
Use as a short cut accent or social reveal. It is finite by design and should
land on a motivated edit, capture, pose, or beat. It is not a permanent glow or
a replacement for shader Bloom.
### Organic Light Leak
Use sparingly for memory, warmth, film-language transitions, or a motivated
light event. It is a CSS/GSAP overlay rather than a LUT and does not permanently
change source color.
### Freeze-Frame Dressing
Use with a held frame or a subject cutout. The block provides paper, tape, and
flash dressing; it does not perform background removal itself.
For a transparent subject:
```bash Terminal
hyperframes remove-background subject.mp4 -o subject.webm
hyperframes add freeze-frame-dressing --no-clipboard
```
See [Remove Background](/guides/remove-background) for cutout and background
plate behavior.
## How Overlays Combine with Pixel Treatments
The compositing model is:
```text
source <img>/<video>
-> color correction and grading
-> LUT and shader effects
-> visible media canvas
-> HTML/CSS/GSAP overlay layers above it
-> captions and other composition layers
```
Color grading does not recolor overlay text or graphics. This is intentional:
HUD labels remain crisp, captions remain readable, and an organic light leak
can be adjusted independently from the source grade.
If an overlay should appear behind a subject, separate the subject into its own
transparent media layer and place the overlay between the background and
foreground layers.
## Editing and Reuse
Registry overlays are copied into the project rather than fetched at render
time. This keeps rendering deterministic and lets teams:
- Change the HTML and CSS.
- Replace labels and colors.
- Retune the paused GSAP timeline.
- Reuse the block across multiple compositions.
- Remove it without changing the media's `data-color-grading` payload.
Keep host-specific timing and placement in the host composition when possible.
That preserves the Registry block as a reusable visual unit.
## Related Guides
<CardGroup cols={2}>
<Card title="Color Grading" icon="palette" href="/guides/color-grading">
Correct and grade source media before adding visual dressing.
</Card>
<Card title="Media Effects" icon="wand-magic-sparkles" href="/guides/media-effects">
Apply shader-based optical, retro, print, and art treatments.
</Card>
<Card title="Remove Background" icon="scissors" href="/guides/remove-background">
Create transparent subject layers and paired background plates.
</Card>
<Card title="Keyframes" icon="diamond" href="/guides/keyframes">
Edit deterministic overlay and host animation in Studio.
</Card>
</CardGroup>