mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-07 10:06:21 +00:00
d0d281de50e18fab941ea817894ba83ea46d0d10
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
b9be0b2625 |
feat(skills,studio,media-use): the intent layer, review loop, and user memory — BRIEF.md, companion mode, recipes; /website-to-video folds into /product-launch-video (#2133)
* feat(studio,cli): per-frame board comments, self-refreshing storyboard, status-aware preview landing Per-frame comment boxes on the storyboard board batch into .hyperframes/frame-comments.json (a resubmit wins per frame; unconsumed comments on other frames are kept). Submitted-but-unconsumed comments stay visible — a toolbar banner plus a per-tile echo — until the agent consumes the file; the banner also says what to do next (reply anything in the agent chat). The board keeps itself current: GET /projects/:id/signature exposes the watcher-cached project signature, the storyboard payload carries the signature it was derived from, and the view polls at 2s (hidden tabs skipped, re-checked on visibility), refetching in place with no loading flash. Posters bake the signature into their URL so tiles fill in as sketches land and a poster that failed mid-write retries on the next version; the empty state upgrades itself when STORYBOARD.md appears, and its handoff prompt now points the agent at the review loop and uses the parser's real status vocabulary (outline, not planned). preview lands the browser on the storyboard view while the board is the review surface — any frame built, or pure planning (srcs declared, none on disk yet) — and on the timeline once the video is assembled. * feat(skills): the review loop — plan, sketch, build as one shared process hyperframes-core/references/review-loop.md is the single source for the three-pass collaborative review: the plan proposed on a live board (§ 1), wireframe sketches marked built with one layout question (§ 2 — real words on plain blocks, run no CLI; a confirmed board is itself a valid deliverable when the user asked for a storyboard, not a video), the build dressing confirmed layouts (§ 3, worker or inline), and the final look (§ 4). Autonomous runs skip every gate and keep one question before render. The three narrative workflows' Steps 3/4/6 collapse to references plus their sketch stand-ins (captured-asset blocks for product-launch-video, plain code panels for pr-to-video); the confirmed-sketch handoff stays in each frame-worker prompt. general-video plans on a board for multi-scene narrative pieces in collaborative mode — its sketch pass is layout-before-animation with the user watching. The router treats "I want a storyboard" as a process request rather than a route, and closes exploratory intake by recommending a route plus how the run will review. The supporting contracts land next door: the comments channel (silent submit, one reply picks it up, check the file before the words) in brief-contract § 1; the sidecar schema and the built status rung in storyboard-format; the mode question asked first and alone in the three workflows' Step 0. * feat(media-use): user memory — remembered preferences and frozen recipes Two tiers of memory on media-use's existing two-tier storage split. Preferences (lightweight): confirmed brief answers — destination, aspect, language, mode, voice, style preset — recorded to the project's .media/preferences.json (committed, the team inherits it) and promoted to the personal ~/.media/preferences.json once the same value is confirmed in two different projects (a sightings ledger accumulates the cross-project evidence user-side, since project files can't see each other). prefs.mjs get/record; merge reads project-over-user; a changed value restarts its provenance. Recipes (heavyweight): one approved run frozen as a named, versioned bundle — frame.md, the storyboard skeleton (structure kept: durations, transitions, srcs, Video direction; statuses reset to outline; content blanked to per-frame fill-ins naming the beat's role), and the confirmed brief values. Named folders, not content hashes: re-freezing bumps version and archives <name>@v<N>; a freeze is already confirmed, so it promotes to the user tier immediately. recipe.mjs freeze/list/use, plus resolve --type recipe --entity <name> delegating like grade/lut. 16 new node --test cases; the media-use lib suite is 168/168. * feat(skills): wire user memory into the brief and the review loop brief-contract § 2 gains Remembered defaults: read the merged preferences before Round 2 and let a remembered value become the recommended option with a receipt naming its source project. Memory changes the default, never the question — every ask-marked field still gets asked, and what the request says this time beats what was picked last time. Record only what the user actually confirmed (a defaulted voice nobody chose is not an answer; a "go" that accepts the recommended defaults is). The first record announces itself once; after that the receipts carry the reminder. In autonomous mode a remembered value becomes the decided value, receipt included. The three narrative workflows read the remembered defaults before Round 2, record the confirmed answers at the Step 0 gate, record the chosen preset at the Step 2 gate (pr-to-video excepted — its preset is fixed), and fall back to the remembered voice when the request names none. general-video's discovery reads the same defaults. Recipes wire in at both ends: Step 0 checks for a matching recipe before the mode question — one question, plural-aware, and adopting one fills the brief, skips the design step, and drafts the storyboard from the frozen skeleton while every review gate still runs. The review loop's final look (§ 4) offers the freeze once after approval, and the confirmation teaches the recall phrase — the name is something the system reminds the user of, never something they must remember. The router recognizes a named recipe or "like last time" as a route. * docs(skills): the sketch pass names check, not the deprecated validate * feat(skills): intent-layer references — process, route briefs, capability menu, BRIEF.md format * feat(media-use): brief skeleton as the recipe's fourth artifact; flow/storyboard preference keys * feat(skills): the intent layer conducts every brief — workflows execute BRIEF.md * feat(skills): retire the mode preference key; sync catalog surfaces for intent layer * refactor(skills): dedupe router vs intent-layer guidance — one owner per rule * feat(skills): the design ask — own spec, pick by eye from showcases, or defer * docs(skills): the design ask says the honest line on capture routes * feat(skills): product-launch-video absorbs website-to-video as the tour angle * refactor(skills): keep product-launch-video pristine — a tour is brief intent, not a pipeline branch * feat(skills): production loop + genre lenses; general-video goes freeform (route yours, laws hold) * refactor(skills): /hyperframes is the front door - route tables and scope lists leave the workflows * docs(skills): review-loop pass across skill catalog * fix(cli): pass project dir to openStudioBrowser in background-server path * feat(skills): add pitch-round reference - verbalized sampling concept gate * feat(skills): wire pitch round into intent layer - completeness triage + route eligibility * feat(skills): editorial capability recommendations, handoff disciplines, menu-probe split * feat(skills): pitches carry their machinery; source-only-formed requests pitch the telling * feat(skills): companion goes director - ceiling treatment plus blueprint/rule citation discipline * fix(scripts): sandbox npx-leak guard - private npm global prefix keeps npx on the branch CLI * chore(skills): resync manifest hash after formatter pass reflowed general-video tables * fix(skills): recipe freeze reads workflow from BRIEF.md; style_preset records require workflow scope Two holes found by a live companion-run freeze: the agent-supplied --workflow contradicted the run's actual workflow (recipe.json said faceless-explainer, brief-skeleton said general-video), and the style_preset lookup missed because the preference had been recorded under the bare key. - freezeRecipe resolves the workflow from BRIEF.md frontmatter; the flag is a fallback for briefless projects and a contradicting flag is ignored (noted). - recordPreference refuses a bare style_preset — the scoped key is the only writable shape; freeze tolerates legacy bare records via read fallback. - review-loop § 4 / media-use SKILL / brief-format wording follow the machinery. |
||
|
|
a4303137cb |
fix: storyboard-angle review follow-ups (M1 bg-on-clip, B3 slideshow, parser guard, CLI fixes) (#1791)
* fix(skills): storyboard review — bg-on-clip rule, slideshow output, parser parity guard Addresses the storyboard-angle review (jrusso1020): - M1 (invisible text): frame-worker.md (x3) + SKILL.md Step 5 (x3) now require a frame's full-bleed background on a class=clip layer, never the #root / data-composition-id element (the root is clip-gated to its scene window, so a background on it is not a dependable ground and dark text can land on the black host body). The assembler already paints frame.md's canvas onto index #root as the base ground; the per-frame clip rides on top. - B3 (slideshow truncates to slide 1): slideshow/SKILL.md gains an Output section (decks render via 'present'; 'render index.html' captures only the first composition; linear main-line MP4 export is deferred). - Parser drift: vendoredParity.test.ts guards the three vendored storyboard.mjs copies (byte-identical + parse-parity with @hyperframes/core). - skills-manifest.json regenerated for the edited SKILL.md files. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): storyboard review — lint, validate help, snapshot, inspect, capture, render Addresses the CLI findings from the storyboard-angle review (jrusso1020): - lint (@hyperframes/lint): accept vendor-prefixed system-font keywords -apple-system / BlinkMacSystemFont so a system stack with a generic fallback no longer trips font_family_without_font_face (+ test). - help: list 'validate' under Project in 'hyperframes --help' (was runnable but undocumented). - snapshot: honor -o/--output (the flag did not exist; output was hardcoded to snapshots/). The dir is resolved once and threaded through capture + contact sheet + Gemini. - snapshot: split font status into loaded / error / unused with a one-line summary; only a real 'error' is reported as FAILED (an unrequested @font-face is 'unused', not a contradiction with 'loaded'). - inspect: suppress text_occluded across a scene-to-scene crossfade (occluder in a different data-composition-id mount while a scene is mid-fade); a same-scene or two-settled-scenes overlap still flags. - inspect: suppress content_overlap between in-flow siblings governed by the same flex/grid container (tight stacks / number lockups are layout slop). - capture: record source resolution (videoWidth/Height) in video-manifest.json alongside the DOM display box; consumers size off the source dims. - render: warn when the target carries a slideshow island (render captures only the first scene, so the MP4 is truncated to slide 1; use 'present'). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
7cb8386539 |
refactor(skills): rebuild faceless-explainer + pr-to-video on the shot-sequence architecture (#1778)
* refactor(skills): rebuild faceless-explainer + pr-to-video on the shot-sequence architecture Move both skills onto the current shot-sequence authoring architecture and the latest shared engine, then re-narrate to each domain. The engine fixes (pre-assembly frame guards + BGM loop-extend in assemble-index, dark-ground caption contrast, brand-accent selection + mono role in tokens, dark-mode polarity invert / weight-clamp / icon-font filter in build-frame) had only landed in one copy; both skills were a generation behind. - Authoring model: visual-design now writes a time-coded shot sequence (Scene windows paced to the voiceover) instead of the older effects-id phased note; motion-language carries the move vocabulary + the tightened motion doctrine (smooth over bouncy) + the seek-safe core (fromTo entrances, no CSS-transition motion); add cut-catalog (within-frame velocity-matched seams); frame-worker and SKILL Step 4/5 move to blueprint instantiation + shot-sequence fidelity. - faceless-explainer: fold the standalone composition.md into visual-design (inventing-the-visual / portrait / caption geometry); keep the explainer story doctrine; graft cue-segmented VO + candidate-blueprint-from-Step-3. - pr-to-video: keep the ingest pipeline (fetch-pr / ingest / fetch-people-avatars) and code-vocabulary; preserve the code-beat (code-* block as focal, Scenes choreograph the surround) and mechanism-beat treatments under the new model. - Drop stage-assets from both (no captured assets to stage); remove derivation references so each skill reads standalone. bun run scripts/lint-skills.ts passes; all scripts node --check clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(skills): resync skills-manifest after format pass The pre-commit skills-manifest hook hashed the skills before the format hook reformatted cut-catalog.md, so the committed manifest lagged the on-disk content and CI's "Skills: manifest in sync" check failed. Regenerated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
54cab331d0 |
feat(cli): shared TTS/BGM auth preflight + caption and skill-workflow fixes (#1697)
* fix: handle caption skin workflow * docs(skills): simplify the finalize step across video workflows - Drop --strict-layout; all skills use plain `hyperframes inspect` - Add the caption text_box_overflow false-positive note to faceless-explainer - On a failed check, the orchestrator makes the cheapest safe edit itself (no worker re-dispatch / Step 3 backtrack language) - Snapshot: glance at the stitched contact-sheet.jpg and move on Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(auth): onboarding-first `auth status` + shared TTS/BGM preflight When no HeyGen credential is configured, `hyperframes auth status` now prints registration-first guidance instead of a terse error: - Interactive / agent-driven sessions get sign-in guidance led by `hyperframes auth login` (the OAuth step that also creates an account and is shared with heygen-cli), and never steer users to a per-repo `.env`. CI / non-interactive runs get a terse note. Exit 1 is kept so the "am I logged in?" `$?` contract still holds. - It probes which local engine voice/music will fall back to (Kokoro / MusicGen, mirroring the skill resolution order) and whether their Python deps are installed, with a pip hint when missing. `--json` exposes `recommended_action` + `offline_engines` for skills to branch. - `doctor` gains matching "TTS (Kokoro)" / "BGM (MusicGen)" checks via the same shared probe (findPython/hasPythonModules extracted to tts/python.ts; provider resolution in audio/providers.ts). Every TTS/BGM workflow now relays this at Step 0 (setup) instead of improvising its own "missing key" prompt: pr-to-video, product-launch- video, faceless-explainer, website-to-video, music-to-video. The canonical behavior + key-priority table live once in hyperframes-media. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pr-to-video): scale recommended video length to PR change size Step 0 led with a fixed ~60-90s length default. Now the recommended length is derived from the PR's diff stat (lines added+deleted, nudged by file count) on a tier scale (trivial ~20-40s → large ~110-180s, hard cap ~3 min), reusing the same PR peek already done to infer the angle. The agent states the basis when proposing it, and a huge PR with one headline change still stays tight. User can always override. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(captions): embed brand fonts whose files use separators brandFontFaces() matched font files by stripping only whitespace, so an underscore/hyphen-named file (TT_Norms_Pro_Bold.woff2) never matched the family key "ttnormspro" — captions shipped with no @font-face, the font_family_without_font_face bug. Now both family and filename normalize away all non-alphanumerics; families match longest-key-first so a parent family can't swallow a more specific one's files (TT Norms Pro vs Mono); each file is claimed once; "demibold" ranks before "bold"; and when nothing matches it warns loudly at build time instead of returning "". Also: parseFonts() falls back to h1/h2/title/hero display roles, and the frame-worker + caption authoring docs spell out that only shipped font files render — no system CJK/Devanagari families on the headless renderer. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(hyperframes-media): enforce sign-in preflight on standalone BGM/TTS A one-off "generate me a BGM" request went straight to local MusicGen without recommending sign-in: bgm.md/tts.md framed the no-credential path as an automatic fallback, so the generation path bypassed the Preflight stop, and the preflight used a bare `hyperframes auth status` that isn't on PATH in a fresh `npx skills` project. - Preflight now applies to one-off generation as well as workflows, uses `npx hyperframes auth status`, and says: if the CLI can't run, still recommend signing in and STOP — never treat "no credential" as a silent green light for local generation. - bgm.md and tts.md point at the Preflight before generating, reframing local generation as the fallback the user opts into, not a default. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(auth): add Authentication & API keys guide Document signing in, the keys each capability (voice, music, capture) uses, their resolution priority, and the fully local fallback. Add the guide to the nav and cross-link it from the cloud deploy note and the CLI env-var reference. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(lint): strip HTML comments in a fixpoint loop (CodeQL) Single-pass <!-- --> removal can re-form a complete comment from adjacent markers (e.g. `<<!-- -->!-- ... -->`), letting a decoy <template> survive and hijack the template-boundary match. Loop to a fixpoint, mirroring the captions.mjs precedent; add a regression test that fails on single-pass (2 root findings) and passes on the loop. Also wrap the build-frame.mjs node:fs imports to satisfy oxfmt — the new copyFileSync import pushed the line past the width limit, which was the sole cause of the Format / Preflight CI failures. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(lint): strip HTML comments with a linear scan (CodeQL ReDoS) The fixpoint loop still ran a /<!--[\s\S]*?-->/ regex per pass, which backtracks O(n^2) on inputs with many unterminated "<!--" — CodeQL js/polynomial-redos (high). Looping the same regex (the prescribed fix) never addressed this; only the regex itself does. Replace it with an indexOf-based linear strip in utils.ts (stripHtmlComments), kept in a fixpoint loop so markers that re-form when a comment is removed are still stripped. 200k unterminated "<!--" now strips in ~3ms instead of quadratic time; behavior is otherwise unchanged — unterminated comments are kept verbatim, as the old regex left them. The re-forming regression test still guards it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(auth): make TTS/BGM sign-in guidance accurate and runnable From team review of the not-signed-in onboarding: - OAuth is a `hyperframes auth login` feature only. The separate `heygen` CLI is API-key-only — `heygen auth login` stores a pasted key, it is not OAuth and does not create an account. Stop presenting the two CLIs as the same OAuth/sign-up step. - Use `npx hyperframes` in every imperative and runtime hint. Bare `hyperframes` is not on PATH on a fresh machine (command not found); only `npx hyperframes` is guaranteed. Also updates the JSON recommended_action. - Drop `heygen auth login` from the terminal/skill onboarding: it needs its own install and there is no `npx heygen`, so it was a command-not-found trap. The shared-credential fact stays in the reference docs. Covers the `auth status` guidance + tests, the Authentication docs, the shared hyperframes-media preflight (SKILL, requirements, tts, error hints), and the `npx hyperframes auth status` preflight in every TTS/BGM workflow (pr-to-video, product-launch-video, faceless-explainer, website-to-video, music-to-video). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
1967901b57 |
refactor(skills): move product-launch / pr-to-video / faceless-explainer onto the script-driven architecture (#1635)
* refactor(product-launch-video): restructure onto script-driven architecture Move product-launch-video onto the shared script-driven authoring flow: build-frame remixes a hyperframes-creative preset onto brand tokens, audio routes through the shared hyperframes-media engine, per-preset caption skins, and every frame is authored as a directed shot. Removes the old bespoke scripts (captions/validate/prep/hoist/…) in favour of the shared lib. assemble-index.mjs keeps upstream #1629's blank/partial scene-file guard (reject an empty or markup-less scene file at assembly, before emitting data-composition-src, and re-dispatch) carried onto the restructured reader. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(pr-to-video): restructure onto script-driven architecture Move pr-to-video onto the shared script-driven authoring flow: ingest.mjs folds the gh PR artifacts into the synthetic capture package the shared backend (build-frame / captions / assemble-index) reads, add the mechanism beat, route audio through hyperframes-media, and remix a hyperframes-creative preset onto brand tokens via the shared lib. - Fix skill name: pr-to-video-refactor -> pr-to-video (match directory). - Drop a stale faceless-explainer-refactor reference in an ingest.mjs comment. - assemble-index.mjs keeps upstream #1629's blank/partial scene-file guard. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(faceless-explainer): restructure onto script-driven architecture Move faceless-explainer onto the shared script-driven authoring flow: every visual is invented (typography / abstract graphics / diagram / data-viz) and authored through the shared backend (build-frame remixes a hyperframes-creative preset onto tokens, audio via hyperframes-media, assemble-index builds the standalone index.html) using the shared lib. - Fix skill name: faceless-explainer-refactor -> faceless-explainer (match directory). - assemble-index.mjs keeps upstream #1629's blank/partial scene-file guard. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(skills): refresh test-skills-fresh.sh workflow roster Update the install-and-verify harness to the current surface: 10 workflows (adds website-to-video, embedded-captions, graphic-overlays, slideshow; drops the removed footage-recut) and refreshed example prompts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(product-launch-video): oxfmt storyboard.mjs Run oxfmt over lib/storyboard.mjs — formatting only, no logic change. Fixes the Format / Preflight CI check. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(studio): import commitGsapPositionFromDrag from its actual module The function was split out into gsapDragPositionCommit.ts in #1605, but the test kept importing it from ./gsapDragCommit, which no longer exports it — yielding 'is not a function' at runtime. Import from the correct module. Inherited main breakage (same fix as #1631); fixes the Test CI check on this branch independently of merge order. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(hyperframes): refine router skill metadata tags Update the entry router's metadata tags (video / animation / router focus); oxfmt collapses the now-shorter metadata to a single line. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(skills): tighten caption comment-strip + document audio --only merge Review follow-ups (#1635): - captions.mjs (x3): the HTML-comment strip used a single global replace, which CodeQL flags as incomplete multi-character sanitization (a nested/partial pair can re-form a marker the single pass misses). Strip in a fixpoint loop instead. Input is preset-library content, not user-controlled, so this is lint- cleanliness, not XSS defense. - audio.mjs (x3): document that fetch-sfx (--only sfx) MERGES into the neutral audio_engine_meta.json sidecar — the engine reads prev and recomputes only the sfx section, so voices/bgm from the generate pass are preserved (review Q). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(skills): remove existsSync->write TOCTOU in workflow scripts Clears the 9 js/file-system-race CodeQL alerts (captions/audio/transitions x3). Each was an existsSync precheck followed by a later write of the same path: - captions.mjs: caption-overrides shim -> atomic writeFileSync({ flag: 'wx' }). - audio.mjs (sync-durations) + transitions.mjs (inject): drop the existsSync precheck and read directly, surfacing the same friendly error from a try/catch on readFileSync — no check->write gap. Behavior is unchanged (same error messages); these are local single-process deterministic scripts so the race was never a real risk, but this clears the gate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(skills): paint root composition ground color in assemble-index Per-frame roots carry data-start/data-duration and get clip-gated against the global timeline at render, so only the first frame's window overlaps global 0 — a frame's own full-bleed background can't serve as the video ground, and every frame after the first renders on the bare body color (black). Paint the ground on the always-present root composition using the project's frame.md canvas color (the same role the caption skin maps to --cap-canvas); fall back to the body letterbox color when frame.md is absent or has no resolvable ground. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(hyperframes): drop router-tag edit (moved to the foundation PR) The entry SKILL.md is rewritten wholesale by the frame-presets/media foundation PR (#1632); editing it here too guaranteed a merge conflict. Restore this file to main and let the router-tag tweak live with the rewrite in #1632, so the two PRs no longer both touch it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |