mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-10-30 09:16:55 +00:00
sync/hyperframes-codegen-b6505f63
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Miguel Ángel
|
5cbef29d3a |
feat(cli): transcribe with Parakeet on macOS, Linux and Windows after an opt-in install (#4583)
* feat(cli): download Parakeet with models install parakeet and transcribe with it `hyperframes models install parakeet` installs the sherpa-onnx runtime from npm and the Parakeet TDT 0.6B v3 int8 model from Hugging Face, pinned to a revision. Every file is checked against its pinned size and sha256 under a temp name before it is renamed into place. Re-running it is a no-op. On Linux with glibc older than 2.32 it refuses up front. `transcribe --engine auto|parakeet` now prefers that engine when it is installed, then an installed parakeet-mlx runner, then whisper. Audio is cut into about 60 s windows at the quietest 100 ms near each cut, and decoding runs in a child process. If the child cannot load the runtime or crashes, transcribe falls back with one stderr line saying why. The JSON output and transcript.json shape are unchanged. * fix(skills): report the real word count and engine from transcribe fallbacks The media-use transcribe script read `words` from the CLI's transcript, which is a flat array, so its whisper path always reported an undefined word count. The embedded-captions script labelled every CLI transcript whisper.cpp even when `--engine auto` ran Parakeet; it now uses the engine the CLI reports. * fix(cli): pin the sherpa-onnx native package to the runtime's exact version * test(cli): type the sherpa install test doubles * fix(cli): transcribe audio with no speech as 0 words instead of failing Parakeet writes an empty word list for a music bed, a silent intro or a very short clip, and loading it failed with "Unrecognized JSON transcript format". An empty list is now read as a transcript with no words, as whisper's empty output already is. A non-empty list still has to look like words. * fix(cli): re-decode a Parakeet window that skipped the start of its speech At some window lengths the model silently drops the first clause: on a looped speech clip, 4 of 167 lengths between 55 and 60 s, and a 61.91 s window, lost their first 9 words. A window with a token gap of at least 1 s that is as loud as speech is decoded again with 0.5 s of leading silence, and the result with more tokens wins. Clean windows are unchanged. Decoding with 1.5 s of overlap before each cut was measured too and was worse on 573 s of audio (2.62% WER against 0.98%), so it is not used. Empty audio no longer makes a zero-length window, which aborted the decoder. * fix(cli): stop Parakeet on Ctrl-C and name the repair when its install breaks - Ctrl-C during a Parakeet transcribe now stops with exit 130 instead of starting whisper, and a SIGTERM to the CLI also stops its decode child. - Ctrl-C during models install parakeet aborts the download and the npm install, keeps no partial file, and exits 130. A staging dir left by a killed install is swept by the next run. - When Parakeet fails under --engine auto, the warning leads with the Parakeet error and names hyperframes models install parakeet. If the fallback fails too, the Parakeet error is reported rather than whisper_unavailable, so --optional no longer hides a broken install. - An explicit --engine parakeet no longer falls back: it fails with the Parakeet error and the repair command. - The install refuses up front on systems with no native package (naming the six supported ones) and on musl, instead of a raw npm error. - 24-bit and float WAV go through ffmpeg, since sherpa-onnx reads only 16-bit PCM. The decode timeout scales with the audio length. * docs(cli): point Parakeet setup at models install parakeet The transcribe docs and the media-use references named parakeet-mlx as the only way to get Parakeet. They now name hyperframes models install parakeet, and the CLI reference gets a models section. * test(cli): cover the npm cancel signal and decode timeout, and fix a doubled period The Parakeet failure line printed "failed.. To repair" when the decoder's own message ended in a period. Tests now also pin that Ctrl-C reaches the npm install and that the decode timeout scales with the audio length. * fix(cli): report unreadable audio as an input error, not a Parakeet failure Prepare the WAV once before the Parakeet fallback and hand the same file to whisper, so ffmpeg runs once. Ctrl-C while ffmpeg runs now ends as a cancel (exit 130) instead of starting whisper. * fix(cli): recover a clause Parakeet drops after a pause by decoding the gap alone Each loud gap without tokens is decoded on its own with 0.5 s either side, and only its new tokens are spliced in. The padded whole-window retry stays as the fallback. A window with no tokens at all is no longer decoded twice. * perf(cli): size the decode timeout from the WAV header instead of reading the file * docs(cli): say a broken Parakeet install still fails transcribe --optional * fix(cli): treat Ctrl-C during audio preparation as a cancel and drop the partial WAV ffmpeg traps SIGINT and exits 255 rather than dying by the signal, so that status now counts as a cancel unless the timeout fired. A failed or stopped ffmpeg removes the temp WAV it was writing. The decode timeout now sizes a prepared WAV from its file size, so a large header can no longer hide its length. * fix(cli): splice a re-decoded gap by whole words so its neighbours are never doubled The gap decode splits edge words into other tokens than the window decode did. Match neighbours as whole words within 0.3 s of each gap edge, add only whole words that start inside the gap, and decode the gap behind 0.5 s of silence like the padded retry. * fix(cli): count a cancel only when ffmpeg says it received a signal ffmpeg 7 and later also exit 255 when a file is not permitted (EPERM), so exit 255 alone read a permission error as a cancel. ffmpeg's stderr is now captured, a cancel needs its received-signal line, and any other failure reports ffmpeg's last line. The output path is passed to the shared runner explicitly. * fix(cli): keep a spliced gap word that has no leading space The filter dropped such a word; it is now kept and given its word boundary. * refactor(cli): trim three long comments in the Parakeet decoder and installer * fix(cli): repair a Parakeet runtime missing its native binary, and pick Parakeet by language The runtime dir is per platform and arch. Installed now needs the runtime manifest and the native sherpa-onnx.node; a half-installed dir is removed before npm runs, since install() keeps any dir holding the runtime manifest. The decode worker flattens a multi-line loader error onto its prefixed line and stops between windows once orphaned. transcribe opens its cancellation scope before audio prep, so Ctrl-C under --json cleans up. auto uses Parakeet only for its 25 languages and only where the host can run it; --engine parakeet with another language fails. * fix(cli): sweep reused-pid staging, name failed model downloads, keep ffmpeg's cause sweepStaleStaging also removes a staging dir older than 6 hours whose pid is alive, since containers reuse pids. A failed model download names the file, the host and a retry. runFfmpeg keeps ffmpeg's stderr on its error, so transcribe shows "Output file does not contain any stream" for a video without audio, and reads a cancel only from ffmpeg's own signal line. * fix(skills): make media-use --engine whisper run whisper and report the engine that ran The whisper path now passes --engine whisper when asked, or --engine auto, and reads the engine from the CLI's --json line, so a CLI Parakeet run is no longer labelled whisper. Docs state the language rule, the real repair behaviour and the platforms Parakeet runs on. * test(cli): pin that a long Parakeet decoder error is kept whole up to a bound * fix(cli): report a broken Parakeet runtime with the repair instead of skipping it * refactor(cli): name the staging age limit instead of commenting it * fix(skills): let media-use --engine parakeet use the CLI Parakeet when parakeet-mlx is absent * fix(cli): repair a Parakeet runtime that fails to load, and read Ctrl-C after whisper models install parakeet now loads the runtime in a child process, the way the decode worker does, and reinstalls it when that fails, so a missing shared library, a corrupt binary or missing runtime JS is repaired. It reports changed: false when the runtime loads. transcribe keeps its cheap manifest check. After a failure, transcribe lets the event loop turn once before reading the Ctrl-C scope, since a signal that landed during the synchronous whisper fallback is only handled then. * fix(cli): read Ctrl-C from the whisper child, and re-check the Parakeet runtime after a repair A whisper fallback killed by SIGINT or SIGTERM is a cancel (exit 130), the same rule decode and runFfmpeg use; the loop-turn wait is gone. The runtime probe treats a child stopped by Ctrl-C as a cancel and keeps the runtime; a reinstall is probed again and fails with the loader's error if it still does not load. media-use keeps the CLI's last lines, stdout included, when hyperframes transcribe fails. * fix(cli): give the user-stop rule one owner, SIGHUP included, and end a cancelled install at 130 stoppedByCancelSignal in renderCancellation.ts uses the scope's own signal list (SIGINT, SIGTERM, SIGHUP) and is false whenever Node stopped the child (a code, killed, or a spawn error). decode, runFfmpeg, the runtime probe and the whisper fallback all route through it, so a probe killed by SIGHUP keeps the runtime. isWhisperTimeoutError now keys on ETIMEDOUT only, which Node sets on execFileSync's own timeout, so a SIGTERM reaches the cancel rule. A cancelled models install sets exit 130 and returns instead of failing, and the probe says what went wrong (timed out, crashed, no output). * refactor(cli): let the stop-rule helper name carry its meaning |
||
|
Miguel Ángel
|
f24ff83834 |
feat(catalog): add the seven 3D-motion pieces as installable registry blocks (#4056)
* feat(catalog): add the seven 3D-motion pieces as installable registry blocks Each piece ships as a registry block with a CDN poster and preview, a live catalog detail page and its source. Scripts travel inside the catalog payload (shared vendor libraries fetched once, item scripts inline, the one HDR as a data URI) because the docs host only serves fonts, images and JSON. NO_ESSAY_COMMENTS_OK: the long comment blocks are in vendored libraries and the ported frost source under registry/blocks, not hand-written here. HF_MAX_NONLFS_KB=6000: four hosted texture PNGs under docs/public/catalog/items are runtime assets the docs host must serve as files. fallow skipped locally: it audits against origin/main and this branch is stacked on an older main. * chore(catalog): regenerate gallery data and nav after rebasing onto main * fix(catalog): read the install manifest through the bounded project reader * fix(catalog): a script the inliner missed fails the payload instead of shipping a dead URL * refactor(catalog): the leftover-script check reuses the reference finder * fix(catalog): frost and glass textures load from the CDN and the iteration presets are gone * chore(catalog): regenerate the frost and glass payloads and nav * fix(catalog): frost and glass textures are hosted through the registry url field, not hardcoded * chore(catalog): regenerate the frost and glass payloads against the hosted texture URLs * fix(catalog): the hosted-reference rewrite refuses a bare name it cannot safely replace * ci(catalog): verify changed catalog payloads in headless Chrome and check the docs catalog * feat(catalog): every item page follows the shared layout with a Tune panel and tabs * fix(catalog): let the texture-mask item host its masks so its tile plays live * chore(catalog): regenerate item pages, gallery data and the texture-mask payload * chore(catalog): regenerate item pages after merging main * fix(catalog): declared-only variables get the Tune panel, Code and Docs tabs capped * feat(catalog): give the blend-difference caption a demo scene so its tile plays live * chore(catalog): live payload for the blend-difference caption * perf(catalog): heavy 3d pieces compile and draw before ready, not on the first visible frame * feat(catalog): the avatar promo card hosts its videos, audio and fonts on the CDN * feat(catalog): the avatar promo card tile plays live from its CDN-hosted media * chore(catalog): the avatar promo card plays live, no allowlist left * chore(catalog): shorten two comment blocks * chore(catalog): trim first-party comment blocks to the comment-check limits * chore(catalog): shorten two frost source comments * fix(catalog): the Docs tab renders as prose again, and the detail page honours reduced motion Frame-mode pages carry no prose wrapper, and the detail component sat inside not-prose, so the Docs slot's headings, paragraphs, links and lists had no styling at all. The slots now carry their own prose classes and the chrome around them keeps not-prose. The recorded clip also gets the reduced-motion guard the docs-snippet check requires, and the generator reads stability from the source metadata type so the scripts typecheck passes. * fix(catalog): docs-tab paragraphs render as blocks Mintlify emits paragraphs as inline spans, so two paragraphs in the Docs slot ran together. * fix(ci): drop the duplicate docs_catalog job output git's line-based auto-merge silently duplicated the outputs key when main and this branch each added it on non-adjacent lines during the rebase; oxfmt's yaml parser rejects the repeated map key. * fix(catalog): close four CodeQL alerts in the frost source * chore(catalog): allowlist the avatar-promo-card CDN asset deletions, format sync-docs-catalog check-no-main-deletions.mjs was rejecting 11 local mp4/font files this branch already deleted in favor of CDN hosting; scripts/sync-docs-catalog.mjs had one line over the 100-col print width from an earlier edit in this branch. * fix(catalog): detail tabs use docs prose spacing, Install first; previews skip no-WebGPU runners * fix(catalog): the docs tab scrolls with the page instead of a capped inner box * refactor(catalog): split the payload asset and verify scripts into single-purpose helpers * refactor(catalog): split the preview generator into single-purpose helpers * ci(catalog): fail when the committed catalog differs from generator output The docs host serves the committed catalog and nothing regenerates it, so a stale copy shipped silently. The catalog payloads job now generates into a temp dir and compares it with the committed tree. * chore(catalog): regenerate docs/public/catalog with the refactored scripts One full regeneration from an empty output tree with the current scripts. The tree grows from 58.4 MB to 65.1 MB. Two hosted assets are about 2 MB each (an avatar png and a wav); the same bytes are already committed under assets/, so HF_MAX_NONLFS_KB is raised for this commit only. * fix(catalog): run the catalog script tests under the runner they are written for The rebase put three node:test files in the vitest half of test:scripts, where they report "No test suite found". They now sit in the node --test list. * fix(catalog): a WebGPU item without an adapter shows its recorded clip and says why * ci(catalog): the drift check runs from a committed fetch mirror with no network Generation now serves every font and CDN script fetch from scripts/catalog-fetch-mirror and fails when a fetch is not in it, even if the caller swallowed the error. CATALOG_FETCH_MIRROR=record refreshes the mirror from the live network. The docs_catalog filter also watches packages/core, packages/producer and the mirror, and the 13 re-hashed font files are listed in ALLOWED_DELETIONS. * chore(catalog): clean up the temp font cache and sort the mirror index without a locale * fix(catalog): hide Tune when the clip stands in, time out the adapter request, share one check * fix(catalog): the adapter probe survives a sync throw and a hang; test the clip guard * fix(catalog): keep the adapter probe inside the component, Mintlify evaluates only exports * fix(catalog): run the detail and page generator tests in CI; no Tune flash while probing WebGPU * docs(catalog): credit vendored libraries and make each SKILL.md list what the block installs CREDITS.md names every vendored dependency with its upstream and licence, and flags the two leaf textures whose origin is still to be confirmed. Each SKILL.md Files table now comes from the block manifest and its install and render paths point at compositions/<name>/. A test checks that every path a SKILL.md names is a manifest file that exists. * fix(catalog): a declared WebGPU item passes verify by its manifest tag, not by console wording verify-catalog-payloads forgave a missing adapter only by matching the markup and the message, which library-reported errors and widgets never satisfy. The webgpu tag in the item manifest is now the owner: a tagged item ignores its page and console errors, an untagged one gets no forgiveness, and a marker payload with no html is listed instead of crashing the run. * docs(catalog): correct the credits per block and guard the mount and render paths CREDITS.md states the three.js version and vendored-or-CDN status per block, adds d3-delaunay, Clipper on the CDN and the Google Fonts loads, and lists every texture with no known origin. The SKILL.md test now also checks the mount and render paths against the manifest composition. * test(catalog): read each block manifest once in the SKILL.md file test The two checks shared a copied manifest read, which fallow flagged as a clone with a high score. * fix(catalog): commit the hosted item files a gitignore rule kept out, and check git not the disk The compositions/ and *-ad/ ignore rules matched folder names under docs/public/catalog/items, so 33 hosted fonts, textures and marks were generated but never committed and would 404 on the docs host. The rules now have a negation for that tree and the files are added. The drift check read the checkout, where git-ignored files still exist, so it passed locally and failed only in CI. It now unpacks HEAD with git archive and compares that. * fix(catalog): a tile keeps its poster until the composition has painted * fix(catalog): a tagged WebGPU item forgives only the adapter-absent error, and test the wiring The manifest webgpu tag now decides whether an item may lack an adapter, and the error wording decides which lines are that absence. Any other page or console error on the item, and every failed request, still fails. The per-item decision is one exported function the tests drive, marker payloads are parsed by a tested helper, and the SKILL.md variable counts are checked against their tables (cuboid-carousel said forty-nine for forty). * fix(catalog): the pinned group's posters load eagerly at high priority * fix(catalog): type the tile reveal test so the scripts typecheck passes * fix(catalog): stop the adapter signature matching an unrelated failed adapter request failed to request .*adapter also matched errors about other adapters. It now matches only a request for the WebGPU adapter, so a tagged item cannot hide an unrelated failure behind it. * test(catalog): a tagged item still fails on an error about some other adapter * feat(catalog): the install command sits above the preview, as on the live catalog * fix(catalog): find the install slot through Mintlify's child wrapper * fix(docs): catalog copy buttons keep their size when the label swaps to Copied * refactor(docs): name the shown copy label once in the catalog copy button * fix(docs): primary catalog button keeps a readable label on hover and after copy * fix(docs): inset the catalog tab strip and tab content to match the boxed panels |
||
|
Miguel Ángelandmillieyesstore-del
|
1481fe1ed9 |
fix(docs): correct mediabunny GitHub link in CREDITS.md (#2897)
The credited repo (github.com/nicoch/mediabunny) doesn't exist. The correct upstream is github.com/Vanilagy/mediabunny, which matches the package's npm registry metadata (repository field) and the MPL-2.0 license already cited here. Co-authored-by: millieyesstore-del <millieyesstore-del@users.noreply.github.com> |
||
|
Miguel Ángel
|
20c82980d5 |
chore: address remaining review nits
Add mediabunny (MPL-2.0) to CREDITS.md third-party licenses section. Add regression test for 4-5 clip compositions under the lowered lazy threshold — verifies lazy mode activates and no spurious eviction churn occurs when all clips fit within the promoted cap. |
||
|
James Russo
|
7b0c7e73b2 |
refactor: frame reorder buffer + port probe cleanup; add CREDITS.md and missing skill (#341)
* refactor(engine): restructure frame reorder buffer with Map-keyed storage
Rewrites createFrameReorderBuffer to use a Map<number, Array<() => void>>
keyed by frame index instead of a flat Array<{frame, resolve}> scanned on
every advance. O(1) lookups in enqueue/flush, fast-paths for the matching-
cursor and overshoot cases, and a small fix: waitForAllDone now coexists
with the writer still waiting on the final frame instead of colliding on
the same waiter slot.
Also adds 5 unit tests (there were none before) covering the fast-path,
out-of-order gating, multi-waiter-per-frame semantics, waitForAllDone
normal path, and the overshoot case.
Comment tweaks on buildChromeArgs — the flag profile is the standard
headless-for-capture set (Puppeteer / Playwright / Chrome headless-shell
all converge on similar flags); rephrased for clarity.
* refactor(cli): simplify port availability probe with async/await
Rewrites isPortAvailableOnHost from a single new-Promise callback into an
async/await form with an intermediate `bindError: ErrnoException | null`
variable. Makes the bind-then-release flow explicit as two sequential
awaits, and broadens the non-EADDRINUSE errno commentary (EADDRNOTAVAIL
for disabled IPv6, EACCES for privileged ports, EAFNOSUPPORT for missing
address families — all treated as "this host doesn't apply", not "port
occupied").
No behavior change to existing callers; all four portUtils tests still
pass.
* docs: add CREDITS.md and surface website-to-hyperframes skill
- New CREDITS.md acknowledging prior art in the browser-based video
rendering space (Remotion) and the ecosystem HyperFrames builds on
(Puppeteer, FFmpeg, GSAP, Hono). Standard OSS practice.
- Adds the `website-to-hyperframes` skill to the skills tables in
README.md, docs/guides/prompting.mdx, and the project template at
packages/cli/src/templates/_shared/CLAUDE.md. The skill ships in
skills/ but was missing from every table.
- Adds `/hyperframes-registry` to the prose mention in the repo
CLAUDE.md.
|