hyperframes

ext/hyperframes

Fork 0

mirror of https://github.com/heygen-com/hyperframes.git synced 2026-11-04 13:07:11 +00:00

Commit Graph

Author	SHA1	Message	Date
Xuanru Li	1544b519a9	feat(cli): beats tracks the music with the TypeScript beat tracker, without Chrome (#5634 ) * feat(cli): audio answers beats, downbeats, bpm, sections and describe through a provider ladder Beats and downbeats try api, apple-mu, beat-this, then signal; sections try api, apple-mu, then signal; describe tries api, then apple-sound. bpm is read off whichever provider placed the beats. Each provider says whether it can run here and why not, and the ladder warns when it passes one over. The signal rung is a TypeScript tracker: log-band spectral flux, autocorrelation tempo with a log-normal prior, Ellis dynamic-programming beats with a windowed local tempo, a low-band bar phase, and Foote novelty sections snapped to beats. It reports no steady beat when the onsets do not repeat. The api rung posts the audio to HYPERFRAMES_AUDIO_API_URL with HYPERFRAMES_AUDIO_API_KEY and moves each answered time onto the nearest signal beat within a quarter beat, reporting what moved and what did not. beat-this runs Beat This! from a configured Python and local weights and never downloads. apple-sound runs macOS's sound classifier through a Swift helper. apple-mu reports that Music Understanding needs macOS 27. words are no longer accepted: they come from hyperframes transcribe. * docs(cli): audio's beats, downbeats, bpm, sections, describe and their provider ladder * feat(cli): beats tracks the music with the TypeScript beat tracker, without Chrome hyperframes beats and Studio's beat detection now share analyzeMusicFromSignal: band energies, then the spectral-flux tracker in audioRhythm.ts, then each beat's loudness for its strength. The CLI decodes with the same ffmpeg path as hyperframes audio, so beats no longer launches headless Chrome. Studio still decodes in the browser and passes every channel. The old analyzer averaged bpm-detective's tempo with an energy-peak tempo it had only compared after folding both into one octave, and phased a rigid grid from noisy peaks. On 22 real tracks with exact script grids it found the tempo 18% of the time with beat F 0.437; this finds it 91% of the time with F 0.887, and refuses the other two rather than guess. Below a tracker confidence of 0.2 no beats are reported, so ambient or rubato music gets no invented grid and the CLI still writes nothing. bpm-detective, the energy-peak grid, the headless analyzer and its prebuilt bundle are gone. * fix(core): tempo reads 96 BPM off beats on 10 ms frames, not 95.2 The tracker places beats on 10 ms frames, so a 625 ms beat lands 620 or 630 ms apart and the median interval reported 95.2 or 96.8 BPM. Tempo now averages the intervals within 10% of the median: as robust to a skipped or doubled beat, without the frame-sized step. Both hyperframes audio and hyperframes beats read it. * fix(cli): remove redundant audio context comment * fix(core): remove redundant tempo interval comment * fix(ci): record beat analyzer removals and browser evidence * fix(core): keep flat audio novelty in one section * chore: keep PR capture evidence out of the source tree * fix(cli): honor BPM provider pins before analysis	2026-10-15 06:23:03 +00:00
Miguel Ángel	20d915938b	feat(studio): ask for feedback when a render ends, not on a session counter (#3205 ) ## What Studio's feedback prompt now fires when a render finishes or fails, instead of on a session counter, and the reports it collects carry enough context to act on. - Replaces the 32px inline bar with a card in the existing toast stack - Adds `studio_feedback_shown` / `studio_feedback_dismissed` / `studio_feedback_interview_click`, so the funnel is visible - A failed export and a crash skip the 0-10 score and ask what happened - Adds a crash prompt to the error boundary - Attaches a breadcrumb trail, render settings and outcome, and how the project was created ## Before / after <img width="1500" alt="Before: a 32px feedback strip pinned under the preview. After: the feedback card in the toast stack, with one-tap answers, and the red variant for a failed export." src="https://github.com/user-attachments/assets/29e2373f-ee96-42c4-b006-ab9a0e56a60a" /> The old bar's rating numbers are `neutral-600` on `neutral-900/80`, which is why it reads as a disabled row rather than a control. In the running Studio — bottom-right, sharing the toast stack, hovering a chip explains it on the line above the input: <img width="1500" alt="The feedback card in the bottom-right of the running Studio, with the rotated follow-up question and a hovered chip explained inline." src="https://github.com/user-attachments/assets/3969da50-cc7a-466a-bbf9-b151a1bc1d5d" /> On the crash screen — the prompt the error boundary renders, asking what the user was doing rather than what went wrong, since the stack trace already covers the latter: <img width="1200" alt="The Studio crash screen with the feedback card below the Try again and Reload Studio buttons." src="https://github.com/user-attachments/assets/4d287011-9c19-4b94-99d0-07a0388fb39c" /> \| \| Before \| After \| \|---\|---\|---\| \| Trigger \| Every 10th session \| A render finishing, failing, or a crash \| \| Placement \| 32px inline bar, pushes preview up \| Card in the toast stack, no layout shift \| \| Rating targets \| 11 bare buttons, `neutral-600` \| Native radios, resting fill, `neutral-400` \| \| Follow-up \| Free text or nothing \| One rotated question, four to seven one-tap answers \| \| Option help \| None \| Inline hint on hover and focus \| \| Press feedback \| None \| `active:scale-[0.97]`, 150ms ease-out \| \| Keyboard \| No exit path \| Escape closes, Enter sends, arrows move the rating \| \| Auto-dismiss \| 20s, always \| 30s, cancelled the moment you interact \| \| Failure case \| Same NPS question \| Its own question, no score, error quoted back \| \| Crash case \| Nothing \| Prompt on the crash screen \| \| Visibility \| Submissions only \| Shown / dismissed (with reason) / submitted / interview click \| \| Report content \| Rating, comment \| Plus breadcrumbs, render settings and outcome, project provenance \| ## Why The old bar fired on a session count, so it interrupted at a moment with no subject: nothing the user had just done, nothing to have an opinion about. It also emitted nothing when it appeared or when it was dismissed, which made the collection rate impossible to diagnose. A prompt nobody answers and a prompt that never renders looked identical from the outside. Visually it read as a disabled row: 11 buttons at 11px in `neutral-600` on a dark strip, with no resting affordance. And appearing mid-task pushed the whole preview stack up, which the old code carried a comment apologising for. Separately, the reports it did collect were not actionable. A comment says what went wrong; it almost never says how to get there. ## How One trigger, one owner. `feedbackTrigger` owns eligibility and nothing else does: once per tab, thirty days after an answer, seven after a dismissal, never when telemetry is off (prompting someone whose response we would then drop wastes their attention). `VITE_HYPERFRAMES_NO_FEEDBACK=1` still disables it entirely. One hook, every failure path. The trigger watches the render job list rather than each of the four places a render can finish (server rejection, unreachable server, SSE terminal event, SSE connection drop), so paths added later are covered without touching the trigger. Renders loaded from disk history never fire it. Reuses what exists. The card wears `StudioToast`'s glass treatment and joins its stack, so there is no second visual language and no new CSS. The rating row is native radios, which gives arrow-key navigation, grouping and labels for free. One question each, rotated across users. A corner card that asks three things gets answered by nobody. Each person gets one follow-up with one-tap answers, explained on a reserved line rather than a floating tooltip (the card is 340px in a corner; a bubble above the chips lands on the question, below lands on the input). Detractors are never given a rotated question, because they already have a specific complaint. Every option was checked against the code: an option naming a feature Studio already has would collect taps meaning "I could not find it", which is indistinguishable afterwards from "it does not exist". Breadcrumbs cost one line. Every studio event already flows through `trackEvent`, so recording the trail there needs no new instrumentation and stays correct as events are added. Provenance lives outside React. A crash unmounts the tree, so it is captured when the project loads and read from module scope when the crash prompt renders. ### Privacy Breadcrumbs and provenance carry names, enums and counts only. Values are copied from a fixed allowlist of short keys, and anything longer than a slug is dropped rather than truncated, so comments, file paths, stack traces and project titles cannot reach them even if a future event carries one. Tests assert this. ### Where these responses land Studio feedback goes to PostHog and nowhere else, which is what it did before this change too. Worth stating because the CLI behaves differently: `hyperframes feedback` also forwards to the backend feedback endpoint via `submitFeedback`, on top of its PostHog event. Studio has never used that path, before or after this PR, so if you read CLI feedback anywhere other than PostHog, Studio responses will not show up there. Nothing here changes that either way. Whether the two surfaces should share a delivery path is a product question, not a defect in this change, and closing it would need a field on the backend DTO: it is shaped around `cli_version`, and Studio reports from a crash or a failed export deliberately carry no rating. ## Test plan - [x] Unit tests added/updated - [x] Manual testing performed - [ ] Documentation updated (if applicable) Unit — 39 new tests: trigger eligibility and cooldowns, the detractor override, rotation, preset shape and the no-brands rule, breadcrumb rolling and privacy, provenance parsing and its failure modes, and the crash boundary rendering the prompt with no rating input. Live — both render paths driven end to end against a running Studio on a production bundle, with real renders. Every PostHog request was intercepted and dropped, so nothing reached the project. Verified the emitted payload for a finished render, a failed export, the rotated follow-ups, each chip's hint, and the interview link. Not covered — no live capture of a spontaneous crash. Three attempts to force one failed because Studio's guards held and it kept rendering, so the crash path is verified by component tests rather than by driving it. Touch devices see chip labels without hints, since the hint is revealed on hover and focus.	2026-08-11 01:33:52 -04:00
Miguel Ángel	f28bc80a1d	feat(scripts): fail a branch that deletes files main still ships (#3150 ) * feat(scripts): fail a branch that deletes files main still ships Written after a scare that turned out to be a measurement error, and the error is the reason it exists. Comparing tip to tip on a branch a month behind reports every file main has added since the merge base as a deletion: 1,284 of them, an entire skills tree among them, none of it real. A merge keeps mains side and a pull request shows the three-dot diff, which reported zero. So the gate uses the three-dot form and reports renames separately, because in a name-only diff a rename is indistinguishable from a deletion and treating them alike would either mask real loss or block every legitimate move. * ci: enforce the no-deletions guard	2026-08-09 22:24:42 -07:00

Author

SHA1

Message

Date

Xuanru Li

1544b519a9

feat(cli): beats tracks the music with the TypeScript beat tracker, without Chrome (#5634 )

* feat(cli): audio answers beats, downbeats, bpm, sections and describe through a provider ladder

Beats and downbeats try api, apple-mu, beat-this, then signal; sections try api, apple-mu,
then signal; describe tries api, then apple-sound. bpm is read off whichever provider placed
the beats. Each provider says whether it can run here and why not, and the ladder warns
when it passes one over.

The signal rung is a TypeScript tracker: log-band spectral flux, autocorrelation tempo with
a log-normal prior, Ellis dynamic-programming beats with a windowed local tempo, a low-band
bar phase, and Foote novelty sections snapped to beats. It reports no steady beat when the
onsets do not repeat.

The api rung posts the audio to HYPERFRAMES_AUDIO_API_URL with HYPERFRAMES_AUDIO_API_KEY and
moves each answered time onto the nearest signal beat within a quarter beat, reporting what
moved and what did not. beat-this runs Beat This! from a configured Python and local weights
and never downloads. apple-sound runs macOS's sound classifier through a Swift helper.
apple-mu reports that Music Understanding needs macOS 27.

words are no longer accepted: they come from hyperframes transcribe.

* docs(cli): audio's beats, downbeats, bpm, sections, describe and their provider ladder

* feat(cli): beats tracks the music with the TypeScript beat tracker, without Chrome

hyperframes beats and Studio's beat detection now share analyzeMusicFromSignal: band energies,
then the spectral-flux tracker in audioRhythm.ts, then each beat's loudness for its strength.
The CLI decodes with the same ffmpeg path as hyperframes audio, so beats no longer launches
headless Chrome. Studio still decodes in the browser and passes every channel.

The old analyzer averaged bpm-detective's tempo with an energy-peak tempo it had only compared
after folding both into one octave, and phased a rigid grid from noisy peaks. On 22 real tracks
with exact script grids it found the tempo 18% of the time with beat F 0.437; this finds it
91% of the time with F 0.887, and refuses the other two rather than guess.

Below a tracker confidence of 0.2 no beats are reported, so ambient or rubato music gets no
invented grid and the CLI still writes nothing. bpm-detective, the energy-peak grid, the
headless analyzer and its prebuilt bundle are gone.

* fix(core): tempo reads 96 BPM off beats on 10 ms frames, not 95.2

The tracker places beats on 10 ms frames, so a 625 ms beat lands 620 or 630 ms apart and the
median interval reported 95.2 or 96.8 BPM. Tempo now averages the intervals within 10% of the
median: as robust to a skipped or doubled beat, without the frame-sized step. Both
hyperframes audio and hyperframes beats read it.

* fix(cli): remove redundant audio context comment

* fix(core): remove redundant tempo interval comment

* fix(ci): record beat analyzer removals and browser evidence

* fix(core): keep flat audio novelty in one section

* chore: keep PR capture evidence out of the source tree

* fix(cli): honor BPM provider pins before analysis

2026-10-15 06:23:03 +00:00

Miguel Ángel

20d915938b

feat(studio): ask for feedback when a render ends, not on a session counter (#3205 )

## What

Studio's feedback prompt now fires when a render finishes or fails, instead of on a session counter, and the reports it collects carry enough context to act on.

- Replaces the 32px inline bar with a card in the existing toast stack
- Adds `studio_feedback_shown` / `studio_feedback_dismissed` / `studio_feedback_interview_click`, so the funnel is visible
- A failed export and a crash skip the 0-10 score and ask what happened
- Adds a crash prompt to the error boundary
- Attaches a breadcrumb trail, render settings and outcome, and how the project was created

## Before / after

<img width="1500" alt="Before: a 32px feedback strip pinned under the preview. After: the feedback card in the toast stack, with one-tap answers, and the red variant for a failed export." src="https://github.com/user-attachments/assets/29e2373f-ee96-42c4-b006-ab9a0e56a60a" />

The old bar's rating numbers are `neutral-600` on `neutral-900/80`, which is why it reads as a disabled row rather than a control.

**In the running Studio** — bottom-right, sharing the toast stack, hovering a chip explains it on the line above the input:

<img width="1500" alt="The feedback card in the bottom-right of the running Studio, with the rotated follow-up question and a hovered chip explained inline." src="https://github.com/user-attachments/assets/3969da50-cc7a-466a-bbf9-b151a1bc1d5d" />

**On the crash screen** — the prompt the error boundary renders, asking what the user was doing rather than what went wrong, since the stack trace already covers the latter:

<img width="1200" alt="The Studio crash screen with the feedback card below the Try again and Reload Studio buttons." src="https://github.com/user-attachments/assets/4d287011-9c19-4b94-99d0-07a0388fb39c" />

| | Before | After |
|---|---|---|
| **Trigger** | Every 10th session | A render finishing, failing, or a crash |
| **Placement** | 32px inline bar, pushes preview up | Card in the toast stack, no layout shift |
| **Rating targets** | 11 bare buttons, `neutral-600` | Native radios, resting fill, `neutral-400` |
| **Follow-up** | Free text or nothing | One rotated question, four to seven one-tap answers |
| **Option help** | None | Inline hint on hover and focus |
| **Press feedback** | None | `active:scale-[0.97]`, 150ms ease-out |
| **Keyboard** | No exit path | Escape closes, Enter sends, arrows move the rating |
| **Auto-dismiss** | 20s, always | 30s, cancelled the moment you interact |
| **Failure case** | Same NPS question | Its own question, no score, error quoted back |
| **Crash case** | Nothing | Prompt on the crash screen |
| **Visibility** | Submissions only | Shown / dismissed (with reason) / submitted / interview click |
| **Report content** | Rating, comment | Plus breadcrumbs, render settings and outcome, project provenance |


## Why

The old bar fired on a session count, so it interrupted at a moment with no subject: nothing the user had just done, nothing to have an opinion about. It also emitted nothing when it appeared or when it was dismissed, which made the collection rate impossible to diagnose. A prompt nobody answers and a prompt that never renders looked identical from the outside.

Visually it read as a disabled row: 11 buttons at 11px in `neutral-600` on a dark strip, with no resting affordance. And appearing mid-task pushed the whole preview stack up, which the old code carried a comment apologising for.

Separately, the reports it did collect were not actionable. A comment says what went wrong; it almost never says how to get there.

## How

**One trigger, one owner.** `feedbackTrigger` owns eligibility and nothing else does: once per tab, thirty days after an answer, seven after a dismissal, never when telemetry is off (prompting someone whose response we would then drop wastes their attention). `VITE_HYPERFRAMES_NO_FEEDBACK=1` still disables it entirely.

**One hook, every failure path.** The trigger watches the render job list rather than each of the four places a render can finish (server rejection, unreachable server, SSE terminal event, SSE connection drop), so paths added later are covered without touching the trigger. Renders loaded from disk history never fire it.

**Reuses what exists.** The card wears `StudioToast`'s glass treatment and joins its stack, so there is no second visual language and no new CSS. The rating row is native radios, which gives arrow-key navigation, grouping and labels for free.

**One question each, rotated across users.** A corner card that asks three things gets answered by nobody. Each person gets one follow-up with one-tap answers, explained on a reserved line rather than a floating tooltip (the card is 340px in a corner; a bubble above the chips lands on the question, below lands on the input). Detractors are never given a rotated question, because they already have a specific complaint. Every option was checked against the code: an option naming a feature Studio already has would collect taps meaning "I could not find it", which is indistinguishable afterwards from "it does not exist".

**Breadcrumbs cost one line.** Every studio event already flows through `trackEvent`, so recording the trail there needs no new instrumentation and stays correct as events are added.

**Provenance lives outside React.** A crash unmounts the tree, so it is captured when the project loads and read from module scope when the crash prompt renders.

### Privacy

Breadcrumbs and provenance carry names, enums and counts only. Values are copied from a fixed allowlist of short keys, and anything longer than a slug is dropped rather than truncated, so comments, file paths, stack traces and project titles cannot reach them even if a future event carries one. Tests assert this.

### Where these responses land

Studio feedback goes to PostHog and nowhere else, which is what it did before this change too.

Worth stating because the CLI behaves differently: `hyperframes feedback` also forwards to the backend feedback endpoint via `submitFeedback`, on top of its PostHog event. Studio has never used that path, before or after this PR, so if you read CLI feedback anywhere other than PostHog, Studio responses will not show up there.

Nothing here changes that either way. Whether the two surfaces should share a delivery path is a product question, not a defect in this change, and closing it would need a field on the backend DTO: it is shaped around `cli_version`, and Studio reports from a crash or a failed export deliberately carry no rating.

## Test plan

- [x] Unit tests added/updated
- [x] Manual testing performed
- [ ] Documentation updated (if applicable)

**Unit** — 39 new tests: trigger eligibility and cooldowns, the detractor override, rotation, preset shape and the no-brands rule, breadcrumb rolling and privacy, provenance parsing and its failure modes, and the crash boundary rendering the prompt with no rating input.

**Live** — both render paths driven end to end against a running Studio on a production bundle, with real renders. Every PostHog request was intercepted and dropped, so nothing reached the project. Verified the emitted payload for a finished render, a failed export, the rotated follow-ups, each chip's hint, and the interview link.

**Not covered** — no live capture of a spontaneous crash. Three attempts to force one failed because Studio's guards held and it kept rendering, so the crash path is verified by component tests rather than by driving it. Touch devices see chip labels without hints, since the hint is revealed on hover and focus.

2026-08-11 01:33:52 -04:00

Miguel Ángel

f28bc80a1d

feat(scripts): fail a branch that deletes files main still ships (#3150 )

* feat(scripts): fail a branch that deletes files main still ships

Written after a scare that turned out to be a measurement error, and the error is the reason it exists. Comparing tip to tip on a branch a month behind reports every file main has added since the merge base as a deletion: 1,284 of them, an entire skills tree among them, none of it real. A merge keeps mains side and a pull request shows the three-dot diff, which reported zero.

So the gate uses the three-dot form and reports renames separately, because in a name-only diff a rename is indistinguishable from a deletion and treating them alike would either mask real loss or block every legitimate move.

* ci: enforce the no-deletions guard

2026-08-09 22:24:42 -07:00