mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-11-17 07:05:28 +00:00
526 lines
18 KiB
HTML
526 lines
18 KiB
HTML
<!doctype html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="utf-8" />
|
|
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
|
<title>HyperFrames vs Editframe Devbox Eval</title>
|
|
<style>
|
|
:root {
|
|
color-scheme: light;
|
|
--ink: #111;
|
|
--muted: #5b6168;
|
|
--line: #d9d9d2;
|
|
--paper: #f7f5ed;
|
|
--panel: #fffdf8;
|
|
--accent: #0d6b68;
|
|
--warn: #9f5b00;
|
|
--bad: #a11;
|
|
--good: #146c2e;
|
|
}
|
|
* {
|
|
box-sizing: border-box;
|
|
}
|
|
body {
|
|
margin: 0;
|
|
font-family:
|
|
Inter,
|
|
ui-sans-serif,
|
|
system-ui,
|
|
-apple-system,
|
|
BlinkMacSystemFont,
|
|
"Segoe UI",
|
|
sans-serif;
|
|
background: var(--paper);
|
|
color: var(--ink);
|
|
line-height: 1.5;
|
|
}
|
|
main {
|
|
max-width: 1180px;
|
|
margin: 0 auto;
|
|
padding: 48px 28px 80px;
|
|
}
|
|
h1 {
|
|
font-size: clamp(38px, 6vw, 76px);
|
|
line-height: 0.95;
|
|
margin: 0 0 18px;
|
|
letter-spacing: 0;
|
|
}
|
|
h2 {
|
|
margin: 48px 0 14px;
|
|
font-size: 28px;
|
|
}
|
|
h3 {
|
|
margin: 28px 0 10px;
|
|
}
|
|
.lede {
|
|
font-size: 20px;
|
|
color: var(--muted);
|
|
max-width: 900px;
|
|
}
|
|
.grid {
|
|
display: grid;
|
|
grid-template-columns: repeat(2, minmax(0, 1fr));
|
|
gap: 18px;
|
|
}
|
|
.card {
|
|
background: var(--panel);
|
|
border: 1px solid var(--line);
|
|
border-radius: 8px;
|
|
padding: 18px;
|
|
}
|
|
.metric {
|
|
font-size: 34px;
|
|
font-weight: 800;
|
|
}
|
|
.label {
|
|
color: var(--muted);
|
|
font-size: 14px;
|
|
text-transform: uppercase;
|
|
letter-spacing: 0.04em;
|
|
}
|
|
table {
|
|
width: 100%;
|
|
border-collapse: collapse;
|
|
background: var(--panel);
|
|
border: 1px solid var(--line);
|
|
}
|
|
th,
|
|
td {
|
|
text-align: left;
|
|
border-bottom: 1px solid var(--line);
|
|
padding: 10px 12px;
|
|
vertical-align: top;
|
|
}
|
|
th {
|
|
font-size: 13px;
|
|
text-transform: uppercase;
|
|
color: var(--muted);
|
|
}
|
|
.pill {
|
|
display: inline-block;
|
|
padding: 3px 8px;
|
|
border-radius: 999px;
|
|
font-size: 12px;
|
|
font-weight: 700;
|
|
background: #eee;
|
|
}
|
|
.HF-edge {
|
|
background: #e6f4ed;
|
|
color: var(--good);
|
|
}
|
|
.EF-conditional-edge,
|
|
.EF-source-doc-edge {
|
|
background: #fff2d8;
|
|
color: var(--warn);
|
|
}
|
|
.split {
|
|
background: #edf1ff;
|
|
color: #26408b;
|
|
}
|
|
.not_final,
|
|
.not_tested {
|
|
background: #eee;
|
|
color: #555;
|
|
}
|
|
pre {
|
|
white-space: pre-wrap;
|
|
background: #151515;
|
|
color: #eee;
|
|
padding: 14px;
|
|
border-radius: 8px;
|
|
overflow: auto;
|
|
}
|
|
a {
|
|
color: var(--accent);
|
|
}
|
|
video {
|
|
width: 100%;
|
|
border: 1px solid var(--line);
|
|
border-radius: 8px;
|
|
background: #000;
|
|
}
|
|
.shots {
|
|
display: grid;
|
|
grid-template-columns: repeat(4, minmax(0, 1fr));
|
|
gap: 10px;
|
|
}
|
|
figure {
|
|
margin: 0;
|
|
}
|
|
img {
|
|
width: 100%;
|
|
display: block;
|
|
border: 1px solid var(--line);
|
|
border-radius: 6px;
|
|
background: #111;
|
|
}
|
|
figcaption {
|
|
color: var(--muted);
|
|
font-size: 12px;
|
|
margin-top: 4px;
|
|
overflow-wrap: anywhere;
|
|
}
|
|
.callout {
|
|
border-left: 5px solid var(--accent);
|
|
background: #eef8f5;
|
|
padding: 16px 18px;
|
|
border-radius: 6px;
|
|
}
|
|
@media (max-width: 820px) {
|
|
.grid,
|
|
.shots {
|
|
grid-template-columns: 1fr;
|
|
}
|
|
}
|
|
</style>
|
|
</head>
|
|
<body>
|
|
<main>
|
|
<h1>HyperFrames vs Editframe Devbox Eval</h1>
|
|
<p class="lede">
|
|
Execution date: 2026-04-30 on <code>ip-10-0-9-220</code>. This report evaluates one
|
|
precise-timing fixture plus public/docs/API evidence. It is not a blind taste benchmark.
|
|
</p>
|
|
<section class="callout">
|
|
<strong>Bottom line:</strong> HyperFrames completed the default local H.264 MP4 render path
|
|
on this devbox. Editframe's default local CLI MP4 render failed because this
|
|
Chrome/WebCodecs build does not support the selected AVC encoder config, but the same
|
|
Editframe fixture rendered successfully through its browser API when VP9 was selected
|
|
programmatically.
|
|
</section>
|
|
<h2>Automated Render Results</h2>
|
|
<div class="grid">
|
|
<div class="card">
|
|
<div class="label">HyperFrames</div>
|
|
<div class="metric">Pass</div>
|
|
<p>
|
|
Published <code>hyperframes@latest</code> linted, validated, snapshotted, and rendered a
|
|
10s 1920x1080/30fps H.264 MP4.
|
|
</p>
|
|
<ul>
|
|
<li>Command elapsed: 5971ms</li>
|
|
<li>Duration: 10.000000s</li>
|
|
<li>Size: 700 KB</li>
|
|
<li>Codec: h264</li>
|
|
</ul>
|
|
<p>
|
|
<a href="../artifacts/devbox-2026-04-30/hyperframes/renders/render.log">render log</a> ·
|
|
<a href="../artifacts/devbox-2026-04-30/hyperframes/renders/ffprobe.json">ffprobe</a>
|
|
</p>
|
|
</div>
|
|
<div class="card">
|
|
<div class="label">Editframe</div>
|
|
<div class="metric">Mixed</div>
|
|
<p>
|
|
Published <code>@editframe/cli@0.49.7</code> default AVC render failed. Browser-driven
|
|
<code>EF_RENDER.renderStreaming({ codec: "vp9" })</code> produced a 10s 1920x1080/30fps
|
|
MP4 containing VP9 video.
|
|
</p>
|
|
<ul>
|
|
<li>Browser VP9 elapsed: 1280ms</li>
|
|
<li>Duration: 10.000000s</li>
|
|
<li>Size: 421 KB</li>
|
|
<li>Codec: vp9</li>
|
|
</ul>
|
|
<p>
|
|
<a href="../artifacts/devbox-2026-04-30/editframe/renders/render-cli-avc.log"
|
|
>failed CLI log</a
|
|
>
|
|
·
|
|
<a href="../artifacts/devbox-2026-04-30/editframe/renders/ffprobe-vp9.json"
|
|
>VP9 ffprobe</a
|
|
>
|
|
</p>
|
|
</div>
|
|
</div>
|
|
<h2>Rubric Findings</h2>
|
|
<table>
|
|
<thead>
|
|
<tr>
|
|
<th>Category</th>
|
|
<th>Weight</th>
|
|
<th>Finding</th>
|
|
<th>Evidence</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody>
|
|
<tr>
|
|
<td>Visual output quality</td>
|
|
<td>18%</td>
|
|
<td><span class="pill not_final">not final</span></td>
|
|
<td>
|
|
Not blindly judged. Both controlled fixtures produced screenshots; this run should not
|
|
decide taste/video quality.
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<td>Timing and motion control</td>
|
|
<td>14%</td>
|
|
<td><span class="pill HF-edge">HF edge</span></td>
|
|
<td>
|
|
Both can seek exact timestamps. HyperFrames validated and snapshotted the GSAP
|
|
timeline through first-party CLI; Editframe snapshotted via ef-timegroup.seek and
|
|
rendered through browser API.
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<td>Timeline and editability</td>
|
|
<td>12%</td>
|
|
<td><span class="pill split">split</span></td>
|
|
<td>
|
|
Editframe has richer public editor primitives; HyperFrames source remains plain
|
|
HTML/data attributes and passed lint/validate.
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<td>Agent success rate</td>
|
|
<td>12%</td>
|
|
<td><span class="pill HF-edge">HF edge</span></td>
|
|
<td>
|
|
HyperFrames fresh init plus lint/validate/snapshot/render passed. Editframe fresh init
|
|
passed, but advertised --skip-skills flag failed and default CLI render failed on AVC.
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<td>Media handling</td>
|
|
<td>10%</td>
|
|
<td><span class="pill not_tested">not tested</span></td>
|
|
<td>This devbox run did not exercise external media, audio, captions, or alpha.</td>
|
|
</tr>
|
|
<tr>
|
|
<td>Render fidelity</td>
|
|
<td>10%</td>
|
|
<td><span class="pill HF-edge">HF edge</span></td>
|
|
<td>
|
|
HyperFrames produced default H.264 MP4 at 1920x1080/30fps/10s. Editframe CLI default
|
|
failed; browser VP9 fallback produced valid 1920x1080/30fps/10s MP4.
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<td>Performance and cost</td>
|
|
<td>8%</td>
|
|
<td><span class="pill EF-conditional-edge">EF conditional edge</span></td>
|
|
<td>
|
|
Editframe VP9 browser render elapsed 1280ms; HyperFrames draft H.264 render command
|
|
elapsed 5971ms. Codecs/render paths differ, so this is directional only.
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<td>Observability and debugging</td>
|
|
<td>7%</td>
|
|
<td><span class="pill HF-edge">HF edge</span></td>
|
|
<td>
|
|
HyperFrames surfaced lint/contrast checks and progress. Editframe error was clear, but
|
|
the CLI left a 28-byte output file and exposes no codec flag.
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<td>Deployment and API ergonomics</td>
|
|
<td>5%</td>
|
|
<td><span class="pill EF-source-doc-edge">EF source-doc edge</span></td>
|
|
<td>
|
|
Editframe public docs include cloud/API render jobs, file processing, webhooks, and
|
|
CPU/GPU backend selection. This run did not authenticate to test it live.
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<td>Human iteration workflow</td>
|
|
<td>4%</td>
|
|
<td><span class="pill not_final">not final</span></td>
|
|
<td>
|
|
Not exercised with human editors. Public docs suggest both have timeline/editor
|
|
workflows; Editframe has broader standalone editor widgets.
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
<h2>Codec Probe</h2>
|
|
<p>
|
|
Editframe's browser renderer depends on WebCodecs encoder support. On this devbox Chrome
|
|
supports VP9/VP8/AV1 but not the AVC config selected by the CLI.
|
|
</p>
|
|
<table>
|
|
<thead>
|
|
<tr>
|
|
<th>Name</th>
|
|
<th>Codec string</th>
|
|
<th>Support</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody>
|
|
<tr>
|
|
<td>avc</td>
|
|
<td>avc1.640028</td>
|
|
<td>not supported</td>
|
|
</tr>
|
|
<tr>
|
|
<td>vp9</td>
|
|
<td>vp09.00.10.08</td>
|
|
<td>supported</td>
|
|
</tr>
|
|
<tr>
|
|
<td>vp8</td>
|
|
<td>vp8</td>
|
|
<td>supported</td>
|
|
</tr>
|
|
<tr>
|
|
<td>av1</td>
|
|
<td>av01.0.05M.08</td>
|
|
<td>supported</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
<h2>Videos</h2>
|
|
<div class="grid">
|
|
<div>
|
|
<h3>HyperFrames H.264 MP4</h3>
|
|
<video
|
|
controls
|
|
src="../artifacts/devbox-2026-04-30/hyperframes/renders/hyperframes-precise.mp4"
|
|
></video>
|
|
</div>
|
|
<div>
|
|
<h3>Editframe Browser VP9 MP4</h3>
|
|
<video
|
|
controls
|
|
src="../artifacts/devbox-2026-04-30/editframe/renders/editframe-browser-vp9.mp4"
|
|
></video>
|
|
</div>
|
|
</div>
|
|
<h2>Timestamp Snapshots</h2>
|
|
<h3>HyperFrames snapshots</h3>
|
|
<div class="shots">
|
|
<figure>
|
|
<img src="../artifacts/devbox-2026-04-30/hyperframes/snapshots/frame-00-at-1.2s.png" />
|
|
<figcaption>frame-00-at-1.2s.png</figcaption>
|
|
</figure>
|
|
<figure>
|
|
<img src="../artifacts/devbox-2026-04-30/hyperframes/snapshots/frame-01-at-5.0s.png" />
|
|
<figcaption>frame-01-at-5.0s.png</figcaption>
|
|
</figure>
|
|
<figure>
|
|
<img src="../artifacts/devbox-2026-04-30/hyperframes/snapshots/frame-02-at-8.5s.png" />
|
|
<figcaption>frame-02-at-8.5s.png</figcaption>
|
|
</figure>
|
|
<figure>
|
|
<img src="../artifacts/devbox-2026-04-30/hyperframes/snapshots/frame-03-at-9.4s.png" />
|
|
<figcaption>frame-03-at-9.4s.png</figcaption>
|
|
</figure>
|
|
</div>
|
|
<h3>Editframe snapshots</h3>
|
|
<div class="shots">
|
|
<figure>
|
|
<img src="../artifacts/devbox-2026-04-30/editframe/snapshots/editframe-1200ms.png" />
|
|
<figcaption>editframe-1200ms.png</figcaption>
|
|
</figure>
|
|
<figure>
|
|
<img src="../artifacts/devbox-2026-04-30/editframe/snapshots/editframe-5000ms.png" />
|
|
<figcaption>editframe-5000ms.png</figcaption>
|
|
</figure>
|
|
<figure>
|
|
<img src="../artifacts/devbox-2026-04-30/editframe/snapshots/editframe-8500ms.png" />
|
|
<figcaption>editframe-8500ms.png</figcaption>
|
|
</figure>
|
|
<figure>
|
|
<img src="../artifacts/devbox-2026-04-30/editframe/snapshots/editframe-9400ms.png" />
|
|
<figcaption>editframe-9400ms.png</figcaption>
|
|
</figure>
|
|
</div>
|
|
<h2>Important Failure Evidence</h2>
|
|
<h3>Editframe default CLI render failure</h3>
|
|
<pre>
|
|
- Initializing...
|
|
✔ Ready
|
|
- Rendering video...
|
|
✖ Render failed
|
|
node:internal/process/promises:394
|
|
triggerUncaughtException(err, true /* fromPromise */);
|
|
^
|
|
|
|
page.evaluate: Error: This specific encoder configuration (avc1.640028, 8000000 bps, 1920x1080, hardware acceleration: no-preference) is not supported by this browser. Consider using another codec or changing your video parameters.
|
|
at http://localhost:5176/node_modules/.vite/deps/chunk-M5YSC7MS.js?v=622dd3ba:23270:17
|
|
at async VideoEncoderWrapper.processAndEncode (http://localhost:5176/node_modules/.vite/deps/chunk-M5YSC7MS.js?v=622dd3ba:23120:13)
|
|
at async VideoEncoderWrapper.add (http://localhost:5176/node_modules/.vite/deps/chunk-M5YSC7MS.js?v=622dd3ba:23073:7)
|
|
at async renderTimegroupToVideo (http://localhost:5176/node_modules/.vite/deps/chunk-67JHYUZS.js?v=622dd3ba:330:9)
|
|
at async Object.renderStreaming (http://localhost:5176/node_modules/.vite/deps/@editframe_elements.js?v=622dd3ba:34967:7)
|
|
at async eval (eval at evaluate (:302:30), <anonymous>:2:7)
|
|
at async <anonymous>:328:30
|
|
at /tmp/hf-editframe-eval-run/editframe-smoke/node_modules/@editframe/cli/dist/commands/render.js:132:17
|
|
at async withProfiling (/tmp/hf-editframe-eval-run/editframe-smoke/node_modules/@editframe/cli/dist/utils/profileRender.js:5:31)</pre
|
|
>
|
|
<h3>Editframe creator flag mismatch</h3>
|
|
<p>
|
|
The help text advertises <code>--skip-skills</code>, but the parser rejects it in this
|
|
published version.
|
|
</p>
|
|
<pre>
|
|
> npx
|
|
> create-editframe html -d should-fail -y --skip-skills
|
|
|
|
node:internal/util/parse_args/parse_args:102
|
|
throw new ERR_PARSE_ARGS_UNKNOWN_OPTION(
|
|
^
|
|
|
|
TypeError [ERR_PARSE_ARGS_UNKNOWN_OPTION]: Unknown option '--skip-skills'. To specify a positional argument starting with a '-', place it at the end of the command after '--', as in '-- "--skip-skills"
|
|
at checkOptionUsage (node:internal/util/parse_args/parse_args:102:13)
|
|
at node:internal/util/parse_args/parse_args:376:9
|
|
at Array.forEach (<anonymous>)
|
|
at parseArgs (node:internal/util/parse_args/parse_args:373:3)
|
|
at main (file:///tmp/hf-eval-npm-cache/_npx/a680ecfe04f5894d/node_modules/@editframe/create/dist/index.js:104:34) {
|
|
code: 'ERR_PARSE_ARGS_UNKNOWN_OPTION'
|
|
}
|
|
|
|
Node.js v22.22.2
|
|
npm error code 1
|
|
npm error path /tmp/hf-editframe-eval-run
|
|
npm error command failed
|
|
npm error command sh -c create-editframe html -d should-fail -y --skip-skills
|
|
npm error A complete log of this run can be found in: /tmp/hf-eval-npm-cache/_logs/2026-04-30T19_01_19_210Z-debug-0.log
|
|
</pre
|
|
>
|
|
<h2>Conclusions</h2>
|
|
<ol>
|
|
<li>
|
|
<strong
|
|
>For devbox-local MP4 rendering, HyperFrames is currently the safer default.</strong
|
|
>
|
|
It completed the exact CLI path users would run and produced a standard H.264 MP4.
|
|
</li>
|
|
<li>
|
|
<strong
|
|
>Editframe has a strong composition/editor architecture, but local rendering is
|
|
codec-sensitive.</strong
|
|
>
|
|
Its browser API rendered fast with VP9, but the published CLI does not expose a codec
|
|
override and failed on AVC here.
|
|
</li>
|
|
<li>
|
|
<strong>Editframe's public API/cloud story is stronger on paper.</strong> This run did not
|
|
authenticate to cloud render, so that remains source-evidence only, not live proof.
|
|
</li>
|
|
<li>
|
|
<strong>Do not use this report to claim better final video taste.</strong> The fixture was
|
|
intentionally controlled and no blind judges scored visual quality.
|
|
</li>
|
|
<li>
|
|
<strong>The eval suite should keep both tracks:</strong> default user path, and
|
|
expert/fallback path. A tool that can render only after hidden codec intervention should
|
|
not get the same reliability score as one whose default command works.
|
|
</li>
|
|
</ol>
|
|
<h2>Artifacts</h2>
|
|
<ul>
|
|
<li>
|
|
<a href="../artifacts/devbox-2026-04-30/hyperframes/index.html"
|
|
>HyperFrames fixture source</a
|
|
>
|
|
</li>
|
|
<li>
|
|
<a href="../artifacts/devbox-2026-04-30/editframe/index.html">Editframe fixture source</a>
|
|
</li>
|
|
<li><a href="devbox-2026-04-30-results.json">Machine-readable results JSON</a></li>
|
|
</ul>
|
|
</main>
|
|
</body>
|
|
</html>
|