Commit Graph
8 Commits
Author SHA1 Message Date
Miguel Ángel f55990ebca fix(cli): name the network cause of a failed fetch in the error box (#7236) 2026-11-05 13:08:09 +00:00
Miguel Ángel 11f43268a1 fix(cli): name the refused file and parameter when a cloud upload is rejected (#7155)
* fix(cli): name the refused file and parameter when a cloud upload is rejected

* fix(cli): blame the file type only for type refusals at upload complete
2026-11-04 11:21:44 +00:00
Miguel Ángel b306438e8b feat(cli): cloud render uploads heavy media as HeyGen assets (#5931)
* feat(cli): cloud render uploads heavy media as HeyGen assets

Media of 5 MB or more leaves the project zip: it uploads through the
direct-upload flow as a HeyGen asset and the zip's HTML and CSS point at
the asset URL. Uploads are cached by content hash and re-checked before
reuse. Media the asset API refuses or caps is re-encoded at visually
lossless quality first. The zip must end under 200 MB or the command
stops and names the largest files left.

* feat(cli): cloud render offloads GIFs, still images and script-referenced media

Animated GIFs in the entry composition go through the renderer's own GIF
preparation and upload as the resulting video. Still AVIF, GIF, BMP and
TIFF images convert to PNG. Plain string literals in scripts and JSON that
name offloaded media point at the asset URL. ffprobe calls end their
options with "--", and two comment lines are trimmed for the ratchet.

* feat(cli): cloud render uploads only the parts of long media a composition plays

A video or audio file that has to be re-encoded is cut to the union of the
windows its clips play, with half a second of margin on the source's frame
grid, and the staged composition's media starts shift to match. Clips in
sub-compositions, looping clips, speed ramps and files a script names keep
the full file.

* fix(cli): cloud render no longer decodes media files as text when rewriting references

* fix(cli): cloud render keeps media chosen at render time and encodes the staged bytes

Only media a reference points at is offloaded, so a file picked through
--variables or built by a script stays in the zip. Variable-bound clips and
clips with a media fragment keep their full length. Entry GIFs resolve the way
the renderer rebases a nested entry and are prepared only from 5 MB, and every
encode reads the staged bytes its cache key hashes. Query strings and media
fragments survive the rewrite, a null cache file reads as empty, and the docs
say asset URLs are public and unsigned.

* fix(cli): cloud render comments cite only what this repo holds

* fix(cli): cloud render handles media over 2 GiB and explains a too-large upload

Media of 5 MB or more stays on disk until it is used: it is hashed and uploaded
as a stream, and a file edited after staging stops the run instead of caching
the wrong upload. Files left in the zip are sized before any is read, so a
4 GB file reports "too large" instead of crashing.

The asset API rejects an over-limit upload as invalid_parameter with "File too
large"; that now gets the dry-run hint instead of the --help one.

* fix(cli): cloud render checks and reads each staged file through one handle

Staging, the on-disk staged read and the re-encoded output read each opened the
path twice: once to stat, once to read. A file swapped or edited between the two
could be read without the size or change check seeing it. Each now opens once,
checks the open file's stat before and after the read, and reads from it.

* test(cli): a heavy file a script builds from split strings stays in the cloud render zip

* fix(cli): cloud render keeps media scripts may reach and never trims clips scripts retime

- An offloaded file keeps its zip copy when script code holds both its stem and
  extension, so a path joined at runtime still loads.
- Entry GIFs are prepared from a private copy checked against the staged file,
  so the GIF cache never stores a later edit.
- No cut when any script can write a clip's timing attributes, dataset,
  currentTime, playbackRate, src or loop.
2026-10-21 14:34:00 +00:00
James 3aa2404747 refactor(cli): centralize process lifecycle 2026-07-20 09:04:03 -07:00
James e73304fb0e feat(cli): make cloud archives size-aware 2026-07-17 18:44:58 -04:00
James RussoandClaude Opus 4.7 a59ff0d91b feat(cli): migrate cloud-render upload to /v3/assets/direct-uploads (200MB) (#1844)
* chore(cli): regenerate cloud client for createAssetUpload + completeAssetUpload

Regenerated from experiment-framework `master` at commit `e74815f7af` (the
merge of EF#41085, which added `/v3/assets/direct-uploads` +
`/v3/assets/{asset_id}/complete` to the `TARGET_ENDPOINTS` allowlist in
`scripts/generate_hyperframes_cli_client.py`).

The `sync-hyperframes-codegen.yml` workflow that normally auto-opens this
PR failed with a `gh: Not Found (HTTP 404)` on the PR-creation step (run
28556975483); regenerated manually with:

  cd experiment-framework
  PYTHONPATH=. python3 scripts/generate_hyperframes_cli_client.py \\
    --out /path/to/hyperframes-oss

This commit is codegen-only — no hand edits. The direct-upload wire-up
that consumes the new `createAssetUpload` + `completeAssetUpload` methods
lands in the follow-up commit.

— Jerrai

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(cli): migrate cloud-render upload to /v3/assets/direct-uploads (200MB)

Replaces the legacy `client.uploadAsset(...)` multipart POST to
`/v3/assets` (32 MB in-memory proxy path) with the three-step direct-to-
S3 flow that lifts the practical per-project ceiling to 200 MB:

  1. `POST /v3/assets/direct-uploads` — declares filename, content-type,
     size, and SHA256 checksum; returns `asset_id`, presigned
     `upload_url`, and required `upload_headers`.
  2. Raw `PUT` to `upload_url` with the zip bytes + `upload_headers`
     verbatim. No CLI auth attached — the presigned URL signature carries
     authorization, and any extra headers would break the signature.
  3. `POST /v3/assets/{asset_id}/complete` — finalizes into a reusable
     asset. Retried up to 5x on 409 ("Uploaded object not found yet"), a
     documented race between S3 write consistency and the finalize check.

The returned `asset_id` is the same namespace the legacy path produced
(both write into `movio_asset`), so the downstream render submission at
`createRender({project: {type: "asset_id", asset_id}})` is unchanged.

Server-side context (EF#41085): the direct-upload endpoint now accepts
`application/zip` via a scoped `_ZIP_MIME_TO_EXT` map — the shared media/
PDF allowlist stays zip-free. The exact-MIME cross-check at the sniff
step guards against zip<->PDF confusion under the shared 'document'
category. Canonical S3 key layout matches the legacy proxy path
(`document/{asset_id}/original.zip`), so the render-side head_object
gate is transparent to which upload path produced the asset.

The prior codegen commit added the generated createAssetUpload +
completeAssetUpload methods this commit consumes.

— Jerrai

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-07-02 16:01:08 -07:00
James Russo f53f4a7a08 fix(cli): drop misleading hint on hyperframes_project_invalid (#1127) 2026-05-29 18:22:32 -04:00
James Russo ce5e872e51 feat(cli): add hyperframes cloud render/list/get/delete commands (#1110)
* feat(cli): vendor initial hyperframes cloud client codegen

Generated by experiment-framework/scripts/generate_hyperframes_cli_client.py
(see heygen-com/experiment-framework#37896). Sets up the baseline for the
sync workflow to diff against on future spec changes.

The follow-up PR adds the orchestration layer (zip + upload + poll +
download) and the user-facing 'hyperframes cloud render/list/get/delete'
commands on top of this generated client.

The fallow ignore pattern is necessary because the generated request()
method is intentionally a single switch that handles all 5 endpoints
in one place; refactoring it here would just be re-introduced on the
next codegen run.

* chore(cli): regenerate cloud client with mimeType parameter on multipart uploads

Adds optional mimeType arg to uploadAsset (and any future multipart
endpoints). Without it, FormData sends application/octet-stream which
is correct for the documented media surface (png/jpeg/mp4/etc.) but
ambiguous for the private-beta zip uploads the cloud render flow uses.
Callers that pass `mimeType: "application/zip"` tag the multipart
part with the right Content-Type so downstream proxies, WAFs, and any
future server-side change that keys off the part MIME (instead of the
current magic-byte detection) all see the intended type.

Addresses review feedback on heygen-com/experiment-framework#37896.
Generated by scripts/generate_hyperframes_cli_client.py with the
matching update to the multipart emit path.

* feat(cli): add hyperframes cloud render/list/get/delete commands

Hand-rolled orchestration layer on top of the auto-generated cloud
client (vendored in the previous PR):

- cloud render <dir>: zip via createPublishArchive → upload to
  /v3/assets → submit /v3/hyperframes/renders → poll
  /v3/hyperframes/renders/{id} every 10s (max 60min) → stream the
  signed video_url to disk.
- cloud render --no-wait: submit and exit with the render_id.
- cloud render --asset-id / --url: skip zip+upload and use a
  pre-uploaded asset or public HTTPS zip.
- cloud render --variables / --variables-file: same UX as the local
  render command; variables are validated against
  data-composition-variables only when there's a local project.
- cloud list / cloud get / cloud delete: thin wrappers around the
  matching client methods, with cursor-pagination support on list.

Auth comes from the existing cli/src/auth/ chain via cloud/auth.ts —
no new credential store, no new env var. The cloud client receives a
getAuthHeaders() callback that re-resolves credentials on every
request, so OAuth refreshes mid-poll are picked up automatically.

Also extracts a parent-scoped path lookup in help.ts so 'cloud render
--help' surfaces the right examples instead of falling through to the
top-level 'render' command's examples.

* fix(cli): address 15 code-review findings on cloud commands

Correctness fixes
- delete: require --no-confirm when stdin isn't a TTY OR --json is
  passed; previously both silently auto-bypassed the irreversible-
  delete prompt. Explicit decline now exits 2 (distinct from API/system
  errors which still exit 1).
- render: mutex check now counts the positional dir alongside
  --asset-id / --url; `cloud render ./foo --asset-id X` now errors
  instead of silently dropping the dir.
- render: docstring updated — only --no-wait short-circuits the poll
  loop; --callback-url is independent (webhook fires either way).
- render: removed dead try/catch around resolveProject (it calls
  process.exit, never throws). resolveVariablesAndValidateIfLocal also
  takes the resolved project source instead of re-parsing args.
- render: createPublishArchive errors now surface via errorBox instead
  of bubbling a raw stack trace past citty.
- help: loadExamples now only catches ERR_MODULE_NOT_FOUND; real load
  errors (syntax error, broken import) propagate so a broken
  cloud/render.ts no longer silently shows the local render command's
  examples. Also skips the parent-scoped lookup when parentName is the
  root command ("hyperframes").
- list: fetchAll gained a 50-page safety cap + duplicate-cursor
  detection so a buggy backend serving the same next_token on a loop
  can't OOM the CLI.
- download: drain await now listens for error / close / abort so a
  failing write stream (ENOSPC, AbortSignal) rejects promptly instead
  of hanging forever. Partial files are unlinked on any error so the
  caller never observes a truncated MP4. content-length is verified
  against the actual byte count.
- poll: default sleep is abort-aware so Ctrl+C feels immediate instead
  of waiting out the full interval.
- pollWithProgress: ANSI carriage-return redraws now gated on
  process.stdout.isTTY — non-TTY runs (CI, file redirects) emit one
  line per status transition instead of polluting the log with
  literal escape codes.

Cloud client: 401-retry-with-refresh
- createCloudClient now wraps the generated client with a Proxy that
  catches HyperframesApiError(status=401), force-refreshes the OAuth
  token via forceRefreshCredentials, and retries the call exactly
  once. Mirrors AuthClient's onUnauthenticatedRefresh so server-side
  revocations and clock-skew rejections recover automatically.
- auth.ts gained forceRefreshCredentials() and now updates expires_at
  on the refreshed credential it returns (fixed stale-expiry race).

Shared helpers
- cloud/errors.ts: reportApiError(stage, err, opts) is the single
  error-funnel. ERROR_CODE_HINTS now applies to every subverb — fixes
  hyperframes_render_not_found being unreachable from get/delete and
  cuts ~70 LOC of duplicated try/catch/instanceof from render/list/
  get/delete.
- cloud/parsing.ts: parseIntFlag / parseNumericFlag / parseEnumFlag
  strict-mode parsers reject trailing garbage that Number.parseInt
  silently accepts.
- cloud/ansi.ts: stripAnsi / visibleLength / padEndVisible — covers
  ESC + 24-bit truecolor (c.accent palette) instead of the previous
  regex which undercounted overhead and missed truecolor.

JSON-output consistency + _meta envelope
- Every cloud subverb's --json output now goes through withMeta(...)
  so it carries the standard _meta envelope documented in cli.mdx.
- Single-render outputs use {render: detail} across get, delete,
  render-no-wait, render-failed, and render-success. list uses
  {renders: [...], has_more, next_token?}. delete adds deleted: true.

Tests
- 25 new tests across ansi.test.ts, parsing.test.ts, plus truncation
  + abort-cleanup tests for download.test.ts.
- 589 / 589 total CLI tests pass.

* fix(cli): address Vai's review on cloud commands

- render: pass mimeType: "application/zip" to uploadAsset so the
  multipart Content-Type is correct (was application/octet-stream).
  Server currently magic-byte-detects from file bytes so this is
  belt-and-suspenders today, but any downstream proxy / WAF / future
  server change that keys off the part MIME now sees the intended
  type instead of relying on detection.
- render: poll error path now surfaces "Resume with: hyperframes
  cloud get <renderId>" via reportApiError's new `suggestion`
  option, matching the PollTimeoutError handler. The server-side
  render keeps running through a transient 5xx; the user just
  needs the right command to pick it back up.
- list: fetchAll now errorBox-exits on the malformed
  {has_more: true, next_token: null} shape instead of silently
  returning a truncated list (matching the duplicate-cursor guard).
- download: closeFile now listens for 'error' on the write stream
  in addition to the end() callback, so a late ENOSPC during flush
  doesn't leak an unhandled error onto the stream and resolves the
  finally promptly.
- errors: reportApiError accepts an optional `suggestion` that's
  used as the errorBox third line when no code-specific hint
  matches — gives callers a place to surface always-actionable
  recovery context.
- docs(cli): document --idempotency-key as the safe-retry mechanism
  for the upload step. The 401-retry Proxy replays POST requests
  on a stale token; without an idempotency key, the upload may
  land twice. A UUID per logical render is the recommended pattern.
2026-05-28 14:10:05 -04:00