* ci(regression): compute the shard matrix from recorded fixture timings
* ci(regression): refresh shard timings from a green post-PSNR run
* fix(ci): close two silent-skip holes in the shard schedule contract
* ci(regression): schedule the new static-volume-future-set fixture
* test(producer): regenerate static-volume-future-set golden in the pinned container
* fix(producer): accept partial color metadata in plan v2
* test(engine): make partial color probe hermetic
* fix(producer): validate plan v2 sentinels in fallback mode
## What
- model expected video-frame counts using the same rounding contract as the extraction branch:
- CFR `-vf fps=<fps>`: nearest output-frame boundary
- VFR `-fps_mode cfr -r <fps>`: ceil
- retain fail-closed ceil behavior when extraction metadata is missing
- keep positive sub-frame clips at a minimum of one expected frame
- preserve the existing 95% truncation gate and source-duration credit
## Why
The coverage gate universally used `ceil(duration * fps)`, but FFmpeg's CFR fps filter rounds to the nearest boundary. This made successfully extracted short CFR clips such as 0.616666s at 30 fps look truncated (18 captured vs 19 expected).
In the dashboard window, 124 of 335 video-coverage failures were exactly one frame short. Historical logs do not include `isVFR`, so that is the maximum addressable cohort rather than a guaranteed reduction. Zero-frame and materially truncated extraction failures remain fail-closed.
## Safety
- VFR still uses ceil and the existing strict coverage threshold.
- Missing extraction metadata still uses ceil.
- No retry, fallback, Temporal workflow, or render-plan behavior changes.
- Intended rollout is through the producer sidecar canary with explicit internal in-process jobs before dev and production promotion.
## Test
- `bunx vitest run packages/producer/src/services/render/videoFrameCoverage.test.ts` (28 passed)
- `bun run --filter @hyperframes/producer typecheck`
- `bunx oxlint ...videoFrameCoverage.ts ...videoFrameCoverage.test.ts`
- `bunx oxfmt --check ...videoFrameCoverage.ts ...videoFrameCoverage.test.ts`
- `bunx fallow audit --base origin/main --fail-on-issues`
- repository pre-commit gates
## Summary
- classify per-source video download/probe/decode/extraction failures with a bounded taxonomy and safe producer-facing summaries
- add candidate-only, at-most-one transient retry with cleanup and retry telemetry
- preserve default engine/producer behavior when the policy is off
- carry allowlisted extraction error codes through blocking JSON and SSE responses
## Stack
Depends on #2774 for atomic remote downloads and its single owned download retry. This PR is intentionally based on `fix/atomic-video-download-retry`; rebase/change the base to `main` after #2774 merges.
## Default compatibility
`HF_VIDEO_EXTRACTION_FAILURE_MODE` defaults to `off` and forces `maxTransientRetries=0`.
With the feature off:
- metadata probe failures keep the legacy Promise rejection
- grouped extraction keeps the existing grouped-to-direct fallback
- no new producer failure gate is enforced
- render-plan schema, Plan v1 artifacts, chunk routing, and distributed execution are unchanged
Typed metadata aggregation is explicit and enabled only by the candidate enforce lane.
## Retry ownership
- remote downloads: exactly one retry owned by #2774
- metadata/FFmpeg extraction: at most one retry only when `HF_VIDEO_EXTRACTION_MAX_RETRIES=1`
- invalid, missing, rejected, out-of-range, zero-output, cancellation, and unknown/internal failures do not retry
- non-finite or invalid runtime retry budgets fail closed to zero
- the superset optimization is never retried; on failure it preserves direct-member fallback, and only the individual ranges can use the bounded retry
- retry counters increment when a retry is scheduled, including exhausted retries
The internal sidecar and Experiment Framework must treat both exhausted stage codes as workflow-terminal after the producer-local budget. Candidate enforcement must not be enabled until those companion mappings are deployed, or Temporal can multiply producer attempts.
## Failure contract
- `VIDEO_SOURCE_UNRENDERABLE`: at least one deterministic/unknown source failure
- `VIDEO_EXTRACTION_FAILED`: all source failures are transient but the producer-local budget is exhausted
Only the allowlisted code and kind/count summaries cross JSON/SSE. Raw diagnostics remain engine-local because they may contain signed URLs or local paths.
## Rollout
1. merge and deploy with stable/candidate both `off`
2. candidate `observe`, retries 0
3. candidate `observe`, retries 1
4. deploy internal + EF terminal transport mappings
5. candidate `enforce`, retries 1
6. keep stable off until success delta, retry counts, extraction latency, CPU/disk, and queue backlog are acceptable
## Validation
- engine focused suites: 105 passed
- producer focused suites: 15 passed
- full engine suite: 1,176 passed, 3 skipped
- full producer unit lane: 32 Vitest files / 393 tests plus all classified Bun unit tests
- engine and producer typechecks passed
- oxlint, oxfmt, Fallow, tracked-artifact, and commit hooks passed
- independent review: approved for merge default-off; candidate enforcement held on companion transport rollout
## Summary
- stage remote video downloads in private per-attempt directories and atomically publish only complete non-empty files
- keep the deadline active through response-body streaming and retry one bounded transient failure
- preserve render cancellation without sharing abort ownership across independent callers
- manually follow at most five redirects, validating HTTPS/public-host policy before every hop
- remove stale zero-byte finals and avoid a permanent render-scoped cache map
## Root cause
The previous downloader wrote directly to the final cache path and cleared its timeout as soon as response headers arrived. A body timeout or mid-stream socket reset could therefore leave a truncated file at a path that later extraction treated as complete. The downstream symptom was zero extracted frames followed by a generic video coverage failure.
## Retry policy
Exactly one retry is allowed for 408, 429, 5xx, timeouts, empty successful bodies, and network/socket failures including nested Undici errors. Cancellation, 404/410, other 4xx, URL/redirect validation errors, and filesystem errors are not retried.
## Security and portability
- `mkdtempSync` creates an unguessable same-filesystem staging directory; exclusive writes plus atomic rename prevent symlink planting and partial publication
- redirects use `redirect: "manual"` and every resolved `Location` is revalidated before the next request, blocking redirect-to-private/IMDS bypasses
- the partial file is opened read/write for `fsync`, which preserves flush semantics and avoids Windows `EPERM`
## Rollout safety
This does not change render-plan schema, Plan v1 artifacts, chunk routing, or distributed rendering semantics. It is suitable for the candidate sidecar lane first; stable can remain pinned while we compare video extraction and coverage failures.
## Validation
- focused urlDownloader suite: 28 passed
- full engine suite: 1,169 passed, 3 skipped
- engine typecheck passed
- oxlint and oxfmt passed
- fallow audit passed
- independent code review: approved, no blockers
## What
- add an exact, symlink-safe plan-size breakdown with fixed or hashed component labels
- reject oversized plans after the compiled tree stabilizes, before video extraction
- reject again after artifact promotion but before the full freeze/hash read
- retain the exact post-freeze ceiling check and existing `PLAN_TOO_LARGE` contract/message prefix
- remove only freeze-owned stale metadata before the preliminary check when callers reuse a plan directory
## Why
The existing ceiling only runs after a multi-GiB plan has been fully materialized and hashed. Production examples range from roughly 3 GiB to 12.6 GiB. This gives us enough attribution to design Plan v2 from real data while avoiding obviously wasted extraction and hashing work.
## Compatibility and rollout
- no v1 plan artifact or `plan.json` format changes
- no queue, routing, retry, worker-count, or steady-state render changes
- under-limit plan bytes/hashes remain governed by the existing freeze path
- intended for the candidate sidecar and canary lane before broader rollout
This does not interrupt a single FFmpeg extraction once it is running. A canaried extraction-budget monitor is a separate follow-up.
## Validation
- focused distributed-plan tests: 54 passed
- full producer unit lane: 31 Vitest files / 384 tests plus all classified Bun lanes passed
- typecheck, lint, format, Fallow, tracked-artifact checks passed
- independent code review approved
## What
- publishes Plan v2 content-addressed artifacts directly from the Cloud Run planner's private staging directory to GCS
- commits the Plan v2 manifest only after every referenced artifact is durable
- removes the second local Plan v2 CAS directory and preserves the existing Cloud Workflows wire contract
- exports a reusable GCS publisher adapter
## Why
Cloud Run planner, chunk, and assembler requests are independent containers and do not share a filesystem. The distributed contract must contain only durable GCS locators.
This layer makes the Plan v2 publication path object-store-native. Like the AWS parent PR, it still relies on one planner-local frozen v1 tree inside the producer; eliminating that remaining staging tree requires direct Plan v2 emission in a later layer.
## Design invariants
- local paths never cross a request or worker boundary
- manifest and artifact locators derive from one validated GCS output prefix
- immutable objects use generation-zero conditional creation and exact digest/size verification
- retries may reuse exact objects but never overwrite conflicts
- the manifest is the final publication commit point
- chunk and assembler requests independently download and verify only their target artifacts
## Test plan
- [x] all 95 GCP Cloud Run package tests pass
- [x] package typecheck passes
- [x] package build passes
- [x] changed-file lint, format, fallow, and repository commit gates pass
- [x] end-to-end adapter test covers plan, target-scoped chunk, and assemble through Fake GCS with no shared local directory
## What
Normalize both padded and trimmed render audio on the decoded sample timeline, encode the result as AAC in M4A, and keep the no-op path on stream copy. Final video muxing remains stream-copy only.
## Why
AAC packet-copy operations cannot guarantee the requested presentation duration. Trimming could retain a packet-boundary tail, while padding by concatenating a separately encoded raw-ADTS silence segment could create a timestamp/bitrate discontinuity that surfaced as roughly 606 ms of apparent audio drift in the production-style regression.
## How
- Trim with `atrim` plus `asetpts`, then AAC-encode into M4A.
- Pad with `apad=whole_dur`, cap the output at the target duration, and AAC-encode the continuous sample timeline into M4A.
- Preserve the M4A encoder-delay edit list when the normalized audio is copied into the final MP4.
- Use the normalized M4A path in both local and distributed assembly; leave already-correct audio unchanged.
- Remove the obsolete raw-ADTS concat helpers and the disproven final-mux duration workaround.
## Test plan
- [x] 18 focused pad/trim unit tests
- [x] Real-media integration regression covering the packet-timeline boundary
- [x] Producer typecheck and repository commit hooks
- [x] Faithful Docker `style-3-prod` regression: video/audio approximately 16.07 s, 1 ms drift, zero visual failures
- [x] Exact-head required CI was green before the review-cleanup commit; fresh exact-head CI is running
- [ ] Documentation updated (not applicable)
The regression was reproduced before the fix with 0.563416 s of reported audio/video drift. The sample-timeline normalization removes the malformed tail while preserving stream-copy final muxing and preview behavior.
James