ci: bound the ffmpeg apt fetch so a stalled mirror costs a retry, not the job (#3356)

* ci: bound the ffmpeg apt fetch so a stalled mirror costs a retry, not the job

Hosted runners intermittently stall on an apt mirror, and an unbounded
apt-get inherits the whole job budget. The producer integration lane normally
finishes in ~11 minutes against a 20 minute cap; on a stalled fetch it ran to
the cap and failed. Same step, same shape, reproduces on main's tip — it is not
specific to any one PR.

The cost is not one red check. On the run that prompted this, four went red off
that single step: the two jobs that install ffmpeg, plus a Test gate and a
preview-regression gate that both fail closed when their dependency does not
succeed. So a mirror stall reads as a producer defect and a preview defect.

Each attempt is now bounded and retried three times, and the five workflows
that installed ffmpeg share one action instead of five copies of the command.
Deliberately still apt: caching the binary would strip it from the shared
libraries it links against, and switching to a static build would change the
ffmpeg under the producer's output comparisons. Neither belongs in a fix for a
network stall.

* ci: drop the stray version echo left in the player-perf ffmpeg step

Converting the step to the shared action left the trailing `ffmpeg -version`
line behind, and YAML folded it into the `uses:` value — so the runner looked
for an action at a path with the command appended and failed all four perf
shards.

It parsed cleanly, which is why validating with a YAML load did not catch it:
`uses: ./path\n  ffmpeg -version` is a legal folded scalar. The check that
does catch it asserts every local `uses:` resolves to a directory containing
an action file, which is now what I ran. The action prints the version itself.

* ci: bound the ffmpeg fetch at the connection, not with a wall-clock kill

The first version wrapped apt in `timeout` and retried. A passing run showed
why that is the wrong shape: the mirror is slow rather than hung — the install
spent ~15 minutes pulling packages from azure.archive.ubuntu.com and finished
successfully. Killing it at 300s discarded a download that was making progress
and started over, so the retry turned a slow mirror into a slower one, and the
worst case of three attempts exceeded the job's own 20 minute cap.

Bound the connection instead. Acquire::Retries re-fetches the one package whose
connection stalled while keeping everything already downloaded, and
Acquire::http::Timeout caps how long any single connection may sit idle. That
addresses the stall the original report described without punishing the slow
case that is far more common.
This commit is contained in:
Miguel Ángel
2026-08-19 18:39:05 -04:00
committed by GitHub
parent d464f60b96
commit 7e96e60fe2
5 changed files with 39 additions and 12 deletions
+1 -1
View File
@@ -66,7 +66,7 @@ jobs:
# file copy), so the job never needed it and ubuntu-latest does not ship
# it.
- name: Install ffmpeg
run: sudo apt-get update -qq && sudo apt-get install -y --no-install-recommends ffmpeg
uses: ./.github/actions/install-ffmpeg-linux
- name: Render changed block/component previews
env:
+2 -6
View File
@@ -322,9 +322,7 @@ jobs:
node-version: 22
- name: Install FFmpeg for integration tests
if: matrix.lane == 'integration'
run: |
sudo apt-get update -qq
sudo apt-get install -y --no-install-recommends ffmpeg
uses: ./.github/actions/install-ffmpeg-linux
- uses: ./.github/actions/prepare-ffmpeg-bin
- run: bash scripts/ci/install-workspace-dependencies.sh
- run: bun run --filter '@hyperframes/{parsers,lint,studio-server}' build
@@ -733,9 +731,7 @@ jobs:
with:
node-version: 22
- name: Install FFmpeg
run: |
sudo apt-get update
sudo apt-get install -y ffmpeg
uses: ./.github/actions/install-ffmpeg-linux
- uses: ./.github/actions/prepare-ffmpeg-bin
- name: Install dependencies
run: bash scripts/ci/install-workspace-dependencies.sh
+1 -4
View File
@@ -105,10 +105,7 @@ jobs:
# to just install it here so the shard never trips on infra.
- name: Install ffmpeg (parity shard only)
if: matrix.shard == 'parity'
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends ffmpeg
ffmpeg -version | head -n 1
uses: ./.github/actions/install-ffmpeg-linux
- name: Run player perf — ${{ matrix.shard }} (measure mode)
working-directory: packages/player
+1 -1
View File
@@ -93,7 +93,7 @@ jobs:
run: bun run --cwd packages/producer parity:fixtures:ci
- name: Install ffmpeg
run: sudo apt-get update -qq && sudo apt-get install -y --no-install-recommends ffmpeg
uses: ./.github/actions/install-ffmpeg-linux
- name: Set up Chrome
id: setup-chrome