fix(skills): pin UTF-8 in Python scripts instead of the platform code page (#3298)

Windows sizes Python's stdio and text-mode file IO to the ANSI code page
(cp1252), not UTF-8. Every skill Python script relied on that default:

  * analyze-beatgrid.py --print writes the glyphs cp1252 has no slot for
    (delta, arrow), so the brief died with UnicodeEncodeError on every Windows
    run — the reported crash;
  * its audiomap write_text() pairs ensure_ascii=False with the default file
    encoding, so a non-ASCII payload is unwritable there too;
  * lint_source.py read_text() raises UnicodeDecodeError before any rule runs
    when a Remotion source carries an em dash or a curly quote;
  * gen-stroke-path.py reads an SVG font whose glyph keys ARE literal
    characters, so a mis-decoded key stops matching the requested text.

Stdio is reconfigured to UTF-8 at import and every text-mode IO call names its
encoding. `errors` is carried across the reconfigure: it resets to "strict",
and CPython gives stderr "backslashreplace" on purpose so the diagnostic path
can never itself raise.

extract-audio-data.py also decoded ffmpeg's stderr strictly while reporting a
failure, which would bury the very error being reported on a Windows ffmpeg.

skills/python-encoding.test.mjs guards the class: it fails if any skill Python
script drops the stdio block or omits encoding= on a text-mode IO call. The
mode is read as a whole comma-delimited argument of mode characters only, so a
payload key like {"bpm": 120} cannot spell the check away.

Verified with a cp1252 stdio stream installed before module load, matching how
Windows starts the interpreter: pre-fix UnicodeEncodeError, post-fix both
glyphs present in the UTF-8 bytes. Not run on real Windows hardware.
This commit is contained in:
Miguel Ángel
2026-08-17 21:44:08 -04:00
committed by GitHub
parent ad84b00c90
commit f8a1e2d315
7 changed files with 172 additions and 10 deletions
@@ -35,6 +35,15 @@ import librosa
import numpy as np
import soundfile as sf
# Windows sizes stdio to the ANSI code page (cp1252), which cannot encode the glyphs
# the brief prints (Δ, →) — every `--print` run died with UnicodeEncodeError. These
# scripts emit UTF-8 on every platform; say so instead of trading the glyphs away.
# Carry `errors` across: reconfigure() resets it to "strict", and CPython deliberately
# gives stderr "backslashreplace" so the diagnostic path can never itself raise.
for _stream in (sys.stdout, sys.stderr):
if hasattr(_stream, "reconfigure"):
_stream.reconfigure(encoding="utf-8", errors=_stream.errors)
SR = 22050
HOP = 512 # ~23 ms frames
AUDIOMAP_VERSION = 2
@@ -517,7 +526,9 @@ def main() -> None:
a = ap.parse_args()
d = analyze(a.audio, phrase_bars=a.phrase_bars)
if a.out:
Path(a.out).write_text(json.dumps(d, ensure_ascii=False, indent=2))
# ensure_ascii=False means the payload can carry non-ASCII, so the file
# encoding cannot be left to the platform default (cp1252 on Windows).
Path(a.out).write_text(json.dumps(d, ensure_ascii=False, indent=2), encoding="utf-8")
dens = " ".join(f"{s.get('level', '?')}:{s.get('density', '?')}" for s in d.get("energy_phases", []))
print(
f"[analyze-beatgrid] wrote audiomap {a.out} · {len(d.get('energy_phases', []))} phases · density [{dens}]",