fix(skills): pin UTF-8 in Python scripts instead of the platform code page (#3298)

Windows sizes Python's stdio and text-mode file IO to the ANSI code page
(cp1252), not UTF-8. Every skill Python script relied on that default:

  * analyze-beatgrid.py --print writes the glyphs cp1252 has no slot for
    (delta, arrow), so the brief died with UnicodeEncodeError on every Windows
    run — the reported crash;
  * its audiomap write_text() pairs ensure_ascii=False with the default file
    encoding, so a non-ASCII payload is unwritable there too;
  * lint_source.py read_text() raises UnicodeDecodeError before any rule runs
    when a Remotion source carries an em dash or a curly quote;
  * gen-stroke-path.py reads an SVG font whose glyph keys ARE literal
    characters, so a mis-decoded key stops matching the requested text.

Stdio is reconfigured to UTF-8 at import and every text-mode IO call names its
encoding. `errors` is carried across the reconfigure: it resets to "strict",
and CPython gives stderr "backslashreplace" on purpose so the diagnostic path
can never itself raise.

extract-audio-data.py also decoded ffmpeg's stderr strictly while reporting a
failure, which would bury the very error being reported on a Windows ffmpeg.

skills/python-encoding.test.mjs guards the class: it fails if any skill Python
script drops the stdio block or omits encoding= on a text-mode IO call. The
mode is read as a whole comma-delimited argument of mode characters only, so a
payload key like {"bpm": 120} cannot spell the check away.

Verified with a cp1252 stdio stream installed before module load, matching how
Windows starts the interpreter: pre-fix UnicodeEncodeError, post-fix both
glyphs present in the UTF-8 bytes. Not run on real Windows hardware.
This commit is contained in:
Miguel Ángel
2026-08-17 21:44:08 -04:00
committed by GitHub
parent ad84b00c90
commit f8a1e2d315
7 changed files with 172 additions and 10 deletions
@@ -10,10 +10,21 @@ Prints: the path `d` string + layout info on stderr.
"""
import re, sys
# Windows sizes stdio to the ANSI code page (cp1252). These scripts emit UTF-8 on
# every platform; say so rather than depending on the console's code page. Carry
# `errors` across: reconfigure() resets it to "strict", and CPython deliberately gives
# stderr "backslashreplace" so the diagnostic path can never itself raise.
for _stream in (sys.stdout, sys.stderr):
if hasattr(_stream, "reconfigure"):
_stream.reconfigure(encoding="utf-8", errors=_stream.errors)
font_path, text, target_w, baseline_y, x0 = (
sys.argv[1], sys.argv[2], float(sys.argv[3]), float(sys.argv[4]), float(sys.argv[5]))
svg = open(font_path).read()
# SVG fonts are UTF-8 and each glyph's `unicode="…"` attribute IS a literal character.
# Decoded with the platform default, a non-ASCII glyph key silently becomes the wrong
# character (or raises) on Windows, so that glyph never matches the requested text.
svg = open(font_path, encoding="utf-8").read()
glyphs = {}
for m in re.finditer(r'<glyph\s+unicode="(.)"[^>]*?horiz-adv-x="([\d.]+)"(?:[^>]*?d="([^"]*)")?', svg):
ch, adv, d = m.group(1), float(m.group(2)), m.group(3) or ""