mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-03 04:38:33 +00:00
fix(skills): pin UTF-8 in Python scripts instead of the platform code page (#3298)
Windows sizes Python's stdio and text-mode file IO to the ANSI code page
(cp1252), not UTF-8. Every skill Python script relied on that default:
* analyze-beatgrid.py --print writes the glyphs cp1252 has no slot for
(delta, arrow), so the brief died with UnicodeEncodeError on every Windows
run — the reported crash;
* its audiomap write_text() pairs ensure_ascii=False with the default file
encoding, so a non-ASCII payload is unwritable there too;
* lint_source.py read_text() raises UnicodeDecodeError before any rule runs
when a Remotion source carries an em dash or a curly quote;
* gen-stroke-path.py reads an SVG font whose glyph keys ARE literal
characters, so a mis-decoded key stops matching the requested text.
Stdio is reconfigured to UTF-8 at import and every text-mode IO call names its
encoding. `errors` is carried across the reconfigure: it resets to "strict",
and CPython gives stderr "backslashreplace" on purpose so the diagnostic path
can never itself raise.
extract-audio-data.py also decoded ffmpeg's stderr strictly while reporting a
failure, which would bury the very error being reported on a Windows ffmpeg.
skills/python-encoding.test.mjs guards the class: it fails if any skill Python
script drops the stdio block or omits encoding= on a text-mode IO call. The
mode is read as a whole comma-delimited argument of mode characters only, so a
payload key like {"bpm": 120} cannot spell the check away.
Verified with a cp1252 stdio stream installed before module load, matching how
Windows starts the interpreter: pre-fix UnicodeEncodeError, post-fix both
glyphs present in the UTF-8 bytes. Not run on real Windows hardware.
This commit is contained in:
@@ -10,10 +10,21 @@ Prints: the path `d` string + layout info on stderr.
|
||||
"""
|
||||
import re, sys
|
||||
|
||||
# Windows sizes stdio to the ANSI code page (cp1252). These scripts emit UTF-8 on
|
||||
# every platform; say so rather than depending on the console's code page. Carry
|
||||
# `errors` across: reconfigure() resets it to "strict", and CPython deliberately gives
|
||||
# stderr "backslashreplace" so the diagnostic path can never itself raise.
|
||||
for _stream in (sys.stdout, sys.stderr):
|
||||
if hasattr(_stream, "reconfigure"):
|
||||
_stream.reconfigure(encoding="utf-8", errors=_stream.errors)
|
||||
|
||||
font_path, text, target_w, baseline_y, x0 = (
|
||||
sys.argv[1], sys.argv[2], float(sys.argv[3]), float(sys.argv[4]), float(sys.argv[5]))
|
||||
|
||||
svg = open(font_path).read()
|
||||
# SVG fonts are UTF-8 and each glyph's `unicode="…"` attribute IS a literal character.
|
||||
# Decoded with the platform default, a non-ASCII glyph key silently becomes the wrong
|
||||
# character (or raises) on Windows, so that glyph never matches the requested text.
|
||||
svg = open(font_path, encoding="utf-8").read()
|
||||
glyphs = {}
|
||||
for m in re.finditer(r'<glyph\s+unicode="(.)"[^>]*?horiz-adv-x="([\d.]+)"(?:[^>]*?d="([^"]*)")?', svg):
|
||||
ch, adv, d = m.group(1), float(m.group(2)), m.group(3) or ""
|
||||
|
||||
Reference in New Issue
Block a user