feat(skills): wake-word-training — teach Hermes to train custom wake words

Adds an optional skill so Hermes can build a custom openWakeWord model for a
phrase other than the bundled "hey hermes" (defaults to "hey <profile>"). The
skill drives openWakeWord's own trainer via the terminal tool — no heavy
training stack ships in core, and the existing resolver already loads the
resulting .onnx by path, so there are zero changes to the wake runtime.

- SKILL.md: end-to-end procedure (synthetic positives, augmentation/negatives,
  train, export into the profile's ~/.hermes/wakewords/, wire config, restart).
- scripts/generate_positives.py: OpenAI-TTS positives (default when
  OPENAI_API_KEY is set) with import-lazy synthesis; pure planning helpers.
- scripts/make_training_config.py: emits the openWakeWord training YAML.
- references/platforms.md: Colab fast path + macOS caveats + dataset sources.
- tests: frontmatter contract + pure helper logic, stdlib-only, no network.
- docs: point the wake-word "different phrase" section at the skill.

optional-skills/productivity/wake-word-training/SKILL.md

@@ -0,0 +1,142 @@
+---
+name: wake-word-training
+description: "Train a custom on-device wake word from synthetic speech."
+version: 1.0.0
+author: Brooklyn <brooklyn> & Hermes Agent
+license: MIT
+platforms: [linux, macos]
+metadata:
+  hermes:
+    tags: [Voice, Wake Word, openWakeWord, On-Device, Training]
+    category: productivity
+    related_skills: []
+---
+
+# Wake Word Training Skill
+
+Trains a custom openWakeWord model so Hermes answers to a phrase other than the
+bundled "hey hermes" — e.g. "hey morgane" or "hey <profile>". Detection stays
+100% on-device; only the optional voice synthesis (OpenAI TTS) touches the
+network, and only during training. This skill drives openWakeWord's own
+training pipeline through the `terminal` tool — it does not reimplement it.
+
+Custom models are trained on **synthetic speech**: you never record yourself.
+A TTS engine speaks the phrase a few thousand times across many voices, the
+clips are augmented with noise/room impulses, and a small classifier is trained
+on top of openWakeWord's frozen feature extractor. The output is a sub-100 KB
+`.onnx` the existing wake runtime loads with zero code changes.
+
+## When to Use
+
+- The user wants Hermes to wake on a different phrase ("hey morgane", a
+  nickname, "hey <profile>").
+- The user asks to "make/train my own wake word".
+
+Do **not** use this for the default "hey hermes" — that model already ships.
+For a phrase you already have an `.onnx` for, skip training and jump to
+**Wire It Up**.
+
+## Prerequisites
+
+- **Time & hardware.** Fast on Linux/GPU (~1 hr) or a free Colab GPU. On macOS
+  it is single-threaded and slow (~3–5 hr) — see `references/platforms.md`.
+  Prefer Colab if the user just wants the file quickly.
+- **Disk.** Several GB for augmentation/negative datasets (room impulses,
+  AudioSet/FMA noise, LibriSpeech negatives). Use a scratch dir, not the repo.
+- **Voices.** `OPENAI_API_KEY` for high-quality OpenAI TTS positives (~$0.04 a
+  run); otherwise the offline Piper generator openWakeWord ships with.
+- **A scratch training venv**, created ad hoc via `terminal` — never install the
+  heavy training stack into Hermes's own environment.
+
+## How to Run
+
+Confirm the phrase and where the model belongs, then train, then wire it up.
+Resolve the profile-aware model directory once and reuse it:
+
+```bash
+python -c "from hermes_constants import get_hermes_home; print(get_hermes_home() / 'wakewords')"
+```
+
+Default the phrase to `hey <profile>` (the active profile's name; `hey hermes`
+is the default profile) unless the user gives one. The model file is
+`<phrase-slug>.onnx` (lowercase, spaces → underscores), e.g. `hey_morgane.onnx`.
+
+## Quick Reference
+
+| Step | Command / action |
+| --- | --- |
+| Model dir | `get_hermes_home() / 'wakewords'` (profile-aware) |
+| Positives (OpenAI) | `scripts/generate_positives.py --phrase "hey morgane" --out-dir <dir>` |
+| Training config | `scripts/make_training_config.py --phrase "hey morgane" --out <cfg.yml>` |
+| Train | openWakeWord's `train.py --training_config <cfg.yml>` (in the scratch venv) |
+| Wire up | set `wake_word.openwakeword.model` + `phrase`, then restart the listener |
+
+## Procedure
+
+1. **Agree on the phrase and paths.** Slugify the phrase for the filename.
+   Resolve the model dir with the one-liner above and `mkdir -p` it.
+
+2. **Make a scratch training dir + venv** somewhere with room (not the repo):
+
+```bash
+mkdir -p /tmp/oww-train && cd /tmp/oww-train
+python -m venv .venv && . .venv/bin/activate
+pip install openwakeword
+```
+
+3. **Generate positives.**
+   - *OpenAI TTS (default when `OPENAI_API_KEY` is set):* run
+     `generate_positives.py` — it speaks the phrase across many voices with
+     small text/pacing variations and writes 16 kHz mono WAVs.
+   - *Offline:* let openWakeWord's own Piper generator produce them (the
+     training config's `target_phrase` drives it). See `references/platforms.md`.
+
+4. **Fetch augmentation + negative data.** Follow openWakeWord's automatic
+   training notebook (room impulses, AudioSet/FMA noise, LibriSpeech/precomputed
+   negatives). `references/platforms.md` has the exact dataset sources and the
+   Colab shortcut that downloads them for you.
+
+5. **Write the training config** with `make_training_config.py` (point it at the
+   positives/negatives/output dirs), then run openWakeWord's `train.py` against
+   it. Cross-check the emitted YAML against openWakeWord's `custom_model.yml` for
+   your installed version.
+
+6. **Place the model.** Copy the exported `<slug>.onnx` into the resolved
+   `wakewords` dir.
+
+7. **Wire It Up.** In the active profile's `~/.hermes/config.yaml`, set:
+   - `wake_word.openwakeword.model` → the absolute `.onnx` path
+   - `wake_word.phrase` → the human label ("hey morgane")
+   - `wake_word.enabled` → `true`
+   Then restart the listener so it reloads: `/wake` off then on, or restart the
+   app/gateway. (Wake config is not part of the prompt, so no cache concern.)
+
+## Pitfalls
+
+- **Don't install the training stack into Hermes's venv.** It's large and
+  macOS-flaky; keep it in the throwaway scratch venv.
+- **openWakeWord needs its base feature models** (melspectrogram + embedding)
+  for any model — the trainer downloads them; don't delete them.
+- **macOS bus errors** from openWakeWord's threadpool + mmap: train
+  single-threaded, or use Colab. See `references/platforms.md`.
+- **Config `model` must be an absolute path** (or a bundled/built-in name). A
+  bare filename won't resolve.
+- **Sensitivity.** If the new phrase over/under-triggers, tune
+  `wake_word.sensitivity` (0–1, higher = stricter) before retraining.
+
+## Verification
+
+- The `.onnx` exists in the profile's `wakewords` dir and is > 50 KB.
+- Smoke-test detection before enabling it live:
+
+```bash
+python - <<'PY'
+import wave, numpy as np
+from openwakeword.model import Model
+m = Model(wakeword_models=["/abs/path/hey_morgane.onnx"], inference_framework="onnx")
+a = np.frombuffer(wave.open("a_clip_of_the_phrase.wav").readframes(1<<20), np.int16)
+print("best:", max(max(m.predict(a[i:i+1280]).values()) for i in range(0, len(a)-1280, 1280)))
+PY
+```
+  A clear utterance should score well above `wake_word.sensitivity`.
+- After wiring up, `/wake` on and say the phrase — Hermes opens a fresh session.

optional-skills/productivity/wake-word-training/references/platforms.md

@@ -0,0 +1,59 @@
+# Platforms, Datasets & the Fast Path
+
+openWakeWord's automatic trainer is heavy (torch + a synthetic-speech +
+augmentation stack) and its speed depends entirely on where it runs. Pick the
+environment first, then follow the data steps.
+
+## Where to train
+
+| Environment | Time | Notes |
+| --- | --- | --- |
+| **Colab (free GPU)** | ~1 hr | Easiest. Recommended when the user just wants the `.onnx` quickly. |
+| Linux + CUDA GPU | ~1 hr | Full local control. |
+| Linux CPU | ~2 hr | Fine, just slower. |
+| **macOS** | ~3–5 hr | Single-threaded only — see the bus-error note below. |
+
+### Colab shortcut (recommended)
+
+openWakeWord ships an automatic-training notebook that installs the stack,
+downloads every dataset below, generates positives, trains, and exports the
+model — in one run on a free GPU:
+<https://colab.research.google.com/drive/1q1oe2zOyZp7UsB3jJiQ1IFn8z5YfjwEb>
+
+Set the target phrase (and, if using the OpenAI-TTS positives from this skill,
+upload the generated WAVs and point the config's positives dir at them), run it,
+download the `.onnx`, then follow **Wire It Up** in `SKILL.md`.
+
+### macOS caveat
+
+openWakeWord's data pipeline uses a threadpool over memory-mapped files that
+triggers bus errors on macOS. Train **single-threaded** (set the trainer's
+worker/`n_jobs` to 1) or, better, use Colab. Expect 3–5 hours locally.
+
+## Datasets the trainer needs
+
+Four kinds of data (openWakeWord's notebook pulls them from HuggingFace):
+
+1. **Room impulse responses** — `davidscripka/MIT_environmental_impulse_responses`
+   (reverb augmentation). → `./mit_rirs`
+2. **Background noise** — a slice of `agkphysics/AudioSet` (16 kHz). → `./audioset_16k`
+3. **False-positive / music negatives** — `rudraml/fma` (start with ~1 hr).
+   → `./fma`
+4. **Precomputed openWakeWord negative features** — from the openWakeWord
+   releases; used as generic negatives + validation for early stopping.
+
+Base feature models (`melspectrogram`, `embedding`) are fetched automatically by
+openWakeWord — don't delete them. Budget several GB of scratch space; keep a
+local copy to reuse across runs.
+
+## Positives: OpenAI TTS vs Piper
+
+- **OpenAI TTS** (`generate_positives.py`, default when `OPENAI_API_KEY` is set):
+  10 voices, small text/pacing variations, ~$0.04 per run, higher quality. Point
+  the training config's positives dir at the generated WAVs.
+- **Piper** (offline): openWakeWord's built-in generator synthesizes positives
+  from the config's `target_phrase`. Fully local, no key. Note: Piper's sample
+  generator is best-supported on Linux.
+
+More positives → better accuracy, with smooth diminishing returns. A few
+thousand is the practical floor; the notebook defaults to 5,000.

optional-skills/productivity/wake-word-training/scripts/generate_positives.py

@@ -0,0 +1,136 @@
+#!/usr/bin/env python3
+"""Generate synthetic positive samples of a wake phrase for openWakeWord.
+
+OpenAI TTS (``gpt-4o-mini-tts``) speaks the phrase across many voices with small
+text/pacing variations, resampled to the 16 kHz mono WAV openWakeWord trains on.
+This is the higher-quality alternative to openWakeWord's built-in Piper
+generator; use it when ``OPENAI_API_KEY`` is set.
+
+Heavy deps (openai, soundfile, numpy, scipy) are imported lazily inside the
+synthesis path so the pure planning helpers stay import-light and unit-testable.
+"""
+
+from __future__ import annotations
+
+import argparse
+import re
+
+# OpenAI's standard TTS voice set — cycling these gives speaker diversity, which
+# is what drives custom wake-word robustness (openWakeWord's own models lean on
+# exactly this kind of multi-voice synthetic data).
+OPENAI_VOICES = (
+    "alloy", "ash", "ballad", "coral", "echo",
+    "fable", "onyx", "nova", "sage", "shimmer",
+)
+
+TARGET_SAMPLE_RATE = 16_000
+
+
+def slugify(phrase: str) -> str:
+    """`"Hey Morgane!"` → `"hey_morgane"` — the model filename stem."""
+    s = re.sub(r"[^a-z0-9]+", "_", phrase.strip().lower())
+    return s.strip("_") or "wake_word"
+
+
+def build_variations(phrase: str) -> list[str]:
+    """Small textual variations so the TTS doesn't render one frozen prosody."""
+    core = phrase.strip().rstrip(".!?,")
+    # Punctuation nudges cadence/intonation; duplicates are dropped by dict order.
+    seen = {core: None, f"{core}.": None, f"{core}!": None, f"{core}...": None}
+    return list(seen)
+
+
+def voice_list(engine: str = "openai") -> list[str]:
+    if engine == "openai":
+        return list(OPENAI_VOICES)
+    raise ValueError(f"unknown engine: {engine!r}")
+
+
+def plan_clips(phrase: str, count: int, voices: list[str]) -> list[tuple[str, str, str]]:
+    """Deterministic (voice, text, filename) plan for *count* clips.
+
+    Voices and text variations are cycled independently so the set stays evenly
+    balanced across speakers regardless of *count*. Filenames are zero-padded and
+    unique, ready to drop into openWakeWord's positive-samples directory.
+    """
+    if count < 1:
+        return []
+    if not voices:
+        raise ValueError("need at least one voice")
+    variations = build_variations(phrase)
+    stem = slugify(phrase)
+    width = max(4, len(str(count - 1)))
+    plan = []
+    for i in range(count):
+        voice = voices[i % len(voices)]
+        text = variations[i % len(variations)]
+        plan.append((voice, text, f"{stem}_{i:0{width}d}.wav"))
+    return plan
+
+
+def _resample_to_16k_mono(pcm, src_rate: int):
+    """int16 mono PCM at *src_rate* → int16 mono PCM at 16 kHz (lazy scipy)."""
+    import numpy as np
+    from scipy.signal import resample
+
+    if src_rate == TARGET_SAMPLE_RATE:
+        return pcm
+    n = round(len(pcm) * TARGET_SAMPLE_RATE / src_rate)
+    out = resample(pcm.astype("float32"), n)
+    return np.clip(out, -32768, 32767).astype("int16")
+
+
+def _synthesize(plan, out_dir, api_key):  # pragma: no cover - network/heavy deps
+    """Call OpenAI TTS for each planned clip and write a 16 kHz mono WAV."""
+    import io
+    import wave
+    from pathlib import Path
+
+    import numpy as np
+    import soundfile as sf
+    from openai import OpenAI
+
+    client = OpenAI(api_key=api_key)
+    out = Path(out_dir)
+    out.mkdir(parents=True, exist_ok=True)
+    for voice, text, name in plan:
+        resp = client.audio.speech.create(
+            model="gpt-4o-mini-tts", voice=voice, input=text, response_format="wav"
+        )
+        data, rate = sf.read(io.BytesIO(resp.read()), dtype="int16")
+        if getattr(data, "ndim", 1) == 2:  # stereo → mono
+            data = data.mean(axis=1).astype("int16")
+        data = _resample_to_16k_mono(np.asarray(data), rate)
+        with wave.open(str(out / name), "wb") as w:
+            w.setnchannels(1)
+            w.setsampwidth(2)
+            w.setframerate(TARGET_SAMPLE_RATE)
+            w.writeframes(np.asarray(data).tobytes())
+    return len(plan)
+
+
+def main(argv=None) -> int:
+    import os
+
+    ap = argparse.ArgumentParser(description=__doc__)
+    ap.add_argument("--phrase", required=True, help='e.g. "hey morgane"')
+    ap.add_argument("--out-dir", required=True, help="positive-samples directory")
+    ap.add_argument("--count", type=int, default=500, help="clips to generate")
+    ap.add_argument("--engine", default="openai", choices=["openai"])
+    args = ap.parse_args(argv)
+
+    api_key = (os.getenv("OPENAI_API_KEY") or "").strip()
+    if not api_key:
+        ap.error(
+            "OPENAI_API_KEY not set. Set it, or use openWakeWord's offline Piper "
+            "generator instead (see the skill's references/platforms.md)."
+        )
+
+    plan = plan_clips(args.phrase, args.count, voice_list(args.engine))
+    written = _synthesize(plan, args.out_dir, api_key)
+    print(f"wrote {written} positive samples to {args.out_dir}")
+    return 0
+
+
+if __name__ == "__main__":  # pragma: no cover
+    raise SystemExit(main())

optional-skills/productivity/wake-word-training/scripts/make_training_config.py

@@ -0,0 +1,85 @@
+#!/usr/bin/env python3
+"""Emit an openWakeWord automatic-training YAML for a custom phrase.
+
+Writes the config openWakeWord's ``train.py`` consumes, pre-filled with the
+target phrase, model name, and the positives/negatives/output paths. It covers
+the common fields; always cross-check against openWakeWord's ``custom_model.yml``
+for your installed version, since the schema evolves.
+"""
+
+from __future__ import annotations
+
+import argparse
+import re
+
+
+def slugify(phrase: str) -> str:
+    s = re.sub(r"[^a-z0-9]+", "_", phrase.strip().lower())
+    return s.strip("_") or "wake_word"
+
+
+def build_config(
+    phrase: str,
+    output_dir: str,
+    *,
+    positives_dir: str | None = None,
+    background_dir: str = "./audioset_16k",
+    rir_dir: str = "./mit_rirs",
+    false_positive_dir: str = "./fma",
+    n_samples: int = 5_000,
+    n_samples_val: int = 1_000,
+    steps: int = 10_000,
+) -> dict:
+    """Build the openWakeWord training-config dict.
+
+    When *positives_dir* is given (OpenAI-TTS clips already generated), the
+    trainer uses those instead of synthesizing its own with Piper.
+    """
+    model_name = slugify(phrase)
+    cfg: dict = {
+        "target_phrase": [phrase.strip()],
+        "model_name": model_name,
+        "output_dir": output_dir,
+        "n_samples": n_samples,
+        "n_samples_val": n_samples_val,
+        "steps": steps,
+        "target_accuracy": 0.7,
+        "target_recall": 0.5,
+        # Augmentation + negative sources (see references/platforms.md).
+        "rir_paths": [rir_dir],
+        "background_paths": [background_dir],
+        "false_positive_validation_data_path": false_positive_dir,
+        "augmentation_rounds": 1,
+        "layer_size": 32,
+    }
+    if positives_dir:
+        # Reuse pre-generated positive clips rather than Piper-synthesizing them.
+        cfg["custom_positive_samples_dir"] = positives_dir
+    return cfg
+
+
+def main(argv=None) -> int:
+    ap = argparse.ArgumentParser(description=__doc__)
+    ap.add_argument("--phrase", required=True, help='e.g. "hey morgane"')
+    ap.add_argument("--out", required=True, help="path to write the YAML config")
+    ap.add_argument("--output-dir", default="./oww_out", help="trainer output dir")
+    ap.add_argument("--positives-dir", default=None, help="pre-generated positives")
+    ap.add_argument("--steps", type=int, default=10_000)
+    args = ap.parse_args(argv)
+
+    import yaml
+
+    cfg = build_config(
+        args.phrase,
+        args.output_dir,
+        positives_dir=args.positives_dir,
+        steps=args.steps,
+    )
+    with open(args.out, "w", encoding="utf-8") as f:
+        yaml.safe_dump(cfg, f, sort_keys=False)
+    print(f"wrote training config for {cfg['model_name']!r} to {args.out}")
+    return 0
+
+
+if __name__ == "__main__":  # pragma: no cover
+    raise SystemExit(main())

tests/skills/test_wake_word_training_skill.py

@@ -0,0 +1,129 @@
+"""Tests for the wake-word-training optional skill.
+
+Pure-logic + frontmatter contract only — no network, no heavy deps. The scripts'
+synthesis paths (openai/soundfile/scipy) are import-lazy, so importing the
+helpers here needs nothing beyond stdlib.
+"""
+
+import importlib.util
+import re
+from pathlib import Path
+
+import pytest
+import yaml
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+SKILL_DIR = REPO_ROOT / "optional-skills" / "productivity" / "wake-word-training"
+
+
+def _load(module_name: str):
+    path = SKILL_DIR / "scripts" / f"{module_name}.py"
+    spec = importlib.util.spec_from_file_location(f"_wwt_{module_name}", path)
+    mod = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(mod)
+    return mod
+
+
+@pytest.fixture(scope="module")
+def positives():
+    return _load("generate_positives")
+
+
+@pytest.fixture(scope="module")
+def cfg_mod():
+    return _load("make_training_config")
+
+
+# ── Frontmatter contract ─────────────────────────────────────────────────
+
+
+def _frontmatter() -> dict:
+    text = (SKILL_DIR / "SKILL.md").read_text(encoding="utf-8")
+    m = re.match(r"^---\n(.*?)\n---\n", text, re.DOTALL)
+    assert m, "SKILL.md must open with a YAML frontmatter block"
+    return yaml.safe_load(m.group(1))
+
+
+def test_description_is_short_one_sentence():
+    desc = _frontmatter()["description"]
+    assert len(desc) <= 60, f"description is {len(desc)} chars (max 60)"
+    assert desc.endswith("."), "description must end with a period"
+
+
+def test_platforms_declared():
+    # POSIX-ish training tooling — must declare supported platforms.
+    assert _frontmatter().get("platforms"), "platforms gating is required"
+
+
+def test_author_credits_a_human_first():
+    author = _frontmatter()["author"]
+    assert not author.lower().startswith("hermes"), "credit the human first"
+
+
+# ── generate_positives pure helpers ──────────────────────────────────────
+
+
+@pytest.mark.parametrize(
+    "phrase,expected",
+    [("Hey Morgane!", "hey_morgane"), ("  Hey   Hermes  ", "hey_hermes"), ("!!!", "wake_word")],
+)
+def test_slugify(positives, phrase, expected):
+    assert positives.slugify(phrase) == expected
+
+
+def test_variations_are_unique_and_nonempty(positives):
+    v = positives.build_variations("hey morgane")
+    assert v and len(v) == len(set(v))
+    assert all(x.lower().startswith("hey morgane") for x in v)
+
+
+def test_plan_balances_voices_and_is_unique(positives):
+    voices = positives.voice_list("openai")
+    plan = positives.plan_clips("hey morgane", 25, voices)
+    assert len(plan) == 25
+    filenames = [name for _, _, name in plan]
+    assert len(set(filenames)) == 25, "filenames must be unique"
+    # Voices cycle evenly: with 25 clips over 10 voices, counts differ by <= 1.
+    from collections import Counter
+
+    counts = Counter(v for v, _, _ in plan)
+    assert max(counts.values()) - min(counts.values()) <= 1
+
+
+def test_plan_empty_for_nonpositive_count(positives):
+    assert positives.plan_clips("hey morgane", 0, ["alloy"]) == []
+
+
+def test_plan_requires_a_voice(positives):
+    with pytest.raises(ValueError):
+        positives.plan_clips("hey morgane", 5, [])
+
+
+def test_unknown_engine_rejected(positives):
+    with pytest.raises(ValueError):
+        positives.voice_list("piper")
+
+
+# ── make_training_config pure helper ─────────────────────────────────────
+
+
+def test_config_sets_phrase_and_derived_name(cfg_mod):
+    cfg = cfg_mod.build_config("Hey Morgane", "/tmp/out")
+    assert cfg["target_phrase"] == ["Hey Morgane"]
+    assert cfg["model_name"] == "hey_morgane"
+    assert cfg["output_dir"] == "/tmp/out"
+    assert isinstance(cfg["n_samples"], int) and cfg["n_samples"] > 0
+
+
+def test_config_uses_custom_positives_only_when_given(cfg_mod):
+    without = cfg_mod.build_config("hey morgane", "/tmp/out")
+    assert "custom_positive_samples_dir" not in without
+
+    with_pos = cfg_mod.build_config("hey morgane", "/tmp/out", positives_dir="/tmp/pos")
+    assert with_pos["custom_positive_samples_dir"] == "/tmp/pos"
+
+
+def test_config_is_yaml_serializable(cfg_mod):
+    cfg = cfg_mod.build_config("hey morgane", "/tmp/out", positives_dir="/tmp/pos")
+    round_tripped = yaml.safe_load(yaml.safe_dump(cfg))
+    assert round_tripped == cfg

website/docs/user-guide/features/wake-word.md

@@ -124,6 +124,15 @@ wake_word:
     model: ~/.hermes/wakewords/computer.onnx   # or a built-in name like hey_jarvis
 ```
 
+Or let Hermes train one for you. Install the **wake-word-training** skill and
+ask it to make a phrase (defaults to `hey <profile>`); it drives openWakeWord's
+trainer end-to-end — synthetic voices (OpenAI TTS or Piper), model export into
+your profile's `~/.hermes/wakewords/`, and the config wiring:
+
+```bash
+hermes skills install official/productivity/wake-word-training
+```
+
 Training references:
 
 - [openWakeWord](https://github.com/dscripka/openWakeWord)