Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,10 +69,12 @@ stackvox speak --normalize --file post.md # normalize, then synthesize
stackvox say --normalize "**Shipped** 1,198.9 MPG" # normalize, then send to daemon
```

`--normalize` is off by default on `speak`/`say`, so notification phrases synthesize exactly as before. The tuning flags mirror the library defaults (all on except emoji-strip): `--no-markdown`, `--no-dev-terms`, `--no-expand-units`, `--no-expand-numbers`, `--no-pauses`, `--tables {drop,csv}`, `--code-blocks {drop,placeholder}`, `--strip-emoji`, `--no-terminal-stops`, `--locale`, and a `--pronunciations FILE` (a JSON `{"written": "spoken"}` map, applied whole-word and case-insensitively).
`--normalize` is off by default on `speak`/`say`, so notification phrases synthesize exactly as before. The tuning flags mirror the library defaults (all on except emoji-strip): `--no-markdown`, `--no-dev-terms`, `--no-abbreviations`, `--no-filenames`, `--no-expand-units`, `--no-expand-numbers`, `--no-pauses`, `--tables {drop,csv}`, `--code-blocks {drop,placeholder}`, `--strip-emoji`, `--no-terminal-stops`, `--locale`, and a `--pronunciations FILE` (a JSON `{"written": "spoken"}` map, applied whole-word and case-insensitively).

**Dev-output handling** (on by default; `--no-dev-terms` to disable) covers two things espeak reads badly: acronyms it says as words (`CLI` → "C L I", `AWS`, `URI`, `IAM`, …) and file-line references, which it voices literally as "dot py colon". A `file.ext:line` ref is re-spoken lead-with-the-line: `engine.py:42` → "line 42 of engine py", `cli.py:100-118` → "lines 100 to 118 of cli py", `foo.ts:666:10` → "line 666, column 10 of foo ts". Times, ratios, and versions (`12:30`, `3:1`, `1.2.3`) are left alone.

**Chat shorthand** (on by default; `--no-abbreviations` to disable) is expanded to the words it stands for, so a response that reads "lmk" is spoken as "let me know": `lmk`, `iirc`, `afaik`, `idk`, `tbh`, `imo`, `btw`, `fyi`, `asap`, `wrt`, `tl;dr`/`tldr`. Whole-word and case-insensitive. Shorthand that doubles as a real word or is already voiced correctly (`rn`, `eta`, `aka`) is deliberately left alone; a `--pronunciations` entry overrides any of these.

Fenced code blocks are dropped by default. `--code-blocks placeholder` instead speaks a short stand-in so a skipped block doesn't sound disjointed (ChatGPT-style) — set the wording with `--code-placeholder` (e.g. `--code-blocks placeholder --code-placeholder "you can see the code in the chat history"`); consecutive blocks collapse to one.

Bash completion:
Expand Down
25 changes: 24 additions & 1 deletion docs/speech-normalization.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ def normalize_for_speech(
*,
markdown: bool = True, # strip Markdown structure to prose
pronunciations: dict[str, str] | None = None, # {written: spoken}, whole-word, case-insensitive
abbreviations: bool = True, # chat shorthand -> words: lmk, iirc, btw, tl;dr …
expand_units: bool = True, # £/p/kWh/MPG/kg/km, ÷ × =
expand_numbers: bool = True, # thousands commas removed; decimals → "X point d d"
pauses: bool = True, # dash → beat (…); comma before "("
Expand All @@ -68,7 +69,7 @@ Individual stages also exposed for composability (e.g. `expand_numbers(text)`,
3. **Numbers** (if `expand_numbers`): strip thousands commas (`1,198.9`→`1198.9`) *then* decimals→words (`1198.9`→`1198 point 9`).
4. **Units/symbols** (if `expand_units`): currency **before** decimals is impossible if decimals ran first, so units run here and the decimal pass must see `£1.63`→`1.63 pounds` first → **currency/units run before the decimal split**. (This is why `read-aloud.py` orders units → decimals.)
5. **Pauses** (if `pauses`): ` - `/`—`→` … `; word`(`→`word, (`.
6. **Pronunciations**: whole-word, case-insensitive, from `pronunciations`.
6. **Pronunciations**: whole-word, case-insensitive: built-in abbreviations (if `abbreviations`), then built-in dev terms (if `dev_terms`), then `pronunciations`, which overrides both.
7. **Whitespace collapse**; **terminal stops** (if `terminal_stops`).

> Ordering note to preserve: **currency/unit expansion must precede the
Expand Down Expand Up @@ -161,6 +162,28 @@ Own flag, `filenames`, deliberately **not** `dev_terms`: the acronym dict and
filename handling are unrelated, and coupling them meant `--no-dev-terms`
silently disabled file refs too.

### Chat shorthand

`_ABBREVIATIONS` expands chat shorthand to the words it stands for: `lmk` →
"let me know", `iirc`, `afaik`, `idk`, `tbh`, `imo`, `btw`, `fyi`, `asap`,
`wrt`, `tl;dr`/`tldr`. Claude responses and blog prose are full of it, and
espeak has no reading that helps: it either spells the cluster out or tries to
say it as a word.

It rides the same `apply_pronunciations` pass as `_DEV_PRONUNCIATIONS` but is a
**separate dict behind a separate flag**, because the two do different jobs:
`dev_terms` fixes how a term *sounds*, `abbreviations` changes *which words are
said*. Coupling them would repeat the `filenames` mistake; one switch silently
disabling an unrelated stage. The two dicts share no keys, and neither one's
spoken form contains a key of the other, so the sequential passes can't feed
each other.

Deliberately excluded: shorthand that doubles as a real word, a Greek letter or
a name (`rn`, `eta`, `aka`), where the false positives cost more than the
expansion buys. `fyi` keeps its letters ("F Y I" is how it is said aloud), so
the dict is an *expansion* table, not a spelling-out table, with one exception
that earns its place.

## 4. CLI surface

Backward-compatible additions to the existing `speak`/`say`:
Expand Down
9 changes: 8 additions & 1 deletion stackvox/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ def _configure_logging() -> None:
;;
esac

local norm_flags="--no-markdown --no-dev-terms --no-filenames --pronunciations --no-expand-units --no-expand-numbers --no-pauses --tables --code-blocks --code-placeholder --strip-emoji --no-terminal-stops --locale"
local norm_flags="--no-markdown --no-dev-terms --no-abbreviations --no-filenames --pronunciations --no-expand-units --no-expand-numbers --no-pauses --tables --code-blocks --code-placeholder --strip-emoji --no-terminal-stops --locale"

case "$subcommand" in
speak)
Expand Down Expand Up @@ -220,6 +220,12 @@ def _add_normalize_args(parser: argparse.ArgumentParser, *, with_switch: bool) -
action="store_false",
help="Do not spell out dev acronyms espeak mispronounces (CLI, CI, IDE, AWS, URI, IAM, ...)",
)
parser.add_argument(
"--no-abbreviations",
dest="abbreviations",
action="store_false",
help="Leave chat shorthand as written (no 'lmk' -> 'let me know', 'iirc', 'btw', ...)",
)
parser.add_argument(
"--no-filenames",
dest="filenames",
Expand Down Expand Up @@ -300,6 +306,7 @@ def _normalize_kwargs(args: argparse.Namespace) -> dict:
"markdown": args.markdown,
"pronunciations": _load_pronunciations(args.pronunciations),
"dev_terms": args.dev_terms,
"abbreviations": args.abbreviations,
"filenames": args.filenames,
"expand_units": args.expand_units,
"expand_numbers": args.expand_numbers,
Expand Down
41 changes: 38 additions & 3 deletions stackvox/text.py
Original file line number Diff line number Diff line change
Expand Up @@ -484,6 +484,36 @@ def apply_pronunciations(text: str, mapping: dict[str, str] | None) -> str:
}


# --------------------------------------------------------------------------- #
# Abbreviations #
# --------------------------------------------------------------------------- #
# Chat shorthand, expanded to the words it stands for. Distinct from
# _DEV_PRONUNCIATIONS above, which fixes how a term *sounds*; this changes
# *which words are said*, so it gets its own flag (`abbreviations`). That is the
# lesson `filenames` taught when it was coupled to `dev_terms`.
#
# Applied whole-word and case-insensitive, so "LMK", "Lmk" and "lmk" all match.
# Deliberately excluded ("rn", "eta", "aka" and friends): real words, Greek
# letters, or acronyms espeak already voices correctly, so expanding them costs
# more in false positives than it buys. "fyi" keeps its letters because that is how it is
# said aloud, not "for your information".
_ABBREVIATIONS: dict[str, str] = {
"lmk": "let me know",
"iirc": "if I recall correctly",
"afaik": "as far as I know",
"idk": "I don't know",
"tbh": "to be honest",
"imo": "in my opinion",
"btw": "by the way",
"fyi": "F Y I",
"asap": "as soon as possible",
"wrt": "with respect to",
# Both spellings of the same abbreviation; the bare form is at least as common.
"tl;dr": "too long, didn't read",
"tldr": "too long, didn't read",
}


# --------------------------------------------------------------------------- #
# Pauses #
# --------------------------------------------------------------------------- #
Expand Down Expand Up @@ -687,6 +717,7 @@ def normalize_for_speech(
markdown: bool = True,
pronunciations: dict[str, str] | None = None,
dev_terms: bool = True,
abbreviations: bool = True,
filenames: bool = True,
expand_units: bool = True,
expand_numbers: bool = True,
Expand All @@ -701,9 +732,13 @@ def normalize_for_speech(
"""Normalize ``text`` into speakable prose. Returns paragraphs joined by
newlines. See ``docs/speech-normalization.md`` for the full contract."""
expand_units_flag, expand_numbers_flag = expand_units, expand_numbers
# Built-in dev-term fixes first; caller-supplied pronunciations override them
# (keyed case-insensitively, so a caller's "CLI" beats the default "cli").
effective_pronunciations: dict[str, str] = dict(_DEV_PRONUNCIATIONS) if dev_terms else {}
# Built-in fixes first; caller-supplied pronunciations override them (keyed
# case-insensitively, so a caller's "CLI" beats the default "cli"). The two
# built-in dicts share no keys, and neither one's spoken form contains a key
# of the other, so the sequential passes can't feed each other.
effective_pronunciations: dict[str, str] = dict(_ABBREVIATIONS) if abbreviations else {}
if dev_terms:
effective_pronunciations.update(_DEV_PRONUNCIATIONS)
for written, spoken in (pronunciations or {}).items():
effective_pronunciations[written.lower()] = spoken

Expand Down
1 change: 1 addition & 0 deletions tests/test_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,7 @@ def _norm_ns(text=None, file=None, **overrides):
markdown=True,
pronunciations=None,
dev_terms=True,
abbreviations=True,
filenames=True,
expand_units=True,
expand_numbers=True,
Expand Down
41 changes: 41 additions & 0 deletions tests/test_text.py
Original file line number Diff line number Diff line change
Expand Up @@ -487,3 +487,44 @@ def test_caller_pronunciations_override_dev_terms():
out = normalize_for_speech("Use the CLI.", markdown=False, pronunciations={"CLI": "command line"})
assert "command line" in out
assert "C L I" not in out


def test_chat_abbreviations_are_expanded():
assert normalize_for_speech("lmk what you think", markdown=False) == "let me know what you think."
assert "if I recall correctly" in normalize_for_speech("iirc it shipped", markdown=False)
assert "by the way" in normalize_for_speech("btw the build is green", markdown=False)
assert "as soon as possible" in normalize_for_speech("ship it asap", markdown=False)
assert "with respect to" in normalize_for_speech("wrt the plan", markdown=False)


def test_chat_abbreviations_are_case_insensitive():
assert "let me know" in normalize_for_speech("LMK if that works", markdown=False)
assert "to be honest" in normalize_for_speech("Tbh it is fine", markdown=False)


def test_tldr_expands_in_both_spellings():
assert "too long, didn't read" in normalize_for_speech("tl;dr it works", markdown=False)
assert "too long, didn't read" in normalize_for_speech("tldr it works", markdown=False)


def test_chat_abbreviations_are_whole_word_only():
# must not rewrite the middle of a longer token
assert "let me know" not in normalize_for_speech("lmkfoo", markdown=False)
assert "in my opinion" not in normalize_for_speech("imos", markdown=False)


def test_chat_abbreviations_can_be_disabled():
assert normalize_for_speech("lmk soon", markdown=False, abbreviations=False) == "lmk soon."


def test_caller_pronunciations_override_abbreviations():
out = normalize_for_speech("lmk soon", markdown=False, pronunciations={"LMK": "tell me"})
assert "tell me" in out
assert "let me know" not in out


def test_abbreviations_independent_of_dev_terms():
# the `filenames` lesson: an unrelated switch must not silently disable this
out = normalize_for_speech("lmk about the CLI", markdown=False, dev_terms=False)
assert "let me know" in out
assert "C L I" not in out