diff --git a/README.md b/README.md index ff6bbf1..12a0b88 100644 --- a/README.md +++ b/README.md @@ -69,10 +69,12 @@ stackvox speak --normalize --file post.md # normalize, then synthesize stackvox say --normalize "**Shipped** 1,198.9 MPG" # normalize, then send to daemon ``` -`--normalize` is off by default on `speak`/`say`, so notification phrases synthesize exactly as before. The tuning flags mirror the library defaults (all on except emoji-strip): `--no-markdown`, `--no-dev-terms`, `--no-expand-units`, `--no-expand-numbers`, `--no-pauses`, `--tables {drop,csv}`, `--code-blocks {drop,placeholder}`, `--strip-emoji`, `--no-terminal-stops`, `--locale`, and a `--pronunciations FILE` (a JSON `{"written": "spoken"}` map, applied whole-word and case-insensitively). +`--normalize` is off by default on `speak`/`say`, so notification phrases synthesize exactly as before. The tuning flags mirror the library defaults (all on except emoji-strip): `--no-markdown`, `--no-dev-terms`, `--no-abbreviations`, `--no-filenames`, `--no-expand-units`, `--no-expand-numbers`, `--no-pauses`, `--tables {drop,csv}`, `--code-blocks {drop,placeholder}`, `--strip-emoji`, `--no-terminal-stops`, `--locale`, and a `--pronunciations FILE` (a JSON `{"written": "spoken"}` map, applied whole-word and case-insensitively). **Dev-output handling** (on by default; `--no-dev-terms` to disable) covers two things espeak reads badly: acronyms it says as words (`CLI` → "C L I", `AWS`, `URI`, `IAM`, …) and file-line references, which it voices literally as "dot py colon". A `file.ext:line` ref is re-spoken lead-with-the-line: `engine.py:42` → "line 42 of engine py", `cli.py:100-118` → "lines 100 to 118 of cli py", `foo.ts:666:10` → "line 666, column 10 of foo ts". Times, ratios, and versions (`12:30`, `3:1`, `1.2.3`) are left alone. +**Chat shorthand** (on by default; `--no-abbreviations` to disable) is expanded to the words it stands for, so a response that reads "lmk" is spoken as "let me know": `lmk`, `iirc`, `afaik`, `idk`, `tbh`, `imo`, `btw`, `fyi`, `asap`, `wrt`, `tl;dr`/`tldr`. Whole-word and case-insensitive. Shorthand that doubles as a real word or is already voiced correctly (`rn`, `eta`, `aka`) is deliberately left alone; a `--pronunciations` entry overrides any of these. + Fenced code blocks are dropped by default. `--code-blocks placeholder` instead speaks a short stand-in so a skipped block doesn't sound disjointed (ChatGPT-style) — set the wording with `--code-placeholder` (e.g. `--code-blocks placeholder --code-placeholder "you can see the code in the chat history"`); consecutive blocks collapse to one. Bash completion: diff --git a/docs/speech-normalization.md b/docs/speech-normalization.md index 780a458..28f3d89 100644 --- a/docs/speech-normalization.md +++ b/docs/speech-normalization.md @@ -45,6 +45,7 @@ def normalize_for_speech( *, markdown: bool = True, # strip Markdown structure to prose pronunciations: dict[str, str] | None = None, # {written: spoken}, whole-word, case-insensitive + abbreviations: bool = True, # chat shorthand -> words: lmk, iirc, btw, tl;dr … expand_units: bool = True, # £/p/kWh/MPG/kg/km, ÷ × = expand_numbers: bool = True, # thousands commas removed; decimals → "X point d d" pauses: bool = True, # dash → beat (…); comma before "(" @@ -68,7 +69,7 @@ Individual stages also exposed for composability (e.g. `expand_numbers(text)`, 3. **Numbers** (if `expand_numbers`): strip thousands commas (`1,198.9`→`1198.9`) *then* decimals→words (`1198.9`→`1198 point 9`). 4. **Units/symbols** (if `expand_units`): currency **before** decimals is impossible if decimals ran first, so units run here and the decimal pass must see `£1.63`→`1.63 pounds` first → **currency/units run before the decimal split**. (This is why `read-aloud.py` orders units → decimals.) 5. **Pauses** (if `pauses`): ` - `/`—`→` … `; word`(`→`word, (`. -6. **Pronunciations**: whole-word, case-insensitive, from `pronunciations`. +6. **Pronunciations**: whole-word, case-insensitive: built-in abbreviations (if `abbreviations`), then built-in dev terms (if `dev_terms`), then `pronunciations`, which overrides both. 7. **Whitespace collapse**; **terminal stops** (if `terminal_stops`). > Ordering note to preserve: **currency/unit expansion must precede the @@ -161,6 +162,28 @@ Own flag, `filenames`, deliberately **not** `dev_terms`: the acronym dict and filename handling are unrelated, and coupling them meant `--no-dev-terms` silently disabled file refs too. +### Chat shorthand + +`_ABBREVIATIONS` expands chat shorthand to the words it stands for: `lmk` → +"let me know", `iirc`, `afaik`, `idk`, `tbh`, `imo`, `btw`, `fyi`, `asap`, +`wrt`, `tl;dr`/`tldr`. Claude responses and blog prose are full of it, and +espeak has no reading that helps: it either spells the cluster out or tries to +say it as a word. + +It rides the same `apply_pronunciations` pass as `_DEV_PRONUNCIATIONS` but is a +**separate dict behind a separate flag**, because the two do different jobs: +`dev_terms` fixes how a term *sounds*, `abbreviations` changes *which words are +said*. Coupling them would repeat the `filenames` mistake; one switch silently +disabling an unrelated stage. The two dicts share no keys, and neither one's +spoken form contains a key of the other, so the sequential passes can't feed +each other. + +Deliberately excluded: shorthand that doubles as a real word, a Greek letter or +a name (`rn`, `eta`, `aka`), where the false positives cost more than the +expansion buys. `fyi` keeps its letters ("F Y I" is how it is said aloud), so +the dict is an *expansion* table, not a spelling-out table, with one exception +that earns its place. + ## 4. CLI surface Backward-compatible additions to the existing `speak`/`say`: diff --git a/stackvox/cli.py b/stackvox/cli.py index b08ff91..6357a5e 100644 --- a/stackvox/cli.py +++ b/stackvox/cli.py @@ -87,7 +87,7 @@ def _configure_logging() -> None: ;; esac - local norm_flags="--no-markdown --no-dev-terms --no-filenames --pronunciations --no-expand-units --no-expand-numbers --no-pauses --tables --code-blocks --code-placeholder --strip-emoji --no-terminal-stops --locale" + local norm_flags="--no-markdown --no-dev-terms --no-abbreviations --no-filenames --pronunciations --no-expand-units --no-expand-numbers --no-pauses --tables --code-blocks --code-placeholder --strip-emoji --no-terminal-stops --locale" case "$subcommand" in speak) @@ -220,6 +220,12 @@ def _add_normalize_args(parser: argparse.ArgumentParser, *, with_switch: bool) - action="store_false", help="Do not spell out dev acronyms espeak mispronounces (CLI, CI, IDE, AWS, URI, IAM, ...)", ) + parser.add_argument( + "--no-abbreviations", + dest="abbreviations", + action="store_false", + help="Leave chat shorthand as written (no 'lmk' -> 'let me know', 'iirc', 'btw', ...)", + ) parser.add_argument( "--no-filenames", dest="filenames", @@ -300,6 +306,7 @@ def _normalize_kwargs(args: argparse.Namespace) -> dict: "markdown": args.markdown, "pronunciations": _load_pronunciations(args.pronunciations), "dev_terms": args.dev_terms, + "abbreviations": args.abbreviations, "filenames": args.filenames, "expand_units": args.expand_units, "expand_numbers": args.expand_numbers, diff --git a/stackvox/text.py b/stackvox/text.py index de74909..5579492 100644 --- a/stackvox/text.py +++ b/stackvox/text.py @@ -484,6 +484,36 @@ def apply_pronunciations(text: str, mapping: dict[str, str] | None) -> str: } +# --------------------------------------------------------------------------- # +# Abbreviations # +# --------------------------------------------------------------------------- # +# Chat shorthand, expanded to the words it stands for. Distinct from +# _DEV_PRONUNCIATIONS above, which fixes how a term *sounds*; this changes +# *which words are said*, so it gets its own flag (`abbreviations`). That is the +# lesson `filenames` taught when it was coupled to `dev_terms`. +# +# Applied whole-word and case-insensitive, so "LMK", "Lmk" and "lmk" all match. +# Deliberately excluded ("rn", "eta", "aka" and friends): real words, Greek +# letters, or acronyms espeak already voices correctly, so expanding them costs +# more in false positives than it buys. "fyi" keeps its letters because that is how it is +# said aloud, not "for your information". +_ABBREVIATIONS: dict[str, str] = { + "lmk": "let me know", + "iirc": "if I recall correctly", + "afaik": "as far as I know", + "idk": "I don't know", + "tbh": "to be honest", + "imo": "in my opinion", + "btw": "by the way", + "fyi": "F Y I", + "asap": "as soon as possible", + "wrt": "with respect to", + # Both spellings of the same abbreviation; the bare form is at least as common. + "tl;dr": "too long, didn't read", + "tldr": "too long, didn't read", +} + + # --------------------------------------------------------------------------- # # Pauses # # --------------------------------------------------------------------------- # @@ -687,6 +717,7 @@ def normalize_for_speech( markdown: bool = True, pronunciations: dict[str, str] | None = None, dev_terms: bool = True, + abbreviations: bool = True, filenames: bool = True, expand_units: bool = True, expand_numbers: bool = True, @@ -701,9 +732,13 @@ def normalize_for_speech( """Normalize ``text`` into speakable prose. Returns paragraphs joined by newlines. See ``docs/speech-normalization.md`` for the full contract.""" expand_units_flag, expand_numbers_flag = expand_units, expand_numbers - # Built-in dev-term fixes first; caller-supplied pronunciations override them - # (keyed case-insensitively, so a caller's "CLI" beats the default "cli"). - effective_pronunciations: dict[str, str] = dict(_DEV_PRONUNCIATIONS) if dev_terms else {} + # Built-in fixes first; caller-supplied pronunciations override them (keyed + # case-insensitively, so a caller's "CLI" beats the default "cli"). The two + # built-in dicts share no keys, and neither one's spoken form contains a key + # of the other, so the sequential passes can't feed each other. + effective_pronunciations: dict[str, str] = dict(_ABBREVIATIONS) if abbreviations else {} + if dev_terms: + effective_pronunciations.update(_DEV_PRONUNCIATIONS) for written, spoken in (pronunciations or {}).items(): effective_pronunciations[written.lower()] = spoken diff --git a/tests/test_cli.py b/tests/test_cli.py index f19c03a..b764266 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -129,6 +129,7 @@ def _norm_ns(text=None, file=None, **overrides): markdown=True, pronunciations=None, dev_terms=True, + abbreviations=True, filenames=True, expand_units=True, expand_numbers=True, diff --git a/tests/test_text.py b/tests/test_text.py index e4b7bb3..8dae216 100644 --- a/tests/test_text.py +++ b/tests/test_text.py @@ -487,3 +487,44 @@ def test_caller_pronunciations_override_dev_terms(): out = normalize_for_speech("Use the CLI.", markdown=False, pronunciations={"CLI": "command line"}) assert "command line" in out assert "C L I" not in out + + +def test_chat_abbreviations_are_expanded(): + assert normalize_for_speech("lmk what you think", markdown=False) == "let me know what you think." + assert "if I recall correctly" in normalize_for_speech("iirc it shipped", markdown=False) + assert "by the way" in normalize_for_speech("btw the build is green", markdown=False) + assert "as soon as possible" in normalize_for_speech("ship it asap", markdown=False) + assert "with respect to" in normalize_for_speech("wrt the plan", markdown=False) + + +def test_chat_abbreviations_are_case_insensitive(): + assert "let me know" in normalize_for_speech("LMK if that works", markdown=False) + assert "to be honest" in normalize_for_speech("Tbh it is fine", markdown=False) + + +def test_tldr_expands_in_both_spellings(): + assert "too long, didn't read" in normalize_for_speech("tl;dr it works", markdown=False) + assert "too long, didn't read" in normalize_for_speech("tldr it works", markdown=False) + + +def test_chat_abbreviations_are_whole_word_only(): + # must not rewrite the middle of a longer token + assert "let me know" not in normalize_for_speech("lmkfoo", markdown=False) + assert "in my opinion" not in normalize_for_speech("imos", markdown=False) + + +def test_chat_abbreviations_can_be_disabled(): + assert normalize_for_speech("lmk soon", markdown=False, abbreviations=False) == "lmk soon." + + +def test_caller_pronunciations_override_abbreviations(): + out = normalize_for_speech("lmk soon", markdown=False, pronunciations={"LMK": "tell me"}) + assert "tell me" in out + assert "let me know" not in out + + +def test_abbreviations_independent_of_dev_terms(): + # the `filenames` lesson: an unrelated switch must not silently disable this + out = normalize_for_speech("lmk about the CLI", markdown=False, dev_terms=False) + assert "let me know" in out + assert "C L I" not in out