# Sentence playback latency — 2026-09-26

The owner's direct Bob say and inbox-summary playback both had audible gaps. A
system-audio recording of the same full 115-word, six-sentence text measured mean
sentence gaps of 1.437 s directly and 1.687 s through Spark and the Mac worker.
The full spoken text, engine and voice matched. Queue wait was excluded.

## Causes and change

1. Each sentence started and drained a new `afplay` process. A controlled 0.2 s
   silent WAV took 1.716 s on average over three direct invocations. Keeping an
   AVAudioEngine output open avoids this per-sentence overhead. The first sentence
   still pays device startup; later sentences reuse the device.
2. The summary launch agent explicitly used `ProcessType=Background`. Repeating
   the identical afplay test under launchd without that setting averaged 1.738 s;
   with Background it averaged 2.228 s. The inbox installer now uses Interactive
   for this explicitly requested audio work. These small samples demonstrate
   scheduling overhead, not a fixed universal penalty of exactly 250 ms.

`SpeechEngine.openSession` gives each answer its own resources. The ElevenLabs
session lazily spawns a stdin-driven native helper, then closes it in the speak
pipeline's finally block. The helper acknowledges each file only using Apple's
[dataPlayedBack completion](https://developer.apple.com/documentation/avfaudio/avaudioplayernodecompletioncallbacktype/dataplayedback),
which accounts for device output latency. Existing sentence-level ledger writes,
resume semantics and prefetch remain in place. An incomplete sentence is not
marked heard. The signal cleanup kills the owned child, and a parent-watch covers
hard parent death. No persistent service or listener was added.

The trim now retains 200 ms of trailing silence: a deliberate sentence breath,
rather than relying on accidental player startup/drain delay. Missing native
builds warn and retain afplay as a compatibility fallback.

Build with `bun run build:player` after updating the Swift source. The compiled
helper is ignored by Git; the source and build recipe are versioned.

## Validation

- 796 tests passed, 17 skipped; TypeScript typecheck passed.
- Real native-player test: full-duration acknowledgement, reuse across three
  clips with a latency ceiling that the former afplay path fails, and missing-file
  failure. Uses silence, no synthesis or network call.
- Protocol tests: one child across sentences, delayed acknowledgements, premature
  child exit, malformed acknowledgement and close during playback.
- SIGINT/SIGTERM tests exercise the new player's actual cleanup registration.
- Existing ledger, prefetch, lock and fallback tests pass; new lifecycle tests
  cover answer ownership and cleanup on failure.
- Inbox's isolated real-worker check passed: duplicate exclusion, PTT interruption,
  repeated-sentence resume, explicit Stop, and independent Done.

## Live recordings after activation

Both paths replayed the identical complete source text with the same engine and
voice; both reported successful, untruncated playback. The native helper was
observed as the live child process. The worker finished at 6/6 sentences, with
no playback error, and the summary was not marked Done.

| Path | Before: mean sentence gap | After: mean sentence gap |
| --- | ---: | ---: |
| Direct Mac Bob say | 1.437 s | 0.317 s |
| Spark app → Mac worker | 1.687 s | 0.284 s |

After-fix gaps by boundary (milliseconds): direct 307, 317, 317, 308, 337;
app 227, 316, 313, 271, 294. The earlier app penalty is absent in this pair.
Generated speech differs on each synthesis request, so this does not assert
identical gaps on every future run. Slow synthesis or output-device latency
can still increase a gap; prefetch remains one sentence ahead.

Measurement: actual ScreenCaptureKit system-audio capture, matched to the copied
source clips by waveform correlation (after-fix scores 0.981–1.000). Speech
edges use 5 ms RMS at -45 dBFS. No microphone, no saved screen image. Artifacts:
`~/Downloads/bob-audio-comparison-2026-09-26/` (before) and `fixed/` (after),
including original recordings, source clips, analysis scripts and JSON.

Activated code: Hey Bob `2f71a32`, inbox installer `3f73026`. Both were merged
fast-forward to local main. The native helper was built in the live Hey Bob
checkout, and the Mac launch agent was reinstalled with Interactive scheduling.
No Spark app/CLI binary or database schema changed; no push was performed.