Turning spoken commands into JSON tool calls on iPhones (opens in new tab)

Covers 4 stories including GitHub here . You can follow the build instructions below as well. Change -DGGML_CUDA=ON to -DGGML_CUDA=OFF if you don't have a GPU or just want CPU inferen...Discussed on Hacker News

We benchmarked on-device speech-to-tool latency across iPhone generations. The fastest two-step Whisper Tiny plus LFM2 350M path was about 388 ms median, while the best direct LFM path on iPhone 16 Pro landed at 726 ms.

Read the original article