Achieve state-of-the-art inference latencies with speculative decoding (opens in new tab)

How Modal and Decagon worked together to cut inference latency - and you can too.