Pytorch Loves Vllm Meta Ray Summit 2026

Summary

Large language models demand significant computational power for inference, and TPUs are becoming a first-class target for ... Agentic production traffic brings long multi-turn sessions, heavy prefix reuse, and bursty, heterogeneous requests that challenge ... An Open Source Post-Training Stack: Kubernetes + State-of-the-art speculative decoding lives or dies on the drafter, and training drafters well at scale is a production issue. At The keynote continues with an engaging panel discussion featuring Robert Nishihara, Dawn Chen (Software Engineer at Google) ... Serving LLMs for agentic workloads at Chase scale means balancing latency, cost, and resilience. The team cut latency from ... A simpler setup delivered more tokens per second, with the first token arriving sooner for a single request, than the model card's ... Uber scaled both feature preprocessing and training data loading with