Serving Vllm On Intel Gpus Cpus And Gaudi Intel Ray Summit 2026

Summary

In this session, we explored the latest updates in Agentic production traffic brings long multi-turn sessions, heavy prefix reuse, and bursty, heterogeneous requests that challenge ... Large language models demand significant computational power for inference, and TPUs are becoming a first-class target for ... A simpler setup delivered more tokens per second, with the first token arriving sooner for a single request, than the model card's ... Training trillion-parameter MoE models on long-horizon agentic tasks demands that inference and optimization scale ... As LLMs grow in size, context length, and architectural complexity,