# Vllm > Markdown twin of https://roboterminal.app/companies/vllm · https://roboterminal.app/companies/vllm.md ## On shared sources - [NVIDIA](https://roboterminal.app/companies/nvidia) (9) - [AMD](https://roboterminal.app/companies/amd) (6) - [Inferact](https://roboterminal.app/companies/inferact) (5) - [SemiAnalysis](https://roboterminal.app/companies/semianalysis) (5) - [DeepSeek](https://roboterminal.app/companies/deepseek) (3) - [Google](https://roboterminal.app/companies/google) (2) - [Ollama](https://roboterminal.app/companies/ollama) (2) - [vllm_project](https://roboterminal.app/companies/vllm-project) (2) - [Amazon](https://roboterminal.app/companies/amazon) (1) - [Anthropic](https://roboterminal.app/companies/anthropic) (1) - [AWS](https://roboterminal.app/companies/aws) (1) - [Huawei](https://roboterminal.app/companies/huawei) (1) - [Hugging Face](https://roboterminal.app/companies/hugging-face) (1) - [Meta](https://roboterminal.app/companies/meta) (1) - [Moonshot AI](https://roboterminal.app/companies/moonshot-ai) (1) - [OpenAI](https://roboterminal.app/companies/openai) (1) ## Pulse - [@stretchcloud on X](https://x.com/stretchcloud/status/2112849877696958782) NVIDIA already has TensorRT-LLM. It also has Dynamo. This week it shipped a third local inference server anyway, and the part worth sitting with is why. Flare runs on Rust, not Python, and it's built … - [@stretchcloud on X](https://x.com/stretchcloud/status/2112649305836409272) Every hyperscaler racing to dent Nvidia's lead just converged on the same move within a day of each other. Google open sourced TorchTPU at the PyTorch Conference in San Jose this week, co-built with M… - [@stretchcloud on X](https://x.com/stretchcloud/status/2112573053477728345) Model weights are free. Running them at scale cheaply is not, and that's the part DeepSeek open sourced for everyone this week. DeepInfer is the inference engine serving DeepSeek V4.1 Flash in product… - [@SemiAnalysis_ on X](https://x.com/SemiAnalysis_/status/2111938013852762351) BREAKING NEWS🚨: @Inferact & Google submit their FIRST preliminary TPUv7 Kimi K3 2.8T Model Submission, and it is already delivering strong perf per TCO against GB200 NVL72, B300, and MI355 in the ult… - [@SemiAnalysis_ on X](https://x.com/SemiAnalysis_/status/2111532833114054891) TOKENMAXXING: @vllm_project and @inferact achieve 1.3 BILLION tokens per $ of TCO (hyperscaler volume pricing) via disaggregated serving on AgentX. This result is 4x better than the B300 aggregated co… - [@SemiAnalysis_ on X](https://x.com/SemiAnalysis_/status/2110560412907262343) BREAKING NEWS🚨: Google has submitted its FIRST TPUv7 FP8 vLLM results to our Agentic Inference Benchmark and has demonstrated 1.67x better perf per dollar than even GB300 NVL72 FP4. 🚀 It is amazing … - [@SemiAnalysis_ on X](https://x.com/SemiAnalysis_/status/2110496505031192895) ALERT🚨🚨: @vllm_project, @Inferact & NVIDIA TOOK CO-DESIGN TO THE EXTREME WITH A GB300 NVL72 INFERENCE OPTIMIZATION TO GET 🚀 7.2x 🚀 BETTER PERF PER TCO THAN LITERALLY EVERY OTHER AI CHIP, using an … - [@SemiAnalysis_ on X](https://x.com/SemiAnalysis_/status/2109841575547211955) ALERT🚨: AMD JUST LANDED SUPPORT FOR MEGAKERNEL DECODE IN UPSTREAM VLLM WELL BEFORE NVIDIA. This marks one of the first times that a major vLLM feature is only supported on ROCm and not on CUDA yet. M… - [@SemiAnalysis_ on X](https://x.com/SemiAnalysis_/status/2108604228440658369) HOLY ALERT🚨: NVIDIA vLLM INFERENCE RUBIN IS 3.2x BETTER IN PROFIT💰️ PER GIGAWATT & HAS UP TO 🚀 10x BETTER PERF PER DOLLAR THAN EVEN GB300 NVL72. This is on the widely used production LLM engine cal… - [@SemiAnalysis_ on X](https://x.com/SemiAnalysis_/status/2100948236139712988) Great blog from Kevin Lu at @vllm_project about quality control at the inference engine. Unfortunately, @AnushElangovan has not provided enough stable AMD QA clusters to vLLM, which leads to an order … - [@SemiAnalysis_ on X](https://x.com/SemiAnalysis_/status/2098618867035557984) POWER OF CUDA MOAT ALERT🚨: 2 days after CUDA vLLM supported DeepSeekv4.1 Flash, AMD finally publicly released its DeepSeek v4.1 Flash image. Functionally, it works out of the box, but performance-wis… - [@SemiAnalysis_ on X](https://x.com/SemiAnalysis_/status/2098501866401218813) On the Day 0 release of DeepSeekv4.1 Flash, NVIDIA vLLM works out of the box with zero issues across all 6 SKUs: H100, H200, B200, B300, GB200, GB300! Amazing work by the NVIDIA & Inferact teams! In c… - [@ricci_nov on X](https://x.com/ricci_nov/status/2098447871154823642) PATTON maps production LLM KV-cache serving onto commodity PIM DRAM: 1.95x average speedup and 4.83x better energy efficiency over baselines, with no changes to PIM processing units. Hit rate stays co… - [@SemiAnalysis_ on X](https://x.com/SemiAnalysis_/status/2098238630015705267) ALERT🚨 GB300 NVL72 NVLINK BACKPLANE ENABLES 13x BETTER PERF PER DOLLAR THAN HOPPER ON AGENTIC INFERENCE Through extreme co-design, GB300 NVL72 uses an innovative copper backplane to expand the scale-… - [@stretchcloud on X](https://x.com/stretchcloud/status/2097542010143990196) Every time an agentic system routes a task from one model to another, the receiving model pays a prefill tax. It has to re-read the entire conversation from scratch, driving up both cost and latency. … - [@SemiAnalysis_ on X](https://x.com/SemiAnalysis_/status/2097445137521524787) Spec Decoding (DSpark) finally works with Pipeline Parallelism in vLLM!! 🔥 A lot of the GPU poor/proletarian have been asking for this feature for a while now, but it took the recent Kimi K3's massiv… - [@SemiAnalysis_ on X](https://x.com/SemiAnalysis_/status/2095744598073053470) On total tokens per $ TCO, a new AMD MI355x submission beats B300 at lower interactivity ranges on AgentX. Shoutout to vLLM, AMD, and LMCache engineers. https://t.co/1oYYeYYu9S - [@SemiAnalysis_ on X](https://x.com/SemiAnalysis_/status/2095731112207061029) On total tokens per $ TCO, a new AMD MI355x submission beats B300 at lower interactivity ranges on AgentX. Both high throughput configs use disaggregated setups. Shoutout to vLLM, AMD, and LMCache eng… --- Source: Roboterminal · https://roboterminal.app/companies/vllm.md