What is true on Inference?

A Python pipe buffer was holding camera frames until 64 KB had piled up. That quietly added hundreds of milliseconds to our π0.5 rollouts. New from @castorhat: how we found it, and what changed on the robot once it was gone. https://t.co/xTAueR4wOV

Why hardware specialization is back

In just the first 72 hours since Gemini 4 Argon launched, we processed ~25 trillion tokens, the demand for intelligence is incredible and still under appreciated!

NVIDIA just released Flare, a local inference server built for DGX Spark and RTX Blackwell GPUs - Built for long chats, agents and tool calls - Serves from Rust, no Python - Plans memory up front - FP8 KV cache by default Interesting timing... right around when DeepSeek open-sourced their inference engine. Link below.

China wants to control exports of its most powerful AI models. A new draft adds 23 AI technologies to its export control list, including large models, AI agents and computing clusters. Frontier models above 2 trillion parameters, or trained with more than 10^26 FLOPs, would need a license before their weights or source code leave the country. Open source models that are already public are exempt.

We are excited to work together with @FireworksAI_HQ on the early-launch of EAGLE-4. Check out our technical blog to understand how EAGLE-4 works and why it matters for creative workflows: https://t.co/YnYzQ89uCv https://t.co/02VMIubVgg

We're announcing six new features this week: Flex VMs, our own VM product that makes it significantly easier for us to onboard capacity. Spot, a mechanism for our suppliers and commitment users to monetize idle resources. CPUs and Apple Silicon, diversifying our compute resources for specialized use cases. MCP and CLI, making it seamless for humans and agents to access the world's compute. Learn more: https://t.co/ai73PLDbZY

🚀 Introducing DeepInfer, our in-house inference framework, now fully open source! Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable. 🔗 https://t.co/JGAik1kgk0 1/4

PyTorch 🤝 Google TPUs. TorchTPU is officially open source! 🚀

If we’re entering the era of Robotics APIs, we need to start thinking about the security threats that come with them, just like we’ve seen with LLM APIs. Except this time, the models act in the physical world. https://t.co/qE9ghIMw5H

PyTorch 🤝 Google TPUs. TorchTPU is officially open source! 🚀

Former $GOOGL employee on why data center capacity and electricity are the real bottlenecks, not GPU production itself, and how unreliable some of the neoclouds are ($AMZN, $GOOGL, $MSFT, $NVDA): 1. According to the expert, open-source models are closing the gap with frontier models faster than expected, with customers increasingly replacing Anthropic and OpenAI models in multi-agent architectures with open-source alternatives starting from Q2 of this year. The shift started with simple tasks like tool calling and MCP, but has moved up the stack to low-level orchestration. 2. The expert identifies capacity as the biggest constraint at the moment, with certain companies' traffic growing 5x while GPU availability, particularly in the U.S. where latency requirements are tightest, simply cannot keep up. Sourcing capacity overseas in Asia or Australia is technically possible but adds latency that most customers will not accept. 3. Hyperscalers offer reliability but are expensive, require multi-year commitments of three or more years, and have limited availability of the latest hardware, with customers already chasing B300s. AI-native companies tend to gravitate toward neoclouds due to hyperscaler pricing and contract rigidity, but neoclouds come with their own reliability issues, with GPUs dropping off unexpectedly and requiring engineering resources to manage fleet stability around the clock. 4. The third tier consists of smaller independent operators who have invested in GPU racks, seeing the opportunity to lease out capacity and recoup costs within a year or two, with the remainder being pure profit. The expert describes these as essentially unusable in practice, with operators who do nothing more than find land, install racks, connect electricity, and set up cooling, with no real cloud management, maintenance, or operational expertise behind them. 5. The expert expects overall GPU availability to increase meaningfully over the next year or two, with $NVDA continuing to ramp production. The real bottleneck is not GPUs themselves but data center capacity and electricity in the U.S. On the supply side, the expert also sees players $SPCX beginning to lease out spare capacity to the market, with more likely to follow. 6. The expert notes that U.S.-based neoclouds typically require 20-30% prepayment upfront, with the remainder paid monthly or quarterly over the contract term, and rebates negotiated at the end for any reliability incidents like traffic loss. APAC-based providers operate differently, with no prepayment required at all. found on @AlphaSenseInc

A new frontier for on-device speech. We're open-sourcing Phonon, a suite of TTS models under 50m parameters, setting a new standard for naturalness and accuracy across languages, with the broadest inference support to date. Phonon-31m is available under Apache 2.0. https://t.co/d3qa9qmnzY🧵

WTF DeepSeek just gave away its production inference stack. Big move. DeepInfer, the engine behind DeepSeek-V4.1-Flash, is now open source, plus a component that makes community deployments match the official API. DeepSeek’s stated goal is lower compute costs and cheaper AI for everyone.

How could you POSSIBLY be bearish on compute right now when GPU rental prices are doing THIS? According to data from FastGPU, GPU rental prices increased as many as 81 times across providers over the past 60 days, with some of the most important chips used for AI training and inference seeing MASSIVE price increases. NVIDIA B300 rental prices increased 52.8%, going from $8.50 to $12.99 per GPU hour. H200 prices jumped 33.3%, B200 prices increased 27.8% and H100 prices rose 30% and the RTX PRO 6000? Rental prices surged an absolutely INSANE 307%. We're watching AI infrastructure companies get absolutely DESTROYED in the stock market, with investors freaking out about debt, financing costs, oversupply, and whether these companies will ever generate attractive returns on their investments. Meanwhile, the actual cost of renting the underlying compute is going UP. If we were genuinely heading toward a massive compute oversupply, you would expect rental prices to start collapsing as providers competed to fill their available capacity. Instead, we're seeing pricing strength across some of the most important GPUs powering the AI industry. And remember, we're still in the EARLY stages of AI adoption. Frontier labs are scaling their training clusters, inference demand is exploding as more people and businesses adopt AI, and the amount of compute required to serve increasingly sophisticated models continues to grow. This is EXACTLY why I've been so bullish on companies like $NBIS , $MU , $CRDO , $IREN etc. I genuinely believe compute will remain one of the most valuable resources in the AI economy for years to come. And while Wall Street continues panicking over the short term volatility, I'm BUYING THE DIP. If you want to know exactly what I’m buying on this dip and how I’m positioning across the compute buildout, check out portfolio below. https://t.co/Hkywss4mqJ

We're witnessing one of the BIGGEST supply demand imbalances in the history of the technology industry and it's showing NO signs of slowing down. According to a new report from TrendForce, hyperscalers are already locking in hard disk drive (HDD) supply for 2028, and even after securing long term agreements with manufacturers, they STILL expect shortages. HDD lead times are currently sitting at 50 weeks, compared to just 16 weeks in a balanced market. In other words, it can take nearly an entire YEAR to secure hardware that would normally take just a few months. TrendForce is reporting that DRAM lead times have reached 20 weeks compared to a normal 8 weeks, enterprise NAND storage is at 16 weeks versus 8 weeks, ABF substrates are sitting at an incredible 48–56 weeks compared to just 12 weeks and even MLCCs, which are critical components used in electronic circuits and power delivery, are experiencing significant supply constraints. Meanwhile, GPU lead times are actually relatively balanced. And I think this tells us something VERY important about where the AI buildout is heading. As AI inference scales, hyperscalers need enormous amounts of storage to support expanding datasets, model outputs, retrieval systems, and the infrastructure required to serve millions of users. And unlike GPUs, which handle computation, HDDs provide the cost effective, high capacity storage needed to retain enormous amounts of data. This is also why demand for enterprise SSDs is accelerating. When hyperscalers cannot secure enough HDD capacity, they increasingly explore high density QLC SSDs as an alternative, which can create additional pressure on NAND supply. And this is where things get REALLY interesting from an investment perspective. Companies like $WDC and $STX are positioned to benefit from constrained HDD supply, while $MU, $SNDK, and other memory manufacturers could benefit from increasing demand for DRAM and NAND. When customers are willing to secure capacity YEARS in advance, suppliers can gain stronger pricing power, better revenue visibility, and potentially higher margins. And that's why I continue to believe the AI infrastructure buildout is going to create enormous opportunities across the ENTIRE supply chain, not just GPUs. We're moving into an environment where having access to critical components, securing long term supply agreements, and actually executing on infrastructure deployment could become major competitive advantages. Of course, shortages don't automatically mean every supplier will see higher profits. Higher prices can eventually encourage supply expansion, customers may shift technologies, and elevated infrastructure costs could pressure demand but the BIGGEST takeaway for me is that hyperscalers are making purchasing commitments extending into 2028 while STILL anticipating shortages.

Today, Dream Labs is coming out of stealth with Dream-1, a robot foundation model that controls bimanual robots it has never seen, on tasks it was never shown, in places it has never been, from $3/hour. Dream API is live for everyone, today. 🧵 https://t.co/MB1snhUOAJ

Amazon is opening AWS Trainium to PyTorch developers. With TorchNeuron, existing PyTorch AI workloads can run on Trainium with virtually no code rewrite. Less migration work. More hardware choice. More competition in AI chips. #AI #PyTorch #AWS #Trainium https://t.co/mxnq3tikSu

Introducing SGLang Low-Latency Mode, built with @NVIDIAAI and @AMD. On DeepSeek-V4.1-Flash with DSpark, it delivers up to 1.38× P90 per-user token speed on NVIDIA GB300 (671 to 928 tok/s) and 1.56× on AMD MI355X (332 to 516 tok/s) in @SemiAnalysis_'s AgentX trace replay. Highlights: - NVIDIA: overlapping kernels cut decoder layer time from 96.8 to 68.4 µs - AMD: 2 persistent kernels per layer replace dozens of small ones, with cross-GPU communication overlapped inside them. - New plugin system: custom kernels plug in from external libraries, with automatic fallback. - Low-latency kernels for GLM-5.3, K3, and MiniMax M3 are in progress and will plug in next.

The AI infrastructure buildout is about to enter an ENTIRELY different phase in 2027. SemiAnalysis just released a new report forecasting 51.2 GW of new data center capacity in 2027, compared to 25 GW in 2026 and just 15.6 GW in 2025. We're talking about more than DOUBLING annual capacity additions in a single year. To put that into perspective, SemiAnalysis estimates that the industry could add roughly as much data center capacity in 2027 alone as it had built cumulatively through 2024, excluding China. And here's where things get REALLY interesting. While some investors, including Brad Gerstner, have questioned whether the industry can actually deliver this much capacity, SemiAnalysis points to satellite imagery showing more than 24 GW of data centers already in advanced stages of construction, with those facilities expected to be delivered in 2027. On top of that, the firm reports roughly $1 TRILLION in binding, non cancellable data center leases signed in 2026 so far. Now, think about what this means for the ENTIRE AI ecosystem. Every additional gigawatt of data center capacity creates enormous demand for GPUs, HBM memory, networking equipment, cooling systems, electrical infrastructure, and POWER. And remember, adding data center capacity doesn't automatically mean every facility will be fully equipped with GPUs on day one. In fact, SemiAnalysis is already raising the possibility that as power and construction constraints ease, the next major bottleneck could once again shift towards semiconductors and memory. And here's what I think a LOT of investors are missing. More compute capacity doesn't necessarily mean demand suddenly gets saturated. As inference becomes cheaper, AI adoption can accelerate across autonomous agents, enterprise applications, robotics, and eventually physical AI. Lower costs can unlock entirely new use cases that simply weren't economical before, creating another wave of demand for compute. If SemiAnalysis is even remotely close to being right, we're about to witness one of the largest infrastructure expansion cycles in history. And I believe the companies supplying the compute, memory, networking, and power behind this expansion could be some of the biggest beneficiaries over the next several years. If you want to see how I’m investing around this next phase of the AI infrastructure buildout, check out my full portfolio below: https://t.co/Hkywss4Ugh

Anthropic says Claude will lead 80–90% of building the next Claude. The question for the rest of us is which 10% we keep. At the IMF AI & Cyber Risks panel, Anthropic's Michael Sellitto said Claude's share of leading next-gen model work went from 0% in January to 3% in April and 26% in August, and could hit 80–90% sometime in 2027. https://t.co/b2u0M9aTXj

Power is the BIGGEST constraint in the entire AI buildout right now. Morgan Stanley estimates that US data centers could require roughly 160 GW of power by 2029, but even after accounting for data centers under construction, additional utility grid access, behind the meter generation, and potential nuclear capacity, there could STILL be a 76.5 GW power shortfall. Think about how INSANE that is. We're talking about nearly HALF of projected US data center power demand potentially going unmet and here's why this matters so much. We can manufacture more GPUs, build better AI models, and raise hundreds of billions of dollars to finance new infrastructure. But none of that matters if you can't actually power the data centers. Electricity isn't something you can simply scale overnight. Building new transmission infrastructure, expanding grid capacity, securing interconnection agreements, and developing new power generation can take YEARS. And with AI workloads becoming increasingly compute intensive, demand for electricity is only going to accelerate as inference, autonomous agents, robotics, and physical AI continue scaling. This is exactly why I think POWER is becoming one of the most valuable assets in the entire AI ecosystem. And I think @elonmusk understands this better than almost anyone and it's also why I remain so bullish on neoclouds like $NBIS, $IREN, $SPCX. These companies are becoming incredibly important because they're helping solve one of the biggest bottlenecks in the AI industry turning massive demand for compute into actual operational capacity. The companies that can secure large amounts of power, develop data center campuses, deploy GPUs, and get that infrastructure operational FAST are going to have an enormous competitive advantage. And as power becomes increasingly scarce, the value of existing infrastructure with secured electricity and grid access could become significantly more valuable. You can't have an AI revolution without compute, and you can't have compute without POWER. If you want to see how I’m investing around the AI power bottleneck and the infrastructure companies I think benefit most, check out my portfolio below. https://t.co/MMkdjfqLsf

Announcing the beta release of the Pi API. We’ve partnered with companies across diverse industries to adapt our models to their robots and applications, from harvesting strawberries to assembling servers. Now we’re opening access to more teams. https://t.co/0T9tTFjVqM

2026 年 Q3，具身智能领域最有冲击力的变化，来自一个通用模型：9 月初 OpenAI 发布 GPT-6 Astra 后，具身从业者们发现，Astra 不只理解任务、拆解任务，还能输出机器人末端执行器的位姿，完成了本该属于 VLA 等物理 AI 模型的部分工作，而且还做得很不错。 在部分测试中，直接把 Astra 当做策略模型（VLA、WAM 等都是机器人上的策略模型类型）的成功率已超过了一些专门为机器人任务和物理世界操作训练的具身模型。 这动摇了数字世界 AI 和物理 AI 的边界，也引发了一个灵魂拷问： GPT、Fable 之类的通用模型是否会吃掉 Robotics？ 同时已经发生的事实是：OpenAI、Anthropic、DeepSeek、Kimi 等多数通用前沿模型公司都正酝酿进入 robotics。 除了 Astra 冲击外，本期具身季报还关注以下 4 个行业进展和事件： - AMD 拟以 82 亿美元收购 World Labs - 宇树 IPO 及连锁反应 - 有可能大幅压缩部署成本的 “上下文学习” - 美国 FCC 实际禁止了中国机器人进口 对一些人来说，Astra 的变化已经撼动了独立的具身智能公司的发展叙事和估值逻辑。至少，现在到了重新思考物理 AI 与通用 AI 边界的时刻。 https://t.co/ZRNrzkIDMe

Announcing the beta release of the Pi API. We’ve partnered with companies across diverse industries to adapt our models to their robots and applications, from harvesting strawberries to assembling servers. Now we’re opening access to more teams. https://t.co/0T9tTFjVqM

The cost for frontier LLMs to complete a robot subtask halved every 3 weeks, from $2.36 in July to $0.38 in September 2026. https://t.co/6XHOI3ZaSk

🚀RP1 is officially here. 🌍Putting control of robotics back in developers’ hands. Introducing RP1 (ROBOTO 01) by RoboParty—a high-performance, fully open-source humanoid robotics platform for frontier robotics research and embodied AI model development. 🤖 Open across the full hardware and software stack: • Robot platform: mechanical CAD, bill of materials, and assembly documentation • Actuators: in-house ROMOMO joint designs and related technical resources • Control: electrical systems, firmware, low-level drivers, and motion control implementations • Development: simulation models, SDKs, data processing tools, and training tools • Engineering foundations: technical documentation and research materials Open the black box. Understand the system. Make it your own. A deviation in motion can originate in the model, control policy, actuator response, or mechanical structure. RP1 opens up the design and implementation of these layers so developers can trace problems, modify the system, and validate changes on real hardware. Open-source algorithms for advanced motion control RP1 has demonstrated continuous walking, recovery from disturbances, getting back up after a fall, and human motion tracking. UFO and MimicLite connect motion learning, control policies, and execution on real hardware. The open toolchain connects data collection, simulation training, hardware deployment, and experimental validation—supporting joint iteration across models, control, and hardware. From RPO to RP1, we’re turning one team’s engineering experience into a starting point for many more. Opening the future of embodied AI to everyone ready to explore it. 🔗RP1 on GitHub: https://t.co/kMqFLi3eJ3 #RoboParty #RP1 #OpenSourceRobotics #EmbodiedAI

Announcing the beta release of the Pi API. We’ve partnered with companies across diverse industries to adapt our models to their robots and applications, from harvesting strawberries to assembling servers. Now we’re opening access to more teams. https://t.co/0T9tTFjVqM

interesting how all the ai researcher led robot foundation model startups are opening up their api now it’ll really expose who’s got a good and a not so good model and who has real generalization and that the model layer alone was never the moat this entire time when real world performance as the product is the way, as frontier models get better, they get better along with them

Metal Arm is available now Get yours at https://t.co/poa3dtPhOZ 6 DoF. 3kg Payload. Full CNC construction. Ships in 1 week. DMs open. https://t.co/BsDVyvZ1Dh

Introducing Natura-1, our first general foundation model for the physical world. It follows natural-language instructions zero-shot and turns them into action with dexterous hands. Natura-1 is a 50-billion-parameter mixture-of-experts (MoE) model pretrained on one million hours of egocentric human experience. On top of this foundation, we conduct efficient post-training on a limited amount of robot data to adapt pretrained world knowledge to dexterous manipulation, yielding a model that generalizes to a wide range of unseen scenarios. Natura-1 follows complex natural-language instructions zero-shot, reasoning through spatial constraints, multi-object relationships, and negative conditions. It can also autonomously carry out long and continuous workflows to perform long-horizon executions. Natura-1 provides early evidence that generalization—and even emergent behavior—can coexist with the complexity of dexterous hands. Further progress depends on the full stack: infrastructure for faster training and real-time inference, pretraining data at greater scale and coverage, and models that transfer to robots with less adaptation. We expect it to be a long-term research and engineering effort.

Announcing the beta release of the Pi API. We’ve partnered with companies across diverse industries to adapt our models to their robots and applications, from harvesting strawberries to assembling servers. Now we’re opening access to more teams. https://t.co/0T9tTFjVqM

Announcing the beta release of the Pi API. We’ve partnered with companies across diverse industries to adapt our models to their robots and applications, from harvesting strawberries to assembling servers. Now we’re opening access to more teams. https://t.co/0T9tTFjVqM

Announcing the beta release of the Pi API. We’ve partnered with companies across diverse industries to adapt our models to their robots and applications, from harvesting strawberries to assembling servers. Now we’re opening access to more teams. https://t.co/0T9tTFjVqM

Announcing the beta release of the Pi API. We’ve partnered with companies across diverse industries to adapt our models to their robots and applications, from harvesting strawberries to assembling servers. Now we’re opening access to more teams. https://t.co/0T9tTFjVqM

Morgan Stanley's Adam Jonas in new @SpaceX note: "The part of the SpaceX story that we believe holds the greatest growth potential, and that is being largely ignored/not contemplated by investors, is the Distributed Inference Cloud. Investors are already witnessing some degree of corporate/regulatory/social angst toward data centers. We see SPCX's efforts as instrumental in helping to get the Super Intelligence Economy and the Physical Intelligence Economy on the next leg of growth and economic development. As the 'density' of physical agents increases, we expect incrementally greater intelligence capability of the collective mobile swarm. We ask investors to be prepared to hear about the Distributed Inference Cloud and physical AI swarms in the coming year. From our seat, we think this subject is very close to breaking out in the investor discussion. Gradually… then suddenly." https://t.co/8Sua6sjcjE

Runway has been on a tear over the past couple months. Viral HORSE Video contest Solaris: the world's first Interface World Model Praxis-1: open-weight World Action Model for robotics Project Continuum: operating system research application built around real-time interfaces A video model. An interface model. A robotics model. An operating system. Are you getting it? These are not separate pretrains. They’re different applications, built around the same General World Model backbone, with different post-training on top. All of them are examples of what you can do when you can simulate any pixel. So I asked co-CEO Anastasis Germanidis to co-write an essay with me explaining how video, interfaces, operating systems, and robotics all fit together, why video was the right starting point for World Models, where the world is heading if Runway’s bets are right, and whether or not, as someone who simulates worlds, he believes we’re living in a simulation. Welcome to the future. Enjoy! (Video, of course, made with Runway)

Introducing Natura-1, our first general foundation model for the physical world. It follows natural-language instructions zero-shot and turns them into action with dexterous hands. Natura-1 is a 50-billion-parameter mixture-of-experts (MoE) model pretrained on one million hours of egocentric human experience. On top of this foundation, we conduct efficient post-training on a limited amount of robot data to adapt pretrained world knowledge to dexterous manipulation, yielding a model that generalizes to a wide range of unseen scenarios. Natura-1 follows complex natural-language instructions zero-shot, reasoning through spatial constraints, multi-object relationships, and negative conditions. It can also autonomously carry out long and continuous workflows to perform long-horizon executions. Natura-1 provides early evidence that generalization—and even emergent behavior—can coexist with the complexity of dexterous hands. Further progress depends on the full stack: infrastructure for faster training and real-time inference, pretraining data at greater scale and coverage, and models that transfer to robots with less adaptation. We expect it to be a long-term research and engineering effort.

Introducing Natura-1, our first general foundation model for the physical world. It follows natural-language instructions zero-shot and turns them into action with dexterous hands. Natura-1 is a 50-billion-parameter mixture-of-experts (MoE) model pretrained on one million hours of egocentric human experience. On top of this foundation, we conduct efficient post-training on a limited amount of robot data to adapt pretrained world knowledge to dexterous manipulation, yielding a model that generalizes to a wide range of unseen scenarios. Natura-1 follows complex natural-language instructions zero-shot, reasoning through spatial constraints, multi-object relationships, and negative conditions. It can also autonomously carry out long and continuous workflows to perform long-horizon executions. Natura-1 provides early evidence that generalization—and even emergent behavior—can coexist with the complexity of dexterous hands. Further progress depends on the full stack: infrastructure for faster training and real-time inference, pretraining data at greater scale and coverage, and models that transfer to robots with less adaptation. We expect it to be a long-term research and engineering effort.

Introducing Natura-1, our first general foundation model for the physical world. It follows natural-language instructions zero-shot and turns them into action with dexterous hands. Natura-1 is a 50-billion-parameter mixture-of-experts (MoE) model pretrained on one million hours of egocentric human experience. On top of this foundation, we conduct efficient post-training on a limited amount of robot data to adapt pretrained world knowledge to dexterous manipulation, yielding a model that generalizes to a wide range of unseen scenarios. Natura-1 follows complex natural-language instructions zero-shot, reasoning through spatial constraints, multi-object relationships, and negative conditions. It can also autonomously carry out long and continuous workflows to perform long-horizon executions. Natura-1 provides early evidence that generalization—and even emergent behavior—can coexist with the complexity of dexterous hands. Further progress depends on the full stack: infrastructure for faster training and real-time inference, pretraining data at greater scale and coverage, and models that transfer to robots with less adaptation. We expect it to be a long-term research and engineering effort.