AI's New Bottlenecks: Power, Portability & Memory—Not Just More Chips

AI infrastructure has three constraints that more accelerators cannot fix. Power, software portability, and memory. That’s the entire story this week, Admin, because it signals the end of one era and the start of another. Last week, in episode 145, we talked about the real-world roadblocks for AI, like public pushback on data centers. Patrick Moorhead's point just now proves that the roadblocks are also inside the building. The problem isn't just getting more chips; it's what you do with them, and what it costs—in watts and dollars—to actually run them. So, the conversation on AI is changing. It's getting quieter, more technical, and a lot more serious. Here’s the rundown of the signals you need to hear. First, the big one.

According to Gartner data, global spending on AI inference—that’s the cost of running the models—has now officially surpassed the cost of training them. This is the tectonic shift. It means we've moved from the R&D phase to the production phase. And production has rules. It has budgets. It has security audits. And right now, it’s a mess. Which brings us to the next headline: eighty-eight percent of enterprise AI agent pilots are being blocked by security and isolation problems. Let me say that again. Eighty-eight percent. This isn't about model capability. This is about the basic blocking and tackling of letting autonomous code run inside a company network without it stealing the crown jewels or setting the servers on fire.

The diagnosis from architects on the ground is that you can’t just prompt your way to safety. You need hardware-level isolation and transaction logging. You need to treat an AI agent like a new type of employee with a badge, a boss, and a budget. And that budget is getting serious scrutiny. Boardrooms are now demanding headcount-style governance on AI spending. The era of swiping a credit card for unlimited API calls is OVER. Executives want to see ROI. They're asking why a model is being used, what it's costing per transaction, and whether a cheaper, open-weight model running on their own servers could do the job instead. This is forcing a massive shift toward private cloud infrastructure and open-source models—not for ideological reasons, but for cold, hard cash reasons.

Compounding this is the problem of "Shadow AI." Security audits are revealing that the number of unmanaged AI agents running wild in corporate SaaS systems is five times higher than what IT teams are even tracking. It's the new shadow IT. Employees are connecting tools, creating automations, and opening up pathways to core databases without anyone in charge knowing. It’s a ticking time bomb. Meanwhile, the big players are still making moves. IBM and OpenAI just launched a strategic alliance to embed frontier models directly into enterprise backends. This is the counter-move to the open-source trend: go deep, get embedded in mission-critical workflows, and become too essential to rip out. OpenAI also just previewed "Ultrafast mode" for GPT-5.6, promising up to fourteen times the speed, and released a specialized cybersecurity model.

Sam Altman’s message was clear: "please consider using our models to help defend your systems." It’s a classic move: sell the solution to the problem you helped create. But even as the big models get faster, the open-source world is getting smarter. A trend called "open autoresearch" is spreading, trying to create autonomous AI systems that can conduct scientific research. The results so far are… not great. One experiment gave an AI agent six days, internet access, and a three-thousand-dollar budget. It produced papers that human reviewers scored a 1 out of 6 and a 2 out of 6, citing flawed reasoning. So, the Skynet takeover can wait. But the ambition is there. And companies like Z.ai are pushing ahead, preparing to open-release a model called GLM-5.3 specifically for cyber defense, arguing it’s a responsible path.

Finally, there's a fascinating new idea bubbling up about what all this means for software itself. An analyst named Artemis just dropped a thesis that as AI makes building software cheaper, Figma—the design tool—won't just be for humans. It will become the shared design layer for humans and agents. A place where we define the visual rules and workflows that our new AI coworkers will use to get things done. So what does it all add up to? The age of AI spectacle is fading. The age of AI plumbing has begun. Let's go deeper on that, because the real story is in the details of how enterprises are trying to tame this technology. The work has shifted from speculative pilots to the hard reality of production. And the person who has the clearest view of this is an enterprise architect who goes by Shakthi.

His daily briefings are a masterclass in what’s actually happening on the ground. The first thing to understand is that the primary barrier to scaling AI is no longer the model. It's what Shakthi calls the "business context translation gap." A model can be brilliant, but if it doesn't understand the dynamic, real-time operational rules of a business, it stalls. It's like hiring a genius who doesn't speak the language and has no access to the company directory or sales reports. They can’t do anything useful. Millions of enterprise context switches—the little gaps in workflows where a human has to step in—are holding back autonomous agents. The models need structured, contextual memory fed into them live, during the transaction.

This is why that Gartner data point is so critical. The moment inference—running the model—costs more than training it, the entire economic equation flips. You're no longer paying for a static asset. You're paying for a continuous, operational process. And this changes the infrastructure you need. It’s not about massive training clusters anymore. It’s about dynamic, low-latency inference fabrics that can handle constant traffic from thousands of tiny agentic tasks. It’s about re-architecting your entire network to handle things like continuous KV-cache exchanges, which is the technical term for an AI's short-term memory. Now, remember that eighty-eight percent failure rate for AI agent pilots? Here's the core of the problem.

Companies tried to scale these pilots using the same security model they use for human employees or traditional software. It failed. According to Shakthi's analysis, scaling autonomous multi-agent systems requires a completely new security paradigm built on three pillars. One: Hardware-level microVM isolation. A microVM is a tiny, super-lightweight virtual machine. The idea is to wrap every single task an AI agent performs in its own disposable, isolated container. If one task gets compromised or goes haywire, you just kill the container. The blast radius is tiny. It's the digital equivalent of giving every worker their own sealed-off office for every single task they perform. Two: Strict identity mapping.

Every agent, every action, needs to be tied to a specific identity that your security systems can track. You can't have anonymous, untraceable AI agents running around your network. It sounds obvious, but it’s incredibly hard to implement when agents can be created, cloned, and deployed in seconds. And three: SIEM-integrated transaction logging. SIEM stands for Security Information and Event Management. It's the central nervous system for a company's security team. The point is that every single thing an agent does—every query, every API call, every file access—has to be logged and fed directly into the existing security dashboard. The security team needs to see agent activity with the same clarity they see human activity.

Without this, you have that five-fold "Shadow AI" sprawl we talked about. This is the hard, unglamorous work of enterprise AI in 2026. It's not about prompt engineering. It's about systems engineering. It's about governance, risk, and compliance. And it’s why the companies that figure this out are the ones who will actually make money and create value, while everyone else is still stuck in pilot purgatory. So if that's the messy, complicated backend, what about the frontend? How do we, the human users, actually work with these things? This brings us back to that thesis from Artemis: Figma will become the design layer for humans and agents alike. This is one of those ideas that sounds a little abstract at first, and then it clicks and you realize it might be the future.

Think about it. Right now, we interact with AI mostly through chat boxes. We give it a text prompt, it gives us a text response. Or maybe it generates an image. It's a very simple, one-to-one interaction. But that's not how real work gets done. Real work involves complex, multi-step workflows, collaboration, and a shared understanding of a visual interface. This is where Figma comes in. For years, it’s been the place where designers and developers meet to create the user interfaces for our apps and websites. It’s a shared canvas, a single source of truth for what the software should look like and how it should behave. Now, imagine an AI agent is a new type of user on your team. This agent's job is to, say, process customer support tickets.

It needs to understand the interface of your help desk software. It needs to know where the "escalate" button is, what the different ticket statuses look like, and how to fill out the form to close a ticket. You could try to teach it this by writing thousands of lines of code and descriptive text. Or… you could just point it to the Figma file. If Figma becomes the "design layer for agents," it means we start designing our software not just for human eyes, but for machine eyes. The Figma file becomes the instruction manual. An agent could "look" at the design, parse the components, and understand the layout and the intended workflow without a human having to explicitly code every single step. The design itself becomes the API.

This connects directly back to the enterprise problem of context. Remember how agents stall because they lack structured operational context? A well-designed Figma file is structured operational context. It defines the rules of the road for the user interface. This flips the whole script. The fear has been that AI will replace designers and developers. But this model suggests something different. It suggests the value of design and clear system architecture becomes EVEN MORE critical. If your Figma file is a mess, your AI agent will be a mess. If your workflow is illogical for a human, it will be impossible for an agent. Garbage in, garbage out. This also creates a powerful feedback loop. An AI agent, as it performs its tasks, could provide feedback on the design.

It could say, "This workflow is inefficient. I'm performing these three clicks a thousand times an hour. Can we create a shortcut button?" The agent becomes a QA tester and an optimization engine, all based on the shared language of the design file. This is not science fiction. Companies are already building tools that turn Figma designs directly into code. The next logical step is to have agents that can read those designs and operate the resulting software. The phrase "design layer for agents" is doing a lot of work, as one commenter pointed out. But it's pointing in the right direction. It suggests a future where AI isn't just a tool we use, but a collaborator we design for. The conversation is moving past what the models can do in a vacuum, and toward how we integrate them into the messy, visual, and collaborative world of human work.

The chatbox was just the beginning. The real revolution starts when AI gets a user interface. This week marks the end of the AI gold rush. The frenzy of prospecting for new models is giving way to the much harder work of building the towns, the railroads, and the laws that make a civilization run. The constraints are no longer theoretical; they are here. Power, security, cost, and context. These are the new keywords. The winners won't be the ones with the biggest model, but the ones with the best plumbing. The future of AI isn't being written in research papers anymore. It's being written in budget approvals and infrastructure diagrams.