In my previous blog, I explored why infrastructure is no longer just an IT consideration. AI, sovereignty, cyber resilience and rising consumption costs are turning infrastructure into a business decision, with consequences that extend well beyond the datacentre.
Following that blog, and after presenting on infrastructure economics at Dell Technologies Forum and JuiceIT in Sydney, I had several illuminating conversations with CIOs, CISOs and technology leaders about what comes next. While the discussion started with infrastructure, it quickly shifted to governance. Specifically, what happens when organisations begin deploying not dozens, but potentially thousands of AI agents across their business.
One CIO summed up the challenge in a way that really stuck with me. After discussing the scale of change agentic AI could introduce, he joked that he was glad he was approaching retirement. It was an offhand remark, but I think it’s a genuine concern about what this means for organisations responsible for securing and governing increasingly autonomous systems.
That concern isn’t hypothetical either. Gartner forecasts that the average Fortune 500 organisation could be running more than 150,000 AI agents by 2028, while many organisations are still working out how those agents should be monitored, controlled and governed.
My first blog focused on why infrastructure economics matters. This piece gets into the meat of what those infrastructure decisions are protecting: governance, security, accountability and control in an environment where AI agents could soon become as common as human employees.
Most organisations already have established ways of managing employees, devices and applications, but agentic AI introduces something different: systems that need identities, permissions, monitoring and a defined lifecycle. Organisations need to know what agents exist, what they can access, who is responsible for them and when they should be retired. The challenge is that many of these controls become harder to implement after deployment rather than from the outset. As agents become connected to data, applications and business processes, organisations need a clear way to govern how they operate, what they can access and how their activity is monitored.
Not every agent carries the same level of risk, however, and applying the same controls to every agent can create unnecessary friction. Applying too few controls can leave higher-risk agents operating without appropriate oversight. Organisations need a way to apply controls according to risk rather than treating every agent the same.
Without visibility and audit trails, governance gaps often surface only after an incident. An organisation that cannot see what its agents are doing cannot apply risk-based controls, cannot demonstrate what its agents are costing or delivering, and is less likely to identify a problem before it reaches production.
Following a recent presentation on agentic AI governance, I spoke with a Chief Security Officer in the transport sector who raised a concern. His view was that agentic AI doesn’t just expand the attack surface, but also creates the possibility of agents themselves being used to generate activity at a scale existing controls were never designed to manage. His immediate questions were simple: what governance policies exist for agentic AI today, and who owns them?
Like many security leaders, he was trying to understand how existing governance models apply when software is no longer just processing information but increasingly interacting with systems, accessing data and carrying out tasks on behalf of users.
A similar conversation followed my session at Dell Technologies Forum in August 2026. One security leader told me the discussion had completely changed the way he thought about the challenge. Up until that point, the focus had been on reducing the attack surface, but the bigger issue, he argued, was that agentic AI changes what a secure environment needs to look like. What this told me was that alongside traditional security concerns, organisations are now considering how autonomous systems should be governed, monitored and controlled as they become embedded across the business.
The conversations I’ve had with customers all come back to the same challenge. Governance sounds straightforward in principle, but becomes much harder in practice as organisations deploy more agents across more systems and business processes. Knowing what agents are running, what they can access, how they’re being monitored and when they should be retired requires more than policy alone. It also requires infrastructure designed to support those controls.
This is where the Dell AI Factory with NVIDIA comes into the discussion. As organisations look to apply governance, monitoring and lifecycle management at scale, the supporting infrastructure becomes an increasingly important part of the conversation.
| Gartner requirement | Dell AI Factory with NVIDIA | Nemotron / NeMo layer |
| Data sovereignty and control | On-prem/private cloud (PowerEdge, PowerStore, PowerScale) keeps agent data inside the org’s security boundary. | Nemotron is open and deployable on-prem (PowerEdge, Dell Pro Max, Deskside Agentic AI), helping organisations keep AI workloads and data within their own security boundary rather than relying on third-party AI APIs. |
| Full-stack observability | Integrated, validated Compute/storage/network/software stack instruments agent activity end-to-end. | Nemotron ships as Nano/Super/Ultra/Lightning; NeMo Switchyard auto-routes each task to the right-sized model. |
| Right-sized deployment | Infrastructure tiers matched to each agent’s actual risk and scope, not one-size-fits-all. | Nemotron is available in multiple model sizes, while NeMo Switchyard helps route tasks to the most appropriate model based on workload requirements. |
| Lifecycle identity management | Dell APEX and the AI Factory management layer help organisations manage agent lifecycle, access controls and operational governance at scale. | Open, post-trainable models let orgs retire/replace narrow agents without vendor lock-in. |
Beyond the models themselves, NVIDIA’s NeMo framework provides many of the governance and lifecycle capabilities needed to support agentic AI at scale. NeMo Agent Toolkit helps organisations monitor cost, latency and accuracy across agent frameworks, while NeMo Evaluator benchmarks agents before production. NeMo Guardrails/Safety applies policies at the individual agent level, and NeMo Switchyard automatically routes tasks to the most appropriate model.
Think of Nemotron as the models doing the work and NeMo as the management layer providing governance, monitoring and control capabilities around them. Together they help organisations manage agentic AI on infrastructure they control end-to-end.
Governance and cost are closely connected because an organisation that can’t see what its agents are doing will also struggle to understand what they’re consuming, what value they’re delivering and whether they should continue running.
Agentic workloads consume significantly more tokens than traditional chat-based AI interactions and can generate substantially higher inference demand. At that scale, consumption becomes a financial consideration on top of a technical one, with token usage, infrastructure capacity and ongoing operational costs all needing the same level of oversight as any other business investment.
Dell refers to this discipline as “tokenomics”: matching workloads, model size and deployment tiers to the task at hand so costs remain aligned to business value rather than growing unchecked with usage.
Independent analysis from Signal65 found that on-premises Dell AI Factory with NVIDIA infrastructure reduced costs by between 28 and 90 per cent compared to cloud AI APIs, with breakeven periods measured in months rather than years. The study found savings increased as workloads scaled, reinforcing the importance of matching infrastructure strategy to long-term consumption patterns.
The same visibility needed to govern an agent estate also helps organisations understand what those agents cost, how effectively they’re being used and whether they’re delivering the outcomes expected of them. In many cases, governance and cost control are two sides of the same challenge: understanding what is running, why it exists and whether it continues to justify the resources it consumes.
Many organisations are still working out what agentic AI governance should look like in practice. Infrastructure, governance, cost management and operational requirements all need to be considered together as adoption grows.
Data#3 and Dell Technologies help organisations assess those requirements early, validate workloads and governance needs, and design environments that can support agentic AI at scale.
Key areas of support include:
As agentic AI adoption grows, organisations will need to balance governance, security, performance and cost while maintaining visibility and control across their environments. Data#3 and Dell Technologies bring together the infrastructure, platform expertise and delivery experience to support that journey.
Book an Agentic AI Governance and Infrastructure Assessment workshop.
Information provided within this form will be handled in accordance with our privacy statement.