How AI agents are changing the rules of software governance
InfoWorld ·

The conversation around AI governance is already happening. Organizations have spent the past few years accelerating AI adoption across software development. Now they face a different problem. AI systems can act faster than the organizations responsible for them can understand or govern what is happening. Organizations are thinking about compliance, data privacy, security, and model reliability while boards are demanding policies and regulators are developing frameworks. The challenge now is how to turn those newly implemented levers into tangible controls that work at the pace of today’s AI-driven software development without creating a bottleneck. AI has significantly lowered the barriers to generating code. Assistants can suggest individual functions, while increasingly capable coding agents can work across repositories, run tests, make changes, and execute multi-step development tasks. For software development organizations, the goal now is to scale AI across development workflows while ensuring that the people ultimately responsible for the software can still understand, verify, and stand behind what those systems produce. AI is changing the developer workflow Traditionally, AI assistants operate within relatively clear boundaries such as a coding assistant recommending certain functions or completing a section of code. Agents, however, operate entirely differently. Rather than responding to a single prompt, they can perform multi-step workflows. They can access tools, retrieve data, write code, trigger actions and coordinate tasks on behalf of users. For engineering teams, the ability for AI to increasingly take action within developer environments is the most profound change in how they work. When a developer accepts an AI-generated code suggestion, accountability is seemingly straightforward. The developer reviews the outputs and catches any errors before the code gets shipped into production. However, governance becomes murkier when AI agents are not only generating code but testing it, deploying it, opening a pull request, modifying multiple files, or interacting with other systems. The distance between decision and action becomes much shorter, which simultaneously means that the ability for humans to retain enough access to understand and monitor each step becomes much harder. As agents take on more of the execution, the developer’s role changes. Developers still need to understand the code, architecture, and systems they’re responsible for, but increasingly they must also decide what work to delegate, what context an agent needs, and how to verify the result. This is a fundamental change in the development workflow. A developer can delegate a task without delegating responsibility for the outcome. For engineering leaders, the challenge is creating workflows that make this practical across teams, without requiring developers to inspect every individual action an agent takes. Governance must move from policy to engineering practice Most organizations already have policies governing software security, access, and compliance. But policies alone cannot govern AI agents operating across repositories, tools, and development environments. As agents take on more complex work, the challenge is ensuring those policies translate into controls that operate within the development workflow itself. That means defining what agents can access, what actions they can perform, how their work is recorded and where human approval is required. These controls need to be built into the environment where development happens, rather than applied after an agent has completed its work. The goal isn’t to slow developers down with additional compliance requirements. It’s to create an environment where teams can confidently delegate more work to agents without losing visibility, quality, or control. Create an audit trail The first control to consider is traceability. Developers must understand what AI has contributed to a codebase. That means tracking AI-generated changes and making them auditable so that teams can still read, review, and understand the output. For example, if an agent makes changes across multiple files and runs tests, the development team should be able to identify exactly what actions were taken — including what was changed, which instructions were received, and which tests were run. If something goes wrong in production, this control arms teams with the knowledge they need to understand how exactly AI was involved and why something happened. Maintain human accountability Developers are ultimately responsible for software that reaches production, and there are significant consequences if something goes wrong — regardless of whether a portion of it was generated by AI assistants or agents. If a system fails, customers don’t care who (or what) wrote the code. Accountability ultimately sits with the organization and its engineering team. However, the goal should not be to put a human in the loop for every individual action carried out by AI. Instead, organizations need to clearly define where human judgment is required. For example, AI might be permitted to generate code and run tests automatically, while changes to production infrastructure or other critical systems must require explicit human review and approval. What’s important, though, is that these boundaries are defined in advance by organizations. The developer oversight should be weighed against the risk of a specific action and the need to scale that process throughout the entire development workflow. Define AI’s parameters Development organizations also need to clearly define what AI coding tools and agents can access, and which actions they can perform versus which should be prohibited. An AI tool that can modify application code does not necessarily need access to production infrastructure. A coding agent that can run tests does not necessarily need permission to change deployment configuration. This approach very much mirrors security principles that organizations already understand and implement. Systems should have only the permissions and privileged access they need to perform their assigned tasks. As AI becomes more deeply integrated into development environments, permissions need to become part of the engineering workflow instead of an afterthought. Measure AI usage and cost Another control that’s come under scrutiny — and is seemingly a bit nebulous — is understanding how AI coding tools are being used and what they cost. “ Tokenmaxxing ” has been one of the defining AI trends of 2026, yet there is growing realization among organizations that AI activity and AI value are not the same thing. Measuring usage rates, prompts, and token consumption can tell organizations how much AI they are using, but it does not necessarily tell them whether AI is delivering better outcomes or creating meaningful business value. Compounding that challenge, agentic systems can perform long, multi-step workflows and consume significant resources without a developer manually initiating every individual step in the process. Developer organizations therefore need a clear line of sight into which tools are being used, how frequently, and where costs are accumulating. That is starting to drive interest in what the industry is now calling a “ software harness ” — a layer of software around AI models that can provide context and give organizations great control over tasks and token costs. The objective for this control should ultimately be to understand more than just how much developers are using AI, but also what that usage is producing and whether that cost, risk, and level of autonomy are justified by the outcome. Governance must scale with AI autonomy Governance becomes more complicated as AI adoption moves beyond individual developers. What works when one developer is directing one agent becomes harder when multiple agents are operating across repositories, teams, and shared systems. This is where organizations need to move beyond individual controls and establish consistent standards for permissions, traceability, and human approval across development workflows. Without that shared foundation, the same autonomy that makes agents useful can also create fragmentation, inconsistent practices, and new risks. The challenge is not simply scaling the number of agents an organization can deploy. It’s scaling the organization’s ability to understand, coordinate, and take responsibility for their work. Scaling AI without surrendering control The most successful organizations will not be those that deploy the largest number of AI agents but the ones that are able to create the clearest lines of accountability around them. With all the discussions centered around autonomous systems replacing human work, it’s essential to remember that, in practice, when an AI output fails, customers blame companies not agents, and therefore responsibility will always land within engineering teams. Because of this, developers are no longer simply responsible for the code they write, they’re also increasingly responsible for reviewing, governing, and understanding the work of systems capable of generating far more software than any individual engineer could ever produce alone — and eliminating blind spots along the way. With the future of software development undoubtedly shaped by AI agents, the question is not whether organizations will adopt them but how they can remain in control of them. The competitive advantage will come from how much autonomy can be created without losing accountability and trust. As such, developer organizations need a new model built around visibility, accountability, and developer oversight from the start. The goal is not to limit what AI systems can do, but to ensure its capabilities remain transparent, auditable and ultimately directed by human expertise. — New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com .