What Should a Software Team Spend on Agents? Simulation-Based Cost Optimization of Mixed AI-Human Teams
Steffen Herbold
Abstract
Context: AI agents are rapidly adopted in software development, but their token costs are rising and it is unclear how much a team should spend on agents, and on which tasks. Objective: We determine how teams should divide spending between human developers and AI agents to deliver the features a project demands at the lowest total cost. Method: We model mixed teams of human developers and AI agents with eight specialized roles as a discrete-time stochastic process covering bug injection and detection, technical debt, human supervision of agents, and release cycles with feature freezes. Using Monte Carlo simulation and evolutionary search, we determine cost-optimal teams for 432 scenarios varying developer capability, project velocity, debt dynamics, agent capability and price, and bug costs, and compare them with a human-only baseline. Results: Escaped bugs are the dominant cost driver, and 61% of the agent budget goes to finding bugs. Agent capability matters more than token price, since higher-quality outputs reduce downstream quality assurance costs. In teams with capable agents, human developers mostly fix bugs and leave the implementation of new features to the agents. Teams with high-capability agents are typically cheaper than human-only teams, but low-capability agents offer no consistent advantage. At realistic velocities, the optimal teams still spend more on developers than on agents. Conclusions: There is neither a universal token budget per developer nor a universal agentic team structure. As in traditional development, this depends on many factors and teams need to be carefully tailored to the project context to achieve the optimal cost-quality trade-off.
Create a lesson
Related papers
Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents
Yuyao Ge, Shenghua Liu, Peizhong Gao et al.
SWE-Stream: Scaling Executable Training Data for SWE Agents via Progressive Construction and Elastic Streaming
Chengyang Liu, Wen Yang, Shifeng Chen et al.
The Code of Theseus: Can Agents Rebuild Software from Recovered Design Documents?
Cezar Sas, Sandra Mitrović, Fabio Barth et al.
SWE-Nexus: Automated Composition of Software Engineering Tasks from Test Execution
Songwei Yu, Ziqi Miao, Jingxuan Xu et al.
LintEval: Benchmarking LLMs on Linter-Guided File Refactoring
Tongwei Deng, Jinyu Zhu, Yue Cao et al.
Temporal Adaptive Fine-Tuning for Code Comprehension
Alex Wolf, Pooja Rani, Harald C. Gall