Rethinking Technology Stack Selection with AI Coding Proficiency

Abstract

Large language models (LLMs) are now an integral part of software development workflows and are reshaping the whole process. However, existing technology selection methods mainly focus on the inherent attributes of technologies, overlooking whether the LLM can effectively leverage the chosen technology. Therefore, teams using LLM assistants risk choosing technologies that cannot be used effectively by LLMs, yielding high debugging effort and mounting technical debt. We foresee a practical question in the LLM era, is a technology ready for AI-assisted development? In this paper, we first propose the concept, AI coding proficiency, the degree to which LLMs can utilize a given technology to generate high-quality code snippets. We then conduct the first large-scale empirical study examining AI coding proficiency across 170 third-party libraries and six LLMs. Our findings reveal that libraries with similar functionalities can exhibit up to 84% differences in the quality score of LLM-generated code. These gaps can be translated into real engineering costs and steer developer choices toward a narrow set of technologies, threatening technological diversity in the ecosystem. We call on the community to integrate AI coding proficiency into technology selection frameworks and develop mitigation strategies, preserving competitive balance in AI-driven development.