Beyond Black Boxes: Designing and Testing Agentic AI Systems for Strategy

This study explores the effect of different types of AI (general and agentic) on strategic decision making. We conducted a randomized controlled trial with 976 managers trained in the theory-based approach to strategy and tasked with solving a strategic challenge.We show that both general and theory-based agentic AI assistance increase participants’ confidence in their theories of value, but not their quality and probability of success as evaluated by experts and LLMs. AI assistance also reduces AI aversion and increases AI complacency and bias.We explain the de-coupling between confidence and quality of theories through “cognitive miserliness”, the preference to use minimal cognitive resources when processing information, surfacing how confidence, quality and probability of success of theories interact with AI complacency and aversion for different types of managers. Our heterogeneous treatment effect analysis shows that highly educated managers become more overconfident, AI complacent, and biased, especially if exposed to like-minded agentic AI systems that are trained to reason causally. Instead, more experienced managers do not inflate their beliefs and significantly improve the quality and probability of success of their theories of value. We contribute to the growing literature on how AI affects strategic decision making by revealing how managers’ and AI’s characteristics interact differentially impact strategic decisions.