CompeteAI: An Artificial Intelligence AI Framework that Understands the Competition Dynamics of Large Language Model-based Agents
Competition significantly shapes human societies, influencing economics, social structures, and technology. Traditional research on competition, relying on empirical studies, is limited by data accessibility and lacks micro-level insights. Agent-based modeling (ABM) emerged to overcome these limitations, progressing from rule-based to machine learning-based agents. However, these approaches still struggle to accurately simulate complex human behavior. The advent of Large Language Models (LLMs) has enabled the creation of autonomous agents for social simulations. While recent work has explored LLM-based agents in various environments, studies specifically examining competition dynamics remain sparse. This gap hinders a comprehensive understanding of competition across different domains.
Empirical studies on competition have uncovered valuable insights, such as inter-team competition fostering intra-team cooperation and the “Matthew Effect” in academia. However, these studies face limitations in controlling variables and collecting comprehensive data. Recent advancements in LLM-empowered-ABM have revolutionized social simulations. Notable projects include the Generative Agent, which established a foundational framework for agent designs, and studies exploring information dissemination, recommendation systems, and macroeconomic environments. Significant progress has also been made in collaborative cooperation simulations.
Despite these advancements, research on competition mechanisms using LLM-based agents remains limited. Existing studies have explored auction scenarios and corporate competition, but they fall short of simulating complex competitive environments and thoroughly analyzing competitive behaviors and system evolution. This gap in research presents an opportunity for more comprehensive studies on competition dynamics using LLM-based agent simulations, which could overcome the limitations of traditional empirical studies and provide deeper insights into competitive phenomena.
Researchers from the University of Science and Technology of China, Microsoft Research, William & Mary, Georgia Institute of Technology, and Carnegie Mellon University introduce CompeteAI, a comprehensive framework to study competition dynamics between LLM-based agents. The framework consists of environment selection, setup, simulation execution, and analysis. Using GPT-4, researchers developed a virtual town simulation with restaurant and customer agents. Restaurant agents compete to attract customers, driving continuous evolution and innovation. Customer agents, with diverse characteristics, act as judges by selecting restaurants and providing feedback. This setup allows for a detailed examination of competitive behaviors and system evolution. The framework begins with selecting an appropriate competition context, followed by environment setup, running experiments to capture agent interactions, and finally analyzing behaviors to derive insights into competition dynamics. Also, the framework’s core component is creating a competitive environment with meticulously designed competitors, judges, and interactions. Constraints, such as resource and service limitations for competitors or financial restrictions for judges, are crucial for success. The design is inspired by resource dependence theory, where competition for resources influences organizational behavior and strategies.
The CompeteAI framework implements a simulated small-town environment with two competing restaurants and 50 diverse customers. The simulation runs for 15 days or until one restaurant quits. Both restaurants and customers are powered by GPT-4 (0613) LLM-based agents. Restaurant agents manage their establishments through pre-defined actions like modifying menus, managing chefs, and creating advertisements. Customer agents, either individuals or groups, choose restaurants daily based on provided information and leave feedback after meals.
To overcome challenges in practical implementation, the researchers developed a comprehensive restaurant management system with APIs, allowing text-based LLM agents to interact effectively with the simulated environment. The system incorporates diverse customer characteristics and relationships to trigger more realistic competitive behaviors. Restaurant agents analyze daily information, design strategies, and interact with the management system, storing summaries for future planning. Customer agents, with varying characteristics and group dynamics, make decisions based on restaurant information, personal preferences, and group discussions. Also, this framework includes a dish quality evaluation mechanism, considering factors such as the chef’s skill level, dish cost, and selling price. This empirical approach ensures a realistic representation of service quality in a competitive environment.
The researchers conducted experiments with 9 runs for individual customers and 6 runs for group customers. This analysis covered both micro-level and macro-level perspectives:
Micro-level results revealed the sophisticated behavior of LLM-based agents in the CompeteAI framework. Agents demonstrated contextual perception, analyzing scenarios from “shallow to deep” – examining customer flow trends, dish feedback, and rival actions before deeper strategic analysis. They employed classic market strategies including differentiation, imitation, customer orientation, and social learning. Customer decisions were influenced by multiple factors, with “satisfaction of needs” being crucial for all. In particular, individual customers valued the restaurant’s reputation more, while groups were more open to exploring new options, showcasing the framework’s ability to simulate diverse consumer behaviors.
The macro-level analysis uncovered several significant phenomena in the simulated competitive environment. Strategy dynamics exhibited a complex interplay of differentiation and imitation behaviors between competing restaurants. The Matthew Effect was observed, where initial advantages led to continued success for one restaurant through positive feedback loops. Interestingly, customer grouping diminished the “winner-take-all” phenomenon, occurring less frequently for group customers (16.7%) compared to individual customers (66.7%). Perhaps most importantly, competition consistently improved overall product quality. In 86.67% of cases, the average dish score in at least one restaurant improved over time, with average dish scores increasing by 0.26 for Restaurant 1 and 0.22 for Restaurant 2 from Day 1 to Day 15.
These findings demonstrate the complex dynamics of competition between LLM-based agents and provide insights into market behaviors, customer decision-making, and the impact of competition on service quality in simulated environments.
The CompeteAI framework introduces an innovative approach to studying competition dynamics using LLM-based agents. By simulating a virtual town with competing restaurants and diverse customers, the study reveals sophisticated agent behaviors aligning with classic economic and sociological theories. Key findings include the emergence of complex strategy dynamics, the Matthew Effect, and the impact of customer grouping on market outcomes. The research demonstrates that LLM-based agents can effectively simulate competitive environments, consistently improving product quality over time. This innovative framework offers valuable insights for future studies in sociology, economics, and human behavior, providing a promising platform for interdisciplinary research in controlled, realistic settings.
Check out the Paper and GitHub. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter..
Don’t Forget to join our 47k+ ML SubReddit
Find Upcoming AI Webinars here
Asjad is an intern consultant at Marktechpost. He is persuing B.Tech in mechanical engineering at the Indian Institute of Technology, Kharagpur. Asjad is a Machine learning and deep learning enthusiast who is always researching the applications of machine learning in healthcare.
