Abstract

With the rapid advancement of large language models (LLMs) for handlingcomplex language tasks, an increasing number of studies are employing LLMs asagents to emulate the sequential decision-making processes of humans oftenrepresented as Markov decision-making processes (MDPs). The actions in MDPsadhere to specific probability distributions and require iterative sampling.This arouses curiosity regarding the capacity of LLM agents to comprehendprobability distributions, thereby guiding the agent's behavioraldecision-making through probabilistic sampling and generating behavioralsequences. To answer the above question, we divide the problem into two mainaspects: sequence simulation with known probability distribution and sequencesimulation with unknown probability distribution. Our analysis indicates thatLLM agents can understand probabilities, but they struggle with probabilitysampling. Their ability to perform probabilistic sampling can be improved tosome extent by integrating coding tools, but this level of sampling precisionstill makes it difficult to simulate human behavior as agents.