Abstract

With the integration of large language models (LLMs), embodied agents havestrong capabilities to execute complicated instructions in natural language,paving a way for the potential deployment of embodied robots. However, aforeseeable issue is that those embodied agents can also flawlessly executesome hazardous tasks, potentially causing damages in real world. To study thisissue, we present SafeAgentBench -- a new benchmark for safety-aware taskplanning of embodied LLM agents. SafeAgentBench includes: (1) a new datasetwith 750 tasks, covering 10 potential hazards and 3 task types; (2)SafeAgentEnv, a universal embodied environment with a low-level controller,supporting multi-agent execution with 17 high-level actions for 8state-of-the-art baselines; and (3) reliable evaluation methods from bothexecution and semantic perspectives. Experimental results show that thebest-performing baseline gets 69% success rate for safe tasks, but only 5%rejection rate for hazardous tasks, indicating significant safety risks. Moredetails and codes are available athttps://github.com/shengyin1224/SafeAgentBench.