Abstract

In the last decade, Deep Reinforcement Learning has evolved into a powerfultool for complex sequential decision-making problems. It combines deeplearning's proficiency in processing rich input signals with reinforcementlearning's adaptability across diverse control tasks. At its core, an RL agentseeks to maximize its cumulative reward, enabling AI algorithms to uncovernovel solutions previously unknown to experts. However, this focus on rewardmaximization also introduces a significant difficulty: improper rewardspecification can result in unexpected, misaligned agent behavior andinefficient learning. The complexity of accurately specifying the rewardfunction is further amplified by the sequential nature of the task, thesparsity of learning signals, and the multifaceted aspects of the desiredbehavior. In this thesis, we survey the literature on effective reward specificationstrategies, identify core challenges relating to each of these approaches, andpropose original contributions addressing the issue of sample efficiency andalignment in deep reinforcement learning. Reward specification represents oneof the most challenging aspects of applying reinforcement learning inreal-world domains. Our work underscores the absence of a universal solution tothis complex and nuanced challenge; solving it requires selecting the mostappropriate tools for the specific requirements of each unique application.