Abstract

Large language models (LLMs) suffer from temporal misalignment issuesespecially across long span of time. The issue arises from knowing that LLMsare trained on large amounts of data where temporal information is rathersparse over long times, such as thousands of years, resulting in insufficientlearning or catastrophic forgetting by the LLMs. This paper proposes amethodology named "Ticktack" for addressing the LLM's long-time spanmisalignment in a yearly setting. Specifically, we first propose to utilize thesexagenary year expression instead of the Gregorian year expression employed byLLMs, achieving a more uniform distribution in yearly granularity. Then, weemploy polar coordinates to model the sexagenary cycle of 60 terms and the yearorder within each term, with additional temporal encoding to ensure LLMsunderstand them. Finally, we present a temporal representational alignmentapproach for post-training LLMs that effectively distinguishes time points withrelevant knowledge, hence improving performance on time-related tasks,particularly over a long period. We also create a long time span benchmark forevaluation. Experimental results prove the effectiveness of our proposal.