Abstract

Causal discovery is essential for understanding complex systems, yettraditional methods often depend on strong, untestable assumptions, making theprocess challenging. Large Language Models (LLMs) present a promisingalternative for extracting causal insights from text-based metadata, whichconsolidates domain expertise. However, LLMs are prone to unreliability andhallucinations, necessitating strategies that account for their limitations.One such strategy involves leveraging a consistency measure to evaluatereliability. Additionally, most text metadata does not clearly distinguishdirect causal relationships from indirect ones, further complicating theinference of causal graphs. As a result, focusing on causal orderings, ratherthan causal graphs, emerges as a more practical and robust approach. We proposea novel method to derive a distribution of acyclic tournaments (representingplausible causal orders) that maximizes a consistency score. Our approachbegins by computing pairwise consistency scores between variables, yielding acyclic tournament that aggregates these scores. From this structure, weidentify optimal acyclic tournaments compatible with the original tournament,prioritizing those that maximize consistency across all configurations. Wetested our method on both classical and well-established bechmarks, as well asreal-world datasets from epidemiology and public health. Our resultsdemonstrate the effectiveness of our approach in recovering distributionscausal orders with minimal error.