In-Context Molecular Property Prediction with LLMs: A Blinding Study on Memorization and Knowledge Conflicts (opens in new tab)

The capabilities of large language models (LLMs) have expanded beyond natural language processing to scientific prediction tasks, including molecular property prediction. However, their effectiveness in in-context learning remains ambiguous, particularly given the potential for training data contamination in widely used benchmarks. This paper investigates whether LLMs perform genuine in-context regression on molecular properties or rely primari...