Metrics for Evaluating Biological AI Model Predictive Accuracy at the Data-Substrate Level (opens in new tab)

Reports in the biological literature disagree on whether a given model can predict a biological outcome from a given data sample --- one study finding a model capable, another, on the same kind of data, finding it is not. This is particularly a challenge in relation to LLMs--where the models are large and opaque, with weights and training data inaccessible.textbf{ }Such disagreements cannot be settled by directly inspecting the model. To address this challenge, we considertextbf{ }an alternat...