LLM Targeted Underperformance Disproportionately Impacts Vulnerable Users (opens in new tab)

submitted by yogthos to technology2 points | 0 comments the thing that doesn’t get talked about enough. Everyone’s worried about AI taking jobs or whatever. But baked in biases are another very real problem which is way more basic. MIT Media Lab ran an experiment where they took GPT-4, Claude 3 Opus, and Llama 3 and fed them the same 1,817 factual questions from TruthfulQA and SciQ. Then they tried changing the user bio with one personal being a Harvard neuroscientist from Boston, another a P...