conventional wisdom on pull request incentives is incomplete2025-12-07

The general adoption of LLM tools for programming have led to an increase in garbage-quality PRs. The problem is only getting worse and I believe there is a new reason for why this is happening.

Conventional wisdom

Historically, PRs that were made had a few different major motivators:

1. desire to fix or add to a tool they use
2. working to build status in OSS - short term, resume building for a job
3. working to build status in OSS - long term, becoming a community member
4. giving back to a community from a gift giving perspective

Sure, there are many other reasons and things get complicated, but this is the realm of what existed before mass adoption of LLMs for writing code.

Generally speaking, the worst PRs in this era were from #2. It represented a cohort of developers that were looking to boost their resume as quickly as possible, then never be heard from again. There is no incentive for these developers to write something maintainable given they don’t plan to do other than close an issue and share that with a recruiter. This mindset of transience is what lends to worse quality PRs than 1, 3, and 4.

What’s going on today?

As we fast forward to today: Claude Code, Cursor, and every other tool in this category have become commonplace for developers. Even if you don’t use them; certainly you hear about them and are asked about them all of the time now.

Recently, there’s been more maintainers posting about an onslaught of garbage quality PRs submitted:

The problem with adding "help wanted" issues on @github.com is that people just feed those to their agent, and loop over PR review comments like drones 🫠 I don’t have a problem with using AI, but submitting vibe-coded slop is wasting everyone’s time. I doubt this will be enough, but it’s a start:

— François Best (@francoisbest.com) 2025-12-07T08:23:46.885Z

Trend I’m seeing getting worse recently: a legit GitHub issue gets filed. Somehow people latch onto it as something they can spam us on. Multiple similar PRs, often from different people sharing bits of code, that look 90% plausible but with blatantly obvious tells.

— Joe Savona (@en-js.bsky.social) 2025-12-06T20:05:27.441Z

I am burning out on this... Multiple PRs created by AI-bot accounts are trying to solve the same issue that has not yet even been identified, with verbose plain-text PR descriptions. I don’t even know if I should bother to reply... Any ways to stop this? 😇

— Anthony Fu (@antfu.me) 2025-12-03T07:06:26.338Z

It’s clear that this is a real problem. It also feels clear that GitHub is unlikely to take meaningful action to prevent this sort of activity. Here is an unanswered discussion where community members are simply asking to block PRs and Issues directly generated by Copilot. It goes against the product roadmap of GitHub to police this activity.

Drawing conclusions from historical knowledge

If we use our existing knowledge for what motivates someone to make a PR, the most likely reason would be assumed to be some variant of #2 - working to build status in OSS - short term, resume building for a job. From talking with developer friends and seeing speculation in threads, this is the sort of reasoning I see most people point towards when guessing why these PRs are made. They are PRs generated en masse to build profiles for a resume or portfolio.

Something that added to my confidence that this is the current conventional wisdom was putting the following prompt through kagi assistant (my daily driver LLM search tool).

Prompt:

why do people crawl github for issues and submit ai generated prs without reading them

Response:

People crawl GitHub for issues and submit AI-generated PRs without reading them primarily for low-effort contribution farming and resume padding. This behavior stems from several motivations:

Key motivations behind this behavior:

• Quick contribution credits - Contributors want to appear active on GitHub profiles without investing real effort
• Hallucinated solutions - AI tools generate plausible-looking code that appears genuine until closely examined[^1]
• Template replacement - Submitters often replace PR templates entirely with AI-generated text, indicating they haven’t read project requirements[^3]
• Volume over quality - The ease of generating AI content enables mass submissions without understanding the actual issues

Why this is problematic:

The AI-generated submissions often introduce security vulnerabilities and correctness bugs because the AI fails to respect API contracts or hallucinates non-existent functionality[^2]. Unlike human contributors, there’s no one available to discuss or refine the code when issues arise.

Maintainer response:

Many projects are now implementing explicit policies to reject low-effort AI-generated PRs, as these submissions have become the dominant form of low-quality contributions that waste maintainer time[^1].

[^1]: Policy: Reject low-effort AI-generated PRs · Issue #893... (48%)

[^2]: Should we have a policy on AI-submitted pull requests? (29%)

[^3]: Something about AI-generated pull requests · Issue #62666 - GitHub (23%)

The work needed to create a PR has lowered

This means that anyone working for any mix of the motivations stated at the top of the post is able to post more PRs if their bottleneck was typing. This, of course, is not always the case:

We ran a randomized controlled trial to see how much AI coding tools speed up experienced open-source developers. The results surprised us: Developers thought they were 20% faster with AI tools, but they were actually 19% slower when they had access to AI than when they didn’t.

— METR (@metr.org) 2025-07-10T19:46:11.941Z

I’ve personally been able to see more of my ideas come to life, but I’ll leave the productivity discussions for another day. For this post, it’s undeniable that it is significantly easier to make a PR without any time spent thinking or reading about an issue. We now have full-blown developer guides for how to make GitHub PR bots in "15 minutes".

Barrier to entry is lowered to the floor

With the cost of PR creation drastically lowered, people with new motivations and skillsets are welcomed into the space. These barriers of entry that used to exist are all gone

• knowing how to use git
• knowing how to read & write code
• having the time to spend reading through an issue and running the codebase locally

This creates a world where non-developers can make PRs in an automated way. With the lowered barrier to entry, we get a whole new category of users on GitHub. For better and worse.

New people means new motivations

Over the past few years we’ve seen an increase in supply chain attacks

• Shai-Hulud
• eslint prettier plugin attack
• Log4Shell
• plenty of others

There’s been some reporting on how phishing emails have been used as a way to get maintainer’s publishing tokens. And thankfully some moves towards more security in publishing.

There’s a concept in phishing called "chumming" or a "double-barreled attack." Attackers send an initial low-effort email not expecting it to succeed, but to see who bites. A reply marks you as someone who engages with strangers. Then comes the real attack.

AI slop PRs map onto this uncomfortably well. Generate PRs en masse across repositories. Cost is near zero. No expectation that most will be merged.

A maintainer who merges the PR reveals they don’t review carefully. A maintainer who engages extensively, explains problems, requests changes, goes back and forth, reveals they’re stretched thin and responsive. A maintainer who closes immediately with visible frustration reveals burnout. Each of these are useful signals.

The follow-up could be phishing for publishing credentials. It could be a second, cleaner PR with something malicious buried in legitimate-looking changes. It could be social engineering to become a maintainer on an understaffed project. It could be simply getting added as a new maintainer, like in the Log4Shell attack. Or it could just be waiting. Burned out maintainers eventually hand off keys or stop reviewing carefully.

Even without a follow-up, the attacker wins something. Every minute spent triaging slop is a minute not spent on real contributions or security reviews. It’s a denial of service that doesn’t look like one.

What to do about it

Policing this stuff is messy and hard. I don’t pretend to have a good grasp on how it can all happen. That said, I’d like to call GitHub to action on this. There is an opportunity to be pioneers in the Trust and Safety space and show it off in a very public way. I’d like the story of GitHub Trust and Safety to mirror the way they leaned into Dependabot years ago. These are very visible and helpful toolsets that can bring happiness, trust, and productivity to the platform.

In the meantime, there have been good attempts from community members to work towards some level of defense for projects:

• automated PR bots: https://octo.guide
• there might be space for prompt injections in PRs
• AI disclosures, but these can erode trust with good faith actors as well

This post exists to collect my thoughts and share how I find this to be very similar to traditional phishing. If platforms want to defend against supply chain attacks, slop PRs should be treated as malicious attacks and not just endless clutter for maintainers to clean up for free.