Hear, hear!

I've been proclaiming this exact point as part of my "Resurrecting the Dead" talk.

Code review of GenAI PRs is no longer about training the next generation of potential maintainers, making it just a time sink, sucking the joy out of being a maintainer.

Well done AI companies.
OSS can't survive without community, so destroying the communities is a great step towards destroying open source (in favour of the AI closed garden where tokens costs 100x what they do now)....

@jrf_nl @elazar The fact that AI can’t learn is one of the worst parts about using it. There are certain things that are so prevalent in the training data that any amount of “memory” files stressing the importance of doing things a certain way will not overcome the tendency to follow what it was trained on.

One of those things is using commas instead of concatenation operators in PHP echo statements.

@ramsey As it happens, I came across this article today, and think it makes a similar and relevant point, albeit in a round-about and likely unintentional kind of way.

"Retrieval is the wrong tool for finding code an agent can fetch itself, and the right tool for remembering things an agent has no way to rediscover from the repo as it stands today."

A limited and often lacking ability to curate and navigate memories heavily curtails the usefulness of models.

https://www.qodo.ai/blog/we-built-a-state-of-the-art-rag-system-for-code-review-in-qodo-2-4-we-took-most-of-it-out/

@jrf_nl

@elazar @jrf_nl The best way to enforce things, even according to Claude’s docs, is to use static analysis and coding standards tools as part of pre-commit hooks. This will restrict the AI from making the commit until the issues are fixed. It sucks, though, because it’s a waste of tokens for it to churn like that.

@ramsey @jrf_nl I do appreciate the validation that AI has brought to the best practices that have become established over the last two decades or so: the importance of context in architecture and implementation, documentation and automated tests as first-class deliverables, static analysis tools, etc. I think it's given me a new appreciation of the fact that what makes developing software easier for us also makes it easier for AI agents.

@ramsey @elazar I favour building community and upskilling people over using a completely unethical tool just because it is convenient, so, to me, forbidding any kind of AI usage is still the right choice.

Especially for software which requires a lot of contextual knowledge to create a good (correct) patch, like PHP_CodeSniffer. Even more so, knowing that PHPCS is used by AI to "keep it honest" 😄

@jrf_nl @elazar Yeah, and this is what I worry about: death of communities and loss of skilled programmers.

@ramsey @elazar Research has already demonstrated that usage of AI leads to "deskilling" of people. After about three months they don't catch mistakes the AI makes anymore....

If AI usage in software can only safely be allowed in combination with strict code review, then deskilling the people who need to review is not the way to go.

But hey, what do I know ?