Hear, hear!

I've been proclaiming this exact point as part of my "Resurrecting the Dead" talk.

Code review of GenAI PRs is no longer about training the next generation of potential maintainers, making it just a time sink, sucking the joy out of being a maintainer.

Well done AI companies.
OSS can't survive without community, so destroying the communities is a great step towards destroying open source (in favour of the AI closed garden where tokens costs 100x what they do now)....

@jrf_nl @elazar The fact that AI can’t learn is one of the worst parts about using it. There are certain things that are so prevalent in the training data that any amount of “memory” files stressing the importance of doing things a certain way will not overcome the tendency to follow what it was trained on.

One of those things is using commas instead of concatenation operators in PHP echo statements.

@ramsey As it happens, I came across this article today, and think it makes a similar and relevant point, albeit in a round-about and likely unintentional kind of way.

"Retrieval is the wrong tool for finding code an agent can fetch itself, and the right tool for remembering things an agent has no way to rediscover from the repo as it stands today."

A limited and often lacking ability to curate and navigate memories heavily curtails the usefulness of models.

https://www.qodo.ai/blog/we-built-a-state-of-the-art-rag-system-for-code-review-in-qodo-2-4-we-took-most-of-it-out/

@jrf_nl

@elazar @jrf_nl The best way to enforce things, even according to Claude’s docs, is to use static analysis and coding standards tools as part of pre-commit hooks. This will restrict the AI from making the commit until the issues are fixed. It sucks, though, because it’s a waste of tokens for it to churn like that.

@elazar @jrf_nl It also has to be part of an enforced hook because, if you leave it up to the AI to remember to always run them before committing, there will be many times they’ll just “forget” and commit straight away. When called out, it’s an “Oh, yeah! I was supposed to do that!” moment.

They cannot be relied on to be deterministic. They’re still just advanced auto-complete.