Jon

And this goes for program analysis as well! Sure, it's impressive that Mythos-class LLMs can be used to identify oodles of problematic constructs in code that's been shipping for years, including tends of thousands of real bugs some of which are security vulnerabilities. It was also very impressive that PREfix and PREfast (the program analysis tools I worked on in back in the day) and the more-powerful tools that followed like Coverity could do it. Where would the program analysis field be today if billions of dollars had been invested in building on these tools instead of "AI"?

But none of these analysis tools change the underlying causes of the bugs -- software engineering processes that value time-to-market over security, unsafe libraries and languages, leaving security as an afterthought, etc etc etc. Don't get me wrong, finding and fixing bugs has value; one net effect this wave of LLM program analysis is likely to be useful hardening of existing software. But all the resources going to that aren't going to addressing the underlying issues -- and also reinforcing all the ethical, sustainability, and power-concentrating consequences of LLM usage.

An excellent thread here. So much of what I see people pointing to as LLM's benefits for coding relates to long-standing problems in software engineering that the field just hasn't addressed. And LLMs don't solve these problems, at best the just paper them over and make dealing with them less tedious -- while reinforcing the problematic dynamics.

So yes it's great that people with no programming skills can create software to solve their prolems. But if we had collectively spent a chunk of the literally billions of dollars that are going to "AI" building on the early approaches to this from 25+ years ago (Hyperscript, Logo) that don't have the same downsides, we'd be in a much better place today.