re: aipol
@whitequark After burying my head in arxiv for a while I've decided LLMs themselves are interesting (but don't exist in a vacuum). When people call them fancy autocomplete, or make a big deal about non-determinism, I think they're directionally correct in noticing the stank around LLMs and avoiding them, but it still comes from a place of either wilful ignorance or uncritically repeating things they heard on the internet. I don't look up to that or aspire to be like that.
To make a more specific point, I'd like people to stop interpreting everything that comes out of a softmax() as a probability density function just because it's non-negative and sums to 1. Next-token prediction is a useful objective for pre-training because it's a way to kickstart the model to learn higher-level representations unsupervised, but the reason it's called pre-training is you *keep going after that.*
If it wasn't for:
* the data theft,
* the vandalism of public internet infrastructure,
* the environmental damage,
* the widespread copyright laundering and "clean room" re-implementation of code that is in the training data,
* the erosion of social norms around contributing to open source,
* their ability to one-shot intelligent people into believing bullshit,
* their corrosive effect on your intellectual abilities,
* the concentration of the means to write code in a handful of US companies,
* the impending financial crisis,
...then I'd support their use. I don't see most of these changing though. The only actually-open-recipe-open-data models I'm aware of are Nemotron, and they underperform given their size and architecture, so perhaps the secret ingredient is crime.
