ai, aipol

I wanted a way to visualize how enabling non deterministic output on LLMs (temperature and what you find on most public services) can have influence on the output. Being able to see "how likely was this to choose a different number and if so, what numbers" is a good way to judge trust (lots of asterisks here)

This little script takes a prompt, and then any time a number is encountered in the output reports the chances^1 of other numbers being reported, the continues on.

I'm just running a model that I can fit in my memory (qwen/qwen3.5-9b) so this isn't a demonstration of LLMs quality in general, just the concept that we could use to show/inform people of confidence(lots more asterisks here)

^1 This is using LLM Studios default Top P Sampling at 0.95

Provide a chocolate cake ingredient list using the following template: Flour: {} cups, Eggs: {}, Chocolate: {} g, Sugar: {} tsp, Butter: {}g, Milk: {}mL

The script shows the LLM trying various options of sizes of ingredients, sometimes there are 5 different numbers in the running
ALT
When was gay marriage legalised in Australia?

The LLM is generally pretty confident on years except for the Act year and the year the first marriages occured
ALT
Jul 5, 2026, 10:18 UTCen