It’s just not, no matter how many times people say it. When it regurgitates data, that’s an artifact of the training process. It means that information was not properly encoded into the whole

And it most certainly is creative, as the eldrich mathematical construct it is, it can produce truly new concepts. More creative than humans, even - we very, very rarely have a brand new idea. Instead, we’re masters of lateral thinking - we take patterns we learn in one place and apply them to something entirely different. Our creativity is probably significantly more useful, but LLMs are creative in a way unbounded by human logic. It’s fair to say they’re unhinged in their creativity

The first time I really understood this was when, after hallucinating hide and seek several times (at the time I was running an LLM with no context window or system prompt just to get a feel for how it worked), it came up with a game called hide the comma, where it gave you a sentence with missing punctuation.

I’m pretty confident no human has ever done that, because it’s really fucking stupid. It’s not useful at all, you can put a comma in all sorts of places and completely change the meaning of a sentence. The AI would agree to any answer you give, because they’re chaotic little chat bots wired to meet expectations

But it was a brand new idea, an unhinged one, but creative in an inhumanly creative way