I usually don’t even understand the lingo they use. “Open-weighted” is the most recent one, then it usually goes down to specific “models” that everybody is supposed to know about.
These are my thoughts (I will stick to the vague “it” for now, but of course therein lies another question: “and how does all this apply to various specialised AIs”):
- Is it really feasible to run it 100% locally? I know there’s plenty of people with very powerful rigs indeed, but still. Or are 99% of these people really saying “it would, in theory, be possible to run that locally, therefore your concerns are invalid”?
- If yes to the previous: the software doesn’t come from nowhere and ultimately still relies on gas-turbine-powered datacenters and stolen IP and stolen personal data, no?
If what I wrote above is true, what exactly are people arguing when they say it’s still possible to use LLMs ethically or true to FOSS philosophy, because … ???
edit
Thanks to all who answered.
I guess it’s my fault for asking several questions in one, but this thread has attracted exactly the type of people I’m writing about; several even used the term “open-weighted models” without explaining it.
Asking to get arguments explained, I got more arguments instead.
Worth noting that a request is often dozens of requests now that there’s “reasoning,” even for search. As I understand it, a model will take a question , figure out the context (one request), reform the question so it yields better results (another request), if it’s doing a web search there’s requests for each result, another to compare, another to check if it answers the original request, if not it loops and does it all over again. So one request is easily, and often, dozens of requests. This is one way Ai companies can say to investors, “see, look at how much usage has increased.”
Also, we need to factor in the power to scrape and train each of those models, build the datacenters which is near impossible as these companies are not transparent about it and actively try to obstruct investigations into it. Then there’s the redundancy of all these different companies competing and doing roughly the same thing at the same time, as fast as they can, so it’s orders of magnitude inefficient energy consuming before it gets its first user prompt.
Comparing it to other assaults on the environment is not only hard to do, but a case of “the worse negates the bad” fallacy.
I do see what you’re saying. You can account for all of these factors and it still turns out that, largely, individual LLM use just doesn’t use that much power compared to most things people do day to day. Inference is so cheap that even dozens of requests don’t amount to much. I could look up and give you a bunch of numbers, but I don’t think that’s likely to convince anyone who doesn’t do the research themselves. It’s so easy to come up with sources that say what you want. I’d encourage you to actually look into this yourself.
Training costs are higher, but you train once and use repeatedly. Right now, total training costs are stupidly high, but that’s because we’ve got an arms race between the frontier labs to spend as much money and compute as they can for truly marginal gains in quality. The solution to that problem isn’t for individuals to stop using AI, it’s to stop those assholes from wasting so much power.
Individual LLM use is so cheap, that it really isn’t worth wasting people’s energies thinking about limiting that. Instead of being distracted by attempts to make this an issue of personal responsibility, we should be focusing on what will actually make a difference. We should be focused on supporting policies that lead to systemic change. A carbon tax would change corporate behavior right quick, and not just for AI companies.