What tools and projects do you use for your local LLM chats and agents?
I’m trying to sift through the mess of LLM harness and tools to find how to really get the most out of local LLMs.
What tools and projects do you use for your local LLM chats and agents?
I’m trying to sift through the mess of LLM harness and tools to find how to really get the most out of local LLMs.
AI effectively raped and pillaged all human information for the benefit of a handful of corporations. It’s use has been shown to reduce the skills of school children and even deskill those with established knowledge. Then there’s the environmental considerations in a time where the world is decades behind appropriate climate change management… Id say it’s pretty understandable that it evokes passionate responses.
Copying information is not theft. Learning from copied information is not theft.
Theft is when you take something and deprive the owner of what you took. If I make a copy of your car, I am not stealing your car, you still have it and can drive it, wreck it, whatever.
So google ai summary now taking content from the web and putting at the top of their search, thus generally stopping people clicking on websites and taking their revenue -> theft.
I’m sure OPs RTX 3060 will have a tolerable effect on the school children and the rainforest.
@FauxLiving @YeahToast “I’m sure my driving to work will doom the world” same mentality.
Also, it’s not so much use of a model as the creation of the model. Training is expensive in terms of resources and energy. Using the model and normalizing AI usage leads to more AI usage and more AI normalization, which leads to new models being trained…
Put it into perspective:
People driving cars pollutes more as does global shipping, regular old consumerism, etc. The environmental impact of data centers is negligible in the grand scheme of things.
Now that AI is making open source software the most secure software ever written (the hard way, hehe), not using it to scan your stuff is now considered worst practices and basically idiotic negligence. No company that uses software or anything with firmware can afford to ignore it anymore.
There’s no way out of it. Accept the new AI reality and push for good regulations. Example: If we made it so that all AI data centers had to be powered via renewable energy on site that would allay most of the energy use/environmental concerns.
I don’t buy it. If Datacentres were the powersipping butterflies you suggest, why all the new fossil power plants everywhere? I would probably not have given a shit about the datacentres, if not for all the new fossil pollution that is part and parcel of the construction.
Lies, damn lies, and statistics.
Here’s the thing, datacenters could be powersipping butterflies if they built solar farms and ground source heat pumps when they build out the data centers. But the Epstein class are too cheap, greedy and selfish to do that without being forced to. They are accustomed to having the government, directly or indirectly, lower their costs and guarantee them profits.
@riskable Justify using an unethical technology however you want, but it’s still the same basic argument that drivers often make (which is what I was pointing out). There is always some worse impact to the environment, and that is why we don’t need to change our behavior.
Ethics are for suckers, I guess?
AI isn’t inherently unethical. The very premise of saying such a thing is flawed.
Is it unethical to use AI to find new drugs and medical treatments? To use it for physics research? What about regular research? Writing code?
Where do you draw the line?
Also, AI was used to design the current crop of fusion reactors and the full potential of that is only just getting started. If it turns out that AI solves all the issues and generates a working fusion reactor design, it’ll make every one of your arguments moot. Because that one achievement will be worth everything. It would “save the world” so to speak (from an energy/emissions standpoint).
Chatting with an LLM is less impact than driving a car. Think about that for a second before you say using it is unethical. It’s literally more negative for the world for someone to drive a car every day to work than it is to ask an AI to find something on the Internet or write a program in Python.
Or perhaps your underlying complaint is more about capitalism than it is about AI?
How AI was trained is unethical. There was no consent, limited renumeration to individuals for using their work. If I drive to work, I provide value to people . Having AI scrape and summarize searches (incorrectly at times) has no benefit. Forcing all work meetings to use AI, has no benefit.
When I learn skills using books from the library, am I morally obligated to pay the authors a cut of my future earnings?
… No? The library purchased a book? You are also an individual. AI is a corporation that has been ripping resources without paying for them plus commercializing the content that’s completely different. I believe you’d find you wouldn’t be allowed to get a book from the library then turn it into an ebook to resell.
If OP is doing orbital dynamics in their day job, they probably live in California. JPL, Raytheon, LLNL, LBL, and Berkeley are all here.
Even if they don’t have rooftop solar, most of our grid is renewable. Their local GPUs have a decent chance of being more environmentally friendly than charging your cell phone, depending on where you live.
And using it for science and programming is really different than using ChatGPT to write you an email. I’m home right now, but I have two agents at work running debug and trial experiments, and they’ll be running it all night until I get in in the morning. I’ll review their work and steer them all day while I’m in the office; and while those two completely different projects are running, I’ll be personally working on a third. They are enormous productivity multipliers when you have a well-defined and deterministic target.
My grievance isn’t so much with current household models with PCs sitting there running off solar (that’s starting to get very bloody specific though). But how did you get to a point where you could get a local model on your PC. All the pillaging of content, hundreds of Billions to Trillion(s) of dollars spend on a product that didn’t renumerate anyone who created their learning data.
It’s actually my setup.
It’s a standing on the shoulders of giants thing. That’s how humanity have been making progress historically.
People running local LLMs with open weights abliterated models, especially with solar PV power (perhaps not as rare as you think, I’m upgrading my solar PV system in the next couple weeks for that explicit purpose), is about freedom and privacy. At some point there should be enough of them to link up into a grid over the Internet, collaborate or donate resources to distributed community projects. This is a way to allow for the “pillaged” content to pay back for itself.
Do you have qualms with public pirate libraries, like SciHub and LibGen, too? These have also been used for training LLMs.
We involuntarily contributed to their creation, the least we get to do is enjoy the fruits of that theft. More should be given, financially reimbursement through taxation, but until that happens, free use of the technology is consolation.
Besides, following through with your analogy, if a baby is born from rape - does that baby deserve hate for who their father is?
Unpopular opinion time: I’m torn as to where that theft line is.
If a turn of phrase in Lord of the Rings inspires something new in me, do I owe royalties to the Tolkien estate?
The stories on FanFiction.net are derivative by definition — is it theft? I love “Harry Potter and the Natural 20”, a 4th-wall breaking D&D and Potter universe mashup story (seriously, it’s hilarious). Does that story owe royalties to Wizards of the Coast and (ugh) JKR?
Maybe those are super transformative. What about opinion pieces where the author redigests information from all over, many paid writings? Are they selling the original authors’ product, even though restated?
Maybe it’s ok if they paid for the subs. But what if it came from free library books?
My wall has a bunch of art from artists at SDCC. I’m looking at Spider Gwen; Thor, Iron Man, and Mjolnir Cap charging into battle; Kaladin and Syl before the Stormfather; and a map of all the places the Gaang went in ATLA. Those artists all ingested the original work to make those derivatives; were they not allowed to make them if they borrowed their friends’ copy?
It feels kinda wrong to vacuum up public data to train models, kinda? But I don’t know that I can fairly call it immoral without condemning all these other things I think aren’t immoral. I kind of want to, but that’d be really inconsistent, and that’s profoundly frustrating in its own way.
It’s messy and complicated and I think the gut response of most people isn’t actually a reasoned one.