Yeah, that’s the neat part about servers, they don’t necessarily need to be snappy. Most of what it does is background work and it doesn’t have anything else to do. DDR3 and DDR4 get the job done just as well, and save you a buck.
My NAS has 16gb of ddr3 ecc, it has 10tb of ZFS pools and it runs an embedded OS. No problems with that. Not going to use it to host Gitlab lol, but it gets the job done for what it has to do.
We’ll they said they only scanned stuff that would have allowed them implicitly, now did they really respect that? Their ‘‘stack’’ is public, so people can review it. It’s opensource, people can audit it at least.
The big giants can lift whatever they want from Github and we wouldn’t have the means to prove it. I’m sure Microsoft is using private repos as they like. It’s not a coincidence that Copilot was one of the earlier coding LLMs.
Yeah, they aren’t supposed to scrape that stuff. That’s kind of where I’m wondering if they limited their collections.
Yeah, its a strategy when you want someone to understand something they clearly haven’t. You restate it in different words. You may have addressed it, that doesn’t mean you understood my position. I don’t mean that to sound bitchy btw. 😅 We just chatting. We cool. 👉 👉
You’re just refusing to engage with point. ¯_(ツ)_/¯
Would you find it acceptable if their claims at face value were true?
Why are you assuming I am excusing their behaviour?
We’d have an even bigger RAM shortage if so.
Sure but you moved the goalpost from ‘‘they weren’t allowed’’ to “they aren’t being honest”. There’s people in this thread who are in the repos, so we can find that out without assuming.
Also, at least their model is opensource and can easily be selfhosted on consumer hardware. Helps with your datacenter point from earlier. Like if this project does what it says its supposed to do, it’s genuinely interesting. If they are pushing the boundaries of permissive licensing then yeah that is really concerning.
So they scrapped stuff that wasn’t permissive? Interesting… though not really surprising.
Their claim was they followed the licenses and only used code they were implicitly allowed to use. I’d like to see if that was just astroturf and bullshit.
Honestly, though this is one of the reasons why people prefer copyleft and avoid permissive licenses, because yeah anyone can profit monetarily off your work otherwise.
I do not believe a permissive license has any notion of consent. You can’t stop someone from forking your code as long as they follow the rules of the license, like crediting you.
Even in GPL, you can’t stop someone from using your code, see Gnome’s recent arguements with Mint over their usage of an old version of their Calender app.
Others in the thread have mentioned we’ll see permissive licenses with exceptions for Ai in the near future. It’s a solution because that door is currently open.
But if it’s permissively licenced, couldn’t I just copy bits and pieces for my own project without asking?
Like I understand asking is always better and an opt in process for “the stack” or wtv would have been better received. Nevertheless, did they really have a legal obligation, rather than moral obligation, to ask given how this is licensed?
Taking GPL code would be a different situation right? They would have to also include it in their model creation… which theoretically they just could.
In theory, it’s all supposed to be permissively licenced code and the opt out is more than other models give. I saw Starcoder as one of the more ethical models. I thought the underlying principals to be fair at least.
I’m interested in this gut hostility to it regardless. Kind of shows how you can’t present LLMs in a positive angle no matter what. They shouldn’t be using anything GPL or similar.
Do you really need 64gb though? Do you want a PCIe in a low wattage, low air circulation mini PC? Are you running a docker swarm of VMs within the same system? Why?
What’s your use case for this?
You won’t get Local LLMs if the PSU is minuscule. Best get a big old workstation with good cooling. Fried motherboards overloading them with local LLMs. That stuff is mean on hardware.
16gb is still fine for most people.
At least it supports 64 bit then. 😅
Listen, can’t speak for 20 year old cpu, but yeah snaps are more optimized now than in 2018.
Yeah an appliance distro would be a different consideration. For me Ubuntu is really flexible and a great docker host. I feel it’s safer to tinker with since you get those extra layers of containment and sandboxing.
If I was to build an appliance I might also start with Debian, but so far I have used FreeBSD. Works great on my NAS, where I use XigmaNAS. You can transfer a surprisingly lot of skills. Learning what was common to FreeBSD and Linux really helped me with cli.
I’m curious to try making stuff with Ubuntu Core. I think it has potential, but that’s a future deep dive and project. lol
I discovered Tresorit and I like it so far. $150 usd a year for 1 TB. Owned by the Swiss Post of all things. I prefer state owned or non profit companies where possible as service providers.
I also wanted something GDPR compliant and outside the scope of the Cloud Act. I didn’t want my data accessible by a hostile nation.
I like their cli tools, though they are setup a bit weirdly on linux. It’s more like running binaries and scripts than installing a package. Nevertheless, it gets the job done. Upload speed isn’t amazing, but it is overseas. Took nearly 24 hours to send 100gb or something. My server handles it, so its not a concern.
I’m not sure I could recommend it as a replacement for a Google Drive since I don’t use it that way. I just turn the sync on once a week and it’ll diff what’s on my backups.
I might setup a different folder to use as a more daily cloud storage and see how that goes. There’s desktop and mobile apps.
Selfhosting Gitlab takes a bazillion RAMs lol. I think I got it up to 12gb idling without doing anything.