If you run Nemotron, from open datasets.
See the dataset table here. There are sections for pretraining and for specific tasks/topics, like STEM, code and such: https://huggingface.co/nvidia
Chinese models are starting to open their pretraining datasets too, albeit slowly.
this new model, couldn’t universities rebuild it with different contexts and study it in ways you can’t reproduce in other models? like, is k2 horizons a good scientific foundation on which to study machine learning?
Not on a university budget, I don’t think.
But there is all sorts of interesting stuff that can be done with a truly open model other than replicating it.
On top of what others said, Nvidia Nemotron is truly open, with an open dataset.
There have been others (like Olmo), but Nemotron was unique in that it was reasonably useful/cutting edge, while other open models before were basically research artifacts.
…This is exactly what I am talking about.
If EU leftists adopt that way of thinking, the far right will happily sweep up all those disparaged “evil or stupid” voters, with zero shame.
It’s not theoretical. It’salready happening.
This is dismissive.
I know otherwise smart, college or post grad people who voted Trump. I know working professionals, who aren’t religious or anything, who would probably vote for him again over whatever Dem runs. And the left blanket labeling them as evil idiots is exactly what Fox News, WSJ, or their social feeds harp on.
I’m not trying to drag the US into this, I’m just positing that Europe is falling into the same trap. You can dismiss these people, but the far right won’t; they will happily take their vote.
Same thing that happened to the US.
I see a lot of blame on “brainless idiots” or similar labels when discussing this, but I think that’s dismissive of issues many parties seems to be ignoring, or exacerbating, and that the alt-right is pouncing on.
Not everyone who voted Trump is an idiot, and doubling down on that thinking will make the issue worse.
Another is that the far-right parties tend to be “modern” and very social media savvy, whereas establishment centrist parties are out of touch, and campaign in archaic ways. And (from my impression on the outside) the more leftist European parties are prone to disunity and infighting. They don’t benefit from scandal and controversy like the far right does.
Hence “Inflation” is a loaded term that has come to encapsulate other issues consumers are feeling, like wage stagnation, wealth inequity, seemingly high taxes with rising debt and borrowing costs, enshittification, supply issues, purchasing power issues, and other systemic problems like that.
I don’t like it. It’s unscientific and masks too much. But when Americans say “inflation,” I really think they’re expressing feelings for much more than that.
Its the content you don’t see though.
IG’s thirst farming stuff is the absolute creepiest to me.
YouTube too!
YouTube is just… apathy. Like it could be a scathing expose of YT itself, and the comments would all be interchangable with vapid bot spam.
Fighting local models is playing right into enshittification, though.
They WANT people to be scared of “AI.” They want people to keep talking about it as this existential anthropomorphized danger, not a tool theyve enshittified.
Because the thing that keeps Altman/Modi up at night is the proliferation of trifty, dirt cheap open weights models. Ones presented as tools, not intelligence. And every bit people rail against them is more ammunition to regulate those open models away, so they have a monopoly.
Rare clearly shouldn’t be trusted with solo developing another game and neither should Bungie.
I find the phrasing of this amusing. In my head, I’m picturing Arkane and Bungie in a room, and Sharma is pointing at Arkane and saying “Dont leave Bungie alone in a room with the cookie jar. Dont do it.”
Honestly, it gets real yellow outside in Texas. I imagine it must in Mexico too.
I mean, it was fanservice to sell copies too. Even my child ass could see that back then.
I mean, do we really want the Destiny 2 dev working on Halo now?
I say give it to a studio with proven writing chops (Arkane, if they hadnt killed them?) or let it die.
Man, an Arkane Halo game…
Halo 4 was terrible to me.
I marathoned it blind, and I wasnt a Halo lore nut or anything, and I love me some space opera, yet it was just… a soap opera in a bad way? I couldnt believe it was a Halo game.
I heard even worse things about 5. And Infinite was such a waste of potential given the massive budget and the giant open world they barely did anything with.
So I say it was pretty dire.
https://www.orionsarm.com/xcms.php?r=oa-timeline
We will all either be transcendent uploaded cyber-furry collectives spread across space governed by dyson sphere gods (or some variant of that).
Or we will be dead.
The Fermi Paradox kinda points to the latter.
OA is a somewhat reasonable extrapolation, though. Its optimistic, but even then, they almost didn’t make it.
I mean. Step back, and its the same either way.
But I suppose the anguish is on the “liker,” as the undecided is the one who gets to ponder their feelings and decide the fate of the date.
…So if I was being selfish, I’d want to be the “undecided.” Generous, and I’d want to be the “liker,” so the heartache would be on me instead of my date.
If I had knowledge of this cosmic situation, I guess I’d be the liker. I don’t want my date to have to sweat while I decide; I want them to have fun. And I’d classify myself as “looser than average,” so I wouldnt mind and just enjoy the time with the attractive person.
Yeah.
Don’t get me wrong, I don’t want to touch projects vibe coded to this level, going solely on past experience with them.
But I am scientifically curious about its functionality.
Yeah, in that case it’s sus. I’d be scared to look at the codebase.
Does it actually work, though?
Admittedly, no, but this is slowly improving. Stepfun published their SFT dataset, smaller labs are publishing their datasets for task specific tunes. I believe there was another Chinese lab that published bulk pretraining data, but I can’t find it in my history at the moment.
And, notably, these comparatively tiny labs generally aren’t scraping the internet so abusively like OpenAI/Meta. They don’t need as much. Going by statements in their papers, they tend to use existing archives of web data, buy commercial data, or (more recently) generate a lot synthetic data.