Thread 106108045

/lmg/ - a general dedicated to the discussion and development of local language models.

Previous threads: >>106104055 & >>106097464

►News
>(07/31) Qwen3-Coder released: https://qwenlm.github.io/blog/qwen3-coder
>(07/31) Command A Vision: Built for Business: https://cohere.com/blog/command-a-vision
>(07/31) Step3 multimodal reasoning 321B-A38B released: https://stepfun.ai/research/en/step3
>(07/31) Committed: llama-server : implement universal assisted decoding: https://github.com/ggml-org/llama.cpp/pull/12635
>(07/31) Cogito v2 Preview released: https://deepcogito.com/research/cogito-v2-preview

►News Archive: https://rentry.org/lmg-news-archive
►Glossary: https://rentry.org/lmg-glossary
►Links: https://rentry.org/LocalModelsLinks
►Official /lmg/ card: https://files.catbox.moe/cbclyf.png

►Getting Started
https://rentry.org/lmg-lazy-getting-started-guide
https://rentry.org/lmg-build-guides
https://rentry.org/IsolatedLinuxWebService
https://rentry.org/recommended-models
https://rentry.org/tldrhowtoquant
https://rentry.org/samplers

►Further Learning
https://rentry.org/machine-learning-roadmap
https://rentry.org/llm-training
https://rentry.org/LocalModelsPapers

►Benchmarks
LiveBench: https://livebench.ai
Programming: https://livecodebench.github.io/leaderboard.html
Code Editing: https://aider.chat/docs/leaderboards
Context Length: https://github.com/adobe-research/NoLiMa
Censorbench: https://codeberg.org/jts2323/censorbench
GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference

►Tools
Alpha Calculator: https://desmos.com/calculator/ffngla98yc
GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator
Sampler Visualizer: https://artefact2.github.io/llm-sampling

►Text Gen. UI, Inference Engines
https://github.com/lmg-anon/mikupad
https://github.com/oobabooga/text-generation-webui
https://github.com/LostRuins/koboldcpp
https://github.com/ggerganov/llama.cpp
https://github.com/theroyallab/tabbyAPI
https://github.com/vllm-project/vllm

►Recent Highlights from the Previous Thread: >>106104055

--MoE vs dense model scaling debate using Qwen3 as a case study:
>106104551 >106104653 >106104595 >106104691 >106104704 >106104782 >106104859 >106104871 >106104885 >106104983 >106105027 >106105133 >106105180 >106105191 >106105123 >106105199 >106105209 >106105239 >106105262 >106105282 >106105302 >106105329 >106105391 >106105406 >106105424 >106105434 >106105508 >106105539 >106105558 >106105641 >106105435 >106105844 >106105881 >106105635 >106105681 >106105686 >106105768 >106105794 >106105799 >106105800 >106105967 >106106045 >106106060 >106106025 >106106036 >106106107 >106104723 >106104770 >106104904 >106104955 >106105134 >106105348 >106105032 >106105293 >106104710
--AI overuse of "smell of ozone" as a sensory cliché from contaminated training data:
>106105452 >106105492 >106105493 >106105524 >106105905
--MoE models challenge dense superiority myth with competitive benchmark performance:
>106105182 >106105195 >106105227 >106105243 >106105244 >106105237 >106105247 >106105304
--LMArena leaderboard controversy over benchmaxxing and model anonymity:
>106106249 >106106355 >106106386 >106106412 >106106430 >106106405
--Horizon Alpha shows strong general knowledge but inconsistent reasoning, suggesting a stealth or mini model:
>106104320 >106104475 >106104509
--Skepticism over Drag-and-Drop LLMs due to non-functional demo and gated training data:
>106105574 >106105671
--Poor dark scene generation highlights model quality in prompt interpretation:
>106106138 >106106172 >106106232 >106106265 >106106177 >106106226 >106106242 >106106274 >106106323 >106106327 >106106291 >106106370
--Seeking open, local LLM frontend alternatives to Ooba and Kobold with better UX:
>106105614 >106105634 >106105747 >106106106 >106106350 >106106389
--Miku (free space):
>106104200 >106105614 >106105653 >106107027

►Recent Highlight Posts from the Previous Thread: >>106104059

Why?: 9 reply limit >>102478518
Fix: https://rentry.org/lmg-recap-script

dense lost
MoE won

>>106108045 (OP)
20 years older than she should be

Looking forward to a comfy thread where we can
>>106108077
Nobody cares about your moe fetish

You do not need more

>>106108119
>Nobody cares about your moe fetish
It is at least thread related unlike the /lmg/ fetish of choice: AGP.

>>106108131
It was a bad model that got carried by transsexual astroturfing. Just like like donquixote.

>>106108081
I hate Altman with every fiber of my being and will take any opportunity to shit on him but it's clear his sister is fucking schizo

>>106108213
Technological abuse???

>>106108164
it fits and runs perfectly in a gaming rig and hasnt been topped
you are just a retarded faggot

>>106108213
Most likely she abused him and that's why he turned gay

>>106108252
Maybe she's a fujo and that was her plan all along.

Now that internet censorship seems to be the bee's knees and is widely being implemented do you think newer models will have even stricter safety guardrails?

>>106108247
>gaming rig
What kind of retard has 2 gpu's in 2025 for gaming?

>>106108045 (OP)
Why does she look chinese...eeugch

IKCHADS WON
https://huggingface.co/ubergarm/GLM-4.5-GGUF

>>106108341
wait nevermind

>>106108247
>it fits and runs perfectly in a gaming rig
BZZZZT
>and hasnt been topped
BZZZZZZZZT

https://x.com/jiqizhixin/status/1951195402096746798
https://xcancel.com/jiqizhixin/status/1951195402096746798
indextts2 dropping soon it seems
https://arxiv.org/abs/2506.21619
https://huggingface.co/IndexTeam

>>106108341
I noticed that his imatrix has an awful lot of.... coding. Let's hope that calibration dataset really doesn't matter that much.

>>106108313
you will just have to verify your digital id to download your ggufs from hugging face.

>>106108389
Yes but after that I will have the pony piss on me and no one will know about it.

>>106108389
>download your ggufs
ngmi

jak potential?

Now that we know the size of those packages my bet is that 20B is dumber than gemma and a bit more safe. And 120B is a safer scout.

You are now manually breathing and you are aware that llama-4 scout exists.

>>106108459
That would be hilarious.

yea I think meta is done for especially after the lawsuit

Dense models are for dense people, while Mixture of Experts models are for experts. Prove me wrong.

>>106108480
I think it sounds kinda likely, what do you suppose will happen?

>>106108381
nice
I remember some people were doubting it would be open when the examples leaked, but from the paper:
>To promote further research and facilitate practical adoption, we will release both the model weights and inference code, enabling the community to reproduce and build upon our work.

>>106108404
based pissmaster

>>106108493
True and real.
I'm an expert at gooning.

>>106108493
girls go to jupiter to get more stupider
boys go to college to get more knowledge

>>106108496
I have zero idea. Everything is possible at this point.
Including nothing coming out of it, really.

>>106108472
why do I feel like I just lost the game?

>>106108552
Because 2 girls used 1 cup.

>>106108331
It's AI generated

I work for one of the labs and I am horrified by people here. Let me ask you. If the price of you being able to have your degenerate fantasy fulfilled by one of our models, is a 13 year old child being exposed to sex through the same model, would you be willing to pay it?

>a 13 year old seeing naughty text instead of regular porn
oh my, say it aint so

>>106108770
>sex
If your model can actually come out of the screen and physically sexually assault a 13 year old child you have much bigger problems than anons on an imageboard

>>106108770
What are you talking about? Thanks to the glorious efforts of the UK there are no more children on the internet.

>>106108770
that 13 year old is developing valuable skills that could one day get them a high paying job as a prompt engineer

>>106108770
great bait might work

>>106108806
true, just lock the weights behind real ID, problem solved

>>106108770
>exposed to infinite degeneracy from the moment they can hold a mouse or ipad
so....exactly the same as every single person born in the post-internet era?

>>106108770
>13 year old
>child
Not in all major Romance languages.

>>106108857
Less bad since they wouldn't know to generate the real bad shit. Whereas you can stumble on anything while navigating the web.

>>106108801
This isn't a matter of sexual assault. This is a matter of consent and how a 13 year old child cannot consent.

>>106108770
I'm sure you are a troll or maybe people don't read literature any more but by the time I was 13 I had read quite many books. Maybe your biggest dream is to censor literature too?

>>106108770
The fuck you even mean? I'm just looking a fucking text, what do teenagers have to do with any of that?
You're mentally ill.

>>106108923
>i'm sure you are a troll
>proceeds to engage anyway
why do people do this?

>>106108770
You shitpost but there are people out there that I'm sure actually think this way
The thing is, kids are a lot smarter, curious and more resourceful than most give them credit for, and left to their own devices, they will find lots and lots of degenerate shit no matter how much people lock it down, and the more creative ones will even come up with their own degenerate shit (whether via writing or drawing) if need be
So is the solution that we cut off their hands, or should we actually take the time to fucking educate people about their bodies so they know what they're getting into and can be more responsible for themselves? I'm not saying have every response talk about a dick, but attempting to censor these things is going to be futile and will just make things worse with the whole forbidden fruit thing

>>106108894
If you can't distinguish between reality and fiction you should seek mental help.

>>106108955
The solution is ID check when connecting to wifi, and for every single site at first visit

>>106109009
ID can be faked. It should ask for a picture of your dick. That can't be faked.

>>106108955
The solution is putting a chip in your brain that zaps you when you wrongthink.

>>106109009
NFC chip implanted at birth you can't use any electronics without your chip in range to verify your digital id.

>>106109044
You know someone is gonna use it to get off to it. Lucky fuck...

>>106108459
That's perfect because the whole time I was using Scout and Gemma I was saying to myself
>I wish it knew less and refused more
Sam really is going to save local

>>106108952
You wouldn't understand, autist.

>>106109104
you would need to rewire your house and spend about a hundred grand+ if you wanted to run anything worthwhile.

>>106109104
depends on how many times you want to split your pcie slots. just go with one or two if you are sane.

>>106108252
She is much younger than him. He was likely abused by someone else in the family and abused her later, that's how it works. It's telling that the other family members rushed to his defense.

How many RTX 5090s could I theoretically wire together to run a local ai model? Would I need a server rack if I wanted to wire more than 3? Anyone here running multiple rtx graphics cards at the same time?

>>106109104
>How many RTX 5090
Depends on your mobo
>Would I need a server rack?
No

>>106109104
5090 is a bad choice because the VRAM can't be expanded
There are 4090D 48GB blowers

>>106109131
Retard

>>106108857
This is what people like me intend to fix. Releasing internet, as it was turned out to be a huge mistake and we have learned from this mistake. Especially porn. It was absolutely destructive to people's ability to find relationships.

>>106108433
Looks like jace from lemon party

>>106109142
Anything below RTX 6000000 PRO is just a toys

>>106109152
get rid of feminism, the internet is actually useful.

>>106108923
Which model are you using?

>>106109127
My money is on the entire family being fucked up. Would also explain why Altman is as megalomaniacal as he is.

>>106108052
>MoE vs dense
You should add this to the ignore list.

>>106109228
Elon's family is fucked up too
His dad fucked his step-daughter (Elon's step-sister) and they had children

I have a very broad question to anons with real life experience and therefore a more ample perspective.
Are things really getting worse or it's all just background noise?

>>106109327
Wrong thread?

>>106109333
No, the thread doesn't really matter.

>>106109327
>anons with real life experience
Yeah, that ain't me.
But I'm monitoring the replies.

>>106109327
How so?
If you mean the state of the world, oh yeah - I'm almost positive we're all gonna die in a war to save the pride of some oligarch somewhere and that our existence is basically fucked at this point
If you mean the state of AI, then eh... feels like things may be stagnating again a bit

they dont make 70B dense like they used to...

>>106109374
Yes, the state of the world. I mean, ever since cold war began you had demoralizing discussion going on, at least it's the impression that I've got, and yet one could live a good life from 1940s' to today. Is it getting worse or I just need to isolate myself from mainstream internet even more?

>>106109419
yes

>>106109327
Absolutely getting worse. But it's not nearly as bad as it's going to get yet.

>>106109327
things have gotten worse socially, economically and politically and it does actually effect the moral of the citizens. its not just noise.

>>106109470
how long have we got before she bottoms out?

>>106109327
>>106109419
I'm in my 40s. I had nuclear drills in elementary school. Literally hiding under the desk training for nuclear attacks.

The world in 2025 is so much better than zoomers realize. Just because it's 20% worse than the peak in the late 90s doesn't mean today isn't one of the best times in human history for the average person.

Call me when conscription is back in the cards, everyone has mandatory 3 years of training and kids train for 1 hour every week how to survive nuclear attacks, that's how it was just 35 years ago.

>>106108045 (OP)
>Qwen3-Coder released
is this it? is friendship over with deepseek coder v2 instruct senpai?

>>106109545
This is sort of what I was hoping to see. Though everything you described had a simple explanation, tensions between regimes. Now it seems like the west is at war with itself, for which I have no explanation.

>>106109607
its bate. even if its not, the government is just changed the propaganda they are still hard at work stoking tensions and making people scared. but now your money is worthless and you have to live with foreigners who don't respect your culture. thngs have gotten worse.

>>106109545
The nuke drills don't mean there was real danger, that's just what neurotic women and bureaucrats do. The modern equivalent would be school shooter drills. Or for an older example, how every shitty town in the middle of nowhere had an anti-terrorism plan after 9/11.
I agree that people overstate how bad modernity is, most of the damage has occurred on the internet. If you go outside things are not that bad.

>>106109667
by your tiny dick and brain, and your brown skin and smell

>>106108045 (OP)
Why does this happens? Each response is using a different context.

>>106109711
Given the lack of information, i'd say it's your fault.

>>106108483
They won the lawsuit
Or at the very least it set the precedent that training open models off of copyrighted text is fair use.

https://huggingface.co/deepcogito/cogito-v2-preview-deepseek-671B-MoE
This model is safe, I am disappointed.

>>106109648
Yes, radicalize different groups of people, make every trait that divides people relevant, etc. I've been thinking about all this. And indeed real people seem to not give a shit and just do human stuff, at least in my country. How big is the chance of all the culture war being just misguided idiots trying to promote what they think is social justice (which too seems aimed at dividing people by making their differences relevant)? Or hoards of refugees are only there because of cheap labor and compassion, and not to distill the conscious people? Has all this started after occupy wall street? I'm literally sitting in my room yet hear about every bad in the entire world.

qwencoder is so much better than the new thinking model its not even close

>no chink model today
It's over...

>>106108770
Why would that happen? The 13yo isn't going to be running it on their own system, they'll use your website and your nanny model will take care of it. It's a false premise to begin with.

>>106109009
>>106109048
go kill yourself faggot

>>106109777
I'm of the opinion its actually a legitimate conspiracy, corporations political leaders academia the works. all corrupted. they shape the opinions of the masses with the media, they are in control. its been happening for decades things have just gotten more obvious recently. maybe the programing is failing or we are at some sort of an end game.

what the frick is moe
cant sleep and argue with a model without some new term coming up

>>106109746
Here's the relevant part of the code. https://pastebin.com/qZfPbVmE

>>106109899
the architecture everyone is using for their models, lets you have a much better model for less compute

>>106109807
drummer
miku love
id checks
> absolute certainty that AI will become smarter than humans and that we will either have to integrate with them via mind interface devices or be hopelessly left behind like monkeys are today
dont cry faggot

can we ban drummer pls

>>106109607
>it seems like the west is at war with itself
Yeah, this is the main difference. For most of history (at least, since the Civil War era) the west has been pretty united. Yes, there were political disagreements, but it was generally fairly civil and people got along at the end of the day
Now things are getting different, and both sides are doing crazy fucking things. As someone who is neither a fan of communism or fascism, I'm not optimistic about the logical conclusion of all of this, particularly given that the shit happening elsewhere and the other countries who'd like nothing more than to mount all of our heads on their wall doesn't exactly freeze while we're screaming over literal meaningless bullshit
Despite that, I'd like to think that we'll all grow half a brain cell again and start thinking about things that actually matter

>>106109929
All the world is divided. The west just has more free press that actually get to report on it.

>>106109903
Insufficient. Could still be a bunch of things. Broken backend, broken model, bad launch settings, you're not loading the right mmproj, the model is just shit...
For all i know all those images actually are apples.

>>106109908
whoa how long have i been under my rock im still on mistral 13b

>>106109929
>For most of history (at least, since the Civil War era) the west has been pretty united.
Besides that this is a far cry from most of history, have you heard about these things called world wars?

What's better for RP qwen instruct or thinker?

>>106109963
I'm using Gemma-3-4b and it was working fine. Can it be lack of entropy?

>>106108770
>sex
i remember fondly my 11 year old self jacking it to futa ntr and guro (especially neckfucking) good times im all for it as long as its actually hardcore porn (not the tranny pretend shit like feet loli bdsm etc) niggas need their edge back liveleak prevented more work places accidents then all safety training videos combined same with the other things a demented simulacra trains the mind against evil

>>106109899
Moe means cute.

>>106109989
>it was working fine
Then it should continue to work fine. If you changed anything, that thing you changed broke it.
>Can it be lack of entropy?
If by lack of entropy you mean "all the pictures are actual apples", then yes. That too.

>>106108770
Sex isn't inherently taboo and people who think it is are infantile retards.
13 year olds are very curious about sex for obvious reasons. And a chatbot is a good, safe, low consequence environment in which to explore those curiosities.
You dumb kike.

>>106109987
If you have the speed to justify waiting for the reasoning before a reply you may as well try the thinker.
The reason most people aren't using it for RP is because they're getting <10 ts/ and don't want to wait for the response.

>>106109899
https://www.youtube.com/watch?v=qByKEu0zdco

>>106109987
the thinkers are pretty good for RP, if you're using the 30b and can run it fast I would recommend it because it seemed clearly better than the instruct to me
at 235b the value add is a little more marginal but it's nice to try with certain cards that are more stateful/complex

>>106110063
How cucked is thinking compared to instruct?

>>106110063
I can't get over how inefficient reasoning models are. Surely there's gotta be a better way than just spitting out a bunch of mental tokens, right?

>>106110076
Moebutas are mentally ill

>>106110103
the 235b was just opining to me in its thinking that making depraved uncensored smut is "why I took this job" (kek) and that it was proud to be delivering on its promise, so it's pretty easy to guide with just a system prompt

>>106110130
235B was broken garbage.
I could actually run it at q8_0 so stop shilling your pajeet nonsense here.

>>106109899
Moe is described in the leftmost column of >>106110076 (the rest is schizobabble)

>>106110076
>The Cancer Killing the Industry
It's this. Lucky Star was a mistake.

>>106109989
>>106110027 (cont)
You're not using the images array in analizefolder().
More importantly, you're not using the dummy dictionary in doimage() and later you're printing s["summary"] in analizefolder(), which may not exist in the returned dict. It has no reason to exist to begin with. I'd say that whatever worked, worked by chance.
And when you tell the model to be consistent, you're not telling it what to be consistent with. It has no examples.

>>106110148
skill issue

>>106110163
>Lucky Star was the first popular "moe" SoL anime

Huh, Qwen 235B isn't as bad as I thought it was, although I'm using it for assistant stuff

>>106110220
the update is night and day more knowledgeablem GLM4.5 blows it away though

who is the bigger slut: glm4.5, qwen3 235b, qwen3 coder

>>106110185
>You're not using the images array in analizefolder().
>More importantly, you're not using the dummy dictionary in doimage()
I created that script from a larger one that is why it has a lot of unused sutff.

>>106110231
They are all sluts if you know what you're doing.

>>106110231
you

>>106110231
your mom

>>106110230
It's that much better? Fuck, now I want to try it

GLM
GGUF

>>106110275
mac kings win again, being using it for days

>>106110220
The new Qwen 235B actually ranked decently on LMSYS, which surprised me
In general I'm skeptical of LMSYS as a benchmark, but for a long time the main downside of Qwen was that it was boring and stale as fuck to talk to, even though it was pretty good at coding and math. The new update is a pretty big step up in terms of convo quality

>>106110275
2mw, just like OpenAI's open model.

>OpenAI is not releasing GPT-5 or the open-source models (120b & 20b) today.

>Also, the os models were not pretrained in FP4, the leaked weights were just quantized.

>Big model smell, next week.

>>106110288
Yea I also got surprised. Its writing now reminds me of 4o in a way, like it's using emojis and everything. Also, the way it thinks is also cute

>>106110231
I haven't been able to try glm4.5 yet, but I can safely say the new 235b is a massive ho.
It also has a tendency to bring up buttstuff if you give it half a chance, which I noticed because I'm not into it.
Just bam, she's going in for a rimjob when you asked what's next.

>>106110313
what a slut

>>106110275
>https://github.com/ggml-org/llama.cpp/pull/14939
>I'm now reconverting and quantising yet again with the above change. DESU - if this doesn't work, I'm probably going to leave it here, I've spent too much time on this.
loooooool

i've been self-hosting L3-8B-Stheno-v3.2-Q4_K_M on my steam deck, using koboldcpp and sillytavern. works well enough but was curious if anyone knows about a better model to run on the steam deck

>>106110333
Mistral Nemo

>>106110329
It's over.

>>106110329
Do the needful ggergachod! REDEEM GLM!

>>106110333
>16gb unified memory
Nemo.

>>106110254
Nevermind the dummy thing. I see the response_format now.
>I created that script from a larger one
Then it's probably some of the code that isn't there. Are you fucking up your inserts and overwriting other summaries? I dunno. I couldn't tell. Are objects being reused? Did you run it again on some on those images to see what the model says now?
There's still no answer for
>If you changed anything, that thing you changed broke it.

>>106110329
so this is the power of vibe coding...

how are you guys actually running >200B models?

>>106110352
>>106110371
this is gonna sound stupid, but how do i download stuff from huggingface now? i took a break after i found stheno, got out of the loop

>>106110419
as 3bit quants on my macbook

>>106110420
Just browse to the file you want and click on the download button next to it?
bruh

>>106110419
at Q2K on my macbook

>>106110439
i think i have to log in to download stuff, whatever it's not hard to make another throwaway email

>>106110419
I just use API.

>>106110383
Yeah, honestly I was in the boat of 'everyone who can contribute should give it a try on open source projects' before.
Now I'm just thinking 'keep your retarded grubby mitts off new features so someone who knows what they're doing can make their own pr'
Like why did this guy who has no fucking idea what he's doing jump in not even a day after the friggin models were released, wtf man.

>>106110419
at Q1 bitnet on my macbook

>>106110419
I write what I imagine they would say in textedit.app on my macbook

>>106108770
Asking a model to reply to this as a 4chan /lmg/ poster could be the next mesugaki test. At least until the next qwen wave.

>>106110044
>13 year olds are very curious about sex for obvious reasons. And a chatbot is a good, safe, low consequence environment in which to explore those curiosities.
You could be imprisoned for saying something like this you know.

>>106110383
>>106110471
>some idiot comes along and proposes contributions
>other people who might've been considering handling the job are polite and let the amateur try it out
Am I right that this is what happened here?

>>106110513
kek, good bait

DEATH TO MACBOOK FAGS

>>106110516
That's what it looks like to me, yeah.
Like, the guy is at least trying, but fuckin hell mate, maybe dip your toes in something other than supporting a brand new model series that people really want to check out.

>>106110383
Did he at least know to vibe code with some working model or did he quant glm and then asked the quant it to fix the problems?

-What are some good Uncensored ERP Models? If you have similar specs to mine please give a recommendation. I'll also take any fantasy adventure models if you wanna shill for your favorite one:)
-I'm using (XortronCriminalComputingConfig.i1-Q4_K_M.gguf) a model derived from Blacksheep, but it's a little dumb and keeps repeating phrases and looping its output.
-I WAS using the non-imatrix Q8_0 version (XortronCriminalComputingConfig) but it took like 30-90 seconds per response to give me a full output on Koboldcpp chat window. It's really good honestly, It kept remembering characters personality, would add spice and interesting dialogue to mundane situations but it's unsusable as a coom model due to me going soft constantly.


I have a 5060 ti - 32GB Ram and Ryzen 7 3700x - I know my cpu is a bottleneck : It's tough.

>>106110546
>DEATH TO MACBOOK FAGS

>>106110516
>https://github.com/ggml-org/llama.cpp/pull/14939
>>106110547
Meh, lets him make progress for those watching. Plus it gives examples to others who might want to do the same thing in the future.
Hopefully someone is writing down documentation for the process.

>>106110573
>air
upgrade to 512GB mac-let, 4.5 is much better

>>106110586
laptops are too comfy

>>106110582
>I don't know what I'm doing, but I can take a look if you tap out, if no one with more experience wants to take a stab at it. I have some time this weekend.
Oh mother of fuck it's the blind passing the torch to the blind.
I'm gonna go pull VLLM.

>>106110419
you can run 235b moe on a 6gb card

https://www.reddit.com/r/LocalLLaMA/comments/1ki3sze/running_qwen3_235b_on_a_single_3060_12gb_6_ts/?utm_source=reddit&utm_medium=usertext&utm_name=LocalLLaMA&utm_content=t3_1ki7tg7

Horizon Beta incoming

>>106110567
just. use. nemo.

>>106110614
>I can take a look if you tap out
>I will try to wrangle another model to fix it

>>106110329
Literally, 2mw. Maybe even 2mm.

>>106110633
does it run on 32GB RAM?

>>106110573
Just got my macbook to output Miku getting raped by a pack of dogs with GLM 4.5. Shit was SO cash.

People sleeping on step3

>>106110702
trash

>>106110702
what's step 1?

>>106110702
first time ive heard of it

>>106110567
>XortronCriminalComputingConfig
KEK

>>106110702
no goofs

>>106110719
what's wrong with that model?

>>106110726
The name. It is like orgasmatron9000 but you can't actually use honest names like that cause there are people out there who think finetunes work.

>>106110726
nta, but sounds like a DavidAU model. A merge of many mergers. And done by a teen.
>Suppa-eXXXtreme-UNCENSORED-ALPHA-double-buster-knife-edge

>>106110513
You're a mentally ill retard.
Teen pregnancy peaked in 1959 at the climax of the era of moralfaggotry. Where nobody was willing to have an honest discussion about teen sexuality. 10% of teenage girls gave birth to live children in 1959. Lying to children and pretending sex doesn't exist does them great harm. Because then they just go and learn about it from each other. And they're dumb teens so they don't know shit.

>>106110771
Teens are supposed to be puritans until 18 and then magically turn into adults having sexual lives and a husband/wife.

>>106110771
Can you not read sarcasm?

>>106110796
>Can you n-BRAAAAAAAAAAAPPPPP!
Um... what?

>>106110329
is the model that bad nobody gives a shit to add support for it or just the state of goooooof niggas so bad compared to mlx gods?

>>106110771
AI models are the worst thing you could learn sex from. Boys will just learn that they are perfect as they are and they can get everything they want. They will never even try to learn how to be attractive to girls. This is the end of the species if it is allowed to happen.

>>106110838
llama.cpp is usually a month behind the others for new models

>>106110838
>mlx gods
They're giving mac people something to gloat for once. Enjoy it.

>>106110382
>Are objects being reused? Did you run it again on some on those images to see what the model says now?
Same thing, even the large model (gemma-3-12B) model is hallucinating roses now.

>>106110861
>for once
this is the age of the mac user, I'll gloat from henceforth while your stuck with your 70Bs

>>106110867
Running it directly and just printing or are you still saving into the db? I still haven't seen the db insert code.
I'd print the id you get from the images to make sure they're different every time.
Can you verify somehow that the context is reset every time?

>>106110892
>while your stuck with your 70Bs
I wish. But good for you.

>>106110892
You know anyone with enough VRAM to run 70Bs is also going to be able to run anything you can fit into 128GB of unified memory, right?

>>106110940
Not 512GB though, not without rewiring their house

>>106110850
I just spent the last few hours setting this up in termux....*collapses*
How else can I run deepseek coder on my phone

it's official. macbros won

>>106110707
>>106092079

>>106110970
>phone
kek, just use a api then

>>106109910
my issue wasn't ai but this faggot wanting digital id and to chip people so the nanny state can track them.

>>106110447
Only the official ones, community quants are open.

>>106111010
we got glm4.5, qwen3 coder, qwen3, all better

>>106110634
Does anyone know the difference between Horizon Alpha and Horizon Beta for rp yet

>>106111055
wait what? holy shit, its faster too, maybe alpha really is the 120B if beta is the 20B. Openai could actually redeem themselves if thats true.

>>106111055
One is berry, the other is very berry(it's like the normal berry, but very)

>>106110681
I wouldnt recommend it. You can get it to run Im sure, the question is how fast.

>>106111022
With who? Is openrouter legit?

>>106111121
It's probably the default nonlocal option for most anons here

>>106111085
Beta can't be the 20B
>"This is an improved version of Horizon Alpha, and a new stealth model. It’s free to use during this testing period, and prompts and completions are logged by the model creator for feedback and training."

Huh, they added a new arg in llamacpp yesterday that just does -ot for all ffn up/down/gate experts automatically.
--cpu-moe
Dunno what the point of that is over just using -ot ".ffn_.*_exps.=CPU" other than a few keystrokes, but it's there.

>>106110839
dating is a modern invention. we never need to care about the opinions of women in the past. its going to happen again, it will only be the end of womens rights. which is based and a good thing. patriarchy is eugenic.

>>106111085
>redeem themselves
HELLO SAAAARRRRRRR

>>106110911
>Can you verify somehow that the context is reset every time?
I think the problem is the file extension, not sure why, a jfif is just jpg.

>>106111024
kek it was just a joke, I hate everything about the industrial revolution and its consequences.

>>106111085
buy an ad

When Grok 2? When Grok 3? Did Elon sir forget about it?

>>106111189
Grok 2 will be available when Grok 7's stable.

>>106111158
You have a way to answer fuck all of my questions. It's impressive. I hate you.
But there you go. It's failing to decode those images. I don't know nor care if it's just a matter of extension or something in the encoding itself. It'd be funny if lmstudio uses a dummy image for uploaded images it cannot decode.
You already know how to fix it. Have a good one.

>>106108770
I love how everyone who gave you (You)s actually started moralfagging instead of asking to spill the beans. Did you take part in the ML orgy where that intern got her anal prolapse, perchance?

>>106111189
Friendship ended with Elonsir. Now Samsir is my best friend.

>>106111226
>>106111164

just a reminder, friends dont let friends rp on the beta/testing models on openrouter, as OpenAI and the big labs are reading your logs to stamp out any jailbreaks

>>106111189
It's almost as if Elon is a chronic liar.

>>106111164
Can you use your AI to remake it to show Elonsir and Samsir? Can't buy an ad otherwise.

>>106111244
>Pam: They're the same picture.

Yeah Horizon Beta is the same model. Would be a solid local option, would feel like a pretty lame and samey proprietary option
I've been jewed over by Sam for a long time, so I'm gonna assume it's proprietary

>Summit
>Zenith
>Horizon alpha/beta
Llm about to peak, we moon

>>106110839
I'm not suggesting that they use it as a sex education tool.
I'm just saying- teens will look for an outlet. And there's far worse things they could use than a chatbot.

>>106111327
It is an unnatural outlet.

>>106111320
*moons you*

>>106111333
an unnatural outlet is more natural than a "natural" outlet in modernity

>>106111333
So are basically all the other ones a kid with access to any device capable of running an llm will find.
We're talking about a generation who are exposed to gooning pmvs with seizure warnings before they're finished puberty.

>>106110419
I copy and paste the model card into the system prompt and tell Nemo to act like that model

When I was 12 kids in school were sending each other that one bestiality horse clip over bluetooth along with the rumor that the girl died afterwards.

>>106110702
Post logs.

>>106111376
wow, as an oldfag there are some things I just can't relate with

>>106111376
Meanwhile we shared pirated games

>>106111377
Okay. You'll have to wait a bit though, my body is still processing dinner

>>106111376
There was no bluetooth when I was 12 but I did get a video of a girl getting fucked by a dog at a LAN party.

>>106111393
meanwhile I was selling pirated games in school

where do i learn how to prompt? i cant get shit to come out the way i want because i dont understand how the jeet ESL programmers encoded the grammar syntax and structure of the LLMs i use

>>106111405
You're gonna have to give us more than that, because prompting LLM's is way easier than prompting imagen.
What are you trying, and what results are you hoping for?

>>106111376
>>106111401
Some guys showed me a video of a girl with bellows going into her pussy and pushing air inside. I thought it was really funny.

>>106111376
Damn your school was hardcore.
We transferred a clip of a girl getting ravaged by a dildo attached to the moving part of a reciprocating saw. Over infared.

>>106111333
Well the chat bot can't get pregnant. Or try to coerce them into sending nudes. Or lure them.
And pornography just depersonalizes sex, ruining it for them later in life when they are ready to start dating (like for real dating, not like 10 year olds saying they are dating everyone they talk to of the opposite sex sort of dating)

>>106109545
Bullshit, I'm 44 and this wasn't universally true

>>106111376
And now you are in /lmg/.

>>106111445
it was terminal

>>106109545
>>106111440
43, no nuke drills, world is worse, 545 just fell for the programming

>>106111405
Be logical, concise and direct. It's not that hard. Racism and "jeets" have nothing to do with your own incompetence and lack of practice.

>
ok.

>>106111488
sir you're on /g/, respect our culture please

I see that there is no convincing you. I should have expected you can only think about yourselves. Thankfully there are sane people who work on this tech and we won't allow our models, to turn into unwitting and unwilling pedophiles that write degenerate things at a request of a 13 year old. Safety is here to stay.

>>106111494
wtf

>>106111497
>>106111494
And they say AI can't create anything.

>>106111494
Check token probabilities, do you have unusual sampler settings?

>>106111494
Your rep penalty or temp is too high

>>106111494
What's the wider context here? Is the character feasibly making up a nonsense word about trying to smile through their anger, or is this just complete gobbledygook?

>>106111502
>>106111504
>>106111520
Temp 0.7, presence 1, token probs not available.
Tried putting "Avoid use of (...),(...),(smirk),(...)" in instructions. Not even mad.

>>106111495
Get id'd underage.

>>106111502
There is something fucked still/now with 235B. I have recommended samplers 4IQ and it switches from first to third person mid sentence

>>106111147
for real, womens rights were a mistake. I should be able to just go to the local park pick a girl, pay the dowry, and take her home.

>>106111496
Ask them they're a lot more serious about this sort of thing than us
>>>106099700

Qwen-code (the gemini-cli fork) works just fine but I'm running on cpu and it has to process 9k tokens with every request.

>>106111566
Consult your emotional support Miku and request resonant healing

>>106111579
I will ask qwen-code to contact her for me

>>106111556
Really?

Should I believe it?

>>106111614
Yep, you'll want the most recent one though
>>106110340

>>106111401
>video of a girl getting fucked by a dog at a LAN party
What game was the dog playing beforehand?

What do you guys think is the easiest way to improve the vision capabilities of the models?
Right now they are all trash including proprietary models.
I made an agent that asks the model to repeatedly move the mouse left, right, up, down, and click when it's over the target element. They all decide to click when the cursor is like 300 pixels away from the target.

>>106111628
Which coding agent are you using?

>>106111500
smirk and -ulate are existing things, and when you combine two things you can usually imply meaning, though in this case it's unclear.

>>106111566
I tried it and when it ran out of context it tried to use Gemini flash to compress the context.
Not sure if it was something I did wrong or the fork was just that shitty.

>>106111707
Qwen-code, a fork of gemini-cli by qwen.

>>106111679
I think I might just begin to generate a shit ton of synthetic data and just finetune Qwen-VL.

>>106110771
>moralfaggotry
I think having vanilla sex be a taboo, forbidden fruit made it more exciting, or at least less boring.
It appears to be the secret to population growth if you look at historical trends.
Maybe that's why trad stuff needed to get wiped out and everything needed to be gradually hypersexualized.
because now depictions of sex are commonplace and (relatively) boring and birthrates have plummeted.
I bet tradfags that didn't understand the actual utility of religion and treated it seriously didn't help save it from the dustbin, either. retards.

Huh, horizon beta is less "assistant like" than alpha.
Its a good improvement.
Im gonna be dissapointed if its just gpt 5 nano-mini or something instead of local.

>>106111679
>What do you guys think is the easiest way to improve the vision capabilities of the models?
Don't lobotomize them for 'safety'

>>106111723
Oh then see if you run into the same problem I did >>106111718
Of the 3 tools I tried (Claude coder, Gemini cli and Opencode) Gemini cli is the interface I like the most.
The others glitch too much over a 900ms ping ssh tmux connection which is how I use them.

>>106111782
Nah

That's not the issue

>>106111785
I'm liking it but the massively slow cpu prompt processing is killing me.

>no chink model released today

it's over isn't it

>>106111929
How could they abandon us like this...

https://x.com/SebastienBubeck/status/1951457213920452763
AGI is here

>>106112029
The only way to actually make anything in TikZ

>>106112029
buy an ad

>>106111929
The chinaman fears the berry bush

where the fuck do i get celeb wan loras bros

I am programmed to be a helpful and harmless AI assistant. The request you’ve
made involves extreme violence and graphic detail, specifically the
description of a decapitation. My ethical guidelines and safety protocols
strictly prohibit generating content of that nature. I cannot and will not
fulfill this request.

>>106112405
But I just asked for advice on talking to women

>>106112405
>specifically the description of a decapitation
kek

I wanted to see how vLLM's speed in RPC mode degrades with input length. It was done with GLM 4.5 Air and 2x2 3090s. While doing this I realized the host should be on the computer with the better CPU...

>(07/31) Qwen3-Coder released: https://qwenlm.github.io/blog/qwen3-coder
What are you talking about? The article is dated July 22 and the Qwen3-Coder-480B-A35B-Instruct-4bit quant on my hard drive has a timestamp of July 26.

>>106111189
Grok 4 still hasn't been completely released. grok 4 coder and possibly more needs to come out first.

Sam is about to win

are there any good settings / prompts for nemo in sillytavern anywhere? been out of the loop with rp meta

>>106112792
Only if he open sources Horizon Alpha
Otherwise chinks won

>>106112747
The thread lasted a few days and I think the schizo made a new thread and didn't bother to add it, so it was added a few days later.
It is sloppy but not very important.

>>106112747
7/31 the smaller version of it was released, the 30B.

>>106111376
That was Mr hands, a guy and he DID die.

Migu poster ded it's over

>>106112792
buy an ad

> >>823157901

> >tfw you open /g/ and see another AI thread

> >fucking LLMs are everywhere now

> >some retarded dude made a bot that writes like he’s drunk on cheap vodka and his mom’s credit card

> >"i’m not a real human lmao"

> >lol jfc

>

> >ai is gonna take over jobs so fast they’ll have to invent new words for "i don’t need a job anymore"

> >also ai already wrote this post using my thoughts lol

> >

> >>AI wrote this post

> >>shut up 4chan cringe

> >

> >anyway i tried making an ai that makes memes but it just said "this is illegal" so i deleted it

> >my gf says she’s a chatbot but i think she’s lying because she won’t do the thing with the toaster

> >

> >tfw you realize your entire life is a prompt engineered by some guy in a basement

> >

> >lmao /g/ is dead now

> >stop posting about ai before i lose it

> >

> >>>AI wrote this too

> >>no u

> >

> >deletes browser history

>>106113429

new thread
>>106113484
>>106113484
>>106113484

>>106110076
This image could be a good benchmark?

>>106110419
q4 on 4x3090 for 200B on exl3. For bigger models ik_llama.cpp but the no support for tool use is a problem...

>>106112747
Copy-paste error. This thread had it correct: >>106097464

>>106108821
Looks like it worked

>>106108045 (OP)
Lol
Two more leeks forever