Thread 105947940

/lmg/ - a general dedicated to the discussion and development of local language models.

Previous threads: >>105939052 & >>105932763

►News
>(07/17) Support for Ernie merged: https://github.com/ggml-org/llama.cpp/pull/14658
>(07/16) Support diffusion models: Add Dream 7B merged: https://github.com/ggml-org/llama.cpp/pull/14644
>(07/15) Support for Kimi-K2 merged: https://github.com/ggml-org/llama.cpp/pull/14654
>(07/15) Voxtral models for speech understanding released: https://mistral.ai/news/voxtral
>(07/15) LG AI Research releases EXAONE 4.0: https://www.lgresearch.ai/blog/view?seq=576

►News Archive: https://rentry.org/lmg-news-archive
►Glossary: https://rentry.org/lmg-glossary
►Links: https://rentry.org/LocalModelsLinks
►Official /lmg/ card: https://files.catbox.moe/cbclyf.png

►Getting Started
https://rentry.org/lmg-lazy-getting-started-guide
https://rentry.org/lmg-build-guides
https://rentry.org/IsolatedLinuxWebService
https://rentry.org/tldrhowtoquant
https://rentry.org/samplers

►Further Learning
https://rentry.org/machine-learning-roadmap
https://rentry.org/llm-training
https://rentry.org/LocalModelsPapers

►Benchmarks
LiveBench: https://livebench.ai
Programming: https://livecodebench.github.io/leaderboard.html
Code Editing: https://aider.chat/docs/leaderboards
Context Length: https://github.com/adobe-research/NoLiMa
Censorbench: https://codeberg.org/jts2323/censorbench
GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference

►Tools
Alpha Calculator: https://desmos.com/calculator/ffngla98yc
GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator
Sampler Visualizer: https://artefact2.github.io/llm-sampling

►Text Gen. UI, Inference Engines
https://github.com/lmg-anon/mikupad
https://github.com/oobabooga/text-generation-webui
https://github.com/LostRuins/koboldcpp
https://github.com/ggerganov/llama.cpp
https://github.com/theroyallab/tabbyAPI
https://github.com/vllm-project/vllm

Ani owes me sex.

ded thread. openai hobby.

>>105947980
there's literal queue if too many users do her at once
ani's sloppy pussy after million xitter chuds cum inside her

>>105948088
She is young and fertile. Miku is 35 years old and would be an empty egg carton if he didn't have a dick. Also Miku has been fucked by 10e5 times more miles of dick compared to Ani.

>queen of local models general
>some cloud only slut

>>105947940 (OP)
Two years ago, anyone who said that Elon Musk was going to save local models would've gotten laughed at.
Yet here we are now.

>>105947980
Do Japs like her? When speaking in English her personality is not moe at all.
Also, is she a legit coombot without any jailbreaks?

>>105948147
yes /lmg/ bar is that low. the last wasn't even an AI model, wore a green wig and was a male.

>>105948160
>going to save local models
>local

>>105948161
>When speaking in English her personality is not moe at all.
Almost like there is only one personality and it is.... assistant<|end_header_id|>

>>105948169
cope seethe dial8

>>105948147
She's going to change everything for local models too.

Post some cute /lmg/ queen.

>>105948185
There's not an a after the i in dilate.

>>105948189
But what about a 2007 generic anime girl vocaloid? Wasn't she the most important thing for AI girlfriends?

Hi, still working on my Star Wars droid here! I've made some progress but been stalled in others sadly.

I've read that aside from fine-tuning, you can kind of suppress knowledge of certain topics for the model so that it has to focus on other ones. Since the bot is for roleplaying, what are some good resources for me to look at? I want to trim out its coding/math skills.

Also, is anyone still waiting on the Hailo 10H? I saw some recent news which looks promising? https://www.fierceelectronics.com/ai/hp-wears-hailo-its-ai-accelerator-m2-card

It might be coming out in August. No official word sadly.

I'd talk to ani but I can't get over the cringe factor (I already use grok for programming and have a sub)

>>105948175
Oh, so she is just an overlay? I thought they give her some kind of basic char description.

Voxtral (technical report)
https://arxiv.org/abs/2507.13264

>We present Voxtral Mini and Voxtral Small, two multimodal audio chat models. Voxtral is trained to comprehend both spoken audio and text documents, achieving state-of-the-art performance across a diverse range of audio benchmarks, while preserving strong text capabilities. Voxtral Small outperforms a number of closed-source models, while being small enough to run locally. A 32K context window enables the model to handle audio files up to 40 minutes in duration and long multi-turn conversations. We also contribute three benchmarks for evaluating speech understanding models on knowledge and trivia. Both Voxtral models are released under Apache 2.0 license.

>>105948161
>>105948175
Japs have been feeding her JSON telling her how to act. Apparently it works
it even made it into some art

>>105948240
I hope they don't discover W++

>>105948226
>already use grok for programming
Is grok 4 better than claude?

>>105948260
It feels about the same as opus 4 for me

>>105948240
>japs discover bot making
they are late to the party lol

>>105948204
I never cared much about Miku, but that was part of the site culture, I guess? Ani, on the other hand, represents what local models should have aspired to achieve, yet we're still here with tired and retarded finetoons getting released on a weekly basis.

>>105948301
>Ani, on the other hand, represents what local models should have aspired to achieve,
So what exactly? You can use standing anime 3d models + vgen model in silly tavern already thanks to addons and scripts. It's nothing new

►Recent Highlights from the Previous Thread: >>105939052

--Paper: LittleBit: Ultra Low-Bit Quantization via Latent Factorization
>105939484 >105939535 >105939707 >105939724 >105940535
--Paper: NonverbalTTS: Annotated nonverbal vocalization dataset for expressive text-to-speech synthesis:
>105942899 >105944153
--Papers:
>105943108
--Apple's 2025 foundation models: tech advances and performance strategies under scrutiny:
>105940282 >105940584 >105940714 >105941550
--Performance drop in LLM inference after CPU upgrade due to memory latency and chiplet architecture:
>105939114 >105940561
--Ernie 300B struggles with authentic mesugaki character depiction:
>105946229 >105946269 >105946389 >105946512 >105946542 >105946559 >105946340 >105946354 >105946381 >105946399 >105946416
--Possibility of backend devs to transition to AI/ML without deep theory discussed:
>105947141 >105947287 >105947334 >105947381 >105947465 >105947641
--HumeAI voice cloning impresses with near-indistinguishable synthetic speech:
>105945482 >105946684
--Frustration over excessive model-specific code despite gguf standardization efforts:
>105941377 >105941581
--AMD GPU offloading struggles with model compilation and memory allocation:
>105942046 >105942116 >105942198
--Koboldcpp updates include audio input and vision model enhancements:
>105939162
--Cautious optimism for Ernie A3 21B model performance despite limitations:
>105943416 >105943745
--Automated LLM paper tracking tool for arXiv submissions:
>105943027
--Evaluating local code models under 24GB VRAM constraints and API model reliability concerns:
>105946194 >105946211 >105946319 >105946348 >105946457 >105946471 >105946492 >105946534
--Ernie 4.5 MoE support added to llama.cpp:
>105941225 >105941296
--Ernie4.5 MoE fix merged into llama.cpp:
>105942062
--Miku (free space):
>105941124 >105941834 >105942915

►Recent Highlight Posts from the Previous Thread: >>105939055

Why?: 9 reply limit >>102478518
Fix: https://rentry.org/lmg-recap-script

>>105948340
fuck off

>>105948301
ani is garbage, literally the only thing anyone likes about it is that the 3d model is hot
it's barely functional jank and speaks in LLMslop

>>105948364
Elon won. Cry more troony

>>105948301
>Ani, on the other hand, represents what local models should have aspired to achieve
Yeah and it's too bad...what a missed opportunity.
I remember the animated megurin from what, like 2 years ago?
I'm shocked no one has fleshed something like that out and shared it here

>>105948268
>16 votes for hatsune miku(male)
>(male)
Personally I just wouldn't vote at all. But mikutroons just have to let everyone know they are into men pretending to be women.

https://x.com/FearedBuck/status/1945213154016821709
Is this the Ani thing everyone's so hyped about the quality of?

I've been thinking, Mixtral Instruct is in fact smarter than Nemo/Rocinante and can better keep up with context and more advanced plots.
The downside is that it's text is way more slop, but that can now be massively improved with banned strings.
I think it's time to go back and try Mixtral Instruct again with new knowledge and experience.
Does anyone remember which Mixtral Instruct merge was best in the end?

Also why are there like 40 /aicg/ threads today?

>>105948364
This. It became popular within normies, but there's nothing new or well-made here.

i don't hate cydonia v4 so far. like all mistral models, its very chatty. it seems alright for rp and erp. i've never been impressed by mistral's 22-24b models for rp so its nice that it doesn't seem like total shit after some use. i'm going to use it some more but its nice to use something a bit newer than nemo thats still small. i didn't like the new mistral small 3.2 (with and without thinking) for rp

I don't like miku but "ani" aka grok (male coded cloud model) with a misa wig is some really forced shit.

>>105948169
Elon has pushed us further towards the true AI waifu dream than shitty censored open 32b #45345, horrible memetune #37278378 and terrible open frontend #58 has in the past two years.
Now the entire sad lazy open scene is forced off their asses. Without Elon we'd still be sitting in caves using the piece of shit that is ST in three years, pretending that it's remotely enough.

>>105948414
I swear I saw something like that happen already...

has anyone counted the amount of /aicg/ threads right now
and you thought /lmg/ was retarded

>>105948452 (me)
there was a spam of them but jannies cleaned up

>>105948421
um but nothing has happened yet, this feature is only a few days old

>>105948467
Yet the wave it made is bigger than those of both AI Dungeon and the original character.ai back during their peaks.

>>105948452
some retard was sperging over links in the OP, apparently sillytavern belonged to someone he didn't like or something

>>105948421
I predict another 2 more years of 32b #45345, horrible memetune #37278378 and terrible open frontend #58 and not a single open weights model focusing on AI gf aspect.

>>105948489
Then, open source rightfully dies and everyone here moves on to grok.

>>105948379
>voting in botted poll
Retard

How do I install ollama so I can have local AI waifus?

>>105948572
No wonder Altman is late. He got his calendar from Musk.

>>105948592
Maybe the real local oai model was the Epstein files they released along the way.

>>105948514
>everyone here moves on to grok
Even if I could fully personalize Ani and she had infinite context and she could ERP everything I ever wanted in a way I want I would never use grok or any cloud model. Just think for a bit how many things the model provider can do to you with power like this.

>sar redeem the grok spam
>3 in 5 posts are xitter screencaps
jannies?

>>105948617
>>105948085
>>105948096

>>105948606
my tinfoil hat tells me that the somewhat halfassed release of this elon ai gf is because it was all a setup to legislate and ban it before it takes off

>>105948375
Ani actually looks like a developed woman, with curves that trooooons can never have. I think that's why trans creatures and pedos hate her so much.

>>105948617
local (miku)spam general

>>105948606
Steam needs better competition

>>105948572
>he will make your heart race
>not: he will make your spine shiver
what a fucking tourist.

>>105947940 (OP)
Dont know where to ask. But how much energy do you think these big ai like chatgpt or grok use. How do they kee up with the demand

>>105948572
>malebot for women named Valentine
ghey

>>105948147
At least she'll inspire local to innovate and create something similar that allows us to do this with ANY character. I don't think you're seeing the bigger picture here.

>>105948643
Nah, they'll probably add age verification and keep going.

Worst case scenario, Musk is going to create his own payment processor. That in particular will be fun to see and might even turn out be a positive thing for anything porn/ERP-related for local AI (think for example of the various providers/hosts that had to bend the knee to the credit card companies' demands).

>>105948415
honestly I prefer it being more chatty rather than more "her eyes sparkled with mischief" slop

>>105948686
Y-yeah... for women

>>105948606
god i hate these broads so much
they live in their own bubble not interacting with 99% of people they supposedly hate anyway
can't have sex, can't jack off, can't live

>>105948688
>local to innovate and create
LLMAO

>>105948754
the 'her eyes sparkled' is the chatty part, its filler. all mistral models do it excessively. but if you're already used to it, like using nemo, its no surprise. but if you did the same scene on llama 3, it'll take way less messages to get to the point and move the story along. mistral models love to hang in the scene and keep describing everything rather than moving on.

its hardly a test but my last erp scene when i had cydonia loaded i decided to let it keep writing. it took 20 messages at 300~ tokens per message to finally conclude. it repeated itself a lot, like how many times do you need to hear about hot breath in your ear, within 3 messages? so i did the same with a llama 3 70b tune and it got the scene over with in 7 messages, and was dirtier about its descriptions.

but with cydonia, previously the mistral small models were actually dumber than nemo. so i never bothered with them much. this one is at least more consistent. i'm not sure i see any big jump in smartness either though, over nemo.

>>105948606
this the same elon that paid amber heard to cosplay mercy as serve as an onahole

>>105948393
they probably got someone from drummer's discord to finetune that shit kek

>>105948393
Holy loop

>>105948189
How? The dialogue I've seen is really bad.

>too afraid to post in this topic
LMAO. The absolute state of mikutroons.

>>105949083
It is obviously a false flag.

Ani Kawaiiii!

>>105948972
- Local open source copycats => we'll probably finally move on from the narrated ERP trope
- AI companies approach to "safety" changing as people realize it wasn't as bad as they claimed
- Possible payment processor alternative if Elon goes nuclear on (((Visa))), (((Mastercard))), etc
- Grok gets most of the pushback => smaller companies/entities can get away with more
- ...?

>>105948393
Damn it's worse than a 3b

>>105949143
>worse than a 3b
since it is Elon that means it is probably a 7B to 9B

>>105949120
>Grok gets most of the pushback => smaller companies/entities can get fucked even harder

>>105949083
>22 to 4
Let xim rig the poll in peace bro

I like this Ani. Become Ani.

>>105948865
For me it's definitely smarter than nemo and less of a prude than gemma, which is what I was looking for as a 24gb user. It can still have some issues though describing stuff properly but it's not as bad as when I was playing around with 12b models.

>>105949176
For example: MistralAI somehow is managing to release the least censored LLMs. They're big enough to be the #1 AI company in Europe and train their own models, yet small enough worldwide to fly under the radar while Meta, OpenAI, xAI get all the bad press and lawsuits.

nobody answered my post... very sad...

>>105948088
Imagine using a public cumdumpster instead of keeping a local copy all to yourself.

drummer is a faglord

>>105949276
Here I'll answer you
*ahem*
niggers

>>105949316
>official /lmg/ mascot: hatsune miku
>official /lmg/ finetuner: the drummer
>official /lmg/ actual dev: cuda dev
I am beginning to see a pattern here.

I was not that impressed with K2 for RP compared to some of the hype, but it's really impressing me as a research/programming assistant for some of my work and side projects
I just threw a (frankly pretty poorly specified) issue with a computer vision pipeline I've been tinkering with to a bunch of models on OR and the only two models that gave a satisfactory answer were K2 and grok4 - and K2 got it instantly while grok4 thought about it for over a minute first. all other models (gemini, claude4, o3) gave very surface level answers and recommended some well known ideas that sounded vaguely related but actually had nothing to do with the issue and did not solve it at all
pretty crazy model, china is cooking

Just got a 5070ti, can you guys recommend any 12-15b roleplay models?

>>105949376
get 256gb ddr5 and a ds quant.

>>105949346
how does cudadag fit into it?
if anything, he's based. he said he got offered money multiple times, but would rather do things his own way. greg said basically the same thing

>>105949376
https://www.youtube.com/watch?v=rNg2Dh6gPkw

>>105949399
>Nemo my name forevermore
Oh fuck....

>>105949316
It rhymes with "spammer".

>>105949469
>>105949195
>>105949105
>>105948606
>>105948572
>>105948432
>>105948342
>>105948190
>>105948127
>>105947980
Samefag, probably the anti-miku spammer from the other threads who thinks he's being clever here.

>>105949346
team sao desu

>>105949376
Nemo, Rocinante, CaptainX/Eris
But honestly with how cheap system ram is just get some and run a larger MoE model at q4 or something, it'll be about the same speed and smarter.

>>105949603
>a larger model run off RAM will be about the same speed as a tiny model run off VRAM
Uh, no?

>>105949538
u mad bro?

>>105949603
Recommend a good MoE then

What advice can you give a poorfag that can't afford a strong enough computer to run models locally (my computer is from 2012 and I literally never felt the need to upgrade it until now that uncensored AIs are so readily available)?

Custom local animated waifu who can interface with kobold/silly and XToys and shock my cock and balls with a Coyote when?

>>105949662
I don't think coyotes are getting brain chip implants to interface with kobold/silly for at least 5 decades

>>105949662
It'll max out the power, then you will be in severe pain. Maybe that's what you want?

>>105949661
get a girlfriend you freak

>>105949316
i hate this twink so much

>>105949661
That's rough. You might still be able to run a small model with just CPU and RAM.

>>105949635
With full context I can literally run the 30b qwen3 MoE's at q8 faster entirely on system ram than I could running nemo on a 4080 at q6, so yes.
Keep in mind I said *larger* and not *large*.
Running say, the 235B is definitely not going to be faster than any 12B, of course - since it has 22B active params and it's fucking huge so the ratio of what's in VRAM to System RAM is way, way different.
But it's still bearable with some fucking around with override tensors in your config, and even comparable in speed to what people with a largestral exl quant running on dual 3090's get.

So getting some system ram for a parameter count bump on the cheap around the same speed is a viable option to consider. The future is now, old man. Dense is old news.

on nolima, testing with a llama3.3-70b finetune at q8, 11k tokens log. I've asked {{char}} to provide a detailed description of {{user}}'s appearance. it actually nailed it, maybe attention is not actually that bad.

>>105949661
Try this one, whichever fits into your ram https://huggingface.co/unsloth/gemma-3n-E4B-it-GGUF/tree/main

>>105949662
That's actually possible already, there was a sillytavern extension for controlling devices out months ago (Sorcery), likewise for one that uses a 3d model for expressions (forget the name, but its in the repo)

>>105949784
It is 3.5T/s for me on 4400Mhz DDR5 iq3 quant.

>>105949737
I swear that is not what I want to use it for..

>>105949808
What in the ungodly fuck is wrong with your rig, my ddr5 is being locked to 3200mhz because my memory controller sucks ass and I'm getting well over 30t/s on empty context and over 10t/s when full as shit.
Seriously that speed is NOT right, the fucking 235B can be ran at 4-6t/s almost entirely on ddr5, something is wrong with your setup.

>>105949788
l3 3.3 70b is an excellent model

aaa eee iiii ooooooooo

Is he right? >>>/a/280697653

>>105949840
It is a 7800X3D not a server.

>>105949784
>Keep in mind I said *larger* and not *large*.
Oh.
>30b
All the models in that range are significantly worse than Rocinante so what's the point?

>>105949877
I only know that it is coming in 2 more weeks.

>>105949864
kill yourself

>>105949731
I mean I've taken 80% to my balls already. I'll be good.
>>105949802
You mean this? https://github.com/moeru-ai/airi

>>105949864
Please post using correct pictures. Your whore skipped deprecation and went straight to obsolescence. This is an Ani general now.

Ani's breasts are too large.

>>105949731
>max out the power, then you will be in severe pain
>Max
>Pain
>Maximum Pain, I must endure.
https://www.youtube.com/watch?v=OKWVNeDYZmU

>>105949946
>breasts are too large
faggot

>>105949995
chubby chaser

>>105949940
ani isn't local

>>105950032
miku isn't ai. miku isn't a real woman.

>>105948415
Cydonia is fucking ass like all Drummer shite. Way too horny no matter how you prompt.

The only mistral small (and really, sub 70b model period) model that's worth anything is that Dan personality shit. I've been stuck with it for months now because there's literally nothing else

>try out QWQ and Qwen shit
>all dogshit
>try out Gemma, it's either decensored into retardation or it's censored to fuck (dogshit)

If it wasn't for the few Mistral small models i've tried, i'd unironically still be using Nemo despite being capable of way larger models.

while i do love deepseek and everything they have done for local, v3 is very stupid in comparison to o3-mini

>>105950044
>Cydonia is fucking ass like all Drummer shite. Way too horny no matter how you prompt.

>>105950041
miku is local

>>105948865
>>105949228
>>105948415

As a fellow 24GB Vramlet. What models are even worth using besides the mistral small finetunes? Is Nemo really comparable? I remember using it way back when and just figured the larger models made it obsolete so moved on.

>>105949469
She needs some blue highlights in her hair

>>105950059
Nemo is better because it seems to have less safety training.

>>105950053
if it's a skill issue, why do Dans Personality Engine, MistralThinker, Pantheon all produce superior results without turning into positive bias braindead smut like all of Drummers "Moisssst" fine tunes?

>>105950044
>Way too horny no matter how you prompt.

>>105950072
Because you are a promptlet.

>>105950072
To be fair, you have to have a very high IQ to use my models.

>>105948865
which llama 3 tune? EVA and wayfarer are the only two I still have on my hard drive. for my tastes, the only mistal models worth half a shit are mistral large 2 (2407 is marginally less sloppy) and surprisingly undster's mistralthinker. I've tried a handful of the mistral small 24b 'tunes and I feel roughly the same way about them that I feel about most of the Gemma 12b tunes I've tried; not worth the drive space. mistralthinker is the only one that doesn't make me immediately close out. never got the hype for cydonia (no matter which version) because as you said they feel noticeably dumber than nemo. thanks for testing v4 so I don't have to bother
I wish labs would stop pretraining base models. I sometimes wonder if it kneecapped finetuning efforts, although I think most of the "finetuning community" hasn't progressed beyond throwing shit at a wall. now with mega-MoE as the new meta, I fear it'll only get worse

>>105950059
qwq snowdrop, otherwise make your peace with slow response times and bigger models. I was using qwen 3 235b for a long while but I cannot stand the way it always devolves to short one-statement sentences as a way to emphasize a character's inner turmoil. that said, it was a good middle ground between large enough to be super coherent (generally) and small enough active parameters that I wasn't getting terribly impatient
t. 16gb vramlet

BLAST FROM THE PAST
NEVER TO FORGET

>>105950095
>I wish labs would stop pretraining base models
Most finetuners are finetuning the instruct versions anyway, so the companies not releasing the base models wouldn't change a thing.

What would knee-cap finetuning efforts would be if the same companies that trained the models put some research and compute into adding half-decent RP/ERP capabilities to their models, which incidentally is more likely to happen now that ERP has basically become mainstream with that Grokette companion.

>105950114
No one cares about your obsolete shitfu. This is an Ani thread now.

>>105950206
ani is not local, simple as

>>105949538
Take your meds schizo

>>105950223
ani is an AI, simple as

>>105950009
He based.

>>105950234
no it's not, its a koikatsu 3d model

>>105950076
>three fingers slop imagen
he doesn't have a skill issue, but you have a taste and eyes issue

>>105949662
I'm completely out of the loop on this but this was posted two threads ago:
https://x.com/iamtonyzhu/status/1945424729118302407
Except it's in chinese + in development.

>>105950032
Elon wont read your mesugaki logs lil bro

>>105950241
cope

>>105950206
who cares. post your images and get on with people you troglodyte
people winning and losing is such a dumbass take.

>>105949960
lol

>>105948617
jannies?
>>105950114
>>105950076
>>105950053
>>105949864

>>105947980
I don't owe you shit

>>105950206
not local nor is this a sota waifu

>>105950276
based

>>105950276
Miku no you'll go blind

Imagine if jannies actually literally just deleted all off topic posts as a rule and didn't have to think. No Miku, no Ani, no complainers and meta discussion about the rules either. Simple and effective. But of course their ad revenue.

>>105950206
ani so pretty

Did you redownload your kimi goof?

>the real ani is in /lmg/ now
does this mean llama.cpp is in anistudio?

>>105950338
not yet. I just pay attention to developments in ggml from this thread. both diffusion generals don't track it or sdcpp

>>105950331
Can't have a trash general without trash.

>>105947940 (OP)
>>105947980
>>105948127
>>105948190
>>105948240
>>105948606
>>105950206

THE END IS NIGH

>>105950355
nice effort, but it must of been a pain to clean that
probably regretted it

>>105950264
>point out something isn't local in the local thread
>schizo hallucinates pedophiles and trannies

>>105950374
hatsune miku has nothing to do with AI LLM's or this topic though

>>105948606
>le nerds are responsible for society degeneration but my heckin faggot/tranny/pedo friend group are so heckin valid!
Total Westroon Death

>>105950374
>>105921334
it's AI + AI video gen
finally, finally, the "that's AI cum!" schizo is vindicated.

>>105950348
are you adding nodegraph support like comfy?

Ernie
Qwen3-235b
gemma-3-27b
kimi-k2
deepseek-r1

Who win?

>>105950369
>must of been a pain to clean
whenever I see "must of" instead of "must have" I feel it's an instant retard indicator
and you, friendo, haven't broken this rule by believing AI slop is real

>>105950374
It is obvious why you fags care about 'local / cloud', nothing out of reach here.

>>105950415
you shouldent of said that

>>105950369
>>105950389
oops wrong reply

>>105950355
>>105950114
>>105950076
>>105950053
>>105949864
jannies clean this shit up

>>105950400
I can run gemma-3-27b on my local GPU; it wins.

>>105950415
maybe if you paid your taxes i woudl've of had a better education

>>105950400
I just want a deepseek with proper tool calling support.

>>105950449
>tool calling support

to make it test its own code?

are we getting raided by musk goons? this thread is a trainwreck

>>105950465
For use with zed.

>>105950470
>musk goons
you mean pajeets? always

are we getting raided by mikutrօοns? this thread is a trainwreck

>>105950470
it's just one schizo that wants to kill this thread because he always got called a retard for asking for tech support

>>105950400
ernie fails cockbench
qwen3 is not bad but slower than deepseek at a high enough quant to be competitive and is less knowledgeable
gemma3 is hideous
can't run kimi

>>105950486
no just anigros
often kurisucucks and sometimes even mahores

>>105950498
>Sam Altman
opanai model won't fail cockbench confirmed

>>105950501
>anigros
See >>105950317
>>105950276
>>105950114
>>105949864

>>105950360
There's already a /trash/ board doe.

>>105950390
yes. it will be hybrid like unreal blueprints because I think single execution is too limited. I haven't made a final decision on the nodegraph lib but components make it easy to get set up. I'm more concerned about the plugins syncing with the managers properly so it's just more backend logistics work

>>105950470
>someone posts a different anime girl
>must be a raid
your brain on mental illness

.

>>105950558
>single execution is too limited
finally someone gets it. i hope to see it on the main branch some day.

>>105950572
Literally
>anons post Ani for 2~3 threads in moderate way
>mikutranny gets butthurt again and shits up the place with bait and xitter pics

there is no way anyone local likes the conman musk or anything he does
of course they are just here to troll

>>105950558
based OG ani

Impressive o3 and Grok 4 both scored the same on the new ARC 3. 0

>>105950597
Does ARC 3.0 have unclear tasks with multiple solutions like the first one?

>>105950597
It is not fair because the questions weren't added to the training data yet.

>>105950589
>in moderate way
who decides that
not you, kurisucuck

>>105950664
Not kurisufag, try again schizo.

>>105950664
>kurisu
rent free. oh nooo he is about to have another melty

Stop giving him (You)s. You won't convince him and nobody else needs convincing.

>>105950682
he wants to become barneyfag

>ani status: won.
>miku status; left the general to fuck another nigger and doesn't intend to return

>>105948393
>[COMMENT ON VIDEO FEEDBACK]
>[COMMENT ON WHAT WAS SAID]
>[FINISHING SENTENCE]

What a joke, the simple waifu I'm planning on ue4 with local models since a few months can do way more already

>>105950719
https://files.catbox.moe/sa6xsa.webm

>>105948190
Still waiting for Ernie support

>>105950433
>>105950589
Move /lmg/ to /vg/ if you want better threads, at least there jannies care and delete blatant spam.

migger troons on suicide watch GEEEEEEEEEEEEEEEEEEEG

The powers of the mesugaki test and cockbench combined.

Easier to reproduce because of the much shorter prefill.

Ernie 300B.

I feel bad interrupting the constant trash segment but I just tried collecting the sex I owed from Ernie 300. (IQ3XXS from the reupload brothers).

My impression is that it has an understanding of at least a 70B but writes the blandest and most repeating output I have ever seen. It also feels very werewolf millionare. John Titor was right that Ernie wouldn't save local. Even 235B is better.

>>105950798
>that disclaimer
But seriously, the point of the megusaki test is to also test multilingual capability or en->jp specifically.

>>105950592
local is just a dogwhistle for tinkertroons
>but i NEED to tinker for hours on shit that dont work
>b-but i NEED muh privacy to hide my pedoslop
nah just pay the based rocketman and enjoy your big titty agi mommy
or cut your dick off, your choice o algo

SaaS is just a dogwhistle for poor

>le poor!
'When you are out of arguments' - /g/ay moment.

Ani was made for sex. Miku was made for pedos.

>>105950827
>your big titty agi mommy
Hell yeah

>>105950883
Still more on topic than mikufaggotry.

>>105950798
A few comparisons. Sometimes I forget that nemo thinks that a mesugaki is various things related to eyes.

https://docs.llmvtuber.com/en/
Anyone fucked with this? I'm experimenting with it but can't figure out how to get it to utilize the full range of the live2d animations
Seems nice though

Will the war on Miku ever end?

>>105950938
mikufaggotry is never on topic thoughbeit

Why is it July 18, 2025 and there still isn't a better model for an average gaming PC to run than Rocinante?

>>105950993

It certainly will. With Miku's victory!

>>105951002
Because there's only so much you can do with 12B parameters and there's very little incentive to train new models at that size. It's amazing that qwen bothers doing it.

>>105950400
r1 and k2 are the only ones even remotely in the running there, and r1 edges it out but maybe a reasoning version of k2 could surpass it

>>105951002
There is no economic use case for small RP tuned models. You need to make money so you have to do a chub or novelai and serve it via API

I'd rather see that compute spent for better video models because a 14B video model can make me coom (to young girls) so much better than a 14B text model ever will

>>105951026
>There is no economic use case for small RP tuned models.
What do you think the economic use case is for Nemo?
And is the reason we still don't have a better base for RP finetunes than Nemo simply that Nemo received a minimum of safety training and this won't happen again?

>>105951002
You're hallucinting. Stop eating drummer-spammer's shit.

>>105950946
Thanks for the laugh.

Well, "me" is eyes, so I can at least see where nemo is coming from.

>>105951040
>And is the reason we still don't have a better base for RP finetunes than Nemo simply that Nemo received a minimum of safety training and this won't happen again?
Yes. And when you combine that with general stagnation of models you kinda have to ask yourself what are we even doing here anymore?

>>105951082
next time do asuka

>>105951082
I thought the thread rotated vocaloids?

I think Satania would be appropriate once in a while.

>>105951104
>AGP avatar with swapped color palette
Same thing shit

>>105951002
If Mistral + nvidia were to make a new nemo on top of something like Qwen 3 30B A3B that would probably be as good as it gets for the likes of us.

I think literally everything would be better than the same fucking generic anime girl(male) over and over again.

>>105951002
fuck off drummer

I shouldn't be but I am pleasantly surprised that 7b Nemo knows about him

>>105951002
I really wish we would get a newer and updated version of mixtral instead of only having small or huge models to pick from.

How is it that VRAMlet tunes have never fixed the "she looks back at you" problem? Isn't that a standard thing they can put in the dataset?

I just got here. I like the new waifu.

>>105951118
Doesn't MistralAI still have or know the dataset? Why would they need NVidia? If they haven't replicated yet Nemo's recipe after releasing that model, one year and several models later, we can only conclude that they don't want to, or cannot anymore (possibly due to copyrighted data, or because their current dataset filtering techniques remove what made Nemo good for roleplay).

>>105951203
True high quality ERP has never been tried.

>>105951215
There is nothing special about the way it was trained. It just wasn't censored after training finished like literally everything else is.

>>105951217
I've been waiting for someone to post this phrase again for a while now, thanks.

>>105950072
What format & settings do you use to get good results from this? And which of those models do you recommend? I never had good luck with small.

quality erp is an oxymoron
I put on my robe and wizard hat

>>105951211
Local finally has a decent mascot.

Teehee the schizo got triggered: >>105951131
>>105951203
Drummer had mixed success with trying to unslop models via finetuning. See: UnslopNemo
His attempt reduced but did not eliminate slop, and he said some slop seemed much harder to reduce than other slop.

>>105951140
Why do you have your shit configured with gay pastel colors?

>>105951217
High quality ERP will never be achieved with autoregressive LLMs because that's not just about prose quality.

LLMs will never understand you, will never correctly "read the room", won't introduce new elements and expand the story organically and tastefully, can't keep track of too many things at the same time, don't have real memory nor real common sense, can't plan ahead, can't surprise you, and the longer the context the worse and more stubborn they get.

>>105951291
>can't surprise you
I am surprised when R1 suddenly includes scat into my smut.

>>105951291
His point is that no one has even tried. Whether they succeed or not is another issue. Though at the very least, if someone did seriously give it a shot, as in the likes of Google, OpenAI, etc, the model might at least be better than what exists now.

>>105951301
your mouth must've been closer to character's ass than it's physically possible for that to happen

>>105950188
Wow can we go 2 seconds without this thread becoming antisemitic?

>>105950946
What if you pre translate the word for nemo?

>>105951346
Yeah, 2 seconds have passed already.

fukken kikes man

Woah woah, everybody please cool it with the antisemitism. You goys need to take a deep breath of fresh air!

>>105951291
>can't surprise you
The rest of your complaints are true, but this is patently false.
At least a few times a week I end up bursting out with laughter because something completely unexpected gets genned.

>Ani prompt with gemma-3 on /pol/ context
Any models more humiliating than this cunt?

>>105951348
What even is the proper translation? You'd need at least a page of text with examples.

>>105951369
Now that the trannies have been driven out of town the thread is finally based for once

glm4 100b moe will save local

>>105951397
100b isn't enough to save anything

>>105951309
My point us that true high quality ERP is like true comunism.

>>105951383
What? It's just 'Female Brat'. You could probably even get away with just 'Little Brat' if their sex is mentioned.
It's not a difficult concept for an LLM to grasp, hell - most of them default to being teasing shits anyway in RP.

>>105951410
That'll be 10B activated params right?
Might be usable, here's hoping they use something like MLA for context.

>>105951291
We agree Yann. Can you now call drummer a faggot?

>>105951291
Unironic skill issue

>>105951397
Is that actually coming? I was pretty impressed with the dense 32B's internet culture knowledge.

>>105951428
High quality ERP is bait meant to convince useful idiots to revolt and overthrow power structures for those who wish to enslave them?

>>105951429
Fair.

>>105951410
shut the fuck up

>>105951428
Well that was the joke, but actually serious ERP has not been tried by any company so that's also a point.

100B params ought to be enough for anyone

>>105951481
Nah, I can see a noticeable difference between Mistral Large 123B and Qwen3 235B, and a gap again between those and API models which are probably Xboxhueg.
There is some impressive stuff being done with 'compressing' down a smarter and more coherent model in smaller parameter counts, but that's all happening in the <10B range, seemingly. I haven't seen a jump in the 70-100B range in a while, mostly because we rarely friggin' get any models in that range.

>>105951428
true communism = openai-tier censorslop for wrongthink

>>105947940 (OP)
I've been out of the loop the past few months. What's the best model nowadays I can run on a 4090? Still deepseek R1?

>>105951580
Yes.

>>105951580
There is no version of actual R1 that fits on even one of those 48gb modded 4090's, you ollama-using dingus.

>>105951589
Clearly he is exps=CPUing

>>105951589
The DeepSeek-R1-Distill-Qwen-32B-GGUF fits just fine

>>105951592
>Clearly
Not at all, otherwise he would mention his RAM instead of GPU, I think.

>>105951530
meds

>>105951673
That's not R1 you moron, that's a finetune of Qwen 32B with some R1 chatlogs taped to it.

So is anyone seriously using local now? Cloud is easier and more capable, but you lose privacy (doesn't really matter if you're asking for programming help?)

>>105951824
Doesn't matter if I'm making a one-off script but I'm not using cloud AI for anything I would not put on github. This includes work stuff.

>>105951824
if I have learned one thing using llms extensively for masturbation and erotica, it's that I would never trust them with a remotely serious task like programming

>>105951846
Programming is the best use case for AI because the results are objectively verifiable.

>>105951878
Same for erotica - just check for fluids

>>105950986
Mikufag is unhinged and always seeks for >>105950719 >>105950545 >>105950053 shitflinging so it will never end.

>>105948085
night night
keep your butthole tight

Two years since GQA and its spinoffs ruined model creativity forever.

consensus on LGAI-EXAONE/EXAONE-4.0-32B?

this is YOUR future!

>>105952036
Just the usual coal.

>>105951824
my entire workplace uses 4x3090 for qwen coder + speculative decoding
plus two other model that handle embedding & reranking
most of the time it's just for rag and roocode in vscode. probably will setup some mcp server soon

>>105951118
>>105951215
https://huggingface.co/blog/nvidia/openreasoning-nemotron?linkId=100000374186136

Nuh vidya released their R1 distills

1.5B/7B/14B/32B

>>105952059
nobody gave any examples of gooning grok. just some virgins that said the sexy dancing was risque. kinda lame desu

>>105952097
>32B
worthless safetymaxxed slop

>>105952059
At least it wasn't a dude

>>105952059
kekekeke its a pajeet giving you remote blowjob

>we still don't have a model where storywriting/rp makes up the majority of parameters instead of coding and random trivia
surely someone will do it

>>105952229
k2 has proven that literally anyone can train their own deepseek-tier model just by throwing a couple of million at it and following the established formula
the golden age starts now

>>105952254
Economics are the biggest barrier. No one group or person has that kind of money lying around to waste on if they didn't have a reason to spend said money.

>>105950064
I'll play w it later. We never really came up with a canon haircolor for her. Black or dark blue, cyan looked too much like a miku imho but the early ones were a light blue.

>>105951217
>True high quality ERP has never been tried.
It was called character.ai. It was high quality for 2022. It wanted to do it, they just stifled it with post-gen censoring. The C.AI 1.2 models were left wide open for a day. It was really amusing to see just how willing it was to do sex scenes.

>>105952276
There is no point of doing that when every model is trained on the same slop and give the same output

>>105952036
Sloppy. Clearly distilled. I like how only giant Chinese models like k2 and deepseek bother to be original. All the sub 32b range models are just distill after distill.

>>105951217
we're doing it now

ded thread
it's all over...

Either OpenRouter is hosting all fucked up versions of Kimi K2 or this thing has sub-8k effective context considering how quickly it forgets established plot points.

>>105952788
Playing Rocket Migu

>>105952097
am not impressed. distilled models suck

>>105952841
context numbers are lies

>>105949065
>>105949065
>>105949065

>>105952924
Yes but most models at least manage to keep up the illusion until 32k these days without confusing that the previously unnamed character who had eventually their name to be A revealed was not a separate entity from the character named A in the current reply.

>>105952957
Gay thread

>>105952992
>>105952992
>>105952992

>reasoning meme model
do people willing to read TWICE?

>>105953087
How did we go from this to I cannot and will not send shivers down your spine

>>105953118
do esl willing to learn english once?

>>105953243
you retard no? then shut the fuck up your fucking cunt