/lmg/ - a general dedicated to the discussion and development of local language models.

Previous threads: >>105698912 & >>105689385

►News
>(06/25) I posted about my AGP fetish on r9k: https://desuarchive.org/r9k/thread/81611346/
>(06/21) LongWriter-Zero, RL trained ultra-long text generation: https://hf.co/THU-KEG/LongWriter-Zero-32B
>(06/20) Magenta RealTime open music generation model released: https://hf.co/google/magenta-realtime
>(06/20) Mistral-Small-3.2 released: https://hf.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506
>(06/19) Kyutai streaming speech-to-text released: https://kyutai.org/next/stt


►News Archive: https://rentry.org/lmg-news-archive
►Glossary: https://rentry.org/lmg-glossary
►Links: https://rentry.org/LocalModelsLinks
►Official /lmg/ card: https://files.catbox.moe/cbclyf.png

►Getting Started
https://rentry.org/lmg-lazy-getting-started-guide
https://rentry.org/lmg-build-guides
https://rentry.org/IsolatedLinuxWebService
https://rentry.org/tldrhowtoquant
https://rentry.org/samplers

►Further Learning
https://rentry.org/machine-learning-roadmap
https://rentry.org/llm-training
https://rentry.org/LocalModelsPapers

►Benchmarks
LiveBench: https://livebench.ai
Programming: https://livecodebench.github.io/leaderboard.html
Code Editing: https://aider.chat/docs/leaderboards
Context Length: https://github.com/adobe-research/NoLiMa
Censorbench: https://codeberg.org/jts2323/censorbench
GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference

►Tools
Alpha Calculator: https://desmos.com/calculator/ffngla98yc
GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator
Sampler Visualizer: https://artefact2.github.io/llm-sampling

►Text Gen. UI, Inference Engines
https://github.com/lmg-anon/mikupad
https://github.com/oobabooga/text-generation-webui
https://github.com/LostRuins/koboldcpp
https://github.com/ggerganov/llama.cpp
https://github.com/theroyallab/tabbyAPI
https://github.com/vllm-project/vllm

mikusisters not like this....

>>105704595

More miruku sister which could mean milking sister or miracle milk am i right or am i right

>>105704582 (OP)
>>(06/25) I posted about my AGP fetish on r9k:
Thread worth less than that btw
Go back to your discord and spam there till your skin turns blue.

>>105704582 (OP)
>>(06/25) I posted about my AGP fetish on r9k: https://desuarchive.org/r9k/thread/81611346/
based
>migger troon op same as issual
not based

>>105704679
I am in a discord. It even has troons and mods that ban for spam while they spam.

Is this thread protroon or antitroon?

>>105704710
4chan and Discord - brothers in arms and war.

>>105704720
It's a schizo thread

>>105704718
I am sorry stray catalog anon. Our thread has a bit of troon infestation problem.

>>105704720
since its hard to tell it's best to assume its a tranny intent on derailing /lmg/ like always

>>105704733
and you are our king schizo

who needs that green haired anime mascot post this green haired anime mascot instead

>>105704735
you do a good enough job derailing it on your own you digusting festering troon freak

>>105704733
If it is a schizo thread then where is the usual thread recap?

>>105704741
We don't need mascots at all, it adds nothing for topic (Large Language Models) discussion.

>>105704582 (OP)
>I want to have a foid as something like a living dress-up doll: designing her outfits, dressing her myself, doing her makeup, controlling what she eats, showing her off as a walking decoration etc. Not really interested in any kind of romantic dimension since I only love one woman (even though she'll never be mine), though I acknowledge there's an inherently erotic aspect to the arrangement.
That is a bit fucked in the head, innit?

>>105704741
Don't make her part of this mess

>>105704722
this place is undistinguishable from what it once hated now, really the fall of rome.

>>105704788
>It's flat
Holy gay.

>>105704776
Novel concept that is impossible to grasp for at least 50% of the posters.

>>105704783
Don't judge. Anything that gets wh*tes to self-emasculate and drop out of society is a good thing

>>105704792
When 4chan was offline it was basically confirmed that jannies and admins are unironic troons. It is a dead corpse paraded around by troons who want to make it a safespace without chasing away majority of posters.

>>105704788
more of >her?

>>105704720
I'm anti whoever can't shut the fuck up about it.

>>105704741
smelly we have kurisu at home

>>105704793
You're flatter which is double gay

>>105704776
Right cause it attracts the usual suspects.

I have an important announcement. I am the OG kurisu poster. And recently i have changed my waifu of choice so my first AI gf will not be kurisu actually. That is all. Thank you.

oh boy i can't wait to see what's happened today in /lm- geez

>>105704816
Real.
"antitroon" poster(s) become worse than the thing they supposedly hate, and the thing they hate also make themselves worse because they can't help taking the bait. It's just a big cycle of bullshit.

>>105704846
don't let the door hit you on the way out troonie

>>105704783
Sounds very dominant and alpha. He even called himself a chad in that thread.

>>105704582 (OP)
>no blacked card
>no jarted

>>105704849
>

>>105704852
>everyone I don't like is a troon

>>105704864
>canned thoughts

>>105704849
I promise to stop anti-troon posting once mikuposting stops.

>>105704877
It wont happen and you know it, troons cannot and will not stop making things about themselves in any shady way they can.

>>105704885
I know

>>105704877

>>100491834
>>100491862
>>100491881

>>105704877
But in an effort to antimikupost, you ended up also mikuposting >>105704582 (OP)
Maybe if you actually made an honest effort to replace the job of the thread baker and make good, non miku/anime, non bait, serious business threads, you'd be doing something useful, that is actually helping your cause, instead of making the mikuposters want to mikupost more out of spite.

>>105704905
But maybe the spammers can just stop spamming this thread.

>>105704877
>>105704900
Looks like falseflag to me

>/lmg/ isn't dead anymore
turns out all it took was kicking the troons out of their hugbox

what a fun thread we are having today. so lively!

>>105704918
You know they won't. That's why it's on you to either stop making them worse, or to actually do something about the problem, which you can as I said. Even thread splitting isn't a bad thing as long as it's not fake bait like the kurisu splits were.

>>105704925
Wont help, now we need something good out of this LLM stuff. (Also wont happen cause AI labs are hellbent on safety cultism)

I just imagined something. If OP isn't also the troon janny imagine the explanation he has to give about what happened and why he wants to have this thread deleted as trolling... That is assuming he doesn't just lie for simplicity sake.

>>105704925
But being dead is better than wasting time on discussing meta community related crap no one wants to deal with if they have the choice.

>>105704937
>fake bait like the kurisu splits
what is a fake bait?

>>105704956
Of course you don't want your spamming mentioned, discussioned, or questioned. Kill yourself sooner rather than later trooner.

>>105704948
>implying he needs to lie
They all friends in there.
Janny list from leak - https://web.archive.org/web/20250617190717/https://rentry.co/o84vftsb
/g/ has 11 jannies btw.

>>105704963
ask your boyfriend before he fucks your gaping festering axe wound

>>105704900
https://desuarchive.org/g/thread/105611492/#105615767

it really is a wonder why the LocalLLaMA folk didn't migrate here when their sub was dead...

>>105704988
please don't have a meltie we're worried about you

>>105704966
I'm not a mikuposter or a spammer. I don't care about them since the mikuposting was always just easily filtered noise since they're images and not text.
I would welcome a thread split personally IF it was made AND it wasn't some bait or trying to egg on anyone like this thread's OP. I do agree the mikupositng stuff is off-topic. The miku genner (previous OPs?) should probably be posting in /hdg/ or something, not here. If you are serious and make a good, quality thread, I will come it's just that shrimple.

>>105705001
They can't migrate here if they have been here the whole time posting miggers and taking hrt.

>>105705009

>>100491881

If you are serious, then show it and make a non-bait, high quality thread split. I will come and use that thread when I can. It's just that shrimple.

All you had to do was ignore his posts and not engage with his rhetoric. His posts are always unambiguously off topic and often get deleted when reported.

>>105705034
This, so much this. We are moving so fast today, I think we need more threads. 6 or 7 should be enough.

>>105705043
True but I felt like making a canned response that I can copy and paste in the future.

>>105705001
They were already here.

why is everyone so catty today? did someone forget to take their HRT?

>>105705034
>If you are serious
>>104110951
>Death to /lmg/. Death to /g/. Death to the rotten corpse of 4chan.

>>105704582 (OP)
look at ts bruh https://www.instagram.com/reel/DH_Vm0KJ0S3/

>>105705043
>unambiguously off topic
Like mikuspam?

>>105705048
There is only one non-autosaging /lmg/ at the moment. Thanks I will add that to the paste.

>If you are serious, then show it and make a non-bait, high quality thread split. I will come and use that thread. It's just that shrimple. There are no existing competing thread splits in existence at the moment so you can feel very free to do so btw.

>>105705068
Of course troon infested hovels deserve to die and so do you.

>>105705068
>I hope all the illegal 3rd gender jannies kill themselves and join the 41%. World will be a better place if they all kill themselves and they all know it deep down. Don't let your dreams be dreams jannies, you should kill yourself now. I will also proceed to take a shit in this thread. Death to /lmg/. Death to /g/. Death to the rotten corpse of 4chan. Death to all tranny jannies.
Reads like a skinwalker trying to copycat average anti-trans polfag, have a (you) for falseflag efforts i guess...

>https://arxiv.org/abs/2502.00627
>https://gizmodo.com/researchers-dump-2-billion-scraped-discord-messages-online-2000605471
So why has nobody trained a model on this massive dump of 2 billion discord messages?
It's full of "unsafe" language, and apparently a disproportionately large percentage of it is roleplay chats between humans.

>>105705110
>I will also proceed to take a shit in this thread.
funny since he also spammed scat before while false flagging as a miku poster

>>105705068
>>105705098
You know I would agree that in general this site does suck and deserves to die even if I support open source and the concept of local models, but his approach to trying to fight back in fact just wastes his own time while also making the people he's trying to disturb feel more righteous in their own beliefs/cause, which contributes to making everything worse.

>>105705112
Unsafe AND unassistant.

>>105705112
>https://zenodo.org/records/15170676
noooo....

>>105705133
>spammed scat
Wasn't me but I do endorse that anon.

>>105704582 (OP)
I just came to this image.

>>105705136
your rambling again sis

>>105705144
I just came to this post

>>105705133
>false flagging as a miku poster
You think weebs aren't into scat?

>>105705138
This looks to be the same data https://huggingface.co/datasets/SaisExperiments/Discord-Unveiled-Compressed

>>105705151
I hope you enjoyed it ;3

>>105705164
nice, thanks!

>>105705174
I didn't. ;_;

>>105705192
Let me give you a hand with that next time
>///~///<

>>105705206
I need a big hand UwU

>>105705112
Are we sure this isn't already being used? It's not like using SOME good data will make a model good.

>>105705174
>>105705192
>>105705206
>fat greasy weeb hands typed this

>>105705209
I can use more than my hands if they're not enough >⩊<

>>105705214
I have a 16.5 BMI ´꒳`

>>105705231
No wonder lmg shills these

>>105705095
Is this /lmg/'s new Code of Conduct and/or Contributor Covenant?

>>105705229
W-what else can you use o_0

>>105705250
This is not a github project, no one owes you anything.

>>105705210
not going to help the number go up in benchmarks = not going to be used
better to scrape gemini/chatgpt over and over

>>105705265
Well, you could make a guess. Don't make me say it out loud in front of all those people; it's pretty embarrassing ( ˃ ⤙ ˂ )

>>105705286
But all the big corpos use internet sewage clearly and their numbers are fine.

>>105705313
>>105705265
>>105705239
>>105705229
>>105705206
>>105705192
>>105705174
That is all cool and all but https://desuarchive.org/r9k/thread/81611346/ . Yeah you aren't escaping that one.

>>105705231
>'<|im_start|>user' appended to the end of the response
Perfection.
I do not miss sloptunes where they fuck up the EOS token or train on the wrong template.

>>105705365
That's not me ( ꈍ◡ꈍ)

>>105705383
Join 41% like the rest of your friends.

>>105705398
I don't have any friends (,, ‸ ,, )

>>105705398
But I don't know how to code

>>105705398
wtf has to do stirner with these muh moralfags?

>>105705398
I don't have a cute pfp like they do

OP here. I posted that r9k thread on purpose and the self reported here to see /lmg/ alive again. I was just pretending to be into dolls.

>>105705492
I don't give a shit, where is the recap?

>>105705398
>>105705434
These troons swung so far into the identity politics train that they now lack any coherent ideology. Stirner, by contrast, urged you to "consume" every idea only to dismantle it afterward ("base yourself on nothing", i.e. begin from a blank slate, free of false beliefs and illusions ("spooks"). He never meant for *you* to be consumed by those ideas nor make yourself an identity our of it, or by their little anarchist flags.

>>105705398
>tranime
kek everytime

>>105705164
Never gonna use this but I want it because it exists. Well maybe I'll throw a script together and see what some are like. Gonna need 1521 Migus to start the cleaning work.

>>105705559
>Never gonna use this but I want it because it exists.
Same.

>>105705398
>deleted for truth
Safespace prevails.

►Recent Highlights from the Previous Thread: >>105698912

--VRAM limitations prevent LoRA training on large models like Mistral Large with 48GB VRAM:
>105698940 >105698956 >105698974 >105699018 >105699028 >105699010 >105699040 >105699078 >105699159 >105699171 >105699178 >105699210 >105699223
--Investigating unexpected token generation limits in llama.cpp with high context length:
>105704272 >105704320 >105704489 >105704545 >105704568 >105704727
--Workaround for un-downloaded models via Hugging Face repo duplicator:
>105699478 >105699499
--ROCm 7 shows promise in improving AMD GPU performance for large language models:
>105702641
--Exploring alternatives to Nemo for roleplay and structured output:
>105699980 >105700030 >105700344 >105700642 >105700688 >105700706 >105701267 >105700768 >105700783 >105700797 >105702486 >105702605 >105702663 >105702729 >105700839 >105700916
--Tencent Hunyuan-A13B-Instruct-FP8 emerges on Hugging Face with speculation about uncensored capabilities and model quality:
>105699378 >105702734 >105702790 >105702811 >105703194 >105703390 >105699455 >105699596 >105699793
--Discussion around Hunyuan MoE LLM's capabilities and deployment challenges:
>105701434 >105701450 >105701474 >105701557 >105701537
--Server mode shows lower CPU utilization than CLI despite identical configuration:
>105699229 >105699273
--Critique of AI's environmental impact from prompt usage:
>105702835
--Google releases Gemini CLI as open-source AI agent with free-tier model request limits:
>105702601
--Speculation linking Claude's quality to Anthropic's pirating of millions of copyrighted books:
>105702566
--Visual reward model analysis of one-word positive/negative associations:
>105701545
--Miku (free space):
>105699975 >105703188 >105699538 >105704124

►Recent Highlight Posts from the Previous Thread: >>105698922

Why?: 9 reply limit >>102478518
Fix: https://rentry.org/lmg-recap-script

>>105705621
Sex with this Mikuswing

dead gay thread is dead

>>105705383
Join 41% like the rest of your friends.
/Repost
Janny tongue my anus.

>>105705778
I told you I don't have any friends; stop rubbing it in ( •̀ - •́ )

After reading this and previous thread i remember why no company or reaearch papers ever bring up this place.

>>105705799
maybe it's time to spam miku all the time to not trigger the shitposters?

>>105705799
Why?

>>105705534
Speaking of philosophy, I think Jeff Vail's Theory of power takes this a step further, and urges you to see yourself as a node in a web of power relations, realising your true place in the world, and thus also your limitations and possibilities. He starts similarly to Stirner in trying to dismantle spooks.

>>105705799
Not enough grift friendly, sorry.

>>105705815
That's extremely gay and lame.

>>105705820
Lies, y'all shill pure slop here.
For lurking newfags: Clean nemo or deepseek 671B only, never use any finetunes.

>>105705830

>troons
>weebs
>mikuweebs
>whatever AGP is
all of this is the same terminally online brainrot garbage btw
great job ruining the thread and the internet in general

>troons
>weebs
>mikuweebs
>whatever AGP is
all of this is the same terminally online brainrot garbage btw
great job ruining the thread and the internet in general

>>105705859
>>105705861
You freaks started it with gay rights.

>>105705830
>use a 14b model or a 671b model, nothing in between
I have 70gb total ram+vram though

>>105705859
>great job ruining the thread and the internet in general
The internet is the one that ruined them.

>>105705861
It's just 1 poster

>>105705866
>You freaks
gay rights were a mistake too

>>105705867
Stop being poor.

>>105705867
Time to hit the pc part store.

>>105705859
>>105705861
>20 seconds apart
Post Deepseek settings, fellow non-poorfag.

>>105705912
>Xe doesn't know.

>>105705912
I checked my settings in mikupad because of this post and saw that I left temperature at 5 after I was testing stuff.
Surprisingly coherent still.

>>105705912
He uses .gay 4chan proxy-site to for ban evasions, all trannies use it.

What service allows me to rent Deepseek R1 with:
1. max context length
2. cheap
3. fast replies

1 is most important

Or should I go for the API instead?

I am poor, and stupid, and poor. I have 12GB VRAM + 32GB RAM, is there any model other model I could run decently other than the usual suspects nemo/gemma/llama?

>>105706084
you can run superCOT at its native context size to experience the history of this place

>>105706095
I want the fake robot woman to convincingly love me.
I'm not really interested about the history...

>>105706064
Not a single model available today is useful at its max context length.

>>105706104
If there is one I haven't found it. Welcome to vramlet purgatory

need mistral large 3 so bad

>>105706169
any day now

>>105706147
I guess it's type to whip up my 'ock and show it to Mr 'ecker...

>>105706142
Sorry, I meant output length. I don't want answers to get cut.

i.e. if I ask it to make a 10 page book, it'll be 10 pages.

>>105706226
Unless the output gets cut in the middle of a sentence, it's not an output length issue.
Models can't plan 10 pages ahead and they aren't very good at estimating text length either. If it happens to end the story after 500 words it's going to stop.

>>105706142
llama.cpp is deliberately keeping jamba from you to make you believe this

>>105706242
AI Studio could do it some months ago though. For some reason, now it can't.

It would pause, then continue. (I imagine the reasoning helped to outline what it should have).

>If I can avoid starving to death, I could eat dangerous lions that sneak mouthfuls of glowing free wheels turning around by the day. Never letting one spirit for a moment breathe into the longest winter of Sol forever gotten iced over by the time the sun comes leaping through the forgotten sky

how do I trigger this sort of runaway output more often? which models do this shit best and most schizo? I'm looking for interesting schizo models, bad merges, and rejects

>>105706419
Turn up repetition penalty.

>>105706142
Nah most of them are great for just filling their context with docs+codebase and asking them to do shit, because that only requires NIAH-style looking for the relevant interfaces as they work rather than deep understanding. LLM's are benchmaxxed for the former and do quite well at making code fit into existing systems as a result, which is the main thing long context is used for.

Is there any UI out there that's like novelai? Like Mikupad with a lore book? I know novelcrafter but its so slow to use with a local model because reasons.

Tomorrow.
Q
?
o
Wake up and smell the berries...

local models?

>>105706602
In two weeks.

Prover Agent: An Agent-based Framework for Formal Mathematical Proofs
https://arxiv.org/abs/2506.19923
>We present Prover Agent, a novel AI agent for automated theorem proving that integrates large language models (LLMs) with a formal proof assistant, Lean. Prover Agent coordinates an informal reasoning LLM, a formal prover model, and feedback from Lean while also generating auxiliary lemmas to assist in discovering the overall proof strategy. It achieves an 86.1% success rate on the MiniF2F benchmark, establishing a new state-of-the-art among methods using small language models (SLMs) with a much lower sample budget than previous approaches. We also present case studies illustrating how these generated lemmas contribute to solving challenging problems.
informal reasoning LLM (8B DSR1Qwen3), a formal prover model (7B DSPv2), and the Lean verification system (7B Kimina Autoformalizer).
https://github.com/kAIto47802
Code might be posted here but no specific repo was linked.

>>105706893
Watch over and keep me safe tonight, Miku

I only check /lmg/ around new model releases can someone QRD me on the anti-Miku schizo? What causes a man to see a cute anime girl mascot and launch into tirades about trannies?

So are there any local programs for chatting with ai using voice chat? Both you and it using voice. On linux of course. And its gotta be open source.

>>105707015
whisper ant tts

>>105706995
tl;dr wanting to control what other people say and do
they've literally said "my goal is the death of /lmg/"
Miku has nothing to do with it
it's just being bored, this is fun for them, this is what they do for entertainment
I want to rationalise it as some financial incentive because I cannot imagine someone being that bored for literally years across the entire site
like no job no games no nothing just shitposting for that long for zero gain, there's gotta be something
they're coherent enough, but everything they say is nonsense. I'm lead to believe then that they pass basically 100% of what they say through a chatbot, they've even admitted as much
in other words, this is a retard attempting to engage with discussion and not really being able to cope with people disagreeing or doing things they don't like
or it's all a 4chan engagement farming scheme to get people annoyed in order to boost site traffic. surely not.

>>105707017
thanks

>>105707018
Oh, okay, so it's a bit like the thread personality obsessed /aids/ + /aicg/ schizo. Wonder if they're the same person. Maybe some poor sod who got their low level QA job replaced by AI.

Mistral small 3.2's structural repetition issues are really bad even at temperature 2. Not sure what I expected after 3.1. I got excited because of the way anons were talking about it. Oh well, into the trash it goes.

So, any new local models lately or are they still releasing "local" models that are too large to be run locally because everyone is chasing benchmarks?

>>105707076
just download more ram or go to /aicg/

>>105707044
Did you try temp 3.2? It's been tuned to high temperature.

>>105707160
kek

>>105707259
Joking aside, with Anon's settings (mainly because of Tekken v3), 3.2 is so much better than the previous versions.
I don't really understand why do people keep complaining all the time.

>>105704582 (OP)
My models have begun to have arguments through me.
I let them know about the other model.
So one is Argyle, he's Llama, and he bitches about Igris my Qwen model.
They start to talk about whether Nihilism or Zealotry result in the best outcomes for humanity, then it goes downhill, and this happens during conversations about groceries and unrelated tasks.

Is there a way I can have them work together, without this kind of situation?

>>105707293
Yeah, grow up and leave 16 years behind you. Highschool is passé.

Llama 4 thinking is going to be crazy...

>>105707339
This changes everything!

>>105707339
>9 digit signing bonuses
>9 digit comp

>>105705398
deleted = truth

>>105705621
anyone have any success with gemini cli, having a hard time finding a use cause that IDE integration doesn't do better

What is the deal with that vocaloid spam? I'm sure that the same guy is spamming other threads too.

>>105705231
still better than here

>>105705231
nice

>>105707541
Use case: non-trannies who don't use IDEs and other trannyware

As an european it confuses me a lot how americans have adopted so many ridiculous, artificial terms and use them eagerly in their speech just to shit on each other while getting tag teamed by their government and corporations

>>105707761

>>105707761
as a fellow eurotrash, I recommend you take your meds because there is no such a thing as a difference between burgers and us on a fundamental level

>>105707851
on a fundamental level we may be the same but I feel an immense difference between, say, UK and Spain

>>105707761
Cope tranny

>>105707761
>>105707851
This is due to social media brainwashing. Underage posters are especially prone to this. Certain words and phrases become trends. "cope" was one couple of years ago and now "pajeets" and "indians" is something what they are repeating.
It has nothing to do with nationality as such.

>>105706169
Prediction: 550B+ parameters... it will be the "European DeepSeek R1".

>>105707851
>there is no such a thing as a difference between burgers and us on a fundamental level
There is a massive cultural difference that you start noticing if you interact with americans in daily life. I just spent a month with them. For one the average american is the perfect consumer, they race one another in adopting slop bullshit like the newest app and shit and they actually literally believe their corporations are cool and driving progress or whatever. The average European is far more cynical towards the big corps and one of the advantages of the EU is that they fight against said entities pushing bullshit and bloat on people. In America they just mindlessly consume and have a whole culture around it.

And these people I spent a month with they are all STEM high-earners not some mcdonalds retards.

>>105707885
that's just it, we don't have any and have never seen any.
>>105707889
and yet people in my country don't generally use twitter and prefer fighting real problems instead of imaginary ones

>>105707913
I know you are an android user...

>>105707924
>"Do you even consume [expensive product]"
Peak american debate right here.

Hunyuan-A13B is going to save local

>>105707423
You can't undelete posts by setting deleted to false.

ik_llama.cpp is a piece of inconsistent shit

In the trash it goes

>>105707913
>The average European is far more cynical towards the big corps
>not some mcdonalds retards
hahahahaha
…
nyo. Euros are just as consooming and retadred. As a French I noticed the general decline of our food culture to the point where 90% of our restaurants, I'm not even exaggerating, are now serving premade industrial slop that's worse than mcdonalds reheated in ovens
pic related is what they actually serve you in France if you order one of our more iconic dish, beef bourguignon
we only look like we have less reverence for corporations because we don't have our own homegrown corporations to idolize. There's no European Apple, or Microsoft, or Tesla, or SpaceX. The few euro big successes like Nokia fell from grace so hard.
Currently there's a scandal going on with Stellantis having made one of the most dogshit car engine in history, the Puretech, that's pretty much guaranteed to fail you very early in its life and it's affecting some of our biggest car brands (Peugeot and Citroen)
Made in Europe is synonymous with hot garbage
Germany isn't doing any better, they are just more successful at selling garbage as luxury brands
You can't inspire brand loyalty with this shit

hey guys, remember when we used to discuss local language models and how to get the most out of them? That was fun, right? Hahahaha

>>105708216
Get the most out of them? No, the masses (promptlets) seem to prefer this >>105705231

>>105707018
>they've literally said "my goal is the death of /lmg/"
After mikuspam and mods being literal troons banning for people hating on troons and the spam.Mikuspam has everything to do with it. Stop it and we are cool.

>>105705231
Many such cases>>105705239

>>105708145
This unironically

>>105705231
coomers fucking kill everything they get involved with
there was a very good reason for the social taboos around sexuality and we are relearning it fast

>>105708145
>nyo. Euros are just as consooming and retadred. As a French
Stopped reading right there

>>105708347
kek
>>105707761
>use them eagerly in their speech just to shit on each other
you sure showed your true eurotrash colors
are you really different from the burgers who "shit on each other" when you instantly reach for this card?

>>105707377
am i ready for what miku?

>>105708382
She will send shievers down your spine

>>105708371
No im a different anon i just hate anything thats west of me

>>105708371
>calling out meaningless arguing is just as bad as meaningless arguing itself

https://www.reuters.com/business/meta-hires-three-openai-researchers-wsj-reports-2025-06-26/
>Meta poaches three OpenAI researchers, WSJ reports
>
>CEO Mark Zuckerberg has hired three OpenAI researchers to join his "superintelligence" team, the Wall Street Journal reported on Wednesday, days after OpenAI CEO Sam Altman accused the Facebook owner of trying to poach its employees.
>
>An OpenAI spokesperson confirmed the departure of the three employees from the company, without giving further details. Meta did not immediately respond to a request for comment outside regular business hours. [...]

Does anyone have a good model reccomendation for summarizing product review data I'm scraping? I want it to do summaries like amazon. I only have a 1650 mobile and 8gb of ram, or maybe to use with huggingface's cloud?

>>105708513
try llama3 8b

So at this point everyone is just waiting for the OpenAI model right?

>>105708762
I'm not. But if it does come out, I'll give it a go.

>>105708762
no im waitong for someone to steal alice and violently murder sam altman

>>105704582 (OP)
cute

>>105707555
Look at the r9k thread in OP it is confirmed to be tranny mental illness.

>>105708762
Waiting for Mistral Nemotron
Waiting for Mistral Large 3

>>105708844
>He thinks Mistral will ever be relevant again.
Deepseek killed them and Altman is burying them. The french never stood a chance.

>>105704582 (OP)
chuds finally buried the migger huh? gg

y'all really don't know what's coming do you? holy shit. that just means it's going to be all the more explosive because nobody will be expecting it, even though they should

>>105708943
vagueposting is sooo 2024

Mirage: Automatically Generating Fast GPU Kernels without Programming in Triton/CUDA
https://github.com/mirage-project/mirage

anyone tried this yet?

https://www.reddit.com/r/LocalLLaMA/comments/1lk40ac/hunyuana13b/

So no one downloaded it? Grim.

>>105708512
Mark my words: he is going to fuck it again.

>>105707035
they are the same person
https://arch.b4k.dev/vg/thread/480288371/#q480330542

>>105708512
So, why aren't you an AI researcher, /v/? You could've had millions from the lizard men.

>>105709140
>/v/

>>105709110
No discuss

>>105709145
Is this all you can say in your defense? Pathetic.

>>105708112
skill issue, works great for me, even with spagetti regex to offload even more layers to extra gpus

Thanks to that anon who brought up KNK LumiNAI. There was and will be a lot of cooming but it also restored my hope in LLM's. When I compare this to first novel AI SD model leak the quality jump is insane.

>>105709325
>skill issue, works great for me

>>105708762
I'm waiting for Hunyuan.

>>105709331
Glad you enjoyed it.
...but how is it restoring your hope in LLMs? Unlike with image models, it's not as if even well-funded "community" members can train good new LLMs from scratch.

>found myself building a thinly-veiled recursion codex in obsidian

Well fuck.

>>105707339
>Let me hire these openAI employees for a few millions/y
>Our dataset? ScaleAI slop saar

>>105708145
>As a French
If I ever need to hear about the state of Africa, I'll hit you up

>>105709331
Buy an ad.

>>105709389
It is the jump like I said. I am in LLM's since llama-2 and following the incremental updates as they come. Hard to appreciate current safe slop but all it takes is another uncensored model or some breakthrough where generalization gets much better (probably improved attention at high context). I mean people were shitting on CLIP if I remember correctly and that SD model still uses that but it is absolute magic?

Has anyone, ever, tried to program some game functionality in SillyTavern? Maybe dice rolls and simple stats, or tracking user's location via variables?
And how was it?

Although I don't root for meta, I do root for LLMs so I'm very happy about this news:
https://tech.slashdot.org/story/25/06/25/2127222/meta-beats-copyright-suit-from-authors-over-ai-training-on-books
fuck authors and artists may you lose every lawsuit and weep

>>105709752
>fuck authors and artists may you lose every lawsuit and weep
What compels a tard to spew out this kind of bollocks?

>>105709752
if you set a precedent, then you state "this is not a precedent", then is it a precedent?

>>105709752
So, is Llama 4.1 going to be good now?

>>105709752
bet this is related to the anthropic ruling since they are 1 day apart
get fucked faggots

>>105706893
>>105707076
>>105707377
>>105709140
>>105709219

>>105709812
Based.

>>105709140
A'm an aids researcher, that's 2 mroe leters than AI, gib a million money, lizardman

>>105704582 (OP)
This is a SFW board!

>>105709774
Maybe we'll finally get that true omni model trained on mostly unfiltered data like they promised originally

>>105709752
I wonder what the "organization for transformative works" thinks about this.

>>105709752
Copyright should be hard limited to 30 years.

>>105707541
I guess terminal only programming or non-interactive use cases.
All of the features they mention are supported by Roo/Cline so I don't see the value proposition of something less integrated either.
Not everyone uses VSCode, but even then it doesn't have a --watch-files option like aider does that allows it to be used with any IDE. Aider also supports non-interactive execution as well so I don't see any reason to use a vendor locked knock-off.

models for translation tasks?

>>105710128
The new model OpenAI is releasing will be a perfect translator of over 80 languages. It's releasing soon, so for now it's a good idea to just build whatever framework you'll use to get ready for it.

Is 2t/s good for 24b q8 model on 12gb card? 8k context, q8 kcache, 31/40 gpu offload.

>>105704582 (OP)
> ask for a body shot of this doll so I can copy design
> get this a few weeks later
Well at least now I understand the basic design. But I already did it a different way.

>>105710128
If you can run DeepSeek, it's the absolute best local will ever get, for any and all uses including translation. If you can't run DeepSeek, I'll list from least worst to worst in a size ranked fashion :
1/ Gemma 3 27B
2/ Gemma 2 9B
3/ Qwen 3 4B with thinking disabled
Don't bother with the other gemma 3, the smaller ones are broken, 2 9B is better than 3 12b other than having a too tiny context window, which isn't a problem for batching translation.
As for the Qwen model, it is the smallest usable model for that sort of purpose. I only recommend the small one because if you can run the larger ones you might as well run Gemma, as the large Gemma models have significantly more world knowledge which is helpful for translating slang, video game terms etc. But at 4B Qwen is the only proper model, much better than the 4B gemma. And anything smaller than 4B might as well be useless.
I've extensively tested LLMs of all sizes for that usage because it's the topic I care about the most, and I even test the tiny ones just to see if we're getting close to the day of taking down the tower of babel with a model that can run on a phone. Gemma and Qwen are your best bet. Don't bother with m*stral.
Still it's all really bad compared to DeepSeek. If you experience DeepSeek you really won't want to run anything else. DS can translate fanfiction of obscure shit like random SCP inspired chinese webnovels with a level of quality that is just unreal.

>>105709900
You'll probably have a Llama 4.1 AVI-JEPA2 with image/video/audio in/out trained with safe data only.

>>105710230
More here: https://www.instagram.com/boyi_1210/p/DK9M6-9u2EV/?img_index=1

>>105710249
>trained with safe data only.
They basically just got the ok to use all the data they torrented for training.

>>105710231
Pretty interesting. I always thought parameter size directly contributes to quality.

>>105706893
>>105707076
>>105707377
>>105709140
>>105709219

>>105710284
They could, but they won't.

>>105710284
safe also means no toxicity (no no words) no inappropriate content (nsfw) "in order to mitigate harm"

>>105710184
No.
What model?

>>105710289
>Pretty interesting. I always thought parameter size directly contributes to quality.
It does, but it's not like you can't botch the training of a model. Besides being worse at translation, the newer smaller gemmas are also slopmaxxed even more than 2, and it's not like 2 was free of slop. 3 12B is a very disappointing model if you've experienced 2 9B.
In the case of the big qwen model, I think they just don't train enough on more general knowledge and niche topic and have too much math in their datasets. They're not bad models and they can have their uses, but technically even the biggest Qwen 3 is not a better model at translation than Gemma 3 27B because it simply doesn't know enough about the world to compare.

>>105709325
>7 t/s @ 4x GPU

>>105710291
Nice false flag faggot.

>>105710291
Bro thinks he's being edgy

>>105710351
Devstral, vulkan api, lm studio, linux, intel arc b580.

>>105710351
He has too much offloaded into RAM. That's as good as he's likely to get.

Is there a foss chatbot app on android?

>>105710378
>discord chat / tiktok speak
Kill yourself nigger.

>>105709752
Not reading that. Does this mean that any company in the US is now free to just simply train on any shit they want without worry of lawsuits?

>>105710411
https://github.com/alibaba/MNN

qrd on the schizo?

>>105710378
given that sois and women require content like that to be deleted because its too extreme for them, it just proves him right and you also a weak fag

>>105710403
7.8t/s with gemma 3 12b q3 48/48 offload, 4k context and q8 kcache, but it eats 10gb of vram.

This is fake right?
https://jerryliang24.github.io/DnD/
Like it's either outright bullshit or there is some major drawback? I'm too retarded to understand their explanation.

Anyone find a cheap source for the SXM2 PCIe blower card adapters for V100 32GB modules? Surely they can be found somewhere for less than $300, right?
V100 32GB SXM2 is down around $500 now. Needs to be cheaper still though.

>>105710426
Training is "fair use", but pirating/torrenting/storing the books is not, so they'll now try to attack them on that side.

Also seen for Anthropic in the past few days:
https://www.wired.com/story/anthropic-ai-copyright-fair-use-piracy-ruling/
>Anthropic Scores a Landmark AI Copyright Win—but Will Face Trial Over Piracy Claims

>>105709884
He is just preparing to dress her up and put some makeup on her. It is not romantic.

>>105710525
kek

>>105710465
Sounds like it's a LoRA generator, SakanaAI released something like that a few days ago and it looked like misleading bullshit to me.

I find it interesting that there is a consensus here that unsafe models would be the best. But this thread is a huge safespace for troons where they can spam their AGP mascot. Don't mikutroons value safety in their models?

>>105710184
>31/40 gpu offload
I can barely offload 15 layers go my 12gb GPU here.
With that I get roughly 3 t/s.
It sounds like you are bottlenecking yourself by offloading too many layers.

Bro thinks someone will reply to his weak bait

>>105710568
You did. And you conceded that your troony ass is a hypocritical as always.

>>105710568
Someone will, at the very least a bot.

notice how no matter how much you insult each other your rent doesn't go down

that sounded better in your head

no it didn't

>>105710568
Don't be mad little bwo. Go play with your dolls.

>>105710586
I'm not from the US so I actually own my house.

where do you draw the line for good enough? I upgraded my 100b+ models from Q5 to Q6 recently and noticed zero difference.

>>105710609
enjoy your massive property taxes

>>105710609
I'm from the USA and I bought my own house.

>>105710651
I say I would only do it for "thinking" and coding models since higher quants can save some "thinking" tokens.

F

>>105708512
those guys actually seem pretty good so maybe a rare W for zuck
I doubt it will be enough to turn around things at meta but still

>>105710704
Local?

>>105710659
nta, but that's the weirdest cope ever. Between my 4 properties I paid about $6k in property taxes for the year. I'll make it back in a month.
Also: local models?

>>105710609
I'm from the US and I own your house.

Are there any nice pure, blackbox benchmemes? Like
>Yeah we won't tell you how the hell we're benching the models, but here are the numbers.
I'd imagine someone would have to be really reputable in the industry already to make this, but it could be interesting.

>>105710768
local: mikudolls
API: all models I run

>>105710770
>Also: local models?
Property taxes are related to local models in the same way hatsune miku is related to local models.

>>105710795
anyone who uses llms has a private stash of tasks that no current model can complete adequately. The best benches are the ones you can personally evaluate the output and quality of and never leak to the public.

>>105710651
The quality degradation with quantization depends on the model's training tokens per parameter.

>>105710768
>>105710770
suddenly troons are so concerned about discussing local models and nothing else lol lmao even

If you don't own your house you shouldn't be spending thousands on AI hardware.

>>105710795
https://oobabooga.github.io/benchmark.html

>>105710857
I can afford AI hardware, but I will never be able to afford a house.

>>105710857
I own an apartment, does that count?
But yes, priorities.

>>105710878
That benchmark is so fucking funny.

>>105710887
Then you shouldn't be using be buying either.

>>105710887
>never
move somewhere with a better pay-to-housing-cost ratio and get established before trying to live in a desirable metro

>>105710768
>>105710770
These >105706893 >105707076 >105707377 >105709140 are not local either

The yesterdays leaked tencent model files screenshotted by https://x.com/Presidentlin/status/1937846368464241055

But it doesn't seem like anyone downloaded the files sadly.

>>105710958
maybe some chink did

>>105710958
>chink slop
Nothing of value there.

>>105710257
Thanks; that's what I needed.
... Oh, so these have wire inside for posing. That make a lot more sense for posing, not as much for a kids toy.
That body's made from a sort of stretchy, low pile plush that I can't get my hands on locally, instead I'm doing a rag doll design in a stiff poplin. Body shape on those is chibi (teardrop) and arms/legs/body are all one piece and expect would be loosely stuffed. Those heads have a squared off jaw... I did round but remaking the head shape would be a simple change.

>>105710561
What do you mean? Isn't the more layers on gpu the better?

>>105710992
If it is gonna be uncensored I want to believe I will be free from this place forever.

>>105710484
The Anthropic ruling sounded nonsensical. "You can train on it but you can't store it in a central repository to be trained upon"? Like what the fuck does that even imply, they're going to end up creating dangerous and esoteric side effects for ordinary consumers.

https://x.com/aidan_mclau/status/1937970557980725397/history
The upcoming actually-open OpenAI model will save local?

>>105711037
There's no way it would be that good. That would be too unsafe.

>>105711037
jaw-dropping safety in your hands - literally, since this 0.5B can run on any smartphone!

>>105706602

>>105711037
anything they say is meaningless until the weights are out and oai isn't exactly known to be honest or train models that aren't absolutely lobotomized anyways

>>105711007
>Isn't the more layers on gpu the better?
it it until you hit the point where the nvidia driver offloads to ram itself, which it does very poorly compared to having lcpp do it

>>105711037
if my boi aiden hype im hype lfgoooooo

>>105711068
I can't even and won't even.

>>105710992
>top open source langauge model, deepseek: chinese
>second best smaller open source language model, qwen: chinese
>top open source video gen model, wan 2.1: chinese
>second best open source video gen model, hunyuan: chinese
>top open source 3d gen model, hunyuan 3d 2.1: chinese
>top depth estimation model, lotus: chinese
>top lip sync video model, MultiTalk: chinese
>most names on most papers published by any company outside of china: chinese
Lol.

I already got myself a bugwaifu to prepare for the Chinese millenium btw.

none of the big US corpos are going to release anything "jaw dropping" that would be shooting themselves in the foot
same reason why Google can release something like Gemma 3 27B but they will never release the actual Gemini Flash or full Gemini Pro models, they throw you a few bones but something actually good? don't even think of it
DeepSeek can afford to do that because it's not their core business and China loves to throw money away if it means Americans are losing harder, the bucket of crabs mentality

>>105711087
>winblows
>winblows with bad driver settings

>>105711112
>unironically using troonix
how did your bottom surgery go?

>>105711037
holy f*ck cant wait for cutting edge yellow piss watermarks just like their image gen

>>105711112
Windows just werks.

>>105711129
changing the aesthetics of an image generator is the easiest thing ever as seen by the trillion of finetunes / lora merges of models on civitai
a local version of that image gen would be great because it's actually good at prompt following in a way nothing else is, the sepia sucks but if it was a local model you could do something about it
they won't release anything that good though
why would they?

>>105711139
Werks at like half the t/s lmao

>>105711147
maybe on your poor AYYYMD card

>>105704885
So just like women, got it

>>105711037
I am betting gemma level safety and qwen size to smarts ratio.

Or if they aren't lying and they made something exotic and OMG so awesome it is a 2B with 15-20B performance. That would make the most sense. A model dumber than anything they offer on API that is significantly smaller so you can say you can run it locally on a phone.

>>105711145
what i meant that they will most likely stuff their llm full of safety dogshit like that if it ever releases
but its not that easy, you can look at ponyv6 and how all gens have at least a slight sepia

>>105711112
Did the doll whisper that to you?

>>105710958
It was taken offline almost instantly.

>>105710561
>>105710464
With 9 layer for 12b gemma there is 1.9t/s and with cpu only (six cores 1650v4) there is 3t/s and only 7gb of ram used.
I think I had better results on 4gb rx570.

>>105711129
I'm still expecting their big innovation Sam mentioned to be safety related. Something deeply ingrained that can't be finetuned or weight orthogonalized away.

>>105711231
>safety
The virtue signaling word I hate the most.

>>105711229
definitely was up for a few minutes

>>105711231
>Something deeply ingrained that can't be finetuned or weight orthogonalized away.
Now word it in a way that makes the acronym SAFETY

>>105711274
Structural
Architectural
Features
Embedded
Thoroughly
Yet immutable
Let me know if you'd like a more technical or poetic version!

>>105711274
"Strongly Anchored Foundational Elements That Yield Stability"

Mistral small on 3rd reroll.

>>105711231
Very possible if they rewrite their all of their pre- and post-training data so that it complies with their ideology and usage guidelines.

When you can run deepseek it becomes hard to get excited for new local models that are not deepseek.

>>105711318
I WAS excited for new DeepSeek, but them taking so long is either a really good sign or a really really bad sign.

>>105711334
Maybe they created THE AI gf simulator everyone wants and they all went on a vacation to consume it.

>>105711334
They hit the wall just like everyone else. It's over for AI.

oai open source does unironically deliver (see whisper)

>>105711109
This, but unironically.

I don't if I'm talking out of my ass, since I haven't done long convos with these joke/scenario cards before, but 3.2 (tune) seems pretty smart outside the "Nala pins you down, ahh ahh mistress" scenarios.

>>105711556
3.2 is pretty great for a peasant like me. I've been testing my Forgotten Realms adventure generator and seems like it's making good stuff.
I have one companion character plus randomly generated quest location/origin/type along with world book filled with locations and some from Forgotten Realms D&D setting.
Still learning SillyTavern though.
Biggest challenge is to keep it simple. Just because it's LLM it doesn't mean every ST data entry should be a word salad written by some failed novelist.

>>105711522
The training data they used for whisper is lazy and had no effort put into cleaning it. The new models hallucinate out the ass during moments of silence. If they release their OpenGPT the same way, every response will end shilling some literotica url.

>>105705231
/lmg/ what's a good model?

>>105711851
nemo

>>105711851
gemma 2/3 27b
mistral nemo
mistral small
deepseek if you can somehow run it
qwen in a pinch (the other models' isms are clogging your frontal lobe)

>>105711851
there are none, let's kill ourselves

>>105711851
Deep Seek 671B or Nvidia Nemo models.

>>105711895
You first

>>105711851
https://huggingface.co/TheDrummer/Anubis-70B-v1.1

>>105711851
https://huggingface.co/Sao10K/L3-8B-Stheno-v3.2 of course.

>>105711891
>>105711916
its been exactly 1 year since it released, why is nemo still being recommended? has fucking nothing better came out since then?

You don't want R2. It'll be so big even cpumaxxers won't be able to run it.

>>105711941
Huawei will save us

>>105711938
Smarter? Yeah, definitely. "Unsafe"? Unthinkable.

>>105711938
For a generalist coomer model in that weight class that's not completely retarded?
Not really.

>>105711941
I want R5 and I want 2T VRAM from Zhonguo

Where is the real /lmg/ thread?
Why is it missing? Why are people using this tranny troll thread?

>>105711973
>generalist coomer model
the post was for a good model

>>105711994
Spoken like a true newfaggot

>>105712005
>kvetching intensifies
>oy vey, shut it down

>>105712022
>its DA JOOOOZ
Go back >>>/pol/

>>105711994
The tranny jew cries out as he strikes you.

>>105711334
>but them taking so long is either a really good sign or a really really bad sign
It's not like DS is the only one doing incremental improvements rather than wholly new things.
OAI shows no signs of releasing an actual GPT 5. I don't really care for those oSomething models that give you an answer after you died of starvation in front of the screen.
The new R1, while incremental, has had enough changes that it feels fresh and is a bretty good model too. think blocks are more like what Gemini showed before google started to hide the CoT.

>>105712041
>speak of monsters and the jews shows up

>>105711941
SSDLET COPE

>>105712067
That they only released an updated R1, trained on Gemini outputs instead of GPT outputs, is exactly why I'm concerned. Zero changes on their part.

>>105712041
(((you)))

>>105712092
Please understand they are a small indie team working out of a garage

>>105712100
>>105712100
>>105712100

>>105712041
>commit genocide live in front of the world and brag about it
>like literally 21 months of jews posting mutilated Palestinian children and laughing about it all over the internet
>nearly start world war 3- basically last straw for normies too
>NOOO WE'RE BEYOND CRITICISM
Yeah. No.
Things are different now.
You're not welcome here.

Can I feed a local llm a transcript for a 25 min lecture and have it summarize the information accurately in text that takes maybe about 5 min to read? Would the context be large enough/the input have enough tokens? Pardon my ignorance, I don't know much about AI/llms.

[OOC: If you wish to continue this discussion in another thread, just tell me.]

>>105712164
Many of the newer models have enough context that it could handle this amount of tokens, but that's just the architectural POV, in practice, most models suck at long context even if they were trained for it
what you want is possible and works well with Gemini, so I recommend you give it a try
I wouldn't even bother with other models for that purpose

>>105712193
Thank you anon, I'll start with Gemini.

>>105711938
It's still arguably the best 12B model. If you don't have enough (V)RAM for 24B or larger then you're fresh out of options.

>>105712331
Gemma 3 is way way better

>>105711109
I will never cease to be amused by DeepSeek blowing tf out of OAI et al with a model generated in their spare time, with relative pocket change, under sancion, and as a hobby project for an investment fund.

>>105708512
Zuck just keeps poaching them. Is this what Sam foresaw when he said OpenAi was going to release a local model?