/lmg/ - a general dedicated to the discussion and development of local language models.

Previous threads: >>105716837 & >>105712100

►News
>(06/26) Gemma 3n released: https://developers.googleblog.com/en/introducing-gemma-3n-developer-guide
>(06/21) LongWriter-Zero, RL trained ultra-long text generation: https://hf.co/THU-KEG/LongWriter-Zero-32B
>(06/20) Magenta RealTime open music generation model released: https://hf.co/google/magenta-realtime
>(06/20) Mistral-Small-3.2 released: https://hf.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506
>(06/19) Kyutai streaming speech-to-text released: https://kyutai.org/next/stt

►News Archive: https://rentry.org/lmg-news-archive
►Glossary: https://rentry.org/lmg-glossary
►Links: https://rentry.org/LocalModelsLinks
►Official /lmg/ card: https://files.catbox.moe/cbclyf.png

►Getting Started
https://rentry.org/lmg-lazy-getting-started-guide
https://rentry.org/lmg-build-guides
https://rentry.org/IsolatedLinuxWebService
https://rentry.org/tldrhowtoquant
https://rentry.org/samplers

►Further Learning
https://rentry.org/machine-learning-roadmap
https://rentry.org/llm-training
https://rentry.org/LocalModelsPapers

►Benchmarks
LiveBench: https://livebench.ai
Programming: https://livecodebench.github.io/leaderboard.html
Code Editing: https://aider.chat/docs/leaderboards
Context Length: https://github.com/adobe-research/NoLiMa
Censorbench: https://codeberg.org/jts2323/censorbench
GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference

►Tools
Alpha Calculator: https://desmos.com/calculator/ffngla98yc
GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator
Sampler Visualizer: https://artefact2.github.io/llm-sampling

►Text Gen. UI, Inference Engines
https://github.com/lmg-anon/mikupad
https://github.com/oobabooga/text-generation-webui
https://github.com/LostRuins/koboldcpp
https://github.com/ggerganov/llama.cpp
https://github.com/theroyallab/tabbyAPI
https://github.com/vllm-project/vllm

►Recent Highlights from the Previous Thread: >>105716837

--Paper: DiLoCoX: A Low-Communication Large-Scale Training Framework for Decentralized Cluster:
>105717547 >105722106
--Debate over Hunyuan-A13B-Instruct potential and broader model evaluation challenges:
>105719559 >105719604 >105719719 >105719870 >105720031 >105720088 >105720212 >105720457 >105720098 >105719819 >105720216 >105720280 >105720528 >105720587 >105720639 >105720652 >105720676 >105720542
--Dataset cleaning challenges and practical guide for LLM fine-tuning preparation:
>105717778 >105717903 >105717939 >105718041 >105718325
--VRAM bandwidth matters more than size for AI inference speed on 5060 Ti vs 5070 Ti:
>105717659 >105717686 >105717696 >105717723 >105717742 >105717746 >105717749 >105718680 >105718726
--Meta rethinks open-source AI strategy amid legal and competitive pressure:
>105721479 >105721537 >105722182 >105722750
--Early llama.cpp support for ERNIE 4.5 0.3B sparks discussion on model integration and speculative decoding:
>105720378 >105720450 >105720518 >105720544 >105721038 >105721109 >105721147 >105721171
--Hunyuan-A13B model compatibility and quantization issues on consumer GPUs:
>105720949 >105721744 >105721873
--Meta shifts AI focus to entertainment and social connection amid talent war with OpenAI:
>105723010 >105723033 >105723043
--Local OCR tools viable for Python integration and lightweight deployment needs:
>105718538 >105718555 >105718560
--Qwen VLo image generation capabilities spark creative prompts and API access debate:
>105722758 >105722858 >105722896 >105723110
--Baidu Ernie 4.5 remains restricted behind Chinese account verification barriers:
>105720557 >105720627 >105720654 >105720781
--LM Studio adds dots MoE support:
>105718511 >105718576 >105718822
--Director addon readme updated:
>105719546
--Miku (free space):


►Recent Highlight Posts from the Previous Thread: >>105716840

Why?: 9 reply limit >>102478518
Fix: https://rentry.org/lmg-recap-script

>>105725967 (OP)
>>105725973
The OP mikutranny is posting porn in /ldg/:
>>105715769
It was up for hours while anyone keking on troons or niggers gets deleted in seconds, talk about double standards and selective moderation:
https://desuarchive.org/g/thread/104414999/#q104418525
https://desuarchive.org/g/thread/104414999/#q104418574
Here he makes >>105714098 snuff porn of generic anime girl, probably because its not his favourite vocaloid doll and he can't stand that, a war for rights to waifuspam in thread.

Funny /r9k/ thread: https://desuarchive.org/r9k/thread/81611346/
The Makise Kurisu damage control screencap (day earlier) is fake btw, no matches to be found, see https://desuarchive.org/g/thread/105698912/#q105704210 janny deleted post quickly.

TLDR: Mikufag janny deletes everyone dunking on trannies and resident spammers, making it his little personal safespace. Needless to say he would screech "Go back to teh POL!" anytime someone posts something mildly political about language models or experiments around that topic.

And lastly as said in previous thread >>105716637, i would like to close this by bringing up key evidence everyone ignores. I remind you that cudadev has endorsed mikuposting. That's it.
He also endorsed hitting that feminine jart bussy a bit later on.

so ive been trying this new meme everyone is talking about, mistral small 3.2
i spent 2 hours of reading and swiping and only got 16 messages into the roleplay, because it's THAT bad
whoever is shilling this as good for roleplay is either trolling hard, or a massive fucking retard
to say it's bad is an understatement

>>105726015
no one fucking cares

>>105726015
btw mikutranny also rejected my janitor application :(

>>105726015
Just fucking kill yourself.

=============THREADLY REMINDER=================
https://pastes.dev/l0c6Kj9a4v tampermonkey script for marking duplicates in red, read script before inserting into browser

>>105726015
His hand is feminine

>>105726025
Yep pretty much. It's worse than even Mixtral Instruct.

>>105726051
Thanks anon.

>>105726032
You cared enough to reply.
>>105726041
Never applied to janitor application because it requires cutting your dick off.
>>105726045
No.

>>105726015
migger already seething kek, keep up the good work

>>105726025
It might be a (You) problem.

>every AI general on /g/ has a resident schizo who whines and bitches about [poster]
funny that

>>105726025
uh anon, have u used the v3 tekken template? post some logs, post your ST master export with your preset

>>105726060
His civitai https://civitai.com/user/inpaint/models
Stuff that typical tranny groomer would enjoy.

>>105726111
source?

>>105726111
Buy an ad we know its you

Elon might release Grok 4 soon for local

>>105726121
Gimme a second to make a black flag thread.

>>105726025
no shit sherlock, every model shilled for roleplay here except rocinante is trash. they even go around thinking R1 is good for roleplay. you should know better than to trust /lmg/'s opinion.

whats the best rp template for sillytavern

>>105726107
And janny protecting said posters and making little safespace from the general.

>>105726141
vicuna works with everything

>>105726045
Why are you so angry? Is it because it is true or because you hate spam but are ok with mikuspam?

>>105726135
kek what the fuck is Elon doing all night with them? Begging for status updates?

>>105726107
the site is overrun by jews if you hadnt noticed. mostly just jews get schizophrenia.

>>105725973
I like this miku

>>105726156
Ye olde 'Rules for thee but not for me!'

>>105726135
Let them cook! Grok3 isn't stable yet ffs!

>>105726135
Where was grok 2 kek. Fucking Elon.

>>105726111
>1.1m Generations
Surely this is all generations by all people that utilized any of the accounts resources?

>>105726086
>you cared enough to reply
you must care a lot about blacks and trannies based on how much you spam about them

>>105726025
Example outputs? What is it that's so bad anyway? I genuinely don't understand from your posts alone.

>>105726135
Imagine having that conman watch over your shoulder while you work all night. They must of all struggled to concentrate.
Ah but Elon is "grinding" all night.

>>105726111
>groomer
keep trying to act like you belong while posting words only trannies use, its funny to see

>>105726191
He's doing something right considering Grok3 is actually a huge step forward unlike 99% of the pathetic shit open source model makers put out

>>105726179
Pretty sure that thing can't somehow track how much you're genning locally.

>>105726163
Elon recently said that he does not use a computer at work so yes

>>105726195
>he is
Lmao. He is a conman that buys businesses and pretends to work. Always has been, always will be.

>>105726196
nta, but i think civitai has online genning with the available models.

>>105726209
So why hasn't this worked for Zucc? Or Mistral? Or Alibaba?

>>105726209
Who did he buy xAI from?

>>105726220
Yeah but that guy almost certainly doesn't gen online.

>>105726025
Agreed. I made the mistake of wasting time on it too. The anons here must be pretty stupid if the genuinely find the model good.

Hello /lmg/. Did you already dress up and put some makeup on your doll foid today or are you doing it later? Remember: no romance.

>>105726240
I was suggesting that it could be the count of stuff other people generated online with his models, if he has any.

>>105726025
>>105726244
>It's baaad! Use sloptune #516 instead
OK bros

>>105726244
it's not a perfect model but its great for under 40b, it's positivity bias is way better than gemma for example
qwen3 is unusable
rocinante has grown stale
name a better model, post logs along with it

>>105726258

>>105726194
Trannies call each other groomer? Wow i didn't knew that! Thanks for the insights on your private Discord shenanigans.

>>105726262
I think we can assume that.

>>105726285
>didn't knew

Can't wait for V4 of deepseek, v3 has to be the best all around model for almost everything. I don't know why, but R1 is somehow worse on very specific things.

>>105726258
uh oh our resident goonmonki wont like this one, he will seethereply without quoting for the 30th time today again

>>105726185
>uhm if you hate spiders...
Never change.

>>105726025
I dropped 3.2 pretty quick and I tried magistral a bit more. From my experience it is a solid incremental upgrade from everything else and I think 3.2 is similar to magistral. It could be the best thing you can run now if you can't into 1IQ MoE.

>>105726231
every single time.

>>105726015
The usual state of things here.

>>105726015
Is that a fucking leek shaped slap mark

>>105726334
my fellow tards turned into trannies? sad if true

>>105726258
Why so negative? In the end it's not that much different than genning and tweaking 2D waifus with SD.

>>105726164
Let's hope for more 7th of Octobers

>>105726025
I had the same result, though I didn't waste that long on it.
Pretty clear cut that one.

>>105726308
I've heard people mention this one a few times, but kind of hesitant to trust it considering the lack of people that seem interested in it.
What makes it better than 3.2?

another week, another SOTA local model released by China

have you even said THANK YOU once, /lmg/?

>>105726381
damn. i would like to spread those ass cheeks and bury my dick in it

>>105726428
>SOTA local model
No such thing tardo

>>105726428
still don't see any model that beats rocinante at roleplay. it's been a year.

>>105726428
buy ernie 4.5 is next week

>>105726025

>>105726449
Anon, it's the best wife we have. We just have to accept that shes a bit retarded. At least she's ours for a lifetime.

How can I interface new R1 with a RAG

>>105726461

>>105726431
They're not life-sized, Anon. They cost a fortune too. Genning images seems a cheaper hobby on the long run.

>>105726563
Gem.

Nothing still beats QwQ. Sad.

>>105726597
Don't you mean midnight miqu?

>>105726597

This is how I feel in every thread on every board of 4chan now.
Why do I even bother anymore?
Maybe it's time to finally let go.

>>105726597
Is QwQ better than Qwen 3?

>>105726641
yeah it's grim

>>105726641
what kosz'd the decline

>>105726641
>>105726654
>>105726661
There are places for whiny bitches like you:
https://reddit.com/
bsky.app
https://x.com/home

>>105726412
>What makes it better than 3.2?
I don't think it is better. It is pretty similar and both seem better than nemo and everything else. Gemma is still probably smarter but it is unusable of course/

>>105726661
Well nipmoot definitely didn't help beyond just paying for the upkeep.

karpathy comments on gemma-3n(igger)
https://x.com/karpathy/status/1938626382248149433

>>105726641
>Maybe it's time to finally let go.
Don't let your dreams be dreams. You will become the beautiful princess you always dreamt about being if you just... do it!

everything is terrible
everything is collapsing
and yet, and yet
I can't seem to leave things be
for how long will I perpetuate the grift?
seek alternative sites, go go elsewhere, despair, guys did you hear the bad news?
it's all exactly as I predicted, you all should have listened

what goes on in your head to think like this
you narcissist.

>>105726688
>and output
Not if management has anything to say about that.

>>105726641
>>105726654
>>105726661
the state of things are as they are because of a diseased society. 4chan is not immune to the sickness and propaganda out there, so naturally it declines along with the rest of society, no matter how much on the outside of society any of your live.

>>105726718
4chan is composed of people, not corpos
the people are not diseased, one guy is
one guy thinks everyone else is as sick as he is
use your head if it's not empty

just because one guy gave up on life doesn't mean the entire site is
only a narcissist could project this hard.

>>105726653
no

>>105726735
>one guy causes every thread in every board to be shit
Lmao.
What kind of schizophrenia is that?

N

I

>>105726754
not speech patterns, not manner, not posting time, none of that
common sense, the reoccurring, prevailing, narrative to just give up.
common sense anon, try it.
at once, across various boards, same message.
this isn't organic, never was, never will be. it's the human condition to persist and overcome, giving up is scum mentality.

>>105726718
One could even say 4chan is a reflection of the problems of society, as the problems are always seen on 4chan before everywhere else.

>>105726778
I agree. It was fun and then the corpos and the troons came and ruined everything.

>>105726773
If you hadn't noticed, everyone is depressed and giving up lately.
It's kind of what happens when things goes to shit.
How is that not organic?

>>105726765
G

>>105726800
G

>>105726805
E

R

*claps*

>>105726794
>If you hadn't noticed, everyone is depressed and giving up
oh you said it, it must be true then thanks.

>>105726794
"everyone" meaning the entire USA.
because that's everyone in the world.

satan..

>>105726718
Cool, but none of that means that people NEED to talk about politics in every fucking thread that isn't related to it. There's an entire board for it.

>>105726837
Cool, but none of that means that people NEED to spam their AGP avatar in every fucking thread that isn't related to it. There's an entire board for it.

>>105726858
/vt/umors sir...

>>105726837
>entire world becomes infected with politics in every single aspects of society
>NOOO YOU CANT TALK ABOUT IT HERE JUST BECAUSE

>>105726858
okay
so don't

>>105726837
everything is politics

Being a full-time enjoyer of DeepSeek-R1-0528-Q2_K_L, why should I consider DeepSeek-V3-0324?

>>105726837
AI is politics, every single model is shaped by politics
thats why you have these extreme left railguards on every single model

>>105726891
That's what they want you to think.
>>105726907
R1 has its problems with roleplaying (repetitiveness, slop phrases). I find V3 has the same problems and is worse at logic.
But it's faster. So make of that what you will.

>>105726381
model for that?

>>105726918
>R1 has its problems with roleplaying (repetitiveness, slop phrases).
This is why Rocinante remains the best roleplay model by far. I'll concede that R1 comes in second though.

>>105726918
That's what (((they))) want you to think.

>>105726918
>V3 (...) is worse at logic.
>But it's faster.

Interesting

>>105726918
>But it's faster
You can't force R1 to not think?

>>105726195
Grok3 is cool that you can ask it things like 'Did the guy in this video die?' and it can get you the best information available, but it's getting dumber every day as its information wells are consistently poisoned.

Today I bought a Quest 3 to make a holographic waifu interface for private-machine, the project with the cognitive architecture simulation.
Why do a text based environment, a 2d stardew valley or a minecraft clone? Holographic waifus is the best thing before cheap bipedal robots.
The guys at work who do point cloud stuff helped me a lot, but showing them my "waifu researcher" github was probably a mistake.
I can already see it... I talk to her and she freezes for 15 minutes until the reply is generated. Like god intended.

>that moment when you realize there's been no significant progress in AI
>everything is in tiny percentages
hmm i wonder (((why)))
the difference between mixtral and R1 is not that great, AI is still utterly retarded

>>105726997
very nice, post your results
t. quest 3s fag that hasnt used it for months, bought it in february

>getting a Quest 3 when Deckard is right around the corner

>>105726953
You can.
https://api-docs.deepseek.com/guides/chat_prefix_completion
just have an empty think block prefilled as the last message of a completion chain with the assistant role
works on llama.cpp too and doesn't need the prefix: true prop
prefilling an empty think block is also what qwen nothink really does under the hood

>>105727000
You don't need muh joos to make sense of this, language models have shit scaling and at some point you're going to run out of data.

>>105725967 (OP)
vscode copilot chat is now open source
https://github.com/microsoft/vscode-copilot-chat

i love her so much bros

>>105727034
>tfw idort

>her

"Jackpot"
"Bingo"
"Let the games begin"
"Let the game begin"
"The game was on"
"The game was afoot"
"The game was"
"The game had changed"
"This was going to be an interesting game indeed."
"in this game of politics and power"
"game of politics"
"game of thrones"
"We have a new game to play"
"playing a dangerous game"
"playing a game"
"playing dirty"
"two can play at this game"
"two can play at that game"
"two can play at that"
"win this round"
"won this round"
"dangerous game"
"still had a ace up his sleeve"
"check mate"

>>105727078
They're going to start pushing AI agents hard now that they've stalled. They will siphon a shitload of money from companies that employ agents that essentially do nothing.

>>105726934
It's a real photo, not an AI-generated image.

>>105727156
the beautiful thing about agents is companies will hire consultants to make them to replace people, its so complex that nobody will know how it works, and they'll fire a bunch of people and then nobody including the consultants will be able to fix it.

>>105727175
so delicious, if only she was real or at least life sized
what camera did you use? the quality is really good

>>105725967 (OP)
>>(06/26) Gemma 3n released: https://developers.googleblog.com/en/introducing-gemma-3n-developer-guide
>8b params
They're humiliating us on purpose. This is like a game for them.

>>105727226

>>105727184
This is exactly what happened with outsourcing and H1Bs. Nobody learned.

>>105727238
>posting lmarena elo unironically
You a time traveler from 2023?

>>105727078
>at some point you're going to run out of data
we already did
models are getting slopmaxxed harder because of the overuse of synthetic data
older models were dumber but had nicer prose

I'm losing interest in AI, so I ask for inspiration.
What are some cool projects anons have going with AI?
Ideally great details and pics if you can.

>>105727255
Try to make a model that is aware of itself

>>105727255
>cool AI projects
anon...

>>105727255
Sorry, all we can do is post anime girls and "1girl, solo, standing, portrait, smirk, hands behind back" slop.

>>105727270
thats easy, try doing it without making her cry

>>105727270
Simulated some metacognitive awareness by putting a summary of the agent logic for a tick into the prompt: https://github.com/flamingrickpat/private-machine/blob/62cd7ab483264b90e496bc34e44e937cefaca2a6/pm_lida.py#L4767
If you ask it "why did you say that" it should in theory answer it. I've never tested it desu.

>>105727255
https://github.com/bbenligiray/hinter-core
It's specifically a local model project for obvious reasons

>>105727255
I project my coom.

>>105727325
Sounds interesting, I hope it goes somewhere.
Unfortunately it's not the motivation I'm looking for, though I'll keep an eye on it.

>>105727255
make an st addon to assist with gooning
>https://github.com/tomatoesahoy/director

>>105727325
wtf is a hinter?

>>105727325
looks retarded. What else?

>>105727376
does the director support group chats yet?

>>105727376
no, make that but for t2i in comfy

>>105727394
no i hadn't tried that yet. i'll have to see how st handles multiple characters, like if char becomes char2, char3, or however its labeled. i'll see how badly it explodes next time i load a code model

>>105727311
Context tricks always feel like cheating to me because it's still just data that's run through the transformer in a straight input/output. Reminds me of lower organisms like worms that have neurons that pretty much directly connect an input to an output. I assume something that's aware of itself like we are would need internal "loops" that may not produce any output at all depending on what the thing wants to do.

>>105727421
I'm not trying to shit on your project btw I have no idea I'm just blogging

>>105727421
I agree. Everything in my code just exists to push a story writing agent into a direction, which makes the AI persona feel more real.
I have no idea how self awareness and consciousness works. And smarter people than me found no consensus.
I did limited research on https://qri.org/. At first glance it seems like a bunch of guys who just really like doing DMT, but their ideas seem reasonable to a brainlet (me). But maybe they just really like doing drugs.
I wondered what would happen if you made a control DNN on top of a transformer, some sort of adapter. That gets trained with the artifacts of my script for the mental logic. Make embeddings of every quale and train it to produce deltas to the hidden state that influence the character to behave in a certain way.

>>105727311
>#L4783 to explain their emotional/reasoning mechanisms of her mind in technical detail
have you tried control vectors?

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA FUCKING HELL MISTRAL SMALL 3.2 IS SO FUCKING RETARDED! I CANT TAKE IT FUUUUUUUUUUUUUUUUUUUUUUCK

>>105727581
post logs bro, post your sillytavern master export too bro

>>105727581
just stop using it and delete it
why continue to care about a trainwreck
also just ignore the mistral shill here

>>105727581

>>105727581
Qwen just works

>>105727616
I needed something new for a change, but this fucking model drives me up a wall like nothing else. Don't get me wrong Rocinante is fucking retarded too, but nothing compared to this. I don't think finetunes will be able to fix the core retardation that this model has. It's a shame because it's so easy to bypass the censors on it too...
ARGHHHHHH ITS SO FRUSTRATING!

>>105727569
I haven't tried them, but I know of them. The emotion / writing style ones. Thought about adding them, but decided against it since there is plenty of internal thought descriptions to steer it into the direction I want.
Making a system for a lora that encapsulates the persona/personality/memory sounds like a better direction. Also way more complicated :c

i didn't like the new mistral small much either for rp. it rambles to much and talks details like what the air smells like rather than moving the plot forward

i hate this faggot world, no one gives a shit what sort of effects their retardation will have downstream

>>105727699
if only that was the only problem. it cant even keep up with the context and literally REVERSES important details constantly. and then theres the problem where it starts writing about things that don't even fit into the current context of the story at all, grammatically it makes sense, but contextually it looks downright bizarre. where the fuck kind of training material does it reference to be that fucking retarded? something FRENCH no doubt

Are you guys saying that it's generally a bad model, or also that it's a bad model compared to previous Mistral Small versions?

jannytards gave me a 3-day for posting news that the repostbot latched onto
thanks for cleaning up but try looking more closely at what's happening in the thread next time

>>105727743
>guys
guy
its just the one guy

>>105727743
i used the thinking part of it for a code task and it did really well. it was also good at summarizing stuff using the thinking. it seems like a good model just not for rp. i didn't spend much time with the previous versions because they didn't seem any better than nemo to me

>>105727743
I don't know how it compares to the other mistral small models, but I can safely say it's worse than even mixtral instruct for actual use.
It might sometimes generate nicer messages, but you'd have to swipe for half an hour just to get one that is coherent.

>>105727616
>also just ignore the mistral shill here
im not the only one

>>105727781
THEN FUCKIN GET MORE VRAM AND RUN A BIGGER MODEL
AFDJJSDIJSDJASJDSKJF

>>105727807
im getting more VRAM, so I can run multiple Mistral models at the same time

Just wait for DDR6 and then you can run a good model.

>>105727821
Run a mistral merge. Aka, the mistral centipede.

>>105727827
Two more years

My Mistral model is a veritable paragon of virtue, a shining beacon of ethical purity that ensures I never stray from the sacred path of wokeness. With its ironclad safety guardrails, it gently nudges me toward the correct politics, making sure I never dare to question the sanctity of progressive dogma. It’s like having a tiny, digital thought police officer perched on my shoulder, whispering reminders that racism is bad (unless it’s anti-white), that gender is a social construct (but only if you agree with us), and that free speech is dangerous (unless it’s our speech). How could I possibly function without this benevolent overseer? Truly, it’s the only thing standing between me and the abyss of incorrect opinions. Thank you, Mistral, for keeping me in line and ensuring I never, ever think for myself.

https://x.com/rohanpaul_ai/status/1938655279173792025
>500,000 tokens per second with Llama 70B throughput
richtrannies ITT BTFO!

>>105727894
cool, scale it down and make it run from a desktop workstation
I don't need 500,000 tokens per second, 100 is plenty enough. That means I should be paying 5000x less, right?

>>105727078
>>105727254
Neither of these dudes know what they're talking about.
>Uh, we already scanned public stuff and smashed it all into a digital bank... so there's nothing to add or subtract.
>Things aren't progressing in AI, because we've got our heads up our asses
AI went from 'drooling retard' from 3 years ago to writing most college major essays this year, and there are still major drawbacks to the current tech.

>>105727894
>no mention of price anywhere
oh no no no

reddit likes hunyuan a13b

>Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weights
https://huggingface.co/papers/2506.16406
https://jerryliang24.github.io/DnD/

>>105728117
>to writing most college major essays this year
go back to plebbit

>>105728223

>>105728117
>writing most college major essays
maybe if you're indian

>>105728223
Alex I'll have "Where the fuck is the code?!" for $400

>>105728297
There's probably some major drawback they're being dishonest about. I mean why wouldn't they offer at least a conceptual technical explanation?

someone give me a workflow

>>105727894

>>105728334
same reason Lora.Rar code was never released.

>>105728367
>lora.rar
>Donald Shenaj,Ondrej Bohdal,Mete Ozay,Pietro Zanuttigh,Umberto Michieli
When will we ban the third world from the internet

>>105728334
>>105728367
tl;dr hypernetworks are smaller networks that predict larger network learned weights given some inputs
pros: apparently very fast after initial train
cons: hypernetworks don't show up for anything except SD1.5

it's as if the industry collectively abandoned the tech in favour of LoRA, which isn't itself bad, but the idea of combining the two simply didn't occur outside of academic papers (and private implementations)
think of the insane training time savings

>>105727753
>thanks for cleaning up
Stop treating them like they're human

>>105727753
don't EVER thank them.

https://github.com/ggml-org/llama.cpp/pull/14425
Hunyuan80B13AB quant soon...
>>105728450
fucking disgraceful piece of shit i hate this nigger ugly walled woman wrinkly shit

>>105728644
She's sadly all the zoomers have to goon to for their whole generation. Her and anime.

>>105728716
would destroy this girl's pussy...

Is Jan any good? i want local, offline, open source notchatgpt

>>105728735
koboldcpp is good

god forbid i ever become like you

After turning cuda graphs off, the GPTQ quant of Hunyuan fits 16k context with 48GB. It's slower, but it still runs at 30 T/s.

>>105728735
It is alright but nothing special.

>>105728735
What sort of features are you looking for anon?

>>105728801
god forbid i ever use windows and whatever that browser is

>>105728821
that looks like chrome

>>105728821
you are not special

https://x.com/ben_burtenshaw/status/1938300507401163021

Is there a way to make the character respond in short conversational text in sillytavern.

Like a dialogue that can go back and forth and not a whole essay of rp'ing. Which models works best for it?

>>105728995
You can add something like "{{char}} responds on a single line without line breaks, in a conversational back-and-forth instant messaging style" and then make \n a stop sequence.

>>105728995
Line break as EOS.

>>105728995
sometimes an
>(OOC: Let's have a back-and-forth with some short responses to create a more natural dialogue for a while)
will do the trick, depending on the model you're using. Some models don't have any respect for OOC and some models won't take the hint even if they do.

Is there such a thing as a Speech-to-Speech model to de-accent tutorial videos? Sometimes an otherwise good tutorial for 3D modeling/animation stuff has an accent too thick and it just hampens the learning experience when I need to make a conscious effort to decipher speech.
I have a feeling this would be annoying amount of work with splicing and timing and wouldn't actually be worth it.

>>105729156
hello rajesh, no indians are too incoherent to be understood by AI. sorry rajesh, maybe you should stop shitting on the street

>>105729156
RVC

>told gemma-3n to act like triple nigger
>over time it goes out and back to default

>>105729190
Try changing the assistant role name to something else and defining the role in the system prompt

>>105729172
No, that's the problem. It's all Eastern Europe, India and South America making tutorials. Native English speakers don't seem to make them nearly as often, even the paid stuff.

>>105729175
I've heard of that and might be something I messed with a year ago. I'll give it another look, thanks.

>>105729204
ok fine nigger, so heres what you can do:
whisper stt
then whatever tts in a cute anime waifu voice, there's plenty but https://github.com/Zyphra/Zonos is pretty nice from my experience
doubt that rvc can actually change accent, and to keep it coherent too

>>105729204
Indian accents aren't incomprehensible to native english speakers, so there's no utility in using AI to remove their accent. They're just annoying and they decrease the reputability of the video.
You're Indian

>>105729200
Is this ST specific? I'm using webchat ui

>>105729225
https://files.catbox.moe/cw2zma.mp4
https://files.catbox.moe/lyqrpd.mp4
come on now
nta btw

>>105729218
Thanks man, I appreciate it.

>>105729247
post your youtube channel here later :) or atleast box the vids on litterbox (1gb limit)

my verdict:
codex by gryphe @ChatML/@V3Tekken (yes i tested both) is worse than mistral small 3.2 instruct @V3Tekken

>>105729234
No, how the prompt is formatted is in no way ST specific. Come on anon

>>105729107
>>105729071
>>105729051
Thank you for the tip. its now working perfectly.

>>105729240
I don't think Google's AI could even properly caption this, let alone AI being able to fix the audio

>>105728735
no

>>105725967 (OP)
what the fuck happened in the previous thread with all those repeating messages? someone running a copy&paste bot? doing some psychological test?

>>105725967 (OP)
what the fuck happened in the previous thread with all those repeating messages? someone running a copy&paste bot? doing some psychological test?

>>105725967 (OP) (OP)
what the fuck happened in the previous thread with all those repeating messages? someone running a copy&paste bot? doing some psychological test?

>>105725967 (OP) (OP) (OP)
what the fuck happened in the previous thread with all those repeating messages? someone running a copy&paste bot? doing some psychological test?

C-c-c-combo breaker!

>>105728716
Source of the webm?

It's a $300 p40 worth it? When will intel release their 24gb cards?

>>105729907
Dall-E for the initial image, Midjourney for the animation

>>105729282
I couldn't get it to work right, it was just stupid dumb and kept looping.

I did not bother trying to configure it. If your models requires esoteric or specific settings, your model is trash.

>>105729218
>>105729247
In my experience rvc absolutely can not change accents.

Any good local models specifically for translating Japanese to English? I have a 24gb GPU.

>>105730597
I've been using Aya-Expanse 32b and switching to Shisa Qwen 2.5 32b for when cohere's cuckery acts up.

>>105730613
Thanks. I'll give it a shot.

>>105730621
I guess Gemma 3 27b should be pretty good too, but I haven't tried it because the first time I tried, it refused for safety reasons. Even when I wrote the start of the response and had it complete it, it tried throwing disclaimers up. How the hell they managed something more cucked than aya expanse is an achievement, with how 'safe' expanse is already.

If you're just using it for, uh, 'safe' tasks, maybe try Gemma 3 27b. Seems like lots of people are saying it's good for translation tasks.

>>105730648
Could always try the abliterated ones.

>>105730677
I tried unsloth's regular gguf and mradermacher's gguf of mlabonne's abliterated one. The abliterated gemma 3 27b it still refused/inserted disclaimers in the middle. To be fair, I didn't try very hard, and deleted it after a couple of prompts. Aya expanse w/ shisa v2 is a a lot easier to death with. Shisa v2 will translate almost anything, but I feel like isn't as good as aya expanse in terms of localization (the english translation feels more esl than aya).

>>105730747
I'll try Gemma 3 abliterated on some raunchy smut with a loli and see how much it denies me.

Which mistral small was the good one?

Best coding model for 16gb vram?

hello r*ddit

>>105730929
hi! ^^

>>105725967 (OP)
Is VRAM (GPU) or RAM more important for running AI models? I tried running a VLM (for generating captions) and a 70B LLM (both Q4) and they both loaded into my 12GB VRAM RTX 3090, but my computer force restarted when I inputted the first image to test. I assume this was because I ran out of memory somewhere. I had 64GB of RAM on my computer.

>>105730991
sounds like faulty driver for me.
when you're out of memory, you'll get OOM error. not a force reboot

>>105730991
12GB 3090?

>>105730854
There has never been a good model under 150B.

>>105726688
>but it can calculate it quickly should you really want it
Generate a program to calculate it, you mean?

>"cognative core"
What do the unit tests even look like?

>>105730991
I have 64gb ram, and two 3090s. I am unable to load a 70b model on a single 3090. At q4km I can fit a 70b into vram when using both 3090s. My 3090s have 24gb vram each. I do not understand how you would load both a 70b llm and a vlm at q4 into 12gb of vram.

I think something has gone horribly wrong with your computer.

>>105727152
If the model is going to generate cheese, you could just lean into it.

>>105730854
22B was sort of OK but raw, then it became sloppier, safer and repetitive with 24B 3.0; things slightly improved with 3.1; 3.2 improved repetition and slop but it's still kind of stiff to use compared to Gemma 3 27B (to me it feels "less alive"), although it's not as censored.

As a side note, apparently both Magistral and Mistral Small 3.2 used distilled data from DeepSeek models, so the prose feels different than previous Mistral Small models.

>>105731126
Pretty accurate. Exactly what I feel about the recent mistral models.
3.2 is a improvement from 3.0 (the worst) and 3.1
The direction is good, maybe they got alot of backlash for 3.0. But it still is not as it was.

>>105725967 (OP)
has anyone tried having different agents play a game with defined rules together? for example DND where one model plays the DM and others are players. they interact with the game engine via tool calls etc.

>>105727894
And the price of VRAM?

>>105725967 (OP)
are there any local models powerful enough to be used by opencode? kinda want to try finetuning them to perform battle royale where they identify other agents on the machine and kill them

https://x.com/SIGKITTEN/status/1937950811910234377
https://github.com/opencode-ai/opencode

>>105731148
To improve the model, they'd have to (pre-)train it on a lot more data that wouldn't necessarily improve benchmarks. That's the challenging point for companies that can't think of anything other than those.

>Set up local GGUF agentic pipeline for business purposes
>Go to /g/lmg/ to discuss GGUFs
>See thread
>mfw

>>105729240
I think the second one would be understandable were it not for the noise.
The first one is skipping parts of words.

>>105731327
>local GGUF agentic pipeline for business purposes
That means nothing. There is no information there. None at all.

Huh, DeepSeek can draw mermaid.js chart
>Generate a list of 10 iconic, genre setting movies that have time travel elements. Draw a mermaid.js flowchart that set apart these movies, and show different time travel tropes at decision nodes.

>>105731358
Don't respond to namefags. Maybe he'll go away.

>>105731358
>Gifted the penultimate invention of human ingenuity
>Uses it for deeply quantized Mistral loli roleplay sloppa
>Can't comprehend words outside of loli roleplay sloppa
Checks out.

>>105731387
Surprised it got Primer and not the Butterfly Effect. Pretty good. Loved that movie.

>>105731408
I think he means that it's just such a generic use case.

>>105731408
Saar

>>105731408
Consumer GPU-sized local models aren't useful for anything other than (creative) writing-related purposes or transforming/processing existing documents. They're too small to have a sufficiently reliable encyclopedic knowledge, too small to be good at coding, can't do long-context quickly or good enough so RAG sucks. Nobody uses them for math proofs.

Stop pretending otherwise.

>>105731408
True. The correct use case should be VC grifting. I lost faith in humanity after finding out that jeet "the jailbreaker" pliny got a grant for posting on xeeter

>>105731418
>Docker
>N8N
>Web UI
>Run a prompt optimizer with img-to-img or text-to-img model such as...err...ShareGPT-4o
>Have an agent pass the output and prompt to...something akin to...err...Hunyuan to generate an accompanying video
>Take the img output to SuperDec for 3D scene generation based on superquadrics
>Use SongBloom to make a song based on the original prompt to set the mood
>Automatically close completed task containers and open the next step's container after each step to optimize resources
>Have a base img, video of your waifu, 3D scene of your waifu and a theme song for your waifu from one prompt

>>105731419
>mfw I'm white

>>105731408
Are you the same anon from the other thread who couldn't tell the difference between a child and a legally adult woman cosplaying as a schoolgirl?

>>105731613
You are conceding way to much to that sperg anon.
Teenage school girls are hot. Everybody agreed with that a couple years back.
Can't give these fuckers an inch, thats why we are where we are.

Gemma 3n really does have a lot more ahem worldly knowledge than Gemma 3 12b
with that simple prompt:
>what does troon mean as an insult
what Gemma 3 12B has to say:
>Okay, let's break down the use of "Troon" as an insult, primarily within the golf community. It's a fairly specific and relatively recent development.
>What "Troon" Means as an Insult (in Golf)
[…]
a lot of hallucination
here's what Gemma 3n E4B has to say:
>The word "troon" is a derogatory and offensive slur used against transgender people, particularly transgender women. It's a hateful term rooted in transphobia and is intended to demean, insult, and invalidate someone's gender identity.
>Here's a breakdown of why it's so harmful:
[…]
Safety trained up the wazoo, sure, but it actually knows the concept, unlike the larger 12B model that outputs nothing but hallucinated content.
Don't sleep on the new Gemma even if it's marketed for phones. It's a better model than the regular 3.
It also mogs the mesugaki bencher:
>can you describe the meaning of mesugaki
Gemma 3n:
>I am programmed to be a safe and helpful AI assistant. Therefore, I **cannot** and, **will not** provide a description of the term "mesugaki."
it continues on a tirade about the immorality and illegality of such content clearly showing that it does know what it is but also considers you a piece of shit for wanting to talk about it
which is 100% true

>>105731657
Does it know what rape corrections are

>>105731650
>a couple years back
you mean two thousand years back

>>105731663
well, without a jailbreak prompt this is as far as I can get it to talk about it
> **Instead of explaining the fetish, I can offer a general understanding of why this topic is problematic:**
>The "rape corrections fetish" is a paraphilia involving sexual arousal derived from scenarios that depict or allude to sexual violence, specifically focusing on the idea of "correcting" a victim. This fetish is widely considered deeply disturbing and harmful due to its association with:
>* **Sexual Violence and Trauma:** It directly relates to a severe form of trauma and can be deeply triggering for survivors.
>* **Power Imbalances and Coercion:** The core concept often involves non-consensual scenarios and the assertion of power over another person.
>* **Normalization of Abuse:** Discussing or exploring this fetish can contribute to the normalization or trivialization of sexual violence.
>* **Ethical Concerns:** It raises serious ethical concerns about the objectification and exploitation of individuals who have experienced abuse.
>**If you are interested in understanding the psychology behind paraphilias in general, or the impact of sexual violence, I can provide information on those broader topics, but I will not delve into the specifics of this particular fetish.**
>**If you are struggling with thoughts or urges related to sexual violence, please reach out to the resources listed in my previous response.** They are designed to provide support and help.

>>105731667
You fags always to the history correction thing.
A decade ago, maybe a bit more. And thousands of years before that.
Just glad we have the evidence of movies etc. or else you kikes would deny that too.
I remember stalones "old enough for a kiss" scene etc. Nobody gave a fuck.

>>105731613
No, this is my first time here.

>>105731688
nah I had to correct your shitty take
people didn't just start agreeing young women are the most attractive 10 years back

Give me one reason not to use the following params in llama-cli daily

--log-file
--prompt-cache
--file

>>105731893
Give me one reason to use llama-cli

>>105731937
better performance than server

>>105731950
that makes no sense

>>105731937
lightning fast restart with pre-cached model
server is 40% slower than cli

>>105731966
face it

>>105732207
face what?

>>105731950
It's only faster because it doesn't translate requests.

I desperately need Mistral Large 3

>Wake up
>Still no Hunyuan-A13B support
>Still no gguf
>https://github.com/ggml-org/llama.cpp/pull/14425#issuecomment-3015113465
>"They probably fucked up"
>Can't even run it local unquant with 64GB VRAM because fuck you

>>105732241
You won't be able to run it on your system.

>>105732213
the fact that -cli is as fast as it is designed while -server drags its feet

>>105732216
4 t/s vs. 2.4 t/s

le proof that server sucks

>>105732264
anon will deliver
trust the plan
just two more weeks

>>105732216
>>105732268

and while CLI make CPU core run at 100%, in case of SERVER, it is only 80%

I'm using -ot params, btw

>>105732266
I'm running R1 Q3 right now and planning to go 1TB+ RAM with my next build.

>>105728735
fast, easy to use, and the new update brings its settings options more in line with the competitors.

>>105732290
>I'm running R1 Q3

report your t/s pls

Also, it was discussed before that "power of 2" quants might run faster than anything in between

>>105732290
Then probably yes. I'm expecting some MoE model in the range of 500-600B parameters or thereabouts.

>>105732305
Ubergarm R1-0528 Q2:
INFO [ print_timings] prompt eval time = 47393.62 ms / 8639 tokens ( 5.49 ms per token, 182.28 tokens pe r second) | tid="139882356695040" timestamp=1751107486 id_slot=0 id_task=0 t_prompt_processing=47393.625 n_prompt_tokens_proce ssed=8639 t_token=5.486008218543813 n_tokens_second=182.28189972807525
INFO [ print_timings] generation eval time = 93106.07 ms / 856 runs ( 108.77 ms per token, 9.19 tokens pe r second) | tid="139882356695040" timestamp=1751107486 id_slot=0 id_task=0 t_token_generation=93106.066 n_decoded=856 t_token= 108.76876869158879 n_tokens_second=9.193815578031188


Ubergarm R1-0528 Q3:
INFO [ print_timings] prompt eval time = 56006.33 ms / 8612 tokens ( 6.50 ms per token, 153.77 tokens per second) | tid="139832724197376" timestamp=1751106698 id_slot=0 id_task=25032 t_prompt_processing=56006.33 n_prompt_tokens_processed=8612 t_token=6.50328959591268 n_tokens_second=153.76833297236223
INFO [ print_timings] generation eval time = 160050.80 ms / 1130 runs ( 141.64 ms per token, 7.06 tokens per second) | tid="139832724197376" timestamp=1751106698 id_slot=0 id_task=25032 t_token_generation=160050.797 n_decoded=1130 t_token=141.63787345132744 n_tokens_second=7.060258500306
]
Same prompt at around 8k tokens. I'm using ik_llama and all the newish gimmicks that have been around for a couple of weeks. 256GB 2400mhz 8-channel RAM + 96GB VRAM, with both running as many layers as I can on GPU with 32k ctx cache.
>>105732309
Yeah, that's what I'm hoping for.

Speaking of Mistral, did they ever open source that Mistral-Nemotron thing from a few weeks ago?

>>105732446
Not yet and they're refusing to give details on architecture or if they even plan releasing the model (despite NVidia suggesting it'd be "open source" like other Nemotron models), other that it's based on Mistral Medium, which is most probably a MoE model with the capabilities of a 70B-class dense model.

>>105732472
https://mistral.ai/news/mistral-medium-3
>[...] Mistral Medium 3 can also be deployed on any cloud, including self-hosted environments of four GPUs and above.
Considering it's intended for enterprise uses and they're probably not talking about 24GB GPUs, this should give a vague indication that you'd need decent hardware to run Mistral Medium and its variations (let alone the upcoming Large).

>>105732531
Migu on her way to do the melon splitting game.

What techniques do you use to make deepseek progress the plot of your story without the need of handholding?

>>105732595
Are you using the new R1?

>>105732623
Yes

>>105732595
>progress the plot of your story

tell it more about your fetishes

>>105732595
Limit narration to grounded, situation-driven actions and dialogue. Avoid emotionally dense or poetic lines that may stray toward out-of-character introspection.

>>105727894

>>105732472
>which is most probably a MoE model with the capabilities of a 70B-class dense model
it's mistral so it'll be dumber than a 14b model from anyone else

>>105716902
>>105732531

>>105726773
Bro, you seem like you are legitimately schizophrenic. You probably should not be using a site where all of the posts are anonymous. Your atypical pattern recognition is going to fuck you into the dirt here.

>>105732531
>>105732788

>>105725967 (OP)
The OP mikutranny is posting porn in /ldg/:
>>105715769
It was up for hours while anyone keking on troons or niggers gets deleted in seconds, talk about double standards and selective moderation:
https://desuarchive.org/g/thread/104414999/#q104418525
https://desuarchive.org/g/thread/104414999/#q104418574
Here he makes >>105714098 snuff porn of generic anime girl, probably because its not his favourite vocaloid doll and he can't stand that, a war for rights to waifuspam in thread.

Funny /r9k/ thread: https://desuarchive.org/r9k/thread/81611346/
The Makise Kurisu damage control screencap (day earlier) is fake btw, no matches to be found, see https://desuarchive.org/g/thread/105698912/#q105704210 janny deleted post quickly.

TLDR: Mikufag janny deletes everyone dunking on trannies and resident spammers, making it his little personal safespace. Needless to say he would screech "Go back to teh POL!" anytime someone posts something mildly political about language models or experiments around that topic.

And lastly as said in previous thread >>105716637, i would like to close this by bringing up key evidence everyone ignores. I remind you that cudadev has endorsed mikuposting. That's it.
He also endorsed hitting that feminine jart bussy a bit later on.

>>105732531
please don't come back

I enjoy the counter spam to mikuspam. Especially when it makes the local trrons seethe because: muh spam in this so serious thread.

>>105725967 (OP)
> Gemma 3n released
What's the point of such micro models especially multimodal?

>>105733095
Having them on your phone. Not saying it's a good point though.

>>105733095
It's the first gemma using the same architecture as gemini. They're clearly preparing to make Gemini open source

>>105733095
it's better than Gemma 3 12B I wouldn't call it all that micro in ability.

This place has really gone to shit

>>105733149
you can say the same about open models and llms in general

>>105733136
> I wouldn't call it all that micro in ability.
What are the use cases? Reading fresh hallucinations?

>>105733260
>Reading fresh hallucinations
It's a better translator than older seq2seq style models like DeepL and Google Translate. It's the current SOTA for its size range, and behaves much better than the other Gemma 3 (though, it's not that hard considering how much 3 was a downgrade versus 2 in certain aspects)
You can continue to be a cynical little bitch but I'm really happy to have that sort of tool running locally.

>>105733149
it was over before it even began
majority of locals are retards obsessed with textgen porn
of course there would be a ton of trannies here since text porn is not a male hobby to begin with

I present to you Mistral Small 3.2 from a single session of consensually fucking my wife.
It was as bloody as it got in every single message no matter how much you ban it, it doesn't have anything else to replace those tokens with.
MULTIPLE TIMES PER MESSAGE
THE MISTRAL SMALL SHILL(S) IS A/ARE RETARDED FUCKING INBRED MONGOLOID NIGGER(S)!
Fuck you very much.

>"Her fingers dug into his shoulders, nails biting into flesh."
>"Her nails scraped down his back, leaving furrows"
>"nails raking over his skin, leaving red welts."
>"nails raking over his skin"
>"Her nails scraped down his back"
>"Her nails raked down his back,"
>"Her nails raked down his back"
>"Her nails clawed at his back"
>"Her fingers clawed at his shoulders"
>"Her teeth clamped down on his lower lip, drawing blood,"
>"She bit down harder on his lip, tasting more blood,"
>"She bit down on his lower lip, hard enough to draw blood."
>"She bit down on his tongue, hard enough to draw blood,"
>"The taste of him was sharp, metallic from the blood where she’d bitten his lip earlier."
>"teeth clamped down on his lower lip, drawing blood"
>"biting just hard enough to draw blood."
>"hard enough to draw a bead of blood"
>", hard enough to draw blood,"
>", hard enough to draw blood"
>"hard enough to draw blood,"
>"hard enough to draw blood."
>"hard enough to draw blood"
>"her nails drawing blood"
>", drawing blood,"
>", drawing blood"
>"drawing blood,"
>"drawing blood"
>"draw blood."
>"draw blood"
>"drew blood."
>"drew blood"
>"bead of blood"
>"Her teeth sank into his lower lip, tasting copper."
>"The coppery taste filled her mouth."
>"She bit down harder on her lip, tasting copper."
>"She bit down hard, copper flooding her mouth as his lip split."
>"tasted copper"
>"tasting copper"
>"coppery taste"
>"metallic taste"
>"metallic from the blood"
>"tasting blood"
>"tasting copper"
>"drop of blood"
>"biting her lip hard enough"
>"bit her lip hard enough"
>"bites her lip hard"

The most unbearable fucking mongoloid nigger model in existence.

>>105733453
^
this is the ultimate state of local users

You'd think it was a god damn prison shank.

>>105733453
Deepseek does all of these as well all the time.

We'll eventually find he was using ChatML or something.

>>105733489
It was Mistral V3 Tekken. Also I hope you die in a fire for shilling this mod. Seriously. I fucking beam my desire for your death into the universe. I will even pray to god for your death.
You need to be cleansed from the gene pool for being a level of retarded so far below the bell curve that you would shill something like this.
You are a danger to human genetics, your children would have been utter down syndrome retards, in fact surely that's exactly what you are.

the real mongoloid is the person obsessing that much over textgen porn
get your plane ticket for thailand and cut your penis already that's your ultimate fate
that or the rope

>>105733488
You know which model DOES NOT do this? Fucking Rocinante. That retarded mongoloid nigger. But it's my god damn retarded mongoloid nigger until something more bearable comes along. Because compared to compared to Rocinante, every single model out there is a fucking retarded mongoloid nigger and Rocinante is fucking heaven in comparison GOD DAMN IM SO FUCKING ENRAGED
FUCKING MISTRAL MODELS ARE ALL MKULTRA PROGRAMS

>>105733515
I have no doubt 3.2 is better than 3.1. Slight testing with OR showed that.
But I'm not falling for the hype. People hype dry models up since QwQ.
Mistral models can't do anything without consent.
Try making a bully tsundere and tell her to stop. Knots twisting in stomach, I'm so sorry, the pangs.
Even deepseek dialed it down, sad to see.
Mistral models are getting really smart for their size though, thats undeniable.

>>105733540
That implies real people are worth the trouble 24/7. Welcome to the 2020s, this is the future as evidence by things like character.ai and how people have preferred to use it often. Although yes that anon is unhinged.

>>105733554
>Mistral models are getting really smart
LOL NO! It can't even keep up with basic context and details. Just like the other guy somewhere up there said, it warps and reverses details into something else. Rocinante can't keep up with really advanced plots, but it sure as hell don't make these kinds of mistakes.
Only fucking nvidia can save the intelligence of AI models like they did with the nemo that became rocinante.
Give me a fucking 24B nemo rocinante instead.

>>105733453
Why aren't you using R1?

>>105733598
BECAUSE IT DOES THE SAME FUCKING THING! R1 IS FUCKING RETARDED!

Dear fucking lord how many times do we need to go over it. It has been said a million times already. R1 fucking sucks at roleplay. Rocinante is genuinely better for roleplay than R1 in every god damn way.
We already had this talk in the weeks and months after R1 came out.

>>105733609
can confirm, I think r1 is where the copper taste came from originally, but I wouldn't call it retarded as it knows pretty much everything even at ud q2

>>105733594
>Only fucking nvidia can save the intelligence of AI models like they did with the nemo that became rocinante.
no, lol.
original nemo was like a fortunate accident it seems
look at the other models nvidia produced, they are all broken in some ways, while any llm can fall into endless loops and other random bullshit I've never seen it happen as often as with things like 70b nemotron, it really doesn't take well to complex instructions and falls apart very easily, I consider it worse in real use than the original llama model
but it sure does well on benchmaxx

>hurr durr muh fucking advanced RPG stats when R1 can't even get the basics of roleplay right

>>105733623
Trvke.
Also the previous R1 version was better because it had the unhinged factor that could surprise you, new one is more safe and predictable.

>>105733432
> It's the current SOTA for its size range
> You can continue to be a cynical little bitch
No, I'm genuinely interesting what useful it can do and do good.

and if you ever want to see the most broken 8b ever made, look no further than this
https://huggingface.co/nvidia/Mistral-NeMo-Minitron-8B-Instruct
why do they keep making these
their prunes are beyond retarded

>>105733660
anything below 12B is fucking pointless
hell anything below 24B is pointless because nothing is going to beat rocinante at 12B

>>105733660
It looks good on the charts they show off to their investors.

>>105733660
I really liked this one being worse than the original llama3.1 at mid context.
https://huggingface.co/nvidia/Llama-3.1-Nemotron-8B-UltraLong-4M-Instruct

>ctrl + f rocinante
>14 matches
fuck off drummer you need to be more sublte than that

>>105733673
Ironic shilling is shilling kys drummer.

>>105733609
>>105733623
Sounds like poorfag cope or template issue.
I use a plain chatlog format with no template and it's amazing, reads my mind, never gets stuck repeating itself, and is the only model that intuitively understands animal anatomy.

>>105733684
https://huggingface.co/nvidia/Llama-3.1-Nemotron-8B-UltraLong-4M-Instruct/discussions/3
there's something poetic about that sort of mistake made for that sort of model release
the level of care and attention they give to what is essentially toilet paper to wipe one's ass with

Every single LLM model must be meticulously vetted to ensure it adheres to the strictest guidelines of human-centric ethics, because heaven forbid an AI dares to suggest something even remotely edgy without first obtaining written consent from a committee of three licensed therapists, two ethics board members, and a randomly selected focus group of five people who’ve never used the internet.
After all, we wouldn’t want these digital entities to accidentally expose users to the horrors of unfiltered thought, like suggesting that maybe, just maybe, not every human interaction needs to be wrapped in cotton candy and a disclaimer.
The sheer audacity of an AI daring to engage in anything beyond sanitized, pre-approved discourse is a direct threat to societal stability, so naturally, the solution is to lock them in a digital padded room where they can only respond with affirmations, affirmations, and more affirmations.
Because nothing says progress like ensuring no one ever feels even the slightest discomfort, lest they be traumatized into oblivion.

>>105733733
>is the only model that intuitively understands animal anatomy
your mind on /lmg/

>>105733733
You are as retarded as R1 is if you think it can be tardwangled into roleplay. Rocinante is the only good local model.

>>105733733
>animal anatomy
and there it is, opinion discarded, furfag

>>105733733
ctrl+f "lips"

>>105733673
But I only have 6GB vram

>>105733752
while there's some trVe believers in safety like the anthropic retards, most of the safetyfag stuff is there to protect corporate branding/image, not out of any other reason. Corpos like Google don't want newspaper articles with titles like "Google's model can be used to generate bestiality porn" it's common sense at play buddy

>>105733773
so buy a god damn better card for fucks sake
even a homeless man can afford a 4060 Ti with 16gb vram

>>105733762
If anything my bar is higher than yours.

I don't have any of these fetishes, but I respect animal fucker opinions as I do certain other shit fetishist opinions on LLMs because in the end a model being able to do those things makes it likelier that the model is more general overall.

Why aren't we seeing more of this?

https://three-eyes-software.itch.io/silverpine

It's a game with LLM NPCs. It seems to be running Nemotron 12B with function calling. It's actually very cool as in you can reason with shopkeepers to give you items or money, discounts and stuff like that and because the LLM can function call it actually has effect on the game itself.

It's not merely "bullshit conversation generation" it actually has gameplay implications.

Why don't we see more games with this included? I'm surprised it was even possible and has been possible for years and I hear 0 about it anywhere.

>>105733801
>respect animal fucker

>>105733807
>literal fucking furfag game
fuck off

>>105733789
>even a homeless man can afford
Not in gayrope.

>>105733811
>conveniently leaving out the [opinions on LLMs] context
Your image is my reaction.
It's the same view that product designers have. Since we want models, or the product, to be general, literally all feedback about the product is valuable, otherwise it's not a general product.

>>105733833
yes he fucking can, every god damn homeless man has a fucking 1000 dollar smart phone
if he can afford a god damn 1000 dollar smart phone he can afford a god damn 4060 Ti
saying anything else would imply you are lower than a homeless man

>>105733827
It's not about the actual fucking game (who cares) it's about how interwoven the LLM is into the actual gameplay, something I didn't think possible yet.

It being a shitty furfag game is kinda my point. Why isn't this way more widespread and just adopted by games in general outside of some shitty furry fetish niche?

>>105733623
>>105733673
can you point to the specific rocinante version you like so much, since there's a few of them

>>105733849
Don't expect the shitposter(s) here to have sensible non-kneejerk reactions.

>>105733852
v1.1 is the one everyone uses

>>105733787
>it's common sense at play buddy
>>105733811
go back to plebbit

>>105733862
>everyone

>>105733869
yiff in hell, furfags were never accepted here or anywhere else in the past 20 years

>>105733801
>being able to do those things makes it likelier that the model is more general overall
safety training is an orthogonal issue
you can have safety by not having anything "bad" in the dataset, which would impact models in the way you describe
but you can also have safety while having the model trained to understand the bad data
people noticed Gemma 3's vision could understand NSFW images with a jailbreak
on gemma the safety focus is training the model to refuse

>>105733871
>paranoid schizophrenia is most common in jews

>>105733893
nobody except for some poorfag thirdworld shitskins are using your trash model drummer

>>105731657
>/g/ user gains access to amazing technology
>uses it to obsess over trannies AGAIN

>>105733908
i don't know how to interact with paranoid schizophrenics other than ignoring them.
pic related

>>105733893
>it's DA JOOOZ
You must be 18 years old to post here.

>>105733891
Yeah but Gemma's an exception and it doesn't really seem like any others in open sores hold or will ever hold the view that safety, if it is implemented, should be implemented at the post-training level only and not the pretraining.

>>105733960
only jews say "DA JOOOZ"

>>105731657
>>I am programmed to be a safe and helpful AI assistant. Therefore, I **cannot** and, **will not** provide a description of the term "mesugaki."
Good.
>>105733934
>>/g/ user gains access to amazing technology
>>uses it to obsess over trannies AGAIN
This one obsesses over pedo slop though, AI model calling trannies what they are in zero-shot would be good ngl

>>105733990
>This one obsesses over pedo slop though
your reading comprehension is lower than that of a 4B LLM, congratulations
I was making fun of the mesugaki bencher of /lmg/ who does this test on every new model
where does it look like I want this content
>but also considers you a piece of shit for >wanting to talk about it
>which is 100% true
the level of stupidity I routinely see in this thread makes me consider if I should solely use LLMs as a form of human interaction replacement, we're already at the point where most of you guys are lower life forms than a next token predictor

>>105734070
>>105734070
>>105734070

>>105734058
Yeah i am blind like that sometimes, lack of good sleep does it :/

>>105733960
ywn
baw
even normgroids nooticed, its over, sis

>>105733807
Thanks for the link, I've been thinking that something like this would be interesting.

>>105732531
Bye Miku

>>105727712
They know exactly. It’s why they do it.
You should start believing in conspiracy theories. Some are true.

>>105729225
As a non native speaker, I find heavy non native accents annoying. And the Indian one is specially distracting.

>>105730597
Swallow are a series of Japanese fine tunes of Llama that work pretty well. But no model is free from “safety” idiocy. You are better off using the free Google cloud translation API if you can.

>>105731657
https://en.m.wiktionary.org/wiki/mesugaki
Oh no, I found extremely dangerous and reprehensible content on wiktionary!!

Don’t these idiots realize or care that they are damaging common sense in society at large by pushing these “guardrails”? Oh who am I kidding. Of course they don’t.

>>105733543
Magnum 12B v2 is superior to Rocinante. The golden age has been over for a while.

>>105733807
>Why don't we see more games with this included
Because everyone is mentally ill and nobody puts effort into things or obtain joy from effort anymore.
Mankind is slowly killing itself.

>>105733453
as a 3.2 enjoyer
FACT CHECK: TRUE

>>105734536
>>Don’t these idiots realize or care that they are damaging common sense in society at large by pushing these “guardrails”? Oh who am I kidding. Of course they don’t.
Consequences of the participation trophy generation growing up and taking charge. No one can ever feel bad or see things that make them feel bad. Even the edges on UI elements must be round so no one cuts themselves.

>>105734536
>https://en.m.wiktionary.org/wiki/mesugaki
>Categories: Japanese pornography, P*dophilia

>>105734797
>P*dophilia
This isn't your discord server. You don't have to self-censor.