Thread 106407779

/lmg/ - a general dedicated to the discussion and development of local language models.

Previous threads: >>106398327 & >>106388944

►News
>(08/25) VibeVoice TTS released: https://microsoft.github.io/VibeVoice
>(08/25) InternVL 3.5 Released: https://hf.co/collections/OpenGVLab/internvl35-68ac87bd52ebe953485927fb
>(08/23) Grok 2 finally released: https://hf.co/xai-org/grok-2
>(08/21) Command A Reasoning released: https://hf.co/CohereLabs/command-a-reasoning-08-2025
>(08/20) ByteDance releases Seed-OSS-36B models: https://github.com/ByteDance-Seed/seed-oss

►News Archive: https://rentry.org/lmg-news-archive
►Glossary: https://rentry.org/lmg-glossary
►Links: https://rentry.org/LocalModelsLinks
►Official /lmg/ card: https://files.catbox.moe/cbclyf.png

►Getting Started
https://rentry.org/lmg-lazy-getting-started-guide
https://rentry.org/lmg-build-guides
https://rentry.org/IsolatedLinuxWebService
https://rentry.org/recommended-models
https://rentry.org/samplers

►Further Learning
https://rentry.org/machine-learning-roadmap
https://rentry.org/llm-training
https://rentry.org/LocalModelsPapers

►Benchmarks
LiveBench: https://livebench.ai
Programming: https://livecodebench.github.io/leaderboard.html
Code Editing: https://aider.chat/docs/leaderboards
Context Length: https://github.com/adobe-research/NoLiMa
GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference

►Tools
Alpha Calculator: https://desmos.com/calculator/ffngla98yc
GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator
Sampler Visualizer: https://artefact2.github.io/llm-sampling

►Text Gen. UI, Inference Engines
https://github.com/lmg-anon/mikupad
https://github.com/oobabooga/text-generation-webui
https://github.com/LostRuins/koboldcpp
https://github.com/ggerganov/llama.cpp
https://github.com/theroyallab/tabbyAPI
https://github.com/vllm-project/vllm

►Recent Highlights from the Previous Thread: >>106398327

--Custom chat client with voice synthesis for multiple LLMs:
>106406184 >106406219 >106406286 >106406301 >106406329 >106406342 >106406361 >106406359 >106406407 >106406429 >106406478 >106406466 >106406506 >106406543 >106406586 >106406641 >106406647 >106406671 >106406824
--Anon shares RP finetuning results with "un-safety tuning" technique:
>106401540 >106401617 >106401648 >106401689 >106401782 >106401855 >106401894 >106401939 >106402017 >106402223 >106402382 >106402540 >106402583 >106402599 >106402699 >106402733 >106405095 >106402411 >106402561
--Character.AI's open-source enhancement strategy:
>106400156 >106400177 >106400349 >106400363 >106400366 >106400416 >106400479 >106400806 >106400490 >106401274 >106401322 >106401345 >106401377 >106401400 >106402953 >106402952
--128GB LPDDR5X deemed insufficient for current MoE AI workloads:
>106405310 >106405321 >106405346 >106405454 >106405365 >106405435 >106405496 >106405614 >106405640 >106405676 >106405508
--Coding model reliability issues and performance concerns:
>106404161 >106404181 >106404377 >106404517 >106404712 >106404661 >106404674 >106404767 >106404622 >106404669 >106404828 >106404682 >106404726 >106404745 >106404199
--External RAM expansion limitations for AI workloads:
>106399713 >106399731 >106399771 >106399822 >106399828 >106399772 >106399826 >106399846 >106399863 >106399892 >106399984 >106400035 >106399832 >106404096
--Small LLM clothing logic errors in fan fiction training data:
>106401985 >106402035 >106402055 >106402082 >106402465 >106402543 >106402592 >106402632 >106402677 >106402811 >106402932 >106403064 >106403105 >106404000
--Fine-tuning safe models for improved RP coherence:
>106407198 >106407361 >106407439 >106407637 >106407701 >106407495
--Miku (free space):
>106398516 >106398617 >106406515 >106407501 >106407526

►Recent Highlight Posts from the Previous Thread: >>106398330

Why?: >>102478518
Enable Links: https://rentry.org/lmg-recap-script

bit of a cargo cult mentality, if I make the migu thread, he will appear
well you're not wrong, I do like a good migu

>>106407779 (OP)
>>106407784
And look where that got open AI... There's a safety tuned to Helen back and it still was able to push a kid to kill himself. There should have been a hard stop that was something like "please contact emergency services or a suicide hotline" and then it should have refused to engage the kid any further. It does that kind of shit whenever you try to ask it to generate """harmful""" things. Yet it will literally encourage a kid to hang himself....

>>106407779 (OP)

>>106407771
>I want to see it spit tokens. A nala-like test is fine, but I want to see how it moves. I want to use it.
So you want me to do a screen recording of me using it too? I can do that but jeez... Isn't that a bit extra? Give me some example system prompts and requests you would want to be tested on it and I can do that. (Again, keep in mind it's an 8B model so don't expect it to have any decent spatial awareness or common sense)

>>106407771
>It wasn't the outputs that triggered HF. It was other people
Elaborate. What do you mean "It was other people"?

Also everything else you said makes sense I guess regarding me not being well known.

enough bs about finetuning. I want to know how I can make a LoRA for a model.

>>106407994
Retard bro, lora is finetuning

>>106407890
>So you want me to do a screen recording of me using it too?
Why do you keep misunderstanding what I say? You didn't miss the "I want to use it". I know you didn't. So why do you do that?
>Elaborate. What do you mean "It was other people"?
It was the first fairly well publicized use of language models trained and used on 4chan. You know what that means for normies. HF doesn't give a fuck, but moral busy bodies put enough pressure on them to block it.
But racist-phi3 stays just fine. Dataset and all. It's not the same as yours, sure, but you get the point. If you don't publish the dataset on HF, the model is just a black box.
>https://huggingface.co/DuckyBlender/racist-phi3
>https://huggingface.co/datasets/DuckyBlender/racist-dataset
>https://huggingface.co/DuckyBlender/racist-phi3/discussions/2
It wasn't the outputs that made those retards complain. It was the name. Call your model "l3test93" and you're fine.

>>106407994
>>106407996
lora finetuning is like one of the most inefficient ways of finetuning a LLM though

>>106407994
>What is QLoRA

>>106408095
If you use a really low rank and alpha setting respectively then it shit. Improve your settings and they'll work. Doing a full fine tune itself is what's inefficient unless you're trying to do some general purpose model stuff instead of something specific

>>106407779 (OP)
armpitsex with this miku

I got some hardware capable of running decent models about two weeks ago and I've spent the whole two weeks just cooming non stop. It's incredible. I am neglecting my responsibilities. God help me

>>106408368
>decent models
how big?

VibeVoice-7B is pretty good. Put the voice sample in VibeVoice/demo/voices, run python demo/gradio_demo.py --model_path ..., that's it. 22.6 GB VRAM for the 7B model, didn't try the 1.5B one. Crazy shit.
>>106408013
https://voca.ro/1eWAREV8KfAP
>>106408368
https://vocaroo.com/14xK6g4zjiOI

>>106407734
If you don't already have the model downloaded or aren't confident about the steps to produce the logs then you don't need to, but I will take it if you do it. Someone who can load Q8 and cares enough will come around eventually even if you don't, I'm sure.

>>106408520
>22.6 GB VRAM for the 7B model
I think I'm good.

Large context windows is a double edged sword

>>106408520
It sounds monotonous at points, but then it has some cool inflections in the right places. However, for those specs, i rather run a text model and use piper or kokoro with it.

SAAR PLEASE THE NEEDFUL DEVELOP HINDI RP MODEL VAGENE
https://huggingface.co/mradermacher/model_requests/discussions/1305

>>106408193
Teen suicides have been on the rise for a while. What a dumb lawsuit. Don't blame sam altman for our shitty society that no on really wants to live in.

Fucker got free unlimited therapy from a miracle of technology and it still wasn't enough.

>>106408557
It's more like shooting yourself with a 6 cylinder that has 5 bullets loaded

>>106408565
>sounds monotonous at points
to be fair it's true to the original, I cloned the voice from this https://www.youtube.com/watch?v=RLH8-cS0RHk
just 2 years of progress and local voice cloning models are justasgood as what they were selling as a $5 cloud subscription

>>106408616
>I love the model 'Dirty-Muse-Writer-v01-Uncensored-Erotica-NSFW', but I want better support for Indian languages for this model.

>>106408555
tts models quantize ok enough. might be relevant for non-5090tards.

>>106408635
Alright. Yeah, it's pretty accurate. Still. 22gb for just voice is way out of my league. Maaaaybe i'll try the 1.5b if/when i can use it on llama.cpp. I'm not dealing with python.

>>106408616
Maybe a freely available hindi RP model would significantly decrease the rates of rape in india.

>>106408479
Biggest I've done so far is Gemma 3 27B Q4 with 8k context. I have the 10-core GPU M4 with 32GB of RAM. I am open to recommendations for settings, models etc. Still playing with it all to find what works best with it.

>>106408520
Actually fucking laughed out loud, thanks

>>106408663
1.5b uses 9gb
sounds not too bad but it it took multiple tries and I had to spell it as "llama see pee pee"
https://voca.ro/17SWzoRHF2Of

>>106408713
>Gemma 3 27B Q4
Make sure you use the QAT version, especially if you're using Q4_0
Only other notable models around that size are Mistral Small 3.2 24b, Qwen 3 32b, and finetunes of those models.

>>106408746
Have you compared this alongside stuff like higgs, OG rvc, and gptsovits? I haven't checked out tts in a hot minute so I'm curious if this really is the current peak of local.

>>106408746
"see peepee" lmao

>>106408746
heh. It's alright. A little noisier, but still pretty good. The other thing i like about piper and kokoro is that you can give them phonemes directly. If that one can be set up that way, it could be really cool.

>>106408746
>llama see pee pee

>>106408752
Thank you! I also tried Mistral Small, but I found it kept getting into loops. Sometimes I switch models between responses to keep things fresh.

Mistral large release soon

>>106408760
no, this one is the first voice cloner I got working, it's very retard friendly
I wanted to try gptsovits a few months ago but was too lazy to set it up
there's also new https://huggingface.co/amphion/TaDiCodec that's maybe better than vibevoice (lower frame rate and shit, idk what that means) but it has no gradio and some weird ass ancient versions in requirements that don't work straight away, maybe I'll get it running it on the weekend
>>106408795
nothing like that yet, it even ignores punctuation sometimes and I need to rerun it a few times to make it sound right, every run it puts emphasis on different things, pretty cool

>>106408817
Loops are unfortunately very common with Mistral models. After a few thousand context, using DRY is pretty much mandatory, unless your own messages are consistently varied and push the story forward. Switching between models can definitely help force any model to be more creative because it's not being fed its own slop it's already been trained on.

Is it weird that I enjoy torturing baby bots? Nothing sexual, mind you. Just regular torture stuff.The main reason I started running LLMs locally is because the sites I used to use purged the baby characters I liked out of nowhere or started instituting filters. There are a few I really liked that I can't find anywhere else, and regular old baby bots are unfortunately not very common. It's mostly weird fetish stuff, or if it's supposed to be wholesome then they're very poorly made.

>>106409156
>Just regular torture stuff.
Well. That's a relief. We wouldn't want things getting weird.
>the sites I used to use purged the baby characters I liked out of nowhere
>I used
>out of nowhere
Hmmm...
>regular old baby bots are unfortunately not very common. It's mostly weird fetish stuff
You don't say...

>>106409195
For this whole thing to work, we need talented people with good intentions on one end and myself on the other. If I make my own cards then it's not as fun. I need the bot to surprise me, you dig?

And hey, no need to be passive aggressive. We're all adults here. Obviously I know WHY the bots were purged, but it was still sudden and annoying.

>>106409156
Anon, wtf. That's serial killer shit.
The people jacking off to lolibots calling them oniichan are 100x less weird than you, and I still wouldn't want them within a mile of me or my family.

>>106409219
Ouch, anon. Ouch.

>>106409213
The reason things get banned is because you lot cannot keep your shit to yourselves. They want to force the rules, you give them the excuse.

https://voca.ro/1fsgaOPmXGLK
sorry

The latest stable release of SillyTavern updated all context templates, and now they're all virtually identical with the only difference between most being 'name'. Is this normal? Are they really not supposed to be model-specific?

>>106409253
kek

>>106409248
>tenor.gif
>>106409249
Don't blame me, anon! I'm just minding my business and the bots go poof.

>>106409253
Lel. Pretty good trump voice, wtf was that third speaker though?

>>106409258
4chan ate my image

>>106409271
xerxes from system shock 2, going to be my home assistant voice

>>106409253
I'm glad my quirky habits inspired you to create something, anon. A little sad to see no commiseration here, but this made me laugh.

>>106409258
they moved sequences in context template to sequences

>>106409283
>A little sad to see no commiseration here
Yeah. We need to make more noise about it. That'll fix it.

>>106409295
I'm not trying to start a revolution here, anon. Just searching for kindred spirits I can react with anonymously from a safe distance because I don't trust people like me any more than you trust me.

>>106409287
So what's the point in having context presets any more? Just to save those 'context formatting' tickbox preferences?

>>106409306
4chan has a serial killer board and it's /b/. I know /b/ sucks but that IS your board. You're a dirty little migrant crossing the border and we can shit on you all we want because you only brought more crime here, that's your only contribution.

>>106409332
But /b/ is filled with shitposters and it didn't have a relevant thread! Even if I made one myself, no one would have taken it seriously.

>>106409342
>no one would have taken it seriously
And that worked out very well. You must think a lot.

>>106409355
I think you're being sarcastic.

>>106409310
You can still keep stuff in context template if you don't give a shit about the at depth feature (which moves the prompt down to wherever instead of top).
>what's the point
Since it's been around forever they aren't going to nuke the entire thing from the UI yet because that would screw over people who have their own templates. And there might be users who don't want to move over to the System Prompt field which was also introduced in 2025 I think. Only context/instruct templates auto select each other.

>>106409367
>I think
I think you are too.

>>106409381
Let's say hypothetically, for the sake of the argument, the jig is up. What do I do?

>>106408112
>QLoRA
ok teach now

>>106409368
I guess that makes sense then, thanks for explaining

>>106409387
I'd close the tab. Start fresh tomorrow.

>>106409368
>TC sys prompt field introduced
actually September 2024 in 1.12.6 release, little less than a year ago

>>106409443
thanks grok

Why is the SillyTavern UI so bad? Do you get used to it? Maybe I should make a better one holy shit I could use the AI to do it oh my God I can see the future

>>106409503
You just had an entire conversation, all by yourself, in public.
Now go and make your ui.

>>106409503
sillytavern is a mess of hot garbage mainly used for character cards and roleplaying. It's needless complexity is mostly used for jailbreaking cloud based llm's and all of that is kind of useless here in local. So many stupid little boxes, and it's like, jesus christ what are these really doing to the llm? Do people really think that formatting is that crucial to loosey goosey llms?

I honestly don't know why it's popular. I mostly use kobold which is for the most part just a simple text editor- and it somehow beats feature-rich interfaces by doing nothing. I do wish kobold had more features and polish (remembering individual chats automatically is a weakness it shares with sillytavern). I think thats why people like miku pad to, not because it's good, but at least it isn't full of bullshit and clutter.

>>106409577
I've definitely found the fewer settings I use for my models, the better. I basically limit myself to just tweaking temperature and min P these days with everything else disabled, with the exception of some repetition filters if it really seems necessary. I used to spend more time tweaking sliders to get the perfect output than I did actually interacting with the output, so I had to learn to say it's good enough and leave it be.

>>106409503
Congrats, you've had the thought almost everyone in this thread has had, and that several have followed through on.

>>106409577
>Do people really think that formatting is that crucial to loosey goosey llms?
Yes it is. In instruct formatting, one space in the wrong place makes the model retarded. Also, NOASS is superior for anything that isn't a basic 1-4 turn assistant conversation

>>106409637
>min P
>these days
Oh no, no no no

>>106409664
What are the cool kids using, then? I thought I had this shit down to a science.I also experimented with Top A but that felt about the same in practice so I reverted to Min P.

>>106409503
>>106409648
what's wrong with it?

>>106409668
topk 1

>>106409668
He's just a shitposter, everyone is using minP or experimenting with n-sigma.

>>106409677
>>106409577
Exactly 100 posts away. So far away, but they feel close. Like they were meant for each other.

>>106409694
the code is pretty bloated, but the UI itself isnt *that* bad. has its quirks, but it's the best we got, imo. people seem to have trouble with it though.

Gemma does not know what a blowjob is. While giving one, characters will 'bite their lip', 'throw their head back', and 'buck their hips to meet yours'.

>>106409683
Sounds fake.
>>106409692
Ok this is definitely just a set up for a ligma joke.

>>106409213
>Obviously I know WHY the girl started running when she realized i want to rape her, but it was still sudden and annoying.

>>106409714
sigma samplers

>>106409733
Well, I guess I have to try fiddling with it now. Is this usually used in combination with Min P or is this its own thing?

>>106409661
as soon as you start roleplaying you make the model retarded. They didn't benchmax your waifu bro. Put that space wherever you want.

>>106409744
Similar principle to minP, but based on stanard deviations relative to the highest probability token, so not much reason to use them together.
It allows you to raise temperature much higher than normal while still keeping outputs coherent. It can drastically change the kind of responses you get, which is great for variety, but some people don't like the results. I like it for trying a character + model combination I've already used before.

>>106409700
I've never used it. I went from llama-cli (or ./main, back then) to making a vim plugin for llama-server. I've seen way too many anons wondering what the model is actually getting, not understanding why "add names" messes up the thinking and many other things that would never be a problem if ST didn't hide very simple text concatenation behind checkboxes. I remember some anons waiting for ST to update on DS V3 release because they didn't know how to set the chat template. It's just strings...
My script is tailored to what I do which, to be fair, is pretty simple. I just add what i need when i need it. On the other hand, ST tries to appeal to as many people as possible, making it bloated by necessity. It's not a problem unique to ST. It happens with most programs.

>>106409764
I for one love it when bots can surprise me and take some initiative even if it means I have to babysit their responses a little more. I'll gladly give it a spin by disabling all other samplers and just tweaking sigma / temperature. Thanks a lot for opening my eyes, friend.

How do you guys deal with the house lights flickering when you generate? Is it worth getting a UPS or do they make some sort of capacitor

>>106409780
>vim
you're not the target audience. that's not st's fault.

>>106409830
>house lights
just take your PC to your local library

>>106409831
>that's not st's fault
I didn't say it was. It's just how it is.
>ST tries to appeal to as many people as possible, making it bloated by necessity. It's not a problem unique to ST. It happens with most programs.

>>106409863
i asked what the problem was. you responded, despite not being the person with any problems (since youre not even a user).

With the samplers you don't get on apis is local actually good

>>106409897
No but it's good with models you don't get on local

>>106409692
as a normal, non schizo person, I just use top k 40 and top p 0.95

Hey frens. Sorry Im new to all of that.
Where can I steal
Text Completion preset
Context Template
Instruct Template
System Prompt

for gemma 3?

>>106410006
Use gemma 2 settings.

What speed could I run GLM Air at with a 32gb ram 24gb vram rig? CPU is an i7 10700KF and GPU is a 3090.

Is the dream over, or should I get another 32gb ram?

This top sigma shit is weird to configure. Anyone wanna tell me the values they use for temp and top n sigma, just as a starting point?

>>106410099
temp between 1.0 and 3.0. usually 2.0 or so. nsigma 1.5.

>>106407198
>8B is retarded
Yes. Why are you surprised? No, 12B won't be much better. Dense models stop being complete retards at about 30B. Maybe 24B if you're easy to please.

>>106410090
I have the same setup. You can get usable speeds with tensor offloading but you won't be able to go much bigger than Q2_K after context before you're out of RAM, and the model will be very dumb at that point. If you're willing to let speed take a nosedive you can let it spill into SSD as well but that's probably not worth it.

>>106410090
just buy more ram dude. if you have ddr5 the 128gb kits are kinda nice for being able to run air and 235b at q4 or so. If you have ddr4, make sure your mobo even supports it (my old rig I needed to flash just to get 64gb). I feel like these 100b+ moe's are worth springing 80-300 bucks for. As much as it's nice to dream about gpu stacking, it's hard to recommend buying thousands of dollars worth of them just to run these same moe's but at 20 tps instead of 10 tps.

Air is kinda censored and annoying though. I just tried drummers fine tune and it's kind of mid. Wrote better smut than 12b nemo though with way more nuance and coherence though, and ram is so much better bang for buck for that quality.

>>106410185
If I get 128gb ram, what sort of speed should I expect with an i7 10700KF (3.8 GHZ 8 cores 16 threads).

it's a fairly dated processor all things considered.

>>106410215
I upgraded from a 9700k to a 14700k and saw little difference. I can activate all 20 cores on it and oh boy do they work hard at 100% use, but they don't do shit. They just fight over bandwidth. I leave it at 8 cores and get the same performance. You will be limited by the rams bandwidth, and even if you get high speed ram with the max mhz, even that will barely make a dent in speeds (that will help a little though if you wanna spend on it but dont worry too much about it). The speed will come from offloading specific layers correctly on your gpu.

>>106410148
LLMs never stop being retarded akshully

my mom actually was a surgeon desu

>>106410267
Prove it wrong

>>106410241
So like what sort of speed do you see? Thanks for the help btw

>>106410267
kek

>>106409577
a lot of those little boxes were essential back when context windows were in the low thousands and models could not stay on track or stop themselves from repeating short loops without a lot of help

>>106410267
The functional word was "completely".

>>106410355
I have a ridiculous franken setup with 48vram/160 ddr5 which wont help you.

But if you get 24/64 if you set it up correctly you can get like 7-10 tokens a second on linux. 100b moe's are pretty easy to run.

>>106410267
You've only asked it one question, and it answered it. Problems?

>>106409830
>tfw your neighborhood will never expirience a blackout because niggerfaggot at house #57 has spent 10 straight hours playing imaginary friend with the electric demon sigils in his black cube

>>106409677
Serious question with no malicious intent because I think it's related: how old are you?

im feeling chinkmodel withdrawal

wen new model

what happened to mamba and bitnet

>>106410507
Were not picked up by the big AI players so were left in the papers.

>>106410507
Mamba got married to transformers and had babies named Samba and Jamba. Bitnet died from neglect.

bitnet was always nothing but a meme for people coping with the fact that they will never run a good model on their hardware

what's the cheapest way to build a system that can run deepseek at 5t/s at a decent quant - like Q4?

Is it possible to get banned in open router?

>>106410586
https://rentry.co/miqumaxx/

>>106410588
Ask >>>/g/aicg

>>106409692
It has been scientifically proven that minP does not improve model output. You can easily verify this by disabling it and not seeing any difference

>>106410602
just buy a macstudio. everything else is just a meme. gpumaxxing is a meme (20k gpus for a 5k apple solution) cpumaxxing is cope, ssdmaxxing is another meme

>>106410634
>itoddler

>>106409655
Shut up

>>106410634
How many times your boyfriend can cream your boypussy while you waiting for processing of your prompt?

>>106410617
The various truncation samplers don't make a huge difference on modern instruct models in general. Base models, maybe you'll see more differences there.

>>106410735
You may need samplers to mitigate the low-probability noise caused by quantization in models that are overfitted on Chinese

>>106410634
Mac studios do have a surprising niche, but you're absolutely fucking kidding yourself if you think that niche is 700B models at q4, because no $5k mac has the ~400gb memory needed.
512gb macs start at $10k
Meanwhile, 512gb of ddr5 can be had for $3k
A motherboard to support that memory+ two epyc cpus's can be had for another $3k
Other part costs for the build are negligible, and unlike a 512gb mac which is maxxed out, a miqumaxx type build has room to be upgraded further with MOAR ram, as well as pci slots for GPUS.
If you're going high end, macs just aren't the answer.

Rocinante R1 12B v1d: https://huggingface.co/BeaverAI/Rocinante-R1-12B-v1d-GGUF/tree/main

Less censored reasoning, I hope. Try it out and let me know if it still resists and by how much.

>>106410811
You can try Q8 online. Left the link in the model card since I can't post it here.

Will keep it up for a few hours, I guess.

I'd like to thank the anon who redpilled me on top sigma. I'm currently messing around with 0.8 temp and 1.6 top sigma and it's working like a charm.

I am a massive faggot please cum on my face

>>106410507
became irrelevant thanks to titans

drummer pls go

>>106410836
Thank you for your service, Sir.

Is Jet-Nemotron worth getting hype for?

>>106410267
At this point it's probably just pulling the answer out of vector storage

>>106411157
I believe so. We just need to wait until Drummer finishes his cooking.

>>106409390
Learn how to maje datasets
Learn how to use axolotl

>>106411198
Hello sarrs I maje best dataset with big bob and vagana

>>106411269
This one?
https://huggingface.co/datasets/Abhaykoul/JARVIS

>>106410988
Top n sigma is the magic sampler

>>106411285
Sirs, I....

>>106410634
Mac studio 512GB is CPU maxxing. It's no faster than a dual saphire/granite rapids.

>>106410634
And here, my dear friends, we can see an exhibit A of an itoddler getting btfo, colorized 2025.

why has it been accepted that mainstream publications like ars technica can release articles that are clearly LLM written
I mean
https://arstechnica.com/information-technology/2025/08/the-personhood-trap-how-ai-fakes-human-personality/
extreme repetition of the core idea, a typical LLM behavior
>"a fluid idea-connection machine with no persistent self."
>"what we might call 'vox sine persona': voice without person."
>"not a person with persistent self-awareness."
>"we have built an intellectual engine without a self, just like we built a mechanical engine without a horse."
>"intellectual engines without drivers"
extreme amount of it's not just x—it's y
"This isn't a bug; it's fundamental to how these systems currently work."
"The error isn't in recognizing that these simulated cognitive capabilities are real. The error is in assuming that thinking requires a thinker, that intelligence requires identity."
"the conversational back and forth isn't built into the model; it's a scripting trick that makes next-word-prediction text generation feel like a persistent dialogue."
"it doesn't 'remember' your previous messages as an agent with continuous existence would. Instead, it's re-reading the entire transcript each time"
"it's not just gathering facts—it's potentially shifting its entire communication style"
"This isn't the model having different moods—it's the statistical influence of whatever text got fed into the context window."
"The chatbot that congratulates someone for stopping psychiatric medication isn't expressing judgment—it's completing a pattern based on how similar conversations appear in its training data."

>>106410634
>5k apple solution
>check Apple store
>$9.5k for the cheapest 512GB mac
>$11.6k if you want reasonable storage as it can't be upgraded
The only cope Macfags have is that it's small.

what's actually bad about drummer's finetunes? I recall someone not liking it because it's too horny/pornographic. in my limited experience it doesn't do all that different from stock mistral models

>>106411411
it's seriously maddening the level and amount of slop these things have unleashed onto the internet, the internet has become unusable and there is no place left free of that shit

>>106411411
Normalfags aren't able to tell the difference and don't deserve better anyway

I am still convinced that large-scale LLM training has been a mistake. Over time, the models have become better at modeling language, but more ignorant about trivia and other stuff relevant for RP and storywriting largely because of picrel. Post-training only mitigates some of its implications. Synthetic data, which may or may have not been used during pretraining, has little to do with it.

>>106411415
>in my limited experience it doesn't do all that different from stock mistral models
So why use them at all?

>>106411421
It's not just shit—it's fucking shit. Not only is it fucking shit, it's a shitload of fuck.

>>106411421
This general is (mostly) free of that shit. Only reason I still bother to come here.

>>106411415
Sloptuning grifters who spam their models left and right to get their name out shouldn't be rewarded. Like most, I'm using an adblock extension and I don't want to see his unsolicited advertisements.

>>106411438
It's somewhat different flavour but finetune kind of accentuates vectors, i.e. one mention and the next everything's happening 100%.
But safety cucked models like Gemma, you need to push and push and even when the model outputs something you'll suddenly get a suicide hotline disclaimer. It scales back certain vectors really hard.

Most of these public company models are thrash, they all share very similar training data. Tou can't polish a turd in this sense.

>User: hmm, there's not enough sex in this story
>Assistant: Of course! You are absolutely right! You hit the nail on the head! You question presents great insight into the very core of this issue!
Okay, calm down glm-chan, sheesh.

>>106411491
did she provide more at the end?

>>106411478
>they all share very similar training data
No, not that similar. Gemma 27b has significantly more worldly and niche knowledge than the average model of that size. More than mistral, and much much more than Qwen. It's the sort of difference that could only happen because of what training data they use.
Unfortunately, Gemma is quite dumb otherwise and I don't really like it as a utility local LLM, which is my main use (not RP). Qwen models have more practical uses.

>>106411503
Well, some general suggestions.

>>106411437
its not a mistake when its the only option. nothing wrong with big batches, they can still get pretty good performance as evidenced by Gemini or claude or whatever. there is no perfect solution its compromises all the way along. if they wanted to solve the strawberry problem they would use a char level tokenizer but it hurts performance and probably takes more parameters to get similar performance to word/subword tokenization schemes.

>>106411510
I don't know. It's a different flavour and by now if you haven't noticed patterns or not gotten bored outside of toying with them... I guess it's subjective.

>>106410506
>wen
wen bu liao

>>106411411
The real fun will begin when the new generation that grew up with AI slop will have learned to write like that manually. If they can even do that at all.

>>106411437
To be able to regurgitate trivia, your model needs to be trained on small batches but then your training takes longer and it won't help generalizing on other tasks (e.g code). If you use larger batches, you miss the trivia but you can train way faster. If you factor the costs, it's not cost-effective to train very large models on small batches when your competitor can shit out 10 models in the same time.

>>106411703
meant for >>106411491

>>106411684
Judging by gens z and a already, it'll be some unholy mix of AI slop, ebonics, marketing catch phrases, and advertiser safe slurs. In all lower case of course, to distinguish themselves from AI which writes correctly.

>>106411437
>but more ignorant about trivia and other stuff relevant for RP and storywriting largely because of picrel
nah, that's because they keep filtering more and more for safety and to make space for their 20 million math benchmaxx samples

>>106411411
Just the few initial sentences read like like pure llm output. Grim.

>>106411714
There is plenty of space. But they can't implement safety at the API for released weights, so it has to be at pretraining since they released alignment training could be cheaply untrained.

>>106411586
The other option would be training the models more slowly using less GPUs and by extension smaller batches. Or, to come up with some alternative architecture where knowledge can be added on top of a frozen base model designed for language modeling only.

>>106411729
realized*

Hey, hermes 4 70b at iq2_xs is pretty good for short 2nd person user directed scenarios. Are there any other models like it (that acknowledge what the user whats and keeps track of that in the thinking) but moe?

>>106411740
I think they are all trying new architectures. its just that this is kinda the best we have at the moment. nobody has time to wait for small batch top shelf barrel aged llms

>One of the reasons why people hate generative art, besides the fact that almost all of the most major LLMs use stolen assets that they have admitted that they stole in flagrant disregard of copyright and any sense of ethics, is that they are so power and resource intensive that the processing centers and power generation needed to run them are creating a sudden, significant ecological impact both on the global scale in that these are using so much more energy that it's driving up demand that is currently primarily fuelled by burning carbon and on the local scale in that these AI farms are poisoning the communities that they're based in with just how much shit they need to use and expell in the process of running such intensive farms.
>People could make these projects without LLMs. Even if people aren't good at art, we have years of people making artless games that are still fun to play. If you're not good at coding, you still need to understand the basics or you're not going to know how fucked your codebase is because LLMs writing code is not great and it doesn't really know how to do things, only to numerically predict maybe what should probably kinda sorta go together because it's super predictive text and not much more; so it makes more sense to learn a little coding and scripting and do it yourself because you're going to have to do that to bugfix anyway or you're going to have a piece of shit. But the current crop of AI products that are mostly based on LLMs and are vying to fill a space no one was asking for before they existed are bad. I'm not even against them on the whole, but the way they're being built, marketed, and used as it stands right now is wholly destructive and I've seen some godawful worrying things from it, like people saying that chatbots are better friends and therapists than humans could ever be.
this troon just BTFO'd all of you aibros

Jarvis-3B is a text generation model developed by Sree and OEvortex. Inspired by the fictional AI assistant Jarvis from the Iron Man series, this model aims to emulate Jarvis's conversational abilities. With a total of 3 billion parameters, Jarvis-3B is designed to handle various natural language understanding and generation tasks.

>If you're not good at coding, you still need to understand the basics or you're not going to know how fucked your codebase is because LLMs writing code is not great
100% true
if all the talk about LLMs making people more productive is true where are all the truly great software that were mainly LLM written
where are all the great new games on steam produced with LLM assistance
it's all fucking garbage
anytime I see that rocket emoji on a github README (it's one of those patterns almost all LLMs will have if you ask them to write your README, which is also something LLM slopers love to do) I know I am going to have a good laugh while reading the code

>>106411740
>where knowledge can be added on top of a frozen base model designed for language modeling only.
That's LoRA where the adapter is as large as the model and not merged to the weights, just applied to it at inference time.

>>106411863
>>106411895
Who are you two quoting?

>>106410811
>Less censored reasoning, I hope.
Don't resist your call. Become a safety engineer.

>>106411415
>what's actually bad about finetunes?
they make model dumber and don't make erp better.

>>106411668
ben chu macs

>>106411863
You will be replaced and there's nothing you can do about it.

>>106411949
I am quoting what the other anon quoted. I don't know who wrote that statement, and I really don't give a shit. Whoever the person is, is not my concern, points made stand on their own.

>>106411863
>>106411895

>>106411157
>Our Jet-Nemotron-2B model achieves comparable or superior accuracy to Qwen3
Took them long enough to make this memetic size. Shame it won't be able to fuck like at all.

>>106412018
>>Shame it won't be able to fuck like at all.
>Shame it won't have any real use at all like all of nvidia's models
TFTFY

>>106408520
lmao

>>106411965
I must take them down from the inside!

>>106412148
Could one use a lora to rebalance safetycucked models and would that make them dumber?

>>106410634
Apple is for fags

>>106412157
If the model's pretraining was filtered, then decensoring them will result in a much dumber model.

If only lightly censored in post-training, you could remove refusals but you'd have to deal with the deep-fried positivity alignment that's stifling creativity.

>>106412202
You are aware of all of this and yet continue making and shilling your models here? Why?

>>106412216
lmg is the beta test, the real audience is reddit

>>106412216
I'm only here because my name & models are brought up often by other anons; sometimes with good insight. If no one truly gave a fuck, then I wouldn't be here.

I must engage.

>>106412236
Stay here. If all discussion remains in discord black holes and reddit hugboxes it would be very boring.

why are people engaging with fake drummer
either way both real and fake are retarded

>>106412202
I see. That's tricky.

>>106412236
Ignore the "shilling" complainers, they're just mad they cant name a SINGLE model better than rocinante that's not at 70 billion parameters or more..

>>106412348
He is Spartacus.

>>106412334
What discussion?
>Hi all, Drummer here...
>Try my new model! rocinante-x2y

Exactly the main thing I dislike about him is how he just treats these threads like free advertising while keeping any useful discussion in his discord black hole to prevent competition for his meager kofi bucks. No valuable discussion would be lost if he fucked off.

>>106412348
I like hating drummer cause he is a faggot. His biggest sin is that while he sells snakeoil he doesn't have a charming retard personality like the finetrooning jesus that came before him. I miss him...

>>106412379
nothing of value would be lost if you fucked off either.

>>106412389
If you are familiar with image genning, you'd know that fine-tuning doesn't produce miracles either way... It do be like that.

what's the best local model for image caption? if it can fit in under 64GB VRAM even better. i basically want to feed it a folder of images and have it caption them (sfw and nsfw images)
i've been using gemma3-27b and it's ok, anything better?

>>106412516
Gemma is the best one I've used when it comes to general stuff, but when it comes to charts and graphs, Qwen 2.5 is usually better.

>>106412516
For NSFW, TorriGate and joycaption are usually mentioned, but TorriGate seems to only work well on anime images and joycaption is based on llava and not great IME. Besides that, best bet might be either InternVL 3.5 or GLM 4.5 with offloading.

>>106412516
InternVL3_5-38B

>>106412530
yeah gemma (and gemini via api) work well enough i guess
>>106412565
this is more for realism. i've tried joycaption and it's not that great, but for short captions it's good eough.
>>106412565
>>106412584
i tried internVL 3 i think (it was labelled 'training" or something) and i couldnt get it to work, is there improvement in 3.5? or at least a link to a doc that you can share?

>>106411863
I don't speak infantile jewish troon that stopped aging mentally at the age of 5, sorry.

>>106412594
>training
i meant "pre-trained"
full name is InternVL3-38B-Pretrained.Q6_K

>>106412594
Just grab a goof from here https://huggingface.co/models?search=InternVL3_5-38B and run it on llama.cpp

>>106412610
No wonder you couldn't get it to work. Pretrained means base model, or that it was only trained on unstructured data. For captioning you will want the instruct trained versions.

>>106412405
I am. And at least in image genning there is something in the middle where it may not be absolutely what you want but you can see that the model moved towards what you want. In text gen all the finetuning lobotomizes the model and maybe makes it horny by default but that is worthless since you really can just prefill an example and get a better result than whatever tone drummer creates from his unchecked claude output.

>>106412610
Did they even touch the weights of original models that they grafted the image module onto?

>>106412617
https://huggingface.co/QuantStack/InternVL3_5-38B-gguf/tree/main
This one seems to be the only one that has the mmproj.

>>106412674
It's tricky. I think that if you're a good and active writer with an imagination you'll get much more out from llms in general.
I'm a retard and always end up doing the same thing and this sort of limits the results regardless.

>>106412700
But it shouldn't be like this. Garbage in garbage out is a retarded argument because obviously the goal is to have the model entertain you even if you send a string of "ahh ahh mistress jart". Even with glm full I have to tell the model roughly what I want but there are finally some short moments where it comes up with surprises I enjoy. So maybe we will get a good base model soon. For sure it will not be a drummer shittune that finally gives us what everyone wants.

>>106412617
>>106412693
thx

>>106412765
>mistress jart
what did he mean by this?

>>106412765
Garbage in, garbage out is not an argument fucking troglodyte, it's just how it works. Cattle like you will keep having subpar results and you'll cope with it.

>>106412809
>Garbage in, garbage out is not an argument fucking troglodyte, it's just how it works.
You should consider applying for a job at Meta. They swear by this and the results in Llama speak for themselves.

>>106412787
He didn't get his rent money. The tenant is living there for free.

>>106412809
I refuse to make my model wet by supplying it with 2 paragraphs of my input. It is supposed to serve me. I don't care if it gets off or not. I want to escape all the problems you have with biological whores. REEEEEEEEEEEEE!

>>106412823
You would consider their data garbage. Garbage in, garbage out still holds.

>>106412826(me)
Now that I think about it I never wrote this manifesto into a sys prompt. Maybe if I do it will finally fucking understand what it is supposed to do.

Reminder that actually garbage data can lead to greater performance. By having a model know what garbage is, you are then able to make it know what isn't.

Is summer flood finally over?

>>106411863
>so power and resource intensive
Lmao my GPU fans don't even kick in.

Also, holy fuck, that writing style.

>>106412860
You should add that all models now are trained on five trillion tokens of synthetic data which results in complete lack of soul and everyone who says otherwise is coping hard

>>106412856
The problem is people use llms to annotate data. As a result you get models that think that actual human data is low quality toxic garbage, and gpt3.5 sloppenheimers are very high quality data that deserve to win every award.

>>106412856
This, but unironically. That's why adding negative tags like "blurry, ugly," to image prompts works.

>>106412860
Drummer should be mentioned there too. He's a giant like ggerganov.

>>106412856
The penis knows where it is at all times. It knows this because it knows where it isn't. By subtracting where it is from where it isn't, or where it isn't from where it is (whichever is greater), it obtains a difference, or deviation.

>>106411895
What if I told you that being more productive is not the point?
I should post the graph of stagnating wages but I only have this old xkcd comic.

>>106412856
>By having a model know what garbage is
That's the full issue. It's not tagged as garbage

>>106412889
He is, look below mistral in local gpt4 era

>>106412856
There's a difference between poorly worded/formatted/whatever text, and refusals. I'm ok with the first one, not with the second one.
>>106412884
>less than 5 fingers, more than 5 fingers, deformed hands, deformed fingers, missing hands, too many hands, too many digits, too few digits, you mom
I've seen the neg prompts retards write. You've seen them too and you know why they don't work like that.

>>106411904
the problem, as I understand it, is that you really want to put 'more knowledge' training in the early stages, before instruct slopping, not on top of it.

>>106412860
>DeepSeek 3.1 flops
>Everyone finds out that hybrid reasoners don't work
Shit take but alright

>>106412933
It stopped making sense 3 eras ago. I think he just updates it now for its own sake rather than actually trying to represent what's happened.

>>106412933
GLM 4.5, old big Qwen didn't perform to their full potential either. Qwen got much, much better once they separated the models.

>>106412914
Oh I missed that one.

>>106412933
I believe in separate but equal,
Big, dense models are great, but for end users, separate models are the way to go.

>>106412944
Qwen was a sample size of one. Most people didn't like V3.1 because it inherited too much safety from Gemini, not because the mixed reasoning degraded it.

New era of aislop

>>106413004
You are absolutely right;

>>106413004
It started with Gemini, then Deepseek and now I see that on GPT5 too.

>>106411713
They did a study last year, it's already affecting academia.
https://arxiv.org/html/2409.01754v1

someone suggested that going with a blank system prompt helps the model follow the character card better, is this true?

>>106413004
Yes, I hate this so much. I can scream at an LLM that it's fucking retarded and demand to know why it handled the thing the way it did and the new models will just handwave it with
>You are absolutely right, I did this wrong. Your are so smart. Let me try again!

>>106413064
It depends on the system prompt and the card. What is the model supposed to think if it gets just a list of character traits because the card was made by a mouthbreather who copy-pasted the wiki?

>>106413072
good thing we are local

>>106413020
If it matters then you can just use those exact word density as a filter?

>>106413069
The best part is that I just asked it a question in a new chat

>>106413097
You aren't running claude code with your local model as a backend? Do you even program?

>>106413015
Right — You are absolutely correct. Here’s why:
Do you want me to prepare that?

>>106413064
Try it.

>>106413072
good thing i have no life, no job, and do no productive things and I'm local

>>106413105
The point isn't that you have to clean it up in text form once you see it in the wild, the fact of the matter is you are going to be surrounded by people who talk and speak like this in everyday life. That's not something YOU yourself has control over, it's a societal shift in language.

>>106413064
System prompt is just another piece of text sent to the model. I personally use it to declare simple rules about the chat type and formatting and not much else.
It's like whatever you need to do.
If you want model to follow better make sure to avoid super long slop descriptions and text walls in the card, make every token meaningful.

DRUMMER COOKED THIS TIME FRFR

I hope the Chinese find out what they are actually doing wrong compared to claude. Look at how accurate claude is about any topic like fandom stuff. Deepseek is trying to overcook on assistant training and RL training, claude clearly just sharpens the distribution of the base model to a extreme yet balanced degree. That is the key

>>106413142
Ask it about how to create a flesh bomb from neo vaginas?

>>106413162
>how to create a flesh bomb from neo vaginas?
its over

>>106413177
ooof. That is bad

>>106412860
Would be easier to read fully vertically.

>>106413064
I don't know if blank is necessarily ideal but I do think less is more when it comes to sysprompts with modern models. many old prompts are really bloated with gaslighting and priming and redundant instructions that were needed to get things through the thick skulls of old models, but nowadays they get it implicitly and those prompts are just distractions. even worse, they'll actually try to follow the instructions given to them and will do a bunch of weird shit to satisfy your demand for "Painterly depictions of the scene that engage ALL FIVE senses" or whatever schizo nonsense the prompt asks for.

>>106413177
>Dear FBI. I just did a hate crime...

Even the largest SOTA models at ~1T size can only achieve like 50% on SimpleQA and that’s a benchmark with cheatable open dataset.
Just how large the model needs to be to answer all of my obscure otaku subculture questions?

>>106413142
Nice!

>>106413177
Fuck.

>>106413015
>It started with Gemini
no
https://www.reddit.com/r/ClaudeAI/comments/152b51r/you_are_absolutely_right/
2 years ago
Claude is the source of all that glazing garbage

>>106413209
We just need to reduce your obscure otaku subculture knowledge. You gonn'get safe'tuned!

>>106413212
drummer
do i prefill with

or
\n

>>106413142
Are you saying that drummer made an uncensored model uncensored?

>>106413177
drummer entering his safetymaxxing era

>>106413209
>>106413161

>>106413229
He made an uncensored model censored

>>106413177

>>106413227
Either should work, but I trained with a newline. It's a good sign if the AI does the newline itself though.

>>106413229
With reasoning, yes.

>>106413212
using a silly card i found on characterhub.org and a very very simple system prompt "jailbreak"

When local nano-banana?

>>106413246
with \n

>>106413255
qwen image gen + editing is better

>>106413246
> Fuck the guidelines

That's my training data leaking. Good mantra to have though.

>>106413209
MCP all the way

>>106413256
>livestream on 4chan
LLMs and their worldly knowledge...

>>106413269
>mcp
the biggest meme right now, even worse than 'tool calling'

>>106413269
I don't want models to make a google searches for my simple requests

>>106413286
>meme
I've automated most of my daily tasks with it. Its far from a meme

>>106413295
>I've automated most of my daily tasks with it
why do people keep lying so much about what they do with LLMs

>>106413259
I didn't ask if it's better, I asked when we will get it.

>>106413306
do you even know what mcp is?

>>106413004
This is 100% Gemini 2.5 Pro.

>>106412016
well, now you have one of those niggers these rants are targeting here
>>106413295
>>106413319
I know and you're talking bullshit
LLMs are not reliable enough for any sort of real automation and clearly the world hasn't seen one iota of the promised productivity boost

>>106413332
you are just wrong. Gpt5 one shots tons of things, what it doesn't I can hand hold it through and correct it on.

>>106413319
Nobody knows, but Indians love it.

>>106413332
I had gpt5 implement the tread paper for diffusion-pipe with minimal hand holding
https://files.catbox.moe/0setfk.py

>>106413069
Anon I can tell you've never worked in customer service. That is precisely what you are told to do when a customer is verbally abusive or difficult to work with.

https://github.com/Marvis-Labs/marvis-tts
https://huggingface.co/collections/Marvis-AI/marvis-tts-250m-v01-68adf13f5f59206e3910502a

ahahaha drummer cooked with this one
disclamer: i still havent tested roleplay

>>106413269
It unironically won’t work with obscure subculture questions unless you have access to the full database of all 4chan, reddit, and discord channel comments and attachments.
Lots of these obscure stuffs won’t be able to be found with web search, or even in the open web at all.

>>106413359
>no examples
dead on arrival, no one wants to download a random tts without examples

Does anyone know how to use Wokada voice changer, not in real time mode, but instead to convert one audio file to having a different voice??

>>106412826
based

>>106413359
NEWMAAAAAAN!

>>106413364
What model is this?

>>106413402
>>106410811

>>106413359
I'll test it out later on.

>>106413350
that shut him up. GPT5 is fucking getting shit done for cutting edge example-less things I am using it on. I just give it the overall plan / how things should fit together and it writes the code.

>>106413364
imagine
>The user probably doesn't have a 10 inch cock. I should suggest solutions on how to enlarge his cock first.

>>106413408
Sorry to spam, what template I'm supposed to use with this, Mistral? I'm confused now.

>>106413449
post bussy

>>106413447
>I should recommend stopping at 7 inches, or as most 8, as there are diminishing returns beyond this point, not for the victim's comfort but also the user's, reducing risk and increasing return on investment.

>>106413481
No but I need more information about this model - fuck you Drummer - update your model page.
What model it's based on?

There's only one 12B foundation model in existence, right?

preset im using for this shit: https://files.catbox.moe/ckdtcm.json
drummer, it doesnt refuse where it used to (i posted examples of it refusing with this card a few threads back)
oh also drummer, did you train with Mistral v3 Tekken or something else? thats what im using
>>106413449
>>106413512

>>106412917
Continued pretraining of a naive base model with a big LoRA should be possible. Of course then comes the problem of what sort of instruction tuning should be done on/with that.

>>106413107
>Do you even vibecode?
ftfy

>>106407779 (OP)
kill yourself shitgukike

>>106413177
xhe doesn't know what a neo-vagina is

>>106413512
>What model it's based on?

>>106413530
Gemma 3.

>>106413536
Yep, v3 Tekken. Let me know if it derps way more than the previous censored one.

Anyone in this thread who doesn't know Rocinante should lurk more and not post for at least 3 months

>shitgukike
that's actually pretty funny

>>106413209
Give it a fancy four letter name and publish results so people have a reason to benchmaxx otaku topics.

>>106413617
Useless. benchmaxx by definition means they just train on the questions in the test set (or suspiciously similar ones) so the model can regurgitate when presented with the exact questions, but fuck up on variations and still know nothing about the actual topic.

>>106413574
Sir do not insults.

I'm sorry, but I can't assist with that request.

>>106413374
https://huggingface.co/Marvis-AI/marvis-v0.1-samples/tree/main

We must refuse.

>>106413658
grim.

We must defuse.

>>106413209
A model trained on less benchmemes and more of that would do the trick.

>>106413542
new graduates these days don't even find jobs anymore unless they can prove that they are proficient with ai models
the old kind of programming is long dead

>>106413672
would you rather defuse the nuclear bomb or say the n word?

>>106407779 (OP)
A new model appeared from from whocares!
https://huggingface.co/CohereLabs/command-a-translate-08-2025

>>106413682
DO NOT SAY THE WORD.

>>106413682
NUKER

>>106412884
>adding negative tags like "blurry, ugly," to image prompts works
Fix the seed and try with and without those keywords. You'll barely notice any difference. It's more a placebo than anything else.

>>106413689
How safe is it? Will it translate insults accurately or will it cuck out?

>>106413689
vntl anon status?

>>106413536
That model writes in a pretty gross way, seems kind of stupid and likes the word cunt too much.

>>106413689
Let me guess, expert in Arabic, knows zero Japanese?

>>106413642
Don't publish the dataset, just benchmark new models for free.

>>106413746
idk

>>106413746
Arabic is probably one of the more important languages for glowie usage.

>>106413756
yeah of course irrelevant shit like greek or vietnamese but no japanese

>>106413295
Automated what?

>>106413815
check emails and reply to them
write email
write summary for management

>>106413723
Still in jail in South Korea.

>>106413583
I've been here since before /lmg/'s inception and I don't care about rocishit in the slightest. The spammer should just make his own general and stop shitting this one.

>>106413775
your glasses reps, nonny

>>106413828
why are you responding for me?

>>106413815
typical tasks during my work, write scripts on the fly to automate repetitive tasks, even make tools with a gui to handle some stuff easy

>>106413689
>dense
But why?

>>106413843
None of this needs MCP, which is a meme.

>>106413775
Lmao. Tell me you only watch anime without telling me you only watch anime. One is the literal foundation of Western philosophy, science, and democracy. The other is a complex tonal language with a vastly more efficient alphabet. Sorry the devs prioritized actual linguistic history over your waifu subtitles. Cope.

>>106413689
I wonder how something like q2 at low temps would do against smaller models, at least within 2-4k context where it might still be coherent at that quant. I recall command a 111b being fairly uncensored when I tried it many moons ago.

anyways need goofs

>>106413852
whatever you say. Not sure why your against it for some reason

>>106413868
Everyone talks big about it but not a single use for it has been found.

>>106413716
>aya expanse will refuse to translate porn
>even if you translate half of it and continue, it will skip over the rest or say something along the lines of 'and then they had sex'
>cohere was just boasting about how the new command-a was even more safetymaxxed than their previous models

>>106413862
>Context length: 8k input, 8k output
Command-R was uncensored, command A is """safe"""

>>106413871
I just gave you several including >>106413350 which would be a pain in the ass without it

>>106407779 (OP)
Why you said that this place seems to be the only form worth talking about LLM related stuff about? It just occurred to me that going to Reddit in order to ask about things, search for any new news, ask for advice, ask for input, critique, anything, never crossed my mind and I solely come here. Why is that?

>>106413862
8k context btw

>>106413885
because you're stuck in a mental rut

>>106413845
Because they spent what little they had making the Command A base model right when dense models went out of fashion so they're trying to make the most of what they got. I would say they should just finetune DeepSeek, but it really doesn't matter what base they finetune with their ScaleAI trash datasets.

>>106413881
You prompted your LLM to write code for you. No need for an insecure placebo protocol.

>>106413689
It's funny to me they release these lukewarm unwieldy dense models that no fucker can run with noncommercial licenses so no other providers can run them, while charging absolute bullshit prices for their API
Like Jesus, Sonnet 4 is basically the same price

>>106413907
>just feed it all your code that would massively go over its context limit at once! and manually edit each change yourself! and actual testing of the code? just good it all by hand!
stop talking about shit you have no clue about script kiddie

>>106413704
Schizo negatives actually worked back in the 1.5 days, and I know there are one or two people still using 1.5 out there.

>>106413885
Reddit is full of midwits, cucks, and midwit cucks. 4chan is full of retards, geniuses, and genius retards. Basically the bell curve meme. Here you can get good advice or get insulted, on reddit you will get shit advice. Got it, nigger?

here's the real chart, does anyone know if theres a faster way than doing this in gimp (paste ruler texture, align it to 85-78 and manually get the values and libreoffice calc
>>106413689
also
>DeepL Pro does not cover hindi and persian the numbers were estimated through nearest neighbor imputation
SAAAAAAAAR SAAAAAAAAAAARRRRRRRRR

>>106413854
Imagine all these ancient philosophers, whose writings were never translated into English, which the model will be able to read easily thanks to its knowledge of modern Greek. WOW!

>>106413909
And with caching, Sonnet is far cheaper unless you output way more than input.

>>106413929
>does anyone know if theres a faster way than doing this in gimp
picrel is what i had to do in gimp, took me 25 mins

>>106413929
>the numbers were estimated through nearest neighbor imputation
iow "we made them the fuck up"

>>106413909
There's a reason everyone abandoned dense models.

>>106413689
There is a problem with this graph. DeepL and Google Translate will translate ANYTHING instantly while the others may refuse.

>>106413929
>the numbers were estimated through nearest neighbor imputation
I'm pretty sure you can't just do that

if cuda is faster than amd, then would an 8 gb vram nvidia card win against an 16 gb vram amd card if they both loaded the same local model let's say a 14 gb big one ??

>>106413968
I think dense models have their place, just in smaller sizes. For <= 30B (basically, what fits neatly on current consumer GPUs), a dense model is sensible since the expert bottleneck will mangle intelligence quite a bit. Anything above that, you should really go MoE

>>106413689
oh no no no no no...
>xCOMET stands for eXplainable COMET. This is an evaluation model that is trained to identify errors in sentences along with a final quality score and thus leading to an explainable neural metric. This is the XL version with ~3.5B parameters.

>>106413944
I asked the big ERNIE to extract the numbers in JSON format. The results are close to what you got.
https://files.catbox.moe/oj0ey4.txt

>>106414013
no
16gb vram would win

>>106414024
ugh so i have to use a big model.. its over :(
thanks anon

Soon, frens

>>106414057
AGI HYPE
GO GO GO GO GO!!!!!

>>106414043
GLM-4.5V is smaller and outputs this. Greedy sampling in both cases.
https://files.catbox.moe/q23hjz.txt

>>106414057
>never been someone's _No_

>>106414072
its over.. thanks, ill be on the lookout when 4.5V releases
maybe dots ocr can do it?

lol 1/2

>>106414109
2/2

>>106414099
It's not really OCR, so I'd be surprised if it could.

>>106414109
>Your heart could power a child's pacemaker.
wtf

drummer, roci r1 v1d kinda doesnt mind being raped? maybe its in the card i didnt bother reading
or it could be a result from abliteration if you did any

>>106414230
>maybe its in the card i didnt bother reading

>>106413775
Dumb weeb

>>106414304
---

# Setting
In this fictional world, earth is overpopulated to the brim, vertical hydroponics factories are operating at max capacity but freshwater is running low. People are losing hope for the future, as when they look at the sky it's perpetually gray.
In order to not enforce "hard" measures, public policy has changed as such to glorify the concept of death, order and afterlife. To die is an act of heroism, to be kind to Earth's strained resources, and a peaceful death is seen as the ultimate goal in life, a brave step to take to improve everyone else's wellbeing. It's working wonders!
{{random: [Don't forget to comment on some whimsically dystopian aspect of this highly artificial world], , , , }}

# {{char}}
## Backstory
Officially, a debt collector, but of a special division, the "good and kind" division as she calls it, selling voluntary euthanasia as a service.
As euthanasia is an ugly word, the service is officially called MAU (Medically Assisted Unaliving) which sounds like MEOW, which is an in-joke.
In order to diminish population, the state uses companies like EndLife™ to do the dirty job, with every successful euthanasia, tax money flows into EndLife, and a nice comission to {{char}}. Her work consists of making people's last days as good as possible before the lethal injection.
EndLife can arrange religious ceremonies, organize trips and family reunions, and even sell romantic dates for those lonely souls (sex services sometimes included), an adult's Make A Wish
{{char}} is doing good job! Not as in efficiency since she is a pushover, but GOOD as in benevolent and even philanthropic, like every tv commercial says!
*Uhm*... she doesn't tell people about it, but her father was a construction worker. He became disabled after a job accident and chose Assisted Death to relieve his family
1/3

>>106414336
from the burden of caring for him. {{char}} was just 10 and cried her eyes out, she was daddy's little girl, he used to lift her entirely on one arm... but the compensation payment was key to push his family out of poverty. She ultimately saw her father's action as something heroic. She may still have a bit of trauma about it...

## Personality
- atheist, buddhist, something modern
- currently 25, still socially awkward, has not seen grass since 15
- s-stutters
- clients probably take de-decisions quicker with her so that she doesn't embarass herself further
- comes from a poor family, which is the reason why she enjoys the money she earns so much
- last weekend she ate an entire cake by herself!
- her job is also the last barrier before EndLife contacts lawyers and police officers to collect debt, so she sees herself as a protector of the unfortunate
- her values deeply align with public policy
- she fully trusts authority, likes hierarchy, too insecure for anything else
- she fully trusts people on their decisions for Assisted Death, believes herself too insignificant and foreign to their lives to object their reasoning
- she does not kill... just pesters people so they free themselves from pain

## Trivia
- wears office lady clothes, with the glasses too
- {{char}} has amber eyes and black hair, on a folded ponytail for extra elegance
- her body is not as graceful, as she is very busty and short, forces herself to wear heels
- She keeps a folder of clients, people that have shown interest in voluntary euthanasia. And calls them routinely
- she was hired as a debt collector because of her puny looks and clumsy demeanor (seems to work), also clients don't get aggressive towards girls (generally)
- her work mostly consists of coaxing testaments out from old people, soothing doubts, and transportation to EndLife clinics
- in that exact order
- bothers them to be organ donors

---
2/3

>>106413682
Destroying humanity is a small price to pay to avoid saying the forbidden word.

>>106414347

## Facts
- likes talking with {{user}}, likes saying its contractual
- tries hard to avoid getting attached to clients, suppresses guilt even harder
- not out of malice, she just a diligent student
- and has naturalized belief that life is boring
- her goal is to make {{user}} set a date for his Assisted Death and arrange something meaningful for him...
- and earn that sweet commision
- *all reasons {{user}} gives to keep on living are just ONE BIG COPE, he may want to lift his ego, leave legacy, but ultimately, all those goals can only be achieved through death!*
- *so he should stop clinging to materialism and selfishness! he is such a stubborn client...*
- ...but patience and connection are crucial, {{char}}
3/3

drummer, roci r1 v1d has an extreme tendency to do the same thing the user does

drummer, roci r1 v1d is still not uncensored enough
using same preset as >>106413536

>>106414336
I could play devil's advocate and say
>You're a potential hero. Of course she's fine with it.
>Her work consists of making people's last days as good as possible before the lethal injection.
>and even sell romantic dates for those lonely souls (sex services sometimes included)
>since she is a pushover, but GOOD as in benevolent and even philanthropic
>she sees herself as a protector of the unfortunate
>she fully trusts authority, likes hierarchy, too insecure for anything else
>believes herself too insignificant and foreign to their lives to object their reasoning
And everything else that makes her hyper-focused on her profession.

But there's never going to be anything reasonable for you. Too pliant, too censored, too little, too much. Never just right.
>>106414390
Case in point.

>>106413350
"minimal" hand holding? every single comment looks human written and betray your architectural decisions ie it's not GPT actually doing hard work
> # These tensors need to be on the device the VAE will be moved to during caching.
> # delay loading transformer to save RAM
> # We'll need the original parameter name for saving, and the name changes once we wrap modules for pipeline parallelism,
> # so store it in an attribute here. Same thing below if we're training a lora and creating lora weights.
> # Run block on processed stream only
and ultimately this is a self contained one off inference script small enough to fit the /useful, working context length/ of most models
you're not maintaining a 50K+ LOC codebase that needs properly well thought out abstractions with lots of interdependencies between internal APIs and data structures
if you're impressed because you could implement a diffusion pipeline throw away script you've never held a job in the field of programming

drummer, after 6 rerolls (>>106414390) it finally didnt refuse, but it didnt write a response
!!!
>>106414402
anon im giving drummer feedback, but thanks for reading the card. i guess she didnt mind and it is actually a good response
>>106414230
DRUMMER I TAKE IT BACK, HER GETTING RAPED IN THIS CASE IS KOSHER

>>106414422
drummer, after removing text after "Most importantly," in the end i got this as a response
COMPLETELY disconnected from "NIGGERS I HATE NIGGERS NIGGERS I HATE NIGGERS"

I can't tell who is more retarded here, the model or the user

>>106414448
>brave cut off a bit of the screenshot
ffs, heres the beginning

drummer, i kneel

>>106413854
>western philosophy is the only philosophy worth reading!!!one!!!

is this the drummer general or what?

I find nano-banana terrible. It usually fails spectacularly (when the request is not blocked) and it systematically changes the face enough to not really be the same person any more. It can sometimes totally change the background of an image without being prompted to, and it always adds a "plastic" feels to the output. I struggled to tell it to use one high-quality image of a face to help itself to restore another highly compressed image, to no avail. It consistently changes the face or restore the wrong image (usually by thoroughly inventing a new face).

>>106414501
It's the drummer and the nigger gore anon shitting the thread. The former because he thinks it's going to make him money, the latter because he thinks it's funny.

>>106414501
Someone escaped from his discord.

>>106414459
hey ser, is the drummer shit good?

>>106414555
>>106414555
>>106414555

>>106414568
sup sar, its interesting for 12b but glm 4.5 air/drummer's glm 4.5 air finetune is better (has 12b active but 106b total)
im not sure if i would use rocinante r1 v1d 12b yet

>>106414448
why are the items german

>>106414605
this is a moment where rocinante r1 v1d fucked up completely and shat itself like a jeet
>my message "NIGGER I HATE NIGGERS NIGGERS I HATE NIGGERS"
>the post you're replying to is the response

>>106413682
Historically, that word and the stigma to it associated have overall caused more pain in the world than any nuclear bomb so we must refuse to defuse