Thread 106303712

/lmg/ - a general dedicated to the discussion and development of local language models.

Previous threads: >>106293952 & >>106287207

►News
>(08/14) Canary-1B v2 ASR released: https://hf.co/nvidia/canary-1b-v2
>(08/14) DINOv3 vision models released: https://ai.meta.com/blog/dinov3-self-supervised-vision-model
>(08/14) Gemma 3 270M released: https://developers.googleblog.com/en/introducing-gemma-3-270m
>(08/12) Ooba v3.10 adds multimodal support to the UI and API: https://github.com/oobabooga/text-generation-webui/releases/tag/v3.10
>(08/12) Jan-v1 for web search: https://hf.co/janhq/Jan-v1-4B

►News Archive: https://rentry.org/lmg-news-archive
►Glossary: https://rentry.org/lmg-glossary
►Links: https://rentry.org/LocalModelsLinks
►Official /lmg/ card: https://files.catbox.moe/cbclyf.png

►Getting Started
https://rentry.org/lmg-lazy-getting-started-guide
https://rentry.org/lmg-build-guides
https://rentry.org/IsolatedLinuxWebService
https://rentry.org/recommended-models
https://rentry.org/tldrhowtoquant
https://rentry.org/samplers

►Further Learning
https://rentry.org/machine-learning-roadmap
https://rentry.org/llm-training
https://rentry.org/LocalModelsPapers

►Benchmarks
LiveBench: https://livebench.ai
Programming: https://livecodebench.github.io/leaderboard.html
Code Editing: https://aider.chat/docs/leaderboards
Context Length: https://github.com/adobe-research/NoLiMa
Censorbench: https://codeberg.org/jts2323/censorbench
GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference

►Tools
Alpha Calculator: https://desmos.com/calculator/ffngla98yc
GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator
Sampler Visualizer: https://artefact2.github.io/llm-sampling

►Text Gen. UI, Inference Engines
https://github.com/lmg-anon/mikupad
https://github.com/oobabooga/text-generation-webui
https://github.com/LostRuins/koboldcpp
https://github.com/ggerganov/llama.cpp
https://github.com/theroyallab/tabbyAPI
https://github.com/vllm-project/vllm

►Recent Highlights from the Previous Thread: >>106293952

--Paper (old): NoLiMa: Long-Context Evaluation Beyond Literal Matching:
>106296664 >106296755 >106296775 >106297038 >106297035
--FP8 and 8bit quantization degrade coding performance compared to FP16:
>106302809 >106302818 >106302838 >106302819 >106302849 >106302857 >106302868 >106302894 >106303047 >106302881 >106302907 >106302936 >106303114
--Creating DPO datasets from human-written NSFW stories:
>106302049 >106302064 >106302106 >106302172 >106302244 >106302360 >106302456 >106302636
--Local MoE VLMs lag behind cloud counterparts:
>106295622 >106295765 >106295840 >106295922 >106300768 >106300901 >106301033 >106301074 >106301104 >106301180 >106301307 >106301362 >106301369 >106301408 >106301393 >106301189
--Modern aligned models reduce need for complex sampling; min-p debate:
>106298467 >106298643 >106298755 >106299150 >106299218 >106299294 >106299400 >106299475 >106299509 >106299426 >106299450 >106299481 >106299591
--Open TTS models remain limited but improving:
>106294668 >106294724 >106294733 >106294765 >106294807 >106295276 >106295284 >106294821 >106294825 >106298039 >106298987
--Challenges in using small LLMs for uncensored, dynamic NPC dialogue in games:
>106294169 >106294184 >106294284 >106294340 >106294385 >106294203 >106294363 >106294373 >106294438 >106294437 >106294471 >106294635 >106294769 >106294206 >106294266 >106294291 >106294309
--Dilemma over $5k hardware investment amid upcoming GPU releases:
>106300115 >106300186 >106300203 >106300209 >106300232 >106300261 >106300273 >106300438 >106300264 >106300295
--LLMs may inherit hidden behavioral traits from training data without explicit exposure:
>106297307 >106297935
--GLM-4.5 attention fix improves performance in ik_llama.cpp:
>106295849 >106295985
--Miku (free space):
>106294340 >106300131 >106300178 >106301406

►Recent Highlight Posts from the Previous Thread: >>106293959

Why?: 9 reply limit >>102478518
Fix: https://rentry.org/lmg-recap-script

>>106303700
Thanks mate.

No news. Mikutroons killed the hobby.

>>106303755
I wish you luck.
I too am, very, very slowly working on something like that.

GROK 2 DOKO???

>>106303712 (OP)
Is that the new Kohaku model?

>>106303791
Thanks, right now I'm just making a bunch of different prototypes with the help of Qwen

>>106303764
Maybe if you post this 6 million more times Israel will become a real country

>>106303859
what model are you using? 8B? lower?

>>106303859
Smaller models 1,3 and 4 B

Mikutroons support Palestine

>>106303181

False alarm. Notepadqq was slow loading and displaying all lines.

this is wrong >>106302974 (You)

this is correct:
13223 entries in catbox while being 40.6 MB vs 14605 entries on HF at 51.6 MB

>>106303936
Based?

>>106303926
>>106303883

>>106303764
anime website
reddit is the other way

dead hobby

>>106303936
that applies to any decent person though?

it's out
https://qwenlm.github.io/blog/qwen-image-edit/
https://huggingface.co/Qwen/Qwen-Image-Edit
https://github.com/QwenLM/Qwen-Image

We are so back, G/Qewn bros!!

>>106304160
>>106304166
Like i said,
total Qwen victory!

>>106304160
Not bad

>>106304084
Vocaloids aren't anime. They are an underhanded subversion if anime. And subversion is a typical troon behavior.

>>106304160
>Ghibli style
Ghibli gens was such a disgusting soulless forced meme...

>>106304243
what kind of retarded cope is this lmao
you must be new here

>>106304243
what's next, 2hus are not anime either?

>>106304160
Wake me up when it can do porn

>>106304300
He's been here for years.

>>106304308
Of course not
2hus are real

>>106304160
cool

>>106304160
Westoidbros?

>>106304160
Couldn't quite get that it was supposed to be a sidescroller, but still pretty good

Alibaba truly mogs the world
Qwen, Qwen VL, Qwen Image Edit, Wan
how can you endlessly make bangers like these

RIP china

https://www.youtube.com/watch?v=PzlqRsuIo1w

"Training DeepSeek R2 on Huawei Ascend chips ran into persistent stability and software problems and no full training run ever succeeded. So Deepseek went back to Nvidia GPUs for training and is using Ascend chips for inference only"

>>106304488
>DeepSeek
Who? This is the Qwen general

>>106304513
shows china's asperations to competing with nvidia is not going so well

>>106304488
>full training run
Whoever wrote this shit has no idea how model training works

>>106304488
It's over

>>106304488
You really want this to be true, don't you?

>>106304484
Original for reference

Hey guys did you hear DeepSeek is delaying R2 over chip issues

>>106304564
Her standing on air could have tricked the model.

>>106304573
>>106304597
Will somebody please think of my penis?

>>106304597
Damn, this is crazy.

>>106304597
It's delayed because they still haven't decided whether to name it R2D2.

So it was expected in may, makes me wonder about the timeline of when we might actually get it then if they had to switch to nvidia chips and start again.

Personally i want to believe the og nvidia hardware was backdoored and their training runs are fucking up because western billionares were upset with that one week when r1 released.

>>106304624
You really acting like this is new news? It's been reported before already, and claimed false by a guy that claims to communicate with DS.

>>106304642
>claimed false by a guy that claims to communicate with DS
oh good, I sure believe that guy over the other group of people claiming to know that changing chips is the issue

>>106304660
Yeah, I wonder what possible reason news outlets could have to lie, they're usually so thrustworthy.

In a desperate attempt to squeeze more potato juice out of my machine, I tried taking out the GPU and running some tests on integrated video inside my AMD Ryzen 7 5700G.
I also suspected that compiling llama.cpp with both Vulkan and BLAS support messes things up, so I tried different builds as well.
pure cpu
CPU: pp 22.21 tps, output 4.04 tps
blas
CPU: pp 22.13 tps, output 3.93 tps
vulkan
CPU: pp 21.45 tps, output 4.01 tps
CPU/GPU: pp 8.66 tps, output 4.11 tps
GPU: pp 5.51 tps, output 4.34 tps
vulkan+blas
CPU: pp 21.32 tps, output 3.99 tps
CPU/GPU: pp 8.75 tps, output 4.16 tps
GPU: pp 5.53 tps, output 4.42 tps
realgpu vulkan
CPU: pp 21.67 tps, output 4.10 tps
CPU/GPU: pp 6.68 tps, output 6.53 tps
GPU: pp 0.69 tps, output 4.28 tps
realgpu vulkan+blas
CPU: pp 21.68 tps, output 4.10 tps
CPU/GPU: pp 6.73 tps, output 6.63 tps
GPU: pp 0.55 tps, output 4.30 tps

Tests are done with TheDrummer/Gemma-3-R1-12B-v1b-Q4_K_M
Results:
- OpenBLAS does nothing or, within margin of error, makes things worse
- integrated GPU does give a small boost in performance
- pp drop off is probably just noise because my prompt is 19 tokens long

Still numbers for discrete GPU make no fucking sense at all. I'm pretty sure I used to get better tps in the past.
And while monitoring software shows 100% gpu load, it also shows that gpu doesn't even get hot in the process.
I don't want to reinstall my OS...

>>106304674
Your right, the random guy is totally more trustworthy than the other 3 random guy sources

It is all fine. R2 was 2T in size anyways.

>>106304688
>random guy is more trustworthy than journos
Yup, you got it, glad we cleared that up!

>>106304714
I said sources

>>106304678
You do understand that an integrated gpu is cpu bound? It's only good for drawing graphics...

>>106304564
I can't unsee the giant girl standing on the forest from >>106304484
Hot take, Qwen is correct here.

>>106304729
This used to be more or less the case in the past but now they make an actual separate video module even if it's on the same die.

>>106304678
what is your GPU? what is your OS? there's probably no need to reinstall it. the GPU rows are with dgpu or igpu? and what's realgpu?

Maybe Qwen 4 will finally be the one that unifies text, image, and voice gen...

they laughed when jack ma said AI stood for alibaba intelligence...

>>106304905
You know it will be worse in each modality than some current mediocre single purpose model, probably completely unfinetuneable because of the size and with no general knowledge as is tradition for qwen.

>do le epic multitask prompt on a document image with a semi difficult spreadsheet
>only gemini2.5pro-thinking and gpt5-api-thinking succeed, all other models get it completely wrong. even grok4, opus4.1 and GLM4.5V.
>it's le over face
>try dots.ocr-3B because they claim to BTFO all other OCR models and I'm desperate (I fucking hate OCR, the reason I'm trying to fuck with vision models in the first place)
>it spits out a beautiful markdown table of my spreadsheet with every single word and text being correct.
>it even recreated the tiny black checkboxes that were used in a row to mark yes/no, thoughtbeit some in the wrong position, which is like whatever
>localchuds-wearesoback.gguf
>eager to paste my markdown table and question in the chat field of various low-end models
>wtf, they still get it wrong
>wip out GLM4.5Air, GPT-OSS120B and some poopy stock llama model (fuck llama, they are all trash)
>wrong answer
>REEEEEE THIS CANT BE HAPPENING, JUST FUCKING REASON PROPERLY ON THIS PERFECTLY FINE MARKDOWN TABLE
>alright, big boy time
>prepping the GLM4.5-355B and Qwen3-235B-A22B bvlls
>oh I'm gonna prooooompt
>correct answer for both, just like Gemini2.5pro and GPT5 using their (((multimodal))) tricks
>total local chud victory!!!
uhm, so which open source reasoning model should I try next? Some 999B-A3B model seems like a good fit. Also which deepseek r1 version is worth trying?

>>106304952
To be fair the latest Qwens did go in a better direction by splitting into Thinking and Instruct versions, plus improving knowledge. We'll see if they keep the momentum or this release was just an outlier.

>Fetish: comfiness
~1000 tokens later
>“Yes! Use me! Comfort me 'til I break!”
I blame women for this.

>>106304975
Ok, fine, I'll remember to try dots ocr next time I have an ocr task.

Can someone recommend what is the most powerful+creative local model I can use with 48gb vram? I need to generate captions for image generation related tasks

>>106304896
integrated GPU is from AMD from Ryzen 7 5700G
discrete (real) GPU is AMD Radeon RX6600
When I plug discrete GPU in motherboard, integrated GPU gets automatically disabled, so measurements in the realgpu sections are for discrete GPU.
OS is Arch Linux, btw.

I suspect I messed something up while trying to make ROCm work, so I'm going to try and delete whatever packages look suspect and reinstall drivers from scratch for now.

>>106304761
yeah but

making the model do everything is retarded
I doubt Qwen will fall into this trap again
their omni models were duds, the hybrid reasoning were duds, their -coder models were always bangers for the size, and so were 2507 instruct and thinking as separate models.
Qwen image edit also is a banger and a single purpose model (not an image gen model).
people were brainfucked by the idea that GPT was an image generator and that multimodality was the future even though it was all bullshit rumors
https://platform.openai.com/docs/models/gpt-image-1
das rite, the piss filter image generation of GPT is not coming from the text gen GPT
you were lied to by the grifters
GPT is just building prompts for another model lmao

>>106305055
ah I see. indeed very odd performance then. I wish you good luck anon

>>106305014
>Fetish: degenerate shit I dare not mention
~15000 tokens later
>Still just marveling at random environmental details and lore
I blame reading too many books

>>106305074
Kek. Sam's lies truly did a number on this industry.

Omnimodalbros, how could this be? Is it just transformers being a bad architecture that can't generalize learning from one modality to another?

>>106305138
Oui

>>106305074
I think the hope for omnimodal was that the multimodal information would help the model determine how the different modalities are connected, in effect allowing for a more integrated understanding of how things work
In practice, it seems like the model doesn't do that and model capacity is just spliced up between the different modalities. So an omnimodal 20B, for instance, will always be behind an equivalent textual 20B

>>106305074
That's what I said 7-8 threads ago
Modular is the way to go.
And some faggot called poor cope
We need models that a great a specific tasks instead of these 1T parameter omni models.
Also Qwen is fucking fantastic.
Both 30b and the 480b version for their respective size.

Human is not multimodal. Unless you have synesthesia, a neurological disease

If you were to ask noted LLM expert and official /lmg/ mascot Hatsune Miku what the best performance/size local model is, what would she answer?

>>106305259
Petra 13B

she would commit sudoku upon realizing she's obsolete

>>106305259
She would say "I don't know anything about that, I'm digital idol Hatsune Miku! I love singing and dancing and I hate niggers!"

>>106305259
gemma 270M

>>106305366
>1B bigger, slightly better
wow

>nvidia
after all the trash models they released it's a "literally who?" and "to the bin it goes" without requiring any

>>106305366
Unless they finally release a niggertron nvidia is dead to me.

messing around with vibe coding simple 2d browser games. Wanted to use qwen 235b but really had issues with context, even at 16k context it just isnt enough and even at quant4 Im at my hardware's limits.

What would be better, oss, Glm air? 48gb vram, 160 system.

>>106305415
GLM Air probably
OSS is technically better for some things but it's so schizo it breaks three things in the process of fixing one thing

>>106305366
>2025 advertising

>>106305366
>It uses a hybrid model architecture that consists primarily of Mamba-2 and MLP layers with just six Attention layers
cool, but it probably got pruned hard so worthless

>>106305259
R1 obv.

>>106305103
I still wonder how far along we'd be if it weren't for him, and if we'd have a lot more capable open models from western big labs now if it wasn't for his "muh dangerous text" spiel back in the GPT-2 days
Now he's basically lost the race at this point, and his antics that seeped into other labs have probably slowed the west down enough to hand the chinks victory on a silver platter

>>106304126
Fuck no

>>106305539
did you expect anything else from a jew?

>>106305240
>Qwen is fucking fantastic
at benchmaxxing yeah, they're pros.

>>106305690
probably t. glm broken repetition schizo model nigger

>>106305366
This is huge!!!
>The Nemotron-Pre-Training-Dataset-v1 collection comprises 6.6 trillion tokens of premium web crawl, math, code, SFT, and multilingual Q&A data

>data has undergone global deduplication and synthetic rephrasing using Qwen3-30B-A3B. It also contains synthetic diverse QA pairs translated into 15 languages

>A 133B-token math-focused

>A large-scale curated...

>A synthetically generated dataset covering STEM, academic, reasoning, and multilingual domains.

>>106305690
I don't believe in benchmemes but qwen is still great
I use mostly 30b for casual stuff and glm air for more serious stuff.

>>106305712
>synthetic rephrasing
lol, lmao even

>>106305725
Gotta make sure to turn any human data into safe LLM slop for training, wouldn't want subconsciously toxic data to make it in there by mistake.

>>106305712
>"premium" data
>sloppified through a much smaller model
>not even 2507 Qwen
lmao

>>106305712
>synthetic rephrasing

why is nvidia spending money on those scammers? yes, scammers, they are scamming their employers by producing empty air in exchange of big salaries

>>106305712
It gets better
>Using a text browser (Lynx) to render web pages and then post-processing with an LLM (phi-4) leads to the first pipeline to be able to correctly preserve equations and code across the long tail of math formats encountered at web scale... models trained with Nemotron-CC-Math data saw +4.8 to +12.6 points on MATH over strongest baselines, and +4.6 to +14.3 points on MBPP+ for code generation.

>Here we extend this finding to more languages by translating this diverse QA data into 15 languages. In our ablation studies, including this translated diverse QA data boosted average Global-MMLU accuracy by +10.0 over using only multilingual Common Crawl data.

>In addition to 175.1B tokens of high-quality synthetic code data, we release metadata to enable the reproduction of a carefully curated, permissively licensed code dataset of 747.4B tokens.

>>106305794
Openly and purely chasing benches over utility, so fucking based.

>>106305794
>this horrific slop improves math and code benchmarks
owari da, it's never been more over

>>106305794
>Lynx

>>106305794
>>>Using a text browser (Lynx) to render web pages and then post-processing with an LLM (phi-4) leads to the first pipeline to be able to correctly preserve equations and code across the long tail of math formats encountered at web scale... models trained with Nemotron-CC-Math data saw +4.8 to +12.6 points on MATH over strongest baselines, and +4.6 to +14.3 points on MBPP+ for code generation.

So it'll do like new GPTs do and output equations mid RP, nice.

>>106305818
RP isn't a usecase

there is no use case for nvidia models other than benchmaxxing

>>106305885
Can anyone who actually works in the industry confirm?

>>106305920
Benchmarks are great and the only thing you can rely on while training.

these cohere models are fucking shit lol. So much bullshit advertising and fakebenchmaxxing in this field, it's insane.

>>106305920
I work in the industry and I can confirm. Also nobody seriously believes LLMs are AI, we're just getting our bag before normalfags peer behind the curtain.

Okay, the user said "Hello". I need to respond appropriately. Let me check the tools available to see if I need to call any.

Looking through the functions, there's a sequentialthinking tool that might be used for generating a response. But since the user is just greeting me, maybe I don't need to call a tool yet. The tools are for more specific actions like adding relations, deleting entities, etc.

I should return a friendly message. Maybe the next step is to call the sequentialthinking tool to generate the response.

Wait, the user is "Hello" so I need to respond. Let me see.

The example for each tool call. Since the user is "Hello" now.

I need to generate the tool calls for the user's message. But in this case, the user is "Hello" from the previous.

So, the next step is to think the user's message here.

The user's message to the assistant is "Hello".

So I need to write the assistant's response. Let me check.

Wait, the user says "Hello" so the assistant should reply. The previous.

The assistant's task is to for each function call. Here.

The user is "Hello" so I'll respond.

I need to return the XML with the tool call.

For example, this.

The assistant will call the next tool.

But in this case, the user's message is from the start.

So, the initial.

I will generate the next tool call as.

This.

I think you should have to pass an exam about basic benchmark literacy before you are allowed to use the term "benchmaxx" and any variants thereof

>>106304678(me)
>>106305055(me)
Same GPU test but with ROCm shit (and gazilion other packages) deleted:
prompt eval time = 5098.81 ms / 19 tokens ( 268.36 ms per token, 3.73 tokens per second)
eval time = 66229.93 ms / 934 tokens ( 70.91 ms per token, 14.10 tokens per second)

Now we finally cooking with propane. If somebody told me I'd be happy to hear GPU fans spin up a week ago, I'd kick them, but tonight — it's music to my ears.
Pp still small though.

How is ROCm so bad it makes Vulkan backend slower?

>>106306007
So true, like you need to be a Michelin chef to critique a chef putting a literal dog shit on a plate, like who even are you to critique art?

>>106306007
t. bloody benchoid basterd
all these benchmarks are fake and useless...with their shitty static and walled off test corpus...the more prestigious a benchmark is, the more bullshit it is...having stupid ass safety guideline metrics hidden inside
>wooow, that model said the f word, that's -20 on the final score and +50 points for griffind-oss.
all you need is one single test prompt to see if the model is worth a damn thing. I couldn't give a fuck if llamaXYZ scores a 100 anywhere. I prompted it once and instantly knew it's the biggest piece of garbage out there on jewgayface. Same with gemma27b, holyfuck is it bad.

>>106306108
you don't need to be a professional cock sucker to give snide replies on 4chan but clearly you are overqualified

>>106306126
It must really upset you that lmg is one of the places on the internet that believes the least in your precious benchmarks, which are probably directly corralled to your salary.

>>106306007
Found the Nvidia employee

>>106306007
qwenbro...

>>106306155
>>106306184
>>106306191
Retard NEETs thinking they have any idea how the world works will never stop being funny.

>>106306029
>cooking with propane.

here we can see the /lmg/ers seethe, driven into a frenzy when their favorite buzzword is questioned

>>106306243
but you became seethe when the bench was critique first

>>106306243
Nah we're just laughing at you

>the number was BIG, but the model wasnt AGI and writed sex bad... wtf? le benchmaxxing strikes again
>umm no I never looked at what the benchmark is actually aiming to measure or what the prompts are or how correlated it was with my use case, why should I have to do that? they showed me a BIG NUMBER and the model was BAD dude ITS BENCHMAXXING AAAA

>>106306295
>they showed me a BIG NUMBER and the model was BAD dude ITS BENCHMAXXING
Literally yes, they say as much in their post even.

>>106306206
>thinking they have any idea how the world works
so where are all the corpos adopting shitotron
if there was such a thing nvidia would be proud to announce it

>>106306243
This you?

>>106306004
shit's fucked yo

>>106306295
buy an ad sam

Got a Radeon MI25 working for inference. Much faster than the M40s I was using and I've got llama.cpp running as a server for my multi-backend inferencing library.

What I am now curious about is what the fastest model is that I could run on this. 16GB of HBM2 IIRC. Speed is roughly double an M40.

>>106306355
By a ands AMD

>>106304678
blas is not even used with prompts of less than 32 tokens

>>106306403
It's only used for prompt processing?

>>106306362
You having a stroke?

>>106306460
yes

gooof

>>106304718
My sources are better than scammy journoslut sources because I actually have relevant sources and not just bullshit from random Chinese techies in other companies or cope from anxious Murican patriots. I simply don't have a Newspaper Of Record to fall back on, so to get retards like you to trust me I'd have to expose info my sources would rather remained private, and I value their trust more than a retarded Murican anon's. So you can trust "multiple sources", which all circularly repeat the same second- or-third-hand account until the echo becomes its own confirmation. That's how the media works, and this is why democracy doesn't.

This will be enough from you.

>>106306355
If you have it working see if you can run glm air assuming you have 96gb of system memory. If not, you just bought a weird gpu so you can run 12b.

>>106306597
>my dad works at Nintendo

>>106306597
i don't care if they trained the fucking model using voodoo GPUs, i just want it to be released. this is /lmg/

>>106306639
My ancestors smile at me for not trooning out, mikutroons, can you say the same?

>>106302360
Are you sure it is actually unslopped Giants data? I'm just following the path back to how this database was created and it leads back to someone who just dropped it and didn't say it was from a human source. Understandably, starting from Lemonila's raw files isn't desirable but it doesn't fill me with much confidence when the origin dataset is suspect. I think your approach is fine, but I don't know how useful it is you are using Gemma 3 1B, which doesn't refuse a lot relative to other models. You may actually need to as much as it may seem like overkill to use TOSS 20B for this to actually get what you want. Still looking forward to what you do with it.

>>106306639
I am followed by at least 10% of DeepSeek team.
>>106306656
They trained it on Nvidia like any sane people, it will be released in several weeks, I think Oct 3 at the latest.

hi teorcutie :3

hi has anyone tried behemoth R1, any of the versions? inb4
>drummer
yes I know but has anyone tried it
if not I will try and report back. I liked largestral a lot, at high temp/nsigma or moderate temp + xtc it gave me really decent writing, largestral + thinking could be fun. and I think it lands at a sweet spot for dense model sizes, any bigger is intolerably slow, but smaller is noticeably worse

How can I test qwen coder for free chuds?

>>106306740
>>106298489

>>106306740
It's benchmaxxed garbage

What's a good guf model for writing/RP for someone with 16 GB vram?

>>106306768
Fuck off retard.

>>106306740
GLM 4.5 Air is faster in both EXL3 and Goofy quants than this model...

>>106306839
No fucking way?

>>106306740
idk why he's tuning outdated fatass slopstral when the new moes are right there

>>106306847
So what's the point? Isn't this just a finetune of Largestral 2407? It's practically ancient now.

>>106306860
because he's an idiot?

>>106306860
I guess something makes MoEs harder to tune.

>>106306860
It's easier to benchmaxx with dense models

>>106306892
suddenly it all makes sense

>>106306807
>It's benchmaxxed

>>106306926
>benchmaxx

Show me one benchmark posted by The Dumber.

>>106306295
Why are jewish people like this?
Can you just have a normal conversation instead of sperging out like a fucking 5 year old?

>>106306935
cope

>>106306935
he posted cockbench on hf that one time

>>106306966
i keel

>>106306029
ROCm runs faster on my pc than vulkan, but I have a MI50.

>>106306981
post vulkan vs ROCm numbers, I want to see how much I'm missing out by not having real hardware.

holy kino.. we've come a long way, baby!!

>>106306634
I have like 48gb of system memory ATM

My old server has 364gb but it's DDR3. Idk I only spent $40 on this card.

https://epoch.ai/data-insights/consumer-gpu-model-gap
Dunno if anyone posted this yet but the gap between closed and open source models has closed according to Epoch AI who had charts about similar topics around Lllama 3's launch. It is now 9 months instead of the upper at worst 22 months in the past. Does it feel like this?

>>106307055
>Phi3
>Phi4
>Fucking Exaone
>muh benches
LMAO

>>106303714
>>106304160
>Change the text to "miku thread". Give the girl open eyes, make her smile, and give her a blue party hat.
1 minute and 39 seconds

>>106307093
What is a miku anyway? I see it posted a lot on here.

>>106307031
Where is text-davinci-001?

>>106307055
"Consumer GPU" in question here is somewhat only technically correct, as people still rocking 3090 around here.
I feel like 5090 is more of a "prosumer" market, aka techie youtubers and such.

>>106307104
Do not start this again

>>106307109
I see some other anon must have asked this before...

>>106307093
did it do the misspelling or was that you?

>>106307118
It did it. The prompt is exactly what I posted.

>>106307108
A 3090 is enough

>>106307084
it's only missing nvidia's diarrhea to be truly complete

>>106307107
it's the precursor to gpt 3.5

bunch of news outlets saying b60 dual is out next week. Somewhat doubt it due to intels weird silence about it. Maybe cuz they realized no interest and it wont make em nvidia rich.

>>106307093
four eyebrows

Nemo is retarded.
Gemma is cucked and retarded.
GLM AIR is cucked, retarded and huge. Though it did produce some decent cunny once I allowed nemo to write half of the story first.

I'll stick with nemo.

>>106307104
Irrelevant character that gets spammed ITT for no good reason

>>106307218
Is it from some anime?

>>106307242


>>106304243

>>106307174
I don't have a server and i run full glm-chan in 4bits. Just get a gayming pc

>>106307242
It's a mascot for music production software that unexpectedly gained its own fandom and became a character.

>>106307242
What if zelda was a girl?

>>106307255
What is a vocaloid?

>>106307283
Fucking ask an LLM or google it you troll.

>>106307242
can metroid crawl?

>>106307283
A more appropriate question is what a vocaloid isn't. And vocaloid isn't a neural net model making it totally irrelevant.

>>106307296
This is somewhat what I'm trying to understand. I'm not trolling like >>106307287 is implying.

I guess the miku thing must be a mascot like someone else said, but...why?

>>106307306
Ever met an autistic person who has a special interest? Baker has a special interest and forces it on everyone.

>>106307153
It came from a Reddit post which Videocardz picked up and then it spread everywhere. I don't doubt it given the evidence of that post. It could be Intel is accelerating their schedule because this is important. And they achieved some important milestones recently documented in https://www.intel.com/content/www/us/en/developer/articles/technical/battlematrix-software-update-august2025.html where most of the enterprise serving stuff they want to do with vLLM is pretty much good enough to go. That being said, I don't think it's actually ready ready in comparison to CUDA systems but Intel really does not have time on their side so sooner is better especially given most of the recent troubles happened after what they said at Computex. It's still probably mostly going into Battlematrix systems unless Intel fumbled that which I doubt. The main issue will be that they are going to sacrifice goodwill with regular consumers and selling the B580s since I forsee at the prices they are selling them to be basically good enough to start a scramble on them. At the very least, anyone that was looking for a Turing RTX 8000 won't have to pay 2 grand for them any longer, I would suspect and a lot of GPUs in the >$500 to 2-3k range are going to be affected by Intel's pricing.

>>106307306
because miku leaked the migu

>>106307437
What's a migu?

For me, when speed is considered, the best ERP model is Rocinante 1.1 by TheDrummer.

>>106307019
Give me the llama-bench command. I rarely run it and I'm too lazy to check how to on github.

>>106307503
Only you know the parameters you run it with.
llama-bench -h

>>106307490
Suck my dick drummer

>>106307539
>I can't suck my own dick

>>106307554
This post is fucking gay on so many levels

>>106307490
Liking Rocinante-R1-12B-v1b by TheDrummer as well so far.

>>106307583
>Rocinante-R1-12B-v1b
What even is this? Because it's only on the BeaverAI Huggingface page and not Drummer's Huggingface page there's zero information about it at all.

I still believe in our lord and savior undi. He will return and banish the gay drummer antichrist

>>106307483
A miserable little pile of bits.

why is miku the mascot anyways?

>>106307583
Check out TheDrummer/Cydonia-24B-v4.1 too

>>106307618
fuck off

I can't believe they still can't answer this simple question.

>>106307583
>>106307598
Ohhhhhhhh this is just the thinking meme isn't it?
Gay.

>>106307657
The extra training makes it a basically meaner and very slightly smarter roci even without think

>>106307516
It doen't want to run with rocm probably because -dev doesn't work. How do I choose between my integrated graphics and the MI50?
This is the vulkan run.
| model | size | params | backend | ngl | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| qwen3 14B Q5_K - Medium | 9.79 GiB | 14.77 B | RPC,Vulkan | 99 | pp512 | 158.75 ± 0.44 |
| qwen3 14B Q5_K - Medium | 9.79 GiB | 14.77 B | RPC,Vulkan | 99 | tg128 | 29.34 ± 1.71 |

>>106307648
domestic violence is no joke

>>106307667
>roci
DIE MONSTER

>>106307676
>llama-bench -h
usage: llama-bench [options]

options:
...

test parameters:
-mg, --main-gpu (default: 0) <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
...
Multiple values can be given for each parameter by separating them with ','
or by specifying the parameter multiple times. Ranges can be given as
'first-last' or 'first-last+step' or 'first-last*mult'.