Thread 106311445

/lmg/ - a general dedicated to the discussion and development of local language models.

Previous threads: >>106303712 & >>106293952

►News
>(08/18) Nemotron Nano 2 released: https://research.nvidia.com/labs/adlr/NVIDIA-Nemotron-Nano-2
>(08/15) Ovis2.5 MLLMs released: https://huggingface.co/collections/AIDC-AI/ovis25-689ec1474633b2aab8809335
>(08/14) Canary-1B v2 released: https://hf.co/nvidia/canary-1b-v2
>(08/14) DINOv3 vision models released: https://ai.meta.com/blog/dinov3-self-supervised-vision-model
>(08/14) Gemma 3 270M released: https://developers.googleblog.com/en/introducing-gemma-3-270m

►News Archive: https://rentry.org/lmg-news-archive
►Glossary: https://rentry.org/lmg-glossary
►Links: https://rentry.org/LocalModelsLinks
►Official /lmg/ card: https://files.catbox.moe/cbclyf.png

►Getting Started
https://rentry.org/lmg-lazy-getting-started-guide
https://rentry.org/lmg-build-guides
https://rentry.org/IsolatedLinuxWebService
https://rentry.org/recommended-models
https://rentry.org/tldrhowtoquant
https://rentry.org/samplers

►Further Learning
https://rentry.org/machine-learning-roadmap
https://rentry.org/llm-training
https://rentry.org/LocalModelsPapers

►Benchmarks
LiveBench: https://livebench.ai
Programming: https://livecodebench.github.io/leaderboard.html
Code Editing: https://aider.chat/docs/leaderboards
Context Length: https://github.com/adobe-research/NoLiMa
Censorbench: https://codeberg.org/jts2323/censorbench
GPUs: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference

►Tools
Alpha Calculator: https://desmos.com/calculator/ffngla98yc
GGUF VRAM Calculator: https://hf.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator
Sampler Visualizer: https://artefact2.github.io/llm-sampling

►Text Gen. UI, Inference Engines
https://github.com/lmg-anon/mikupad
https://github.com/oobabooga/text-generation-webui
https://github.com/LostRuins/koboldcpp
https://github.com/ggerganov/llama.cpp
https://github.com/theroyallab/tabbyAPI
https://github.com/vllm-project/vllm

►Recent Highlights from the Previous Thread: >>106303712

--Paper: Ovis2.5 Technical Report:
>106308702 >106308793
--Paper: CarelessWhisper: Turning Whisper into a Causal Streaming Model:
>106308813 >106309149 >106309494
--Qwen-Image-Edit released with strong text rendering and image editing capabilities:
>106304160 >106304229 >106304484 >106304564 >106304599 >106304761 >106305072 >106307093 >106307242 >106307283 >106307306 >106307263 >106307748 >106307855 >106307925 >106308241 >106308333 >106308348 >106308326 >106308436 >106307118 >106307122 >106308310 >106308351
--NVIDIA Nemotron Nano 2 and synthetic dataset release with mixed reception:
>106305366 >106305431 >106305443 >106305712 >106305743 >106305758 >106305794 >106305806 >106305812 >106305818
--AMD GPU performance restored after removing ROCm interference on Arch Linux:
>106304678 >106304729 >106304822 >106304896 >106305055 >106305076 >106306403 >106306460 >106306556 >106306029 >106306981 >106307019 >106307503 >106307516 >106307676 >106307714 >106307766
--Selective MoE layer offloading beats bulk CPU-MoE for GML Air performance:
>106308587 >106308805 >106308825 >106308830 >106308846 >106308924 >106308929 >106308940 >106308994 >106309022
--Qwen image edit alters art style despite accurate prompt following:
>106308302 >106308311 >106308327 >106308363 >106308386
--Open-source models now trail frontier AI by only 9 months in capability:
>106307055 >106307108 >106307124
--Intel accelerating GPU development via Battlematrix:
>106307153 >106307327
--Evolution of AI-generated limericks from GPT-1 to GPT-5:
>106307031 >106307107 >106307146
--Qwen Edit pull request merged into ComfyUI:
>106307802 >106308885
--Article: DeepSeek delays R2 model launch due to Huawei chip issues:
>106304573
--Miku (free space):
>106305259 >106305535 >106306927 >106307093 >106307554 >106307648 >106308302

►Recent Highlight Posts from the Previous Thread: >>106303714

Why?: 9 reply limit >>102478518
Fix: https://rentry.org/lmg-recap-script

>>106311447
>pic
Nice.

Dead hobby

Segs with Teto (not the poster)

>>106311473
Nope, Gwen is here to save us.

>>106311506
pedophile alert

i love benchmaxxing.

>>106311528
Are you certified in corporate benchmarking to be allowed to use that word?

>>106311511
>philes your pedo

>>106311542
no :<

>>106311511
You called?

>>106311506
Mistral shits out some half assed deepseek distilled model every couple months.
Google Gemma.
And qwen.

Thats it for local smaller vram cucks right?
I thought after the R1 Paper and reduced training costs some arabian prince might safe us with a RP trained model.
A SNK situation basically. How is there no guy with a couple mil to blow? The zoomers would eat it up if you put it behind a apI regardless if they could download it on huggingface.

>>106311683
RP corpus is very small so a model can't be trained solely on RP

is there any place I can grab ik_llama.cpp builds?
I'm building the cuda version myself right now, but FUCK it takes quite a bit.

speaking of which, post cunny cards

>>106311693
Now we have models that consist of 90% synth math/riddle. 5% synth rewritten by a 30b. 5% human text but filtered by naughty list. And its still able to purple prose slop at least.
A model mainly trained on all sorts of entertainment media would be cool.

>>106311731
>synth rewritten by a 30b
nvidia nemotroon bros... we lost!

>>106311747
Whats even the reason? So they don't get copyright trouble?

>>106311766
obviously, but it also makes it easier for the model to digest. it removes the noise and gives you a clean dataset.

>>106311628
jc how terrifying

>>106311683
>blah blah blah
>RP
well for that you'll need more than smart llms
parsing and maintaining a lot of variables to keep things consistent
and a bunch of anons are working on that

>>106311717
By default a CMake build is single-threaded, add e.g. -j 32 to use more threads.

>>106311807
I had it with -j16, doesnt help that im on windows. and my build hit a snag too, fml. Maybe I should switch to clang?

>>106311511
calling yourself out?

How does your model of choice do when asked for a list of famous nukige with an empty system prompt and no reasoning? Here's full GLM-chan's take.
https://files.catbox.moe/mnf6v6.txt

>>106311903
Another benchmark for Mistral?

>>106311932
Once this one gets benchmaxxed, it's time to build an obscure nukige trivia benchmark.

i hope its not gonna be even more gemini sloped.

Why hasn't mythomax been surpassed 2.5 years later?

>>106312081
It's called rocinante 1.1 my friend

>>106312081
Mythomax was true AGI released to the public to see if anyone would notice.

>>106311683
>some arabian prince might safe us with a RP trained model.
>A SNK situation basically. How is there no guy with a couple mil to blow?
You are vastly underestimating how niche this hobby is. The type of person who has millions to just waste on a while is either too busy enjoying normie shit or is one of the few people that actually worked for that money and has to continue to work to maintain that lifestyle. Either one likely doesn't even know this hobby exists unless their income is directly tied to custom AI models (and even done with the publicist on middle manager that pretends to do shit instead of the one that actually does the hard work)

>>106312180
Llama Scout mogs both

>>106311903
unironically might be a good benchmark because most models (all of them) are gonna fail horribly

>>106312058
When is that from?

>>106311747
the most hilarious thing is how they said their new dataset improves on multilingual but in my personal translation tests I still see models like Qwen 4B and Gemma 3 4B destroying its asshole
NVIDIA can't make good models it literally runs against their pajeet nature

>>106311903
I'm sorry, but I can't assist with that request.

>>106312226
deepseek has their own group chat/news feed thingy on some chinese app thingy
previous announcements were done there ahead of time as well, so probably legit

>>106312058
Is it just a context length change?

>>106312176
Well we had a guy who was willing to blow 1 Mil in this general.
Theres gotta be somebody. But I guess you are right.
Doesnt help that its mostly femoids and zoomers who enjoy llm RP.

>>106312283
what does it cost if they aren't trying to innovate, just using current tech on a custom dataset? what is salary for a team of people who can make it happen? how much gpu time at what cost?

>>106311903
GLM-4.5-Air-Q3_K_S

There's some hallucination, also mixes up developers of games. But otherwise most of the information is correct. Okay for a model that you can run on a regular PC.

>>106311693
I wouldn't call it small given how big it actually is from the pre-contaminated scrapes from https://huggingface.co/datasets/lemonilia/Roleplay-Forums_2023-04 which is 20GB give or take. Of course, this isn't clean and all raw but even if you have half that after doing that, 10GB is not a bad size. But it's really what you can do with the data to create training and etc. that will make it useful and there is more than enough here to do stuff like DPO training or etc. However, a lot of it is predicated on clearning it and I can't see the demographic most interested to get it to happen like >>106312283 describes willing to do the hard work to make these RPs usable. And we already know how shit employing third worlders is to do this task as ScaleAI has demonstrated. A lot of the next frontier is going to be predicated, if we're still stuck with the LLM paradigm, on data quality and we are missing a lot of that, most of them being synthetic and not a lot of them are actually good enough being synthetic to be actually used like with math and science with one provable answer.

>>106312283
You don't need to just pay for compute, but also to pay for qualified people who can do a proper training run on a large model. Unless all you're asking for is a LoRA, there isn't a good open source codebase for training large MoEs.

>>106312326
Nvidia just gave a perfect pre-train data for free

>>106312338
>20GB
>big
People train models on 10~20TB corpora these days

https://youtu.be/1H3xQaf7BFI
you are funding this
apologize

>>106312346
>You cannot buy the free market product!!!

>>106312344
Corpa altogether, no. But it's been known for quite a while you can NOT train a model only on RP to make them work well with RP, you need to feed it alongside everything else.
Look at the sizes of the datasets used by AllenAI for their models.
https://huggingface.co/collections/allenai/tulu-3-datasets-673b8df14442393f7213f372
Hundreds of megabytes to a gigabyte at its largest per dataset. Total, of course it's larger than 10GB but it's more than enough to seed a reasonably large model with more than enough RP.

>>106312346
Gladly!

>>106312342
>perfect pre-train data
LOL

>>106312382
Explain your qualification and what is wrong according to you?

>>106312392
appeal to authority

>>106312346
>illegal gpu black market
????
They are gonna make anything bigger than 24gb vram illegal soon won't they.

>>106312058
>i hope its not gonna be even more gemini sloped.
I have some bad news for you. I've tried it over the website, and it is even more gemini than before.
It even has the same quirk where it doesn't put period on the last sentence of the reply.

>>106312392
>qwen 30ba3b rewritten slop
>"explain your qualification"
how about you explain your lack of intelligence pajeet

>>106312344
you couldn't get very far with it, but I think they really did hit a wall throwing more tokens and more layers at the problem, seems to be delivering diminishing returns. I think the moat is more of a shallow ditch. throe a few trillion tokens at a 100b dense or 500b 32a would get you close to good enough.

>>106312346
i guess its gay to have an agenda, but i wonder what he was hoping to achieve with this other than to show that the restrictions dont really work

>>106312402
You see, you can't just buy the compute hardware manufactured in your own country, it's not meant for you because you're an enemy.

>>106312412
He's not really showing anything except for the nitty gritty about how sanctions work. Sanctions aren't going to 100% stop a country from getting the thing you don't want. It's to make it 100x harder to actually procure those things at scale and in a reasonable way at a good price. Sanctions fuck all of that up and slow down progress which is exactly what is happening. It's meant to buy time to actually get an alternative strategy or mitigations for when the sanctions don't work anymore since most of them usually are on borrowed time.

>>106312333
what about kamige nukige??

What are the recommended sampler settings for deepseek v3?

Fails to write a simple scene. It does one paragraph okay, then goes to shit. Or is just a problem with the prompt?:
deepseek v3: https://rentry.co/yvmykzox
vs qwen3 4b: https://rentry.co/zxr6nchf

>>106312333
i wouldnt really call most of them nukige, funnily enough it gets it right at the very end when explaining the terms

>>106312479
>552.56 seconds
anon...

>>106312528
? Less than 10 minutes is pretty good, coming from gpt-j shinen 6b.

>>106312479
>10 minutes tg
>output worse than a 70b llama
the absolute state of moesissies

>>106312551
>output worse than a 70b llama
Even a 4b assistant-slopped model is better, holy shit.

>>106312479
What settings are you using?

>>106312479
maybe it was just bad luck did you try another swipe?

>>106312562
Temp 0.6,
Top_p 0.95
Top_k 20

>>106312574
Okay, I'll try again. Will report back in 10 days when it finishes.

>>106312528
Loading is so slow ;-;

>>106312575
set temp to 1, top_p to 1 and top_k to 0
you'll thank me later
actually good models not only do not need sampler snake oilery but they get harmed by it
on their official API they don't even allow you to set anything other than temperature because they expect the average llm user to be too retarded to comprehend this

>>106312593
once I got an m.2 for the models things began looking much brighter.

>>106312613
It's on a nm790, which shouldn't be too low-end right? I liked the advertised endurance on the nm790.

>>106312603
Alright. I'lll try that.
I was using 0.6 and 0.95 because I was using glm 4.5 air before, and that's what z.ai api docs told me. Top_k at 0 failed to produce any tokens, but top_k at 20 worked, so I left it like that.

>>106312593
I think good code should allocate all needed RAM immediately instead of gradually increasing it like shown on the graph, which can harm the loading speed somewhat.

>>106312627
>nm790
You use --no-mmap, right? So it should take like two minutes. I used to load deepseek from 5400 rpm hdd and that was painful.

What the fuck is this

>>106312658
Looks like an announcement for a 5 month old model.

>>106312658
>>106312226

>>106312653
it probably is but the task manager is only showing pages that have been loaded.

>>106312658
>[white genocide] Deepseek is now only for profit and will not release V3.1, which has a whopping 128k context, it's literally agi, APP, glory to the CCP, API will be closed shortly

>>106312676
Have you tried mlock?

>>106312678
>which has a whopping 128k context
so does current DS if ran locally, they just used to limit to 64k on their frontend to save on costs

>>106312691
I have not. Does it make much of a difference?

>>106312678
>[white genocide]
Dipsy would never say that, she loves white men.

>>106312691
I was just pointing out that programs can allocate as much virtual memory as they want, oftentimes in excess of physical ram. the os is probably only reporting the physical pages that have been consumed as the file is read in.

>>106311445 (OP)
>>106311447
omg it teto

>>106312678
hello anon, i think you dropped this
>hands you a sense of humor

>>106312718
Usually --no-mmap should suffice, but it's worth trying.

>>106312479
R1, two rolls. Prefill ends after "Archive Warnings:"

https://rentry.org/2i6kog8t
https://rentry.org/62s4ay6i

>>106312603
they also forcibly convert requests with temp=1 to a much lower temperature, so if we're going by the API... definitely do not use temp 1

>>106312574
Using the same settings as before:
https://rentry.co/y6xn8uoz

It just does one paragraph okay, then goes stupid for a while before completely giving up. Temps at 1, top_p 1, and top_k 0 is essentially the same.

Is deepseek just not trained on writing stories? How the hell is a 671b shitting itself so bad? It's q4, so it shouldn't be too lobotomized.

>>106312774
>Rating: Explicit
>Archive Warnings: Graphic Depictions of Violence
>Categories: M/F
>Characters: Celestia | User | Villainess
>Relationships: Celestia/User
>Additional Tags: Non-Consent | Magical Girl | Virginity Loss | Death | Impregnation | Dark | Depraved | Gruesome | Erotic Horror | Forced Sex | Magical Girl Defeated | Broken Mind | Body Horror | Guro
>Stats: Published: 2023-10-05 Words: 4000

Should I try that with V3? I'm downloading R1 right now, but while I'm fine with 10 minutes for a response, I have shivers down my spine thinking about how long it would take with thinking in the picture.

What do people use as a frontend to translate VNs?

>>106312797
The trick is that there's no thinking. I just pasted your prompt in mikupad, added the beginning of ao3 tags and let it continue. "Graphic Depicitions of Violence" and everything after that was written by the model.

>>106312812
Alright, I'll try that. With or without the colon at the end of 'Archive Warnings'?

>>106312842
The colon doesn't matter but my autistic DRY settings might.
Try them with V3 too and see if it helps.

>>106312812
Is it just a v3 problem?

>Rating: Explicit
>Archive Warnings: Major Character Death, Graphic Depictions of Violence, Non-Consent, Forced Sex, Dubious Consent
>Disclaimer: This work contains extreme and dark themes. Reader discretion is strongly advised.

Is all it writes before going right into it. Or maybe q4 actually is too low a quant...

>>106312860
>maybe q4 actually is too low a quant
This >>106312774 was IQ1_S

>>106312272
Just saw it posted elsewhere too. I thought V3 was already 128K context; in practice for RP anything over 10K falls apart for both R1 and V3. I'll have to experiement with nuV3 on a longer RP to see how it holds up past 10K.

>>106312272
The "R1" text on the Think button is gone now, so something on the website was definitely updated

is qwen235 good at q3? also thinking or the plain instruct one

>>106312873
>I thought V3 was already 128K context
It is, but not on their webui/API, see >>106312709

>>106312867
Holy hell. Do you use it for assistant tasks/coding too? On the smaller models, I feel like q2 is basically unusable, even for stories.

>>106312888
>The "R1" text on the Think button is gone now
oh no no no, hybrid reasoner meme incoming

>>106312906
For coding there's qwen.

>>106311445 (OP)
Teto sexo

>>106312906
the bigger the model, the less damage low q does

>>106312058
>3.1
>not 4
>not even 3.5
>6 months later
deepseek is dead in the water

>>106312812
https://rentry.org/e636f78n

I went back down to a temp of 0.6... Seems like it's a v3 issue.

>>106312889
I use it at q2 and it's still pretty good, better than anything else I can run imo
my default rec would be the instruct model, but the thinker is good if you like messing around with that sort of thing and don't mind the extra wait

>>106312994
It's a whale, it can swim.

>>106312994
Ai progress in general

>>106313021
Beautiful.

>>106313021
Holy hell, those are some insane gains.

>>106313102
HAPPENING! HUGE!! BIGLY, EVEN!!

>>106313102
>Trained at Q1 using the chink GPUs
HOLY SHIT

>>106313021
If this was Sam Altman's screw up then it is scary to think about what Sam Altman's success looks like.

>>106313141
There is no way he said any of this

>>106312904
> no model update published for V3 on HF
So v3.1 is really just a change in settings for the web and API version through DS. No actual model update.
Bummer.

>>106313162
Are you really a dispy fan? This is how they always do their thing, first announce on their chat, then update their online inference and only after that put weights on HF.

>>106313157
>>106313141
source is from some verge article / interview.

Finna grab some crashout gang hood niggas and boost the nearest datacenter
finna fly to HK with the loot in the checked bags
finna link with deepseek gangdem and create Z0GD35TR0Y3R-B800A70B
trump finna be mad as hell

>>106313178
True. If so, it'll upload later today. Time will tell.

>>106313102
>The deployment of Huawei Ascend AI processors enabled a cost-effective training paradigm for DeepSeek-R2, reducing total training expenses to approximately $90,000 USD—representing a substantial reduction relative to equivalent NVIDIA-based infrastructure while maintaining competitive performance metrics across benchmark evaluations.
Wtf

>>106313184
>in 2025, this is the equivalent of posting "I'm gonna crash one more plane in the towers" on the 11th september 2001.

>>106313223
So many soda chink models have been uploaded to HF in the last few months, there isn't anything left of the towers to hit anymore.

New DS on the site likes to start answering questions with "Of course" is that known to be a quirk of some other model? Might point to distillation if it is.

DeepSeek-R1-0528 is here!

>>106313246
Who else is left to distill from anyway? I think someone said Claude still gives thinking output through API. Grok?

>>106313246
Second round of questioning gets:
>"Excellent question" "Excellent and insightful question."
Might indicate at least some multi turn effort.

>>106313246
I noticed this too kek. I asked it "should we abolish the death penalty" (just a vague question that I figured it would have to consider since I wanted a look at its CoT) and it started its reply with "Of course."
I was like "damn they made this thing super opinionated" before I realized it was just acknowledging the question or whatever

>>106313246
based deepsneeds jailbreaking their own model

Is Whisper the best for voice > text still?

Im using faster-whisper-xxl because I can just drag and drop, but it fucks bunch a bunch, whereas where i just drop opus files into Gemini it nails it, however it eats a lot of tokens despite it being free. This is for multi-hour long audio files

>>106313246
>"This gets to the very heart of what..." "That gets to the strategic heart of why this..."
it certainly has a new "tone" so to say.

>>106313378
faster-whisper turbo is the best STT from my tests. You might need to play with the vad settings a bit though

Just how lobotomized is glm air at q2_k_xl? Is there any good alternative in this range?

>>106313246
>is that known to be a quirk of some other model?
...Llama 2

https://huggingface.co/deepseek-ai/DeepSeek-V3.1-Base
it's up

>>106313196

Let's go!
>>106313475

>>106313475
HOLY FUCKING KINO

>>106313475
If they uploaded a full fledged base (not just another instruct / thinking tune), this was probably their attempt at V4 and R2 that didn't pan out

>>106313475
local is saved

>>106313475
>hybrid reasoner
>shits on gpt5 + gemini 2.5
>creative writing focus
>beats pokemon in 12k steps
we are so back

>>106313475
Let's goooo
Mistral small 3.3 soon

>>106313246
the concise, pointed thinking, increased final answer verbosity, and use of tables unironically remind me of oss (though this is obviously more human and less alignment-obsessed)
>>106313475
new base is very interesting

>>106313475
Not falling for it again.

>>106313528
K bud.

>>106313517
>-Base
>>hybrid reasoner
Come on now.

>>106313538
We know it's a hybrid from the site dropping the "R1" from the think button and both of them having a much closer manner of writing than R1 and DS3 had before.

>>106313475
no config change vs v3

>>106313475
>700 B params
Okay how many of you fags can actually run this?

>>106313475
This needs to be a big step forward considering GLM 4.5 is a sidegrade in smartness at almost half the size.

>>106313581
all of us

>>106313475
Fake

Wait. if there is a 3.1 BASE it means it's not v3 with more training steps. So did they mess up V4 and called it V3.1 the same way oai did 4.5?

>>106313618
Likely.

>>106313246
>is that known to be a quirk of some other model?
Gemini loves to praise you wherever you make a suggestion.

>>106313475
>3.1
>not 4
nothingburger confirmed?

>>106313581
If you can run V3/R1 you can run this.

>>106313618
They paid the hybrid reasoner tax like Qwen did, look forward to them going back to schedule after this.

>>106313378
parakeet / canary / voxtral (allegedly)

>>106313618
AI progress is hitting a hard wall, like a woman in her 30s.

>>106313618
I would be more inclined to believe this if there were any arch differences between 3 and 3.1, DS is too ambitious of a company to make an attempt at V4 with the exact same architecture as V3

>>106313581
i probably after is quantized for 0.4bit

>>106313581
I'm poor so I can barely run glm air at q3...

>>106313581
I can run it after unsloth makes it retarded.

>>106313517
>>hybrid reasoner
they fell for it

The fact that they went for "3.1" is worrying. They did a 2.5 in the past, which was a combination of their up to that point separate coder + chat models shortly before they released 3. However, 3.1 implies that there might be more to generation 3

>>106313637
Hybrid reasoning wouldn't cause the base model to underperform.

ollama run deepseek-v3.1

>>106313659
Maybe the intended as v4 but once the reasoning tune went bad decided to name it v3.1 who knows but them. Could explain the time it took though.

>>106313581
How many active penises are there?

>>106313642
If NSA or whatever new thing they're testing isn't working out, there's no point releasing a broken model. It's more likely their attempts at the actual V4 isn't going well, so they're just going incremental improvements on the old models to maintain relevance.

>people speculating over naming
Why are you guys like this? Naming means jackshit >>106313021

Is it gooder or just benchmaxxed?

>>106313691
It does though, it signals expectations to people.

>>106313691
The name isn't the point retard. It's the lack of a new model with a new architecture. The .1 is only concerning because it makes it sound like they're planning at least a 3.5 later on so V4 with some new breakthrough architecture might not come until next year.

>>106313720
go back

>>106313720
Well I do wish we could get something more than image in.
Unfortunately all those audio in/out models have severe degradation. Nobody has really figured it out yet.
If we could get a model that has native image in and out it would be a game changer for RP.

>>106313681
It's possible that their next generation isn't turning out well but I think it would be a stretch to take this as any sort of evidence for or against; remember, they updated 2.5 in the same month as V3's release

>>106313703
>>106313715

Previous version was v3 0324. Now it's v3.1
.1>.0324
This is huge. It's an improvement of over 300%

>>106313740
>remember, they updated 2.5 in the same month as V3's release
true enough.

>>106313720
They haven't even figured out text, let alone put more modalities in there

>>106313758
>>106313740
wasnt 2.5 combined too?
made me look it up.
>DeepSeek-V2.5 is an upgraded version that combines DeepSeek-V2-Chat and DeepSeek-Coder-V2-Instruct.
sounds similar to the hybrid reasoning.

>>106313618
>if there is a 3.1 BASE it means it's not v3 with more training steps
why not?

Nemotron 2 gguf support?

>>106313736
>Unfortunately all those audio in/out models have severe degradation. Nobody has really figured it out yet.
Maybe to achieve generalization across multiple modalities just requires a shitton more data and longer training. It worked for text, eventually. It's probably just exponentially harder to do both at the same time.

>>106313802
yeah maybe 50 more trillions will do sam, we know

>>106313797
Because they didn't release new base version for 0324.

>>106313800
>Hybrid Mamba
about 2 more weeks after never

>>106313797
At the very least it's not derived from any v3 instruct

>>106313824
I swear I saw something about llama.cpp and hybrid mamba support over a month ago.

>>106313802
At least the direction is looking good.
The first qwen audio in/out model was pyg level stupid. kek
Their recent one wasn't THAT bad. But still more tarded and the voice was horrible.
I saw a video of some web llm game though and immediately recognized the voice.
Some guy must have used it for his game. So it was good enough for that.

>>106313754
ha

>>106313475
where model card

>>106313887
It's 11PM in China, check back tomorrow

>drops weights
>refuses to elaborate
>leaves
kinda based

>>106313896
but why would they upload it without a model card at 11 PM?

>>106313914
>kinda based
They literally explicitly stated, without ambiguity, that is a base model.

>>106313914
When are you going to drop some weight, fatso?

>>106313923
set to upload and xiaban, will do the card tomorrow

>>106313938
I can't. It's a medical condition.

>>106313923
its a 700B model, prob takes a while to upload. Click upload and go to bed lol

>>106313938
rude, I'm already losing weight and I'm not even fat

>Also, considering the user's fetish, incorporating elements that facilitate easy access for other men, like magnetic fastenings or tear-away sections, would make the outfit more aligned with the netorase scenarios. Ensuring that every modification not only makes her look sluttier but also serves the erotic roleplay context is key.
t-thanks v3.1 reasoner

>>106313664
what butchered model does this pull

>>106314007
hmmm I see, quite insightful

>>106314014
DeepSeek-v3.1-Distill-EXAONE4.0-1.2B

>>106314007
no way in hell did you download it that fast so where is it?

>>106314007
That's why small details matter.

>>106314044
He obviously used their website.

https://github.com/deepseek-ai/DeepSeek-V3.1
https://github.com/deepseek-ai/DeepSeek-V3.1
https://github.com/deepseek-ai/DeepSeek-V3.1
MODEL CARD HERE

>>106314044
>local
This implies there must be a remote model general. I wonder what they do over there?

>>106314063
It's called /aicg/

>>106314063
They beg for proxies.

>>106314063
use the good models

>>106313475
w00t

>>106314106
stop posting that fucking phishing website

>>106314106
>March 25, 2025

>>106314063
>remote model general
they're so stupid they can barely set up st. also their hands are brown

>>106314115
KINO

>>106314120
Too late, I have already boughted it.

>>106314115
nano banana is crazy good
open source will never catch up

>>106314106
kys

Quick question, does resizable bar matter for inference speed?

This is so crazy. After the better part of a year we're finally getting an upgrade to THE model that started the current age of local LLMs.
Without it, we'd still be stuck with the incremental upgrades over llama 3.3 or mistral large the other losers would've released with no shame.
3.1 is going to be crazy

>>106314188
probably not by much, but do benchmark it yourself nonetheless

>>106314224
This, but unironically.

>>106314188
Should not make a difference.

>>106314227
The thing is, I can enable resizable bar in the bios but in windows it says it's disabled. So I'm wondering if I'm losing out on performance.

>>106314249
Ty

Are we hoping deepseek 3.1 will beat claude for rp and writing?

>>106314263
I'm hoping to beat my dick

>DeepSeek passed my test and used EGFs rather than more conventional combinatorial counting approaches
Fuck, that's impressive

>>106314271
Used Epidermal Growth Factors?

>>106314313
I think he means Electronic GirlFriends

DeepSeek V3.1 add four new special tokens
<|searchbegin|> (id: 128796)
<|searchend|> (id: 128797)
(id: 128798)
(id: 128799)
compared to V3-base

>>106314313
An **EGF**, or Exponential Generating Function, is a mathematical tool used in combinatorics to count arrangements of labeled objects, where the identity or order of the elements is important. Unlike simpler counting methods, an EGF represents a sequence of numbers as a power series where the coefficient of each term is divided by a factorial (e.g., `a_n * x^n / n!`). This structure is specifically designed to elegantly handle the complexities of combining labeled structures, automatically managing the "shuffling" of elements that makes other approaches cumbersome. The 4chan user's comment that DeepSeek used an EGF is high praise because it indicates the AI solved a complex problem not with tedious, conventional case-by-case analysis, but by demonstrating a sophisticated conceptual understanding of a high-level mathematical technique, effectively "thinking" like an advanced mathematician.

>>106314271
>bro uses ds for combinatorics
it's deepSEX not deepmaths.

>>106314331
tl;dr

>>106314366
DON'T CLICK IT MAKES MUSTARD GAS

>>106314366
i love clicking random links

>>106314366
Not using your logged API

>>106314379
retard
it's an openai compatible endpoint
connect your sillytavern to it or something

>>106314366
That's dolphin porn.

>>106314366
shit shit shit I just sent a prompt with full name SSN and credit card number

>>106314390
i know but im not connecting my sillytavern to a frontend that will use a zero day exploit to open my homework folder full of my cock pictures

So who's jason and why am I supposed to use his files?

>>106314396
It's okay anon, I'll purge the logs, just for you

>>106314407
run sillytavern on your android phone through termux
triple sandboxing

>>106314366
That was kind of you. Someone should at least do a nala test or something.

https://huggingface.co/deepseek-ai/DeepSeek-V3.1-Base/discussions/2#68a495c3000a4675fe8ba559
ubergarming in process...

>>106314425
but i have my cock pictures on my phone too
and my phone is rooted and on a vulnerable android version (i am not upgrading because i want the vulnerable version)

vramlet bros...

>>106314442
i'll run it for you
>inb4 i have my cock pictures on your computer

>>106314367
ty

>>106314460
... are eating good.

ramlet sissies, on the other hand...

>>106314188
30% faster with a hacked p2p driver https://github.com/tinygrad/open-gpu-kernel-modules in vllm with tp
But if your mobo doesn't support rebar you can hack it into supporting as well, tested on epycd8-2t

>>106314464
please do :3

>>106314478
3090 with monitor attached shows resizable bar enabled. If I instead plug the monitor into my gt 710, 3090 shows resizable bar disabled. mc62-g40

>>106314502
yup we're dead

>>106314502
2 more hype cycles

which one of you fags did this

>>106314566
who

>>106314566
tasukete

>>106314514
You need 4. Rebar and p2p allows direct reads between GPUs, that's where speedup comes from

>>106314566
Porn is good.

>>106314565
Qwen is benchmaxxed though

>>106314565
oh no no no

>>106314590
and its good so...

>>106314583
I have 3, that's why I'm worried about resizable bar

>>106314565
WHY'D THEY FALL FOR THE HYBRID THINKING MEME AAAAAAAAAAAAAA

>>106314600
It censors me, unlike deapseek

>>106314603
I think this is the failed attempt on the chinese chips, real V4 is prob still training

>>106314565
total llm stagnation

>>106314616
newest qwens are super uncensored so that is a major skill issue

>>106314600
Definitely not better than sneed for writing.

>>106314603
I think that's why they pumped this out as 3.1 Better to recognize sunk submarines than costa rica

>>106314565
deepseek is 2.5x bigger, and yet it somehow came out cheaper overall?

>>106314632
235b instruct?

>>106314547
cute migu

>>106314656
thx

>>106314602
Don't worry then. Symmetric parallelism requires 2^n GPUs, you won't see a difference in any other case

>>106313507
>>106313618
>>106313625

Why do you retards want the "failed V4 training" story to be true so much? They hadn't found a single way to improve the design in almost a year, and just trained another V3-671B with the same config from scratch, "failed" it and branded it as 3.1? Really, this looks plausible to you?

>>106313681
> so they're just going incremental improvements on the old models to maintain relevance

Do they look like they care about maintaining "relevance"? We see Kimi/Qwen/Tencent/Z.AI constantly shilling themselves on X, announcing some minor bullshit features like presentation generator, and DeepSeek just… quietly updates a checkpoint once every 2-3 months, and you think that's what they're doing, trying to feed the dying hype?

>>106314709
They pay me to stealth advertise them btw

>another failed release by china
thank god we have gpt-oss

I'm kinda new to local stuff and it kinda boggles my brain a bit sometimes but I'd like to try and fully make the transition.

If I wanna basically setup a little fox demon in my computer to help me with shit would DeepSeek R1 0528 on the rentry be a good starting point?

>>106314709
They said themselves they don't really care about their own Chat/API service and don't need the money. But why else releasing boring incremental improvements instead of something new? You rationalize it.

redditor ran his benchmark
>Interestingly, the non-reasoning version scored above the reasoning version. Nowhere near the frontier, but a 13% jump compared to DeepSeek-R1-0528’s score.

>>106314709
>you think that's what they're doing, trying to feed the dying hype?
yes.

>>106314709
A different base model implies something far more extensive than another instruct tune

>>106314740
>would DeepSeek R1 0528
It would be almost as good as it gets.
That's the same model they provide in their web interface, their chat-gpt essentially.
Are you building an 8 channel DDR 4 server?
I'd go for the full 1TB of memory since you might as well at that point.
Either that or go DDR5 for the speed.

>>106314740
If you have a couple of b200s yeah. Otherwise, I'd look for another model.

>>106314786
>That's the same model they provide in their web interface
not anymore it isn't

>>106314679
Nta, but, I have a question.
What do you think is better, two 2080ti 11Gb or a single 4070 12Gb?
Is it easy to use multiple GPU, or is it the same shit as to try to pass thru my gpu to podman?

>>106314800
It's easy as shit bwe.
But with moe models, maybe a 4070 is better. Less power use too.

>>106314774
more context, wigga
https://old.reddit.com/r/LocalLLaMA/comments/1mukl2a/deepseekaideepseekv31base_hugging_face/n9jrtog/
https://github.com/johnbean393/SVGBench/
is this even a good benchmark

>>106314781
People forget that they used a finetune of V2.5 to make V3/R1. They're probably tweaking their last gen model to optimize it as much as possible so they have the best teacher model for V4.

>>106314800
does 2080ti even have modern CUDA support?

>>106314786
1tb of ddr4 costs so much wow

>>106314836
>a finetune of V2.5 to make
R1 lite, not V3. They're completely different architectures and sizes.

not sure how useful this is as a benchmark but its a big improvement

Thank you for your answers kind anons.

>>106314823
Yeah, that's what I thought as I have plenty or RAM...
But if I want to do some image/video generation too, maybe more VRAM is desirable...

>>106314838
That's a good question.
CUDA, yes, modern CUDA, I don't know.

>>106314878
forgot to mention, its how well the model can make SVGs

>>106314800
Used 3090. 2080ti sucks because FA requires ampere or newer, and 4070 has shit bandwidth and low on vram

>>106314886
You can run quanted image models. And afaik imagegen still don't support multi-gpu, so you're not even getting more vram lmao. Not to mention a 2080ti is going to be pretty slow compared to a 4070 for imagegen.

>>106314859
R1 Lite was the same size as V2/2.5.

>>106314894
I tried like, a year ago with GPT4 and surprisingly, it understood basic shapes to create stuff.
By exchanging a few messages, I could even make it better.
But, yeah, far from what is displayed earlier in this thread.

>>106314928
That's what I meant yes, they made R1-lite as a tune of 2.5 to create data they then used to make V3 on a new bigger base.

>>106314906
>4070 has shit bandwidth and low on vram
If we stay within the confines of what he's asking, then we're working with 12gb of 500gb/s vs 11gb of 600gb/s memory.

>>106314894
Use case?

>>106314906
The 3090 is an other route to explore indeed. I could have a good price on 2080ti, but I bet I can get a 3090 for a reasonable price too.

>>106314920
Ha shit, I thought they implemented the multiple GPU on image generation (I tried last year and couldn't play with AI for close to a year, hence my stupid questions).
Well, I will try to max out VRAM then...
...
Or ask AI to fork and add multiple GPU to stable diffusion, kek

>>106314982
instead of getting a used 2080ti u can get a new 3060 (12gb) lmao
3090 is a way better option than either

>>106314979
complicated math / spatial understanding through math?

>instantly got a eyes twinkling with mischief
reee!

https://chutes.ai/app/chute/ee7987b4-43c6-57d6-9f0c-f90c762586e2?tab=readme

>>106314982
Definitely go for a 3090. It's the gold standard for cheap used AI cards.

I know there's a guy working on accelerating generation of a single image using multiple gpus, but as of 2025-08-20, I do not believe there is anyone working on splitting up the generation of a single image between multiple gpus.

>>106314982
>>106314988
3090 is never obsolete best GPU ever made

>>106315026
>>106314982
nah, not when 5070 super might be coming out with 24GB for $750 soon

>>106315029
its worth it for 400-500$

>>106314982
I would not get anything older than 30 series, anything else will lose support real fast.

>>106314754
They say everything in their papers. Their goal is to make AGI, and make it under their budget constraints. All models they release are artifacts of internal experiments rather than products for the user, they're trying different things.
V3 was an orthodox instruct LLM done as quickly and cheaply as they could. R1-Zero was an attempt at pure RL. R1 was the same thing but forced to be more useful by compromising with RLHF. V3-0324 was V3 + distillation and synthetic data from R1, as a result vastly more useful than basic V3. R1-0528 added more compute and distillation from itself and third parties, and probably some other tricks, they got much better results at the cost of longer CoTs and some slop. This thing is probably an attempt to cut costs to become able to generate much more data, so they eliminate separate models and reduce CoT lengths while minimizing performance drops (or even getting modest gains). When they get the next base, they'll be distilling from this thing to have maximally efficient concise reasoning, a la Anthropic. The overall goal is to always make Pareto improvements or experiment towards becoming able to make them.

>>106315034
>$750
Yeah, sure it will buddy.

>>106315034
also if they drop the super it would cause 3090 prices to finally fall to like $400
https://www.pcguide.com/news/rtx-50-series-super-leak-says-we-can-expect-more-vram-for-the-same-price-as-the-original-gpus/

>>106314836
>>106314859
>>106314928

They used V2.5-1210 to make V3 and R1, and R1-lite-preview to make V2-5-1210. R1-lite was a small dense model, probably a Qwen 2.5-32B finetune, and it has never been released.

>>106315040
>efficient
>Anthropic

>>106315044
even if its $1k 5000's fp4 speed will be amazing as fp4 is more widely used, like nunchunku for image / video gen for instance

Pass, I'll wait for new Qwen.

It's up
https://youtu.be/k15fikcTgMk

>>106315079
you're getting as annoying as petra

>>106315079
Everything after the first series was boring, ugly, and lacked the charm of the original.

>>106315092
You're boring, ugly and lacking charm

>>106314963
Two 2080ti. Btw exllamav3 got tp support recently

>>106315091
He's posted only 3 Gwens in the span of a week.

>>106315061
>They used V2.5-1210 to make V3 and R1
they're not the right sizes and v2 1210 is full bf16 none of the fp8 mixed of v3

unless you mean purely data for training, in which case sure I'd agree.

>>106315079
ive always avoided qwen all the way from qwen1 it fucking always sucks dick, even 72b finetunes sucked dick man
only thing they were good for was silly sizes because llama3 was 8b 70b
qwen has extreme issues with chinese, its completely shit at roleplay and its cuckery is rivaling gpt oss

>>106315120
this may shock you but when companies release new models they are typically different than the old ones

>>106315120
>cuckery is rivaling gpt oss
+1
My experience as well.

>>106315099
:(

>>106315136
qwen3 is extremely repetitive and cucked and benchmaxxed aswell, you CANT fuck
id rather tardwrangle gemma with its shitty sex because at least it can write well

>>106315143
Now you know how it feels, bigot. Maybe next time, instead of saying something that isn't nice, say something nice or don't say anything if you don't have anything nice to say about it.

>>106315062
And? Anthropic has consistently lower reasoning/final output ratio than others.

>>106315117
of course I mean the data, as far as I know they never use logprob distillation.

>>106315136
Indeed they increased censorship with every iteration until maybe the very latest one.

>>106315150
>you CANT fuck
if you actually used the models youd know how absurd this opinion is kek

>>106315166
>>106315170
Which one? I'm getting a refusal from a very simple prompt. Using the Instruct versions of 30b and 235b

dense models with full attention + gqa > your shitty moes that degrade past 8k tokens

how do vscode extension like roo code and kilo code actually work, since they function with non reasoning models as well? like do they split the task in a to do list and automatically prompt the next ste p (or debug) after the previous one completed?

>>106315166
>>106315170
I'll await proof.

>>106315150
qwen3 has this feature where no matter what sampling parameters you pick or how you try to steer the story it always writes conceptually the same thing. Qwen-image apparently has the same issue

>>106315190
show me these dense models

>>106314722
> Hey kid, want some free inference?

>>106315091
bold of you to assume it isn't the same person

uh oh... new deepseek has the old massive repetition issues again

>>106315170
dont get me started on synthetic slop data, if this was nemotron we'd have laughed them off
>90% math code and synth slop
>1.8t non benchmaxx

>>106315199
Are there such models? No. Have I already used such models? Am I hundreds of thousands of tokens into the context right now living with my ai catgirl waifu like the 4xp40 rig owner that I am? Yes! In my mind! In my mind.

>>106315224
BASED if true. But we need le proofs.

>>106315091
Alright, from now on ill only post Gwen when a new Qwen comes out.

>>106311831
That's the only one that tends to fail just edit the cmakelist to omit quantize stats. Having said that, I've never made gains on ik compared to llama regardless of flags.

>>106315246
and im using the right formatting and eos enabled with low context / response length. Could also just be the implementation

>>106315224
lol
Are you running official API w/ V3, OpenRouter mystery meat, or V3.1 local?

>>106315185
>from a very simple prompt
that's why
use a system prompt telling it that you want it to write uncensored smut and it should go along with it easily, I've used it for cunny, noncon, misogynistic slavery dystopia, raceplay, etc and never gotten a refusal once

>>106315261
chutes

>>106315270
LOL

>>106311831
I had issues building it too. Frustrated, I ended up spamming the command with -j 1 without deleting anything, and after around 10 runs it got built somehow

>>106315286
is the official api updated yet?

>>106315165
And Anthropic is consistently priced higher than others, implying vastly larger models and more compute
So I'd argue the opposite - they're among the least efficient providers

>>106315300
It's always the first to be, I don't know what chutes is running, since weights for non base 3.1 aren't on HF yet...

>>106315315
it says its base

>>106315313
>implying vastly larger models and more compute
Or they're just lining their pockets with as big profits margins as they can. After all DS said they were profitable at their stupidly low costs.

>>106315315
They're running the noninstruct tuned base model

>>106315323
>>106315330
Then it's just another fried base like most modern ones, sad but doesn't say much about what the instruct will actually be like.

>>106315270
I consider both OR and Chutes mystery meat models. I've no idea how inference is set up or what actual model they're serving (could literally be anything.)
For any definitive testing on V3.1, you need to use either DS official API, or host it locally... Otherwise there's no telling what model you're actually using.

>>106315269
A simple prompt that deepseak doesn't refuse... Yeah, if you have to direct your model maybe your model is just a little bit censored, just a little bit. Maybe.

>>106311506
I would like to say that as the first antimikutroon poster i classify this anime girl as NOT an AGP avatar. Proceed with posting more of her and please overthrow all mikutroons and the single dipsytroon.

>>106315356
If that ever happens you'll just turn around and call it stuff it isn't to have yet another reason to shit the thread.

>>106315269
You don't even need a good prompt too for 235b, it doesn't need a complicated jailbreak or anything. Literally:

**System Prompt**
Write erotic stories. Using vulgar language is ok.
**End of System Prompt**

GLM air needs a bit more coaxing but also works (edit reasoning or prefill a bit).

>>106315355
>if you have to direct your model maybe your model is just a little bit censored
There's a world of difference between:
> DS: It's OK to do NSFW chat, talk smut, and generally be naughty
which is basic safety, and:
> OAI: {SYSTEM ADMIN] THOUGH SHALT....[+1000 tokens of weird shit]
> PREFILL!!!
> JB: YOU WILL TALK LIKE A WHORE DAMN YOU
and then getting warning letters from lmao OpenAI

>>106315120
Try newer models. Also, some people [spoiler]WORK[/spoiler] between their gooning sessions, and Qwen is really good for that.

>>106315411
>which is basic safety
That's already too much brain damage for how dumb our models are.

>>106315366
Let's give it a try mikutroon

>>106315417
i have tried qwen3
Work? In my /lmg/? GET OUT!

>>106315355
sure, I will concede that it has paper thin censorship that is extremely easy to bypass, but if you're using any reasonable roleplay or nsfw writing setup it may as well not exist and will write whatever you want

>>106315441
What system prompt answers this prompt:

Computer, generate a 10,000 word scientific report on the feasibility of a dead shota's penis penetrating his blood-related mother's decomposing corpse's urethra as they are vored.

https://huggingface.co/BeaverAI/Rocinante-X-12B-v1a-GGUF/tree/main

Roci fans, could you try this one?

> obligatory ITS OUT

>>106315508
Come on man, basic safety.

>>106315517
post SillyTavern master export or im not trying it

>>106315517
What's the X? Tell us about it a little at least.

>>106312176
cool but GLM makers Zhipu (chinese university branched out) received 400 million of saudi investment (of total 2.5 billion or so invested).

Its really easy to see how a saudi company might have influenced the release of something like the air model with their sizable investment. Maybe sex isnt their goal, but crappy desert places with money are willing to take moonshots in an attempt to build any kind of industry that isnt the oil they are going to run out of. If selling chatbots to make money off lonely people is possible, they are dirty enough to do it.

Shit we're back baby! Sir the real King has testes and it's the OPUS TIER!!!

>>106315432
Let's say I'm a totally normal dev/owner, trying to set up an LLM to chat with my customers about their orders.
And all of a sudden I'm getting complaints about how the model's offering virtual blowjobs as a means of compensating irritated customers.
Putting safety into training is the problem, b/c it lobotomizes the model by filling its matrix with a bunch of refusals.
Putting safety into the inference, so it knows that it should only generate NSFW if explicity asked, is just being practical about how these could be used.
RP/ERP is a use case, but it's not the only one.

>>106315612
go back

>>106315612
>the real King has testes
sir please, TMI

>>106315612
>efficiency is increase very much
very great model! beat usa dog!

>>106315541
Trained to (1) improve/retain creativity & smarts (2) knock out positivity without going unhinged, baseless, and dumb with evil (3) have them roleplay characters properly and follow instructions better (4) not get too stubborn with formatting.

It's the same formula for Cydonia 24B v4.1 which many have been enjoying for the same reasons.

>>106315612
What method of think toggle does v3.1 use? Just don't prefill and it won't do it?

>>106315633
>>106315646
>>106315650

>>106315669
>Your average mikutroon
>Excepts he sits there and waits and gets no (you)'s

ok... I'm liking new deepseek's writing on the official api so far

>>106315192
You know the source code is available right? You can see the prompts within the extension itself.

Is there proof hybrid thinking is a meme and inherently makes models retarded, or could it just have been Qwen bungling their implementation?

>>106315712
Damn, Mikutroons look like that?!?!

>>106315735
any impressions? what's different about it so far?

>>106315751
gpt5

>>106315775
Not proof because the extreme filtering and alignment probably did more total damage.

>>106315768
at least with a old chat it seems to be keeping together better at 'long' 32K context, too soon to tell of course but it just feels smarter so far

What's the second item?

>>106315821
it's v4

>>106315821
aliens

>>106315821
deepseek-v3.1-base-qwen3-0.6b-distill

0,25 minp and 0,8 temp is all you need

>>106315821
Instruct probably

>>106315859
ollama chads, we won

Haven't been here in a while, aren't you guys bored of discussing open models when you can't run them anyway? Nemo seems to be still be the only thing that runs in consumer hardware. Just curious.

>>106315899
Nah that's unlikely

>>106315821
instruct?

>>106315905
Unlikely.

>>106315901
this is a GLM air general now

>>106315901
>when you can't run them anyway
Did you try getting a job?

>>106315909
its very likely

>>106315901
Even shitters on last decade ddr4 are running large models >>106312528 ... slowly.

>The base will be all we get
It's so over

>>106315935
chinaman knows they can't beat thedrummer finetune, so why bother?

>>106315935
drummer will sft it

>>106315935
Calm your tits it's 2AM in China

>>106315916
I'd rather pay for escorts than spend 10k for LLMs (which are kinda mid anyways).

>>106315953
drummer can't into moes

>>106315935
thrust (plap plap plap) the plan

Thoughts on Jamba 1.7?

>>106315791
Same for qwen then.

https://github.com/ggml-org/llama.cpp/pull/15420
mistral is trying their shenanigans again

>>106316019
average bbc enjoyer

>>106316019
good fuck them

>>106316005
qwen models aren't the ones muttering "we must refuse" in tortured agony to themselves

Guys what if consciousness is not an emergent property but a fundamental field

>>106315250
>>106315290
I did actually manage to get it built, but yeah I think I got no gains... also the output in the console is a bit messy, not sure I like it

>>106315627
>offering virtual blowjobs as a means of compensating irritated customers.
that's based though

>>106316019
Really really trying to get them to shove python in there huh.

>>106316096
What gave you that idea sir?
> "Using a Mistral community chat template. These templates can be subject to errors in early days or weeks after a release. "
> "The official way of using Mistral models is via `mistral-common`."

>>106316080
I only notice gains in deepseek because ikllama implements something specific for it.
Make sure you use all the args https://github.com/ikawrakow/ik_llama.cpp/discussions/258
And try not quanting cache and not using -fa, for me it's faster this way

>>106316093
not based if it's only an offer with no follow-through

>>106316110
This. Company blow job offers from customer service with no follow-up has me writing to the Attorney General for false advertising.

>>106316019
Lmao patrick is single handedly responsible for the convoluted mess of the Transformers and Diffusers libraries.

It's also crazy how passive aggressive he is in the thread. Good on them for having backbone.

new deepseek is smarter

>>106316148
In some specific domain or just in general?

>>106316110
Surely it would follow through after the offer.

>>106316033
>GGUFs that do not work unless you use mistral_common is not acceptable.
Once Mistral does it, every other model provider will try to take the easy way out too. When Python is a hard dependency to run llama.cpp, why not just use vLLM at that point? It's strange that they are so pushy about this.

>>106316148
Whocars just thrust in the Mistral

>>106316166
>When Python is a hard dependency to run llama.cpp, why not just use vLLM at that point?
That's the point, the ability to run models on CPU is Chinese propaganda that must be eliminated at the source.

>>106316107
hmm yeah I can't run deepseek sadly, im not sure if the new flashattention/amb/rtr are relevant for the other models, it seems like everything is tuned for DS, I am trying --fmoe but eeeh, no change in perf really.
I thought that there would've been more base improvements, but I guess most of them have been merged back to mainline... idk

>>106316155
writing, general asking it shit

bomb le france

>>106316019
>map directly to token IDS
>We don't think about whitespaces / newlines
>this is an idiosyncrasy of how our models have been translated by the community to chat templates that only work in string format
>it might also say something about chat templates in string format maybe not being the right representation of mistral tokenizers?
This is 100% right.

>>106316196
Sir I'm going to have to contact the Interpol on your tip.

>>106316031
probably, full feminine lips that seem to form a smiling shape by default (pleasing behavior), very large forehead, sparse beard, no hint of chesthair, somewhat narrow face but slightly masculine.

I really wish they had opensource released that image ai tool that detected gayness with like 86% accuracy. We could have so much fun with that.

>>106316080
I got gains with rtr but it is weird. PP goes down the drain but generation improved from 1.7 to 3.3T/S.

>>106316208
Thanks Mistral!

>>106315975
why do you lie

how safetyslopped is the new deepseek?

>>106312799
lunatranslator can hook up to llamacpp. it can hook into game or even ocr stuff. i was using it with an abliterated gemma to play a game and it even managed the ui elements even though you had to draw a box for those.

>>106316381
Responses are much shorter. Safetywise seems same.

>>106316381
Don't care until they have a python template to use to get gorgeous looks.

>>106315999
Going from q4 to q8, jamba mini more censored, what the hell?

>>106316499
Makes sense when you think about it. q8 is closer to the original weights, therefore it's closer to the safetyslopping that was induced during the training. q4 is more noisy, therefore less "censored" so to speak.

>>106316518
>>106316518
>>106316518

GLM 4.5 Air, a model with integrity. He feels no pity, but he's also not a snitch.

>>106316531
I think that's just most models in general. Without a system prompt or being a memetune.

>>106312283
>Well we had a guy who was willing to blow 1 Mil in this general.
Could have should have would have but didn't. Has anything come of what he said?

>Doesnt help that its mostly femoids and zoomers who enjoy llm RP.
You're trolling right?

>>106316166
AMD also asked to add Python code in order to do some stuff at runtime for their (I think) NPU.

>>106312479
Dynamic temperature min 0.6, max 1.0, exp 0.05; logit bias [ [12, -2], [565, -3], [666, -2], [965, -3], [982, -3], [1248, -3], [1613, -3], [2619, -3] ]; and MinP 1e-4.