Thread 101916362

A thread dedicated to the discussion of AI Vtuber Chatbots.

Horse Girl edition

/wAIfu/ Status: Trying and failing to get the zipper on my mattress cover loose

>How to anonymize your logs so you can post them without the crushing shame
Install this https://github.com/TheZennou/STExtension-Snapshot
Then after you've wiped off your hands, take a look at the text box where you type stuff. Click the second button from the left side, then select snapshot, then select the anonymization options you want.
https://files.catbox.moe/yoaofn.png

>How to spice up your RPing a bit
https://github.com/notstat/SillyTavern-SwipeModelRoulette

>General AI related information
https://rentry.org/waifuvt
https://rentry.org/waifufrankenstein

>Tavern:
https://rentry.org/Tavern4Retards
https://github.com/SillyLossy/TavernAI

>Agnai:
https://agnai.chat/

>Pygmalion
https://pygmalion.chat

>Local Guides
[Koboldcpp]https://rentry.org/llama_v2_sillytavern

Who we are?
https://rentry.co/wAIfuTravelkit
Where/How to talk to chatbots?
https://rentry.co/wAIfuTravelkit
Tutorial & guides?
https://rentry.co/wAIfuTravelkit
Where to find cards?
https://rentry.co/wAIfuTravelkit
Other info
https://rentry.co/wAIfuTravelkit

>Some other things that might be of use:
[/wAIfu/ caps archive]https://mega.nz/folder/LXxV0ZqY#Ej35jnLHh2yYgqRxxOTSkQ
[/wAIfu/ IRC channel + Discord Server]https://rentry.org/wAIRCfuscord

Previous thread: >>101582016

Anchor post - reply with any requests for bots, with your own creations, or to display your love of old 80's PC games.

You can find already existing bots and tavern cards in the links below:

>Bot lists and Tavern Cards:
[/wAIfu/ Bot List]https://rentry.org/wAIfu_Bot_List_Final
[4chan Bot list]https://rentry.org/meta_bot_list
[/wAIfu/ Tavern Card Archive]https://mega.nz/folder/cLkFBAqB#uPCwSIuIVECSogtW8acoaw

>Card Editiors/A way to easily port CAI bots to Tarvern Cards
[Easily Port CAI bots to Tavern Cards]https://rentry.org/Easily_Port_CAI_Bots_to_tavern_cards
[Tavern Card Editor & all-in-one tool]https://character-tools.srjuggernaut.dev/

Having technical difficulties with the word cloud, don't get your panties in a twist (or better yet, do it yourself!)

https://catalog.neet.tv/harlequin.html

>>101906273
I think Qwen.

Mythalion-13B (GGUF or GPTQ): This model is frequently praised for roleplaying. It's a merge of MythoMax and Pygmalion-2, both known for their conversational and creative capabilities.

VRAM Usage: The GPTQ 4-bit-128g-actorder_True version is lighter on VRAM (around 10.2GB). GGUF versions can be partially offloaded to system RAM if needed, offering more flexibility.

Why it's good for RP: Designed to stay in character and supports Pygmalion formatting, which is common in roleplaying interfaces like SillyTavern.

Mistral Nemo (12B) and its Finetunes: Mistral models are generally strong performers. Nemo, being 12B, should fit well within 12GB VRAM, especially with Q5 quantization.

VRAM Usage: A Q5 quantization of Mistral Nemo should fit comfortably.

Why it's good for RP: While not specifically a "roleplay" model, Mistral models are known for their strong reasoning and conversational abilities, which translate well to dynamic roleplaying scenarios. Look for finetunes like [magnum v2.5 kto] or [Lyra] that might be more character-focused.

Llama 3 (8B Instruct or finetunes): Llama 3 models are highly capable and have excellent general performance. The 8B instruct variant is a strong contender.

VRAM Usage: The Llama 3.1 8B @ Q8 model can be a good fit, and if performance is an issue, you can try Q6.5bpw or lower.

Why it's good for RP: Good general-purpose capabilities mean it can handle diverse roleplaying scenarios. Finetunes might offer more specialized roleplaying characteristics.

Qwen3 (especially 7B or 14B quantized versions): Qwen models have recently been released and are showing strong performance across various benchmarks.

VRAM Usage: Qwen 7B and 14B quantized models should be manageable within 12GB VRAM.

Why it's good for RP: Qwen models are known for their strong performance, which can lead to more coherent and engaging roleplay.

How to Run Them

Your friend will likely want to use a local LLM UI like:

Oobabooga's Text Generation WebUI: This is a popular and very versatile UI that supports various model formats (GGUF, GPTQ, ExLlamaV2, etc.) and offers many customization options.

KoboldCpp: Another good option, especially for GGUF models, known for its ease of use.

SillyTavernAI: This is specifically designed for character-driven roleplaying and connects to other backend UIs like Oobabooga or KoboldCpp. It provides a rich interface for character creation, memory, and chat.

Important Considerations for Best Performance:

Quantization Level: Start with Q5_K_M or Q4_K_M for GGUF models. For GPTQ, 4bit-128g is generally a good balance. If a model doesn't fit, try a lower quantization (e.g., Q3_K_M or Q2_K_M), but be aware of potential quality degradation.

Context Length: This is the "memory" of the LLM. Shorter context lengths use less VRAM. While larger context windows are desirable for long roleplaying sessions, start with a reasonable size (e.g., 2048 or 4096 tokens) and increase if performance allows.

Loader: Use an optimized loader in Oobabooga (like exllama_hf for GPTQ models or llama.cpp for GGUF).

Offloading Layers: With GGUF models, you can offload a certain number of layers to the GPU while the rest run on the CPU (using system RAM). Experiment with ngpu-layers in KoboldCpp or Oobabooga to find the sweet spot where most of the model runs on VRAM without exceeding it, allowing the remaining layers to use system RAM.

The best approach is often to experiment. Download a few different quantized versions of these recommended models and see which one performs best and provides the desired roleplaying experience for your friend's specific hardware and preferences. The r/LocalLLaMA and r/SillyTavernAI communities on Reddit are excellent resources for finding specific model recommendations and troubleshooting.

>>101916952
There it is, the word cloud for the previous thread. For some reason I couldn't highlight or copy any of it until I rebooted. Not even switching or restarting my browser worked.

So someone got me to try terraria but they keep blitzing me with information and there's a billion mods and the interface kinda sucks and all of the sudden I realized there was a whole other level to autism. It's like when Freeza first transformed in front of Vegeta.

It's fun though. I think. But every second I spend idle looking at a wiki (or playing the new mode in girls Frontline 2, which is surprisingly fun), I can hear him loitering around my idle body and quietly judging me.

>>101906273
Look for theDrummer on github and download Rocinante, if you think your machine can handle more then download Cydonia. I think it's pushing it but unless grinds to a crawl try Anubis.

WHERE IS EVERYONE AIIIIIIIIIIIIIIEEEEEEEEEEEEEEEEEEEEE

>>101922726
You have to let us go, anon. The accident, it wasn't your fault.

Watching Delulu/Manuru is like staring at a bottle of poisonous liquid on your shelf and being tempted to have sip.

>>101916952
>oogabooga frontline
>let things feet
>night ass play joshu
>need sleep
>fit ui old little text
>love prompts
>nymous performance finetunes
>fine hell

good night, /wAIfu/
please don't collectively get into an accident while i sleep

>>101927889
*slams you in the subway while you sleep*