☰
×
Login Register an account
Top Submissions Explore Upgoat Search Random Subverse Random Post Colorize! Site Rules Donate
5

What LLM Should I use (download and use locally)?

submitted by rapid_water to AskUpgoat 10 monthsJul 31, 2024 02:55:41 ago (+5/-0)     (AskUpgoat)

I want to test out LLMs. What uncensored version should I get? I'm confused over the many types: mistral, dolphin, phi, llama. Help me out.


4 comments block

sort by: hot new top old low

[ - ] TheYiddler 1 point 10 monthsJul 31, 2024 06:34:30 ago (+1/-0)

Depends what hardware you have. You want a big model, at least 30b, but you can make it easier to run with quantization. A five bit quantization is like a jpg. It's a bit grainy but still very recognizable. Less bits than that and it goes to crap.

Mixtral 8x7b is nice and versatile.

[ - ] Love240 1 point 10 monthsJul 31, 2024 02:59:38 ago (+1/-0)

As far as I understand, the Llama ones are pretty okay. Llama 3.1, I think.

[ - ] MaryXmas 0 points 10 monthsJul 31, 2024 08:25:45 ago (+0/-0)

What is your goal? Find a model that fits that. Zoom over to hugging face and you can use a bunch. I think it is less about the model and more about the training set.