Any relatively new gaming PC from the last, what, 4? Years has enough power to run local LLMs. Maybe not the ginormous 70GB behemoth models, but the toned down ones are pretty damn good and if you don’t mind waiting a few seconds while it thinks, you can run it completely locally as much as you want, and whenever you want.
You would benefit from it with some GPU offloading, this would considerably accelerate the speed of the answers. But you only need enough RAM to load the model at the bare minimum.
I use chatgpt a lot, what is the best non-billionaire funded llm? I really need to change to one that doesn’t worsen the world…
That may be hard, seeing as all AIs use ungodly amounts of electricity. So I’d say they all worsen the world.
While deepseek is billionaire funded it still should be better if run locally I don’t think Foss llms are at that level yet
Thanks, another person mentioned it, I’m trying it now, hopefully it suits my needs.
Try hosting locally DeepSync R1, for me the results are similar to ChatGPT without needing to send any into on the internet.
LM Studio is a good start.
Don’t you need a fast GPU to do so?
Any relatively new gaming PC from the last, what, 4? Years has enough power to run local LLMs. Maybe not the ginormous 70GB behemoth models, but the toned down ones are pretty damn good and if you don’t mind waiting a few seconds while it thinks, you can run it completely locally as much as you want, and whenever you want.
You would benefit from it with some GPU offloading, this would considerably accelerate the speed of the answers. But you only need enough RAM to load the model at the bare minimum.
Thanks I’m trying it now!