F5F Stay Refreshed Hardware Desktop What hardware for AI...

What hardware for AI...

What hardware for AI...

I
InvadedLands
Junior Member
11
06-11-2022, 02:59 AM
#1
Hello everyone.
If you're looking for alternatives to purchasing a threadripper for $1200 and a 4090 at around $1800, what recommendations do you have?
I've set up ollama, Docker Desktop, and an open web UI, then added models such as deepseek r1, llama 3.2, etc.
I'm planning to build a system that can run AI programs like llama or deepseek at high settings. I understand that the better the hardware, the better the performance for AI tasks.
Could you suggest suitable CPU, motherboard, and GPU options that would work well without breaking the bank?
For example, I recently used 7b parameters with DeepSeek and it ran smoothly. I tried 8b and it was still fast. With 14b it took about two minutes for the response to come back. This happened on my spouse's laptop—a ROG Zephyrus G16 with a GPU of that model.
Any advice on hardware types and prices would be really helpful.
Thanks!
I
InvadedLands
06-11-2022, 02:59 AM #1

Hello everyone.
If you're looking for alternatives to purchasing a threadripper for $1200 and a 4090 at around $1800, what recommendations do you have?
I've set up ollama, Docker Desktop, and an open web UI, then added models such as deepseek r1, llama 3.2, etc.
I'm planning to build a system that can run AI programs like llama or deepseek at high settings. I understand that the better the hardware, the better the performance for AI tasks.
Could you suggest suitable CPU, motherboard, and GPU options that would work well without breaking the bank?
For example, I recently used 7b parameters with DeepSeek and it ran smoothly. I tried 8b and it was still fast. With 14b it took about two minutes for the response to come back. This happened on my spouse's laptop—a ROG Zephyrus G16 with a GPU of that model.
Any advice on hardware types and prices would be really helpful.
Thanks!

E
Elias218
Junior Member
5
06-14-2022, 12:27 PM
#2
From my understanding, the higher the #B value, the more memory you'll require. Regarding the CPU, most models indicate that a multicore processor is necessary, and it's likely all these AI applications will also need an RTX to function properly.
E
Elias218
06-14-2022, 12:27 PM #2

From my understanding, the higher the #B value, the more memory you'll require. Regarding the CPU, most models indicate that a multicore processor is necessary, and it's likely all these AI applications will also need an RTX to function properly.

X
xXSuperNovaXx
Posting Freak
811
06-21-2022, 09:24 PM
#3
Don't need TR, just purchase a standard CPU and allocate most of your budget to the graphics card.
X
xXSuperNovaXx
06-21-2022, 09:24 PM #3

Don't need TR, just purchase a standard CPU and allocate most of your budget to the graphics card.

A
AldoMino5
Junior Member
24
06-27-2022, 02:22 PM
#4
Around 3000.
A
AldoMino5
06-27-2022, 02:22 PM #4

Around 3000.