Llama 3.1 405b LOCAL AI Home Server on 7995WX Threadripper and 4090

Running a 405B LLM on your home server is possible? YES! Today we give this a spin on out 7995WX and we also test the 3090 and 4090 head to head on the 6400 MHz DDR5 we have in this system. Two fun topics as we check out performance and uncover a mystery in performance. RTX 4090 https://geni.us/4090GPU_5N_s Threadripper 7995wx https://geni.us/AMD_TR_PRO_7995WX Threadripper 7995wx Build https://youtu.be/YQZ2HWonnGA QUAD 3090 AI HOME SERVER BUILD https://youtu.be/JN4EhaM7vyw PARTS GPU Rack Frame https://geni.us/GPU_Rack_Frame Supermicro H12ssl-i MOBO (better option vs mz32-ar0) https://geni.us/MBD_H12SSL-I-O Gigabyte MZ32-AR0 MOBO https://geni.us/mz32-ar0_motherboard AMD 7V13 (newer, faster vs 7702) https://geni.us/EPYC_7V13_CPU RTX 3090 24GB GPU (x4) https://geni.us/GPU3090 256GB (8x32GB) DDR4 2400 RAM https://geni.us/256GB_DDR4_RAM PCIe4 Risers (x4) https://geni.us/PCIe4_Riser_Cable AMD SP3 Air Cooler (easier vs water cooler) https://geni.us/EPYC_SP3_COOLER iCUE H170i water cooler https://geni.us/iCUE_H170i_Capellix (sTRX4 fits SP3 and retention kit comes with the CAPELLIX) CORSAIR HX1500i PSU https://geni.us/Corsair_HX1500iPSU 4i SFF-8654 to 4i SFF-8654 (x4, not needed for H12SSL-i) https://geni.us/SFF8654_to_SFF8654 ARCTIC MX4 Thermal Paste https://geni.us/Arctic_ThermalPaste Kritical Thermal GPU Pads https://geni.us/Kritical-Thermal-Pads HDD Rack Screws for Fans https://geni.us/HDD_RackScrews Be sure to 👍✅Subscribe✅👍 for more content like this! Join this channel https://www.youtube.com/@DigitalSpaceport/join Digital Spaceport Website https://digitalspaceport.com Patreon https://www.patreon.com/digitalspaceport Buy Me a Coffee https://buymeacoffee.com/digitalspaceport Please share this video to help spread the word and drop a comment below with your thoughts or questions. Thanks for watching! Chapters 0:00 Threadripper 7995WX AI Testing 1:00 Running 405B Local AI Server 5:56 Qwen 2.5 32b on 4090 8:34 Llama 3.2 3b on 4090 10:17 Llama 3.1 8b fp16 on 4090 12:15 Qwen 2.5 32b on 3090 14:32 Llama 3.1 8b on 3090 15:47 Llama 3.2 3b on 3090 ***** As an Amazon Associate I earn from qualifying purchases. When you click on links to various merchants on this site and make a purchase, this can result in this site earning a commission. Affiliate programs and affiliations include, but are not limited to, the eBay Partner Network. *****