@rpdon, thank you so much. Performed your suggested edit and restart.
Ran "ollama run llama3.2-vision" Downloaded 6 GB + 1.9 GB worth of manifest files. Like you said, very slow snails pace, but functional. Using psensor to monitor temperatures. Good that Rpi5 has PCM fan.
Asking, "what are prime numbers between 1 and 100"? took about 15 minutes. CPU temp 58C. Fan RPM max 5300.
Estimating it cost $1B for people, hardware and energy to train Llama3.2, that is quite amazing it will run on Rpi5 4 GB.
I am running a NVME hat which I think has the raspbian OS but there is also a microSD card in the slot. I actually don't know where the RAM disc is operating. Open to any speed improvement suggestions. Thanks again.
energyi
Ran "ollama run llama3.2-vision" Downloaded 6 GB + 1.9 GB worth of manifest files. Like you said, very slow snails pace, but functional. Using psensor to monitor temperatures. Good that Rpi5 has PCM fan.
Asking, "what are prime numbers between 1 and 100"? took about 15 minutes. CPU temp 58C. Fan RPM max 5300.
Estimating it cost $1B for people, hardware and energy to train Llama3.2, that is quite amazing it will run on Rpi5 4 GB.
I am running a NVME hat which I think has the raspbian OS but there is also a microSD card in the slot. I actually don't know where the RAM disc is operating. Open to any speed improvement suggestions. Thanks again.
energyi
Statistics: Posted by energyi — Mon Feb 10, 2025 5:41 pm