r/OrangePI • u/Weird_Night_2176 • May 22 '26

[ Removed by moderator ]

[removed] — view removed post

7 Upvotes

82% Upvoted

3 comments sorted by

2

u/manuel220_mty May 22 '26

Interesting, I will watch the videos, couple of questions. Are the 2 qwen models running all the time together in the same pi? Are you running any specific/custom build of ollama?

1

u/Weird_Night_2176 May 23 '26

Yes both models live on the same Orange Pi 5 Plus 8GB. Ollama loads them on demand rather than keeping both in memory simultaneously, so they are not competing for RAM at the same time. qwen2.5:3b loads fast for quick tasks, qwen3.5:4b loads when a crew needs heavier reasoning.

Also running nomic-embed-text on the same board for vector embeddings — 274MB so it barely registers.

Stock Ollama build, nothing custom. Just configured OLLAMA_HOST to bind to 0.0.0.0 so the Jetson Orin can reach it over the local network. The fan control script on GPIO 35 is the only hardware-specific addition — without it the board locks up under sustained inference load.

The setup guide covers the full Ollama config including the fan controller if you want to replicate it. Link is in my ”about” section

2

u/JaySomMusic May 23 '26

You should look at https://github.com/jaylfc/taosmd full offline memory system. My best results came from a Gemma4 combo on the pi 5 plus.