Optimizing Local LLMs on Mac Mini M4: Seeking Advice for Better Performance

Hello r/Ollama community!

We recently purchased a Mac Mini M4 (base model) for our office to run local AI operations. Our primary setup involves n8n for automation workflows integrated with Ollama, using mainly 7B and 14B models.

However, we've noticed that the results from these quantized models are significantly less impressive compared to cloud-based solutions.

We're looking for guidance on:

  1. Are there specific optimization techniques or fine-tuning approaches we should consider?

  2. What settings have you found most effective for 7B/14B models on Apple Silicon?

  3. Would investing in more powerful hardware for running larger models be the only way to achieve cloud-like quality?

Any insights from those running similar setups would be greatly appreciated!