Archive 2. March 2026

waybarrios/vllm-mlx: OpenAI and Anthropic compatible server for Apple Silicon. I use this to run mlx-community/gemma-3-12b-it-4bit on my MacBook Air. It works very well, a small shell script to start the server and then I am autonomous. Not as comfortable as Ollama, but it perfectly supports Apple's MLX and thus makes good use of Silicon.

mlx-community/gemma-3-12b-it-4bit · Hugging Face is currently the best model for local operation, allowing me to implement image captioning and even local chat. It's not the fastest, as it's quite large, but it's absolutely suitable for offline operation if I come up with a few mechanisms for batch processing of images, etc. This could be super exciting for vacation times. An image description might take a minute, but hey, no dependencies.