juxt/allium: A language for sharpening intent alongside implementation. is a very exciting project. Specifically, a specification language for the behavior of software systems. A language for which there is no runtime environment or compiler, except the LLM. Implemented as Agent Skills. Super exciting because it also includes a Distill, with which you can analyze existing software and retroactively build specifications, or work out a specification in an interview process with the AI that is much more precisely understandable to an LLM than general English. A funny detail on the side: I tried Allium with Qwen3-Coder-Next, my current favorite model for local hosting, in pi.dev. I couldn't install the binary for Allium (a syntax checker and linter) with homebrew, so pi.dev simply downloaded the binary and installed it itself.
Archive 27. March 2026
scitrera/cuda-containers: Scitrera builds of various CUDA containers for version consistency, starting primarily with NVIDIA DGX Spark Containers - I'm currently a big fan of eugr/vllm-node as a base package because it always provides up-to-date versions for vLLM, but if I want to play around with sglang sometime, this is probably the most similar project. I'm particularly interested in EAGLE-3 speculative decoding - basically, tokens are generated in parallel via a very small model and the main model checks what fits and takes it, or generates itself if necessary. This way you can often have a third of the tokens generated via a much faster simple model in the <3B range and only pass every third one through the large model.
thushan/olla: High-performance lightweight proxy and load balancer for LLM infrastructure. Intelligent routing, automatic failover and unified model discovery across local and remote inference backend might be the better choice following the LiteLLM debacle (hacked supply chain with data extractor in the package). For me, it's definitely interesting because I simply want to run two models and make them available under a single endpoint, and all the other packages are significantly overkill for that.