Cactus v2 Brings On-Device AI with Cloud Fallback to Any Arm Device
Cactus v2 is an on-device AI inference platform that runs on any Arm device — from iPhones to Raspberry Pis — with a built-in confidence-based routing system that hands off tough queries to the cloud. It also introduces lossless 4-bit quantization, GPU acceleration via Apple Metal, and a converter for any PyTorch model.