12 stories tagged GPU
A $400 million chip-backed loan from investors who previously financed GPU production now targets inference chips, signaling a major shift in AI infrastructure funding that could lower costs and speed up AI applications for everyone.
Modular has updated its Max AI models to run on Apple Silicon GPUs, making powerful AI tools more accessible. This change allows Mac users to leverage their hardware for advanced AI tasks without needing external servers.
Apex-1-flash is a lightweight, 4-billion-parameter AI model finetuned on an RTX 5070 to perform complex reasoning tasks. It's designed to be highly efficient and accessible, running easily on consumer-grade hardware without requiring expensive servers.
LLM-d is a new open-source tool that facilitates distributed inference for large language models across multiple GPUs. It aims to make running powerful AI models more accessible for hobbyists and small developers without requiring specialized hardware or deep technical expertise.
A tech hobbyist successfully ran a massive AI language model using 768GB of Intel Optane memory and a single graphics card. The setup, called Local Kimi K2.5, achieved roughly 4 tokens per second, showing how powerful consumer hardware can be for cutting-edge AI experiments.
Kog AI has developed a method to run large language models at 3,000 tokens per second on standard GPUs, making advanced AI faster and more accessible. This breakthrough could significantly reduce the cost and complexity of AI applications.
Nvidia's focus on AI has alienated its core gaming community, with gamers expressing frustration over GPU shortages and prioritization. The company faces a delicate balancing act between its lucrative AI business and traditional gaming market.
Laimark is an 8B parameter LLM designed to self-improve, and it runs on consumer-grade GPUs. This development could democratize advanced AI capabilities for individual developers and small teams.
Kepler Communications has launched the largest orbital compute cluster with 40 GPUs, and Sophia Space is its first customer. This marks a significant milestone in space-based AI infrastructure.
Hugging Face has released Waypoint-1.5, an open-source model designed to generate high-fidelity interactive worlds that run efficiently on consumer-grade GPUs. This release lowers the barrier to entry for developers building immersive, real-time 3D environments without requiring massive enterprise infrastructure.
New attacks exploit GPU memory to hijack CPUs. GDDRHammer, GeForge, and GPUBreach give attackers complete control.
MegaTrain enables full precision training of large language models on a single GPU. It stores parameters in host memory and uses GPUs as compute engines.