Japanese AI startup Sakana AI has launched The AI CUDA Engineer, an agentic framework that automates the invention and optimisation of CUDA kernels for improved GPU efficiency.
The corporate claims the framework can generate CUDA kernels with speedups starting from 10 to 100 instances over frequent PyTorch operations and as much as 5 instances quicker than present CUDA kernels utilized in manufacturing.
CUDA is a low-level programming interface that allows direct entry to NVIDIA GPUs for parallel computation. Optimising CUDA kernels manually requires vital experience in GPU structure. Sakana AI’s new system makes use of LLMs and evolutionary optimisation strategies to automate this course of, making high-performance CUDA kernel growth extra accessible.
“The best autonomous coding agent I’ve seen not too long ago: use AI to jot down higher CUDA kernels to speed up AI. AutoML is so again!” mentioned Jim Fan, senior analysis supervisor and lead of embodied AI at NVIDIA. He added that probably the most impactful solution to utilise compute sources is by enhancing the long run productiveness of that exact same compute.
In line with Sakana AI, The AI CUDA Engineer converts customary PyTorch code into optimised CUDA kernels via a multi-stage pipeline. Initially, it interprets PyTorch operations into CUDA kernels, typically enhancing runtime with out express tuning. The system then applies evolutionary optimisation, utilizing methods equivalent to ‘crossover’ operations and an ‘innovation archive’ to refine efficiency.
“Our method is able to effectively fusing numerous kernel operations and might outperform a number of present accelerated operations,” the corporate mentioned. The framework builds on the corporate’s earlier analysis with The AI Scientist, which explored automating AI analysis. The AI CUDA Engineer extends this idea to kernel optimisation, utilizing AI to boost AI efficiency.
Sakana AI reported that The AI CUDA Engineer has efficiently translated greater than 230 out of 250 evaluated PyTorch operations. It has additionally generated over 30,000 CUDA kernels, of which over 17,000 have been verified for correctness. Roughly 50% of those kernels outperform native PyTorch implementations.
The corporate has made the dataset obtainable below a CC-By-4.0 licence on Hugging Face. It consists of reference implementations, profiling knowledge, and efficiency comparisons towards native PyTorch runtimes.
Sakana AI has additionally launched an interactive web site the place customers can discover the dataset and leaderboard rankings of optimised kernels. The platform gives entry to kernel code, efficiency metrics, and associated optimisation experiments.
The put up Sakana’s AI CUDA Engineer Delivers As much as 100x Pace Features Over PyTorch appeared first on Analytics India Journal.
