NVIDIA开源了cuDNN中的超过20个MoE内核和NSA稀疏注意力内核,结束了其12年的闭源状态。此举旨在推动开放内核创新。
For the past 12 years, cuDNN has been completely closed sourced (besides the .h files), until this week! OVER 20 MoE kernels & NSA sparse attention kernels from cuDNN has been open sourced! Great work to @manicely6005 & the rest of the team on seeing that parts of NVIDIA are moving towards open kernels! open source kernels drive innovation! (1/3) 🧵
likes: 434 | retweets: 56 | replies: 6 | views: 30499