Why Software is Critical for AI Inference Accelerators
By Geoff Tate, Flex Logix (April 23, 2020)
Inference accelerators represent an incredible market opportunity not only to chip and IP companies, but also to the customers who desperately need them. As inference accelerators come to market, a common comment we hear is: “Why is my inference chip not performing like it was designed to?”
Oftentimes, the simple answer is the software.
Software is key
All inference accelerators today are programmable because customers believe their model will evolve over time. This programmability will allow them to take advantage of enhancements in the future, something that would not be possible with hard-wired accelerators. However, customers want this programmability in a way where they can get the most throughput for a certain cost, and for a certain amount of power. This means they have to use the hardware very efficiently. The only way to do this is to design the software in parallel with the hardware to make sure they work together very well to achieve the maximum throughput.
E-mail This Article | Printer-Friendly Page |
|
Flex Logix Technologies, Inc. Hot IP
Related Articles
- Why Interlaken is a great choice for architecting chip to chip communications in AI chips
- Revolutionizing AI Inference: Unveiling the Future of Neural Processing
- Partitioning to optimize AI inference for multi-core platforms
- Will Generative AI Help or Harm Embedded Software Developers?
- The Expanding Markets for Edge AI Inference