Faster, Smaller and More Accurate Edge AI Using Deeplite and Andes Technology Software + Hardware

MONTREAL, CANADA and HSINCHU, TAIWAN – December 3, 2020 – The push for low-power and low-latency deep learning models, computing hardware, and systems for artificial intelligence (AI) inference on edge devices continues to create exciting new opportunities. There has been unprecedented interest from industry stakeholders in the development of hardware and software solutions for on-device deep learning, also called Edge AI. This has already begun to yield progress on hallmark applications such as keyword spotting in audio classification, anomaly detection and, in this case, person detection in computer vision applications. Specifically, tinyML, the branch of machine learning tailored to ultra-low power systems, holds tremendous promise. The efficiency of proposed solutions (milliwatt or even microwatt power consumption) and vast applicability and deployment of such devices in real-world settings will lead to over 100 billion IoT sensors and devices expected to ship over the next 5 years 1. The future of deep learning is poised to provide significant benefits to customers and end-users by way of affordable, eco-friendly and more accessible intelligence than ever before.

Today, Andes Technology and Deeplite Inc. are excited to announce the latest results in their partnership for AI-powered applications using Deeplite’s unique optimization software and Andes’ low-power Andes RISC-V CPU cores. The partnership focuses on compressing and accelerating the well-known Visual Wake Words (VWW) application, where a tiny embedded camera can detect a person in images. Together, Deeplite and Andes achieved industry-leading results, producing various optimized INT8 models from the floating point based Mobilenet-v1-0.25x model. The first set of results focused on increasing accuracy. Our accuracy-focused INT8 model optimization achieves 2.7% higher accuracy, 1.7x (172 KB) smaller size, and 9% faster execution. The second set of results focused on maximizing compression. Our model-size-focused optimization achieved 2.3x (121KB) smaller size, slightly higher accuracy (0.7%), and 15% faster execution when compared to int8 model provided by TensorFlow Lite Micro.

Table 1 Trade-offs in model size, accuracy and execution time for reference and optimized models

“We are determined to provide the most efficient and accurate solutions possible for low-power devices, particularly as edge AI is increasingly deployed in smart assistants, security cameras and smart manufacturing applications.” said Dr. Charlie Su, CTO and Executive VP of Andes Technology. “Deeplite’s cutting-edge software offers an effective way to optimize AI models with enhanced performance to current frameworks. We leverage Deeplite’s abilities for our AndeStar™ V5 architecture, the first commercially available RISC-V CPU cores with DSP SIMD ISA, to enable our customers to use the AI models most suitable for their applications.”

“The Andes™ RISC-V CPU cores provide ideal hardware examples to show the benefits of Deeplite’s model optimization, deploying sophisticated intelligence previously not possible in low cost, battery-powered devices,” said Nick Romano, CEO of Deeplite. “As we continue to produce leading results for the industry’s biggest challenges like Visual Wake Words and keyword spotting, we anticipate a major surge in Edge AI applications powered by Deeplite’s software.”

The combination of industry leading optimization software by Deeplite with Andes' state of the art RISC-V CPU cores for tinyML can finally unlock Edge AI use cases like voice recognition or person detection to meet microcontroller-level memory and compute requirements. Device OEMs and application developers may now offer users the benefit of keeping their data on-device while still providing the real-time and seamless responses necessary for AI in everyday life.

To access the technical whitepaper and optimization results, contact Anastasia Hamel, Deeplite Marketing Manager at anastasia@deeplite.ai.

¹https://review42.com/internet-of-things-stats/

About Andes Technology

Fifteen years after starting from scratch, Andes Technology Corporation is now a world class creator of innovative high-performance/low-power 32/64-bit processor cores and associated development environment that serves the rapidly growing global market for embedded system applications. As the founding Premier member of RISC-V International, Andes is the first mainstream CPU vendor that has adopted the RISC-V as the base of its fifth-generation architecture, the AndeStar™ V5. To meet the demanding requirements of today's electronic devices, Andes delivers highly configurable and performance-efficient CPU cores with full-featured integrated development environment and comprehensive software/hardware solutions to help customers innovate their SoC in a shorter time to market. Since 2018, the yearly volume of SoCs Embedded with Andes CPUs has surpassed the 1-billion mark. Andes Technology's comprehensive RISC-V CPU families range from the entry-level N22 (32-bit only), mid-range 25-series, advanced 27-series to high-performance superscalar 45-series.

For more information, please visit https://www.andestech.com

About Deeplite
Based in Montreal, Canada, Deeplite is an AI software company dedicated to enabling AI for everyday life. Deeplite uses a proprietary AI software platform to automatically make other AI models faster, smaller and more energy efficient creating highly compact, high-performance deep neural networks for deployment on edge devices such as cameras, sensors, drones, phones and vehicles. Since its inception in 2018, the core technology from Deeplite has enabled some of the world’s most innovative companies to drive more efficient cloud AI inference and unlock new use cases previously not possible for deep learning on embedded and low-power devices.  Deeplite was named to the 2020 CB Insights AI100 list of top 100 privately-held AI companies and has been featured by Gartner, Forbes, and ARM AI as a premier Edge AI innovator.

For more information, please visit http://www.deeplite.ai