High Performance Scalable Sensor Hub DSP Architecture for Vision AI

SensPro2™ is the highly-scalable and enhanced 2nd generation high performance sensor hub DSP for multitasking sensing and AI of multiple sensors including camera, Radar, LiDar, Time-of-Flight, microphones and inertial measurement units.
Vision AI SensPro2 family is designed to handle multiple sensor processing workload for contextually aware devices and can be utilized in modern intelligent systems in automotive, robotics, surveillance, AR/VR, voice assistants, wearables, mobile and smart home devices.

 

SensePro applications Diagram

 

SensPro2 is a highly-configurable and self-contained architecture that brings together scalar and vector processing for floating point and integer data types, as well as deep learning inferencing.

The 2nd generation of vision AI SensPro DSP family includes 6 self-contained cores, addressing a wider range of computer vision, vision AI and Edge AI applications:

  • The SP100 and SP250 DSPs, with 128 and 256 INT8 MACS, respectively. These DSPs offer the smallest die size and are ideal for a range of imaging and vision applications combined with AI workloads.
  • The SP500 and SP1000 DSPs with 512 and 1024 INT8 MACs, respectively. These DSPs offer the highest performance and precision in the SensPro2 family, with optimal configurability for computer vision, SLAM, Radar, and AI workloads.
  • The SPF2 and SPF4 floating point DSPs, with 32 and 64 single precision floating point MACs, respectively. These DSPs are optimized for electric vehicle powertrain control and battery management systems, complemented by a full suite of Eigen Linear Algebra, MATLAB vector libraries and support for the Glow graph compiler.

SensPro2 maximizes performance-per-watt for multi-sensor processing use cases by utilizing a combination of high performance single and half precision floating point math, point cloud creation and Deep Neural Network processing, along with parallel processing capacity for voice, imaging, and Simultaneous Localization and Mapping (SLAM).

SensPro2 platform approach is paired with advanced software and development tools that include LLVM C/C++ compiler, Eclipse based integrated development environment, Ceva Deep Neural Network (CDNN) graph compiler for neural networks including the CDNN-Invite API for inclusion of custom AI engines, TensorFlow Lite Micro support, OpenVX API, and a multitude of software libraries for OpenCL, Ceva-CV imaging functions and Ceva-SLAM SDK.
These broad set of software and development tools help to minimize cost and improve time to market.

Benefits

SensPro2 is a high performance sensor hub architecture, allowing it to address the processing and fusing of data from multiple sensors for contextually-aware devices. It is a highly configurable architecture that support the data-processing of various sensors in devices that need high performance and ultra-low power consumption.

Self-contained, specialized sensor hub on-device processor that unifies multi-sensor processing with AI and sensor fusion in a unified solution
Six highly configurable cores for different computation needs of various applications, ranging from 0.2 TOPS up to 3.2 TOPS
Multi-Tasking support to switch between AI and DSP sensor processing tasks

Main Features

  • Self-contained, high-performance sensor hub processor family
  • Unifies multi-sensor processing with AI and sensor fusion
  • 8-way VLIW architecture
  • State-of-the-art micro architecture that combines Scalar and Vector Processing Units
  • Application-specific ISA extensions for radar, audio, computer vision and SLAM
  • ASIL-B Automotive safety compliant with ASIL-D support
  • Advanced set of software libraries and development tools
  • Ceva-BX scalar processor for control code execution
  • Up to 1024 integer (8x8) MACs
  • Up to 64 Single Precision and 128 Half Precision floating-point MACs
  • Support for Binary Neural Networks

Block Diagram

Ceva SensPro2 Sensor Hub DSP video demo - AI & Computer Vision for smart edge devices

Ceva’s SensPro2 Sensor Hub DSP is integrated into Novatek’s NT98530 smart camera SoC and evaluation board. The NT 98530 is utilizing the embedded Ceva SensPro2 DSP for running real-time edge AI processing and advanced computer vision algorithms, demonstrating neural networks for face detection and people detection running in parallel.