GenAI v1-Q launched with 4 bits Quantization support to accelerate larger LLMs at the Edge
The new version brings a 276% speed increase for the top LLMs in low-cost systems, while maintaining their intelligence.
Spain, September 24, 2024 -- Raiderchip has presented a new HW accelerator product adding 4-bits and 5-bits Quantization support (Q4_K and Q5_K) to the extraordinary efficiency of the base GenAI v1. The new Generative AI LLM hardware accelerator runs inside FPGA devices, and is ideal for boosting their capabilities with low-cost DDR and LPDDR memories, incrementing inference speed by 276%.
GenAI v1-Q running the Llama 2-7B LLM model with 4 bits Quantization on a low-cost Versal FPGA with LPDDR4 memory
The new acceleration engine increases not only inference speed but also lowers memory requirements by up to 75%, allowing the largest and most intelligent LLM models to fit into smaller systems, lowering the overall cost, while keeping real-time speed, and also reducing energy consumption. All of this with minimal impact on model accuracy and intelligence perception.
The GenAI v1-Q, which like its predecessor is already available for a wide range of FPGAs, aims to expand the range of available features. In the words of its CTO, Victor Lopez, ‘We seek to offer maximum flexibility to our customers, with highly configurable hardware that allows them to balance criteria such as accuracy, inference speed, model size, unit cost of hardware, or energy consumption goals according to their needs, finding the perfect balance that best fits their objectives.’
The current demonstrator accelerates the 4 bits quantized Meta’s Llama 2-7B using barely 4 GB of memory, whereas the vanilla version requires 16 GB of DDR.
Companies interested in trying the GenAI v1-Q may reach out to Raiderchip for access to our demo or a consultation on how our IP cores can accelerate their AI workloads.
More information at https://raiderchip.ai/technology/hardware-ai-accelerators
|
Related News
- RaiderChip unveils its fully Hardware-Based Generative AI Accelerator: The GenAI NPU
- RaiderChip raises 1 Million Euros in seed capital to market its innovative generative AI accelerator: the GenAI v1.
- Silicon Proven AV1 Decoder IP with support for 12-bit pixel size and 4:4:4 Chroma Sub-Sampling Released by Allegro DVT
- RaiderChip NPU for LLM at the Edge supports DeepSeek-R1 reasoning models
- Ceva Expands Embedded AI NPU Ecosystem with New Partnerships That Accelerate Time-to-Market for Smart Edge Devices
Breaking News
- Arteris Joins Intel Foundry Accelerator Ecosystem Alliance Program to Support Advanced Semiconductor Designs
- SkyeChip Joins Intel Foundry Accelerator IP Alliance
- Siemens and Intel Foundry advance their collaboration to enable cutting-edge integrated circuits and advanced packaging solutions for 2D and 3D IC
- Cadence Expands Design IP Portfolio Optimized for Intel 18A and Intel 18A-P Technologies, Advancing AI, HPC and Mobility Applications
- Synopsys and Intel Foundry Propel Angstrom-Scale Chip Designs on Intel 18A and Intel 18A-P Technologies
Most Popular
- QuickLogic Delivers eFPGA Hard IP for Intel 18A Based Test Chip
- Siemens collaborates with TSMC to drive further innovation in semiconductor design and integration
- Aion Silicon Joins Intel Foundry Accelerator Design Services Alliance to Deliver Next-Generation Custom SoCs at Scale
- TSMC Unveils Next-Generation A14 Process at North America Technology Symposium
- BOS Semiconductors to Partner with Intel to Accelerate Automotive AI Innovation
![]() |
E-mail This Article | ![]() |
![]() |
Printer-Friendly Page |