RaiderChip brings Meta Llama 3.2 LLM HW acceleration to low cost FPGAs
The company incorporates the latest model, presented by Meta less than a week ago, into the catalog of LLMs already accelerated on a wide range of FPGAs
Spain -- October 1st, 2024 -- Just 6 days after its launch by Meta, RaiderChip has added support for the new Llama 3.2 model to the list of LLMs hardware accelerated by its GenAI v1 IP core for FPGAs. The supported models, which already included previous models from the same company (such as Llama 2, Llama 3, and Llama 3.1), as well as models from other providers (Microsoft’s Phi-2 and Phi-3), have been expanded again, following RaiderChip’s strategy of adding the most relevant models as they hit the market.
RaiderChip’s Generative AI hardware acceleration IP core is designed to accelerate any model built on Transformers technology, which underpins the vast majority of LLMs. The selection of additional supported models on FPGA is determined by customer choices of specific target devices and foundational LLMs. Supporting a foundational model enables any customer-specific fine-tuned derivative to be accelerated seamlessly, without the need to share its weights.
It is important to note that the various FPGA options available on the market have different sizes and variable logic and memory capacities. These technical factors, combined with commercial decisions such as unit cost, power consumption, or final functionality, mean that the ideal FPGA and LLM vary for each potential final product. For instance, using “smaller” models (such as Meta’s Llama 3.2 1B and Microsoft’s Phi-2 2.7B) or 4-bits Quantization are ideal for products based on simpler, more economical FPGAs; while larger LLMs in their original format, preserving the model’s full floating point precision, require larger and more expensive FPGAs,” explains Victor Lopez, the company’s CTO.
GenAI v1-Q running the Llama 3.2 1B LLM model with 4 bits Quantization on a low-cost Versal FPGA with LPDDR4 memory
Like previous models, the new Llama 3.2 can be tested through an interactive demo based on a Versal FPGA. “The demonstrator exposes not only a simple local and remote API access to the accelerated LLM, but may also be run directly through a chat terminal application that allows users to interact with the different models, while they are HW accelerated by RaiderChip’s IP core on the FPGA. At RaiderChip, we invite our customers to personally experience the real performance of our product, assessing key aspects like intelligence, latency, or tokens per second in a live demonstrator, beyond the usual performance tables based on theoretical or simulation data, which are often confusing,” the team comments.
Companies interested in trying the GenAI v1-Q IP core may reach out to RaiderChip for access to our FPGA demo or a consultation on how our IP cores can accelerate their AI workloads.
More information at https://raiderchip.ai/technology/hardware-ai-accelerators
|
Related News
- RaiderChip launches its Generative AI hardware accelerator for LLM models on low-cost FPGAs
- Lattice Ships First Samples of Low Cost, Low Power LatticeECP4 FPGAs
- Paul Williamson on Edge AI, Llama 3.2 on Arm
- Efinix Releases Topaz Line of FPGAs, Delivering High Performance and Low Power to Mass Market Applications
- Radiation-Tolerant PolarFire® SoC FPGAs Offer Low Power, Zero Configuration Upsets, RISC-V Architecture for Space Applications
Breaking News
- Arteris Wins Two Gold and One Silver Stevie® Awards in the 2025 American Business Awards®
- Faraday Adds QuickLogic eFPGA to FlashKit‑22RRAM SoC for IoT Edge
- Xylon Introduces Xylon ISP Studio
- Crypto Quantique announces QRoot Lite - a lightweight and configurable root-of-trust IP for resource-constrained IoT devices
- BOS Semiconductors to Partner with Intel to Accelerate Automotive AI Innovation
Most Popular
- Andes Technology and Imagination Technologies Showcase Android 15 on High-Performance RISC-V Based Platform
- TSMC Unveils Next-Generation A14 Process at North America Technology Symposium
- Synopsys and TSMC Usher In Angstrom-Scale Designs with Certified EDA Flows on Advanced TSMC A16 and N2P Processes
- Certus Semiconductor Joins TSMC IP Alliance Program to Enhance Custom I/O and ESD Solutions
- M31 Collaborates with TSMC to Advance 2nm eUSB2 IP Innovation
![]() |
E-mail This Article | ![]() |
![]() |
Printer-Friendly Page |