 |
|
How to perform meaningful benchmarks on FPGAs from different vendors
Programmable Logic DesignLine (05/28/08, 01:39:00 PM EDT) As a design engineer using FPGAs, how do you choose which FPGA is the best for your application? Every engineer faces this problem and is inundated with endless, self-promoting marketing claims that many times are taken with a "pinch of salt". Every FPGA vendor asserts that their part is this much faster, takes this much less time to compile, and consumes this much less power as well as claiming numerous other technical specifications. After all, any company can create artificially-tailored benchmarks that can put their FPGA in a better light when compared to that of the competition. So the question is: which benchmarking claims are accurate and which are marketing claims? Altera evaluates and benchmarks its FPGAs in conjunction with its Quartus II software at every process node on a suite of comprehensive and representative customer designs. These customer designs are collected from a variety of market segments, such as networking, telecommunications, wireless, and consumer applications, and a variety of implementation technologies such as ASICs, gate arrays, and FPGAs from other vendors. These customer designs (which are maintained in a secure database) are used to understand how real customer designs perform against the current architecture and with that of the competition. Designs that do not optimize well on Altera's FPGA architecture are evaluated on over 150,000 experiments; enhancements to future incarnations of the FPGA architecture or design tools are then recommended based on the evaluations performed. Based on the evaluation and benchmarking processes introduced above – and the fact that there is no standardized FPGA benchmarking processes – Altera has created what we believe to be a "fair and unbiased" benchmarking methodology, which is based on a set of real customer designs and endorsed by industry experts. But there's a problem: since the existing designs are proprietary to our customers, we cannot provide these designs for evaluation of Altera FPGAs by new/other customers. Although everyone understands the reasoning behind this, it still serves to amplify end-user skepticism. If I was a potential customer of Altera and was told that Stratix III FPGAs are on average 35% faster than Virtex-5 FPGAs, compile three times faster than Virtex-5 FPGAs, and provide 95% core utilization on average, then without the means of evaluating this myself, as an engineer, I would be very skeptical!
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
