Survey finds embedded efforts lagging, lacking

Survey finds embedded efforts lagging, lacking
By Jeff Child, EE Times
April 6, 2001 (4:58 p.m. EST)
URL: http://www.eetimes.com/story/OEG20010404S0064

SAN FRANCISCO — Embedded-systems projects are falling months behind schedule and failing to deliver as promised on performance and features, a survey of developers finds.

EE Times' corporate parent, CMP Media LLC, conducted the survey, which will be discussed at next week's Embedded Systems Conference. Among other things, 1,100 embedded developers reported that more than half their projects were running late, with a four-month lag the norm, and that most had failed to achieve even half their expected performance. Nevertheless, the survey found few shelved projects — and even fewer answers to the problems.

"Some of that [project lateness] is due to poor scheduling or management giving unrealistic deadlines," said Jerry Krasner, executive director of CMP's Electronics Market Forecasters, which conducted the survey. "Still, it looks like it's something endemic. And the results are pretty consistent with surveys we've done before."

A veteran industry analyst, Krasner said the problems uncovered in the survey suggest a need for more-sophisticated tools and a move to simulation modeling. His researchers questioned 3,500 embedded developers who read EE Times and another 3,500 readers of sister publication Embedded Systems Programming.

Indicating widespread slippage in schedules, one survey question put the designs not completed on time at more than 50 percent. The percentages of behind-schedule projects were mostly consistent across embedded processor architectures, types and vertical markets. The rate was 53 percent for both industrial controls and telecom projects and 54 percent for automotive designs. The medical category reported a significantly higher rate of 66 percent, most likely attributed to unique regulatory hurdles.

Among embedded systems, the 8-bit, 16-bit and 32-bit architectures ran into comparable delays of about 55 percent. The number for 64-bit architectures, meanwhile, was slightly better at a bout 45 percent.

Delays generally spread across all microprocessor lines. MIPS Technologies Inc. and Sun Microsystems Inc. were notable, and opposite, exceptions.

Where other embedded designs typically fell three to four months off the pace, MIPS-based embedded systems run an average of 7.4 months behind schedule, according to survey respondents. That result isn't necessarily a knock against the MIPS architecture. MIPS-based embedded systems, the survey said, average 21.2 development engineers per project while other architectures have, on average, five to 10 fewer. That could indicate that project delays lengthen with very large teams. Naturally, it's tougher to manage a larger engineering team than a smaller one.

Another interesting result on the microprocessor family breakout is Sun processors' low rate of designs behind schedule.

Only 37 percent of Sun microprocessor-based embedded projects ran late. By contrast, STMicro-based designs were reported as most often behind schedule at 69.7 p ercent. That's not unexpected, since STMicro microprocessors had the higher number of design starts per developer per year at 7.3. With lowest average number of developers per project, 6, STMicro clearly has a lot of work, perhaps explaining the longer delays.

Embedded-project cancellations averaged about 11 to 12 percent. Interestingly, the average time to cancellation is run at 4.5 months. No significant difference emerged between the number of cancellations in general compared with the number of cancellations caused by late completion. Clearly the time crunch faced by embedded developers is at the heart of their woes.

Krasner pointed to two problems in slipped schedules: basic delays in design and a failure to meet expectations.

The survey asked: "How do you compare your final designs to predesign expectations?"

"Expectations" were broken out into four aspects including performance, systems functionality and features/scheduling. The results revealed that most embedded developers end up wi th final designs that aren't within 50 percent of performance expectation. That result was consistent with the system functionality, features/scheduling category. Here again, the survey took that across the breakouts by microprocessor family, by architecture type and by selected vertical markets. Results were generally consistent across all three of those data sets, according to the survey. One deviation was among ARM-based project developers, which reported nearly 35 percent of users who didn't get within 50 percent of performance expectations.

In a revealing follow-on to the "expectations" question, the survey asked what embedded developers do when they end up with a design that's in its final stages but that's not within specifications. The survey asked respondents to choose from among 13 answers, picking no more than three.

Schedule slips

The results show that the most popular alternative is to let the schedule slip. As a second alternative, developers would "rewrite minor software modules." Interestingly, dropping features came in third and reconfiguring minor hardware problems came fourth. Far down on the list of options, canceling the project was favored by only 4.6 percent of the respondents. Essentially, development teams and their managers said, "We have this sunk cost already spent by working on this, so let's produce something that lets us recoup some of that cost."

No clear-cut answer emerges to the root cause of all these slipped schedules and missed expectations.

Krasner said industry experts offer various reactions. "That's just the way it is," some shrug, suggesting that it's simply the nature of this complex market, he said. And some are even surprised it's not worse than the survey results show. Others claim mismanagement is at the heart of the problem.

For his part, Krasner believes the survey results indicate a vital need for embedded-system developers to invest in and use more-sophisticated software development tools. "I believe this heralds simulation m odeling," Krasner said. "The design process includes many stages, including specifications, coding and integration. But you usually don't test until the end of the process. Each stage introduces bugs. But if you do iterative design and were able to run tests via simulation and find your bugs you'd minimize this. Developers need to embrace newer, better tools."

The old argument among developers is that such tools are too expensive. They're discovering what the costs are in terms of missing windows of opportunity and increased engineering costs caused by delays, Krasner said. "I think we're sitting on the edge of revolution in embedded-software tool demand and I think it heralds specifically a need for modeling and simulation." For more information about this survey and other market reports visit the Electronics Market Forecasters site.