Development work never happens alone. We all work as part of a team, usually with other hardware and software engineers. Our wide-ranging survey of developers from across North America and Europe revealed some generally encouraging information about how developers are working. (We'll present our Asian data later in the year.)
For starters, the mean size of our respondents' project teams was 13.1 people. That number went up to 34 people for developers working in the aerospace industry and went down to just 6.6 for those in the instrumentation business. Security, networking, automotive, and even military developers all ranked their team size right around the mean.
Given that most teams include only a dozen people or so, it's logical that they'd be all in one place. The chart in Figure 1 shows that this is the case, with 57% of respondents saying they and their colleagues all work in the same building. Another 8% are in different buildings spread over a company campus within the same city. About one third of development teams are either spread across different cities within the same country or in different countries. Not surprisingly, it's the larger defense, aerospace, telecommunications, and industrial-control companies that tend to spread out their development teams. Audio, instrumentation, and security firms tended to band together a little more closely.
Figure 1: Location of team members
Figure 2: Team size compared with previous project team size
Figure 3: Team funding compared with previous project team funding
Things are looking up
Overall, project-development teams are getting a bit bigger and better-funded. As Figure 2 shows, about half (52%) of developers we surveyed said their project team now is about the same size it was on their previous project. Of the remaining half, most said their team has grown since the last project, while just 23% of the total said their team has gotten smaller. That's only one team in four or five that has shrunk; the rest are either the same or bigger. Good news for developers.
The same is true of project funding, shown in Figure 3. A little more than half of our respondents (55%) said their current project received about the same level of funding as the previous project. A slight majority of the remainder said they received more funding, and 21% said funding had gone down.
Although it's not shown in the charts, a closer look at the individual responses reveals that the one-fifth minority who had their funding cut are generally the same ones who also saw their teams shrink. No surprises there. About 15% of those with less funding actually saw their teams grow—they've got more people but less money. The reverse case (more money but fewer people) also occurred about 15% of the time. Overall, better funding correlated with larger teams and vice versa.
Where do resources go?
It's become a truism that most development teams are heavily software-biased. Even chip companies usually employ more programmers than hardware engineers. We set out to confirm that suspicion. Sure enough, when asked about the mix of skills in their development team, most respondents (53%) said they had more programmers on the team than engineers. A quarter had equally balanced teams and a minority (20%) reported a preponderance of hardware types within their ranks.
The software bias was true across all industries, with no correlation between the hardware/software mix and business or product segment. Whether developers worked in consumer electronics, industrial control, telecommunications, or other industries they all slanted their teams toward software development. Figure 4 shows the breakdown.
Figure 4: Hardware/software balance among development teams
Figure 5: Stages of product development
As to where those people actually spent their time, the balance was even more lopsided. Two-thirds said they spend more time developing software than they do creating hardware. Fewer than one-fifth (19%) said the balance was even and a scant 14% said their calendar was determined by the hardware schedule. If software-dominated schedules outnumber software-dominated teams perhaps those developers aren't employing enough programmers.
Interestingly, when we select for software-dominated teams the percentage of those who say they spend more time on software development shoots up to nearly 90%. Yet selecting for hardware-dominated teams yields only a 50% hardware response to the question of where the time goes. It seems even hardware-heavy projects are dependant upon their software.
Finally, on the issue of spending we see a now-familiar tendency. A slight plurality (about 44%) said they spend more money on software development, probably on salaries for their programmers. About 38% spend more on hardware development, with the remainder (22%) claiming an even breakdown. Spending is disproportionately weighted toward hardware because of the high cost of hardware developers' components and tools, compared to relatively inexpensive software tools. Hardware development is capital intensive; software development is labor intensive. Although the spending ratios aren't quite as lopsided as they are for time and manpower, the trend is clear: software development dominates every aspect of product development.
Where does the time go?
DSP columnist Jeff Bier has compared debugging to alien abduction: big chunks of time disappear and you can't account for them. We asked our developers where all their time goes and the response was grimly predictable.
Starting at the top and working clockwise, Figure 5 illustrates the progressive stages of a project's development cycle. From initial concept to final production, a developer's time is spent many places. Sadly, only about one-fifth of that time is actually spent in detailed design of hardware or software. That response—which nominally represents the designer's job, by definition—was allotted only 24% of their overall time budget. That bleak allocation didn't change at all between hardware, software, or system-level designers, nor did it change by industry.
What's more distressing is that most of time is spent on debugging. In fact, our respondents said they spend more time debugging their code or hardware than they did creating it. This suggests either a dire lack of talent on their part or widespread deficiency in debugging tools. When professional engineers spend more time fixing their own mistakes than they do making them, there's something fundamentally wrong with the process. Engineering is not supposed to be iterative, at least not within a single product cycle. It's no wonder that managers and marketing departments pull their collective hair out over schedule slips.
Here's a statistic sure to ignite arguments: programmers spend more time debugging than hardware engineers do. Those who identified themselves as software developers said they spend more than 27% of their time (on average) in debug and test, as opposed to an average of just under 21% for hardware developers. Either the programmers are less disciplined than the hardware types or they're more honest. Or more likely, it's because software governs the overall project timetable as we've seen above, so it naturally also dominates the debug schedule.
Not everyone spends time on every stage of our hypothetical development cycle. Simulation was the most frequently skipped task, with a significant portion of survey-takers saying they performed no simulation at all. Prototyping and “sending to production” also were often skipped over.
In future installments we'll look at schedules and how they inevitably slip, plus some detailed information on what chips and advanced tools developers are buying or plan to buy
Jim Turley is editor in chief of Embedded Systems Programming magazine and is the director of the Embedded Systems Conference. You can reach him at .
Thank you for talking about this industry-wide debugging problem.
Regarding “When professional engineers spend more time fixing their own mistakes than they do making them, there's something fundamentally wrong with the process.” I have to agree that our embedded systems industry has not aggressively gone after this problem.
To make it more exciting, product lives are now shorter than the time it takes to develop the product. Average development times have not changed in the last 4 years.
Also, there is a tool that does greatly reduces bugs and reduces development time. Please check out a product called SynthOS at www.zeidman.biz/synthos.htm
– Dave Sell
I have to take exception to the interpretation of the data on time allocation in a project. That developers spend more time debugging than designing does not necessarily mean the developers lack talent or the tools are deficient. Most likely it means the development process is broken, as suggested in the next sentence.
It's well known that shortcutting in the design phase usually means paying the price, with interest, in the testing/debugging phase. Yet many developers are eager to jump in and start cranking code or spinning boards as soon as possible. Unfortunately management usually encourages this because it gives the appearance of rapid progress. Lines of code is an easy metric to include in status reports; progress on design is more difficult to quantify. During design it may appear to management that little progress is being made.
Secondly, design (and thereby engineering) is ABSOLUTELY an iterative process. This point was emphasized throughout my engineering education. What would be the point of testing if we didn't go back and redesign when problems were uncovered?
The ESP survey has collected much interesting data. I look forward to future installments.
– Jason Dougherty