When the IBM 360 tools project started running late managers hired more programmers. And then they iterated this process, eventually boosting team size from the projected 150 to over a thousand developers.
They found that each new batch of hiring slowed the project dramatically. Fred Brooks recounts in his seminal must-read “The Mythical Man-Month” that the effort eventually ran years late and dramatically over-budget.
As a result of this debacle he formulated the thesis now known as Brooks Law: adding people to a late project makes it later.
Barry Boehm’s Constructive Cost Model (COCOMO), used for estimating software schedules, mirrors this finding. As projects grow in size individual productivity plummets. There’s an exponential relationship between the schedule and the size of the project as measured in thousands of lines of code. Typically COCOMO predicts that as a program swells to the million line of code mark programmers will each crank just 17 lines of code per month.
The reason is communication, of course. A small project done by one lone developer has zero comm channels… it’s all in Joe’s head. Two folks have a single link. The number of channels is N(N-1)/2, practically an n**2 rate. Pretty soon, with enough people, everyone is so busy attending meetings, responding to email, writing reports and it’s rare they have time to leak out a line or two of code.
Microsoft puts armies of developers on big projects like Windows and Office. Could this be contributing to the embarrassing delays in Vista (neé Longhorn)?
Of course it’s impossible to deliver 50 MLOC with only a dozen people on the team. But it is possible – perhaps mandatory – to partition projects into small bits and pieces, each logically distinct, each of which can be implemented by a handful of programmers.
What’s the situation in the embedded world? One 2001 survey found that the average embedded project used about 7 developers, a surprisingly small number. Yet firmware content doubles every 10 months to two years, depending on who you believe, so this data may not represent the situation today. It’s not uncommon to have a million lines of code anymore, which is a huge effort likely to consume the efforts of hundreds of developers.
Yet teams of that size are fairly rare. A lot of these giant projects are organic, having evolved over the years from smaller apps as features and functionality get added from one release to the next. Sometimes a newly acquired company has a big code base that gets integrated into the buyer’s products. I’ve run into a couple of groups managing humungous chunks of code with only a handful of developers.
Today’s business mantra dictates doing more with less. But that doesn’t square with skyrocketing code sizes.
How big are your projects… and how many developers do they use? Are there enough… or too many?
Jack G. Ganssle is a lecturer and consultant on embedded development issues. He conducts seminars on embedded systems and helps companies with their embedded challenges. Contact him at . His website is .
I agrees with most of the points made in this article. However, with proper use of code management tools, even small teams can manage a huge software base. This really depend how much the code changes and how well its packaged. I would wager a dedicated team of 10 crack developer can easily manage five times more code than a team 3 times its size.
Jack wrote, “The number of channels is N(N-1)/N…”
To stay on topic, we wouldn't need so many developers to fix problems if we didn't create so many problems in the first place!
– Ed Schulz
Jack Replies Whoops! You're right. I've made the correction. Thanks!
“Crack developers”, eh? We know that special high of belonging to (or leading) a top-notch team of intelligent and hard-working software geeks. It's great! But, it is impractical to rely upon this scenario since most of us work with a mixed group of experienced, competent, diligent, ineffectual, and/or naive people. Its not (code) size, but rather scalable consistency. I think that the top issues (in order) for managing any code base is:
A) Good software architecture – well-organized files, functions, protocols, and command flow that is documented and maintained through all the new features that grow the code.
B) Good software process – a rigid set of rules for fixing bugs and adding subsystems that maintains the architecture. This typically requires code review/analysis and significant oversight from a leader.
C) Good software people – can't find 'em? Make your own with young folks who get trained, mentored, and encouraged!
Its too simple to be true, and too difficult to make happen. I don't see these foundational principles followed very often.
Looking for Henry Ford! Or another genius like him who can break up a problem into a production. How do the Hollywood moguls do it, with the starry tantrums and all that jazz?
– kalpak dabir
The most successful projects I have been on apply the divide and conquer rule. Break the problem down until a very small team can handle it. The dropping price of powerful 8 and 16-bit microcontrollers with great high-level language support makes this even more attractive! Just a nitpick: I think the number of lines of communication is actually N(N-1)/2, where N is the number of people.
– Scott Whitney
A lot depends on the skill level of the developers involved and the degree to which the code is intelligently partitioned. A million lines of modular, well-documented code can be maintained by a smaller team than can a million lines of spaghetti. Also, I think object-oriented techniques allow smaller teams to manage (and generate) larger amounts of code. With huge projects; up-front design, modularity, and good partitioning are the key. The point in the article that really hits home with me is the fact about average team size. In my own experience, I would have pegged the magic number at about 6 or 7. Above that, and your productivity really starts to bog down.
I agree with you, we have a ballpark number of about 1 developer for every 10k lines of C++ code for a maintanence project. The developer is only responsibile for design and coding, requirements are given to him from a different dept.
I should agree that adding a person to a late project makes it later. I now work with a team of 18 FW engineers for a complete and complex SoC on which there are around 35HW engineers in play.
1. It is really difficult to estimate the number of engineers required being with a new technology, new people, and mix of young to veteran engineers…
2. I agree with Brent on the 3 points he has made against using 'Crack developers'. But one thing – the amount of time going into mentoring young engineers when you don't get one – this is a very risky and short term investment for many companies in U.S.
3. I strongly believe in PROCESS ORIENTED ENGINEERING. A good process to manage and maintain the code should be enough for the team to work through any amount of challenges.
4. A mature organization does not depend on crack developers or a few engineers but a ENGINEERING TEAM and a TEAM comes into picture only when they adhere to a code of conduct – implicit or explicit (not so formal as I have worded it over here).
Finally, collaboration & collaboration tools is the key to efficiency in the team is what I believe – big or small (even I would consider a good CM tool develops collaborative atmosplere in a team). Any say on this?
– Saravanan T S