Separating job functions
In responding to my aticle "Design by Clairvoyance" article, a reader named Andy wrote:
Sometime between, say, the construction of the pyramids in Egypt, and the 19th century, the building construction industry bifurcated into two expertise domains: design and construction. Architects designed, and contractors built. And in order for contractors to know what the architects had in mind, several powerful information exchange mechanisms evolved. Architects built models (e.g.: Michelangelo's 1547 model in wood of St. Peter's, presented to the Pope), and produced working drawings. The builders then used the working drawings to accomplish the construction. No competent builder would accept incomplete or incorrect working drawings. To do so would risk bankruptcy.
We aren't quite so disciplined in the construction of software. Though the scope of work in a large project may take years to complete, the work product is mostly invisible, unlike a skyscraper. When software construction is underway, our clients see people sitting at workstations. There are no big holes in the ground, no noisy, heavy equipment, no piles of materials. Lines of code, even if there are millions of them, aren't tangible to our clients. Hence, many projects are cancelled, or are finished "late" (as defined by the original, deeply-flawed estimate), and "over-budget". The software industry needs to develop a discipline of design and construction. Until that happens, software development will largely remain, in the words of a former manager who was a hardware guy, "a thankless job."
Specialization helped fuel the Industrial Revolution. The old model of a village blacksmith doing everything from smelting ore to shaping the product was out; now vast factories employed workers courting a multitude of injuries incurred from endless repetitive motions. The educated class specialized as well; who can count the kinds of engineers that now exist? Or the types of doctors?
Tom DeMarco wrote of separating the coding and debugging functions in his 1982 book "Control Software Projects." Yet two decades have passed and, in general, most software people are the blacksmiths of yore, designing, coding, debugging, testing and documenting their work. Not to mention the special joy of embedded systems: wringing out hardware bugs. Instead of specializing, we embedded people try to be masters of everything, from assembly to C++ and analog to digital.
In the building industry, the architect and the contractor have separate functions, and the division of labor yields some pretty stunning benefits. They communicate by means of drawings and perhaps models of the proposed project. This documentation is a formal contract that tells the contractor what to build. The drawings arbitrate all disputes. Compare that to the standard "but that's not what I wanted" hand-wringing we hear from the 23-year-old marketing droids when they first see our completed embedded system.
Though the customer probably knows little about designing buildings, the drawings and model clearly explain what he's buying. Clarity prevents contention.
It's easy to dismiss an architectural drawing as something much simpler than a big wad of code. Of course the customer can understand the project by referring to the blueprint anyone can. But the drawings do hide a lot of critical detail. Lots of analysis not shown on the page ensures that the building won't fall down, that plumbing and electrical will be adequate for its intended use.
Yet in software everything is in the code. We can't even agree on a standard for design docs. Gurus tout UML as the future, but very few embedded folks use that technology. Can your customers read a UML diagram? We desperately need a way to produce a complete design or model that's comprehensible to all important stakeholders.
Just as architects build scale models of their creations, maybe we can produce some sort of model for our inscrutable designs; something a customer can examine and critique. It's certainly possible to prototype GUIs and PC applications, but embedded systems are tougher. They may have to communicate with other machines, take data, and interact with the environment. The model takes a lot more effort than playing with cardboard and glue sticks.
And worse, a software model never looks like a model. No one has the sense that it's 1/10 the size of the real project. "It works great! Let's ship it!"
An architect's model is very clearly a mock-up.
Perhaps the most interesting part about the building trades is that everyone expects to pay the architect big bucks for a design. One that will take months, maybe years, to produce. The contractor doesn't start pushing dirt around till the drawings are done and the customer has checked every detail of the proposed structure.
In the embedded field the first question we get when the boss wants to start a new project is "How long will it take?" We have a day, maybe a week to hash out enough design to produce the usual wildly optimistic schedule and costs.
Architects don't give a price. The contractor does, once handed a complete design. For only then is it possible to figure the number of sinks, tile, wiring and labor that's needed.
Why is software different?
Jack G. Ganssle is a lecturer and consultant on embedded development issues. He conducts seminars on embedded systems and helps companies with their embedded challenges. Contact him at firstname.lastname@example.org. His website is www.ganssle.com.
Software does not need to be different if (considering UML as modelling tool):
1. All s/w job-holders, including the Marketing droid understands UML 2. Customer understands UML
Point 1 is a matter of discipline in using UML (which is a rarity today!). Point 2 comes the difficult part with Non-technical customers considering the timeline. "BOTTOMLINE: ALL SOFTWARE JOB HOLDERS and STAKEHOLDERS (including embedded folks)NEED TO GET MORE DISCIPLINED IN USING STANDARD MODELLING TOOLS LIKE UML"
The practice of building houses has been around for about five thousand years, yet there exists striking resemblence between what was built then and what is being built now. Is software different?
- David J. Liu
Before everyone starts glamorizing the construction industry, perhaps they should read the following link:
It seems that software design does not have a monopoly on poor planning. It just has more tools to do the wrong thing faster and with little visibility.
- Kyle Heironimus
Q: Why is software different?
A: In the construction industry, you are working with "absolutes", i.e. physical objects in 3D. These are "things" we are familiar with, can pick up, move, and articulate. In the software industry, it's different. We are working with "abstracts", i.e. encoding our thoughts and processes. The best that we have done so far is OOP and modularization. The problem with software is that its data structures can get so complex, it's hard to create a flow diagram entity that all can relate to. Create the symbol for the following that all can understand: An array of hashes of hashes containing a linked list of function pointers. Absract concepts are the hardest to visualize and that is the main reason for our convoluted industry. Look at our tool set. There are many different languages to express our abstractness. Some tools express abstract concepts better than others. Once we find the single tool that can express all our abstractness, maybe then we can standardize our process.
- Steve King
I've been wondering this for over 20 years. Physicists use a different set of notations than Engineers because they are concerned with different things. We don't want to do the science all over again. We want to use that science. Engineers are educated to use the product of physics, math, ...
Computing scientists use lots of specification notations. Model based notations like Z, B and Focus as well as process algebras (CCS, CSP, Pi) are all popular in European CS departments. They teach them to their Undergrads. I use term rewriting systems in my own research. I build abstract models of protocols and can execute them as well as use a model checker. David Harrel is working on some cool stuff that shows promise. Yet all of these are relly more like the diagrams you draw in Physics class. Too detailed and low level for designing a bridge. Great for investigating the kind of stuff we computing scientists are interested in. We need domain specific specification languages. Hopefully ones that can be executed. We need lots of these not bigger ang bigger general purpose notations.
- Alwyn Goodloe
The primary difference is that there is no construction stage in software. It's design all the way from high-level architecture to detailed code. Executable code is simply the lowest level of design. That's why there can be no "hand-over" point.
- Nick Hewish
Jack asked: "Why is software different?"
I believe that the fundamental difference is that you can't see it like you can a building, so outsiders' perceptions are very different. There is also the fact that you can launch into coding in a way that you cannot when constructing a building. Well, nothing larger than, say, a garden shed. Equally the consequences of errors during the construction are minimal for software (timescales and costs notwithstanding). A program that collapses around your ears is minor headache compared to a building doing the same. So I guess that what I am saying is that the perceived need for discipline right from the early stages isn't there for software.
- Andy McC
I think a problem is the way software engineering is taught in school to students. They are taught smaller easy to change programs that teach the "Burn and Learn" method of iterative development -- Much the same way kids at summer camp are taught craft projects like making decorative light switch wall covers. Need 10,000 for a huge project -- not a clue how to do it, it's not taught. Need a disciplined process for something safety critical -- that's likely not covered either. There are several specialties in software that really need to be set up as possibly advanced degrees. (SW Architecture, Safety Critical Systems, Embedded Systems, etc.)
- Wally Murray
Most of the projects I've been involved in are very "time to market" driven. That doesn't allow time for doing comprehensive design and modeling before coding. Use cases seem to be a good tool that the customer can understand without the need for extensive training. As the development stage progresses those use cases "should" be elaborated on and more low level use cases added. Unfortunately the norm for market driven projects seems to be to define some minimal and incomplete user requirements spec and start coding. Do some testing along the way and follow up with the old promise, "we'll update the specs when the code is working". Better use cases to capture the user requirements may not be the final solution but it's a very practical tool that carries little overhead for all involved and it does provide a "blueprint" that everyone can look at and understand.
- Ray Burks
Why software is different:
Customers often know what they want from a building, but don't often know what they want from software. So, a software engineer must often also be a requirements engineer (you forgot to add "specifying requirements" in your list of jobs: ...the blacksmiths of yore, designing, coding, debugging, testing and documenting their work). Besides that, there can also be the task of extracting domain expertise from a client.
The bifurcation of architect and contractor in construction was a natural result of 1) the progression of construction engineering knowledge and 2) increasingly larger scales.
1) In construction, the contractors don't need to have the same level of knowledge about the design as the architect. In software, design and coding go nearly hand-in-hand, and there is usually a feedback loop because of incomplete knowledge at design time. Thus, a coder often needs to have nearly the same level of knowledge about the design and design principles as the designer.
2) In the construction of anything more complex than a house, you must have more than one or two people working on the actual construction in order to get finished in a reasonable amount of time. However, one or two people can complete a respectable software project in a reasonable amount of time.
Incomplete a priori knowledge of requirements and implementation obstacles, the design/code feedback loop, the plasticity of completed software ("just change it"), and the fact that all solutions are "custom" to varying degrees make software more difficult to estimate.
The allocation of responsibility for documentation is about economy of time - it's faster to have the developer write the documentation than to have the developer explain the design/code/operation to someone else, and have the other person write the documentation. Also, detail is bound to be lost in the process.
UML's Use Case diagrams may currently be the best candidate for communicating with a naive' customer about what the software *does* - and that's usually all a naive' customer cares about. However, there are many candidates for diagramming how the software *works*, which is the meat of the architect's and developer's work. There is currently no single modeling method that can articulate all the aspects of the software that architects and developers are usually concerned with, such as contained in other types of UML diagrams, schematic block diagrams, control flow, data flow, state flow, timeline flow and any other type of diagram that you can name. I think that what we really need is a way to hook all of these different model types together in a behavioral sense. I'm all for Model Driven Architecture/Design - further, I'm all for executable models!
I like the provocative idea of splitting up tasks, but try selling the idea of hiring more employees to managers who think that software is already too expensive. Good Luck!
- John Hopkins
The first question is "why to construction morph from the backsmith do everything to design/construct? I suspect economics drove this. People want to know how much its going to cost.
Software is similar. There is the predominate "blacksmith" method and the "create specs and cost estimate/then bang out code".
In construction, I believe there is two estimation phases. One at the Architect/Engineering/Design phase and then the contractor that is going to do the job also does an estimate. If the bids come in too high, then the original estimate was faulty. In software there isn't the second estimate. The coders are told what the timeline and budget are.
I agree a design <-> construction methodology could help deal with the budget over run problem.
- Tim Flynn
I can't argue that better development techniques can and should be realized. However the "construction" metaphor seems forced.
Consider if as developers we were to produce "architectural drawings" for the benefit of "code builders". In building construction a collection of (hopefully) skilled tradesmen know how to implement the design using established codes and standards. The laws of physics, loading, stress analysis, piping losses, etc. are all taken into consideration, both by the architect and by the tradesmen. A carpenter knows about using 2x6's instead of 2x4's for window headers and also when to use a 2x8 or 2x12. He knows about "proper construction techniques" that are well established, codified and legally enforced. What's the equivalent in software development?
Where are the "standard" ways of implementing a particular abstract concept in software? When linked lists, sorting algorithms, timing, interrupt routines and keyboard/ display and GUI interfaces become as ubiquituous and readily understood by software "tradesmen" as doors and windows and 2x4's are to caprenters then maybe this metaphor will apply.
I think software libraries are an attempt in this direction, a way to collect commonly used functions in a reusable, "codified" form. Unfortunately (or perhaps necessarily) these libraries exist in abundance, no two alike, and are generally crafted and reused by small groups. Mass public acceptance of library code (the metaphorical equivalent of standard building construction practices) occurs at the compiler level with the so called run time libraries. Extend these libraries to include entire algorithms (objects anyone?), standardize them and then you'd really have something better.
- Stephen Ciricillo
One major difference between building construction and software construction is how deadlines are determined. Houses can be made quickly, mostly due to the fact that the design is done years before and just "cookie-cuttered" out to make new houses, something akin to burning a release CD based on software already written. When a new building has to be designed, the head of the project doesn't say to the architect and contractor "We need this building done by August 1st." Instead, they know that a proper design takes time, and if it's rushed you end up with a shoddy building that could collapse and result in deaths.
Software deadlines are often created before what the software needs to do is understood.
It's all about understanding that good design takes time.
I talked with an architect about building a health club. They get 10% of the total cost for their work, and he did give me a price. The plans and pricing didn't include the parking lot. Several of my friends had their own homes built, and they worked with high school drop outs who only knew which end of a hammer to hold. Often, the owrk had to be redone. Once, my friend had to go bail the orkers out of jail before getting started for the day. There are too many of these types of people in software. They think they know something because they know C++ syntax. Of those who really do know something, I think they do specialize. Data base design is different from operating system design. Real time is a specialty field. Programming graphics cards and game design are specialties. Windows is different from Linux. Networking, or programming routers is another. O.K. Maybe we do dabble in all and specialize in none. Would someone pay me to do just one thing?
- Dennis Ludwig
In the construction industry, architects and builders understand each other very well and this is not just because of drawings, but also because the domain, the designs, the technologies and the tools change very slowly. The communication languages and channels are all well established.
In software, the design on a VxWorks platform would be significantly different from that on a WinCE.NET platform. How many times have we struggled to communicate, much less convince, our design to a co-designer?
In construction, most of the standard problems are known and standard solutions are used. In software, teams solve same problems and 'find' their way to standard solutions. If we need a strange component, we love building it from scratch rather than changing the design to use an available standard component.
In our industry it is harder to learn from others. If there were easier ways to learn and understand how others have built their software, our own designs would become much better. We will also learn to communicate our designs to others. This would quickly evolve to a set of design patterns/frameworks that everybody understands and uses.
Until we have the means to communicate our domain, architecture, implementation and technology thoughts clearly to others, we would end up doing all by ourselves!
- Shankar Velayudhan
Why is software different?
This is due to two reasons,
1. You cannot breakdown a software system into universally accepted discrete components.A building can be broken down into tiles, bricks, pipes. A chip could be broken down into AND, OR, and other logic gates. Each discreate component would be used to build complex structures. Each discrete component has a well defined input/output interface and specifiction and this remains same across different implementations of the system. But, this is not the case in software. You could argue and say objects/classes could be the discrete components. But no way would we agree on a standard interface.
2. It is difficult to visualize software. Buildings can be visualized by models, drawings. Electrical designs can be visualized by circuit diagrams. But there is no standard way to visaulize a software system. UML could be used, but again everyone must agree on it. And we could debate on it for years on whether is fits all software applications/domains
- Kedar Chitnis
In all this discussion there appears to be little mention of or concern for the client. In the construction industry Architects provide a translation of what the clients want/need, models are used more to confirm that the intrepretation of the clients requirements are appropriate than as a construction guide(different drawings are used for this). I think that one of the shortcomings of we in the software industry is that we get too involved in what we want to do with the technology on a personal level and forget why the code is being written. Good Architects fill their clients needs in all sorts of intangeable ways such as "light", "space" and "feel" so the approach that we're different doesn't wash.
I suggest that we pay more attention to our clients needs when specifying systems.
- Steve Jones
As a software engineer who majored in architectural design (for at least some time) in college, I would assert that, ultimately, the biggest difference between construction and software engineering is the maturity of the former discipline. Software engineering is very much a paradox. While tightly coupled to the "Information Age", both as cause and direct product, we are somewhat pre-industrial in our organization and methods. It's our curse as one of the most fledging of engineering disciplines (aerospace can even consider itself as just EE + avionics in the space sector).
This is one reason that I so enjoy this article and the point it addresses. Unlike the architects of old, we have not learned to unify all stakeholders with formal languages and specification tools. If I'm paying a few million on a new building and don't have someone that can read a floor plan on my team, it's my fault - not the engineering team. And unlike any developer of a product that is mass produced (even tennis shoes, for that matter), we have no clear delineation of career and training between those that design, those that build, and those that certify. Have any of you heard of someone with NO software development experience that has a degree specifically in "Software Architectural Design", and does nothing more than that? Even the fictious widget-makers in college textbooks have evolved beyond the era of apprenticeship and round-trip production based on one person. Of course, this is one of the differences in the new Rational Unified Process: the destinction between engineering stage (analysis and design) and construction stage.
Some people disagree, saying that there are more differences. The concept that there is no "construction" stage in software, and that it's all design is not entirely true. There is a construction stage in software, we just don't have the automation capabilities yet to make mass-production and 100% round-trip engineering possible within our toolchains yet. People would have once made the same argument of crafting firearms once, and I imagine we only fight reform in that direction because we've learned from experience that mass-product means lost jobs. (of course, the history of industrialization also tells us that before full automation comes the employment of cheap and indentured labor to do construction-level tasks: say, from India) I also have a point of contention with the concept that software engineering is so fundamentally different because it is so intangible. Most people believe construction is guided solely by laws of physics and mathematics, and constraints of materials. More of architectual design is based on principles of psychology/sociology, style, and artisianship - intangle requirements and constraints in their own right - than typically understood. Furthermore, interfaces and component decomposition is not much more discrete and staightforward. Classic Roman architecture used three styles - Doric, Ionic, Corinthian - just for pillars; the typical civil engineer would be hard pressed to count on two hands the number of bridge "design patterns" used around the world. Bottom line is that we're a young field that needs to lose the excuses, and build on the lesson learned and evolution of our sibling disciplines.
- Shawn Price->