Separating job functions - Embedded.com

Separating job functions

Click here for reader response to this article

In responding to my aticle “Design by Clairvoyance” article, a reader named Andy wrote:

Sometime between, say, the construction of the pyramids in Egypt, and the 19th century, the building construction industry bifurcated into two expertise domains: design and construction. Architects designed, and contractors built. And in order for contractors to know what the architects had in mind, several powerful information exchange mechanisms evolved. Architects built models (e.g.: Michelangelo's 1547 model in wood of St. Peter's, presented to the Pope), and produced working drawings. The builders then used the working drawings to accomplish the construction. No competent builder would accept incomplete or incorrect working drawings. To do so would risk bankruptcy.

We aren't quite so disciplined in the construction of software. Though the scope of work in a large project may take years to complete, the work product is mostly invisible, unlike a skyscraper. When software construction is underway, our clients see people sitting at workstations. There are no big holes in the ground, no noisy, heavy equipment, no piles of materials. Lines of code, even if there are millions of them, aren't tangible to our clients. Hence, many projects are cancelled, or are finished “late” (as defined by the original, deeply-flawed estimate), and “over-budget”. The software industry needs to develop a discipline of design and construction. Until that happens, software development will largely remain, in the words of a former manager who was a hardware guy, “a thankless job.”

Specialization helped fuel the Industrial Revolution. The old model of a village blacksmith doing everything from smelting ore to shaping the product was out; now vast factories employed workers courting a multitude of injuries incurred from endless repetitive motions. The educated class specialized as well; who can count the kinds of engineers that now exist? Or the types of doctors?

Tom DeMarco wrote of separating the coding and debugging functions in his 1982 book “Control Software Projects.” Yet two decades have passed and, in general, most software people are the blacksmiths of yore, designing, coding, debugging, testing and documenting their work. Not to mention the special joy of embedded systems: wringing out hardware bugs. Instead of specializing, we embedded people try to be masters of everything, from assembly to C++ and analog to digital.

In the building industry, the architect and the contractor have separate functions, and the division of labor yields some pretty stunning benefits. They communicate by means of drawings and perhaps models of the proposed project. This documentation is a formal contract that tells the contractor what to build. The drawings arbitrate all disputes. Compare that to the standard “but that's not what I wanted” hand-wringing we hear from the 23-year-old marketing droids when they first see our completed embedded system.

Though the customer probably knows little about designing buildings, the drawings and model clearly explain what he's buying. Clarity prevents contention.

It's easy to dismiss an architectural drawing as something much simpler than a big wad of code. Of course the customer can understand the project by referring to the blueprint — anyone can. But the drawings do hide a lot of critical detail. Lots of analysis not shown on the page ensures that the building won't fall down, that plumbing and electrical will be adequate for its intended use.

Yet in software everything is in the code. We can't even agree on a standard for design docs. Gurus tout UML as the future, but very few embedded folks use that technology. Can your customers read a UML diagram? We desperately need a way to produce a complete design or model that's comprehensible to all important stakeholders.

Just as architects build scale models of their creations, maybe we can produce some sort of model for our inscrutable designs; something a customer can examine and critique. It's certainly possible to prototype GUIs and PC applications, but embedded systems are tougher. They may have to communicate with other machines, take data, and interact with the environment. The model takes a lot more effort than playing with cardboard and glue sticks.

And worse, a software model never looks like a model. No one has the sense that it's 1/10 the size of the real project. “It works great! Let's ship it!”

An architect's model is very clearly a mock-up.

Perhaps the most interesting part about the building trades is that everyone expects to pay the architect big bucks for a design. One that will take months, maybe years, to produce. The contractor doesn't start pushing dirt around till the drawings are done and the customer has checked every detail of the proposed structure.

In the embedded field the first question we get when the boss wants to start a new project is “How long will it take?” We have a day, maybe a week to hash out enough design to produce the usual wildly optimistic schedule and costs.

Architects don't give a price. The contractor does, once handed a complete design. For only then is it possible to figure the number of sinks, tile, wiring and labor that's needed.

Why is software different?

Jack G. Ganssle is a lecturer and consultant on embedded development issues. He conducts seminars on embedded systems and helps companies with their embedded challenges. Contact him at . His website is .

Reader Response


Software does not need to be different if (considering UML as modelling tool):

1. All s/w job-holders, including the Marketing droid understands UML2. Customer understands UML

Point 1 is a matter of discipline in using UML (which is a rarity today!). Point 2 comes the difficult part with Non-technical customers considering the timeline.”BOTTOMLINE: ALL SOFTWARE JOB HOLDERS and STAKEHOLDERS (including embedded folks)NEED TO GETMORE DISCIPLINED IN USING STANDARD MODELLING TOOLS LIKE UML”

– Saravanan


The practice of building houses has been around for about five thousand years, yetthere exists striking resemblence between what was built then and what is being built now. Issoftware different?

– David J. Liu


Before everyone starts glamorizing the construction industry, perhaps they shouldread the following link:

http://www.poppendieck.com/construction.htm

It seems that software design does not have a monopoly on poor planning. It just has moretools to do the wrong thing faster and with little visibility.

– Kyle Heironimus


Q: Why is software different?

A: In the construction industry, you are working with “absolutes”, i.e. physical objects in3D. These are “things” we are familiar with, can pick up, move, and articulate. In the softwareindustry, it's different. We are working with “abstracts”, i.e. encoding our thoughts andprocesses. The best that we have done so far is OOP and modularization. The problem withsoftware is that its data structures can get so complex, it's hard to create a flow diagramentity that all can relate to. Create the symbol for the following that all can understand: Anarray of hashes of hashes containing a linked list of function pointers. Absract concepts arethe hardest to visualize and that is the main reason for our convoluted industry. Look at ourtool set. There are many different languages to express our abstractness. Some tools expressabstract concepts better than others. Once we find the single tool that can express all ourabstractness, maybe then we can standardize our process.

– Steve King


I've been wondering this for over 20 years. Physicists use a different set ofnotations than Engineers because they are concerned with different things. We don't want to dothe science all over again. We want to use that science. Engineers are educated to use theproduct of physics, math, …

Computing scientists use lots of specification notations. Model based notations like Z, B andFocus as well as process algebras (CCS, CSP, Pi) are all popular in European CS departments.They teach them to their Undergrads. I use term rewriting systems in my own research. I buildabstract models of protocols and can execute them as well as use a model checker. David Harrelis working on some cool stuff that shows promise. Yet all of these are relly more like the diagrams you draw in Physics class. Too detailed and low level for designing a bridge. Greatfor investigating the kind of stuff we computing scientists are interested in. We need domainspecific specification languages. Hopefully ones that can be executed. We need lots of thesenot bigger ang bigger general purpose notations.

– Alwyn Goodloe


The primary difference is that there is no construction stage in software. It'sdesign all the way from high-level architecture to detailed code. Executable code is simply thelowest level of design. That's why there can be no “hand-over” point.

– Nick Hewish


Jack asked: “Why is software different?”

I believe that the fundamental difference is that you can't see it like you can a building, sooutsiders' perceptions are very different. There is also the fact that you can launch intocoding in a way that you cannot when constructing a building. Well, nothing larger than, say,a garden shed. Equally the consequences of errors during the construction are minimal forsoftware (timescales and costs notwithstanding). A program that collapses around your ears isminor headache compared to a building doing the same. So I guess that what I am saying is thatthe perceived need for discipline right from the early stages isn't there for software.

– Andy McC


I think a problem is the way software engineering is taught in school to students. They aretaught smaller easy to change programs that teach the “Burn and Learn” method of iterativedevelopment — Much the same way kids at summer camp are taught craft projects like makingdecorative light switch wall covers. Need 10,000 for a huge project — not a clue how to doit, it's not taught. Need a disciplined process for something safety critical — that's likelynot covered either. There are several specialties in software that really need to be set up aspossibly advanced degrees. (SW Architecture, Safety Critical Systems, Embedded Systems, etc.)

– Wally Murray


Most of the projects I've been involved in are very “time to market” driven. That doesn't allowtime for doing comprehensive design and modeling before coding. Use cases seem to be a goodtool that the customer can understand without the need for extensive training. As thedevelopment stage progresses those use cases “should” be elaborated on and more low level usecases added. Unfortunately the norm for market driven projects seems to be to define someminimal and incomplete user requirements spec and start coding. Do some testing along the wayand follow up with the old promise, “we'll update the specs when the code is working”. Betteruse cases to capture the user requirements may not be the final solution but it's a verypractical tool that carries little overhead for all involved and it does provide a “blueprint”that everyone can look at and understand.

– Ray Burks


Why software is different:

Customers often know what they want from a building, but don't often know what they want fromsoftware. So, a software engineer must often also be a requirements engineer (you forgot to add”specifying requirements” in your list of jobs: …the blacksmiths of yore, designing, coding,debugging, testing and documenting their work). Besides that, there can also be the task ofextracting domain expertise from a client.

The bifurcation of architect and contractor in construction was a natural result of 1) theprogression of construction engineering knowledge and 2) increasingly larger scales.

1) In construction, the contractors don't need to have the same level of knowledge about thedesign as the architect. In software, design and coding go nearly hand-in-hand, and there isusually a feedback loop because of incomplete knowledge at design time. Thus, a coder oftenneeds to have nearly the same level of knowledge about the design and design principles as thedesigner.

2) In the construction of anything more complex than a house, you must have more than one ortwo people working on the actual construction in order to get finished in a reasonable amountof time. However, one or two people can complete a respectable software project in a reasonableamount of time.

Incomplete a priori knowledge of requirements and implementation obstacles, the design/codefeedback loop, the plasticity of completed software (“just change it”), and the fact that allsolutions are “custom” to varying degrees make software more difficult to estimate.

The allocation of responsibility for documentation is about economy of time – it's faster tohave the developer write the documentation than to have the developer explain thedesign/code/operation to someone else, and have the other person write the documentation. Also,detail is bound to be lost in the process.

UML's Use Case diagrams may currently be the best candidate for communicating with a naive'customer about what the software *does* – and that's usually all a naive' customer cares about.However, there are many candidates for diagramming how the software *works*, which is the meatof the architect's and developer's work. There is currently no single modeling method that canarticulate all the aspects of the software that architects and developers are usually concernedwith, such as contained in other types of UML diagrams, schematic block diagrams, control flow,data flow, state flow, timeline flow and any other type of diagram that you can name. I thinkthat what we really need is a way to hook all of these different model types together in abehavioral sense. I'm all for Model Driven Architecture/Design – further, I'm all forexecutable models!

I like the provocative idea of splitting up tasks, but try selling the idea of hiring moreemployees to managers who think that software is already too expensive. Good Luck!

– John Hopkins


The first question is “why to construction morph from the backsmith do everything todesign/construct? I suspect economics drove this. People want to know how much its going tocost.

Software is similar. There is the predominate “blacksmith” method and the “create specs andcost estimate/then bang out code”.

In construction, I believe there is two estimation phases. One at theArchitect/Engineering/Design phase and then the contractor that is going to do the job alsodoes an estimate. If the bids come in too high, then the original estimate was faulty.In software there isn't the second estimate. The coders are told what the timeline and budgetare.

I agree a design <-> construction methodology could help deal with the budget over run problem.

– Tim Flynn


I can't argue that better development techniques can and should be realized. However the”construction” metaphor seems forced.

Consider if as developers we were to produce “architectural drawings” for the benefit of “codebuilders”. In building construction a collection of (hopefully) skilled tradesmen know how toimplement the design using established codes and standards. The laws of physics, loading,stress analysis, piping losses, etc. are all taken into consideration, both by the architectand by the tradesmen. A carpenter knows about using 2×6's instead of 2×4's for window headersand also when to use a 2×8 or 2×12. He knows about “proper construction techniques” that arewell established, codified and legally enforced. What's the equivalent in softwaredevelopment?

Where are the “standard” ways of implementing a particular abstract concept in software? Whenlinked lists, sorting algorithms, timing, interrupt routines and keyboard/ display and GUIinterfaces become as ubiquituous and readily understood by software “tradesmen” as doors andwindows and 2×4's are to caprenters then maybe this metaphor will apply.

I think software libraries are an attempt in this direction, a way to collect commonly usedfunctions in a reusable, “codified” form. Unfortunately (or perhaps necessarily) theselibraries exist in abundance, no two alike, and are generally crafted and reused by smallgroups. Mass public acceptance of library code (the metaphorical equivalent of standardbuilding construction practices) occurs at the compiler level with the so called run timelibraries. Extend these libraries to include entire algorithms (objects anyone?), standardizethem and then you'd really have something better.

– Stephen Ciricillo


One major difference between building construction and software construction is how deadlines are determined. Houses can be made quickly, mostly due to the fact that the design is done years before and just “cookie-cuttered” out to make new houses, something akin to burning a release CD based on software already written. When a new building has to be designed, the head of the project doesn't say to the architect and contractor “We need this building done by August 1st.” Instead, they know that a proper design takes time, and if it's rushed you end up with a shoddy building that could collapse and result in deaths.

Software deadlines are often created before what the software needs to do is understood.

It's all about understanding that good design takes time.

-John Patrick


I talked with an architect about building a health club. They get 10% of the total cost for their work, and he did give me a price. The plans and pricing didn't include the parking lot. Several of my friends had their own homes built, and they worked with high school drop outs who only knew which end of a hammer to hold. Often, the owrk had to be redone. Once, my friend had to go bail the orkers out of jail before getting started for the day. There are too many of these types of people in software. They think they know something because they know C++ syntax. Of those who really do know something, I think they do specialize. Data base design is different from operating system design. Real time is a specialty field. Programming graphics cards and game design are specialties. Windows is different from Linux. Networking, or programming routers is another. O.K. Maybe we do dabble in all and specialize in none. Would someone pay me to do just one thing?

– Dennis Ludwig


In the construction industry, architects and builders understand each other very well and this is not just because of drawings, but also because the domain, the designs, the technologies and the tools change very slowly. The communication languages and channels are all well established.

In software, the design on a VxWorks platform would be significantly different from that on a WinCE.NET platform. How many times have we struggled to communicate, much less convince, our design to a co-designer?

In construction, most of the standard problems are known and standard solutions are used. In software, teams solve same problems and 'find' their way to standard solutions. If we need a strange component, we love building it from scratch rather than changing the design to use an available standard component.

In our industry it is harder to learn from others. If there were easier ways to learn and understand how others have built their software, our own designs would become much better. We will also learn to communicate our designs to others. This would quickly evolve to a set of design patterns/frameworks that everybody understands and uses.

Until we have the means to communicate our domain, architecture, implementation and technology thoughts clearly to others, we would end up doing all by ourselves!

– Shankar Velayudhan


Why is software different?

This is due to two reasons,
1. You cannot breakdown a software system into universally accepted discrete components.A building can be broken down into tiles, bricks, pipes. A chip could be broken down into AND, OR, and other logic gates. Each discreate component would be used to build complex structures. Each discrete component has a well defined input/output interface and specifiction and this remains same across different implementations of the system. But, this is not the case in software. You could argue and say objects/classes could be the discrete components. But no way would we agree on a standard interface.

2. It is difficult to visualize software. Buildings can be visualized by models, drawings. Electrical designs can be visualized by circuit diagrams. But there is no standard way to visaulize a software system. UML could be used, but again everyone must agree on it. And we could debate on it for years on whether is fits all software applications/domains

– Kedar Chitnis


In all this discussion there appears to be little mention of or concern for the client. In the construction industry Architects provide a translation of what the clients want/need, models are used more to confirm that the intrepretation of the clients requirements are appropriate than as a construction guide(different drawings are used for this). I think that one of the shortcomings of we in the software industry is that we get too involved in what we want to do with the technology on a personal level and forget why the code is being written. Good Architects fill their clients needs in all sorts of intangeable ways such as “light”, “space” and “feel” so the approach that we're different doesn't wash.

I suggest that we pay more attention to our clients needs when specifying systems.

– Steve Jones


As a software engineer who majored in architectural design (for at least some time) in college, I would assert that, ultimately, the biggest difference between construction and software engineering is the maturity of the former discipline. Software engineering is very much a paradox. While tightly coupled to the “Information Age”, both as cause and direct product, we are somewhat pre-industrial in our organization and methods. It's our curse as one of the most fledging of engineering disciplines (aerospace can even consider itself as just EE + avionics in the space sector).

This is one reason that I so enjoy this article and the point it addresses. Unlike the architects of old, we have not learned to unify all stakeholders with formal languages and specification tools. If I'm paying a few million on a new building and don't have someone that can read a floor plan on my team, it's my fault – not the engineering team. And unlike any developer of a product that is mass produced (even tennis shoes, for that matter), we have no clear delineation of career and training between those that design, those that build, and those that certify. Have any of you heard of someone with NO software development experience that has a degree specifically in “Software Architectural Design”, and does nothing more than that? Even the fictious widget-makers in college textbooks have evolved beyond the era of apprenticeship and round-trip production based on one person. Of course, this is one of the differences in the new Rational Unified Process: the destinction between engineering stage (analysis and design) and construction stage.

Some people disagree, saying that there are more differences. The concept that there is no “construction” stage in software, and that it's all design is not entirely true. There is a construction stage in software, we just don't have the automation capabilities yet to make mass-production and 100% round-trip engineering possible within our toolchains yet. People would have once made the same argument of crafting firearms once, and I imagine we only fight reform in that direction because we've learned from experience that mass-product means lost jobs. (of course, the history of industrialization also tells us that before full automation comes the employment of cheap and indentured labor to do construction-level tasks: say, from India) I also have a point of contention with the concept that software engineering is so fundamentally different because it is so intangible. Most people believe construction is guided solely by laws of physics and mathematics, and constraints of materials. More of architectual design is based on principles of psychology/sociology, style, and artisianship – intangle requirements and constraints in their own right – than typically understood. Furthermore, interfaces and component decomposition is not much more discrete and staightforward. Classic Roman architecture used three styles – Doric, Ionic, Corinthian – just for pillars; the typical civil engineer would be hard pressed to count on two hands the number of bridge “design patterns” used around the world. Bottom line is that we're a young field that needs to lose the excuses, and build on the lesson learned and evolution of our sibling disciplines.

– Shawn Price

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.