The Commoditization of Software Development
In the early days of computers software development was an arcane art, practiced by people manipulating binary digits with a few tools on machines with woefully inadequate resources. These early practitioners were generally mathematicians or physicists, but the limited resources of these machines meant their programming work was highly creative and in a sense artistic.
Later, high-level languages and associated tools brought computing to the masses. During the 1960s BASIC allowed even nonprogrammers to create programs that actually did useful things. Professionals used the improved technologies to create a more rigorous discipline that helped control the development process. Given the right mix of tools and techniques, the reasoning went, software development could become predictable and repeatable. Many successes bolstered this belief, though today still too many projects are brought to market by heroics. Of course, most of those methods expected that requirements were stable; thus, customer changes destroyed the process.
That expectation was rarely realized. Requirements do change, often at breathtaking speed. Big Up Front Design just can't deal with this, so now some of us are returning to the programmer-as-artist metaphor. Agile methods rely on small but highly focused teams, all composed of extremely bright and unusually competent developers. Obviously, cherry-picked teams will perform well in any endeavor.
But the method zealots and artists are all working on unstable tectonic plates that are shifting the software development landscape away from their highly-paid mastery of the art.
Our software is meant to fulfill an economic need, but we've not done a good job at keeping costs reasonable. Firmware is an odd thing; it has essentially no physical manifestation, yet it's the most expensive thing in the universe. Unfortunately the costs are such that management is looking for alternatives and has found some. Outsourcing is one case in point.
Outsourcing is a new vision of programming that transcends both the artistic and process models. Management increasingly sees software development as a rote task that can be farmed out to the lowest cost provider. Many successes only confirm the front-office's view that developers are simply commodities.
It's easy to see why management is drawn to the seductive siren song of outsourcing. Software costs are spiraling in step with escalating demands for more product features. But sending work overseas is really just a symptom of the commoditization of software engineering.
Another alternative to high-cost programming is www.rentacoder.com. A veritable army of inexpensive developers located all over the world will create software for a pittance. Buyers post problem descriptions and coders (not, mind you, "software engineers") bid for the work.
Here are two examples:
Need a strong ASP programmer to fix an error in our Mail Sending form script so it actually works. Max bid $20.
I have an html file that passes data to a php script--that data is written to a mysql db and is supposed to be passed to a perl script. It's not writing to the db and the data is not passing into the perl script--should be a quick fix. " Max bid: $5
How many of us would consider doing anything for $5? Yet coders swarm over these projects.
Rentacoder buyers include students contracting out their homework, small companies unable to deal with tricky web scripting, and even consumers who want a simple application--or even an Excel macro--to ease their daily work. Some embedded projects surface on the site, though, interestingly, most fail.
It's a cool concept, in a way, which for the first time lets the average Joe get custom-tailored applications for unbelievable prices.
But Rentacoder is the Wal-Mart of code. While I might not like how Wal-Mart affects wages and hurts small competitors, it's hard not to shop there. Ditto for Rentacoder--given a small project that's too much of a nuisance to develop in-house, why not post it on the site for 5 or 20 bucks?
The outsourcing phenomena and the Rentacoder model suggest that, in a limited way, developers are merely interchangeable commodities doing piecework. Perhaps the supply and demand curve has flipped in favor of software consumers. But clearly these signal a new zeitgeist in this industry, where the artists and process devotees of yore are replaced by a flood of cheap and reasonably competent programmers.
What do you think? How will Rentacoder and offshoring change our business and careers? Are you taking any steps to safeguard your career?
(Side note: Join me at the Embedded Systems Conference at the dreadful hour of 7:15 Tuesday morning, March 30th, for a Shop Talk where we'll discuss "The Future of Engineering in an Offshoring Economy.") Editor's note: to take an EETimes poll about offshoring, click here.
Jack G. Ganssle is a lecturer and consultant on embedded development issues. He conducts seminars on embedded systems and helps companies with their embedded challenges. Contact him at firstname.lastname@example.org. His website is www.ganssle.com.
One needs to be careful not to take analogies too far, but...
...I find it very easy to not shop at Walmart. It's been years since I've actually purchased anything from a Walmart (or a KMart or a Target or...) Usually, I don't find what I am looking for in the first place and if I do find it, I don't care for the quality of the offering. Sure, usually it's adequate, but if I can spend a few bucks more to get somethings that works twice as well or lasts twice as long, I'll go for the more expensive product. Low quality at a low price is not the bargain that it appears.
Unfortunately, I seem to be among the minority in the shopping world, just as I am probably among the minority of managers who think that good quality software is worth paying extra for.
- James Thayer
A company that will remain namless did two funtionally similar avionics projects, one outsourced largely to a foreign country, and one done in the US. Both were late, had development problems, were over budget, and ended up costing about the same. The over seas project ate money on air travel, phone calls, management, systems engineering. While the US project ate money on software developer time re-factoring the code. (the foreign code had to be re-factored to, and had subsequent systems engineering costs with that) If all you have are managers and systems engineers then you probably have to outsource, but if you can afford the staff, and can treat them right, (per Peopleware) then do it in house. (Having it's US based SW team in cubicles was probably why they were as in-efficient as they were -- that coupled with three moves during the project.)
- Bill Murray
There is a lot of software embedded and otherwise that is routine and if properly done, e.g. establishing a product line architecture, would greatly reduce development efforts. However the really innovative development e.g. the current effort for the software in the autonomous vehicles racing across the California desert or being used in some biotech applications, needs birght innovative people. There will always be both types of application. Do you think I can get someone to build a Wrap Drive for $25.
- George Goetz
The rentacoder model has lots of problems. The first one is how to specify what work is really to be done. Another is what happens when the work provided does not function as expected. And how trusted is the person doing your work. Is he a seasoned professional playing on the side, or a hacker who has just found a new way to deliver his little bomb that gives him access to your machine. Or maybe the bomb does something very bad, like formats your hard drive. And you watch it happen but cannot stop it...
I would not use this type of service because the risks are just too high.
As with most things, the price is only part of the decision. And, you always get what you pay for.
- Howard Smith
This was the line in the article that got me: "Yet coders swarm over these projects."
The concept of hooking up those looking for some software solution and available programmers is valid, and likely even useful. But doing this for $5, or even $20 is ridiculous. I can't believe this ever works for small projects. I don't think the economics work out, even for a third-world coder.
I just went to the Rent-a-coder site, and there was a request to turn an Excel application into a C# web application, assumably using ASP .NET, though the requirements didn't say. They say the bid is open to fair suggestions, but put it in the small category which is $100. Would a third-world coder (TWC for short) be able to make any money on this project? Let's look at the numbers.
We don't know which version of Excel, or the .NET framework, or ASP .NET is being used. Is it Windows 2000 Server? Windows Server 2003? All of this software is quite expensive. We'll have to assume that all of it is pirated, because TWC certainly doesn't have any money to spend on software. Let's look at hardware and infrastructure requirements, because those can't be pirated. It takes a reasonably good machine to run all of the needed software. What's more problematic is that much of the required software doesn't work well together on a single machine. Two versions of Visual Studio aren't going to be very happy having to co-exist. TWC must have some amazing configuration. Maybe he pirated Virtual PC too. Anyway, lets assume that TWC has computing hardware that's worth $1000. We'll lump Internet access into that as well, since that's what's making all of this possible.
Can TWC pay his bills with this job? Rent-a-coder takes off 15%, so we're down to $85. TWC probably doesn't have such a good financial infrastructure, so I'm not sure how he'll get paid. If he uses a PayPal account, we can yank another 10% off the top for that overhead and possible currency conversion. Now we're down to $76.50. Let's assume that TWC is a tax cheat, and pays no taxes of any kind, so he gets to keep the $76.50. But TWC does have to pay for that $1000 computer and overhead. Just how many $76.50 jobs does it take for TWC to do this? By my calculation, 13 of these will get him to the needed $1000.
A job like this is probably 4 hours, given the overhead of hooking up with the purchaser, downloading, setting up, coding, packing up the results, etc. That's assuming TWC doesn't run into any problems. Everything's worked out for him so far, so we'll make that assumption. To pay his overhead, TWC needs at least 52 (4 x 13) billable hours like this project provides so he can pay for his overhead. I'm not sure how much billing discipline TWC has, but it's got to be pretty good, since everything else is falling into place for him. Let's assume he bills at 80%. That's 65 hours to complete these projects, which is one really long week. I hope TWC can code when he's hungry. Maybe the Internet caf he's sitting in has free snacks.
I've made lots of optimistic assumptions. Even with those, it will be quite a stretch to come out financially with $100 jobs--even for TWC. Who could honestly say that anybody could take a $20 or a $5 task and come out financially?
I guess all those swarming coders haven't done the math.
- Steve Kubis
Jack replies: Steve, I like your approach! It drives our spouses mad, but we engineers do reach for the calculator to check, it seems, every claim.
Steve, excellent analysis, however, you assume they are doing it "on their own". I would guess that many are students or people that still live with their parents and have little or no cost to developing the projects other then their own time, which to them costs nothing.
Lastly, you should never underestimate the learning and fun factor. Many people are willing to make less money for a job if they are enjoying what they are doing.
- Michael Davis
Jack, you're right that engineers immediately grap their calculators to do the math, unfortunately, too many economic analyses, such as Steve's, use what I call "Starbucks Economics". We should not be asking how few double-tall-mochas a $5 American job will buy, but rather how many catties of rice that same job will buy in Beijing, Bombay, or Bangladesh. 65 hours a week would be considered a vacation to a lot of young workers in Cairo. Our global perspective on value and worth are so highly distorted while sitting in our $20,000+ car, waiting for that wonderful $5 Grande mocha, before the long drive to our $200,000+ home. Looks like them enjinneerz in America are going to feel the same pain that so many American farmers, steelworkers, garment industry laborers, and factory workers felt years ago. Is this pain somehow more justified by 4 additional years of education?
- Paul Jusak
I am one of "Top coders" at RantACoder.com While you are reading "two examples" above as cool joke - I have 200+ completed projects last year (via RentACoder only). For each $20 simplest task - I have satisfied buyer with offer for new, $100 project. Of course - I can't compete with "real man's", who asks about $100/h for the same work :-)
I agree that virtually all managers now view programmers as a commodity. It's the in thing these days. Managers who don't share this view lose their jobs to someone who does. This means that they try to get the cheapest person who can get the job done. Forget looking for the best! Forget quality! Everyone has a Wallmart view of the world. They just look to minimize the cost. In fact they have this opinon about all engineering disiplines. Bioengineering is the next big thing to be moved overseas. After all, they are convinced that THEY are the ones who inovate not the dull minded engineer. In his recent and now infamous Mercury News interview Craig Barrett said "If you go to any venture capitalist today and somebody has a bright idea, whatever it might be, probably the first thing the venture capitalist says to the principals of the company is ``of course you are not going to do your R&D here in the States, you are going to do it in India or China or someplace where the costs are only 25 percent of what they are here''. " He goes on to say that tech companies will continue to be founded in the Valley. "It's just that they are not necessarily going to create jobs in Silicon Valley."
Engineering is now a blue-collar profession that requires lots of education. It's no wonder that Bill Gates recent little rah-rah tour of universities seemed to fall on deaf ears. Students at Harvard, MIT, etc are just too bright not to realize that as soon as he can get someone in China or India to do the job he will not hesitate to fire them and tell them it's their fault for demanding a middle-class lifestyle as Microsoft CEO Balmer recently did. Yes we are now a commodity.
- Alwyn Goodloe