The Curmudgeon Strikes Back - Embedded.com

The Curmudgeon Strikes Back

The days of the Bad Guys list have ended. From now on, only the elite Good Guys are worthy of mention. Assume everyone else is bad.

Regular readers know that after finishing a series of mostly technical columns, I like to ramble and reminisce about the good old days (a curmudgeonly pastime) before segueing into the next technical topic. This will be one such column.

I also want to update the Good Guys/Bad Guys list and address feedback from my comments on RTOSes.

But first…

Some clarification

Since my May column came out (“All Problems Are Simple,” p. 7), I've received several e-mails from folks who had trouble following the derivation for the location of the root from Equation 9, reproduced here as Equation 1:

x = x0 + B(y — y0)[1 + C(y — y1)] (1)

The equation and its use has some subtleties that might not be noticed at first glance, so perhaps it's worth explaining here again, in a bit more detail. Note the exact form very carefully. The two terms in parentheses contain different parameters of the fitted points: one is y0, the other is y1. I did this (following Newton-who else?) very deliberately to get the following derivation to come out more simply. You can see that when y = y0, the whole right-hand expression after B goes to zero, and so we are assured that x = x0 as it must. Similarly, when y = y1, the rightmost term in parentheses goes to zero, and we get a simplified form:

x = x0 + B(y — y0) (2)

which lets us solve for B. Write this approach down and mark it as important, because it will save you a lot of time some day. Now, you don't have to use the form of Equation 1-use any old form for the quadratic, and do the math correctly, and you're still bound to get the same final form for the solution. There is, after all, only one quadratic that fits a given set of three points. But if you use the wrong form, you can easily find yourself lost in an endless maze of long algebraic equations.

RTOS feedback

I got many e-mails, mostly complimentary, about my comments regarding real-time operating systems (RTOSes). I wanted to include some of them here, but sadly, my e-mail system is still hosed, so I don't dare try. I didn't mean to imply that one should never use an off-the-shelf RTOS. There are many good RTOSes and they have a purpose. The thrust of my comments can be best summarized, as I did earlier, by quoting race-car designer Harry Miller: “Simplify, and add lightness.” Use what you need-no more, no less. I find that commercial RTOSes are overkill for many of the things I do. I suppose it's nice to have four different mechanisms for one task to communicate with another, but I only need one. The other three not only take up space and add complexity to the RTOS, but they increase my learning curve.

It's nice for tools like Microsoft Excel to have a powerful macro system that lets the people who use such things program the tools to do fascinatingly complex jobs. But I don't need those features, because if I have a fascinatingly complex job, I'll implement it in C or C++ or Matlab or a dozen other possible ways. Trust me on this: I will not be needing an Excel macro any time soon. I'll settle for a spreadsheet that runs without crashing. I have the same attitude about feature-laden RTOSes.

Most of my respondents agreed, mainly because most of them, like me, are old enough to remember having to make the build/buy decision, and, for one reason or another, choosing to build. Like me, most are concerned about current trends to jump to a familiar RTOS without making that critical decision.

Many engineers today are trained early on to use one of the popular RTOSes. The next time they have a problem, it's only natural that they should try the same RTOS again. But remember, to a person whose only tool is a hammer, every problem looks like a nail. Of all the reasons for doing something in a certain way, the worst possible reason has to be that it's the only way you know.

Ever since its inception, this column has been about tools and toolboxes. The title is no accident-a lot of thought went into its choosing. My sense is that the more tools one has in his toolbox, the more likely he is to use the right tool for the job. It's been my goal, as well as my honor, to help you add tools to your toolboxes. All I'm trying to say about RTOSes is keep your options open, be flexible, and use the tool appropriate to the task.

What's that you say? We should include unused features to make room for growth? Well, we could do that. Or we could wait until the need arises. It may never do so. I like what P.J. Plauger has to say: “Never put off until tomorrow, what you can put off forever.”

Many years ago, I built a dynamic simulation for a multi-round, tube-launched missile system. (Side bit of trivia: at the time, the U.S. Army was adamantly opposed to tube-launched missile systems. They were concerned about “tip-off” behavior that occurs when a missile leaves its tube. My boss, Dave Hammock, argued that while tip-off did indeed occur, it was predictable and therefore compensatable. Dave pushed very hard for the use of tube-launched systems-a push that culminated in such successful programs as MRLS. Whenever you see pictures of all those rockets streaking out of tube launchers in videos of Operation Desert Storm, it wouldn't hurt to say a silent word of thanks to Dave, for perservering in his efforts, in the face of withering resistance. I know the GIs who depend on those missiles owe him a debt.)

Anyhow, the simulation was comprehensive, with models for the wobbling of missiles in the tubes, bouncing of the tubes on their support, the motion of the support relative to the tank-even the dynamics between the tank and the ground. I thought I had enough complexity already, and didn't go out seeking more. Nevertheless, when faced with a decision to include things like thrust misalignment or dynamic mass imbalance, I'd always ask Dave if I should.

Invariably, he'd say yes, perhaps we'd better. So I included the effects.

Now, anything that one models in a simulation, needs supporting data. A thrusting rocket model needs to know the thrust. A misalignment model needs values for the angles. After the model was finished, and all effects had been included, I was ready to try some real data. I went to Dave and said, “I need values for all those misalignment and imbalance parameters we included.” He thought for a time, and said, “Just make them all zero, for now.”

Well, you can guess what happened next. That simulation was run many, many times, but the added parameters were never given any values other than zero. I had just included a whole lot of extra multiplications by, and additions to, zero, for no good reason.

I took a vow that very day, that I'd never again add features to a program until I'd established a need for them. And I haven't. And I wish a lot of other folks followed the same philosophy. Never put off until tomorrow, what you can put off forever.

Traffic lights

In my column on RTOSes, I mentioned that Intel's announcement of the 8080, way back in 1974, was accompanied by an article describing how to use the 8080 to switch a traffic light from red to green. I remarked that this was a sad commentary on human nature, that such an application was the limit of our imaginations. It took visionaries like Gary Kildall to see a 64KB personal computer, complete with a full-blown disk operating system, lurking inside that seminal chip.

I got e-mail from one highly incensed fellow who happens to work for a company that makes-wait for it-traffic light controllers. He took serious umbrage at my belittling of the problems of building such controllers, and my suggestion that Intel's original problem didn't deserve a full-blown RTOS.

Oh, well, you can't please all the people all the time. Sorry, sir. No slur was intended, only historical accuracy.

Curmudgeon alert!!

Now back to Dell and Sony, Good Guys and Bad Guys, and the diabolical machinations of the PC industry. When last seen, our hero was trekking west across the Texas badlands in a broken down, barely working covered wagon-er, better make that a broken-down, barely working rental RV, seeking greener pastures in the beautiful desert southwest.

I guess my infatuation with computers must be slipping over the years. Time was, wherever I went, my computer was sure to follow, gently riding in the place of honor, the passenger seat beside me. When I arrived at my destination, the computer was the first thing to be unloaded, set up, and plugged in.

When I left Virginia in 1985, my beloved Kaypro was the last thing to be packed by the movers. It was double-boxed, with lots of paper padding around it. It was the last thing to go onto the moving van. At the destination, it was the first thing to come off, directly from the driver's hands into my own.

As usual, my intent was to put the box containing the Kaypro in its rightful place, in the passenger seat of my car. Never mind that I drove away with the box still sitting on the ground. We'll fast forward over the scene where my wife, driving her own car, ran over it, or the horrifying sounds of tortured metal scraping on concrete that ensued. At least my heart was in the right place. And the Kaypro survived with only a few scrapes and dents to add character. It tended to wobble on the desktop, but its performance was as solid as ever.

I definitely must be slipping. By the year 2001, and after nearly a solid month of arranging, coordinating, and nearly two weeks of packing goods and animals, the best I could do was to tell the movers, from my horizontal position on the floor, “Be careful with the computer.” Definitely slipping.

The movers, of course, ignored me completely, and threw the Dell, with minimal packing, cattywampus into a box, which seems to have been placed at the very bottom of the load, just beneath the pickup truck and commercial-grade lawnmower. By the time the Dell arrived in Arizona-one month late, thanks to a close encounter with a stowaway fire ant-it had seen better days, both externally and internally.

Browser wars

Flashback to two months earlier, when I was having trouble getting my Netscape browser to work properly with eBay. The eBay folks said I needed to “upgrade” to Microsoft Internet Explorer; they didn't guarantee their web site with Netscape. I said thanks, but no thanks, and averred that no Internet Explorer would ever darken the CD-ROM drive door of my computer. Instead, I downloaded Netscape 6.0.

I had trouble getting it to work, on several fronts. I called Netscape's tech support line for help. Now, as you probably know, Netscape, like Microsoft, has a pay-per-incident policy. You can't really fault Netscape for this; after all, they give the software away. They have to make money somewhere.

But when I told the techie that I had several questions, he said, “You realize, of course, that I'm going to have to charge you $30 per question.” I said that I thought not. I suggested that I really had only one problem, which was that Netscape 6.0 wasn't working. He said, “Nevertheless, I have to charge you for each separate question.” I asked to speak to his supervisor, who, after much spirited discussion, basically said, “$150-take it or leave it.”

I thought for about a millisecond, said, “Leave it,” and hung up to download Internet Explorer. I believe at that time it was around 5.0. The new browser installed easily and worked fine.

As near as I can tell, my computer got a virus roughly the next day. You see, virus writers (may they burn in hell and be forced to sit on perpetual telephone hold, listening to Barbra Streisand's greatest hits) target the most common targets. Last year, that was Windows 98, Microsoft Outlook, and Internet Explorer. This year, Windows 98 has become Windows XP. The virus writers look to exploit holes in all the most popular tools. And, wonder of wonders, they find them.

In retrospect, I think my switch from Netscape to Internet Explorer may have been more than a little bit rash. If I had it to do over again, I think I'd bite my tongue and figure out how to use Netscape 6.0. Or, better yet, stick with version 4.xx, which I'm now using again. Even the Netscape tech support folks don't seem to be recommending 6.x. But by then the damage was done.

Understand, I don't hold Microsoft blameless in this. I know they must have their hands full trying to stay one step ahead of the virus writers (may they burn in hell and be forced to download Windows XP for an Apple ][, over a 110-baud Radio Shack modem). But c'mon, gimme a break. The most common way virus writers get in is by exploiting buffer overflows in various tools. The idea is, you write characters to some command-line buffer, until the buffer is full. Then you keep writing, so that the remaining characters now go into program space. The next time that bit of the program is executed, it starts up the virus.

My question is: how many times must virus writers (may they burn in hell and be forced to install updates to McAfee Virus Scan for all eternity) exploit a buffer-overflow flaw, before someone says, “Gee, maybe we'd better check the number of characters we write into this buffer”? Doesn't seem like rocket science to me. We could probably fit the extra code into the picture of that stupid little dog, or the dancing paper clip.

Back to real time

Whatever the reason, my by-now much-detested Dell got banged around a bit, caught a virus, and had to have its hard drive(s) wiped and all the software re-installed. Before the wipe, I installed a CD-ROM burner and saved all my precious data (including all my files for this column) Multiple times, I saved it. After the wipe, I learned two things, to my horror: the Dell was now showing RAM errors, and the data that I thought had been burned to CD-ROMs, hadn't.

In desperation, I rushed out to Comp-USA and bought a brand new Sony, top of their line. The bad news is that, like all other new PC-based computers, the Sony came with Windows XP, Microsoft's latest bid to force yet another upgrade upon us. Only about two days before, I'd vowed to my wife that XP, like IE, would never darken my doorstep. The moral: never say never.

The good news is, the Sony is smaller and much faster than the Dell, and the threefold difference in clock speed almost makes up for the slower performance of the operating system. Almost. Boot time is still measured in minutes.

What's up with that? This computer runs at 1.7GHz. In its two minutes of boot time, I calculate that it's executing roughly 200 billion instructions (assuming pipelining). What can it be doing in there? That's enough instructions to solve the universe. In the '60s, I could have run my simulated Apollo trajectories 1.2 million times. That's probably 1,000 times more than I actually did run them, during ten years of work on Apollo. All in the time it takes Windows XP to say, “WELCOME.”

Sometimes I miss my old TRS-80. You turned it on, it said, “READY>”. That's my kind of computer.

Juggling computers

Well, I won't bore you with the rest of my computer woes. I don't want all this to get cut again. Suffice it to say that, in the long run, I got the Dell's virus cleaned (along with everything else), and found the RAM problem-a loose SIMM strip, probably never latched down properly in manufacture.

I even found one more CD-ROM disk I'd overlooked before, which-miracle of miracles-contained the missing data. Today, both the Dell and Sony have those precious columns restored. Currently, I'm bouncing back and forth between the two computers, trying to keep one virus-free. Unfortunately, the current virus, Klez, not only comes in anew each time I log on, but it's designed to defeat most popular virus scanners. How's that for fun?

The Sony quickly replaced the Dell for detestation value, and not just because it runs XP. Love quickly turned to hate when I tried to delete the game folder and its contents. (I'm not much of a gamer, but I can't be trusted with Freecell. So whenever I get a new computer, I usually go through an orgy of Freecell for a few days, before I wipe the games off in a flurry of guilt.)

In the past, I've always just done a search on winmine.*, sol.*, and so on, and deleted them. Okay, I know this isn't the accepted Windows way, but it's always worked for me in the past. But when I tried this in XP, my Internet dialer went with the games. So did my printer driver. So did Mathcad.

Huh? Wha…? What's up with that? How can deleting game files break the whole system? Is this Microsoft's way of proving to the Justice Department that Windows really is a monolithic whole that can't be subdivided into modules?

Okay, we can talk about the fact that I should have used the Control Panel to delete the files. But save your breath. I've already been sufficiently scolded on that score. I'd still like to know what numbskull at Microsoft thought it would be clever to assign to essential system files, names that begin with WinMine, Sol, and Freecell.

Regardless of the reason, I had a problem. Windows XP was hosed, and needed repair. That's when it slowly began to dawn on me that in my haste to unpack and get the Sony on line, I didn't recall seeing any software disks or manuals. And indeed, there weren't any.

Flashback to yesteryear again. My Kaypro came with a whole boxful of manuals, easily 20lbs of them, plus a huge box of floppies (5.25-inchers, remember them?). The only thing I had to do was to make backup copies of the disks. Until very recently, most systems came with, at the very least, a set of CD-ROMs and a manual for each application.

That was then, this is now. Today, if you want manuals for the installed applications, see the files on the hard drive. If you want a manual for Windows XP, dream on-there ain't one.

What followed was one of my typical, African Queen-like adventures to get a brand new system up and running again. On one of the rare occasions when I was actually able to reach Sony by phone, they told me I should have gotten a set of recovery CD-ROMs. I didn't. They said they'd send more. They didn't. Then they said my receipt was “invalid”-didn't show the store ID number. Sent them another copy. They sent the CD-ROMs-which were the wrong ones. Had to do the whole thing over.

Understand, I'm giving you the short form here, and leaving out the emotions, the loose items being thrown around the room, the red face and high blood pressure, the shouting and cursing, and threats of dire consequences. Eventually, I got my disks, and even got the Sony working again-until the next virus attack, at least.

Oh, yeah, I almost forgot: the Sony comes with PC-Cillin, but it didn't detect the Klez virus. I uninstalled it, and bought and installed a copy of McAfee Anti-Virus-once the best known and most highly respected of all virus scanners. It couldn't even update itself successfully.

I'm now using Norton Internet Security, which includes both Norton Firewall and Norton AntiVirus. It seems to work just fine.

Did I mention that all virus writers should burn in hell?

Optimizing service

A couple of years ago, when I first got the Dell, I ranted against them and had nasty things to say about their tech support and customer service departments (who still owes me, by the way, the $100 service call to Microsoft they promised to repay). The Dell is currently on its fourth video card. Apparently there's a fundamental problem with the design of the board, but Dell won't change to another type. Their attitude is, “You bought this brand, you get this brand.” So they just keep replacing it.

My original reason for buying the Dell had been their reputation for good quality at a decent price. I'd owned four previous Dells, and been totally satisfied with each of them. Dell also had an impressive, seemingly iron-clad guarantee that they could be trusted to repair any problem in 24 hours.

Well, guess what? It really doesn't work that way. Before Dell will honor the guarantee, they put me through a telephone grilling session, followed by walking me through a process of taking the computer apart, checking every possible problem, and verifying that it's what I told them it was in the first place. It does no good to say, “It's the same problem as the last three times.” Procedures must be followed. It took me awhile to realize, but now I'm convinced Dell is using me to perform free diagnostic service.

Once we verify that the video board is bad (again), Dell orders the board. Only when it comes in, four or five working days later, does a technician come out to install it. Then he leaves, not even bothering to put the side panels back on. Dell would do better to just mail me the board.

During my battles with Dell's tech support, which used to be so good, I came up with a theory concerning service, tech support, and optimizing customer satisfaction. As an old curmudgeon, I can remember when firms actually cared if the customer was happy. I can remember that if a problem arose, the vendor, be it grocery store or car dealership, would move heaven and earth to correct the problem, falling all over themselves all the while, begging to be forgiven.

Not anymore. I think it all started when auto manufacturers began to realize that some customers are leaving the fold, no matter what. When the engine falls out of the Chevrolet (as one of mine did), I'd vow never to own another Chevy. When the engine in my Ford blew at 150 miles, I vowed never to own another Ford. Eventually, it all sort of averages out. Someone realized that for every customer leaving Ford for Chevy, another one was on the other side of the cycle, returning to Ford from Chevy. A service department only needs to be marginally better than the competition's.

Same thing with computers and software. I can remember when every call to tech support was a pleasure, as the techies bent over backwards hoping to solve my problem. No longer. According to my theory, some bright accountant had done some kind of Excel spreadsheet study, which proved that lousy service was more cost-effective than good service. This imagined accountant worked out the point at which the total cost of lost sales plus support salaries is minimized. It's okay to lose a calculated number of irate customers, as long as you can cut costs in tech support. Especially if the customers will be back, later, in the Ford-Chevy ping-pong game.

I reported this theory in this column, at the time, and got a nice message from someone in the industry, who told me this theory was exactly right. He said, in fact, that the notion was given in a paper before the National Society of Automotive Engineers, and even gave the reference. Aha! Suspicions confirmed.

In truth, if you don't like the service at one company, where are you gonna go? To another vendor with equally lousy service? If they all read the same paper, they're all going to be doing the same thing, right? Never mind doing the right thing-a concept which today seems totally pass.

No more good guys?

My theory sounded good at the time, but it's two years old now, and things have progressed even further. Today, I find myself slowly formulating a theory that improves on the first one. Today, I think an even brighter accountant, armed with no spreadsheet at all, figured out that no service is even cheaper than lousy service. Bury the customer in a deep enough menu system or layers of web pages, and with any luck, he'll give up and go away. Keep your telephone number a secret, and your odds improve. But a perpetual hold is just as good, and not as obvious.

You think I'm exaggerating? Try to talk to an actual, living, breathing human being in tech support for any technical company. You will not get tech support; you will get a runaround. Call their number, and you will get either an endless maze of menus and wait queues, or a pleasant, recorded message inviting you to visit the company's web site.

If you go to their web site, you will find yourself stuck in some endless, automated diagnostic system that doesn't diagnose anything. You can try in vain to find a telephone number for live tech support.

In trying to reach someone at Sony, I actually had the experience of wandering about five to seven levels deep in their automated menu system. When I pressed the next button, I heard a pleasant recorded voice say, “Goodbye.” Click.

Good luck, also, finding a computer that comes with actual, physical recovery disks, or even more ludicrous, real paper manuals. These days, it's all virtual. In all likelihood, if your manuals exist at all, they are stored on that hard drive that just crashed.

After my latest round of adventures, I'd like to say that I have great new material for my Good Guys/Bad Guys list. Except that the list has almost become irrelevant. Pondering the new reality for a time, I suddenly had the following epiphany:

To a first approximation, all vendors are Bad Guys.

This is important, people. It alters my Good Guys/Bad Guys list beyond all recognition.

Only the good

In keeping with this modern trend, I'm revising my approach to the Good Guys/Bad Guys list. The bad guys are too numerous to mention. I'm only going to tell you about the good ones. If you don't see a given company on the Good Guys list, it's safe to assume they're Bad Guys.

Fortunately, this approach makes the list much shorter, not to mention the fact that it removes from me the burden of curmudgeonliness. Currently, I have only three companies on the Good Guys list. They are, roughly in order of satisfaction:

  • PowerQuest, vendors for Partition Magic, which remains my favorite Windows app. It's my favorite because it's the only one I own that never crashes. Plus, their tech support is not only top-notch, but they have saved my buns more than once, by helping me fix a problem in someone else's software. Tell me another vendor who can make that claim.
  • Mathworks, vendors for Matlab, Simulink, and Stateflow. Yes, their prices are high, but they deliver. Their software (almost always) works. And if there's a problem, they fix it. Mathworks currently seems to be working hard to make their system the environment of choice for generating embedded software, and I think there's a good chance they'll succeed.
  • Vesta, vendors of small, nice single-board computers. Their supported computers range from PICs to PowerPCs, their prices are right, and their support tools are great.

I'm sure I must have left someone out. If I left you out, please forgive me. I'll add names as they come to me. Further, if any of you readers want to suggest additions to the list, just let me know. I'm eager to populate it. My intention is to post the list to the Embedded Systems Programming web site, and update it as new data comes in.

Getting onto the new list is, admittedly, hard. You must provide a product that actually works, and does what it's supposed to do. Always. No exceptions. Further, you must provide decent support. By that, I don't mean perpetual phone queues or automated help daemons. I mean real, effective support by an actual human being. Finally, you must provide real, physical manuals, printed on actual paper.

Getting off the list is easy: screw up one time.

For the record, Microsoft's pay-per-incident and warranty support make the list with ease. Their products, sadly, never will. Neither will Mathcad from Mathtools. You will not see them mentioned again, which I'm sure will make the folks at Mathtools heave a great sigh of relief.

I will say no more about the Sony, except this observation: when I recently re-installed Windows 3.11 on my multi-boot Dell, I couldn't help but remind myself that the OS came on three 1.44MB floppy disks. Well okay, it comes on five, but two of them contain libraries of device drivers, and so on. The installation program only used the first three. That's less than 4.5MB of compressed files.

By the same counting methods, the Sony recovery disks, including Windows XP, came on four CD-ROMs-three for the OS and one for the Sony-installed application programs. Obvious conclusion: in the last seven years, Windows has grown by a factor of 560. Think about that for awhile. It puts a whole new spin on Moore's law.

As Alan Keyes would ask, “Does this make sense?”

Jack W. Crenshaw is a senior software engineer at Spectrum-Astro. He is the author of Math Toolkit for Real-Time Programming, from CMP Books. He holds a PhD in physics from Auburn University. Crenshaw enjoys contact and can be reached via e-mail at .

Return to the July 2002 Table of Contents

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.