Device Drivers for Nonexistent Devices

Procrastination is a bad idea when it comes to developing device drivers. It's better to write as much of the device driver as possible before the hardware is available. Although this can be a daunting task, the payoff can be a shorter overall product development cycle.

Dependencies are an interesting subject. You can be dependent on a variety of chemical substances, some legal and some illegal, but rarely is this type of dependency considered healthy for the dependee. You can also be dependent on people such as a spouse or sibling, which can also be considered unhealthy if taken to an extreme.

We also talk about dependencies in the schedules for our engineering projects. In this situation we are referring to something that must be completed before another step can be started, meaning that the latter task is completely dependent on the output of the former. This procedure can also be considered unhealthy for a project, because it prolongs the time it takes to complete the job, thereby extending the time before it can be delivered to paying customers. Extend this time too much, and the customers will go pay someone else instead. So how can we relieve some of the dependencies regarding our projects? The last time I looked, the Betty Ford center didn't handle an addiction to linear system development. It is up to us to identify areas where work can be efficiently overlapped.

An example of this need for overlap is in the area of developing device drivers. Any system that requires custom hardware will probably need software that is specifically written to operate it. It is almost traditional with such projects to put off the job of creating that driver code until the hardware exists in physical form, which is close to the last step as far as the hardware engineer is concerned.

This linear process of system development is exactly the type of unhealthy dependency I am talking about. The dependency almost becomes an inescapable article of faith"a simple consequence of system development. But there are times when it is possible to break addictions, and this addiction is often one of those cases. Let's take a look at the technical aspects of writing device drivers and see about finding ways to write them without having hardware available.

Traditional implementation technique
The first step is to examine why things are the way they are. Writing device drivers after the hardware is practically done made a lot of sense when the same engineer was doing both jobs. Typically, these engineers were hardware engineers who learned assembly language, but their major job was still more associated with a soldering iron than an editor. Linear development was a consequence of the fact that one person couldn't concentrate on more than one thing at one time.

In most environments, this situation has changed. Engineers are becoming more specialized as each subfield becomes more complex. Hardware designs are often being done at the ASIC level, and entire system designs might be complete and to some extent tested before they ever achieve physical incarnation. Without the intermediate breadboarding step, the time penalty for linear development becomes even more severe.

And the software side of the equation is not getting any simpler either. A device driver used to consist simply of a couple of interface functions that were called directly from the application software. This situation has changed with the requirement for increased multitasking and other operating system services, forcing the device interfaces to adhere to the device driver protocol as dictated by the RTOS.

This increased complexity has typically changed the rules for the upper side of the device driver"the one that interfaces with the application software and the operating system. Also, the increased complexity has sometimes made it necessary to move to higher-level languages such as C, rather than the venerable, traditional assembly language. But opportunity is also offered with this situation. If less of the overall job of writing the device driver is directly involved in controlling the device, then at least that part of the driver can be written without access to actual hardware. In the following sections, I will discuss some of the techniques I have used to do at least that much, and in some cases go beyond.

Stubbing out device interface code
The typical RTOS device driver interface includes a set of six functions. Notice that I am referring to a physical device at this point, not a file system that needs to be able to maintain a coherent structure to the data on the physical device. File systems are usually implemented as a separate layer that uses the physical device driver, rather than being part of the driver.

The first step to writing our device driver involves handling the bookkeeping required for the top end of the driver. As I said, this job has gotten more complex over the years as our software environments have grown in capability. The good news is that little of this increased complexity is dependent directly on the hardware. At a minimum, this procedure will represent the work that can always be done without having physical access to the I/O device itself. The entry points and some discussion of the hardware dependence of each is shown in Figure 1 and detailed below. Create is an entry point typically used to set up the data structures and communication paths that the driver will use. There is usually a set of boilerplate code that sets up the interface between the RTOS and the driver, at the end of which the device is marked as either successfully installed or not.

Create is also where initial readiness of the hardware itself is often tested. It is relatively trivial to stub out this code, reporting either success or failure. Note that it is important to implement both of these possibilities, because either can (and will) happen in real hardware at some point. Remember, the goal is to get as far as possible. Open is another entry point that is often more about the RTOS interface than the device interface. Open typically sets up the data structures that a particular task or function will use to communicate with a physical device or a logical device via the physical device.

Again, most of this code can be written ahead of time. If the device driver is a true one (a network device driver rather than an upper network protocol layer), then the device-dependent portion is probably minimal. Either way, this situation can also be stubbed out by a success/failure return.

Close is the last of our primarily RTOS-centric functions. Shutting down a device is much more about flushing and deallocating data paths than about controlling the hardware. In fact, getting back a bad status from a close routine is a fairly unusual situation, so much so that it may not even be worth testing a failure mode.

With read, we are getting to the heart of the matter. A read function call usually has all the RTOS overhead taken care of ahead of time, and it is typically about getting data from the device in as timely a manner as possible. If the driver design is unbuffered, there is little you can do with this entry point other than pass along dummy data and varied status responses for testing purposes.

Buffered devices provide much more opportunity to do work ahead of time. One of my favorite tricks is to completely implement the read function without the corresponding interrupt service routine for initial testing. Once that path is wrung out, I might add code to a system timer tick interrupt service routine (ISR) that will slowly accumulate data in the read buffers. This technique takes the programmer a long way down the path of debugging the code, even to the point of debugging data access conflicts between the ISR and the non-interrupt device driver code.

With write, we are firmly on the system-dependent side with this entry point. Write should be handled much the same way as the read interface, taking the code as far as is possible before running into the actual hardware. Timer ticks also work here, acting like a clogged output device that can test the limits of the buffering in the system. Remember, the point of debugging is to find the places where the code doesn't work, not to show off the places where it does.

The control interface is the catch-all, usually an ugly reminder of all the imperfect details of working with real hardware. Control is where you do the things that don't fit well with the logical abstractions of the other entry points, so there are usually a polyglot of functions that must be performed. Many of these functions will be heavily hardware-related, but some will probably also be back doors, where the application code and the device driver will be passing information back and forth.

Implement this entry point as soon as necessary, but use it as a last resort as far as the design of your device driver is concerned. An application and a device driver can easily become too dependent on this type of interface, making it harder for either to transport easily to another system.

Plan on the possibility that there will probably be several hardware dependencies here, but don't expect to find all of them before the hardware itself appears. Control is the entry point you may be using to fix the problems you find in the actual hardware that were not evident in the specification.

We have made a good start on our device driver, but the hardware may still not exist. Is it a good time for a little vacation for our device driver programmer? Maybe time to surf the Web a little or check out the latest games? This time might have been playtime in the past, but not these days. Our demanding project schedules require as much work as possible to be done as early as possible, so we need to look at ways to do as much of the remaining code as possible.

Cross-development
The option of cross-development is always worth exploring. If the system you are developing for isn't available, try to set up an environment that is as similar as possible to the development environment you need, and develop most of the software there. It often pays to have that system be as readily available as possible. I have worked on development systems in the past that have to be timeshared by several engineers, and it seems like useful work has barely begun before your timeslot expires.

Fortunately, engineers these days typically have a wealth of computer power at their disposal. The PC that is in front of practically every engineer can provide a mature, accessible development platform for many environments, including RTOS device drivers.

The key question is whether the device you are developing for has a close relative available on a PC platform. Obviously, the best case occurs when exactly the same chip is available, but if that isn't the case, a similar one can usually be found. For example, I once did a set of device drivers for a print server that consisted of a 68000-based single-board computer with a proprietary RTOS. The schedule was ambitious (are there any that aren't these days?), and the hardware was late in coming. The serial and parallel interfaces for the 68K were based on standard PC interfaces, though, so I developed and debugged the drivers on my PC. The total time it took to bring them up on the target hardware was about a day and a half.

But specialized hardware can be a real problem. I also once developed a driver for an ISDN interface chip that was very complex. Luckily, the company had anticipated the complexity of that effort and had built a special daughtercard for an off-the-shelf VME card. The combination of the VME board and the daughtercard gave me an environment similar to the anticipated custom board, allowing us to uncover some interesting anomalies in the ISDN chipset in a stable hardware environment.

The key point to realize in these situations is that there will usually be a period of exploration for the programmer who is working with an unfamiliar I/O device. If the only information the programmer has to work with is a paper description, there will be a period of time where the understanding (or lack thereof) of that description will be tested against the real chip.

If that period of examination is undertaken in combination with an unstable hardware platform or an unstable programming platform, then the time it takes the programmer to get comfortable will inevitably be longer. If that takes place at the end of the project on the real hardware, then the project may well slip its schedule. One way around this problem is to get the programmer a more stable platform earlier in the development cycle. In my experience, it is well worth the effort to build a simple platform for initial testing based on a PC or other stable environment. The cost of this effort can provide effective insurance against a much more expensive slip late in the project.

Device simulation
But what happens if you can't get a similar enough standard platform? This situation usually happens when you are working at either the high or the low end of the computing spectrum. Low-end 8-bit CPUs are turning into ASIC libraries these days and getting modified or integrated with I/O devices to create the long-promised ýsystem on a chip.ý These systems provide maximum cost reduction for systems with high volumes, but they can be a real pain to write software for, due to the fact that they won't physically exist until near the end of the project. And even then, it is often impossible to effectively use an emulator or logic analyzer to debug your software. The same can hold true on the upper end. If you are working with a new CPU, there is usually a lot of system software to be written. Often, it is desirable to have this software execute on a simulation of the CPU to test it before the first silicon is cut.

The only real answer in either of these cases is a simulated hardware environment similar to the one shown in Figure 2. I have not had good luck with these in the past, but it appears that they might be finally coming of age. I wrote an article recently ("Trends in Hardware/Software Codesign," Embedded Systems Programming, January 1996, p. 36) looking at the state of simulation environments, and the fruits of some of the mergers described in that article are starting to be available.

In general, this option has been getting better over the last few years. Simulated environments exist for most of the low-end CPUs and are often considered to be part of the compiler development system. Unfortunately, the weakest part of these simulators is often their support of specialized I/O devices.

The solution to this is a link into an executable model of the chip in question. These models exist for standard devices, but they are often proprietary and expensive. There has been a movement in the hardware community lately to accumulate a series of these models under a public domain code base, modeled after the GNU software effort, but until this accumulation happens this option will be primarily available to those customers of EDA companies who are designing specialized systems and have relatively deep pockets.

But the advantages of this type of simulation environment are undeniable. With them the device driver programmer has complete access to internal states of the device being programmed. This accessibility avoids those frustrating problems where the I/O chip goes off the deep end, providing little or no information about what caused the situation or what it is doing internally. The combination of visibility and early access has been enough to keep interest in simulation alive even when the current implementations haven't lived up to the promise. If the anticipated combination of fast simulation platforms, readily available high-quality simulations, and integration with software-centric debug environments becomes a reality, then simulation may one day become the standard way of writing system software.

Prozac Time
In some ways, this topic is very depressing. Device drivers are a necessity for practically every embedded system development, so you would think that by this point there would be better support for doing the work. There certainly have been serious advancements in both the hardware and software realms over the last few years, but those two environments have been maturing in a vertical fashion"meeting the narrow needs of each specific field without integrating into an overall system development toolset.

The good news is that this situation is seen by some as the next frontier in the embedded system world. Large EDA companies are looking to subsume software into the hardware design systems, so they are merging with the smaller companies that have knowledge about how embedded software development works. Until this happens, though, it is primarily up to the ingenuity of individual device driver programmers to invent ways to get their jobs done earlier in the project cycle. To accomplish this feat, device driver programmers must have a complete understanding of both the hardware options that are available and the capabilities of their software tools.

This situation also means that these programmers should be more demanding in allocating resources for new projects. Force the issue of moving from assembly language to C. Get time added to the schedule to build prototypes dedicated to early experimentation. Device drivers are in many ways the critical foundation of embedded systems, so the people responsible for creating them should not have to make do with arbitrarily limited resources. But once you have those resources, it is up to the programmer to use them inventively. Any field, such as device drivers, that cuts across specialties must always be looking for innovative ways to solve problems. Don't settle for doing things the way they've always been done.

Larry Mittag is a contributing editor for Embedded Systems Programming. He is also the lead consultant for Mittag Enterprises, a consulting firm that specializes in embedded systems software and communications applications. Mittag has over 18 years experience in the embedded systems industry, and holds degrees in physics and secondary education.