Design user interfaces for cooperating devicesIt's hard enough to design a user interface for a standalone product. But when you have two devices that must work in concert, the level of difficulty multiplies.
Making the user interface for one device easy, slick, fun, and fast is a challenge. If you have multiple devices and they need to cooperate, the challenge increases dramatically. As wired and wireless communications hardware gets cheaper, the design opportunities for communicating devices become more common. For example, a Java applet on my cell phone can control my home security system. My digital video recorder (DVR) uses IR to change channels on the set-top box from the satellite TV company. The controls on my car's dashboard can make a phone call using the phone in my pocket. Each pair of devices provides the user with two user interfaces, and those user interfaces must combine to provide a rational final system.
Early adopters will be happy to fiddle with settings that ensure that all of their gadgets talk to each other. More mainstream users will want to minimize the number of times they have to think about the fact that they're controlling more than one device. The classic living room problem is having three remote controls, each of which can control volume, yet only one works on each system. This issue replicates itself in many other areas where a combination of devices must cooperate.
These scenarios provide the user with two different user interfaces on two different devices. The designer must try to ensure that the two, or possibly more, devices present a coherent final system to the user. If you're designing both devices, you want to make the feel of both devices similar.
I recently worked on a design that had two separate GUIsone full-size touchscreen and a smaller "helper" screen for occasional use. The smaller screen needed to look consistent with the full-size screen. Simply scrolling the smaller display to provide all of the information on the larger display would have been a lazy choice by the designer. Careful choices had to be made as to what information normally available on the full display would be left out of the smaller display. Using a similar font and background colors meant that the overall feel of the small display was consistent with the larger display, but the fact that the display was physically smaller made it obvious to the user that this wasn't the primary interface to the device.
Another case of designing a device with two interfaces is a TV with a remote control. If there are buttons on both the TV and the remote, it's trivial for the designers to choose the same symbols and icons on both. A universal remote must allow for variations across a range of TVs. The designer will be faced with compromises, such as having extra buttons for advanced features that provide no function on simpler TVs that lack those features. Alternatively, a minimal universal remote may miss out on some features that are available on the more sophisticated TV's remote.
In many cases, one of the devices is under your control, but the second device already exists. For example, if you are designing a DVR then, you may have to assume that the user already owns a set-top box. Worse yet, you have to assume that they could have one of many set-top boxes, depending on their service provider. In fact, the set-top box they'll be using by the time they purchase your product may not have even been designed yet, so you can not even be sure you'll get to test the combination that will be in the user's home.
If there were only one set-top box to integrate into your product, you could model the DVR's interface on the set-top box, to provide a harmonious combination. Another option might be to minimize the DVR's feature set, and leave as much functionality as possible on the set-top box, but this isn't a satisfactory approach either. The DVR likely has an analog tuner for when a set-top box isn't used, so it must support all functions in that mode.
Many DVR's use an IR blaster to control the set-top box, as shown in Figure 1.
The blaster generates the same IR signals that would be generated by the set-top box's remote control. For example the DVR would use the IR blaster to transmit the required channel number when the time had arrived to record a show. In principle, the idea of one device changing the channel on another is simple. The devil is often in the details. What if the DVR supports channels numbered from 1 to 999, then someone releases a set-top box utilizing a four digit channel number? It's these sorts of mismatches that make compatibility across a range of devices difficult.
It's informative to look at the integration between different PC applications to see some parallels with the model you could implement on an embedded device. For example, many code editors let you check files in and out of a source-code control tool. Usually the check-in, check-out, and get features are provided as buttons in the editor. This interface is a subset of the full version-control interface, because the more advanced features, such as getting an older version or labeled version, aren't available. The other limitation here is that if something goes wrong, the quality of the error messages displayed on the secondary interface tends to be weaker than the messages or warnings seen if using the primary tool.
In an embedded device, it's also important to consider warning messages. If the primary interface has a number of dedicated warning LEDs, and they aren't available on the secondary device, you must consider if that information can be relayed by some alternative means.
Primary and secondary
In some cases, one interface will provide all of the functionality available on the system, and the second interface will only provide a subset. For example, a Bluetooth-enabled watch from Fossil provides a secondary interface to a cell phone. The watch displays caller ID and lets the user hang up a call, but doesn't attempt to provide the full functionality available on the cell phone.
In the evolution of TV remote controls, the remote was previously a secondary interface that wouldn't include the rarely used features, such as tuning the set. Recent TVs tend to put all of the functionality into the remote, sometimes only having minimal functionality available on the TV itself.
When designing a secondary interface, consider whether you want all functionality in that interface or just a subset. Implementing a subset simplifies the secondary interface for both the users and designers. Users will understand why one of the interfaces provides fewer features. It may be because it's physically smaller or because it's expected to be used less often, or because the natural home of certain features is on the other interface.
Ease of use
In some cases, the mixed user interface depends heavily on a particular feature of the secondary interface. I worked on one system that communicated via mobile phone text messages. It required the user to dial a number which appeared in a text message on their phone. On most Motorola models at the time, the user could dial the number that appeared in one key-stroke. However Nokia and other models required the user to select a "Use Number" menu option first and then select the number. Because this feature was used often, it became quite unwieldy on all but a few phone models. Some phones don't let you use the number at all, requiring the user to remember the number and dial it from scratch.
In those cases, the designers need to think hard about claiming compatibility with a wide range devices. If it's going to be difficult to use on partner devices, it may be better to not claim compatibility at all, or find an alternative communications path for those devices.
Many PDAs, mobile phones, and other devices synchronize with the PC. Some have dedicated applications to mirror the data on the PDA. Others synchronize with applications commonly found on a PC, such as Microsoft Outlook. Some phones let you synchronize the appointments on the phone's calendar to appointments on Outlook's calendar, as shown in Figure 2.
One challenge in this type of synchronization is deciding whether the appointments on the PC or the phone should take precedence. If a particular appointment is on the phone, but not on the PC, it gets copied to the PC. If it's on the PC, but not on the phone, it gets copied to the phone. However, in the case of the user deleting an appointment, this arrangement might cause the deleted appointment to reappear after a synchronization.
If an appointment is on both the PC and the phone, the user might edit the description in one place or the other. Ideally, the change is date-stamped and the synchronization software takes the latest version. However, if you're synchronizing with a product such as Outlook, you may find that the kind of history information you require isn't available. That's because the original Outlook wasn't written to cooperate with external devices. Similar issues arise with any standalone application. This is where writing a dedicated application to cooperate with the phone has advantages; that application can be written to handle all the synchronization issues.
Other compatibility issues arise when trying to integrate with preexisting applications. For example, many Nokia phones let you set a reminder for an appointment in the calendar. Similarly, Outlook can remind you before an appointment. It makes sense to transfer the data related to the reminder along with the appointment. However, the nature of a reminder on the PC is different than a reminder on the phone. The phone will ring to get your attention, because it's assumed that you aren't looking at the phone when the reminder is triggered. On the PC, the reminder is more subtle, as it assumes you're already at the PC. A brief beep and a pop-up are what the user will see/hear.
It gets interesting when you set an appointment in Outlook that lasts all day, such as a birthday. When setting the "Wife's Birthday" appointment, if you set no time, Outlook assumes that it's an all-day appointment, meaning 24 hours (from midnight to midnight). The default reminder will prod you 10 minutes before the appointment starts, which is 10 minutes before midnight, in the case of an all-day appointment. A quick synchronization and the same birthday is recorded on the phone. So as a pre-birthday treat, my wife is awoken by my phone 10 minutes before midnight on the night before her birthday. Because the reminder doesn't stop until you press a button on the phone, she gets treated to hearing me cursing as I fumble around the bedroom looking for the phone and throwing it out the window.
On my PC, the reminder would have been sitting there waiting for me the first time I choose to use the PC that day. The reminder on the phone is much more intrusive. This is a danger when taking data from one context and transposing it into another.
The first problem described was an issue of data integritywe tried to establish which version of the appointment data was more recent. This is a technical problem and amenable to technical solutions. The second problem, mapping reminders from one system to the other, isn't technical but is related to the usage style of each device. These issues can be much harder to solve, because there's not always a clear right and wrong answer. The synchronization utility could query the user as to whether he wants to keep the reminder settings that were set on the PC or the one set on the PDA. But if the user has to be asked this question for each differing appointment, it could become tedious.
When two devices are communicating, it's often necessary for one device to be aware of the other's state. For example, if I'm using a TV with a set-top box, some of the commands to the set-top box only make sense if the TV is turned on. However the box may have no way of knowing the TV's state. Similarly a DVR may be set to record a program at a certain time, but it may depend on the set-top box being turned on, and possibly depend on the set-top box already being set to the correct channel. If the communications from the DVR to the set-box is a one way link, such as the IR blaster discussed earlier, there's no way for the DVR to detect the set-top box's state and rectify it or report the problem to the user.
Other possible states for the set-top box could cause problems. Say the user enters a TV-guide mode on the box. The next number received by the set-top box will be interpreted as a channel to display on the guide. The TV will display the day's programs but won't change the channel to show the programs. If the user leaves the box in this state, the DVR won't be aware of it. Later, when a timer tells the DVR to start recording, it'll send the channel number to the set-top box. However, this will only display that channel's guide and it'll fail to record the show.
If your design suffers from this type of state-synchronization problem, one approach is to always reset the secondary device to a known state before issuing the required command. For example, a command to exit the guide could be sent before any channel-change command is sent. In the cases where the set-top box isn't in the guide, the exit-guide command does nothing. But if the user was in the guide, the device has been restored to a known state, where the channel-change commands function as expected. Alternatively, the set-top box could be turned off and then on using the IR blaster, which would restore the set-top box to a known starting point.
The state synchronization problem is a technical issue. A technical solution may be feasible, such as replacing a one-way communications path with a two-way path. In that case, the issue might be completely hidden from the user. As in the appointment synchronization example, the usability issues can be harder to solve. Consider the case where the user is surfing through the TV guide on the set-top box when the DVR decides to record a program. The DVR will try to control the set-top box, interrupting the user. An integrated system would be capable of displaying a message asking the user whether he wants to continue his current activity or start the recording. Such a solution is impossible if the DVR doesn't know the set-top box's current state.
If devices that cooperate with PC applications are of interest to you, it's well worth reading Designing from Both Sides of the Screen.1 This book investigates many of the challenges involved in writting an application that must work on a PC as well as a PDA.
If your system works with many partner devices, the user may have to configure your system so that it can communicate with whichever partner device is in use. Consider the case where a serial link can be used to connect to a cell phone. The first device must know what RS232 settings to use to communicate. Many cell phone makers sell cables that allow the serial port of the cell phone to be brought out to a DB9 connector that can then allow the phone to be used as a modem or to send text messages.
Speed and handshaking settings may vary from phone to phone so the primary device must be able to configure the port. One approach is to let the user enter the Baud rate, parity options, and so forth, to fully configure the serial port. This manual configuration might alienate novice users who have to scour their user manuals looking for serial port setting.
A second approach is to let the user enter the phone's make and model, and the primary device could then derive the serial port settings from an internal database. This multiple choice approach is easier for the user but may fail in the cases where the table of phone models isn't complete. If a new phone comes out next year, it won't be in the table. If the primary device is Internet-enabled, this problem could be solved by periodically downloading a new list of compatible devices.
The third option is automatic identification. The primary device can attempt a number of settings until one works and then issue commands that ask the cell phone to identify itself.
There are other scenarios where these three approaches apply. In some cases, more than one option is made available, so if the multiple choice option fails because a device isn't in the list, the user could be offered the option of manually configuring the interface.
The tortoise and the hare
If the device interacting with the user must communicate with a second device, this always introduces some delay. In many cases the communications is so fast compared with the speed of the human interaction that the user doesn't notice the delay. But there are exceptions. If the communications is over the Internet, the speed might vary considerably. In the DVR example, where an IR blaster is used, these devices send each digit of the channel number at the same speed as a human would type them, so a three-digit channel number can take over a second. This delay isn't a problem if it happens when no one is home and the DVR is preparing to record a program. However, if the user is flicking through the channels using the DVR's remote control, every command will go from the remote to the DVR and then be relayed to the set-top box. There will be a noticeable pause between the user changing the channel and the new channel appearing on the TV.
If data is being presented graphically and in real time, communication delays and throughput issues can limit the functionality. In hospital intensive-care units, it's common to connect several patient monitors to a central station to display gathered data. In some cases, the primary device can display a graph of the patient's vital signs as a graph. However, the throughput to the central monitor might not allow for a smooth time plot. Graphs that have samples spaced by more than 200 ms start to look jerky. The secondary device may be able to show a single bar moving up and down, which doesn't require updates as often as a time plot. If you're designing for a scenario like this and the secondary device needs to provide the full interface available on the primary device, be sure that the communications link is fast enough to give the secondary access to data at the same sample rate as the primary interface.
Having a graphical interface can allow much larger variation in the interface being presented to the user. Some universal remote controls have a graphical display that can present an image that looks like the model of the remote being emulated. This means that all names and symbols can match the terminology and iconology used by the original remote control.
PictBridge is a communications technology that enables digital photographs to be printed directly from a camera over a USB link to a printer. This technology takes advantage of the fact that all PictBridge-compatible cameras have an LCD display, which enables the user to navigate a set of menus on the camera. While the range of cameras that communicate with the printer is large, the common factor is that they can all provide menus in their user interface.
Graphics can allow the displayed UI elements to be defined by either of the devices involved in the communications. In a menu-driven system, the set of menu items might be predefined, or they might be generated at run-time by one device and sent to the other device for display. This can raise interesting issues if the UI must function in a number of languages other than English. It will be important that the language can be specified on the protocol between the two devices, as the menu items will need to be generated in the appropriate language.
HTML allows a browser-capable device to display an interface entirely controlled by the remote device, without any prior device integration required. However HTML interfaces have other problems, which I discussed in more detail in an earlier article.2
Using graphics solves some problems, but it can generate new ones. If the user interface is generated on one device for display on another, there may be many possible variations in resolution and color depth. These variations could mean that the user experience varies greatly from one device to the next. The protocol between the devices may involve a negotiation based on the display's capabilities, and possibly this negotiation will refuse to operate with a device that doesn't have some minimum capability.
If the interoperability problems in your industry are a real challenge, it may be worth considering whether there's a market for an integrated solution. A set-top box with a built-in DVR resolves many of the issues described here. Of course, in many scenarios it's not possible to provide an end-to-end solution.
Niall Murphy has been designing user interfaces for over 12 years. He is the author of Front Panel: Designing Software for Embedded User Interfaces. Murphy teaches and consults on building better UIs. He welcomes feedback and can be reached at email@example.com.
1. Isaacs, Ellen and Alan Walendowski. Designing from Both Sides of the Screen. New Riders Publishing, Indianapolis, Indiana, 2002.