Flash! The strobe winks in your rear-view mirror while you’re navigating a confused hodge-podge of unfamiliar city streets. A computer transcribes your license plate and sends a citation for running the red light.
In London a similar system checks drivers’ tags against a database. If you haven’t paid the day’s five pound fee for driving in the central part of the city, expect an automated fine. According to the NY Times (online registration required) officials are considering a similar system for Manhattan.
Now 150 Florida defendants in unrelated cases want to see the source code of the delightfully-named Intoxilyzer 5000 that found them at least a bit tipsy. The revelers…ah, defendants, feel they have Sixth Amendment rights to understand how their accuser works. The company that makes the device protects the source as a trade secret, a reasonable and common business practice.
The Amendment reads in its entirety: “In all criminal prosecutions, the accused shall enjoy the right to a speedy and public trial, by an impartial jury of the State and district wherein the crime shall have been committed, which district shall have been previously ascertained by law, and to be informed of the nature and cause of the accusation; to be confronted with the witnesses against him; to have compulsory process for obtaining witnesses in his favor, and to have the Assistance of Counsel for his defense.”
As embedded systems become both the accuser and sometimes the sole witness for a variety of crimes, citizens are complaining that their right to be confronted by their accuser is being abridged. Do we have the right to haul the red-light camera from its designated curb onto the witness stand? Though even Clarence Darrow would elicit nothing but silence from even the most aggressive of questioning, shouldn’t we be able to burrow into the device’s brain for evidence?
Computers have regulated our lives in important ways since the dawn of the programming age. Banking, for instance, is entirely digital and is accepted by pretty much everyone. But monthly reconciliations give us recourse. The rare cases where there’s a bank mistake (I’ve had one in my life) get resolved.
That’s not true when a smart device allegedly catches us in some illegal act. Was the speed radar properly calibrated? Is there any chance the drunk-o-meter’s code could get confused by something unanticipated?
A prosecutor might build a case using complex technology like DNA fingerprinting. Few defendants have the expertise to check or even understand DNA analysis. But law is an open process; the accused can have his own experts verify every bit of the evidence.
When an embedded system makes an accusation things change. The units’ operations are secret. You can’t hire an expert witness to probe the devices’ internals. The crime is long over, so it’s impossible to run a duplicate “analysis” using a different device.
Science fiction author Robert Heinlein proposed the idea of using a certified True Witness to confirm the truth in matters of law. Perhaps we need a similar idea for embedded devices, a sort of independent verification process that assures the units are error-free. Maybe the solution is to escrow the source code, making it available to expert witnesses under an NDA seal, which preserves the trade secret protection while affording transparency for defendants.
Open-source code totally alleviates the problem. But I think it’s wrong for the Feds to force companies to release their code under the GPL.
Though an embedded surveillance system could go a long way to making the roads safer and enforcing a variety of laws, I shudder to think that Americans might be herded into compliance by systems of unknown reliability employing secret algorithms.
What do you think?
Jack G. Ganssle is a lecturer and consultant on embedded development issues. He conducts seminars on embedded systems and helps companies with their embedded challenges. Contact him at . His website is .
I think the copyright and patent laws should be changed as follows: “Any company that sells its devices for use by law enforcement agencies SHALL FORFEIT all copyright and patent protection laws.” This will allow several benefits: (1) The accused shall be able to test the accusor. (2) Will prevent collusion between companies and law enforcement. (3) Will prevent bogus scientific claims — meaing companies will have to “prove” their technology actually works. In all, forfeiting copyright and patent protection is a good thing when your devices are used by law enforcement!
– Steve King
I agree that there needs to be an allowance for legal examination of a piece of technology when it stands as a witness, and even more critical when it serves as the sole witness and accuser. The 6th Amendment, in the clause “to be informed of the nature and cause of the accusation,” currently allows for examination of other forms of technical accusators. Why should schematics and firmware be excluded?
On the other hand, companies have a right to trade secrets, and should not be forced to make their work open-source. However, a special-purpose NDA should allow the accused to have an expert examine the system. If a bug is found, then the company has some explaining to do. If it comes clean, then the company will be sure to put extra points on their marketing brochures.
Point: Technology should not be excluded from the 6th Amendment, ESPECIALLY when technology may be the sole witness and accuser.
– B. Smith
Legal introspection and exposition through code inspection! What a
concept. Actually, the courts have had to deal with software-driven legal problems for many years. Decades ago, my dad had a case against a manufacturer that made material-handling equipment many years back. The material handler, installed in a truck assembly plant, had squashed an employee by loading him to the ceiling (Warning: ceiling and walls are closer than they appear). These cases are usually handled by having experts come into court to explain in detail how the equipment in question operates. The plaintiff's attorney then tries to discredit the system by asking the expert hypothetical questions about possible failure scenarios.
In my own experience with radar detectors operated by police, I've discovered that the weakest link is not the software in the detector but the wetware aiming the device.
– Steve Leibson
“I shudder to think that Americans might be herded into compliance by systems of unknown reliability employing secret algorithms.”
As if this was something new, e.g., the latest instance: isn't it beginning to appear that this is what got us into Iraq?
The processes employed by the state have to be transparent when it comes to actions on its part that threaten the freedom or privilieges of its citizens, period. There is no telling what later rulings could be made based on precedent established in a case like this. That doesn't necessarily mean that source must be made known, but certainly the state should have the burden of somehow establishing the validity of evidence to be used in the exercise of its power over a citizen. If the state cannot do that, it cannot convict, period.
– Rick Schrenker
When I first heard about the Intoxylizer case, I was happy to see that someone recognized the need to check these things, but what if the algorithm is flawless and the analog/chemical analysis portion was out of calibration? Its a legal variation of 'if something is wrong its the softwares fault'. The nuances can be hard even for experts.
I like Heinleins concept because it helps quite a bit, but if we could only get non-geeks to read excellent science fiction writers, we might be better off 🙂
– Pat Thomson
Two minor points:
First, it's not a True Witness, but a Fair Witness in Stranger in a Strange Land.
Second, you'd think the system in London that fines you for not paying the fee would instead simply charge the fee to you account, like SpeedPass does for tolls. Clearly the authorities want the revenues from the fines, rather than simplify life for the drivers.
– Mark Dobrosielski
If you can't verify a statement, it is not a fact but a conjecture. The question becomes how much veracity the courts can attribute to the “expert witness”.
This is reminiscent of the Diebold voting machine problem. Some may say that the benefits of the use of such technologies outweigh the drawbacks, but if you can't quantify the failure rate, how do you know if it helps?
– Bruce Salzman
To quote your article”Though an embedded surveillance system could go a long way to making theroads safer and enforcing a variety of laws, I shudder to think thatAmericans might be herded into compliance by systems of unknownreliability employing secret algorithms. “
The problem here is the gap between the “ideal” world and the actual one.In an ideal world, the embedded systems that for example, accuse me ofspeeding in my car, would operate correctly, be properly calibrated andmaintained and there would be no need to question them. Also in an idealworld, every defendent in a court case has access to all the necessary ordesirable resources (lawyers, expert witnesses, time etc). Neithersituation holds true in our actual world.
This means that since many defendants have few resources or none, theaccuser-systems should be all the better validated, verified andmaintained. It also means on the other hand, that defendants in courtcases need better access to resources regardless of their financialposition.
In the distant past, the veracity of witnesses was always suspect (due tofamily alliances, bribery, naked power etc). Over the years we have gottento the point where the right of a defense attorney to judicial discovery,followed by the cross-examination of a witness is sufficient to determinethe “truth”. Attorneys have gotten much better and the public perceptionof honesty and fairness has gotten much keener. Maybe in the future thereliability of embedded systems that accuse us of traffic violations willbecome so much taken for granted that questioning their veracity will bemostly unnecessary. If we (the embedded systems development community) payenough attention to the Ganssle seminars on “Better Software …” suchshould be the case in the not too distant future. (See how trusting I amin the 'march of technology'?)
– jim scandale
I just read your Drunk Driving article on embedded.com, and came across astatement that I think is looking at things the wrong way. The statementwas:
“But I think it's wrong for the Feds to force companies to release theircode under the GPL.”
I agree completely that the feds shouldn't require arbitrary companies torelease their source. However, I think it would be entirely justified forthe Fed's to make it policy that any device sold to or purchased by anylaw enforcement agency have the source code freely and openly available.
The Fed's shouldn't “force” the companies to release their code. But theycan make it a requirement for any devices purchased for law enforcement use.Then it is left as a choice for the company: if they want to be able to selltheir devices to federally funded agencies, or not. It would be theirchoice to make freely, there is no “force”ing about it.
If the market dictates that they will accept no devices without the sourcecode being available, then companies will comply. But it should be left asa market choice. The only federal requirement would be a Supreme Courtruling to require the codes for equipment to be available. Thenif the enforcement organizations want to use those devices, they willobviously pick only the ones where the code is available.
Incidentally, this should be a requirement for ANY devices bought with ANYpublic money, including voting booths and other equipment driven bysoftware.
– Evan Harris
It would appear there are implications here requiring some sort of 3rd party certification. Should there be something like the TUV/UL for certifying design methodology compliance as well as end product testing? If a device was required to pass a certified and rigorous test schedule, would that certification be enough to allow the results from a certain test event to be validated? As long as the device could be shown to still pass the same test schedule after the event in question, then the evidence should be admissible in a court of law.Incidentally, you have discussed the government's design guidelines for electronic voting machines in the past, as well as some of the issues with certain disturbing anomalies related to voting results from some models. It seems reasonable to expect the design process and product for any electronic technology to be audited according to IEC61508 or some similar standard, with the results being made available as public record. Would this focus on the front end design process be enough to protect citizens from tyranical technology, while protecting trade secrets?
– Joe Whinnery
In my opinion, the best solution is to treat breathalyzers and radar speed guns in the same way that other measurement devices that influence financial transactions (like weight scales and gasoline pumps) are treated.
The first step is to define the applications of concern and to define appropriate performance requirements for these applications. Once requirements are defined, a testing program is developed to verify that devices satisfy these requiremens. Manufacturers submit products to an independent lab that certifies the product meets all applicable requirements. A national data base of type approved devices is maintained so that buyers and private citizens can verify that a device is appropriate for the application it is being used in and that it meets legally defined performance requirements.
A review of a devices schematic and source code is unlikely to shed any meaningful light on the situation. Performance requirements for specific applications and independent verification that devices satisfy these requirements is much more useful.
To my knowledge, NIST's involvement with breathalyzers and radar speed guns is limited to supplying traceable calibration standards, but if there is sufficient concern (as there was about scales and gas pumps), then this kind of program could be instituted for law enforcement devices.
The NIST Law Enforcement Standards group has responsibility over breathalyzers and radar speed guns and that is where I would start if I was really concerned about this subject.
– Phil Ouellette
What most people don't know is that most of the speed lasers and red light cameras are put through extensive testing by an independent test lab that the IACP (International Chiefs of Police) endorse. This gives the courts something unbiased and tangible to stand behind. As for dragging the source code or the device into court, that is a bit much. It is aslo what expert technical witnesses are for. Food for thought … I think we should focus on the added safety that these devices provide and stop looking for excuses to make illegal behavior acceptable.
– Lane Mitcham
This all sounds pretty iffy to me. If I were, for instance, subject to a BAC test, I could fail, having never had a drop of alcohol. I have diabetes and if I'm experiencing ketoacidosis, these tests will see that as alcohol. Like human witnesses, they are unreliable.
I do like Steve King's idea.
– Michael Badillo
What about unethical hackers? Would opening up the source code / schematic make mischief easier for the unethical hackers? For example someone accused of fraud in an electronic transaction.Will the banks be forced to disclose too much? Or will such person be able to switch off the security cameras at a traffic junction, in case they controlled by internet? A NDA is a piece of paper, not foolproof.
– kalpak dabir
I like the thought of requiring disclosure of software used to convict people under the 6th Amendment.
I wonder whether, as has happened in the UK on some of their M motorways, whether a preponderance of “enforcement devices” won't just bring about targetted vandalism of these things?
And, lastly, I wonder whether the rule of law that has served humankind for so long hasn't succeeded precisely because of limits to enforceability. If all laws, like, uh, tax laws, for example, could be enforced perfectly 100% of the time, would this really serve us? With 30,000 pages of tax code, what percentage of people who think they are law abiding citizens are really criminals?
Nice discussion. Thanks, Jack.
– Joe Buczek
I used to design and implement breathalyzers for a living.
Each judicial district has a rigorous testing process for accepting a device and/or modifications to the device. The testing process is generally extensive, and is heavily based on available medical and scientific literature regarding how humans metabolize alcohol and other substances.
As for BAC vs. diabetes, the latest generation of devices—including the I5000— can easily discriminate between alcohol and acetone and other compounds, and thusly can distinguish a true drunk (or other drug abuser) from a diabetic requiring immediate medical attention. Indeed, this is one of the many requirements that most judiciaries impose on devices they purchase.
The ability to distinguish alcohol from other compounds is also necessary to be sure the subject is charged under the appropriate set of statutes, since alcohol and other drugs are generally treated separately.
The question that must first be answered here is whether or not the governing judiciary's acceptance tests are rigorous enough to deal with whatever “special case” the defendant is posing. If they are, then I don't see that the defendant has a case— their accuser is the judiciary in question, not the device.
If disclosure of source code becomes an issue, in similar situations the Courts have compelled source code release under NDA to a third party for analysis. I don't see why that couldn't be done here.
In fact, I hereby request that I be considered for the job.
– Bill Gatliff