Advertisement

DavidMCummings

image
Executive Vice President

David M. Cummings is the Executive Vice President of the Kelly Technology Group in Santa Barbara, CA. He has over 35 years of experience in the design and implementation of software systems, many of which are embedded systems. Nine of those years were spent at the Jet Propulsion Laboratory, where he designed and implemented flight software for the Mars Pathfinder spacecraft. He holds a bachelor's degree from Harvard University, and a master's degree and a Ph.D. from UCLA. For more information, please see www.kellytechnologygroup.com.

DavidMCummings

's contributions
Articles
Comments
    • Update: Dr. Koopman’s recent testimony on unintended acceleration in a class action case against Ford was thrown out by a federal judge on 3/26/2018. This is the case that Dr. Koopman refers to in his second paragraph. Because a number of readers expressed interest in my Embedded.com articles about the 2013 Toyota trial and in this response from Dr. Koopman, I thought those readers would be interested to know that it is now public information that the judge presiding over the trial in the recent Ford case to which Dr. Koopman refers found Dr. Koopman’s testimony on unintended acceleration in that case so questionable that he ruled that his testimony be excluded from the trial. Here are excerpts from the judge’s comments (“ETC” is “electronic throttle control”): “In his lengthy reports, consisting of his primary report and his supplemental reports concerning testing and software analysis, Dr. Koopman concludes that Ford’s Gen II ETC system is prone to allowing unintended acceleration due to the design defect alleged by Plaintiffs... Though he summarizes and criticizes Ford’s design, development, and implementation of its Gen II ETC system, his conclusion that the system is defective rests primarily on his testing... Dr. Koopman admits that in many of his tests, he injected faults with voltages designed to produce pedal angles consistent with acceleration... The testing was an artificial demonstration that essentially mimicked intentional acceleration... [H]is testing method requires multiple steps: injecting in two or more sensors, then clearing inputs repeatedly in a sequence that has no “real-world” support... [T]here are profound inconsistencies between the general complaints and his opinions... Ultimately, Dr. Koopman’s testing fails to serve as a reliable basis for his opinions regarding the design defects alleged by Plaintiffs in this case. Therefore, the Court GRANTS Ford’s motion to exclude his testimony.” [See pages 14 – 20 of https://tinyurl.com/y8n2qaqk]

    • Thank you for your comment, and thank you as well to Andrew Banks for his earlier comments. I agree with all your points. And regarding your last paragraph, I have seen two different MISRA C tools greatly overinflate the number of violations for the reasons you describe. Unfortunately, there are members of our profession who make a business out of finding simplistic ways to evaluate other people’s software using metrics such as counting MISRA violations, counting global variables, measuring cyclomatic complexity, etc. Many developers of real-world systems recognize that this is overly simplistic and that these evaluations are largely without merit, consistent with the points you make regarding counting MISRA violations. In addition, there is objective academic research based on empirical evidence that debunks these simplistic metric-based evaluations. Nonetheless, some members of our profession continue to use these simplistic and largely meaningless evaluations to further their own business needs, not only as expert witnesses but also as consultants to industry and government. I spent many years as a government contractor, including 9+ years at NASA’s Jet Propulsion Laboratory, and on a number of projects I saw non-technical bureaucrats and managers hire such people based on the false belief that they had a magic bullet (a “recipe” as these two experts told the jury) for ensuring high quality software. Many experienced developers know that there is no such magic bullet or recipe, but non-technical bureaucrats, managers, judges and juries can be easily misled. So, although the vendors whose tools enable such metrics undoubtedly have good intentions, and although the tools themselves can be helpful to developers who know how to use them properly, unfortunately these same tools are misused by some members of our profession to serve their business needs rather than using them to serve the real technical needs for which they were intended.

    • You left out the second half of Dr. Hatton’s sentence. In its entirety, that sentence and the following sentence, which together complete Dr. Hatton’s paragraph, read: “Although hard real-time systems and scientific subroutine libraries are not the same beast, there is precious little sharing of good experimental data and analysis in any computationally reproducible form on which we could build. It could be argued that there are better, more modern alternatives, in that they make more sense to an experienced engineer, but where is the quantitative evidence that they are ‘better’?” These sentences from Dr. Hatton do not change my point that Dr. Hatton’s academic opinions support this statement from ArtGoldste: “A second disturbing aspect is the way in which Dr. Koopman has selected himself as the arbiter and spokesperson for the academic community when he says ‘“academic standard is there should be zero” global variables’.” Dr. Hatton is a member of the academic community with many publications on software safety, and his opinion about global variables is clearly at odds with Dr. Koopman’s. That was my point. The entirety of Dr. Hatton's paper from which I quoted, which specifically discusses the Toyota trial, makes it clear that his opinion about global variables is at odds with Dr. Koopman’s. Furthermore, in the second Hatton reference that I cited, Dr. Hatton characterizes as “points of folklore” with “no supporting statistically robust evidence” the plaintiffs’ contention in this trial that global variables are associated with defects. This, too, makes it clear that his opinion is at odds with Dr. Koopman’s. Dr. Hatton’s opinions in both of these references support the above statement from ArtGoldste. Your partial quote of just one sentence from Dr. Hatton’s paper does not change that. Nor does your quote from Donald Knuth.

    • Regarding your last paragraph: Yes, my article is about expert testimony at trial, so in that sense it is “’legal’/law focused” as you say. Regarding your first paragraph: When talking about the Obermaisser paper and dangerous faults, Dr. Koopman did not say “potentially dangerous” as you imply. Rather, he told the non-technical jury that Obermaisser said 2 percent of failures are “dangerous” whereas Obermaisser said “arbitrary.” Dr. Koopman misquoted Obermaisser, which I think is misleading. I believe it is crucial that technical experts not misrepresent facts to a jury as Dr. Koopman did. Dr. Koopman then calculated for the jury, based on Obermaisser’s 2 percent number, that a “dangerous” failure would be expected in the fleet of Toyotas every week or two, even though Obermaisser gave no statistics about “dangerous” failures. Then Dr. Koopman went even further and told the jury that, based on Obermaisser’s 2 percent number, an unintended acceleration event would be expected in the fleet of Toyotas every week or two, despite the fact that Obermaisser said nothing about unintended acceleration, or acceleration of any sort. To me this is also misleading. Those are just a few examples. Dr. Koopman also told the jury that Toyota’s code “has far, far too many bugs” without seeing one line of that code. He also told the jury that “there is no reason” for Toyota’s use of global variables even though he did not see Toyota’s code, and even though his own research team’s Ballista software contains explicit comments describing reasons to use global variables. To me, these are other examples of statements that are misleading at best. (My articles provide additional examples.) Regarding your second paragraph: My point was not to simply compare the experts’ software with Toyota’s software. Rather, my point is that the experts’ software shows that they do not practice what they preach about global variables and the use of MISRA C as a predictor of bugs.

    • I agree with your point that Dr. Koopman’s testimony is troubling and raises important ethical issues. And specifically, with respect to your questioning of the appropriateness of Dr. Koopman having “selected himself as the arbiter and spokesperson for the academic community” regarding global variables, I’d like to point out the following: Dr. Les Hatton, a well respected researcher in software safety and author of the well-known book “Safer C,” has not been able to find any statistically significant relationship between the use of global variables and software defects. When discussing the evidence presented at this specific trial, he says: “For example, the use of global variables is an important issue in these reports as a dangerous practice. It is likely that many embedded system engineers would agree, but how dangerous are they and in what context are they dangerous? Everybody will have an example which may have affected them but this does not constitute a quantitative predictive system. Tim Hopkins and I checked this in the NAG library (Hopkins and Hatton (2008)), and there was no statistically significant relationship between the presence of global variables and the presence of defect after 25 years of use.” See: http://www.leshatton.org/Documents/SSS17_Hatton_11-Oct-2016.pdf Dr. Hatton also refers to the plaintiffs’ contention in this trial that global variables are associated with defects, and the plaintiffs’ contention that high cyclomatic complexity and other “measures” are associated with defects, as “points of folklore” with “no supporting statistically robust evidence.” See: http://www.leshatton.org/Documents/SSS_09-Feb-2017.pdf Thus, Dr. Koopman’s testimony to the jury about global variables, as well as his testimony about other supposed “measures” of software quality such as cyclomatic complexity, appears to be contradicted by Dr. Hatton’s research.

    • It would be wonderful if, as you say, these kinds of trials decreased in frequency due to good development processes, increased knowledge, use of best practices, etc. However, that assumes that plaintiffs (and their software experts) behave in good faith, and only file lawsuits accusing software of causing accidents if they have legitimate reason to believe the software was indeed to blame. This trial shows that in fact this is not the case. That’s a key point. The first software expert, Dr. Koopman, told the jury that Toyota’s software was of low quality even though, as I show in the article, his evidence was misleading at best, including misrepresenting facts like changing the wording of a research paper (and making many other questionable assertions). The second software expert, Mr. Barr, also criticized Toyota’s software quality with misleading testimony, as I show. In addition, as I discuss in your reference [1], he claimed to have found that more likely than not Toyota’s software caused the accident (more likely than not is the plaintiffs’ burden of proof). As I show in that reference, his causation theory assumed multiple failures, but he provided no evidence of any specific bugs directly causing any of those failures, let alone the multiple failures assumed by his theory that had to occur together. Furthermore, as I show, he presented misleading slides to the jury, including a slide that conveyed that he had found an actual occurrence in the code of stack overflow, when he found no such occurrence. Thus, unless we find a way to prevent highly questionable testimony from software experts such as the testimony at this trial, I’m afraid that indeed it is the case that “as we become increasingly reliant on embedded software in our daily lives … our legal system is likely to face this issue with increasing frequency.” Sadly, the actual quality of the software at issue is largely irrelevant to testimony such as the testimony presented at this trial.

    • I’m afraid you have been misled by Dr. Koopman. The note you quote from the Ballista website was not in the version of the website I accessed when I wrote my original IEEE articles, which you can see in Footnote 1 here: https://tinyurl.com/yajw76tx. That note was also not in the latest version of the Ballista website archived on the Wayback Machine. Up until this recent addition by Dr. Koopman (perhaps in response to my articles?), there were no such disclaimers on his Ballista website. Thank you for bringing it to my attention. My point regarding Ballista is that Dr. Koopman told the jury that “global variables are evil” and the “academic standard is there should be zero” global variables, but in his own academic code he does not practice what he preaches. Even if, as he is now claiming for the first time on his website, the Ballista code was written by students who are still learning how to create software systems, then why didn’t he provide the right guidance to these students, especially on an issue as important to software quality as he says this is? And of course this is just one of the many questionable aspects of his testimony, including: 1. He told the jury that Toyota “has far, far too many bugs,” and “there is no reason” for Toyota’s use of global variables, even though he did not see one line of Toyota’s source code. 2. He told the jury that for every 30 MISRA C violations, one can expect an average of 1 major bug and 3 minor bugs in the code. In a recent comment on my Embedded.com article, the current Chairman of the MISRA C Working Group said: “This is a metric that the MISRA C Working Group do not recognise, and certainly do not endorse!” 3. He changed the wording of a research paper in order to justify his testimony to the jury that the fleet of Toyotas would be expected to experience an unintended acceleration event roughly every week or two. The actual wording of the research paper does not support the conclusion he presented to the jury.

    • Thank you for your feedback. There are a number of free tools available for static code analysis. Verbose compiler warnings can also be useful. I used a free version of a tool from Gimpel Software for the MISRA C analysis described in this article. However, please understand that no guidelines or tools, including MISRA C, can ensure code quality. I have used static code analyzers and many other tools with success, but not in the simplistic way the two plaintiffs’ experts described. I believe that those experts do a disservice to the embedded systems community (including hobbyists) by saying that MISRA provides a “recipe” for ensuring software safety, and by conveying the impression that if you follow MISRA C your code will be high quality, and if you don’t, it will be low quality. That is just not true. In fact, research has shown that correcting every MISRA C violation can actually make code worse, which the two plaintiffs’ experts apparently do not understand. They gave the jury the impression that the goal should be zero MISRA C violations. This is contradicted by research and by conventional wisdom. Chris Hills, a member of the MISRA C Working Group, has said: “Anyone who stipulates 100% MISRA-C coverage with no deviations does not understand what they are asking for.” He also says: “A ‘tick box’ culture to implementing MISRA-C has developed. As well as giving the programming team many problems, it can also produce horrendous source code.” In addition, I’ve seen different MISRA C tools produce very different results for the exact same code. I’ve also seen replication of single violations reported by at least two popular MISRA C tools, significantly overinflating the actual number of violations. Thus, it is not clear that relying on MISRA C, without applying a very strong filter and/or using other tools as well, is useful. In summary, use static analysis tools, but be sure to use prudence in interpreting the results. Unfortunately, there is no silver bullet.

    • Thank you for your comment. As to your question, there is no simple answer, as much as the two plaintiffs’ software experts in this trial would have you believe otherwise. They conveyed to the non-technical judge and jury that if you follow the right “recipe,” you can build a safe system, saying things like: “So this is a recipe book for how to build safe cars” (referring to MISRA), and “Is your process good? Have you followed a good recipe?” and “Q: Okay. What do you mean by this MISRA is a recipe? A: It's -- we're going to go through that in some detail in slides, but it tells you what you need to do to be safe at great length. … It has everything that you need to know, all the accepted practices for building safety,” and “They don't follow a recipe for making a safe system.” As anyone who has actually designed and written software for safe systems knows, it is not nearly this simple. Yes, there is an important body of knowledge in which one must be grounded before even attempting to architect such a system. However, that is not sufficient. There is no “recipe,” contrary to what the two plaintiffs’ experts say. In fact, there is research showing that many MISRA C violations are false positives, and that fixing every MISRA C violation can actually make code worse. Furthermore, a well respected researcher in software safety, Dr. Les Hatton, has concluded that there is no empirical data to back up the assertion from the two plaintiffs’ experts that global variables are associated with software defects. Dr. Hatton refers to that notion as “folklore.” So what can you do? The character limit for this comment does not allow a complete answer. Let me just say, extremely simplistically, that you must hire the right people to design the architecture of the system and software, based on their experience and track record, and those people must provide continual oversight all the way through design, implementation, and testing. But there are volumes more to say...

    • You bring up several interesting points. Thank you. First, with respect to Dr. Koopman’s academic code, he told the non-technical judge and jury that “global variables are evil” and the “academic standard is there should be zero” global variables. He then said that Toyota’s code was of low quality because it violated this standard. However, talk is cheap, and his own research team’s academic code shows that he does not practice what he preaches. His testimony about global variables, without the judge and jury being aware of the use of global variables in his own academic code, as well as without them being aware of the comments in his own academic code explaining reasons for using global variables, would seem to be misleading at best. Also, as discussed in detail in my article, that is only one of the many highly questionable aspects of his testimony and the testimony of the other expert. (For example, without having seen one line of Toyota’s code, Dr. Koopman told the judge and jury that Toyota’s code “has far, far too many bugs” and that “there is no reason” for Toyota’s use of global variables.) Regarding your comments about the need for independent experts to advise judges in such trials, or the need for technically literate juries, I agree that solutions such as those should be seriously considered by the legal community. But without more input from software experts, the legal community is unlikely to realize the seriousness of this problem. I encourage readers who feel strongly about this issue to speak out. Finding a solution to this serious problem is up to the legal community, but making the legal community aware of the need for such a solution is up to us. At this point, the legal community is largely unaware that they even have a problem. After all, the highly questionable testimony of the two plaintiffs’ software experts in this case apparently seemed reasonable to the non-technical judge, jury, and the judge’s legal staff.

    • Thank you for your comment. In response, I’d like to point out the following: 1. The code from the second expert (the CRC and memory test code) was not code for an academic project. It was code that the expert made available for public use by developers of real-world embedded systems. If Toyota’s number of MISRA C violations means that Toyota’s code is low quality, as he and the other expert told the jury, then his own code (with 2.9 times the violation rate), which he advertises for public use, is much lower quality. He has a double standard. 2. Both experts also told the jury that MISRA C can be used to predict the number of bugs in a body of code (1 major bug and 3 minor bugs per 30 MISRA C violations). Using that bug prediction metric, the code from both experts has many more bugs per million lines of code than they claim Toyota has. If both experts really believed that for every 30 MISRA C violations one can expect roughly 1 major bug and 3 minor bugs, as they told the jury, then one would reasonably expect that they would ensure their own code, which they made publicly available, had zero MISRA C violations, to help minimize the risk of it containing bugs. But they did not do that. As a result, the first expert’s code has roughly 16 bugs in just 591 lines, and the second expert’s code has roughly 19 bugs in just 572 lines, according to their bug prediction metric. Therefore, they apply a bug prediction metric to Toyota’s code that they do not apply to their own code. 3. Thus, my point is not that their MISRA C violations call into question their “expert” status. Rather, my point is that they apply one set of quality standards to Toyota’s code, and a different set of quality standards to their own code, which includes code advertised for use in real-world embedded systems, as well as code advertised by a researcher who advises others on how to build high quality software.

    • I encourage everyone to read Phil Koopman’s so-called rebuttal, as well as my associated comment. As you will see, his rebuttal boils down to the following: 1. He claims that I withheld mention of my work for automotive companies. That is simply not true, as I show in my comment to his rebuttal. 2. He says that my analysis “is factually incorrect,” but he does not point to a single factual error. The closest he comes to addressing any of the facts in my article is his attempt to disavow responsibility for his Ballista code, even though he advertises that code on his website and lists himself as the Principal Investigator of the Ballista project. 3. He does not address my point that his criticisms of the quality of Toyota’s code, without having seen one line of that code, are largely baseless. Without seeing any of the code, he told the jury that it “has far, far too many bugs” and “there is no reason” for Toyota’s use of global variables. Rather than explaining in his rebuttal how he could possibly come to such conclusions without seeing the code, instead he simply says that he tried to get access to the code but was blocked by Toyota. 4. He does not dispute any of the other facts in my article, including the fact that he changed the wording of a research paper in order to justify his testimony to the jury that the fleet of Toyotas would be expected to experience an unintended acceleration event roughly every week or two. As I show in the article, the actual wording of the research paper does not support the conclusion he presented to the jury. Since he does not dispute a single fact in my article, his “rebuttal” is a rebuttal in name only. One reader, who is an ACM Senior Member, reached out to let me know that he wrote to the ACM, as I suggested in the article, to encourage them to address such issues in their upcoming revision to their Code of Ethics and Professional Conduct. I hope other readers are also writing to the ACM about this topic.

    • In response to Philip Koopman’s rebuttal, I would like to point out: 1. His assertion that I withheld information about my work for automotive companies is not true. I am clear about this in both IEEE articles to which I refer in my Embedded.com article. There I say: “Since my review of the trial materials, my company has been retained by several automobile manufacturers. Confidentiality agreements prevent me from revealing details about those engagements.” 2. Dr. Koopman does not dispute the facts I present about his Ballista code. Instead, he contends that he was not responsible for the Ballista code that he advertises on his Carnegie Mellon website. I encourage interested readers to go to his website and draw their own conclusions about who was responsible for that code. 3. I encourage interested readers to read my Embedded.com article, and confirm my analysis for yourselves. Check all the facts in the article, including: a. The fact that Dr. Koopman’s Ballista code does not adhere to his own academic standards for global variables that he presented to the jury. Count the number of global variables yourselves. b. The fact that, according to the metric Dr. Koopman used to calculate the number of major bugs in Toyota’s code for the jury, the largest of his Ballista C files has 2.5 times as many major bugs per line of code as the Toyota code. Read for yourselves how Dr. Koopman told the jury that Toyota “has far, far too many bugs” based on this metric, but without actually seeing the code. c. The fact that Dr. Koopman changed the wording of an academic paper from Obermaisser when he told the jury that the fleet of Toyotas would be expected to experience an unintended acceleration event roughly every week or two. First, Dr. Koopman changed Obermaisser’s word “arbitrary” to “dangerous.” Then he changed “dangerous” to “causes unintended acceleration.” Read the Obermaisser paper, as well as Dr. Koopman’s testimony based on that paper, for yourselves.

    • I am not saying that I don’t believe a car computer could malfunction. But the fact that a malfunction could occur is not the point. The point is that the plaintiffs didn’t show that it is more likely than not that a computer malfunction caused this particular accident. (“More likely than not” is the legal burden of proof required of the plaintiffs.) They claimed they had a theory to show this, but their theory falls apart under technical scrutiny. I understand that to you this may not be that relevant, but in a trial such as this it is highly relevant. The plaintiffs simply didn’t meet their burden of proof.

    • Thank you for your comment. Although coding standards serve a useful purpose (and I have used them on many projects), I agree that just because code adheres to a coding standard doesn’t mean the code is high quality. As you suggest, one cannot meaningfully assess software quality without understanding the code and its underlying design. The plaintiffs’ criticisms of Toyota’s software quality based on metrics, including criticisms from an expert who did not even see Toyota’s source code, are reminiscent of situations I’ve encountered on past development projects. Outsiders were sometimes brought in to those projects who had limited real-world product development experience, and who assessed the quality of the software simply by gathering metrics, without understanding the code. These assessments invariably struck the development teams as being of questionable validity and usefulness. Tools such as static analyzers can be very valuable to development teams, but they can also be misused when in the wrong hands.

    • You make an interesting point. In response, I’d like to first point out that only one of the two embedded systems experts based his testimony on direct examination of the code. The first expert to testify did not see any of Toyota’s source code, yet his entire testimony was directed toward criticizing its quality. This leaves the second expert’s testimony. That expert made his trial testimony and slides available to the public and invited us to “judge for ourselves.” So apparently he felt that his trial testimony and slides were sufficient for the public to draw our own conclusions about his causation theory. And as I show, based on that information, which he provided for us to use in “judging for ourselves,” his causation theory is not credible. Also, of course, the evidence presented in his trial testimony and slides is the exact same evidence he presented to the jury to justify his flawed causation theory.

    • These are important points. Thank you. Due to space limitations, I focused my IEEE article on the plaintiffs’ flawed causation theory. But I have also examined the plaintiffs’ testimony on Toyota’s software quality, and I have serious concerns there as well. Here let me say just a few words about one of your points, 11,000 global variables. I think it is highly questionable to draw conclusions about software quality by simply counting global variables, without looking at why the decision to use them was made, and how they are actually used. There are many reasons why an engineering team might decide to use global variables after performing a risk-benefit analysis. For example, maybe Toyota decided to implement variables as global in order to support engine control calibration, which is not uncommon in automotive systems. Maybe they chose to implement variables as global in order to conserve stack utilization, even though those variables are only accessed in one place (they are not accessed globally), so that the risks of shared access are minimized. Maybe they decided to implement variables as global in order to minimize CPU utilization, even though those variables are only accessed in one place. (These are just a few examples.) The non-technical jury was made to believe by two software experts that the issue is black and white, and that if there are more than a handful of globals -- more than “five, ten” out of Toyota’s one million lines of code according to the first expert’s testimony -- then the software is of poor quality, period. This is overly simplistic and, in my opinion, very misleading.

    • I agree that we are not likely to ever know with certainty what went wrong in this particular accident. With that said, a fundamental issue for many automotive product liability trials ends up being driver error versus vehicle malfunction. There are “black box” recorders on some automobiles that capture pre-crash data, which in principle should provide evidence indicating whether a crash was due to driver error versus vehicle malfunction, and if vehicle malfunction, additional data that might enable a successful post-mortem. However, according to the testimony at this trial, the Camry model involved in this accident did not have such a pre-crash recorder. Furthermore, even if there is a pre-crash data recorder, then as long as it relies on software, care must be taken to minimize the risk that a failure in the computer system can compromise the integrity of the pre-crash data that is recorded.