A rebuttal to “Why every embedded software developer should care about the Toyota verdict”

12/6/2017

To: Editor, Embedded.com, Insights section. 

It is disappointing to see serious problems in the December 3, 2017 article “Why every embedded software developer should care about the Toyota verdict” by David Cummings. The author’s analysis is factually incorrect. Combined with his failure to acknowledge a significant conflict of interest, his article should be considered with skepticism.

The author fails to mention that he and his company are on the payroll of a large automotive company currently involved in an unintended acceleration class action case that also involves the expert discussed in his paper. Since the author saw fit to withhold mention of this extremely relevant conflict of interest, one must question the author’s true motives in publishing a series of pieces aimed at criticizing the expert’s testimony.

The author criticizes Toyota UA case expert testimony, saying it is according to a double standard. He bases this on characterizing the Ballista project as having been written by “the expert and his team.” However, the code in question was developed by a student project group under the supervision of another faculty member in a different department. In other words, the testifying expert didn’t write the code, and didn’t supervise the quality of the code. That project succeeded as a DARPA-funded proof of concept exercise, which was the goal. As readers should expect, non-critical research code written by students still learning basic software engineering practices should not be assumed to strictly follow life-critical software product practices.

There are other technical issues and flaws with the author’s analysis, such as inventing a novel “scale up” global variable metric with no literature support for such an approach. However, details aren’t worth discussing, because the premise of his analysis is fundamentally incorrect.

The author additionally criticizes the expert for not having seen Toyota’s code. That was not for lack of trying. Rather, Toyota spent significant effort preventing access to their code. The expert testimony was, however, based on first-hand knowledge of hundreds of pages of detailed reports from both NASA and other expert analysis of the code. The opinions were based on the totality of that information as well as numerous technical design documents. The full reasoning behind the opinions and use of terminology that the author criticizes is subject to a protective order, and trial transcripts are by necessity summaries of opinions. However, Toyota had full opportunity to challenge and rebut the technical analysis during the case, and was unsuccessful in doing so. The author also does not really discuss the crux of the expert testimony, which had to do with whether Toyota followed accepted practices in creating safety critical software, and the existence of a hardware single point of failure.

If readers want to truly understand the who, what, and why of creating safety critical and mission critical software, including a module with a much more complete story of what really happened in the Toyota UA cases, they are invited to look at freely available course lectures on the topic at:
      https://users.ece.cmu.edu/~koopman/lectures/index.html

Prof. Philip Koopman
Carnegie Mellon University    
Prof. Koopman has testified in both the Toyota UA trial and a currently ongoing unintended acceleration class action lawsuit.

20 thoughts on “A rebuttal to “Why every embedded software developer should care about the Toyota verdict”

  1. “In response to Philip Koopmanu2019s rebuttal, I would like to point out:nn1. His assertion that I withheld information about my work for automotive companies is not true. I am clear about this in both IEEE articles to which I refer in my Embedded.com a

    Log in to Reply
  2. “So at Carnegie Mellon, students are not taught to design to the “best commercial practice” standards? Sounds like doublespeak. You can't have it both ways (displaying the code on the University website as an “example”). And as far as scaling metric

    Log in to Reply
  3. “My wife and I have had “computer run” vehicles since the 1990s. Unexpected acceleration has happened in two of the vehicles. In one case the engine cycled between 500RMP & max (pinned) on seconds intervals, breaking the engine mounts before the key co

    Log in to Reply
  4. “Poor title. Where's the rebuttal? I would expect a professor to have better debating skills than this. Some academics have become too accustomed to mere mortals unquestioningly believing what they say and don't know how to handle questioning. How much

    Log in to Reply
  5. “I read Dr. Cummings article and found it to be totally objective. Every statement made was independently verifiable. The fact that he used a linear extrapolation to scale up the plaintiff's expert's own code, is as every scientist knows the best practice

    Log in to Reply
  6. “Incorrect testimony is genuinely problem as you stated, and it's sad that it wasn't pointed out by the Toyota's legal team. nnHowever, I am somewhat bothered that you attach so much significance to the academic code. The issue here is Toyota's level of

    Log in to Reply
  7. “You bring up several interesting points. Thank you.nnFirst, with respect to Dr. Koopmanu2019s academic code, he told the non-technical judge and jury that u201cglobal variables are evilu201d and the u201cacademic standard is there should be zerou20

    Log in to Reply
  8. “So you have had 2 cars that did this and you are still buying what you call “American”…why? You do know that many Toyotas are built in the US right? I'm ok with people buying Cars because they like the style and features, but question buying on char

    Log in to Reply
  9. “I fear that those, such as Dr. Koopman, who are consumed by absolutes and blind faith, absent scientific method, should go back a short distance in the history of computer science and recover thoughts such as those of Djikstra: nnPlease don't fall into

    Log in to Reply
  10. “Mr Cummings, first of all I appreciate the dicussion you are raising, as a warning for us software developers to keep track of quality of software we are writing. Many times, developers do not know the legal repercussions of what the software they write m

    Log in to Reply
  11. “The correct link to the URL in my previous comment (without the final period) is:nhttps://tinyurl.com/yajw76txnnAlso, the link to the archived Ballista page on the Wayback Machine is:nhttps://web.archive.org/web/20160616215326/https://users.

    Log in to Reply
  12. “What is troubling about all of this is Dr. Koopmanu2019s breach of professional ethics, given the evidence presented that he changed a word in an authoru2019s paper in order to shape a desired outcome.n nI donu2019t know if Dr. Koopman is a member of

    Log in to Reply
  13. “I think the question is not the resource/construct but the way it is (ab)used. IMO global variables fall in the same category as goto: because of the way they were badly used in the past they became anathema, a case of u201cshoot the messengeru201d (th

    Log in to Reply
  14. “I agree with your point that Dr. Koopmanu2019s testimony is troubling and raises important ethical issues. And specifically, with respect to your questioning of the appropriateness of Dr. Koopman having u201cselected himself as the arbiter and spokesper

    Log in to Reply
  15. “It is funny how you did not include the very next sentence after the part you quote: nn”… 25 years of use. Although hard real-time systems and scientific subroutine libraries are not the same beast, …”nnYou should also know that defects is not t

    Log in to Reply
  16. “You left out the second half of Dr. Hattonu2019s sentence. In its entirety, that sentence and the following sentence, which together complete Dr. Hattonu2019s paragraph, read:nnu201cAlthough hard real-time systems and scientific subroutine libraries

    Log in to Reply
  17. “Update: Dr. Koopmanu2019s recent testimony on unintended acceleration in a class action case against Ford was thrown out by a federal judge on 3/26/2018. This is the case that Dr. Koopman refers to in his second paragraph.nnBecause a number of reader

    Log in to Reply

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.