Hacking Forensic Software
For years, some of us have warned about the risks of buggy code in law enforcement software. That is, the target of the investigation will do something to do something nasty to the law enforcement machine, thus evading surveillance or worse. Frequently, this has been in the context of wiretap software; see, for example, Comments on the Carnivore System Technical Review Draft, Tapping, Tapping On My Network Door, and Carnivore and Open Source Software.
We now have some concrete examples in a related field, forensic analysis software. Some researchers have found bugs in forensic software packages. The article quotes one of the researchers: "Basically we can make it impossible to open up a hard drive and look at it." Can it be used to take over the analyst’s machine? They’re not saying yet if their attacks can be used to take over the analyst’s machine.
The societal implications are obvious: defense attorneys are going to have a field day. They’re going to quiz analysts — who are not expert on the internals of the packages they’re using — about why they think the software is trustworthy. "How do you know your buggy software didn’t miss exclupatory evidence?" They’re also going to subpoena vendors and vendor data, including source code, change logs, test plans, bug report databases, etc. If the vendor can’t or won’t produce, they’ll try to get any forensic evidence excluded.
We’re already starting to see things like this in other criminal cases where crucial evidence depends on software. In at least one drunk driving case, a Florida appeals court ordered a vendor to let a defense expert examine source code to an alcohol breath test machine; if the vendor refused, the breath test evidence would be thrown out. The court’s language was very strong: "It seems to us that one should not have privileges and freedom jeopardized by the results of a mystical machine that is immune from discovery."
The obvious counter for law enforcement is to use some form of "certified" software. That is, some outside party would evaluate the software and certify its correctness. Note that this is a very difficult (and expensive) process. Furthermore, for fairness in criminal cases, it probably needs to be an adversarial process. Imagine how this will play out in a high stakes, well-funded white collar criminal case.
Ultimately, of course, this is a question of how to produce and verify high-assurance software. Several decades of work tell us that this is a hard problem.
