Twenty-seven years ago, the Law Commission made a decision to trust computers. They concluded that a provision in the Police and Criminal Evidence Act, which required evidence generated by a computer to prove that the system was functioning properly, was unnecessary. Instead, they believed in a “presumption of proper functioning.” This decision was supported by the government and the provision was repealed in 1999.
However, specialists in IT law now believe that the Law Commission’s analysis was flawed. Dr. Sam De Silva, a partner at CMS and former chair of the Law Society’s Technology and Law Committee, stated that the commission misunderstood the nature of software, hardware, and IT systems. He believes they misinterpreted expert views and used this misguided interpretation to support the repeal of the provision.
The tipping point came in 2019 with the Bates v the Post Office judgment, which revealed that presumptions about the reliability of the Horizon computer system had led to innocent people being imprisoned. Legal experts like barrister Stephen Mason have been warning about the dangers of blindly trusting computers for years.
In November 2020, a group of nine experts, led by Paul Marshall of Cornerstone Barristers, issued a report warning the government about the dangers of presuming computer accuracy. They argued that assuming a computer always works correctly is unrealistic and unsafe. They highlighted that all computers have the potential to fail or function incorrectly.
The report made several recommendations, including strengthening disclosure obligations to include error logs and records of system changes. It also suggested a collaborative approach to disclosure between parties and requiring the party relying on computer evidence to prove its reliability despite any bugs or errors.
Although the Marshall report has been referred to relevant authorities, no action has been taken thus far. However, it is evident that a new approach to handling computer-derived evidence may be necessary, especially with the rise of artificial intelligence software. Some experts suggest subjecting IT designers to ethical standards similar to other professions, while others propose legislation to register programmers of “serious systems.”
In 2018, Sir Geoffrey Vos, former chancellor of the High Court, predicted that computer-generated evidence would reduce court disputes over primary facts. However, it seems that this prediction may have been premature.
Overall, there is a growing recognition that the presumption of computer accuracy needs to be reevaluated in light of the potential flaws and risks associated with computer-derived evidence.