The Unfair Standard: Why We Expect AI to Be Perfect When We’re Not

Written by Amber Schroader

July 9, 2025

There was a reference to a forensic expert who was in a courtroom, attempting to make a case for a new method they’d come up with to figure out a vehicle’s speed from dashcam footage. The case was heartbreaking—a father on a homemade motorcycle, killed by a drunk driver barreling down the road at 105 mph. All they had was a blurry, low-quality video from the driver’s own dashcam. The expert and their team went all-in, reverse-engineering the camera’s setup, matching its exact field of view, and breaking down the footage frame by frame to estimate the speed. It was a first-of-its-kind approach, and it had to face the court to decide if new scientific evidence is legitimate. There were a lot of questions: Had this been peer-reviewed? Was it grounded in solid science? What’s the margin of error?

The expert passed the test. The evidence was admitted, and the driver, faced with hard proof of their reckless speed, took a plea deal. Without that analysis, the charges would’ve been much lighter, and justice might’ve slipped through the cracks. That moment is important not just because of the outcome, but because it showed how much impact careful, science-backed work can have.

Today, there’s an odd double standard when it comes to AI in fields like digital forensics. We seem to hold AI to a level of scrutiny that’s almost absurd compared to what we expect from humans. With AI, people want to know everything—where the training data came from, every possible bias, every detail of how the model works. Even when AI delivers accurate, game-changing results, there’s this lingering doubt, like it must be perfect to be trusted. Meanwhile, no one expects that from a human expert, no matter how experienced they are.

The Self-Driving Car Double Standard

Take self-driving cars as an example. When one gets into an accident, it’s headline news. People lose it, questioning whether the tech can ever be safe. But when a human driver causes a crash and let’s be real, humans cause millions of crashes every year it’s just another Tuesday. We shrug it off as part of being human. Why the different standards? Autonomous systems, even in their early days, often outperform humans in safety. They don’t get distracted, tired, or drunk. Yet, we act like one mistake from a machine is a dealbreaker. A study even showed people tolerate a higher error rate from humans (11.3%) than from AI (6.8%) in fields like radiology. It’s like we’re wired to forgive our own flaws but expect machines to be flawless.

This mindset spills over into digital forensics. A human examiner can make a decision based on years of experience, maybe even a gut feeling, and we’re okay with it as long as they can explain their process. But if an AI analyzes a massive dataset and spots patterns with hardcore stats to back it up, we get skeptical. One potential glitch, and suddenly the whole system’s suspect. It’s frustrating because AI can be more consistent than humans, free from things like exhaustion or bias creeping in. This resistance is holding back progress in digital forensics and it’s holding back justice.

What Makes Something Trustworthy?

Whenever an expert testifies in court, the first thing they get grilled on is their credentials. The system sizes them up through their education, experience, publications, and track record. No one asks them to recite the syllabus from their college algorithms class or dig up the dataset from a random training course they took years ago. They care about whether their methods hold up and if they’ve got a history of getting it right.

So why do we put AI through the wringer in a way we never do with humans? The Daubert standard, which courts use to vet scientific evidence, is a good way to think about this. It looks at whether a method’s been tested, peer-reviewed, has a known error rate, follows clear standards, and is generally accepted in the field. For AI, some of these are trickier. You can test a model and peer-review its algorithms, but pinning down an exact error rate for something like a neural network isn’t always straightforward. And “general acceptance”? That’s tough in a field that’s moving as fast as AI. There’s even a proposed federal rule, Rule 707, trying to apply Daubert to AI evidence, which shows the legal world’s grappling with how to handle this.

If we trust human experts based on their qualifications and results, why can’t we do the same for AI? Why do we need to know every single detail of how a neural network works when we don’t ask a human expert to explain every thought process they’ve ever had? The real question isn’t whether AI is perfect no one and nothing is. It’s whether it meets the same reasonable standards we already use for human experts.

 

AI as a Starting Point, Not the Final Say

Here’s the thing: most AI tools in forensics aren’t there to make the final call. They’re more like a super-smart assistant, pointing investigators toward what might matter. If an AI flags a sketchy message or a weird pattern, it’s not saying, “This is evidence, case closed.” It’s saying, “Hey, take a look at this.” Just like an intern might highlight a document for their boss to review, the human investigator still decides what it all means. AI’s job is to sift through mountains of data way more than any human could handle in a reasonable timeframe and spotlight the stuff worth checking out.

This isn’t new. Forensic teams already work this way. They don’t write down every random thought or dead-end lead that leads to a breakthrough. They focus on the result: the evidence, how they got it, and why it matters. AI should be treated the same. Sure, we need to know its methods are solid, but we don’t need to dissect every line of code in court. If the process is reliable, the value’s in what it finds, not in explaining every single step of how it got there.

Let’s Be Fair About This

AI needs oversight, no question. But so do humans. We’re biased, we get tired, we let emotions cloud our judgment. So why do we trust a human’s gut call over an AI’s data-driven insight? There’s even a term for this “algorithm aversion” where people just don’t trust machines, even when they’re right more often than we are.

The standard for AI in forensics shouldn’t be perfection. It should be about consistency, reliability, and whether it helps. If AI can uncover truths, speed up investigations, or spot connections we’d miss, it deserves a fair shot. The legal system already has tools like the Daubert standard to judge evidence. We just need to apply them sensibly to AI; not hold it to some impossible bar we don’t even set for ourselves.

 

At the end of the day, AI isn’t here to replace human experts, it is here to make us better. It’s a tool, not a magic bullet. If we judge it by the same fair, science-based standards we use for everything else, we’ll see it for what it is: a powerful ally in getting to the truth.

Forensic-Impact Articles

Why do tools show different results?

Why do tools show different results?

Since I started working in the DFIR space many years ago I always remembered the rule of two tools. That rule, although stated, is not always followed by every examiner. With the rising costs of DFIR tools many organizations have only funded one tool for their teams,...

Inside Malicious Office Documents

Inside Malicious Office Documents

Guest Blogger: Luca Garofalo Today whether it is at work, in school or any other context we receive documents. They are very usefull they allow us to keep informations in a more organized way thanks to tables, images and text formatting. However some documents can...