Enhanced stereochemistry labels: advanced techniques for indicating stereoconfigurations
Computer-Assisted Structure Elucidation (CASE): Past, Present, and Future

How to Avoid False Positives and False Negatives in Analytical Chemistry

by Jesse Harris, Marketing Communications Specialist

Is there anything that analytical chemists hate more than false positives and false negatives? Your equipment was operating correctly, you ran your experiment exactly to script, but your result turned out to be an error. We love certainty and accuracy as scientists, but sometimes problems occur despite our best efforts.

While it is impossible to eliminate error completely, it is possible to control it. Understanding the relationship between false positives and negatives allows us to optimize our experimental design. To begin with, what are false positives and false negatives?

False-Positves-blog-2

What are False Positives and False Negatives?

False positives and false negatives are types of inaccurate results. A false positive is getting a positive result that should be negative (this is also known as a “type I error”). A false negative is getting a negative result that should be positive (this is also known as a “type II error”).

Let’s consider an example to understand these in practice. Imagine you need to determine if a specific toxic chemical is present in drinking water. The test is not 100% accurate. In that case, there are four possible outcomes:

Test Result

Chemical Present

Chemical Not Present

Positive Result

True Positive

False Positive

Negative Result

False Negative

True Negative

Both error types can cause problems. In the case of the drinking water, the false positive means you raise an unnecessary alarm about contamination. A false negative would mean the toxic chemical would enter the water supply undetected.

So how do we stop false positives and false negatives? Unfortunately, the two are often connected. For the water testing example, the number of false positives vs. negatives might be related to the concentration of your samples. Concentrating your sample will likely decrease the chance of a false negative but increase the risk of a false positive. Diluting samples will do the opposite.

Adjusting this balance might be sufficient to satisfy you. For water testing, a false positive is inconvenient, while a false negative could be extremely dangerous. But what if we want to reduce overall rates of error? How do we escape this trade-off? Luckily, you have some options.

Improve Your Method

The most effective way to reduce both your false positives and negatives is using a high-quality method. This is particularly important in chromatography, though method development work is necessary in other analytical techniques. Methods are handed down from one scientist to another, sometimes with little training or explanation. These methods may not be optimized for the work you are currently doing.

Why do researchers use underperforming methods? There are many reasons, but one of the most important is because method development is time-consuming. Scientists often use trial-and-error rather than a rational approach when optimizing their methods. You could spend weeks refining a method, only to find you are no further ahead by the end.

Software can significantly reduce your method development time. For example, AutoChrom allows chromatographers to predict separation times for compounds under various conditions. Not only is this more efficient, but it also will enable researchers to optimize a broader range of variables then could realistically be achieved in the lab.

Imad Haider Ahmad, PhD., Associate Principal Scientist at Merck, offered a short demonstration of this software:

Use Multiple Methods

You’ve spent the time to develop the best method possible, but you are still getting too many false positives or false negatives. What do you do next? Using a second analytical method will increase your accuracy even more.

Let’s return to our example of a chemical contaminant in wastewater. This is what the table looks like after a secondary test.

Test 1 Result

Test 2 Result

Chemical Present

Chemical Not Present

Test 1 Positive

Test 2 Positive

True Positive

False Positive

Test 2 Negative

Disagreement

Disagreement

Test 2 Negative

Test 2 Positive

Disagreement

Disagreement

Test 2 Negative

False Negative

True Negative

Additional analysis would be needed in cases where the tests disagree. This does increase the workload of testing, but it will significantly reduce the number of errors. A researcher that uses a single test that was 95% accurate would have a 5% error rate. By using two tests that are both 95% accurate, the error rate drops to only 0.25%! This applies to the rate of both false positives and false negatives.

You can further improve your performance by choosing a secondary method that specifically targets “blind spots.” UV spectrometry is very sensitive to aromatic compounds. IR spectrometry is sensitive to ketones and aldehydes. NMR can be used to test for the presence of specific heteroatoms, such as phosphorous, fluorine, and nitrogen. If you know the cause of your issue, you can pick a method (or combination of methods) that addresses the problem you are trying to solve.

But what about the time investment? Technology can help here too. A 2012 American Laboratory article looked at the time and monetary impact of using Automated Structure Verification (ASV). This software uses NMR and LC-MS data to identify the compounds. Such a system saves scientists time in structure verification as long as false negatives and false positives fall within an acceptable range. The study found that ASV would reduce human error by an estimated 78% with a total error (both false positive and false negative) of the system being ~5%. The authors also estimated that ASV would save money for organizations with 30 or more researchers.

Read the ASV study here: Part 1, Part 2

Understand Causes of Error in Analytical Chemistry

Every experiment has its nuances. Errors can result from random chance, but they often have an identifiable cause. That cause could be the limits of your analytical technique, impurities in your samples, or data analysis issues. If you understand the problem, you can either solve it or work around it.

Imagine you were screening plant material for a specific natural product. Your test is frequently returning false positives. Biological samples are complex and often closely related compounds appear in the same plant. Tinkering with your analytical method might be a waste of time, you should instead work to improve your sample preparation.

You should also know your method’s limit of detection (LOD) and quantification (LOQ). These are the lowest concentration at which a method can accurately detect or quantify the presence of a specific compound. Tests conducted below the LOD/LOQ are going to be highly inaccurate. This may sound obvious, but scientists often inherit methods they didn’t develop. They may not realize that small changes in experimental conditions have led to concentrations dropping under the LOD/LOQ.

The best way to know if your system has unique challenges is to talk with people who have experience doing the experiments you are planning. A 10-minute conversation can save you 10 weeks of work!

Acceptable Error Levels

Unfortunately, there is always a chance of error. Scientists love certainty, but that is not how the world works. If 100% accuracy isn’t possible, what is the highest level you can practically achieve?

You are also limited by time. You could use top-quality standards before every test, run everything in triplicate on multiple state-of-the-art instruments, as well as send samples to a third-party lab for back-up verification. Your accuracy will be incredible, but your project will also be incredibly time-consuming and expensive.

The real trade-off isn’t between false positives and false negatives but between accuracy and efficiency. Experienced scientists know how to identify the importance of each experiment and allocate resources to reflect that. At a certain point, all you can do is hope your results are consistent enough that you can trust them.

Comments

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name is required. Email address will not be displayed with the comment.)