Causes of inaccuracies

Studies have shown that many low-cost pulse oximeters demonstrate highly inaccurate readings. However, some low-cost pulse oximeters have performed with similar accuracy to more expensive devices when used in healthy subjects. Please see our oximeter database for performance information on many low cost oximeters. 

References: Lipnick et al, Anesth Analg 2016

Keywords: low-cost, low cost, cheap, inaccurate

The short answer is no. There is no such thing as a pulse oximeter simulator and none can reliably predict clinical performance for all oximeters.

There are several devices that exist to ‘simulate’ or ‘test’ pulse oximeter function including the Fluke ProSim SPOT, Fluke ProSim8 and Whaleteq AECG100. 

These devices work by detecting LED pulses from an oximeter, then fabricating their own output signal to the oximeter sensor. The Fluke preserves any noise in the LED light signal while the others simply trigger from it and in doing so eliminate all potential errors due to LED noise (a very common problem with implications for oximeter performance). Some of these devices also have a limited range of simulated conditions (ie perfusion) and require prior calibration to the specific oximeter being tested. Thus, for oximeters already calibrated into the in vitro simulator, the simulator can help confirm that a device is working (or at least is sensing signal) as it was designed to do in the factory. For devices not previously calibrated, conclusions about performance are uncertain. 

Multiple studies report to have used these devices to ‘validate’ oximeter performance, yet performance of an uncalibrated oximeter on an in vitro simulator does not ensure good oximeter performance in reality. 

Simulators of this type are useful for determining the operating range of an oximeter.  For example the limits in terms of dark skin and low perfusion can be found.  One can determine if the device reads an erroneous value when pushed beyond its effective range or if it reports nothing.

The OpenOximetry.org Project is working to develop novel in vitro testing devices and protocols that overcome the limitations of prior devices. The hope is that newer techniques can better augment human study subject testing and predict device performance. 

Read here for more on oximeter performance validation requirements

Sickle cell disease (SCD) is a blood disorder that affects the structure of hemoglobin. It impairs the delivery of oxygen to the tissues. Due to this alteration in hemoglobin structure, patients with SCD’s oxygen dissociation curve is often shifted to the right, as shown in the figure below. This leads to lower SaO2 values for a given PaO2, particularly as patients become more hypoxemic. 

The oxygen hemoglobin dissociation curve shows the values for hemoglobin (Hb) A (solid line) and a patient with Hb S (dashed line).

This figure is taken from a Free Access paper: Wagner, M. H., & Berry, R. B. (2007). A patient with sickle cell disease and a low baseline sleeping oxygen saturation. Journal of Clinical Sleep Medicine, 3(3), 313-315.

The change in structure of hemoglobin has been thought to impact pulse oximeter accuracy. Several small studies over the last 30 years have been performed and references are listed below. These studies showed mixed results, ranging from “no clinically significant bias” (Fitzgerald et al, 2001) to a difference between SpO2 and SaO2 of more than 2% in one-quarter of study participants (Ariglani et al, 2020). In the majority of studies, SpO2 underestimated SaO2 by 1-3%. 

The findings of these studies are summarized below:

F Ortiz et al. : Small study in adults with SCD in acute vaso-occlusive crisis. 22 paired SaO2 and SpO2 samples. Found a mean 1.1% underestimate of SaO2 using SpO2. Concluded that “as long as strong and regular photoplethysmographic waves are present, pulse oximeters can be relied upon not to misdiagnose either hypoxemia or normoxemia in SCD.”

Ariglani et al.: Small study in pediatric patients with SCD. Compared 39 simultaneous and paired SaO2 and SpO2 samples. SaO2 samples were taken from arterialized earlobe blood gasses. Findings: “Pulse oximetry was inaccurate in almost a quarter of measurements in ambulatory pediatric patients with SCD, especially at SpO2 values ≤93%.” “In 9/39 (23%) cases, the difference in SpO2–SaO2 was greater than the expected error range ±2%, with SaO2 more often underestimated by SpO2 (6/9), especially at SpO2 values ≤93%”

Blaisdell et al. Pediatric population with SCD. Assessing rate of missed hypoxemia (Assumes SpO2<93% equates to PaO2<70mmHg). Compared 22 pairs of SaO2 and SpO2. Mean difference of 5%, with SD of 5.3%. 

Fitzgerald et al. : Evaluating accuracy of pulse oximetry via correlation with SaO2 in a pediatric population with SCD. 24 patients, total of 21 simultaneous and paired SpO2 and SaO2 samples. Findings: The pulse oximeter correlated well with the cooximeter-measured arterial saturation (r2 = .74). The pulse oximeter significantly underestimated saturation by a mean of −1.6% (95% confidence interval, −0.3 to −3;p = .03). “Pulse oximetry significantly underestimates true arterial saturation, but the bias is clinically insignificant”

Figure from: Wagner, M. H., & Berry, R. B. (2007). A patient with sickle cell disease and a low baseline sleeping oxygen saturation. Journal of Clinical Sleep Medicine, 3(3), 313-315. 

References: 

Ortiz, F. O., Aldrich, T. K., Nagel, R. L., & Benjamin, L. J. (1999). Accuracy of pulse oximetry in sickle cell disease. American journal of respiratory and critical care medicine, 159(2), 447-451.

Arigliani, M., Zheng, S., Ruiz, G., Chakravorty, S., Bossley, C. J., Rees, D., & Gupta, A. (2020). Comparison of pulse oximetry and earlobe blood gas with CO-oximetry in children with sickle cell disease: a retrospective review. BMJ Paediatrics Open, 4(1).

Blaisdell, C. J., Goodman, S., Clark, K., Casella, J. F., & Loughlin, G. M. (2000). Pulse oximetry is a poor predictor of hypoxemia in stable children with sickle cell disease. Archives of pediatrics & adolescent medicine, 154(9), 900-903.

FitzGerald, R. K., & Johnson, A. (2001). Pulse oximetry in sickle cell anemia. Critical care medicine, 29(9), 1803-1806.

Keywords: sickle cell disease, HbS

Anecdotal observations have suggested potential oximeter inaccuracy when readings are obtained through tattoos or henna. Many manufacturers warn this can be a cause of inaccuracy. Controlled studies are lacking to characterize this effect, and performance may vary by oximeter.  

References: Zolfaghari et al, Emerg Med J 2015; Battito, Anesth Analg 1989 

Keywords: ink, tattoo, henna, fingerprint

There are limited data on the effect of severe anemia (hgb <7 g/dL) on pulse oximeter accuracy, and this is still an active area of research. Some studies have suggested that severe anemia may affect pulse oximeter readings. They have shown that in hypoxemic patients (SaO2 is below 80%) with very low hemoglobin concentrations, SpO2 may give falsely low readings and underestimate the real SaO2. However, for normoxic patients, anemia appears to have less of an effect on SpO2 measurements.

Another study has reported that in the setting of acute hemorrhage, pulse oximeters remain accurate even down to a hemoglobin of 2.3 g/dL in patients with SaO2 of >93% (normoxic).

However, there is still a need for further data to better understand how severe anemia affects oximetry accuracy, especially during concurrent significant hypoxemia. This is an active area of research for the UCSF Hypoxia Lab and OpenOximetry.org. 

Severe anemia and concurrent hypoxemia are more common than you might think. Using the example of malaria, one of many common causes of anemia worldwide, severe malarial anemia (Hgb <5 g/dL), accounts for nearly 1 million childhood deaths in sub-Saharan Africa (SSA) each year (18% of all childhood deaths in SSA). The prevalence of severe anemia (Hgb <7 g/dL) from any cause has been reported in nearly 1 in 10 children in SSA.

References: Severinghaus et al, Anesthesiology 1992; Jay et al, Ann Emerg Med 1994; Severinghaus et al, J Clin Monit 1990; Chan et al, Respir Med 2013

Keywords: anemia, low hemoglobin

Elevated bilirubin has been reported by some studies to potentially underestimate oxygen saturation by pulse oximetry, though most available data have not shown any effect on the accuracy of Spo2 with bilirubin levels up to 84.3 mg/dlitre. The light absorption spectrum of bilirubin has a broad peak at 460 nm and two smaller peaks at 560 nm and 600 nm.  Based on this, bilirubin is relatively unlikely to have a detectable effect on the absorption of the 660 nm and 940 nm wavelengths commonly used by pulse oximeters.

References:

https://pubmed.ncbi.nlm.nih.gov/2024749/

https://pubs.asahq.org/anesthesiology/article/70/1/118/30895/Hyperbilirubinemia-Does-Not-Interfere-with

https://pubmed.ncbi.nlm.nih.gov/2967958/

Cooler body temperatures can affect pulse oximeter accuracy. This is because cold extremities can lead to vasoconstriction and poor perfusion, which makes it difficult for the probe to detect a good pulse signal. Warming the body part where the probe is placed can be helpful in some cases.

References: Lifebox Pulse Oximetry Learning Module 

Keywords: temperature, vasoconstriction, cold

There have been a few studies that have evaluated the effects of henna on pulse oximetry results. Most of these studies were conducted on normoxic patients (i.e. healthy patients with normal oxygen levels) and found that the SpO2 values from patient fingers with henna dye were not significantly different from values obtained on non-dyed fingers. One study showed that pulse oximeter saturations from henna dyed fingers were significantly higher in patients who were hypoxemic. This means that in the setting of hypoxemia, pulse oximeters placed on body parts with henna dye may show falsely high SpO2 readings. Another study looked at red and black henna in healthy adults and found that red henna had no effect on pulse oximeter reading, but the black henna caused difficulty in obtaining a SpO2 reading. Given these findings, it may be best to choose a finger that does not have henna dye for placement of the pulse oximeter.

References: Çiçek et al, Emerg Med J 2011; Samman et al, Saudi Med J 2006; al-Majed et al, Trop Geogr Med 1994

Keywords: henna, henna dye, henna paste

Poor oximeter performance (i.e. either inaccurate or absence of a reading) during low perfusion is a known phenomenon in clinical and laboratory settings. If a patient is cold or has peripheral vasoconstriction and poor perfusion, the pulse oximeter may have difficulty detecting a good pulse signal on the fingertip. This affects some oximeters more than others. Some oximeters perform wandering readings while others may produce steady but inaccurate results. At present, there is no requirement by certifying bodies to test oximeter accuracy under controlled or standardized conditions of low perfusion. Nonetheless, some manufacturers and labs do this testing routinely. The OpenOximetry.org Project is working to publish these data in our database while developing and advocating for standards for performance during low perfusion.

References: Lifebox Pulse Oximetry Learning Module 

Keywords: vasoconstriction, poor perfusion, cold

Motion such as patient shivering, tremor, or other movements may affect the probe’s ability to detect an accurate signal. It may give an erroneous value if there is too much motion or prevent the device from obtaining a reading at all.  At present, there is no requirement by certifying bodies to test oximeter accuracy under controlled or standardized conditions of motion. Nonetheless, some manufacturers and labs do this testing routinely. The OpenOximetry.org Project is working to publish these data in our database while developing and advocating for standards for performance during motion. 

References: Lifebox Pulse Oximetry Learning Module

Keywords: tremor, shiver, movement, motion

Nail polish or nail varnish can impact pulse oximeter accuracy, so it is best to remove the polish to allow the light from the probe to pass through the tissue. If it cannot be removed, placing the probe on the toe or on the fingertip sideways can be an option.  

Certain colors of nail polish may lead to underestimation of SaO2 by pulse oximetry. Studies have shown that in healthy adults, blue, green, black, purple and brown nail polish appear to interfere the most with pulse oximeter SpO2 readings. In fact, they can cause readings of up to a 3-5% underestimation in oxygen saturation, though data are mixed. 

Another study, which examined different nail polish colors on 50 critically ill patients, showed that all colors interfered with pulse oximetry readings, but that the most significant interference was due to black, purple, and dark blue polish. All other nail polish colors had a lesser effect on pulse oximeter readings. 

Regarding gel nail polish, a study in 17 healthy adults tested several colors of gel-based manicure, and concluded that certain colors of gel polish resulted in a significant overestimation of SpO2, although their results varied when using different brands of pulse oximeter.

A recent review on the topic was published in 2023 and concluded only marginal reductions on SpO2.

References: Lifebox Pulse Oximetry Learning Module, Ralston et al, Anaesthesia 1991; Adler et al, Acad Emerg Med 1998; Hinkelbein et al, Resuscitation 2007; Yönt et al, Intensive Crit Care Nurs 2014; Coté et al, Anesth Analg 1988; Yek et al, Singapore Med J 2019 Aggarawal et al, Respiratory Care, 2023.

Keywords: nail polish, varnish, nail color

The effect of skin pigmentation on pulse oximeter accuracy remains poorly characterized. Pulse oximeters may be less accurate in patients with darker skin, especially at lower oxygen saturations. Of note, this inaccuracy can tend to overestimate oxygen saturation, which may lead to false reassurance during clinically real hypoxemia. The amount of inaccuracy varies significantly between different devices, manufacturers, and probes. The OpenOximetry.org Project is actively working to better understand the impact of pigment on oximeter accuracy and to develop new strategies to eliminate this source of error.

References: Okunlola et al, Resp Care, Nov 2021, Bickler et al, Anesthesiology 2005; Feiner et al, Anesth Analg 2007; Adler et al, Acad Emerg Med 1998; Bothma et al, S Afr Med J 1996; Zeballos et al, Am Rev Respir Dis 1991; Ries et al, Chest 1989; Emery J Perinatol 1987; Sjoding et al, N Engl J Med 2020

Keywords: skin, pigment, skin tone, accuracy

When you click on a device to view its standard performance details, some of the terms can be unfamiliar. To help, we’ve put together this guide to explain some of these terms and concepts. 

NOTE: If you come across something that isn’t explained here, you can simply hover over the dark grey “i” button for a quick explanation. If you’re still unsure, feel free to contact us, and we’ll clarify!

——————————————————————————————————————————————————–

How do we determine how accurate the oxygen saturation measured by the pulse oximeters is? 

Pulse oximeters estimate oxygen saturation (SpO2), which is the percentage of hemoglobin in the blood that is bound to oxygen. Pulse oximeters do this non-invasively by shining light through the skin. The most accurate way to measure oxygen saturation, however, involves taking a blood sample from an artery and analyzing it with a specialized device called a blood gas analyzer.

To assess the accuracy of pulse oximeters, we compare the oxygen saturation from the pulse oximeter to the oxygen saturation from the blood gas analyzer. We do this in healthy adults, in a controlled lab setting, by gradually and safely lowering their oxygen levels from a saturation of 100% down to 70%. Specialized statistical methods like ‘ARMS’ (described elsewhere in this FAQ) are then used to evaluate the pulse oximeter’s performance. For more information about our process, please refer to our study protocol.

——————————————————————————————————————————————————–

Can you explain what the manufacturer-claimed ARMS (Root Mean Square Error) for the SpO2 range of 70-100% refers to?

The ARMS can be a confusing term to describe pulse oximeter performance, but it is the most common metric used and is required by regulatory bodies. ARMS stands for accuracy root mean square. It tells you, on average, how close the device’s readings are to your true blood oxygen levels, when the oxygen levels are between 70% and 100%, based on tests in healthy adults in controlled laboratory studies.

It’s helpful to understand that a lower ARMS means the device readings are more accurate and precise. For example, with an ARMS of 2%, the device’s readings would typically be within approximately 2-4% of the true oxygen level. So, if your true oxygen saturation is 94%, then the pulse oximeter reading may be between 90% and 98%. 

Under the standard performance information for a particular device, there are two different ARMS values listed: 1) An ARMS that’s reported in the product manual from the manufacture (Manufacturer claimed Arms), 2) The Independent ARMS that we calculate from our testing here at the UCSF Hypoxia Lab using our study protocol.

We follow current FDA guidance and ISO standards for testing and also add more elements to our testing protocols to better account for diversity of multiple factors like different skin colors and different levels of blood flow to the hands (factors that may impact performance of pulse oximeters).

For more information about ARMS, check out this detailed FAQ.

——————————————————————————————————————————————————–

What does the “Independent ARMS Study Cohort Size” mean?

This refers to how many healthy adults were tested using the OpenOximetry.org Protocol for this particular device. We share preliminary results after testing in at least 10 people (which is the minimum number required by the FDA guidance and ISO standards as of October 2024), but we continue testing the devices on as many different participants as possible to improve the diversity of people included. We are also waiting for updated guidelines to know the best number of participants for these tests.

——————————————————————————————————————————————————–

How is skin pigment defined?

Skin pigment refers specifically to the amount and type of melanin in your skin. People with darker skin have more melanin, while those with lighter skin have less.

We’re working hard to understand how skin pigment affects pulse oximetry readings. To learn more about how we measure skin pigment in our studies, check out our Skin Color Quantification page and our FAQ on skin pigment.

——————————————————————————————————————————————————–

Why might openoximetry.org results be different from what a manufacturer or other testing has reported?

While our testing follows FDA guidance and ISO standards for evaluating pulse oximeter performance, some aspects of our procedures may differ from those used in other testing labs. For instance, we test devices on a different group of people than those used by other labs, and device performance can vary between groups. Additionally, one way our testing may differ, while still adhering to ISO standards and FDA guidelines, is in how we modify participants’ physiology. For example, in pulse oximeter testing on the fingers, it is common practice (and allowed under current regulations as of October 2024) to warm participants’ hands to improve circulation and enhance the signal. This may result in better performance for some devices during warming than without warming. However, for the Open Oximetry Project, we do not routinely warm participants’ hands during testing.

——————————————————————————————————————————————————–

Why might openoximetry.org results change over time?

Our results will continue to evolve as we continue testing devices in more people,  learning more about how to best measure performance, and as regulatory recommendations for testing change. This is an ongoing process, and the science is still developing, so it’s not perfect yet. There are many factors that affect how pulse oximeters perform, and we can’t account for all of them with the current testing protocols. As we collect more data from a wider range of devices and diverse groups of people, our understanding and results may change. So, it’s a good idea to check back regularly for updated information!

——————————————————————————————————————————————————–

What does pulsatility amplitude mean?

Pulse oximeters look for the pulsatile (i.e. beating or changing) signal coming from the blood vessels to measure oxygen levels in the blood.  Some devices report the pulsatility in various ways like perfusion index, pulsatility amplitude, percent modulation or other ways. This gives us an idea of how strong the blood flow signal is in the area being measured. For more information, check out our FAQ on perfusion.

——————————————————————————————————————————————————–

What is differential bias?

Differential bias (sometimes called disparate bias) refers to how much a pulse oximeter’s performance can vary between people with light skin and those with dark skin. To measure this, we compare how much the device over- or under-estimates blood oxygen levels (SpO2) in people with very light skin versus those with very dark skin*.

*Using modeling of skin pigment and SpO2 data, very light skin and very dark skin are defined as having an difference in ITA of 100. Read more about differential bias here.

Pulse oximeter measurements can be influenced by many variables such as skin pigment, body temperature, nail polish, movement, and perfusion. If the waveform tracing is erratic and low amplitude, this might be due to a weak signal or due to the patient moving. This can make it difficult for the oximeter to interpret the oxygen saturation accurately. Common reasons for poor waveform can include poor perfusion, irregular heart rate, or poor probe placement.  All of these factors can cause both underestimation and overestimation errors in oxygen saturation readings.

References: Lifebox Pulse Oximetry Learning Module 

Keywords: waveform, perfusion, battery, error

A pulse oximeter photoplethysmograph (commonly referred to as a ‘pleth’) is a graphical display of the pulse oximeter signal over time. Its appearance can vary widely under different clinical scenarios. In healthy patients, the graph should appear as asymmetric humps similar in appearance to an arterial pressure waveform though usually with less level of detail (i.e. the dicrotic notch may not be visible). The waveform should appear at intervals that match the heart rate and regularity. 

Patient motion, tremor and poor perfusion are common factors that affect the pleth. The pleth is an extremely useful feature to help clinicians quickly determine the quality of the oximeter signal. Of note, in the face of a low signal (or low perfusion) some oximeters may auto adjust the scale of the display (y axis) to increase the visual amplitude of the pleth. This can be misleading to clinicians who must also pay attention to signal quality indicators when available. 

References: Jubran, Crit Care 2015 

Keywords: pleth, waveform, PPG

‘Differential bias’ (sometimes referred to as disparate bias) attempts to describe how much a pulse oximeter performs differently in people with light skin pigment as compared to dark skin pigment. To determine this, we compare the device’s bias (how much it over or under-reads SpO2) for people with very light skin and people with very dark skin*. 

Differential bias is calculated independently for two different SpO2 ranges (70-85% and 85-100%). Thresholds for allowable differential bias are not yet established though proposals have targeted 3.5-4% for the SpO2 70-85% range, and 1.5-2% for the SpO2 85-100% range. 

For example:

• Scenario 1: If a pulse oximeter on a person with light skin reads 1% higher than their true blood oxygen saturation (e.g., the oximeter shows 91% when the actual blood oxygen saturation is 90%), and on a person with dark skin it reads 2% higher (e.g., it shows 92% when the actual blood oxygen saturation is also 90%), the differential bias between the two is 1%.
• Scenario 2: If the pulse oximeter on a person with light skin reads 2% lower than their true blood oxygen saturation (e.g., the device shows 88% when the actual blood oxygen saturation is 90%), and on a person with dark skin it reads 1% lower (e.g., it shows 89% when the true value is 90%), the differential bias here is also 1%, but in the opposite direction.
• Scenario 3: If the pulse oximeter on a person with light skin shows no bias (e.g., reads exactly 90% when the actual blood oxygen saturation is 90%), but on a person with dark skin it reads 5% higher (e.g., the device shows 95% when the actual value is 90%), the differential bias is 5%.

These examples demonstrate how differential bias can manifest as either higher or lower readings depending on skin pigment.  

*Using modeling of skin pigment and SpO2 data, very light skin and very dark skin are defined as having an difference in ITA of 100. Read more about ITA here.

The locations for placing pulse oximeters vary by the type of device and probes available, including whether it is a transmittance or a reflectance device. Transmittance devices shine light through a part of the body, so they must be placed on a relatively thin or translucent area such as a fingertip, earlobe, nose or the foot of an infant. It is important that the body part is placed far enough into the probe so that the light shines through the tissue rather than to the side of it. Reflectance devices do not require light to pass through tissue, so they can be placed in a variety of locations including the patient’s forehead, wrist, foot, or chest.

References: Lifebox Pulse Oximetry Learning Module

Keywords: body part, location, placement, finger

Conventional pulse oximeters assume that the major absorbers of light within arterial blood are deoxygenated hemoglobin (HHb) and oxygenated hemoglobin (O2Hb) and calculate SpO2 based on the ratio of measured absorbance of only two wavelengths of light (660nm and 940nm).  Methemoglobinemia (MetHb) occurs when the iron in hemoglobin is oxidized, altering its ability to bind and offload oxygen. 

The molar extinction coefficient of MetHb at 660 nm red light is approximately 1 and approximately equal to the molar extinction coefficient of HHb (see image below).  The molar extinction coefficient of MetHb at 940 nm infrared light is also approximately 1 and significantly higher than the molar extinction coefficient of O2Hb.  As a result of MetHb absorbing 660 nm and 940 nm light approximately equally, the Modulation Ratio (R-Value) approaches 1, this corresponds to an SpO2 of 80-88%. This can result in either an overestimation or an underestimation of SaO2 levels. 

When a patient is cyanotic and methemoglobinemia is suspected, a co-oximeter capable of measuring the absorbance of greater than 2 wavelengths of light is necessary to determine fractional oxygen saturation and MetHb levels. 

Figure: Jubran, Critical Care, 2015

Keywords: Dyshemoglobin, Methemoglobin, Methemoglobinemia, 

References: Jubran, Critical Care, 2015 Chan et al, Respiratory Medicine 2013