Remote photoplethysmography (rPPG) has the potential to greatly expand telemedicine capabilities. In this article, I provide information about what this technology is, how it works, its accuracy, and the potential opportunity around it.
What Is Remote Photoplethysmography (rPPG)?
rPPG is a camera-based technology for measuring vitals and other physiological parameters. For example, you can point a cell-phone camera at your face and measure your blood pressure, without any sensors being applied to your skin. Most of the work with this technology has been done in university research labs to date. However, there are a couple commercially available solutions. [1-3]
How The Technology Works (Basic)
Your face, like other areas of your body, has small blood vessels called capillaries. A camera can detect the ratio of red, blue, and green light reflected off these capillary regions of your face. The ratio and intensity of these colors changes as certain events/physiological changes . Machine learning software have been able to use these subtle color changes in your capillaries to derive vital signs.
How The Technology Detects Individual Vital Signs
Heart Rate: When your heart beats, this leads to blood flow in your capillaries. The changes in the rate at which blood is moving through capillaries changes depending on whether your heart is contracting or relaxing. This variation in the velocity of blood flow through capillaries is reflected in the ratio of red/blue/green light that is reflected off your face and, ultimately, detected by the camera. Backend software can then calculate a heart rate based off the variation of these color ratios due to when your heart is contracting and when it is relaxing .
Sp02: There is a ratio that health care professionals are familiar with called the blood oxygenation ratio (Sp02). It is basically a measure of how much oxygen is in your blood. In red blood cells, oxygen is carried by being bound to a molecule called hemoglobin. When oxygen is bound to hemoglobin, it absorbs (and reflects out) different amounts of different light colors. By measuring variations in the same color ratio outlined above, a camera with the corresponding rPPG software can derive a Sp02 measurement .
Respiration Rate: As you breathe in and out, this changes the pressure placed on the heart. These changes in pressure subtly influence how much blood the heart squeezes out. This translates into small differences in the pulse pressure, which alter the color pixilation detected by the camera .
Blood Pressure: Explaining how rPPG technology can be used to measure blood pressure gets a bit technical. However, briefly, you are probably used to blood pressure being represented as two numbers. There is the systolic, the larger/top number, and the diastolic, the smaller/bottom number. More sophisticated sensors, usually found in a hospital, can represent blood pressure in a waveform . There is a particular marker on the waveform called the dicrotic notch which can be used to separate the systolic from diastolic part of the waveform . Based on the light recorded by the camera, a blood pressure waveform can be approximated. Software can then look at features of the waveform, like slope and amplitude, to approximate blood pressure, and then use the dicrotic notch to separate out systolic and diastolic parts of the pressure . An alternative and/or complimentary way to calculate blood pressure is for the camera to measure something called the pulse transit time (or PTT for short). Basically, by using the camera to track how the pulse waves moves through blood vessels, it can calculate resistance in your blood vessels, and approximate a blood pressure .
Most publicly available accuracy data for rPPG are limited to small proof of concept studies with limited sample sizes. But we are starting to see some larger scale study results become available. Below are some of these studies.
Lifelight: Lifelight conducted a study with over 8,500 participants, including several thousand hospital patients. HR was accurate with a less than a 1 beats per minute (bpm) error. The systolic BP reading was on average 2-3 mm Hg different than gold standard BP measurements. However, the error in BP readings had a quite large standard deviation (close to 15 mmHg) . Meaning there was a large degree of fluctuation in how close the rPPG BP reading was to the true BP. BP readings obtained the traditional manual way do vary, depending on the provider that is obtaining them. ANSI/AAMI/ISO standards is an error less than 5 ± 8 mm Hg . So some degree of variation is acceptable. However, even by the authors’ own admission, this degree of variation is a lot. Sp02 was not investigated in the Lifelight study.
BinahAI: Another rPPG company, BinahAI, has posted validation studies for a lot of their rPPG derived metrics. For their HR study, 221 and 209 participants had their heart rate measured with an Iphone and Android respectively. The mean difference between the rPPG and gold standard HR measurements was 0.6±0.6 on the IPhone and 0.4±0.6 on the Android phone . BinahAI also reported the results of a separate study, in which the BP accuracy fell within ISO standards. BinahAI conducted a study in which 60 and 59 participants had their BP measured via an Iphone and Android phone respectively. BinahAI reported that rPPG derived systolic BP was on average 6.4 mm Hg different than gold standard BP measurements. The standard deviation of this difference was ±4.8 and ±5.2 for IPhone and Android phones respectively . A few caveats for this BP study: the study was done in ideal conditions, with the phone on a preset stand, and the subjects placed a set distance way. Also, they eliminated outlier readings of more than 3 standard deviations. So I would want to understand that decision more. However, these initial results look promising. No Sp02 results were available in the studies BinahAI made available.
Other: There are also various small participant number laboratory studies that investigate the accuracy of rPPG derived vital signs. While conducted in idealized conditions, these studies are generally promising. Some of these studies that investigate Sp02 and blood pressure are in the refernce section [11, 17-19].
Is rPPG accurate enough to be used medically? It depends on the use case. In the technology’s current form, I would have significant reservations about deploying this technology to an emergency room setting, or for monitoring of patients with severe comorbidities. For long-term monitoring of generally well patients, how useful the rPPG technology would be would depend greatly on how much the measurements fluctuate per patient. Most of these studies do report the standard deviation. However, this only applies to the average error collectively of all the vital sign measurements taken in the study. This does not necessarily tell us how much these measurements fluctuate given the same patient. So lets say a patient’s true systolic BP was 120 and the rPPG measurement says this patient has a systolic BP of 112. If the rPPG measurements consistently underreported BP in this patient more or less by the same amount, I could still reasonably use this information to guide treatment. However, if given the same patient, the BP readings fluctuated greatly, then the measurements would be considerably less useful. So if I were looking to partner with an rPPG company, I would dig into their data to get more clarity on this point. Also, a lot of these studies are done in idealized conditions (i.e. the phone distance to patient and patient movement are tightly regulated). So, I would both test out this technology in non-idealized conditions, and, if feasible, I would start out any rPPG partnership with a pilot study, seeing how this technology performed in the patients’ natural environment. Overall, rPPG technology is rapidly improving. Regardless of whether this technology is currently ready to be deployed clinically, the study results to date justify investing in further R&D to reach this technology’s potential.
Other Metrics This Technology Can (Potentially) Measure
BinahAI just announced they can now measure hemoglobin A1c and cholesterol . Hemoglobin A1c is an important marker for diagnosing and monitoring diabetes. In their pilot study with 43 patients, BinahAI reported rPPG A1C was within 1 percentage point of reference A1C . I have not seen any performance data for measuring cholesterol. If they can actually measure these values, that would potentially be a huge remote patient monitoring offering.
FDA/CE/ISO Regulation Status
To my knowledge, there are no FDA approval, cleared, or 510(k) exempt rPPG solutions. Lifelight is CE marked Class I. Some of the companies referenced above have ISO certification [1, 3].
Current Funding Landscape
Binah AI’s most recent raise was a series B for $13.5 million, led by Maverick Ventures Israel. Lifelight has received $10.4 million dollars in funding. It is unclear over how many series Lifelight has raised this capital. MX Labs has raised a total of $2.5M in pre-seed and seed funding. The funding rounds were led by SMOK Ventures.
There is an opportunity for a telehealth provider, such as Zocdoc, to acquire an rPPG company and incorporate it into their platform. This would vastly expand the types of telehealth visits such platforms could provide. For example, imagine that a physician could do blood pressure monitoring for any patient in their respective state. This would be highly valuable to expand the geographic reach of the physician’s practice. This of course assumes that the rPPG company being acquired has achieved sufficient accuracy with the vital sign metrics they measure.
- Binah AI [www.binah.ai]
- MX Labs [https://mxlabs.ai/en]
- LifeLight [https://lifelight.ai/science/]
- Zaunseder S, Trumpp A, Wedekind D, Malberg H: Cardiovascular assessment by imaging photoplethysmography – a review. Biomedizinische Technik 2018, 63(5):617-634.
- Cheng C-H, Wong K-L, Chin J-W, Chan T-T, So RHY: Deep Learning Methods for Remote Heart Rate Measurement: A Review and Future Research Agenda. Sensors (Basel, Switzerland) 2021, 21(18):6296.
- Ni A, Azarang A, Kehtarnavaz N: A Review of Deep Learning-Based Contactless Heart Rate Measurement Methods. Sensors (Basel) 2021, 21(11).
- Guazzi AR, Villarroel M, Jorge J, Daly J, Frise MC, Robbins PA, Tarassenko L: Non-contact measurement of oxygen saturation with an RGB camera. Biomed Opt Express 2015, 6(9):3320-3338.
- van Gastel M, Stuijk S, de Haan G: Robust respiration detection from remote photoplethysmography. Biomed Opt Express 2016, 7(12):4941-4957.
- Lamia B, Chemla D, Richard C, Teboul JL: Clinical review: interpretation of arterial pressure wave in shock states. Crit Care 2005, 9(6):601-606.
- Saffarpour M, Basu D, Radaei F, Vali K, Adams JY, Chuah CN, Ghiasi S: Dicrotic Notch Identification: a Generalizable Hybrid Approach under Arterial Blood Pressure (ABP) Curve Deformations. Annu Int Conf IEEE Eng Med Biol Soc 2021, 2021:4424-4427.
- Molinaro N, Schena E, Silvestri S, Bonotti F, Aguzzi D, Viola E, Buccolini F, Massaroni C: Contactless Vital Signs Monitoring From Videos Recorded With Digital Cameras: An Overview. Front Physiol 2022, 13:801709.
- Fang Y-F, Huang P-W, Chung M-L, Wu B-F: A Feature Selection Method for Vision-based Blood Pressure Measurement. In: 2018; NEW YORK: IEEE: 2158-2163.
- Anesthesia Key: Fastest Anesthesia & Intensive Care & Emergency Medicine Insight Engine [https://aneskey.com/chapter-35-arterial-pressure-waveforms/]
- Heiden E JT, Brogaard Maczka A, Kapoor M, Chauhan M, Wiffen L, Barham H, Holland J, Saxena M, Wegerif S, Brown T, Lomax M, Massey H, Rostami S, Pearce L, Chauhan A: Measurement of Vital Signs Using Lifelight Remote Photoplethysmography: Results of the VISION-D and VISION-V Observational Studies. JMIR Formative Research.
- Heart Rate Validation [https://www.binah.ai/wp-content/uploads/2022/12/MED-000003-Heart-Rate-validation-report_website-version.pdf]
- Blood Pressure Validation [https://www.binah.ai/wp-content/uploads/2022/12/MED-000006-Blood-Pressure-validation-report_website-version.pdf]
- Tran QV, Su SF, Tran QM, Truong V: Intelligent Non-Invasive Vital Signs Estimation From Image Analysis. In: 2020 International Conference on System Science and Engineering (ICSSE): 31 Aug.-3 Sept. 2020 2020; 2020: 1-6.
- Rosa AdFG, Betini RC: Noncontact SpO2 Measurement Using Eulerian Video Magnification. IEEE Transactions on Instrumentation and Measurement 2020, 69(5):2120-2130.
- Shao D, Liu C, Tsow F, Yang Y, Du Z, Iriya R, Yu H, Tao N: Noncontact Monitoring of Blood Oxygen Saturation Using Camera and Dual-Wavelength Imaging System. IEEE Transactions on Biomedical Engineering 2016, 63(6):1091-1098.
- Binah.ai Announces Video-based Blood Test Ability Using a Smartphone Camera [https://www.binah.ai/news/binah-ai-announces-video-based-blood-test-ability-using-a-smartphone-camera/]
- Hemoglobin Validation [https://www.binah.ai/wp-content/uploads/2022/12/MED-000007-Hemoglobin-validation-report-V1-Binah-website.pdf]