The first AI system for seamless study of heart rhythm using smart speakers

Smart speakers, such as Amazon Echo and Google Home, have proven to be able to monitor some health care issues at home. For example, researchers at the University of Washington have shown that these devices can detect a heart attack or monitor babies’ breathing.

But what about watching something even smaller: a momentary movement of an individual’s heartbeat in a person sitting in front of a smart speaker?

UW researchers have developed a new skill for a smart speaker who for the first time will monitor regular and irregular heartbeats without physical communication. The system sends unreadable sounds from the speaker out into a room and, based on the way the sounds are reflected back to the speaker, can recognize individual heartbeats and visualize them. keep them. Because the heartbeat is such a tiny movement on the surface of the chest, the team’s system uses machine learning to help the smart speaker detect signals from all regular and irregular heartbeats.

When the researchers tested this system on healthy participants and heart patients in the hospital, the smart speaker found heartbeats that closely matched the beats detected by conventional heart rate monitors. The team published these findings March 9 in Communication Biology.

“Regular heartbeats are easy enough to detect even if the signal is small, as you can look for an occasional pattern in the data,” said co-author Shyam Gollakota, UW’s associate professor in Paul G. Allen School of Computer Science & Engineering. “But irregular heartbeats are very challenging because there is no such pattern. I wasn’t sure it would be possible to detect them, so I was surprised that our algorithms could detect heartbeats. identify abnormalities during tests with heart patients. “

While many people are familiar with the idea of ​​heart rate, doctors are more interested in assessing heart rhythm. Heart rate is the average heart rate over time, but heart rhythm accounts for the pattern of heartbeats.

For example, if a person has a heart rate of 60 beats per minute, they may have a steady heart rhythm – one beat per second – or an irregular heart rhythm – beats are randomly distributed over that minute but they still average out to 60 beats per minute.

“Arrhythmic disorders are more common than some other well-known heart diseases. Cardiac arrhythmias can cause major diseases such as strokes, but they can be very unpredictable to occur, and therefore difficult to diagnose,” the joint said. senior author Dr. Arun Sridhar, assistant professor of epidemiology at UW School of Medicine. “Access to a low-cost test can be performed frequently and at home convenience can be a game-changer for specific patients in terms of early diagnosis and management.”

The key to assessing heart rhythm lies in identifying individual heartbeats. For this system, the examination for heartbeats begins when a person sits within 1 to 2 feet of the front of the smart speaker. The system then plays a continuous unreadable sound, which kicks off the person and then returns to the speaker. Based on how the returned sound has changed, the system can trigger individual movements – including a rise and fall in the chest while breathing.

“The movement from someone’s breath is orders of magnitude larger on the chest wall than the movement from heartbeats, so that’s a pretty big challenge,” said lead author Anran Wang, a doctoral student at Allen School . “And the breathing signal isn’t consistent so it’s hard to filter out. Using smart speakers that have several microphones, we designed a new beam-shaped algorithm to help the speakers make heartbeats to seek. “

The team designed what is known as a self-directed machine learning algorithm, which learns on the fly instead of from a training set. This algorithm combines signals from the many microphones of the smart speaker to identify the accessible heartbeat signal.

“This is like Alexa being able to find my voice even when I’m playing a video or if there are several people talking in the room,” Gollakota said. “When I say, ‘Hey, Alexa,’ the microphones work together to find me in the room and listen to what I say next. That’s basically what’s happening here but with a heartbeat. “

The heartbeats that the smart speaker detects do not look like the typical peaks typically associated with traditional heart rate monitors. The researchers used a second algorithm to break the signal into individual heartbeats so that the system could calculate the so-called intermittent times, or the time between two. heartbeat.

“With this method, we don’t get the heart’s electrical signal making a contraction. Instead we see the vibration on the skin when the heart beats,” Wang said.

The researchers tested a prototype smart speaker running this system on two groups: 26 healthy participants and 24 hospitalized patients with a combination of heart disease, including atrial fibrillation and failure. heart. The team compared the smart speaker’s interval time with one from a standard heart rate monitor. Of the nearly 12,300 heartbeats measured for the healthy participants, the smart speaker’s median interval time was within 28 milliseconds of the average monitor. The smart speaker performed almost as well as heart patients: of the more than 5,600 heartbeats measured, the median intermittent time was within 30 milliseconds of the standard.

Currently this system is set up for spot checks: If a person is worried about his heart rhythm, they can sit in front of a smart speaker to get a reading. But the research team hopes future versions could continuously monitor heartbeats while people are asleep, something that could help doctors detect conditions like sleep apnea.

“If you have a device like this, you can monitor a patient on an extended basis and identify patterns that are specific to the patient. For example, we can find out when arrhythmias occur for each particular patient and then develop appropriate care plans that are tailored to when patients need them, “said Sridhar. “This is the future of cartography. And the beauty of using tools like this is that they are already in people’s homes.”

###

The paper is co-authored by Dr. Dan Nguyen, a clinical professor at UW School of Medicine. This research was funded by the National Science Foundation.

For more information, contact [email protected].