Popular fitness trackers are unreliable when it comes to calories counting, but it is good at monitoring heart rate, a team of researchers at the Stanford University School of Medicine reports.
An evaluation of seven fitness trackers in a group of 60 volunteers, hand-selected, of 31 women and 29 men tested those devices while walking on treadmills and riding stationary bikes: The Apple Watch, Mio Alpha 2, Fitbit Surge, Microsoft Band, Basis Peak, Samsung Gear S2 and PulseOn.
The group results compared to data from medical-grade devices for measuring both heart rate, the oxygen, and the carbon dioxide in the breath, a commonly standard for metabolism and energy expenditure.
Results from the wearable devices then compared to the measurements from a two “gold standard” instruments.
Results showed that some devices were more accurate than the others. Six devices measured heart rate with an error rate of less than 5 percent. Factors such as skin color and body mass index could and did affect the measurements.
None of the seven devices measured energy (calories) expenditure accurately, the study found.
Even the most precise device was off by an average of 27%. The least accurate was off by 93%.
Euan Ashley, professor of cardiovascular medicine at Stanford and the senior author, said: “People are basing life decisions on the data provided by these devices. But tracker devices aren’t held to the same standards as medical-grade devices, and it’s hard for doctors to know what to make of heart-rate data and other data from a patient’s wearable device,” he said.
Difficult to know home device accuracy
It’s hard to know how accurate information coming from a home device is, or the process that the manufacturers used in testing the devices.
So Ashley and his team set out to independently evaluate activity trackers that met criteria such as measuring both heart rate and energy expenditure and being commercially available.
“For a lay user, in a non-medical setting, we want to keep that error under 10 percent,” Shcherbina said.Heart-rate data reliable
The take-home message, Ashley said, is that a user can pretty much rely on a fitness tracker’s heart rate measurements. But basing the number of doughnuts you eat on how many calories your device says you burned is a bad idea, he said.
The researchers could not be sure why energy expenditure measures were so far off.
Each device uses a proprietary algorithm for calculating energy expenditure, they said.
“It’s likely the algorithms are making assumptions that don’t fit individuals very well,” said Shcherbina.
“All we can do is see how the devices perform against the gold-standard clinical measures,” Shcherbina said.
“My take on this is that it’s very hard to train an algorithm that would be accurate across a wide variety of people because energy expenditure is variable based on someone’s fitness level, height, and weight, etc.” Heart rate, she said, is measured directly, whereas energy expenditure must be measured indirectly through proxy calculations.
