Milwaukee oncologist Michael Thompson, MD, wrote to me yesterday morning (February 10) about an article – “New Study Suggests Apple Watch Heart Rate Sensor Can Predict COVID-19 Up to a Week Before a Swab Test” – promoted on Twitter by MacRumors.com. The story was about the so-called Warrior Watch Study at Mount Sinai Medical Center in New York.
MacRumors is a website that claims that it “attracts a broad audience of both consumers and professionals interested in the latest technologies and products. We also boast an active community focused on purchasing decisions and technical aspects of the iPhone, iPod, iPad, and Mac platforms.”
That broad audience may want to ask for more evidence and fewer rumors. Dr. Thompson did, and didn’t like what he saw – or, rather, what he didn’t see. He tweeted:
You should link to the primary data in @jmirpub . That primary source is not in your article or in Feb 9, 2021 @TechCrunch by @etherington. It is hard to review the data and understand how important this is without that. #scicomm
— Mike Thompson, MD, PhD, FASCO (@mtmdphd) February 10, 2021
MacRumors linked to a TechCrunch.com story which also didn’t link to the journal article describing the research.
The MacRumors editor responded on Twitter:
Wrong. The journal in question is, indeed, open access. They looked for the source article but couldn’t find it, but that didn’t stop them from making sensational claims based on something written by another news source. Wow.
I didn’t have any difficulty finding the journal article online, nor did other MacRumors readers. One of them pointed out the limitations of the research acknowledged by the researchers, something MacRumors and TechCrunch ignored. Those limitations included: a small sample size – which the researchers admit limits their ability to determine how predictive this may really be; only “sporadic collection” of relevant data by the Apple Watch; and the fact that the study relied on self-reported data, “precluding independent verification of COVID-19 diagnosis.” Dr. Thompson pointed out that those who were COVID-infected in the study were relatively young and mainly women – so the data may not be generalizable to the broader population.
As always, when researchers acknowledge limitations of their work like these, it’s important for journalists to convey those limitations to readers before they run off to make their new Apple Watch purchase.
A late change to the story
A link to the journal article was added to the MacRumors story sometime after it was published. The website shows that the original post was published at 1:27 am on February 10 with an edit at 5:27 am. But the Internet Archive Wayback Machine scan of the site at 10:06 am on February 10 did not show a link to the journal article, just as Dr. Thompson had stated. I wrote to the MacRumors editor on Twitter about this but have not yet received a response. Perhaps the change was made in response to reader criticism.
What’s wrong with this picture?
First, it’s weak journalism (can you even call it journalism?) to report a story based at least largely – if not solely – on what another technology website published. This was MacRumors basically scraping what TechCrunch reported (and not very well to begin with) and then throwing it up on their own website 10 hours later.
Second, this isn’t just the latest buzz about the Apple Watch. It’s a story that makes a sensational claim related to early COVID-19 detection. And you’ll see below how some MacRumors readers were sucked in by the story’s claims – all without any data provided.
Third, as originally published, there was no link to – and no independent analysis of – the journal article that reports on the Apple Watch study findings. As Dr. Thompson tweeted, “It is hard to review the data and understand how important this is without that.”
Fourth, it is not optimal web publishing practice to make a change (like adding a link to the journal article hours after publishing) without noting the change for readers. Dr. Thompson, who is one of the most web-savvy physicians I know, wrote to me: “The guy could have just said. Oops, my bad. I added it. Thanks, man.”
Impact on readers
Several dozen reader comments were published on the MacRumors forum.
One reader commented: “News flash. Apple Watch cures common cold and cancer. Quick run out and buy several.”
Another reader – apparently a physician who actually read the journal manuscript, wrote: “This headline (“New Study Suggests Apple Watch Heart Rate Sensor Can Predict COVID-19 Up to a Week Before a Swab Test”) is misleading. The watch CANNOT predict infection a week before the swab detects COVID.”
But another reader comment is exemplary of the other end of the spectrum: “The Apple Watch could very well end up being one of the most significant medical devices ever made in human history.” Unless, of course, this comment was meant to be sarcastic. But there were other gee-whiz reactions from readers – spawned to some degree, in my view, by “journalism” like we saw in this case.
In addition to MacRumors and TechCrunch stories, 24/7 WallSt.com posted something that was based on the MacRumors article, which was based on the TechCrunch article. Dizzy yet? Unimpressed yet?
This wasn’t even news
A pre-print of the manuscript eventually accepted for publication by the Journal of Medical Internet Research was posted on the medRxiv.org website on November 7, 2020 – more than 3 months ago.
In mid-January of 2021 – about a month ago -stories based on that November pre-print were reported by CBS News, AppleInsider.com (based on the CBS story), Tom’sGuide.com, and 9to5Mac.com.
And stories by Bloomberg, USA Today and Forbes – about studies of wearable devices’ ability to predict COVID-19 infection – were published seven to nine months ago.
Summary
If you really wanted to do a decent job on this topic this week, you could have had a reporter studying the November preprint for the past three months, getting many independent expert perspectives. You could have included some of the perspectives reported at least seven to nine months ago about other research in the field.
Instead, readers didn’t get much at all this time around.