Comparing Sleep Trackers: Jawbone Up24, Basis Peak, S+ by ResMed, Sleeprate, Garmin 920XT, Fitbit

Comparing Sleep Trackers: Jawbone Up24, Basis Peak, S+ by ResMed, Sleeprate, Garmin 920XT, Fitbit

In this video (at the bottom of this post), I compare sleep trackers to see which ones function best.

THE CANDIDATES (and their sensors)

  • Jawbone Up24 (3d accelerometer)
  • Basis Peak (continuous heart rate, skin temperature, perspiration, 3d accelerometer)
  • S+ by ResMed (movement as read by the echo-location of low-power signals off the body)
  • Sleeprate (EKG Polar H7 bluetooth chest strap)
  • Garmin 920XT (3d accelerometer)
  • FitBit (all): (3d accelerometer)
  • Beddit (version 1): (bio-electrical impedance)

HOW TO TRACK SLEEP

At a bare minimum, you want to know how long you slept and preferably how many times you woke up. Many devices attempt to provide sleep data but all they can do is say that either you’re “really not moving”, “you’re sort of moving a bit”, or “you’re probably awake” because that’s all the data they can get from the 3d accelerometer. But if you are serious about sleep, you really want to know about REM and deep phases and try to maximize those. REM sleep refreshes the brain and deep sleep refreshes the body. Only three of the five give you that REM/deep information and only two of the three do it reliably.

Screenshot 2015-01-13 20.57.30

(screen clip from Sleeprate)

There is of course a whole other issue of whether or not the data they present is actually clinically valid compared to the gold standard of brain-wave analysis (like the old Zeo used to use.) I don’t have clinical sleep tools here (or my old Zeo) to cross-reference against so I always fall back on the idea that once you have a tool you like at least you can reference its data over time and that still helps you move in the right direction.

SCORES

Shown in the video

In order to score the devices, I looked at their overall balance of data quality, data presentation, ease of use, and mental overhead in using the system daily. Here are the scores. NOTE: This is for SLEEP TRACKING ONLY. This is NOT about sport, activity, or smart-phone functionality.

Basis Peak: I want to like this watch. I’ve tested it for a month. And for sleep, it’s pretty darned good. It senses your sleep and wake periods automatically without you doing a thing. However, it often breaks sleep up into 2-6 “chunks” that have to be reassembled and that often don’t assemble correctly on the mobile app (although they’re often easy to assemble on the web app as demonstrated in the video). So if you really need pretty decent accurate sleep info and want the lowest amount of fussing (and like the idea that you don’t have to turn sleep tracking on and off), the Peak is worth exploring. Another major down-side: you can’t get the data out. Ever. They’ve talked about how they’re building APIs and connectors “real soon now” for a few years. I don’t believe it will ever happen. So basically your data is trapped. (Also be warned, like the model that came before it, it’s still a lousy fitness and sport tracker, something I’ll cover in an upcoming post.)
Grade: C

C: S+ by ResMed: this contactless system sits at the corner of your bed and bounces sound-waves off your body all night and uses the echo-location to figure out your respiration rate and movement. In theory it can then extract your REM, deep, light, and wake states. To me, that seems pretty fantastic since it’s contactless…but they claim it works. I’d love to see third party validation.  To use it, you still have to turn it on at night and answer some questions and then you have to wake up and turn it off in the morning. I’d prefer to see some sort of auto-sensing in place for sleep tracking. People don’t need “one more thing” to think about. This is the death of wearable tech – having to deal with the care and feeding of the system.
Grade: C

Jawbone Up24: Simple to use, and nice UI on the mobile app. Also has great integration to other applications. It also has great battery life. The downside is that it doesn’t track REM or Deep sleep. I’m hoping they’ll fix this in the Up3, due out soon. For real sleep tracking, it’s not that great given that it’s missing phases of sleep.
Grade: D

Sleeprate: I’ve tried this thing for a month. It has been a month of frustration, error messages, server failures, and pesky email messages trying to coach me back to using it. I hate that it uses a Polar H7 Bluetooth EKG strap since that’s both uncomfortable and also one more step to take before crawling into bed. When it *does* work, the reporting is excellent and the data is probably even pretty good given that it’s pulling from an EKG strap but it still gets an F because it’s so damned unstable and I hate that you have to use a Polar HR strap. That’s not a useable long-term sleep tracking solution for that reason alone.
Grade: F

Garmin 920 XT: All of the Garmin products use this terrible sleep tracking module although I have not done a review of the VivoSmart+HR strap because you can only have one source of sleep and step data and I prefer to have the watch over the VivoSmart. This very limited sleep tool is not worth using. It’s too bad because it sullies an otherwise awesome sport tracking watch (I’ll cover the sport tracking functions in an upcoming video – it kicks ass there.)
Grade: F

Not Shown in the Video

Beddit version 1: This is a touch-less, device-less system which is a strip that sticks to your mattress under your sheet. It uses electrical impedance to pick up your heart rate and monitor your sleep. Version 1 was running Bluetooth 4.0 Legacy (i.e. not BTLE), so there was this terrible workflow involved: Open phone. Launch app. Go to iPhone Settings. Find Bluetooth. Connect to Beddit. Go back to App. Connect to Beddit. Answer a bunch of questions. Hit SLEEP. In short, it was unuseable and unsustainable. Version 1 was definitely an “F”. HOWEVER, I’ve spoken to Lasse (the founder/CEO) and they’re in the middle of revising this and rebranding it in the partnership with MisFit and v2 was supposed to be shipped in October but I haven’t confirmed that the MisFit Beddit IOS version is v2 or not. The revised version was supposed to have sorted out a better BTLE connection that would allow the app to automatically connect to the Beddit (the same way that the S+ auto-connects between the iPhone app and the S+ device.)  I’m hopeful that this new version 2 will be a better device, including some very powerful reporting.
Grade: F

F: FitBit (all): I didn’t have any Fitbits in the video (I’ve had the One, Ultra, Flex, Force, and am testing the new ChargeHR). I stopped using all of them for sleep tracking for the same reason – no ability to extract anything useful from their minimal sleep logs. They have tiny little graphs that don’t define sleep phases, so they’re not that useful for anybody serious about tracking sleep.
Grade: F

Without further ado, here is the video. Please excuse the rough audio. I’ll work on improving that for the upcoming series on performance sports tracking.

Edit Notes:

1. After further consideration I downgraded the Basis from B to C because I had forgotten that the data is locked in and there is and never has been any good way to get the data out.
2. I downgraded the Jawbone after reviewing it to see that it still doesn’t show “deep sleep”, just “sound sleep” which is something..well, I don’t know what that is.