Sleep data - Comparing Hello Sense, Fitbit Charge and Bellabeat Leaf

Thought it might be fun to run a comparison of sleep logging on a few of my devices.  

For reference, I wore the Fitbit Charge on my left (non-dominant) wrist, the Bellabeat Leaf clipped to the neckline of my night vest and the Hello Sense is of course clipped to my pillowcase. I sleep primarily on my side/smushed partially face-down and I'm quite a light-sleeper.

In terms of data points I've recorded, here's a break down:

• Sleep score - This is a number the software gives, generally a percentage of total sleep time.
• Total sleeping time - The time the software said I was asleep in some way (with caveats, see next item).
• Deep sleep - This seems to vary by system, some counting what I presume they're inferring to be REM time, but it looks like Fitbit is counting this as all the time I was "asleep" with "total sleeping time" being time in bed total, asleep or not.
• Times woken - Exactly what it says.  I wake at least once a night, every night, for sure.
• Fell asleep - The time the software believes I, well, fell asleep.
• Wake up - The time the software believes I was first awake in the morning.
• Out of bed - The time the software believes I physically got out of bed.
• A ? indicates - The software doesn't offer a value that fits the category.

So, let's have a look at the numbers!  

I've got 3 full days of data, and one partial. Why so little? I forgot to record the Sense data each morning for a few days and it turns out you can't go back and look at previous nights.  Major fail for the Hello Sense right there.  I'm also missing a night of the Leaf's data because I forgot to put it on. Update: The friendly Hello Sense people just emailed me to let me know I can go back and see previous nights data! The gesture was a little non-obvious - but a slide right will go backwards. 

		Fitbit Charge	Hello Sense	Bellabeat Leaf
Monday	Sleep score	96%	77	-
	Total sleeping time	7h54m	6.7h	-
	Deep sleep	7h2m	2.5h	-
	Times woken	2	0	-
	Fell asleep	23:56	0.31	-
	Wake up	07:56	07:12	-
	Out of bed	?	08:35	-
Tuesday	Sleep score	93%	81	97%
	Total sleeping time	10h11m	8.9h	9h24m
	Deep sleep	8h49m	2.7h	?
	Times woken	2	0	1
	Fell asleep	22:41	23.58	23:07
	Wake up	08:51	8.52	08:42
	Out of bed	?	8.48	?
Wednesday	Sleep score	97%	78	98%
	Total sleeping time	9h36m	9.9h	8h58m
	Deep sleep	8h56m	4.8h	?
	Times woken	3	0	2
	Fell asleep	23:39	23:17	00:19
	Wake up	09:14	09:13	09:39
	Out of bed	?	09:24	?
Thursday	Sleep score	95%	81	76%
	Total sleeping time	10h7m	7.6h	5h32m
	Deep sleep	9h35m	3.1h	?
	Times woken	2	0	0
	Fell asleep	22:43	23:06	01:36
	Wake up	08:49	06:42	07:08
	Out of bed	?	09:01	?

On Wednesday, the Sense alarm failed and didn't wake me up, so I overslept and on Thursday only 1 of two Sense alarms went off (I reported the outage, and I think they know what it was and fixed it). Thursday night I *definitely* got up at least twice.  I had a Leaf alarm set every morning, and although I felt it go off once (because I was already awake), it was never the thing to wake me up. The Fitbit alarm works well and has yet to not wake me up when used.  For reference, my first alarm in the morning goes off at 8:00am (lucky me) and then a second to get me up at 8:25 if I haven't shifted by then.

Which feels more accurate?  

Based on what I know about my own sleep habits and routine (don't judge my lazy ass), the Fitbit Charge data fits best and I'd take that one as feeling most accurate - not having any proper professional equipment strapped to me, I can't be certain, of course.  

Comparing the BellaBeat Leaf to the Fitbit data, it does seem to be pretty far off the mark - often not noticing that I've fell asleep until after the other systems.  It was also the weakest in terms of the amount of data and breakdowns in the 3 systems. 

The Sense seems intermittently accurate and is prone to believe I'm awake a lot earlier than I generally am.  I'm very surprised that the Sense hasn't noticed when I've been out of bed at night, but I wonder how much bed sharing effects that (it often just notes "you and your partner were both restless" at certain times that I suspect may be when I've been up or awake).  

Having said that, I don't think any of them are accurately recording how much time I spend in quality sleep, but I don't particularly expect a system that relies solely on tracking movement to be able to do that.  Make of this data what you will , but the bottom line on sleep data by fitness trackers should probably be to take what they tell you with a large pinch of salt.

Hello Sense sleep tracker review

Backed on Kickstarter sometime last year, my Hello Sense unit materialised without much warning. I've had it about 2 weeks now, so I think it's about time for a review. 

The Sense consists of two parts - an orb about the size of a small apple that lives on your nightstand and low-powered Bluetooth trackers called "pills" that clip onto your pillow. 

Physical design

Design wise, this thing is lovely. The unboxing and styling is very Apple-influenced, with all the packaging designed to warm you up to loving the device. The orb glows pleasingly with different colours to indicate what's going on (purple is pairing mode, for example) and it has a sharp modern look about it that makes it feel a bit like an art object on my nightstand rather than just another piece of technology. 

The little pills that clip onto your pillow are pretty cute. They have jaws of doom however and are a complete bastard to get onto the pillow, but once they're on, they're on - so at least they're not going to pop off in the night and disappear into the bed someplace.  I worry about accidentally washing them, but I'll cross that bridge when it happens (extra pills are $59 each, though, so I hope they have decent customer service when that does happen, because that's not cheap).

My one complaint about the design? They made the plug round instead of a neat rectangle (the orb needs to be plugged in, and it's a bit fussy about the cable it gets given), so it takes up two plug spaces on my power strip. I intend to swap it with a neater USB plug at some point.

Setup

Sadly, the setup somewhat ruins the initial design good feels. It was a complete pain in the arse to get connected. Putting the orb on the wifi went smoothly enough, but pairing the pills was hit and miss. I bought two pills, one for me, one for Husband (who doesn't give a monkey's about sleep tracking, but I want a comparison point), and we struggled to pair both. You have to do this shaking thing while the orb is in pairing mode, and the Bluetooth processes kept crashing, so you feel ridiculous and you spend a lot of time waiting for phones to reboot. It all appeared to be faulty software, at least, so hopefully they can iron that out, but I'm not looking forward to having to replace or change the setup any time soon. 

So, what does it do?  

Via the app, you can always see the current status of the bedroom - it shows you the current temperature, light level, sound level and humidity plus all of those over time in a graph. If you pass your hand over the orb, it'll glow a colour to indicate the current room quality based on these values. The Kickstarter advertised a particulate sensor, that at first I thought was missing, but for whatever reason it's not available with the other data in the "right now" screen, you only get to view this via the "before sleep" tab as an AQI value. I assume because they don't want to show the average out of context.

Based on these values and the movement of your pillow, it has a go at telling you when you're sleeping and how soundly. If you have forced your loving bed partner into tracking their sleep too, the app will reflect that in the overnight data, indicating when you were both moving or if it thinks it was just you. In the morning, you'll get a sleep score - mine hovers around 82 most nights. I think that's OK, but I'm not sure what it really means because Husband's is roughly the same and he definitely sleeps better than I do so I really don't know how it's being calculated.

The sleep tracking data quality is on a par with the data I get from my current FitBit Force (and Jawbone Up I had before), but it attempts to quantify it more by telling you about conditions before bed and asking you a few questions about how you slept and your habits. It says things like "you move in bed 13% more than other sense users" and it constantly tells me my room is too warm - got it - but attempts to lower the temp haven't been praised or shown to improve my sleep yet. I want a pat on the head! But my sleep quality is my own fault and I should feel bad, I guess. But in short, the data looks about right. Consumer level sleep trackers just aren't that detailed, and this one is no exception - but at least I don't have to physically wear a device to get a similar quality to that of a Jawbone or Fitbit.

The thing I like most - the alarm. I've switched to just using the sense alarm, because they have really nice sounds that aren't shrill and jarring, it fades up slowly in volume and the little orb glows a pale blue to wake me up. I just have to wave my hand over it to shut it up. It really is that future feeling I want out of my personal tech, plus it stops me reaching for my old alarm (my phone) which helps delay me checking my email. However, it doesn't have snooze so I've just set up a series of alarms 10 mins apart to fake it, but I really hope they add that (snooze could be a short wave over, off could be a triple wave over or something, Hello guys).

The thing I'm saddest about? The Kickstarter said it would have a white/pink noise generator for falling asleep. It's not there. They haven't done it. I use a few apps on my iPad for this already (I actually usually opt for rainstorm sounds - I think I miss the motherland), but I was super excited to have a noise generator connected to a sleep tracker, so I could build up some data on what sounds work best. I'm bummed they didn't add that - I'm hoping they still will, after all, the orb is basically a Bluetooth speaker. If they would just open it up to let me pass whatever audio out I wanted to the Bluetooth that'd work, too.  

Oh, and while I'm begging for features - there is no way to connect this system to anything else in the house/life. I can't even get it to send me an email with the day's stats that I could then IFTTT out to do something else based on that (Slept badly last night? Here's a reminder to go to bed earlier. Room too hot again? Turn on a fan. That sort of thing). If I could communicate with it, I'd be able to use the speaker for notifications or set light sequences for things, too. Oh, and can I have a web interface with an export? I hate having my data locked away and the small screen is not ideal for studying graphs. 

In summary

Pros: 

• Slick modern design
• Nice alarm
• Good for bed sharers
• Data quality on par with FitBit Force
• Don't have to wear anything on my body for tracking

Cons:

• Pain in the butt to setup
• No snooze
• No white noise generator
• No API / integration points
• Lack of detailed particulate information
• Data only accessible via the app

The Sense orb + 2 pills cost me $129 during their Kickstarter campaign. The same setup today would cost $188, which is really pushing their luck on price point.

So, do I like it? Yeah, I do, although it feels a little under developed right now. I don't think there's anything I'm griping about that can't be fixed in the software, so I hope they keep working on that to improve the data / feature set and if they open up an API there's plenty of opportunity there to do some cool stuff. I've got my fingers crossed that this isn't just a one trick pretty pony.