Building rich fitness experiences with Google Fit platform and Android Wear – Google I/O 2016


MUZ MOSTOFI: Hi, everyone. Thank you. Thanks for coming. DAN AMINZADE: My
name is Dan Aminzade. I’m a software
engineer at Google, and I’m also a
dedicated triathlete. So on a good week I’ll run 50
miles, I’ll bike 100 miles, and I’ll maybe write
3,000 lines of code. So maybe it’s no surprise
that I work on the Google Fit app for Android
Wear, which makes really extensive use of
the Google Fit platform APIs that we’re going to be
highlighting for you today. MUZ MOSTOFI: Hi, everyone. I’m Muz Mostofi. I’m product manager on
the Google Fit platform. And I’m a keen,
quantified surfer. I enjoy weightlifting,
running, yoga, and occasionally experimenting
with different diets. Today we’re going to
talk about building rich experiences with the Google
Fit platform and Android Wear. I’ll start out by
talking about why we created the Fit platform,
what problems we set up to solve, and then Dan will
go over its APIs and its data types. Next, we’ll look at some
specific examples of what you can build with
a Fit platform, first basic fitness
tracking, and then a more advanced real
time fitness use case. Finally, we’ll finish up with
some really cool demos coming soon. Give me the clicker. Sorry. Let’s start out by
introducing the Fit platform. We’ll explain why we created
Fit, what problems we set out to solve, and why we believe
fitness is such a great use case for wearables. Before the Google Fit
platform existed life for a fitness application
developer was not easy. It was integration Hell. The first issue was
that of silo data. Users health and fitness
related sensor data was siloed across multiple
apps and companion devices. Second, there was no easy access
and storage of sensor data. No easy way for developers to
read, interpret, and record data from health and fitness
related devices, sensors, and connected companion devices
such as Android devices. The third problem was
poor interoperability. There was no simple, standard
way for fitness devices to interoperate. So in order to integrate
with the existing app device ecosystem app
developers would have to build their own platform
and provide an API. Some of these would
be poorly implemented, leading to user frustration. With the Google Fit platform
we solved that problem by giving apps one
single set of APIs to access, record, and
store phone sensor data, wearable data, and
other app data. So by using my phone or my
wear device as my fitness hub I can connect my
fitness wearable to my favorite nutrition,
and my sleep tracking app to my favorite meditation app. We don’t just
consolidate data, though. With user consent we provide
low level sensor recording, and we perform hard
computations for things like distance and calories. Whether it’s resting heart rate
from heart rate time stamps, distances fused from your
height, steps, and GPS, or calories from heart
rate, the Fit platform makes it easier for health and
fitness application developers. And fitness and wearables
truly go hand in hand. This is for a few reasons. First, the sensors. Your app can get access to
all these ways of measuring fitness activities. Many watches have a GPS
sensor, a heart rate monitor, and potentially
even a barometer. And because the watch
is worn on the wrist it allows us to
detect activities that we couldn’t do with a phone
being in a pocket or a purse. So that’s the first
reason, sensors. Second, you can leave the phone. I love my Nexus 6P, but
running with a device is a bit cumbersome. I have to rely on infrequent
audio queue’s to find out my current pace, because I
can’t see the screen when it’s in an armband or in my pocket. Third, faster input. When you’re on the go you
can have quick interactions with a watch to control music
or activity recording that would take much
longer if you had to rely on pulling
out your phone and opening up the right app. Now let’s talk about what the
Google Fit platform provides that enables you to build
rich experiences in line with what we just discussed. First, let’s talk about
the Google Fit app. So here we have the Google
Fit app on both phone and on Android Wear. If you’re not
familiar with the app you can see that it tracks
walking, running, and biking using the Google
platform’s API that detect activity, associated
steps, calories, and distances. The app also shows
data from connected to third party devices
and apps, such as exercise sessions, weight, heart rate,
nutrition data, and sleep. Our platform
automatically synchronizes between your Android phone,
your Android Wear device, and the web. The Fit platform exists
in all three of these. So you can write in data via
our rest APIs on the web, via our Android
SDKs on the phone, or the Android SDK on wear. The platform handles merging and
digi application of this data intelligently, and we even
support take-out integration if the user wants to
export their data. DAN AMINZADE: Thanks, Muz. So now that you’ve gotten an
overview of the Google Fit platform, and maybe
some motivation for why it might be a good
fit for your fitness app, let me go into a little more
detail about the different APIs that it provides. So basically it breaks
down into four APIs. First, there’s the
SensorsApi, which gives you access to raw
sensory data streams from Android devices
and wearable devices sensors, so things
like heart rate. Then there’s the RecordingApi. The RecordingApi does automated
storage of fitness data for you using subscriptions. You can subscribe to a
data type and Google Fit will store fitness data for that
data type in the background, and it’ll persist to the
subscription for you. Next there’s the
HistoryApi, which gives you access to
the fitness history and lets you perform bulk
operations like reading, and inserting, and deleting
data from the finished history. And then finally
there’s the SessionsApi. Sessions are basically
a metadata layer on top of raw fitness data that
let you store whole fitness activities. So a session is a time
interval during which the user was doing say
a bike ride, or a run, or weightlifting session. As far as what you can
record we support really all the standard data types
for diet, and nutrition, body, activity, sleep. We’re also adding this
new hydration data type. I think this is a pretty
exhaustive list of fitness metrics, but if
there’s one that’s not in here that you’d
like to log in your app you should let us know. So let’s start by talking just
about basic fitness tracking. You can’t measure what you don’t
track so tracking measuring is really, I think, the first
step toward motivating people towards better fitness. I actually first started
tracking my workouts 10 years ago, back
in 2006, when I was training for my first marathon. And I bought my very
first wrist mounted GPS. And it was this massive,
ugly, uncomfortable thing, like the size of a
chinchilla on my wrist. But it did the job, you know? I could plug it into
the serial port on my PC and I could download
XML data of my run. And back then there weren’t
any nice portals or fitness platforms so to
visualize my runs I built this website
with Google Maps API 1.0. And I could put
my runs up there. And there’s something
really thrilling for a data geek like me to
come back from a workout and analyze all my stats
on pace, and heart rate, and elevation. And I could really track my
progress from week to week, and workout to workout. So these days obviously
there are much more elegant and full featured
alternatives where I can analyze my workouts,
and share them with friends, and track my progress
toward fitness goals. And also I don’t have to
wear a big, giant ugly thing on my wrist. I can wear these
beautiful time pieces from TAG Heuer or Michael
Kors, and I can still track and visualize my
progress right on my wrist throughout the day
with the watch face. Now this is actually my favorite
watch face here, the Fit Cat. Yeah. Big ups for Fit Cat. It tries to encourage
you to be more active. So, basically, if you
haven’t seen this before, the more active that you
are, the more steps you take, the more active the
cute, little cat becomes. I love how he’s
wearing the I/O badge and he has the backpack there,
and wearing an Android Wear watch, also. So this seems like a
simple concept, right? The more steps you take, the
more active the cat becomes. But actually trying to build
something like this on your own is not easy. There’s a lot of stuff
you have to do if you want to do it on your own. You have to build a data
store that tracks steps. You’d have to turn on
sensors, get permission from the user to do that. Then you’d have to
be able to record step counts in a way that
won’t kill the watch battery. If you get that part working
then when the user first installs your watch face you
won’t– it will show zero because you haven’t
been recording yet. And then there’s all
these edge cases, like the user takes
off their watch and goes for a walk with just
their phone in their pocket, then you have to write the
logic for merging in that data. And then even if you
solve all these problems your step counts
still is probably not going to be consistent
with other watch faces in other apps. So, fortunately, we solved
all of these problems for you with the Google Fit platform. We store steps in the background
in a nice, low powered battery friendly way, and they’re ready
to be read anytime you like. And you can get
consistent step counts on Android Wear with
just one method call. So let me just show
you how to do that. Typically when you want to
connect to the Google Fit platform you create a
Google API client, right? And normally you create
an authenticated one where you specify the account
that you want to read data for. So here’s how I
build the client. I say that I’m interested in
reading fitness history data so I add the
fitness history API. I add scopes for the particular
types of data I want to read. So in this case activity type,
location, and body sensor data, like heart rate. And then next I add an account. So this is the account I
want to read the data from. And then finally I
add my callbacks. And now I have this
authenticated client that I can use to make queries
to the fitness platform. But wait, you’re saying,
Android Wear watches don’t have accounts. And this is true. Android Wear 2.0 will
actually add account support, but for now we don’t
have accounts on watches. Fortunately, there
is a solution. Now you could sync account
information from the phone, but this would
require a watch face to have a companion
phone app, which is a lot more complicated. So here’s an easier way to
do it without a phone app. You can create a
no account client. So you create a
Google API client. As before, we’re going
to add the History API. And now we just call
set default account. We don’t specify an account. Finish building it, and
now we have a new account client that we can still
use to query step counts. And the way you do that is with
the read daily totals method. So you’re going to
call the History APIs read daily total method. And this is basically going
to give you a simple way to get step counts
for watch face that’s going to be consistent with
other apps and other watch faces. You can set a callback so
that when the result returns you can process it. So here in the on result I’m
basically just extracting out the step count, putting it
into a steps total variable, and there you go. It’s a pretty simple
way to track steps with your watch face. And it’s not just steps that
you can get from the Google Fit platform. You can also read
and write other data, like calories consumed. So here’s an example of how the
Lifesum app displays that data. Here I’m looking at
the amount of calories I have left in my
budget for the day, 911. So I guess I could
eat like two avocados and drink two glasses
of wine tonight. Android Wear 2.0 is
going to have a new way to show data on watch
faces using complications. Maybe you’ve seen some of
this earlier in the keynote. But, basically, on
the right there’s a watch face that has
three complications. And the watch face
lets you select which complications you
want, and then any app can provide data for
those complications. So here’s an example where
Lifesum is exporting data on calories burned
as a complication so that the ranger
watch– it’s switch is on the right– can display
it in one of those three user configurable slots. And if you want more information
on building complications in Android Wear 2.0 highly
recommend you check out the talk– there’s a dedicated
talk at 4pm on this topic. MUZ MOSTOFI: Another challenge
that fitness developers face is how to perform
difficult computations. For example, to derive
distance traveled from multiple sensory
inputs the user’s height, stride length, GPS data,
activity type, active time, and step counts all have to be
fused in together, potentially across multiple sensory inputs. The Fit platform makes this
easy for third party developers through its advanced
transformations for both distance and calories. The basis app takes
advantage of this. Instead of computing
distance themselves they write in steps
and activity data into the Google Fit
platform, and read back our computed values
for distance. DAN AMINZADE: Cool. So let’s take a look at how we
might do something like this with the Google Fit platform. So there is really– there
are two ways of writing data to the platform. You can use the recording
API to do live recording. And then you can use the History
API to insert activity history. So let’s actually start by
looking at live recording. All we have to do
to start recording is to subscribe to the
data type we care about. So here I’m calling
subscribe and I’m saying I’m interested
in activity samples– activity type. And then– I actually, at that
point, I am essentially done. Google Fit will start
storing that data that I care about
in the background. So I don’t need my own database
or my back-end to store it, there’s no listener needed here. Basically, the
Google Fit platform is going to start recording
data and syncing it to the cloud in the background. And when I want it
later I can just read it with the History API. So once I subscribe I’m done. All I really need
to do is just check the status code
in the on result. And that’s just to make
sure that the subscription succeeded. Now let’s look at another
way of getting data into the Fitness history. Suppose I’ve gotten my data
from some external source, like another device. So in the example of basis they
wanted to write in some steps that they had gotten
from another device. So first you create
this data source object, and you specify the type of
data you want to write here. I’m saying I want to
write raw step count data. And then you create a data
set from the data source. And you add one or more
data points to the data set. Each of these points has
a start and an end time. So here I’m adding 950 steps
with a start and an end time. And then, finally, you
just call insert data to put the data into the fitness
history, and that’s about it. All you really have to do after
that is just basically verify that the operation succeeded
before you start querying the data with the history API. And one nice thing is that
the Fit platform will actually validate the data
points that you insert. So, for example, if I put
in an invalid time range, or I put in a really
unrealistically high calorie burn, or cadence, or
something like that, I’ll get an error back in the
client instead of the user history getting all messed up. So here’s how this actually
looks in the basis app. They put in steps,
and then you can see they are able to derive from
that– or through the Google Fit platform they get this
accurate distance traveled. MUZ MOSTOFI: We’ve been
working hard over the past year to improve the quality
of these transformations. The first area with
which we’ve improved them is relation to distance. Distance computed from
steps now has been improved with stride length calibration. From actively trucked
walks and runs to improve the quality
of passively tacked walks and runs. Secondly, for
calories, we will soon be leveraging gender,
age, and heart rate as part of our computations
to improve the accuracy. Thirdly, for merging. Previously leaving
your phone at your desk and going for a walk with
your Android Wear device would result in your step
counts dipping back down and then rubber banding back up. That will no longer be the case
through bi-directional syncing. We’ve also made
merging of our GPS data better by prioritizing the
highest fidelity GPS sources. Finally, with regards
to activities, this is our most complex task,
to infer the activity being performed by analyzing multiple
sensory inputs in real time. It’s really important,
as users get frustrated when their fitness data has
false positive and false negative activity. And they don’t get their
proper calories and step counts as a result. Fortunately, Google
is good and trusted at collecting and
understanding complex data through our advanced machine
learning capabilities. We’ve added dynamic
sampling of sensor inputs to improve quality, and
have made improvements to our algorithms by fusing
steps, position, speed, and user corrections together. The result of this hard
work over the past year is a reduction in our
biking precision error rates from 11% error in early 2015
down to 4% error as of now. And these changes will
be rolling out now. Finally, we’ve added
stride length calibration to the Fit platform. Which we hope will greatly
benefit non-GPS Android Wear devices. So now if a user goes
running initially with their phone and
their watch, which has– the phone which
has GPS and the watch, which doesn’t, the two will
calibrate together and generate a stride model and then they’ll
get accurate distance and pace estimates running
without their phone, just with their non-GPS
Android Wear device. Now that we’ve looked at some
basic fitness tracking examples I want to talk about
our push this year into real time fitness tracking. But why is real
time so important when building a motivating
fitness experience? Well, when I look at
fitness apps out there– and I look at a lot of
fitness apps out there– I see that they
adopt this framework. They start with the
goal of the user. For example, I’d like
to lose 10 pounds. I’d like to sleep better, or I’d
like to finish a half marathon. They recognize that in order
to accomplish this goal they need to encourage a
pattern of motivating actions so that the user achieves
the result that they desire. Let’s drill into these
motivating actions. There are three pillars of
wellness, exercise, nutrition, and recovery. Recovery we mean to mean
sleep and meditation. But these pillars do not
operate independently. And behavior in one needs to
be balanced against the other. By getting a holistic
view of what’s going on in each of
these areas your app can more intelligently
and insightfully present suggestions and nudges
based on a changing fitness context in real time. For example, a sleep
app might observe that you sleep better
when you exercise, or when you eat less
sugar in the evening. An exercise app may
suggest exercise based on your logged
nutrition data so that you meet your
weight loss goals. Note, that in all these cases
timeliness is important. The best time to coach
and motivate a user is immediately in
response to an action that they have taken in
relation to their fitness. So good apps have an inbuilt
awareness of the user’s overall fitness context. One of our partners,
Lifesum makes heavy use of these principles
in their application. And they integrate deeply
with the fit platform APIs, and as a result they’ve seen
double the retention rate for their users connected
to Fit versus their users who aren’t connected to Fit. And who are the APIs
that we’ve added to let you respond in real
time to a changing fitness context for your user? First, there’s Real Time
Data Update Notifications. It allows you to be
notified for certain data types in the fitness
store when they change. The second is Real Time
Session Notifications. It’s actually been in
the platform for awhile, but we’d like to
talk about it today. It allows you to be notified
that the user has started or finished a workout tracked
in the third party application. And, finally, there’s Real
Time Activity Recognition. Coming soon to
Android Wear devices to detect the onset of activity
to launch into your activity– active tracking experience. Let’s dive into the first
API, Real Time Data Update Notifications. This allows us to build
the following experience. Say I just had an
impromptu pizza and I’ve exceeded my calorie
intake goal for the day. I can get a notification from
my weight scale app telling me, listen, more exercise is
needed to keep you on track to meet your weight loss goal. DAN AMINZADE: I do admit I
have a weakness for pizza and that one looked
pretty delicious. So let’s see how we could build
something like that example that Muz just described. What we want to do is
subscribe to updates to the nutrition data type. So first thing we do is
create a PendingIntent. This is typically a
PendingIntent for an Intents service that’s going
to call into your app whenever an update to one
of these data types occurs. What you then do is you
create a DataUpdateListen erRegistrationRequest, which
is a lot of camel case, but it is a pretty
descriptive name. And we’re going to pass
in the type of data that we’d like to
subscribe to updates for. So in this case nutrition,
we’re interested in meals that are being logged. And then we pass on that
PendingIntent we created. And then, finally,
we just call this RegisterDataUpdateListenerMethod
and we pass in our client and our request. And then, voila,
we’ll get an update with the next logged meal. And, of course, you can
register for updates to all different
kinds of data types. For example, if I want
to be notified every time the user steps on their
scale and logs a new weight I can do that with type weight. And here are actually
all of the data types that support real time updates. Pretty much anything that
isn’t updated super frequently. MUZ MOSTOFI: The second API,
Real Time Session Update Notifications,
allows, for example, a nutrition app to be
notified that a user just finished a workout session. So say you just finished
a weightlifting session, your nutrition app can
tell you, it’s time to get a high protein meal in
to maximize your muscle growth. Or, say a hydration logging app. It can suggest
rehydrating immediately after finishing a 21
minute run outdoors. DAN AMINZADE: Yeah, It’s been
so hot here I don’t think– I think we can all use
hydration reminders even if we haven’t been running,
just walking around outside. So let’s build that one. Similar API to the
one that we just looked at with the
HistoryApi, but here we’re going to use the
SessionsApi because we’re interested in
registering for updates when a session starts or ends. So again we create
a PendingIntent that we’d like to fire when
a new session begins or ends. And then we call this
register for sessions method in the Sessions API. And now whenever the user
starts or ends a workout the PendingIntent
will fire and intent will be delivered to our app. And if you want to get more
information on that session you can actually extract
it from out of the Intent. So here I’m pulling out the
start and end time, and then also the type of workout. So whether it was a
run, or a bike ride, or a weightlifting
session, and I can put that into the
notification like the one that Muz just showed. One exciting new
development that’s coming to the Google
Fit platform this year is that we’re actually
going to allow your app to register not only for
these explicit sessions that the user initiate, but
also for implicit sessions. These are ones that
are automatically detected by our platform. So what it means is that your
app can respond in real time to user’s fitness activity. So if I just lace
up my shoes and I start running my
favorite run tracking app can just start up
automatically on my watch. So I’m going to try to show
you a demo of how that works. I’m not really properly
dressed for running, but I’m going to do a
little run around here. OK, I just felt my
wrist vibrate which means that Strava should
have started up on my watch. So let’s go over
here and take a look. And there we go. So you can see that Strava
is now recording my run. And I’ll post that
on Strava later so I hope you can
give me some kudos. Here you go, Muz. MUZ MOSTOFI: Awesome. So we got a couple
of really exciting– well three, actually,
really exciting new features going on Android Wear
towards the end of the year. We’ve added a Beat to Beat API
in Android M. What this allows you to do is get the heart
rate time stamps in addition to the BPMs. And compute heart
rate variability, which is a proxy for
stress levels for the user. Second, we’re
adding a profileApi. This allows you
to easily onboard you new users by reading age,
gender, height, and weight from the Fit platform. And, finally,
GoalsApi allowing you not only to show, for
example, just the step count, but also the progress
against that step goal. But also active time,
distance, and calorie goals as well from Google Fit. There’s another exciting
feature that some of guys might have seen at
the sandbox already. so when I go to the gym I
notice that a lot of people are still using
an old school pen and paper to log their exercise,
their reps, their weight. Seems a bit crazy in 2016. Well Google fit on Android
Wear did count reps for you for three specific exercises,
push-ups, sit-ups, and squats, but only if you told the app
in advance what you were doing. This is still somewhat
limited and cumbersome. Now we’ve got something better. So I’m going to show you a demo. DAN AMINZADE: Oh, come on, Muz. MUZ MOSTOFI: Oh, all right. All right. OK. Maybe. Oh. DAN AMINZADE: Give me a break. MUZ MOSTOFI: OK, so I’m going
to the Google Fit strength training section. And I haven’t told it the
exercise I’m about to do. DAN AMINZADE: Come on. Woo! He’s working hard for you guys. MUZ MOSTOFI: As you can see it
detected bicep curls with 100%. [APPLAUSE] And counted the reps. It was 15 pounds so
I’ll just log that. Let’s do another exercise,
one which involves the legs. Note that I can take some
time just kind of setting up. DAN AMINZADE: Very good form. MUZ MOSTOFI: That’s not. [LAUGHTER] OK. I’ll put these down. And you can see it got deadlift. DAN AMINZADE: Yay. [APPLAUSE] MUZ MOSTOFI: If a rep is missed,
or an exercise isn’t correct, for whatever reason, we
will learn from that mistake and become trained to the
user’s exercise motion. We aim to seamlessly track a
whole workout with minimal user input. We’re planning an API
towards the end of this year and we’re looking for partners. DAN AMINZADE: All right. So that’s our overview of
the Google Fit platform. Before we wrap up we just want
to thank all of our partners who are already using the
Google Fit platform to inspire and motivate people
around the world to lead more healthy
and active lifestyle. You know, at the end of the
day this talk isn’t really about pending intents
and broadcast receivers, it’s about improving lives
and extending life spans. And so we really
hope that you’ll be inspired to get
out there and build the next generation of awesome,
amazing fitness experiences on Android. So thanks so much for
your time and attention. Feel free to contact
us with any questions. Thank you. [APPLAUSE] [MUSIC PLAYING]

You may also like...

Leave a Reply

Your email address will not be published. Required fields are marked *