James's Blog

Sharing random thoughts, stories and ideas.

Personal Analytics

Posted: Jan 26, 2019
◷ 3 minute read

Despite being in the era of “Big Data”, we have very little data on our own personal lives. Of course we know a lot about ourselves anecdotally and qualitatively, such as what food we like, but in terms of quantitative data, we don’t really have any beyond the basic, trivial ones (e.g. age, height, weight). How many calories did I eat today? How many hours did I actually truly focus on something mentally today? It’s not too much of an exaggeration to say that we know more about our cars from the dashboard than about ourselves.

To be fair, we are starting to be able to measure some of the things about ourselves. With the rise of smartphones and wearables, the following personal data has become relatively easy to track:

  • Number of steps taken
  • Flights of stairs climbed
  • Number of hours slept, with estimated sleep quality
  • Estimated caloric expenditure
  • Estimated hours standing, walking, sitting
  • Various data about the heart, including average BPM at rest, average BPM while walking/exercising, heart rate variability
  • Location data, including places visited and duration of stay
  • Some basic financial data, such as rough money expenditure breakdowns
  • Smart phone and computer usage data, including breakdown by app and website

But there are many other things that I’d like to know and track about myself, that are currently infeasible to measure non-intrusively. Some of these include:

  • Caloric intake per meal
  • Detailed nutritional breakdown of food consumed (e.g. amount of protein, vitamins, etc…)
  • Various vital health data, including blood sugar levels, blood oxygen levels, body temperature, hydration data, breathing patterns
  • More accurate and detailed data on sleep quality, with breakdown of the different stages of sleep
  • Various data on mental activities and states, including focus levels, general cognitive performance estimates, mood variations
  • General day-to-day activity data, including breakdown of electronics usage, reading, conversations etc…

One might ask, what is the point of measuring all this data, won’t we just become slaves to these numbers? That is definitely a good question, and a valid risk. But I think the pros of knowing these things outweigh the cons, especially if a healthy mindset about the data is set. Apart from satisfying the morbid curiosity that we might have about ourselves, these comprehensive personal analytics can be used in many ways. For example, we can use this data to find patterns and try to improve the different aspects of our lives. You can fine tune your diet based on how various foods affect your sleep, cognition, and mood, creating a truly personalized food guide. In addition, even without the deliberate effort to look at and analyze the data, just having the information tracked in the background can be useful in surfacing red flag alerts, such as when a pathological trend is detected (e.g. heart condition) and intervention is needed.

We are quite a long way from getting to that point however. Beside the inability to track many pieces of the data, there are several other issues with the state of personal data analytics today. One is simply that the data is not centralized in one place, for ease of access/analysis. Apple has done a great job at consolidating the health related data, but outside of that, one would have to resort to various third party solutions, resulting in siloed data that cannot work with each other. And if this data centralization issue is somehow solved, the next problem would arise, which is the matter of privacy. These personal analytics go way beyond what any company today knows about us, and I think they should stay only in the hands of the individuals. Unfortunately this point may disincentivize people from building the centralized solution to the previous issue. Perhaps techniques such as differential privacy can be utilized, so the data centralizer can benefit from everyone’s data without violating people’s privacy. After all, there is tremendous value in seeing all this data at the population level. We may finally be able to answer questions such as “what do people who live to 100 have in common”.

The field of personal well-being technology is starting to bloom today. With the increase in investments on wearables focused on fitness, and the growing popularity of various health-related movements (e.g. body-hacking), we are in need of a central, private repository for our personal analytics data. I have a feeling that years from now, when people look back at our era, they will be astonished at how we lived so recklessly without even the most basic data about ourselves, just like we find it difficult to imagine how people in the 1800s lived without modern plumbing.