To many, the Sutro time-stamp was moving around like a phantom. Sometimes it would be in the future, other times it would be in the past. Sometimes it would be within minutes of when you looked at it last. It was frustrating for both us and our beta testers since it made a lot of other functionality misbehave.
This felt like our basic physics class where we learned that for an electron, you can’t know where it is and the speed at the same time. It’s all relative.
Anyways, this post digs a bit deeper into the issue, what we fixed, and what went wrong. If you don’t want the details, then you can read the TL;DR below (Too Long; Did Not Read).
TL;DR
- The Sutro stack is a complex road from monitor >> hub >> server >> to your phone.
- Each link in that telephone chain has ways to calculate the time
- The time gets sent up
- , and each element on the chain was not properly talking to each other
- Furthermore, the way that we were looking at UTC (being in the future of pacific time) was affecting how we displayed time to you
- Thus you saw readings in the future
The Sutro Technology Stack
Let us Start with the Monitor Hardware:
There are two crystal oscillators on the device. First, we’ll quickly explain what a crystal oscillator even is.
A crystal oscillator is a small piece of crystal (just like quartz) that uses the mechanical resistance of crystal to vibrate at a given frequency. Kind of like this bridge vibrating at a given frequency. When it vibrates at this given frequency, we know what that frequency is, and so we can count time.
- External crystal oscillator: this is the crystal oscillator on the outside of the PCB (the electronics board)
- Internal crystal oscillator: this is the crystal that is on the inside of the PCB (electronics board).
We won’t go into the nitty-gritty of why we selected one over the other. Still, we ended up realizing that the internal was not as accurate as of the other, and so we moved to the external oscillator. Due to condensation on the earlier models, we ended up having issues with waterproofing on the board outside – and so that started to affect the board. Thanks to Himanshu at Aethon Labs and the fantastic firmware developers at Shenzhen Valley Ventures for getting to the bottom of this.
Firmware on the Monitor
Now, let us move to the firmware on the monitor. The firmware is basically ‘computer code’ that lives on the monitor. It has a tiny computer chip (an integrated circuit) that houses all the code that controls the monitor.
The firmware on the monitor was also incorrect, and instead of taking the entire YYYY-MM-DD-HH-SS-MS, this is the date-time, we were chopping off the date in running our calculations from one side to the other. So in a sense, the device always thought it was the same day every day. It would just keep spinning in a 24-hour cycle.
Firmware on the Hub
The monitor connects to the hub through a sub-gigahertz link. We have what’s called ‘retries’, to make sure that the message properly gets from the monitor to the hub. Now, if the monitor is unable to get the message to the hub, it will retry a few times. But when it’s not able to get the message at the first try, it stores it in its memory with a time-stamp of when it took that reading. Because we have inherent issues with the oscillators, to begin with, that time was off.
The Server (back-end) -> Your Phone
Now we took a test, traced it to the hub, and finally sent it to the Server. The packets are living in the cloud, and are ready to get diced up and sent to your phone. So your beautiful eyes can see what the chemistry is, treat your pool (or spa)… if you need to, and go swimming!
Kelli Rockwell from our development firm, Grio, really helped get to the bottom of it. It was a bit gruesome, but, here’s where it gets more interesting:
- The reading_time, which is what we use to mark the time that the reading was taken, was a little off (due to the issues I mentioned earlier)
- This wouldn’t usually be a problem if we were interpreting the time-stamps as UTC, but React Native (the language that we use) uses a different “JS runtime” for Android that it does for iOS, which meant that the time-stamp was being interpreted as UTC on one platform and the local timezone on the other.
- Since UTC is ahead of Pacific Time, this made the time-stamps look like they were in the future!
- There was also a problem in our logic that determines if we should format the time-stamp as today at X PM or yesterday at X PM, or just as a time-stamp.
- We were computing the time-stamps in UTC, so if reading was taken at 5 PM PT, that’s 12 AM UTC tomorrow. If you looked at the app tomorrow at 10 AM the logic is going to say today is the same date as the time-stamp, and it’s in UTC; it’ll show “Today at 5 PM” (in the future!).
Time is Relative Afterall
As you can see, when you have so many ways to keep and tell time, it can be confusing. This is a common problem with all sorts of systems. Even rocket scientists at NASA have time problems, so we feel a little bit better.
I hope you enjoyed reading, and this should be out in the next fix!