North (formerly Thalmic Labs), the creator of the Myo armband, was acquired by Google in June 2020. Myo sales ended in October 2018 and Myo software, hardware and SDKs are no longer available or supported. Learn more.
This is a guest post written by Ramesh Somasundaram from Irving, Texas. He is an indie dev behind the RoMyo app and the founder of Appestry.
Ever since I saw Tom Cruise scrub images by just waving his hands around in the 2002 sci-fi thriller Minority Report, I always wanted to make things happen by just the wave of my hand! That dream came true when I attended a webinar called Myo Development 101 hosted by Chris Goodine during the 2015 AT&T Bootstrap week. During that webinar, I couldn’t stop thinking of all the ways I could use Myo to make things happen by integrating it with Android and iOS. Having built several apps for Google Play and App Store I saw this as my opportunity to gesture control everything through IoT!
I discovered Myo at the right moment because I was already working on a mobile app to control the Roku streaming media player from an Android Wear smart watch. Originally, I was going to use the sensors of the smart watch to recognize gestures but it proved to be too tricky. So I published the smart watch app with only a push button remote control and saved the gesture recognition for Myo!
Enter RoMyo – control your Roku with Myo! Roku SDK with its well defined command set was an ideal launch pad to go live with Myo. After I got my Myo armband, I jumped right into the SDK to see how it worked with my phone. The sample code was easy enough for me to quickly modify it to forward the gesture input to Roku. I won’t forget the first time when I paused the video on my Roku using just a hand gesture! It was very exciting though I was merely scratching the surface.
Next, I wanted to map all the 12 commands of the physical Roku remote control to gestures because I didn’t want to build an app that couldn’t replace the remote completely. I mapped the four basic commands – play/pause, fast forward, rewind and instant replay – to the four basic gestures – spread fingers, wave right, wave left and fist. That was easy! The remaining 8 remote control commands were primarily used for navigating the Roku menus. I tried to combine fist gesture with motion recognition but that proved to be tricky at times. I had trouble tracking the relative pitch and yaw while tracking the gesture at the same time. I wanted a gesture that was easy to reproduce and reliable at the same time.
Chris introduced me to the concept of toggle zones in his 10ft Media Experience script. It uses the pitch of the Myo armband to find out if the hand is pointed up. I used the onOrientationData callback and Quaternion to find the pitch of the armband. After testing several different positions I arrived at a threshold for the pitch at > 0.9 for the hand pointing upward and at < -0.5 for the hand pointing downward. I negated the polarity of the pitch using the armband’s XDirection whenever the armband was worn upside down. Combining the two toggle zones (hand pointing up and down) with the 4 basic gestures, I now had my 8 navigation commands!
When I put RoMyo app through its paces I noticed that the armband would lock itself before I was done using it. For example, if I was fast forwarding for 5 seconds to skip commercials the default 2 second timeout would lock the armband before I could gesture to resume playing the TV show. I realized that I need to override the standard locking policy. So every time I detected a double tap to unlock the armband I would hold it from locking itself until a custom timeout (using a Handler) or until another double tap to lock it. The custom lock timeout can be adjusted to any value between 2 seconds and 1 minute in the app. I also provided a setting in the app to ignore the double tap for locking and solely use the custom timeout for locking.
Though I had only one Myo armband to test with, I wanted to pair up to four armbands to control the same Roku. Imagine a family of four controlling their TV, each with their own armband! So I took the attachToAdjacentMyos route to pair armbands instead of attaching by MAC addresses or using the ScanActivity as in the HelloWorld sample. This also made it easy to pair the armbands by just tapping the phone on the armband. The MultipleMyos sample app helped me extend the functionality to multiple armbands. Scott Greenberg who reviewed the app before publishing it was instrumental in testing the functionality using multiple armbands. I also added an option to ignore one or more of the armbands just in case a family member wanted to take control of the TV! What can go wrong?!
RoMyo app can be used to control multiple Roku players (one at a time) in the Wi-Fi network. In case of a network issue when the command doesn’t reach the Roku player the armband would long-vibrate to let the user know the connection issue.
After publishing RoMyo app under Appestry in Google Play, I am currently working on an iOS version of the app as well as testing and integrating the Myo armband with all things IoT – Amazon Echo, Fire TV, Chromecast, etc.
You can find more details on how to use the RoMyo app in this help page. Let the gesture control begin!