I like Paint.NET. It's a free graphics program for Windows. Nothing fancy, but it has layers, filters, and a few tools. It's also way easier than GIMP. I thought some gesture-enabled painting would be fun so I whipped up a quick set of key mappings for it with Myo Connect's build in Keyboard Mapper. For this weeks #MyoCraft, we're going to talk about the resulting script. Yes, that's right, if you make some mappings you like, you can export them into the Application Manager so they are automatically active whenever you use that application later. How does it do that? By dynamically writing a Myo Script for you! That's pretty cool on it's own, but there are some neat techniques being used in the scripts themselves as well. Let's dive in!
Using the keyboard mapper was pretty straightforward. I set fist to click and hold the left mouse button, fingers spread to undo, wave left to select my brush, wave right to select the eraser, and double tap to toggle mouse and unlock.
I exported it, making sure it was only active on the Paint.Net application. This was the result:
scriptId = 'com.thalmic.keyboardmapper.Paint.Net'
minMyoConnectVersion = '0.9.0'
scriptTitle = 'Paint.Net'
function activeAppName()
return 'Paint.Net'
end
----------------------------
-- Helpers
function conditionallySwapWave(pose)
if myo.getArm() == 'left' then
if pose == 'waveIn' then
pose = 'waveOut'
elseif pose == 'waveOut' then
pose = 'waveIn'
end
end
return pose
end
unlockType = ''
function unlock(type)
unlockType = type
myo.unlock(unlockType)
end
function getUnlockType()
return unlockType
end
keyPressSuspendedUnlockTimer = false
function keyPress(key, edge, ...)
if edge == 'down' and getUnlockType() == 'timed' then
unlock('hold')
keyPressSuspendedUnlockTimer = true
end
myo.notifyUserAction()
myo.keyboard(key, edge, ...)
if edge == 'up' and keyPressSuspendedUnlockTimer then
unlock('timed')
keyPressSuspendedUnlockTimer = false
end
end
mouseClickSuspendedUnlockTimer = false
function mouseClick(button, edge, ...)
if edge == 'down' and getUnlockType() == 'timed' then
unlock('hold')
mouseClickSuspendedUnlockTimer = true
end
myo.notifyUserAction()
myo.mouse(button, edge, ...)
if edge == 'up' and mouseClickSuspendedUnlockTimer then
unlock('timed')
mouseClickSuspendedUnlockTimer = false
end
end
-- Toggles unlock and keeps mouse control synchronized with unlock state
enabledMouseOnUnlock = false
function unlockAndControlMouse()
unlock('hold')
enabledMouseOnUnlock = true
myo.controlMouse(true)
end
function onLock()
-- If mouse was enabled on unlock then disable
-- it regardless of how the lock was issued
if enabledMouseOnUnlock then
enabledMouseOnUnlock = false
myo.controlMouse(false)
end
end
function onDeactivate()
-- Disable mouse control so that we always resume
-- this script with mouse control disabled
if enabledMouseOnUnlock then
enabledMouseOnUnlock = false
myo.controlMouse(false)
end
end
----------------------------
-- Map poses to functions
LOCKED_BINDINGS = {
doubleTap_on = function() unlockAndControlMouse() end
}
UNLOCKED_BINDINGS = {
waveOut_on = function() keyPress('e', 'press') end,
waveIn_on = function() keyPress('b', 'press') end,
fist_on = function() mouseClick('left', 'down') end,
fist_off = function() mouseClick('left', 'up') end,
fingersSpread_on = function() keyPress('z', 'press', "control") end,
doubleTap_on = function() myo.lock() end
}
function currentBindings()
if myo.isUnlocked() then
return UNLOCKED_BINDINGS
else
return LOCKED_BINDINGS
end
end
function onPoseEdge(pose, edge)
pose = conditionallySwapWave(pose)
fn = currentBindings()[pose .. '_' .. edge]
if fn then
fn()
end
end
---------------------------
-- Script activation handling
function onForegroundWindowChange(app, title)
return platform == 'Windows' and
string.match(app, 'PaintDotNet.exe')
end
---------------------------
-- Set how the Myo Armband handles locking
myo.setLockingPolicy('none')
function onActiveChange(isActive)
if not isActive then
onDeactivate()
end
end
This was my first attempt at painting with my Myo armband. You can tell it's art because it says so right there in the picture.
This should look fairly familiar if you are used to our function binding approach. Unlike our previous examples, however, it's actually USING the binding system to have two separate sets of bindings: LOCKED_BINDINGS and UNLOCKED_BINDINGS. This, naturally, makes it easy to have a different set of bindings for when the Myo armband is locked or unlocked. We do this to let you set any gesture to unlock your Myo armband. The correct binding to use is determined in the currentBindings function based on whether the Myo armband is unlocked or not:
function currentBindings()
if myo.isUnlocked() then
return UNLOCKED_BINDINGS
else
return LOCKED_BINDINGS
end
end
function onPoseEdge(pose, edge)
pose = conditionallySwapWave(pose)
fn = currentBindings()[pose .. '_' .. edge]
if fn then
fn()
end
end
So if it's locked, it will return LOCKED_BINDINGS. If not, you get UNLOCKED_BINDINGS. Then we pull the appropriate function out of that list based on what pose and edge comes in. If it's a valid function (ie, an unlock pose if it's locked, or a pose bound to lock or a key press if it's not), we call it.
Here is my second attempt. Note the subtle use of like, 8 colours. The roof is orange because I couldn't figure out how to make a good brown. Let's just say it's not the Myo holding me back at this point.
The other useful thing I'd like to point out is the inline function definitions. For example, here's LOCKED_BINDINGS:
LOCKED_BINDINGS = {
doubleTap_on = function() unlockAndControlMouse() end
}
The function for doubleTap_on is being defined right there in the array itself. We can do this because functions are first class values in Lua. Remember that the normal function definition syntax of something like
function getUnlockType()
return unlockType
end
is actually just some syntactic sugar for getUnlockType = function () return unlockType end, which is exactly what were are doing in the array. Doing it this way saves some space and makes the script generation less complicated. You may not necessarily want to write your OWN scripts like that, but it certainly does simplify typical pose-key press actions.
So that's how the sausage is made. The Keyboard Mapper isn't doing anything that you couldn't already do with a Myo Script, it just makes it dead simple to map poses to key presses. It doesn't touch the IMU at all, but with a bit of knowledge there is no reason you couldn't start with a basic Keyboard Mapper script and add some motion controls yourself.
This is why they pay me to write. I had a lot of fun painting, but when I asked one of our artists to give it a try she used her Myo armband to whip up some actual art.
That's it for now! Don't forget to submit your projects to MyoCraft@thalmic.com.
Otherwise, see you next week!