The enabling factor here compared to the last time I tried this (quite some time ago) is the addition of custom descriptor support in iWRAP5. I don’t know of any other module which allows the degree of simple customizability in this area that the WT12 does. It was the best module I saw two years ago, and the new features are icing on the cake. (Yes, I have been working at Bluegiga since last October, but bear in mind I was singing this module’s praises long before that started.)

So how does this configuration actually work? There are three important pieces involved:

1. Building the HID descriptor
2. Configuring iWRAP
3. Sending HID report packets

Naturally, I will explain each in order.

Building the HID descriptor

To be honest, this is the part I am least confident about. I have learned a lot about HID descriptors, but only enough to be dangerous instead of simply clueless. In short, they are sent from a device to a host during the connection establishment phase so that the host will know what kind of data to expect from the device, how it will be formatted, and what it should do with it. A HID descriptor is built using a predefined binary “language” of sorts; data bytes either denote a field or setting, or else are the byte(s) contained within the previously denoted field or setting. There are many aspects to these definitions, and I don’t fully understand them all yet. It will be much better if I point you to the resources that were most helpful for me:

• Step 13 of Frank Zhao’s USB Wii Classic Controller Instructable
• Frank Zhao’s USnooBie HID Report tutorial (way to go, Frank!)
• The USB HID device class specification (PDF)
• The USB HID usage tables specification (PDF)
• The USB HID Descriptor Tool
• USBlyzer for complete descriptor analysis of an existing USB device
• The raw HID descriptor structs found in PJRC’s Teensyduino source

Actually, the first and last links ended up being the most practically useful to me. The others are good for cross-referencing things or getting into a detailed understanding. It’s a great list to get started with if you’re looking for HID development resources. Also, USBlyzer can be amazing if you are trying to mimic the behavior of an existing wired USB device, since it can show you both the descriptor structure as well as the actual HID report traffic.

But what’s all this about USB? Wasn’t the whole point of this to be all wireless and Bluetoothy and stuff? Well, it turns out that the important bits of device-specific USB HID descriptors pretty much apply directly to Bluetooth HID, because as best as I can tell, Bluetooth HID is basically USB HID wrapped in a wireless transport layer. It isn’t exactly the same, of course, but the most important pieces—namely, the HID descriptor and corresponding HID reports—match.

Now, what about Bluegiga’s HID documentation? There is a HID application note for iWRAP which is quite good if you are interested in single-function devices, or if you know what you are doing with descriptors. But if you’re only good at following directions and you want a combo device, it’s less obvious. (I will be fixing this shortly, because now I can!) There are example descriptors supplied for a standard keyboard, standard mouse (X/Y + 3 buttons), consumer page control (e.g. calculator and music controls), and a gamepad. But what if you want a keyboard/mouse/gamepad device all in one?

It turns out that this is really not difficult at all. There are two basic rules you have to follow:

1. Take each individual descriptor and mash them all together into one big long one.
2. Make sure each report structure definition includes a “Report ID” byte (0×85) and unique value (0×01 or higher).

The first one is easy. The second one is not so easy if you don’t know where the Report ID byte should go. Based on my tests, it needs to go after the innermost “Collection” start field (0xA1 0xnn) somewhere. I don’t know what the rule is exactly, but I put them (when they weren’t already there) immediately after this “Collection” start field. Each report needs its own unique ID so that the host device knows which kind of report you’re sending, whenever you send one. Pretty straightforward.

Here is the actual descriptor I put together for my successful test last night, in C struct form:

static const uint8_t hid_descriptor_combined[] = {

    // KEYBOARD

    /****/ 0x05, 0x01, /* USAGE_PAGE (Generic Desktop) */
    /****/ 0x09, 0x06, /* USAGE (Keyboard) */
    /****/ 0xa1, 0x01, /* COLLECTION (Application) */
    /******/ 0x05, 0x07, /* USAGE_PAGE (Keyboard) */
    /******/ 0x85, 0x01,   /* REPORT_ID (1) */
    /* 1 byte Modifier: Ctrl, Shift and other modifier keys, 8 in total */
    /******/ 0x19, 0xe0, /* USAGE_MINIMUM (kbd LeftControl) */
    /******/ 0x29, 0xe7, /* USAGE_MAXIMUM (kbd Right GUI) */
    /******/ 0x15, 0x00, /* LOGICAL_MINIMUM (0) */
    /******/ 0x25, 0x01, /* LOGICAL_MAXIMUM (1) */
    /******/ 0x75, 0x01, /* REPORT_SIZE (1) */
    /******/ 0x95, 0x08, /* REPORT_COUNT (8) */
    /******/ 0x81, 0x02, /* INPUT (Data,Var,Abs) */
    /* 1 Reserved byte */
    /******/ 0x95, 0x01,  /* REPORT_COUNT (1) */
    /******/ 0x75, 0x08, /* REPORT_SIZE (8) */
    /******/ 0x81, 0x01, /* INPUT (Cnst,Ary,Abs) */
    /* LEDs for num lock etc */
    /******/ 0x95, 0x05, /* REPORT_COUNT (5) */
    /******/ 0x75, 0x01, /* REPORT_SIZE (1) */
    /******/ 0x05, 0x08, /* USAGE_PAGE (LEDs) */
    /******/ 0x85, 0x01, /* REPORT_ID (1) */
    /******/ 0x19, 0x01, /* USAGE_MINIMUM (Num Lock) */
    /******/ 0x29, 0x05, /* USAGE_MAXIMUM (Kana) */
    /******/ 0x91, 0x02, /* OUTPUT (Data,Var,Abs) */
    /* Reserved 3 bits */
    /******/ 0x95, 0x01, /* REPORT_COUNT (1) */
    /******/ 0x75, 0x03, /* REPORT_SIZE (3) */
    /******/ 0x91, 0x03, /* OUTPUT (Cnst,Var,Abs) */
    /* Slots for 6 keys that can be pressed down at the same time */
    /******/ 0x95, 0x06, /* REPORT_COUNT (6) */
    /******/ 0x75, 0x08, /* REPORT_SIZE (8) */
    /******/ 0x15, 0x00, /* LOGICAL_MINIMUM (0) */
    /******/ 0x25, 0x65, /* LOGICAL_MAXIMUM (101) */
    /******/ 0x05, 0x07, /* USAGE_PAGE (Keyboard) */
    /******/ 0x19, 0x00, /* USAGE_MINIMUM (Reserved (no event indicated)) */
    /******/ 0x29, 0x65, /* USAGE_MAXIMUM (Keyboard Application) */
    /******/ 0x81, 0x00, /* INPUT (Data,Ary,Abs) */
    /****/ 0xc0, /* END_COLLECTION */

    // CONSUMER PAGE

    /****/ 0x05, 0x0c, /* USAGE_PAGE (Consumer) */
    /****/ 0x09, 0x01, /* USAGE (Consumer Control) */
    /****/ 0xa1, 0x01, /* COLLECTION (Application) */
    /******/ 0x85, 0x02, /* Report ID 2 */
    /******/ 0x05, 0x0c, /* USAGE_PAGE (Consumer) */
    /* 8 media player related keys */
    /******/ 0x15, 0x00, /* Logical Min 0 */
    /******/ 0x25, 0x01, /* Logical Max 1 */
    /******/ 0x09, 0xe9, /* Usage (8-bit), Volume Increment */
    /******/ 0x09, 0xea, /* Usage (8-bit), Volume Decrement */
    /******/ 0x09, 0xe2, /* Usage (8-bit), Mute */
    /******/ 0x09, 0xcd, /* Usage (8-bit), Play/Pause */
    /******/ 0x19, 0xb5, /* Usage Min (Scan Next Track, Scan Previous Track, Stop, Eject) */
    /******/ 0x29, 0xb8, /* Usage Max */
    /******/ 0x75, 0x01, /* Report Size */
    /******/ 0x95, 0x08, /* Report Count */
    /******/ 0x81, 0x02, /* Input type 2 */
    /* 8 application control keys */
    /******/ 0x0a, 0x8a, 0x01,/* Usage (16-bit), LSB first, e.g. this is 0x018a Email Reader */
    /******/ 0x0a, 0x21, 0x02,/* Usage (16-bit), Generic GUI Application Control Search */
    /******/ 0x0a, 0x2a, 0x02,/* Usage (16-bit), Application Control Bookmarks */
    /******/ 0x1a, 0x23, 0x02,/* Usage Min (16-bit), AC Home, Back, Forward, Stop, Refresh */
    /******/ 0x2a, 0x27, 0x02,/* Usage Max (16-bit) */
    /******/ 0x75, 0x01, /* Report Size */
    /******/ 0x95, 0x08, /* Report Count */
    /******/ 0x81, 0x02, /* Input type 2 */
    /* Application launch keys + record & rewind */
    /******/ 0x0a, 0x83, 0x01,/* Usage (16-bit), Application Launch Generic Consumer Control */
    /******/ 0x0a, 0x96, 0x01,/* Usage (16-bit), AL Internet Browser */
    /******/ 0x0a, 0x92, 0x01,/* Usage (16-bit), AL Calculator */
    /******/ 0x0a, 0x9e, 0x01,/* Usage (16-bit), AL Terminal Lock / Screensaver */
    /******/ 0x0a, 0x94, 0x01,/* Usage (16-bit), AL Local Machine Browser */
    /******/ 0x0a, 0x06, 0x02,/* Usage (16-bit), AC Minimize */
    /******/ 0x09, 0xb2, /* Usage (8-bit), Record */
    /******/ 0x09, 0xb4, /* Usage (8-bit), Rewind */
    /******/ 0x75, 0x01, /* Report Size */
    /******/ 0x95, 0x08, /* Report Count */
    /******/ 0x81, 0x02, /* Input type 2 */
    /******/ 0xc0,      /* End Collection */

    // MOUSE

    0x05, 0x01,       // USAGE_PAGE (Generic Desktop)
    0x09, 0x02,       // USAGE (Mouse)
    0xa1, 0x01,       // COLLECTION (Application)
    0x09, 0x01,       //   USAGE (Pointer)
    0xa1, 0x00,       //   COLLECTION (Physical)
    0x85, 0x03,       //     REPORT_ID (3)
    0x05, 0x09,       //     USAGE_PAGE (Button)
    0x19, 0x01,       //     USAGE_MINIMUM (Button 1)
    0x29, 0x03,       //     USAGE_MAXIMUM (Button 3)
    0x15, 0x00,       //     LOGICAL_MINIMUM (0)
    0x25, 0x01,       //     LOGICAL_MAXIMUM (1)
    0x95, 0x03,       //     REPORT_COUNT (3)
    0x75, 0x01,       //     REPORT_SIZE (1)
    0x81, 0x02,       //     INPUT (Data,Var,Abs)
    0x95, 0x01,       //     REPORT_COUNT (1)
    0x75, 0x05,       //     REPORT_SIZE (5)
    0x81, 0x03,       //     INPUT (Cnst,Var,Abs)
    0x05, 0x01,       //     USAGE_PAGE (Generic Desktop)
    0x09, 0x30,       //     USAGE (X)
    0x09, 0x31,       //     USAGE (Y)
    0x09, 0x38,       //     USAGE (Wheel)
    0x15, 0x81,       //     LOGICAL_MINIMUM (-127)
    0x25, 0x7f,       //     LOGICAL_MAXIMUM (127)
    0x75, 0x08,       //     REPORT_SIZE (8)
    0x95, 0x03,       //     REPORT_COUNT (3)
    0x81, 0x06,       //     INPUT (Data,Var,Rel)
    0x05, 0x0c,       //     USAGE_PAGE (Consumer Devices)
    0x0a, 0x38, 0x02, //     USAGE (Undefined)
    0x95, 0x01,       //     REPORT_COUNT (1)
    0x81, 0x06,       //     INPUT (Data,Var,Rel)
    0xc0,             //   END_COLLECTION
    0xc0,              // END_COLLECTION

    // RAW 16-BYTE I/O

    0x06, 0xAB, 0xFF,   // Usage Page (Vendor-Defined 172)
    0x0A, 0x00, 0x02,   // Usage (Vendor-Defined 512)
    0xA1, 0x01,         // Collection (Application)
    0x85, 0x04,         /* REPORT_ID (4) */
    0x75, 0x08,         // Report Size (8)
    0x15, 0x00,         // Logical Minimum (0)
    0x26, 0xFF, 0x00,   // Logical Maximum (255)
    0x95, 0x10,         // Report Count (16)
    0x09, 0x01,         // Usage (Vendor-Defined 1)
    0x81, 0x02,         // Input (Data,Var,Abs,NWrp,Lin,Pref,NNul,Bit)
    0x95, 0x10,         // Report Count (16)
    0x09, 0x02,         // Usage (Vendor-Defined 2)
    0x91, 0x02,         // Output (Data,Var,Abs,NWrp,Lin,Pref,NNul,NVol,Bit)
    0xc0,               // End Collection
};

Pardon the mix of comment formatting. I pulled these together from multiple sources and built one of them by hand. The structure of this descriptor is that the standard keyboard report has ID 1, consumer page report has ID 2, mouse report has ID 3, and raw generic 16-byte data packet has ID 4.

However, the above isn’t directly applicable to iWRAP configuration on the WT12 module, since iWRAP doesn’t use C structs as settings. This leads us to…

Configuring iWRAP

Luckily for all of us, the factory default iWRAP5 firmware—currently v5.0.1 build 620—supports all the HID customization we need. (It is also freely user-upgradable if you have or want a different build for whatever reason.) The default settings enable only the serial port profile (SPP), so we have to turn on the HID profile specifically, and write our descriptor. The maximum descriptor length is 255 bytes at the moment. For simplicity, here’s the full set of relevant initialization commands that I’m sending for the Keyglove’s Bluetooth module:

SET BT NAME Keyglove
SET BT CLASS 00540
SET BT IDENT USB:1d50 6025 1.0.0 Keyglove Input Device
SET BT SSP 3 0
SET CONTROL CD 80 2 20
SET CONTROL ESCAPE - 40 1
SET PROFILE HID d 40 100 0 en 0409 Keyglove Input Device
HID SET F2 05010906A1010507850119E029E715002501750195088102950175088101950575010508850119012905910295017503910395067508150025650507190029658100C0050C0901A1018502050C1500250109E909EA09E209CD19B529B87501950881020A8A010A21020A2A021A23022A27027501950881020A83010A96010A92010A9E010A94010A060209B209B4750195088102C005010902A1010901A10085030509190129031500250195037501810295017505810305010930093109381581257F750895038106050C0A380295018106C0C006ABFF0A0002A10185047508150026FF00951009018102951009029102C0

That ridiculously long “HID SET” line at the end is the descriptor itself. The lone preceding “F2″ value is the length (in hex) of the whole thing, which is 242 bytes. I’m right at the edge of the size limit with this one.

Here’s a quick breakdown of these settings (see the iWRAP5 User Guide from Bluegiga for more detail):

• SET BT NAME Keyglove

  - Sets the Bluetooth device friendly name

• SET BT CLASS 00540

  - Sets the device class to “keyboard” (important for iOS compatibility)

• SET BT IDENT USB:1d50 6025 1.0.0 Keyglove Input Device

  - Sets the VID/PID, version, and self-identifying description

• SET BT SSP 3 0

  - Enables PIN-less “just works” secure simple pairing

• SET CONTROL CD 80 2 20

  - Enables GPIO output signals for active link and DATA mode status

• SET CONTROL ESCAPE - 40 1

  - Enable GPIO input control for exiting DATA mode

• SET PROFILE HID d 40 100 0 en 0409 Keyglove Input Device

  - Sets the HID options to be a keyboard class device, English layout, no localization, version 1.0.0

• HID SET F2 ...

  - Explained above. Sets the actual HID descriptor.

Not all of these are strictly necessary for the HID functionality I’m going for (e.g. the “SET CONTROL …” commands and BT NAME), but the rest are. Note also that all of these persist through power cycles, since all of them are stored in non-volatile memory. Once these settings are applied the first time, you must power-cycle the module or issue a “RESET” command to make them take effect—particularly the newly enabled HID profile and HID descriptor.

Now we’re ready to actually send some data.

Sending HID report packets

iWRAP uses a specific encapsulation format for sending raw HID packets. It will actually let you send basic ASCII keyboard characters as plain bytes over UART—so the byte ‘a’ (0×61) sends the keyboard HID reports for pressing and releasing the ‘a’ key—but only if you have the standard keyboard descriptor included at the beginning of your full HID descriptor. As it happens, we do. But full control only comes through the use of raw HID packets, so that’s what we’ll be using here. The raw HID format, as described in the iWRAP HID app note, is always of this form:

0x9F [length] [data ... ... ...]

Where 0x9F is the start-of-frame byte, [length] is how many data bytes follow, and [data ...] is the actual data itself. So, for a 6-byte packet example, it might be formatted like this:

0x9F 0x06 0xAA 0xBB 0xCC 0xDD 0xEE 0xFF

The above packet is invalid given our descriptors, but it illustrates the point. Remember that we have four unique reports in our combo descriptor above, so there are four possible kinds of raw reports we might send:

1. Keyboard

   0x9F 0x0A 0xA1 0x01 [modifier] 0x00 [key1] [key2] [key3] [key4] [key5] [key6]

2. Consumer page

   0x9F 0x05 0xA1 0x02 [field1] [field2] [field3]

3. Mouse

   0x9F 0x07 0xA1 0x03 [buttons] [x-move] [y-move] [v-scroll] [h-scroll]

4. Raw

   0x9F 0x12 0xA1 0x04 [data1 ... data16]

Note the Report ID byte in red. For testing purposes, I like using Realterm because (1) it’s free and (2) it makes sending or displaying binary data in hex format easier than anything else I’ve seen. The UI is definitely engineer-ish, but it doesn’t take too much getting used to, and it has some features that are very hard to do without in cases like these. Realterm has a “Send” tab where you can paste bytes in hex notation (either 0xAA or $AA) and then send them as binary data, which is excellent here.

One important thing to point out here is that for many operations that we tend to think of as a single action, like a keypress or mouse click, there are actually two things going on. One is the press and the other is the release. It is critical that you keep this in mind when sending raw reports. Pressing the ‘a’ key is actually sending one report with the key down and another with no keys down. Here are some examples of each kind of descriptor:

Press and release the ‘a’ key

0x9F 0x0A 0xA1 0x01 0x00 0x00 0x04 0x00 0x00 0x00 0x00 0x00

0x9F 0x0A 0xA1 0x01 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00

Press and release the calculator button

0x9F 0x05 0xA1 0x02 0x00 0x00 0x04

0x9F 0x05 0xA1 0x02 0x00 0x00 0x00

Press and release the left mouse button

0x9F 0x07 0xA1 0x03 0x01 0x00 0x00 0x00 0x00

0x9F 0x07 0xA1 0x03 0x00 0x00 0x00 0x00 0x00

Send 16 bytes of raw data

0x9F 0x12 0xA1 0x04 0x00 0x11 0x22 0x33 0x44 0x55 0x66 0x77 0x88 0x99 0xAA 0xBB 0xCC 0xDD 0xEE 0xFF

(This is a single action and doesn’t require a “release” report)

This successful implementation of complex Bluetooth HID is an excellent next step in the Keyglove development. While the raw HID (as far as I know) is not accessible on iOS devices, this same combo descriptor and iWRAP configuration allows you to connect to a PC, Mac, Linux machine, iPhone, and Android phone—and, I assume and very much hope, Google Glass. We shall see in approximately 10 days.

Until then, have fun with your own Bluetooth HID projects, if you are so inclined!

For one thing, I have now passed the two-year anniversary of the successful Kickstarter campaign. It was one week ago today, in fact: April 27. That is a tremendously long time to wait, for all those of you who backed the project. I am continually encouraged by the ongoing support, and conversely by the conspicuous lack of impatience displayed despite the long, drawn-out development process and the painfully long silences at times (like over the last many months). I want to work on this project, and I want to do it now. This by itself is motivating.

For another thing, although I haven’t had much time to work on the Keyglove, I have still found myself thinking about it quite often and at times doing what could be accurately called brainstorming. Some of the ideas I’ve had should prove to be fruitful once I immerse myself in development again. A corresponding observation to this point is that I think it will be beneficial for me to begin again with the codebase and designs I already have, but with a relatively fresh viewpoint. It will be easy for me to objectively evaluate the systems I have already built and look for any areas which may benefit from certain changes or a re-imagined approach. This kind of analysis is easier to do when you haven’t been neck deep in the project for a while.

Third, there has been quite a bit of relevant innovation in the areas of wearable computing, wireless connectivity, battery technology, and low-power programmable microcontroller systems. Might it be worthwhile to incorporate an Arduino compatible ARM processor instead of the AT90USB MCU I have? Possibly. Bluetooth Low Energy for wireless comms? Very likely, at least as an option. Flexible PCBs? Maybe. New, tiny motion sensor? Sounds great. Bundling everything into a single FPGA? It could be done. The bottom line here is that six or seven months on the embedded system design timescale is significant. There is a lot that has happened in the world since then.

Perhaps the most personally exciting reason for me though is this:

I managed to snag a slot on the new Google Glass Explorers group as part of the #ifihadglass contest that Google put on back in February. I am definitely more excited about this than is healthy for someone as far down the list of early adopters as I am (though I am still thankful that I’m on the list in the first place). My entry tweet was about combining it with the Keyglove, and their reply came a little over a month later. I was (and am) ecstatic.

As it happens, I actually got to try a Glass device back towards the end of January of this year, while having dinner in San Francisco with a friend from college who is now way high up on the Glass development team. It was a pretty short demo, but I was completely enamored. Truth be told, I’d actually had a dream the night before about what it might be like, in subconscious anticipation of that eventuality. I truly can hardly wait for until my turn comes to get a development unit. This is exactly the platform for which I originally created the Keyglove. I don’t know when I will get mine, but I want the Keyglove to be ready to take full advantage of the opportunity.

In my opinion (and others’), Google Glass has tremendous potential. Augmented reality and instant contextual information has a value that we can’t accurately quantify yet. We don’t have enough data to predict with. Glass will allow innovative people to expand the range of possibilities, to give us ideas about things we will find invaluable which today we can’t even imagine since we don’t have a mental framework to put them on. I want the Keyglove to be right there along side this discovery process, solving the input challenge where Glass so (comparatively) eloquently solves the output/display problem of wearable technology.

So, while I’m waiting for Glass…what’s next?

I need to run through the Roadmap again and freshen things up a bit. Then I need to try compiling the last code snapshot I have and see exactly where I left things at the end of September. Then, I will read through the modular code structure I have and see if there are any structural changes to make right off the bat, or if I should go straight to fixing bugs and adding or completing features.

Begone, distractions! I have work to do!

Keyglove #10 – Prototype D Autodesk Inventor COM API Demo from Jeff Rowberg on Vimeo.

This video demonstrates Keyglove Prototype D and the Keyglove Kit v0.4 PCB (v0.5 still pending design completion) along with the alpha Keyglove Manager app integrated with Autodesk Inventor using the COM API for true 3D input. This is just a sample of the kind of thing that can be done with the Keyglove. And, honestly, this isn’t even a very good demo due to the early stage of development of the Manager app and the loose mounting of the older revision of the PCB on the glove.

The sensor connections to the controller board have also been wired into per-finger female headers, which means it’s far easier to swap in a new controller board whenever I update the design. I don’t have to find each individual pin connection and put them in the right order; it’s all conveniently broken out on the PCB, and I can just plug in each header into the six main connectors (thumb, four fingers, and palm). Now I just need to get that new PCB version fabricated so I can mount the board on the glove more easily.

There are two general modes of communication at the physical layer: wired and wireless. This is pretty obvious. For clarity, I am only officially supporting USB (wired) and Bluetooth (wireless) interfaces, but theoretically you can use other types of wired or wireless connections if you’d like. One very smart guy, Mike Cochrane, even hacked a Logitech R400 wireless module into his prototype. But anyway…back to USB and Bluetooth.

Above this physical interface layer, there are a couple of different types of communication that occur between the Keyglove and the host being controlled:

1. Configuration commands sent from the host to the Keyglove
   These commands are used to define the Keyglove’s behavior, enable or disable features, change options, load and save glove-based touchset definitions, query available features, and more. Anything the host might need to know about or change inside the Keyglove is done with a configuration command.
2. Behavior events sent from the Keyglove to the host
   Events including anything that may be triggered by an internal or user-controlled Keyglove action, such as a touch, gesture, battery notification, mode switch, etc. Anything the Keyglove might need to send to the host without the host asking first is an event.
3. Direct input signals sent from the Keyglove to the host (keyboard, mouse, joystick, etc.)
   These are generated by the internal touchset definition, and sent to the host as standard Human Input Device (HID) signals according to established keyboard, mouse, and even joystick signals.

I’ve been promoting #3 for quite some time now because it doesn’t require any drivers, since it enumerates on the host as a regular keyboard, mouse, and/or joystick. This mode of operation is definitely valuable and convenient, and it will continue to be supported in both wired and wireless capacity. There are some great use cases for this mode, particularly those having to do with mobile computing.

However, if #3 was the only method available, the potential for the Keyglove would be severely limited. First, it would be practically impossible to change its behavior without re-flashing new firmware, or implementing some complicated touch-based method of reprogramming. Second, we wouldn’t be able to automatically swap touchsets on the host when switching different applications into the foreground. You could switch manually using a custom touch, but then literally all behavior would need to be defined on the Keyglove itself, which could get complicated and might even be too restrictive, depending on hardware. Certain devices also only support Bluetooth HID and not SPP without lot of extra licensing, such as the iPhone and iPad.

So, I’ve also been working on the bidirectional serial protocol, which takes care of #1 and #2 above. This protocol, combined with some still-under-development OS-specific “Keyglove Manager” applications, allows for greatly extended functionality. Since the direct input mode (#3) will still be implemented and usable if desired, we end up with an optimal solution: convenience for those who prefer it, and power for those who want more control.

Which Interfaces Support Which Modes?

Now that we know what kinds of signals we need to send and receive, we need to know how we can actually accomplish the goal. Based on what I believe (and am totally guessing) will be normal modes of usage, combined with known limitations of each kind of interface, here is what I came up with.

Configuration commands may be sent from the host to the Keyglove over a serial port or custom HID endpoint (not a standard keyboard/mouse/joystick). We also want to make sure that we parse any and every configuration command sent over any interface, to avoid the possibility of locking ourselves out accidentally. Therefore, configuration commands are always parsed from any interface that is supported in the firmware, namely:

• USB serial (wired)
• USB HID (wired)
• Bluetooth serial (wireless)
• Bluetooth HID (wireless)

Behavior events may be sent from the Keyglove to the host over a serial port or custom HID endpoint (again, not a standard keyboard/mouse/joystick). Unlike configuration commands, however, we might not care to do see these on the host—particularly if we’re using direct input signals from an onboard touchset. So while these are enabled by default on any interface which is connected, they may be selectively disabled at any time. This setting persists across power cycles, since it will be stored in non-volatile memory (internal EEPROM). Supported interfaces are as follows:

• USB serial (wired)
• USB HID (wired)
• Bluetooth serial (wireless)
• Bluetooth HID (wireless)

Events are sent out sequentially in the above order (USB serial, then USB HID, then Bluetooth serial, then Bluetooth HID) for any interface that is supported and enabled in the firmware, and which has an active link in the case of Bluetooth. Thanks to the fantastic Bluegiga WT12′s iWRAP firmware, it is possible to have simultaneous serial (SPP) and HID links going—though support for this hasn’t been fully developed yet.

Direct input (keyboard/mouse/joystick) may be sent out only over HID, unlike the other two. But as before, it is possible to send these out over both physical layers:

• USB (wired)
• Bluetooth (wireless)

Direct input is sent out sequentially in the above order (USB HID, Bluetooth HID) if each interface is supported andenabled in the firmware, and at least one internal touchset is defined in memory, and internal touchset usage is enabled (remember that it may be disabled in favor of OS-based custom touchsets).

So What’s “Normal” Behavior?

Currently, the planned default behavior out of the box is this:

1. Parse configuration commands from any interface receiving data. (required)
2. Send direct input using default touchset over USB HID if connected. (optional, enabled by default)
3. Send direct input using default touchset over BT if active HID link established. (optional, enabled by default)
4. Send events over USB serial if connected. (optional, enabled by default)
5. Send events over BT if active SPP link established. (optional, enabled by default)

Aside from these things, there is some other optional behavior which is supported but not planned to be enabled out of the box:

1. Send events over USB HID if connected, using custom HID descriptor. (optional, disabled by default)
2. Send events over BT if active HID link established, using custom HID descriptor. (optional, disabled by default)

There you have it! A basic overview of different methods of Keyglove communication. This allows something for everyone: the ability to plug it in and watch it work without any extra work, as well as the capability for low-level communication and in-depth customization. Now to finish writing the code to support it all…

Of course, this turned out to be a bit difficult to accomplish, due to my lack of knowledge about many low-level AVR hardware features (including timers—*shudder*). I originally hoped I would be able to find an existing chip that did what I needed: 3-channel PWM LED control, on/off digital I/O for the vibrating motor, and frequency-based PWM output for the piezo buzzer. Alas, I found no such device that combined all three of those things together. I guess I shouldn’t be surprised though; that is a pretty unique combination of functionality to pack into a single dedicated pre-fab IC.

Building an ATTiny44-Powered I²C Slave

So, I set about finding an alternative. My choices were to put two or three separate existing chips on the same board to achieve enough functionality—maybe using a couple different PWM control modules—or to create my own using a fully programmable MCU. The MCU approach seemed a bit overkill until I checked the cost of I²C-based PWM modules and compared them with the cost of small AVR modules such as the ATTiny44A (at the moment, $0.83 in single quantities from Mouser).

To make a long story very short, I gave myself a crash course in the ATTinyX4 line of chips, AVR timers, avr-gcc, and the I²C protocol. The guys over at AVRfreaks.net were immensely helpful with pesky objcopy compiler flags with TWI/USI/timer functionality on ATTinyX4s, and Dean Camera’s Newbie’s Guide to AVR Timers was great as well. The final key that allowed me to fully comprehend timers and interrupts came from reading Atmel’s own datasheet for the ATTiny44A, poring over the timer register descriptions and expected behavior. There was a tremendous “AHA!” moment when I rewrote the timer setup portion of my code from scratch while referencing the datasheet, compiled it, and watched it do exactly what I wanted it to on the first try. Amazing.

The I²C/TWI/USI implementation was made many orders of magnitude simpler by using Donald R. Blake’s usiTwiSlave code with a few modifications. I had to add specific chip support for the ATTinyX4 line (most other ATTiny chips were already supported), and then I added callback functions to START/STOP conditions on the I2C bus (only the START one is used in my code; the STOP callback may not actually work due to USI interrupt vector limitations).

Ladies and gentlemen, we’ve progressed to the stage where it’s hard not to justify throwing together a little custom part using a fully programmable microcontroller, simply because they are so darn cheap and flexible. The future is here. (And yes, I realize that this statement will probably seem horribly outdated in a matter of months thanks to Moore’s Law.)

The Eagle schematic and PCB for this module are both available here (always use the latest version, though the one in this post is v2.0).

Programming the Mounted ATTiny44

The current “finished” code is available on the Keyglove GitHub repository here. It uses almost exactly half of the 4K of memory on the ATTiny44 chip, but since the *44 is cheaper than either the *24 or the *84, I decided to go with the *44. What about programming it through? Obviously these things don’t come preloaded with the feedback module slave code. I did all my testing with a DIP package version of the chip, but on the module itself I used an SOIC-14 chip (to keep the same small form factor I used for the original direct I/O version of the feedback board). To be able to program this, I wired up a 14-pin SOIC test clip with appropriate ISP connections for the ATTiny44:

Incidentally, I ran into two difficulties using this approach, both of which I fixed with a small revision to the PCB. First is that the vibration motor is too close to the MCU, so that when it is mounted, the test clip can’t grab the IC. This was easily solved by moving the chip a little farther away from the motor. The second issue wasn’t a problem, but it makes reprogramming the MCU more difficult for anyone who doesn’t have a test clip—namely, only four of the typical six ISP header pins are broken out. These (GND, VCC, SDA, SCL) are needed for regular communication with the module. So, I also added a couple of extra header pins to break out MOSI and RESET. Note that the ATTinyX4 line of MCUs shares the same pins for the SPI and I²C ports.

I did most of my testing and development without the motor mounted, therefore, and I finally got the code working the way I needed it to. Originally I was trying to pack a whole lot of functionality into it, but after a lot of headache and timer troubleshooting, I decided to just strip it down the exactly the same functionality I had before with direct I/O connections. After all, if I could do it that way before, there shouldn’t be an problem continuing. In reality, even adding PWM fade capability to the LEDs was a step up from what I had before, which was simple on/off only.

The feedback I²C slave module is a write-only device (reading is kind of pointless for this application), and it has the following simple register structure:

The default value of all LEVEL registers is zero (off), and the default FREQ value is 0x06E0, which is 1760 Hz, or a high A note. Yes, I chose that arbitrarily.

The code makes use of soft PWM control for the LEDs, and timer-based frequency control for the frequency generator. It is probably not the most efficient way to do PWM, but due to the combination of pins needed for three independent LED channels plus a separate timer-based control of the tone output, I couldn’t figure out a better way, at least with my limited knowledge. Any suggestions are welcome though, if you have any.

The important thing is that the feedback module is now working great while using zero dedicated pins, and I even learned a whole lot about AVR stuff that I didn’t know before. I call this a success.

This recent document from Atmel is also interesting, and may be useful for others (and me, later): Atmel AVR290: Avoid Clock Stretch with Atmel tinyAVR

I’ve just sent in the order for this board, so I won’t have it in my hands for another two weeks or so. In the mean time, I can definitely work with the v0.3 board I’ve got built so far. Functionally they are pretty much identical. Here’s an image that shows the progression in size reduction from v0.1 up through v0.4:

It’s pretty remarkable, really. The v0.4 board will be much more comfortable on the back of the hand. I’ve kept much of the same layout, but moved a couple of the modules over to the right side of the Teensy++ board, so that the whole thing can be about 0.1″ narrower and over half an inch shorter. There’s a kind of “double wing” visual effect now, since the various modules now hang off both sides of the board. Although this would never do for a commercial retail product, it is very valuable for the kit version, since it allows the very rigid PCB that actually sits on your hand to be as small as possible. And this time, I really this it is just about as small as possible. There is almost nothing left to rearrange, and doing so at all would require modifying the WT12 module’s GPIO pins, which I’m not planning to do since it would make that board much less simple to prototype with on its own.

Switching to the MPU-6050

I’m also now officially switching to SparkFun’s new MPU-6050 breakout board instead of using their 6DOF IMU combo breakout board. Both modules have an accelerometer and gyroscope, but the MPU-6050 does it in a single module, faster, more accurately, with no potential axis alignment issues, with a powerful DMP, for $25 less. What’s not to like?

The v0.4 board still has a dedicated header for the 6DOF IMU board, and the embedded software will continue to have support for both, but it is probably best to get the MPU-6050 board instead if you’re looking to build your own prototype and you don’t already have a 6DOF module. It’s just a better option all around, and saving a whole $25 just like that on the overall cost of the kit is pretty significant, especially with the added benefits of future DMP usage (which is close, but still not functionally useful for the Keyglove just yet).

Next up is fixing the ATTiny44-based feedback module. I still haven’t managed that, but this v0.4 revision kind of took me by surprise when I started tinkering with the possibility. Ah, the lure of optimization…

I ran into a behavioral problem with the last revision of the kit PCB (v0.2), where anytime I plugged in the Teensy++ module, it simply stopped working. Not permanently, but as long as I had it plugged in. The blinking orange light that indicates the device is running would go off and stay off, even though it would work just fine with the USB cable connected but not plugged into the board. The really weird thing is that it would still do this even with no other modules or connections on the kit PCB.

There are no shorts or other components built into the kit board itself—it’s just a way to simplify connecting distinct modules together—so theoretically, plugging the Teensy++ into the kit board is electrically identical to merely extending the header pins by a few inches, give or take. And yet, this made it stop working. I checked for shorts across the power bus and found nothing. A visual inspection of the board also didn’t help. It just didn’t work most of the time.

I say “most of the time” because there were short spans of time when it would in fact seem to work right; when I touched some part of the Teensy++ module just right or…I don’t even know, really. I never figured it out, because I replaced it with a new revision of the board:

This board is about as small as it can be without sacrificing functionality or vertically stacking components. It no longer has the internal space for the lithium polymer battery, but if that gets mounted separately on the back of the hand, then the foundational PCB can afford to be smaller, which greatly increases the flexibility possible. The PCB is very rigid, and the v0.2 board was big enough that this rigidity was uncomfortable, even though it was otherwise a very convenient arrangement of parts. Even in kit form, I want this to be comfortable as well as easy, so that’s what the new revision is. Comfortable and small.

Another important technical change, which may possibly have had an effect on the behavior, is that the traces on the board are no longer look like a rats’ nets, full of vias and odd routes. My previous arrangement of touch sensor connections was based on alphabetical order and a very loose correlation of major ports (A, B, C, D, E, F) on the AT90USB microcontroller. However, this doesn’t make a whole lot of sense, because the sensors wires come to the board in finger or thumb groups that are not alphabetical at all: ABCMNO4, DEFPQR5, and so on. With a purely alphabetical pinout, this results in a mess of wires that cross over each other a lot. The v0.3 pinout instead uses the grouped approach. I have modified the pins.h file I’m working with to reflect this, and even included a handy little ASCII pinout in the code:

                             __|||||__
                        GND |   USB   | VCC
                3        27 |         | 26        Y
                      SCL 0 |         | 25        Z
                      SDA 1 |         | 24        a*
                      RXD 2 |         | 23        8
                      TXD 3 |         | 22        9
                2         4 |  37.36  | 21        0
                1         5 |         | 20        A
                      LED 6 |         | 19 INT7
                7         7 |         | 18 INT6
                X         8 |         | GND
                W         9 |         | AREF
                V        10 |         | 38        B
                L    5   11 | 32 . 28 | 39  R     C
                K    G   12 | 33 . 29 | 40  Q     M
                J    H   13 | 34 . 30 | 41  P     N
                6    I   14 | 35 . 31 | 42  F     O
                U        15 |         | 43        4
                T        16 |         | 44        D
                S        17 |_________| 45        E
                         
                          36=INT4, 37=INT5

I will try to create a more helpful graphic diagram with this information before long.

There are a few things to note about this pinout. First, there are three new sensors: a*, 9, and 0. These aren’t that new, exactly, but I consider them new since I’m just now getting around to modifying the code to support them. These three sensors go on the thumb; two are on the back side, behind the nail, and one is on the front side below the two that were there before. This provides a lot of possible functionality, should you choose to use it. Of course, any touchset may include or exclude any sensors you want, allowing you to sacrifice functionality for ease if you choose.

Also note the INT6 and INT7 pins, which are have no sensors connected. These are used for WT12 Bluetooth module management. Technically one of them doesn’t need to be an interrupt pin because it’s an output signal (toggling COMMAND mode when a link is active), but it made sense to keep them next to each other, and keeping the interrupt pin otherwise free allows for more future expansion possibilities. While one pin is used to control the device mode, the other keeps an eye on whether there are any active links, which is a very convenient bit of info to have without needing test which mode the module is in, and without running MUX mode all the time (which is convenient and does work, at the expense of more power required).

There you have it. I am very pleased with this revision of the kit board, and it’s already helping me with my prototyping efforts. I still need to fix one remaining oddity with the ATTiny44-powered feedback module (more on that in a later post), but after that, I have no immediate needs for hardware updates, and I can focus on bringing the embedded software up to speed so that it supports everything I have planned for the current round of features.

You can view a complete photo album showing off the v0.3 board here or below:

Show more

The previous revision of the WT12 UART breakout board that I created only had the basic UART pins, power pins, and the RESET pin (which I have never actually used in any of my projects, leaving me to wonder at its usefulness). I have now created a new version of that breakout board (up for purchase on InMojo) that includes two broken-out GPIO pins to solve precisely the problem mentioned above.

Notice the placement of the extra pins; they are positioned exactly 0.3″ apart from the main 6-pin UART header (which itself is pin-compatible with SparkFun’s BlueSMiRF boards). This spacing and the horizontal alignment mean this module can if desired be placed across the center of a typical solderless breadboard to allow easy access to all of the pins at the same time.

The iWRAP code that I’m working on relies on the ability to determine links status and to control mode switching programmatically for the simplest and most efficient operation. This new PCB revision should therefore help me wrap up the code and finish the keyglove’s Bluetooth integration.

Also, now that Bluegiga has just publicly released their more flexible iWRAP5 beta firmware with full documentation, I should be able to wrap up the customized HID implementation to allow mouse scrolling and few other features that were missing from iWRAP4.

If you need a quick update without waiting for upcoming posts, check out the two links above—especially the Facebook page—and read through a bit of the history until sometime back around Christmas.

Keyglove Kit PCB

One of the things I’ve been working on is a way to make building a Keyglove yourself much easier, either piecemeal on your own or out of an eventual complete kit. Although there are relatively decent instructions available for a partial build already, there are a few reasons why it’s still pretty hard to do. The biggest problem is that there are just so many connections, especially once you start hooking up the sensors. If you’re working with a small solderless breadboard for prototyping, it gets insanely messy. And even if you can get all the connections right by hand, the last thing you want to do is have to disconnect even one pin—especially if it’s a sensor—because it can be a big headache to get it plugged back into the right spot.

The recent addition of interrupt-driven motion detection requires the use of the E4 and E5 pins on the Teensy++ board, both of which are internal and not the standard 0.1″ pitch. Adding the WT12 module with the most recent published design isn’t even possible, since two of the GPIO connections conflict with the RGB/vibe/piezo feedback module. In short, it’s way too easy to get frustrated with building your own, and that’s the last thing I want to have happen.

So, I’ve created a PCB that takes the place of the solderless breadboard, and instead provides a pre-routed board that is essentially just a collection of female or male header blocks, ready for quick connections of pre-built modules or logical sets of sensor cables, where everything is labeled and no jumper wires are necessary. In addition, the physical layout of the board is such that it’s smaller in every dimension than the solderless breadboard, and it includes a strategically placed empty space to hold the recommended lithium polymer battery.

While there’s still a little bit of work that you’ll have to do to use the internal E4/E5 pins on the Teensy++ board, since they’re a weird pitch and too small for normal header pin access, the rest of the board is designed for a simple plug-and-play approach. The board accepts a total of six modules:

• Teensy++ (required)
• SparkFun 6DOF module or Keyglove 9DOF module
• Keyglove LiPower module + LiPo battery
• Keyglove Feedback module
• Keyglove EEPROM module
• WT12 UART module

Note here that the Teensy++ is actually the only part that is strictly required. All you’ll get is touch detection, but it will work just fine that way. The other modules give you great capabilities like motion control, wireless support, feedback, and more flexible touchset programming, but you can easily get by without one or more of those. The motion sensor module can be either the 6DOF board or the custom Keyglove 9DOF board based on the MPU-6050. (Note that none of the Keyglove modules are available for purchase just yet, but they will be as soon as they are adequately revised and tested.)

Further, instead of being manually connected to the right I/O pins on the Teensy++ board, the sensors are broken out into per-finger headers:

• 1x 6-pin header for the thumb
• 4x 7-pin header for each other finger
• 1x 3-pin header for the palm

The bottom of the PCB is clearly labeled to indicate which sensor belongs with which pin. This in particular makes the whole process vastly simpler.

Since the LiPower board (USB/battery charger) requires the power from the USB connection but not the D+/D- data lines, we need a good way to “forward” those connections straight into the Teensy, since the Teensy also uses a mini USB connection, and it would be a huge pain to have to plug two cables in to the kit board every time you wanted to reprogram it and charge the battery at the same time. Further, the Teensy++ must be modified for 3.3v operation, so the VCC/GND pins from the Teensy can’t be used for LiPo charging, even if the current draw was supported. On top of that, the Teensy doesn’t break out the USB D+/D- pins, since under normal circumstances that would be a huge waste of precious board space.

My solution for the Keyglove kit is to use a small USB jumper cable that is connected all the time. The Keyglove LiPower board does break out all five USB lines, so those are brought to a 5-pin header on the edge of the board. This allows a short USB jumper cable (5-pin header on one side, mini USB on the other) to be connected to the Teensy so that the required connections are always present. It’s not the most elegant solution ever, but I think it pretty much is the most elegant solution that doesn’t involve a totally custom controller board—which, of course, is the main goal anyway. The kit is mostly an intermediate/alternative step.

Here are the photos from the Keyglove Kit v0.2 photo album on Facebook: (the v0.1 design never made it off the ground)

Show more

Design Changes for 37 Sensor Connections and Bluetooth GPIO

One thing that I’ve finally been able to accomplish is to allow a full 37-sensor design with the Teensy++ board, which was previously not possible due to a shortage of I/O pins. Since I added three extra sensors on the thumb for more flexibility, I’ve been resigned to the fact that I just wouldn’t be able to use all 37 in the Teensy-powered version, but instead only in the full production version. This is no longer the case.

I’ve taken the feedback module and converted it into an I2C slave device, which means it’s now going to use just the SDA/SCL lines on the Teensy (which are already used for other I2C devices anyway). This freed up five whole I/O lines—red, green, blue, vibration, and piezo—which have been reassigned to the final three sensor connections (labeled 9, 0, and a*) and the two newly added GPIO connections from the Bluetooth module.

The GPIO additions are even more important at this stage than just those last three sensors, honestly, because without any GPIO access, some of the WT12 control becomes significantly harder to accomplish. The iWRAP firmware on the WT12 has two different modes: COMMAND mode and DATA mode. Whenever a Bluetooth link is active, the modules switches to DATA mode, and to take it out of this mode requires an escape sequence (those familiar with old-school modems may remember the typical “+++” escape sequence they used). However, the HID protocol used by the Keyglove actually ignores this escape sequence, since it is reasonable to assume that someone might at some point want to type “+++” without spontaneously altering the state of their wireless keyboard connection.

This leaves a third mode, known as MUX mode or multiplexing mode, where each command or data transmission is wrapped in a special framing packet so that both can be sent at the same time. This works, but it is much more resource-intensive for the WT12′s internal processor, and is therefore more limiting that I would like. GPIO access completely solves this problem by providing a very simple way to trigger a switch to COMMAND mode whenever necessary, and by providing a way to tell whether there are any active Bluetooth connections without querying the device over the UART connection. This will be extremely beneficial.

So what about that new I2C slave feedback module? It isn’t finished yet, but it’s now powered by an ATTiny44 MCU doing the heavy lifting of some PWM fading code, piezo frequency control, and of course the I2C protocol transmissions. It’s almost done, but not quite. The ATTiny’s code is available on GitHub in its current state.

Keep watching for more info. There should be some very good news about a glove manufacturer shortly!

Keyglove #09 – Wireless Mouse Movement Demo from Jeff Rowberg on Vimeo.

Finally! I have managed to incorporate the Bluegiga WT12 sufficiently into the current prototype hardware, along with the still-in-progress iWRAP code library to control it easily and transparently. This video demonstrates the first true combination of Bluetooth wireless functionality and the actual Keyglove code, rather than just a proof-of-concept Arduino sketch. This is a major milestone; wrapping up the remaining core Bluetooth functionality should be pretty straightforward now.

For a quick summary: the board in the video is the same prototype that I’ve been using for a while, powered by the Teensy++ and SparkFun 6DOF module for motion detection. The WT12 Bluetooth module makes it wireless, and a 3.7v LiPo battery powers it all. The glove itself is not attached for this demo, due to the fact that there would be way too many wires there for me to be able to do the quick and frequent adjustments to the board that I’m doing at this time. The touch sensors are working fine, however, and wireless keyboard input would be working as well here if it were connected.