Nonpolynomial Labs

Nonpolynomial Labs Roboexotica Projects

2009-12-03T01:32:20-08:00

For the past month, I've been living in Vienna as part of as artist residency with monochrom. The main goal of this residency was to complete some projects for roboexotica, the robotics cocktail party held each year here in Vienna. This year, roboexotica is being held December 3-6, and now that I've actually seen my projects pour some drinks, I figured it's time to present them to the world.

First off, there's Adult Mario, the mario game that drinks and vibrates!

Then there's Bartris, the tetris that's also a bartender.

I'll have a post next week that goes into the implementation specifics of these projects, but for now, all of the code is available at http://www.github.com/qdot/bartris.

A Day at the Ports

2009-09-08T00:11:32-07:00

While I was out flying my stunt kites this weekend, the wind ended up being a little iffy. So, after flying a bit, I decided to see what kind of things I could put together using just my iPhone. Thus, the 'Day at the Ports' project was born.

This is consists of 2 and a half videos (the half being a test one I took just to see if looping was going to work correctly). all of these were taken using viewfinders around the park I was at, Middle Harbor park in Oakland.

Both videos use a combinations of tracks generated by rjdj, mixed with the wind and environment sounds taken from the iphone mic while recording the video.

The first one is "Construction from Far Away".

Construction from Far Away from qDot on Vimeo.

rjdj page for original audio clip

I was really happy with this one. The quality of the viewfinder along with my shakey cinematography ended up inducing a bit of a dreamlike state in the video, and the soundtrack matched up just perfectly.

The second one is called "A Headless Sutro".

A Headless Sutro from qDot on Vimeo.

rjdj page for original audio clip

This one ended up a little harsher than I think I originally meant it to be. The viewfinders I was using for it seemed to move very quickly, so I decided to use an audio scene that was a little choppier. However, it came out also... schizophrenic in the end.

To show what I was using the take the video, and also to just remind myself where all the buttons were in the video software, I made a little sample video. Uses William Besinski's The River, because, well, water. I'm original like that.

Test Water Loop from qDot on Vimeo.

All in all, I'm really happy with how everything turned out. Managed to make something relatively neat while just screwing around waiting for the wind to adjust its attitude (which it never did. Stupid nature.).

New Software Project Icons and Pages

2009-07-18T10:21:20-07:00

After a couple of months of work, lots of poking at jekyll and github, and very little work on the actual projects themselves, I've finally gotten the webpages for all of my software projects together. They come with nice new spiffy icons from Counter Productive Productions.

[![libnifalcon][4]][5][![libambx][6]][7][![liblightstone][8]][9][![libomron][10]][11][![libtrancevibe][12]][13]

From right to left: * libnifalcon - Cross platform driver for Novint Falcon haptics controller * libambx - Cross platform driver for amBX ambient environment system * liblightstone - Cross platform driver for lightstone biometrics system * libomron - Cross platform driver for Omron USB Blood Pressure Monitor and Pedometer * libtrancevibe - Cross platform driver for the Rez Trancevibrator

All drivers, no applications. Maybe I should actually write something to use this stuff someday...

New Projects at NP Labs

2009-05-03T13:44:51-07:00

Well, another 6 months have come and gone, and I've got a few new projects on the radar here.

libomron

This is a framework for accessing the USB capabilities of Omron medical equipment, specifically the Omron 790IT Blood Pressure Monitor at the moment. There has been some talk online about accessing their pedometers in a cross platform manner (reference: dullest.com), and I'm hoping the protocol for that and their other blood pressure monitors are basically the same, so I can extend the protocol parsing to the whole family of products.

libambx

libambx is an open source, cross platform library for the Ambx ambient environment feedback system that I'm working on in conjunction with the people at Electrosthetics. I'm currently working on reversing the ambx communications protocol (document available here). It's nice to work with someone else on this stuff for once.

libnifalcon updates

libnifalcon is still coming along nicely, if a bit slow at the moment since I'm stuck on some rather difficult problems with firmware loading and kinematics. It's been used in a couple of papers (including this one on teleoperation of an insectbot), and there's also been some work on porting it to Chai3d for use in the Dimple environment for physical interaction with sound.

NP Labs Projects Update

2008-11-02T21:21:20-08:00

Wow. Almost 5 months between posts. I've been so busy on my projects that I have no time to write about them anymore. :)

First off, libnifalcon, the open source driver for the Novint Falcon, is currently in alpha stage for the v1 release, soon to move to beta after a few more bugs are fixed. It's already seeing some usage, too!

Haptic Interfaces for Virtual Instruments using libnifalcon from qDot on Vimeo.

The video above is a demo of a couple of patches for PureData written by Edgar Berdahl for use in Physical Interaction Design for Music class at the Center for Computer Research in Music and Acoustics at Stanford University. The video shows a haptic simulation of a bowed instrument, as well as a training patch for drum rolls. Moving toward the line in the middle of the window causes the falcon to kick back slightly, and the user must hold the end effector steady to keep the sound steady and constant. All of this is running on a beta version of the np_nifalcon external for max and pd, which is available on the libnifalcon site.

Next up is another project that I decided to throw together today to check out the OpenSoundControl protocol.

Kitelight w/ iPhone Controls from qDot on Vimeo.

This is the Kitelight system controlled using the mrmr OSC interface builder for the iPhone. It's connecting to laptop running python using liblo and pyliblo for OSC controls. This goes through a xbee dev kit board to an arduino with xbee shield which is controlling a panel of RGB LEDs attached to the kite.

Usable range on the zigbee is ~100m, more than enough room for flyer to kite communications. Now that this works, I can also run lighting commands from PureData to do music synchonization or whatever else I might think of.

For all the URLs listed above, this project came together rather quick, about 90 minutes from start to finish, though obviously all the boards were already built, so it was just a matter of coding it all and learning OSC and mrmr.

Now, back to working on getting libnifalcon released...

99 Dollar Novint Falcons at TigerDirect

2008-09-05T23:04:31-07:00

Tiger Direct now has the Novint Falcon for $99. Only took a little over a year from release for the price to finally drop to the <$100 level they promised 2 years before launch. :D

So, if you had interest in helping out with the LibNIFalcon project but didn't want to throw the cash down, well, now it's less cash to throw down. LibNIF is cranking right along, the C++ version is nearing the first milestone, and we've now got what will hopefully be a much more stable kinematics model in the works.

Falcon Uncasing

2008-06-07T23:25:22-07:00

Well, after two months of not getting much done I finally manage to zip a directory and call it v0.4 of libnifalcon, and the developer responsible for the SolidEdge simulation does a full uncasing of the Falcon, taking a great picture set along the way. I'm start to feel like I'm falling a bit behind in usefulness here. :)

Image by kouellet of the libnifalcon project

I really like this image for some reason.

SolidEdge Simulation of the Novint Falcon

2008-05-24T13:05:32-07:00

So, I've actually got some people helping out on libnifalcon now, and one of them just posted a neat video of a SolidEdge simulation of the Falcon hardware.

Video from Futuristic Music Design Contest at Yuri's Night

2008-05-18T19:49:04-07:00

At Yuri's Night this year, CreateDigitalMusic.com had a futuristic music design competition to see what new an innovative audio interfaces people could come up with. I filmed the competition, and have finally gotten everything edited and up on YouTube. I apologize in advance for the Blair Witch-esque camera work. :)

libusb, libftdi, latency, and my own stupidity

2008-04-29T22:01:51-07:00

When last we left off, I had an OpenGL kinematics simulator for my current OCD fulfillment object, the Novint Falcon. I'll get to the math behind all that in the next post, but for right now, suffice to say, it just works. The kinematics loops can easily hit 40000 iterations per second, which is more than enough for my first few basic projects, including mouse control. However, that really only mattered on windows, because when I finished the simulator, windows still had a 1000hz update rate, while linux and os x were still running at 70-150hz update rate on a good day with the wind going in the right direction. I spent last week trying to get linux and mac up to the same speed as windows. What follows is the story of that week.

But first off, let's rewind to August of last year. Before Novint put out their own SDK, I was bound and determined to get my own out, just because it seemed like a fun challenge. I managed to document out the firmware download process, but not completely replicate it. My first program worked by starting up one of their test programs to load firmware, then using my own code after that to communicate with the falcon. The hack ended up being successful, if ridiculously unsafe to look at in a normal workplace. The video of the results are at http://www.youtube.com/watch?v=EfrMalBiXuY (NSFW), which is not autolinked because otherwise you'll click it at work, won't you? Yeah. Copy and paste.

Anyways, while doing this, I took some notes on the startup sequence for firmware loading. Here's a picture of those notes:

Now, fast forward back to last week. Time to make everything equally fast.

To start, how the falcon communicates. Using the test firmware that came with the disc in the falcon box (currently the only thing I have mapped, and I honestly don't even know if there's other firmware out there), it receives 16 bytes input, and upon receiving that, gives back 16 bytes as output (it's actually 18 bytes, due to ftdi modem status being tacked on the front, but those are negliable). That's it.

Let's look at the stats that test programs were putting out for each platform (thanks to people on the libnifalcon-devel list for helping out on this):

Linux:

Time between I/O Iterations: 0.015991s

Average Frequency: 62.534351 per second

Windows:

Time between I/O Iterations: 0.000942s

Average Frequency: 980.382853 per second

Yeah. Bit of a difference, eh? So something was causing I/O iterations on linux to take around 16ms each, while on windows, we were getting 1 iteration per millisecond, and achieving near the 1khz that's mentioned by Novint all the time. This is running on the EXACT same code, up to the point of the FTDI library being used. The Windows version was using the ftd2xx drivers, while linux was using libftdi. ftd2xx on linux seemed to run decently fast, though I don't have the number handy at the moment.

Now, I'd read a few things around the net about libftdi being ridiculously slow, mostly in terms of bitbanging though. Either way, I figured it was something in either libftdi or libusb that was causing the slowdown. Rebuilt everything with -gp and let gprof at it for a while, just to see that, nope, it was just sitting there waiting during the I/O loop, for 16ms at a time.

At this point, I start wondering if it's not the synchronousity of libusb-0.1 that's slowing me down. Luckily, libusb-1.0 is in development right now, which enable asynchronous transfers for usb. Pulled the dev branch of that, tried it out. Asynchronous sends, writes are superfast, reads... 16ms.

Damnit.

So, something in the read is taking 16ms. It's time to start playing with our read transfer and see what else we can change. First off, changing the read request size to 64 bytes, the maximum packet size for the endpoint.

Time between I/O Iterations: 0.000919s

Well, that made something happy. The problem there is, we're now sending 1000 input packets, and only getting back ~250 output packets. This means that there was something I was missing about sub-maximal packet sizes.

Much googling insues. No information found. Finally out of frustration, I just google "ftdi 64 bytes".

And I find the FTDI Addendum on Data Throughput, Latency, and Handshaking for the FT232 Series Chips (PDF)

There it is, clear as day. There's a latency timer on the chip that will send bytes to the host assuming one of three conditions:

A serial status line (DTR, RTS, etc...) is flipped
The buffer reaches maximum capacity (Thus our results with the 64 bytes
The latency timer overflows

We're obviously not playing with the first one. The second one we've seen the effects of, but we don't want to have to wait for 62 user bytes at a time.

What's the default starting value of the latency timer?

16ms.

If you scroll back up and look at my notes again, you'll see there's a line there that says "0x9 (latency timer?) 0x1". This was a control message sent over by the Falcon test program. 0x9 is the control message index, and 0x1 is the value. They set the latency timer to 1ms during the initialization stage. I totally skipped over that when transcribing the code for the initialization sequence, and it meant I was sitting on slow code for many, many months.

I added that single line of code to the libusb-0.1.12/libftdi based libnifalcon libraries.

Time between I/O Iterations: 0.002019s

Yyyaaaaaayyyyyyyy!!waitaminute. 2 ms?

And thus, the synchronous call issue comes back to bite me in the ass, except this time, it's actually a correct diagnosis.

You see, when you send a USB message, it's packed into a USB frame. Each USB frame can carry multiple messages, but at a rate of 1 frame per millisecond (we're at USB 2.0 fullspeed here). libusb-0.1.12 only packs one request per frame, so we have 1ms input, 1ms output. 2ms overall, locking our I/O loop to 500hz. DAMNIT.

So, the solution to this is to either figure out a way to get both input and output in the same frame, which may be possible with libusb-1.0. That remains to be seen, and that code is still very, very alpha. I'll keep working with it, though.

Anyways, this still leaves a couple of questions. First off, when I was checking the ftd2xx drivers for linux, I decided to check their symbol table for the dynamic library...

...

         U time




         U tolower




         U toupper




00013673 T usb_bulk_read




0001357d T usb_bulk_write




00017d30 B usb_busses




000132ff T usb_claim_interface




00014309 T usb_clear_halt




000120e6 T usb_close

...

HEY! Those are libusb calls! So ftd2xx is at least partially based on libusb. How they're managing the superfast I/O, I'm not sure. Could be threading, could be they've got their own asynch thing going on.

Secondly, I obviously didn't have the latency timer set on the ftd2xx version of the drivers, either. Why did my drivers run so fast on windows without that? I'm guessing there could be a config file I was missing somewhere, or maybe their drivers just do it themselves on connection or something.

Anyways, the moral of the story: Read the god damn spec sheet. And all the addendums. And pay attention to your own notes.