This section explores some of our initial efforts to program a virtual frog that could be dissected incorporating force feedback using the Novint Falcon. This section is designed for those wishing to continue working on creating an educational haptic application. We have attempted to provide all the resources and information possible to help anyone else looking to pick up where we left off. We only ask that you give us credit for our work.

For convenience sake we have included links to all applications used here, as well as on the downloads page.

Software Development Kit

Novint Technologies released a SDK (Software Development Kit) written in C++ with their initial release of the Falcon in June of 2007. The SDK contains, among other things, all the .dll files needed to run the Novint Falcon, an extensive Programmer’s guide, and a basic programming test written for OpenGL, DX9, VC 6, VC 8 and 3D Game Studio. The SDK also required the files installed when installing the Novint Falcon device itself.

Method Choice

With very little effort we were able to run a basic program in OpenGL that allowed us to use the mouse to move a virtual ball until it hit a will, at which point it would stop. Initially we thought it would be good to just build the entire Frog in openGL, but the finished product would be just that, a final product. With the exception of added functionality, you would still have a frog, and only a frog.

With that in mind, we started looking at other rendering engines, ones that many users were using to create content. It became apparent that virtual worlds were the way to go because of the ease of use, large user base, and community support. Although there are literally dozens of virtual worlds out there two stuck out in our mind: Second Life and Croquet.

Second Life was appealing simply because of the mass numbers of people using it. Furthermore, it has also started to take off in the educational community and several universities, including UNC at Chapel Hill, have invested heavily in buying land in that world. The creators of Second Life actually started their project out with a large haptic device, but soon abandoned it for more financially appealing options. With the Novint Falcon, an affordable option finally now exists. The drawbacks are many though. Although Second Life has promised to release a SDK for their world, they have yet to do so. There is also a significant cost involved in building in Second Life, which would instantly exclude many educators. In the end we decided to look at other options. That is when we found Croquet.

Croquet is an open-source virtual world built using the programming language Squeak, which is a variation of SMALLTALK. It uses peer-to-peer network technology to distribute the virtual world. The big pull for this one was the open-source potential. It also used OpenGL as a rendering engine, and we already had an OpenGL start. In theory we could do what ever we wanted with it. Unfortunately there was a relatively small community that worked with Croquet. There is also no distributable application at this point. It is more of a SDK that developers can use to build their virtual worlds. In the end, the open-source potential won out and we started moving forward with planning to connect the haptic device to a Croquet world.

Our ultimate goal was to get the haptic device talking to the virtual world. That meant both navigation and haptic feedback. We could then build virtual objects with relative ease. Most modern virtual worlds offer easy object importation capabilities, and with resources like Google’s 3D Warehouse there was already a large selection to work with.

Croquet

Having made our decision we moved forward with the programming for Croquet. Because the Falcon SDK was written in C++ and Croquet was written in Squeak we had to figure out a way to get them to talk to one another. We talked with several full-time programmers and they suggested that the easiest way to accomplish that was to create a .dll from the Falcon SDK and then call that .dll in Croquet.

We spent the better part of a month trying to learn to program in Squeak and actually made some pretty good progress, but the project we were attempting was going to take much more programming than we had time to accomplish, so we started looking at other options.

Matt Schmidt from the University of Missouri inspired us when he recently used a Wii remote to navigate Croquet. It turns out that there is a program out there that will turn any device into a pointer with relatively little programming. Because one usually navigates Croquet using a mouse it sounded perfect for us. The program is called GlovePIE (Windows)/Darwin Remote (Mac).

We downloaded the software and after a little searching came across a script written by the developer of the GlovePIE program specifically for the Haptic Device. The program runs in the background and when you want to activate it you click the top button. The haptic device then becomes the pointer with two of the other buttons serving as a mouse buttons. When you click a button and move the device the avatar moves around the virtual space. When you let go, the haptic device becomes a basic mouse. At this point there is no haptic feedback, but navigation and manipulation of objects is possible. Here is the script tweaked a little to run on a single screen and ready to run with GlovePIE.

Demonstration

We also started working on tweaking the script to work with Second Life but didn’t get too far. In Second Life the only way to navigate with a mouse is to bring up an on screen navigation menu that you have to click to move the avatar. It works okay when the avatar is flying, but otherwise you just end up using the buttons on the handle to move around. However, just in case anyone wants to work with the script, here it is.

It should be noted that about the same time we found the GlovePIE software Novint announced they are working on drivers that will work with not only Second Life, but World of Warcraft as well. That should be a huge step forward in this haptic study. We can then build our objects (like a virtual frog) and see how students react to using haptic devices to education.