Using the Microsoft Kinect with ICT’s FAAST and Vizard
This article describes how to use the Microsoft Kinect with ICT's FAAST and Vizard. It includes instructions for setting up FAAST, a sample Vizard script that attaches a sphere to all joints from the FAAST skeleton, and some screenshots of Vizard and FAAST side-by-side.
FAAST installation and setup
- If you have your Kinect plugged into your computer, please unplug it, and uninstall any previous Kinect drivers. Make sure that old drivers are not showing up in Device Manager.
- Follow the steps and directions presented to you within the installer, FAAST_and_Kinect_Installer.exe. The second page of the installer presents four components. All are required for the Kinect sensor. Under ‘Select components to install’, ensure all four boxes are checked, and then click Install. In the case that you have FAAST already installed on your system, the installer will overwrite the existing files. In the case that you have either of the Visual C++, or .NET libraries already on your system, the installer may prompt you to repair or remove these components. If so, choose Repair. After accepting the terms and licensing agreements, installation will complete.
- With the software installed, connect the Kinect: Plug the AC adapter into an electrical outlet, and plug the USB connector into a USB port on your computer. To verify the Kinect is properly recognized, open Windows Device Manager: e.g. search "Device Manager" in the Start menu. It may take some time for the Kinect to appear but, when mounted, you should see a ‘Microsoft Kinect’ node with four different sub-nodes under it.
- Run FAAST (e.g. search "FAAST" in the Start menu). Ensure the Microsoft selection in the Tracker drop-down list, and then press the 'Connect' button. If all has installed properly you should be able to see FAAST successfully connect. Once the Kinect sees your body, move your body so that the Kinect can detect you. Once FAAST calibrates your skeleton, you'll be able to see a stick figure overlay on your body.
That's it for setting up FAAST.
VRPN implementation
Now for Vizard, we'll simply connect to it using the VRPN7 protocol, here's ICT's explanation of their VRPN implementation and skeleton joint mappings:
FAAST streams the user’s skeleton over a VRPN server identified as “Tracker0@ip_address” (“Tracker0@localhost” if running on the same machine as the client). The server automatically starts when the toolkit connects to a sensor. A total of 24 skeleton joint transformations (including position and rotation) are streamed as sensors. Corresponding to the OpenNI framework, the joints are ordered as follows:
| Sensor | Joint | Sensor | Joint |
|---|---|---|---|
| 0 | Head | 12 | Right Elbow |
| 1 | Neck | 13 | Right Wrist |
| 2 | Torso | 14 | Right Hand |
| 3 | Waist | 15 | Right Fingertip |
| 4 | Left Collar | 16 | Left Hip |
| 5 | Left Shoulder | 17 | Left Knee |
| 6 | Left Elbow | 18 | Left Ankle |
| 7 | Left Wrist | 19 | Left Foot |
| 8 | Left Hand | 20 | Right Hip |
| 9 | Left Fingertip | 21 | Right Knee |
| 10 | Right Collar | 22 | Right Ankle |
| 11 | Right Shoulder | 23 | Right Foot |
Example Vizard script
The following example script shows how to attach a vizshape object to all trackers streamed from the FAAST VRPN server. You may need to move the viewpoint in Vizard to get the skeleton data in view:
import vizshape
viz.go()
grid = vizshape.addGrid()
"""
Kinect Tracker object ID's
These are not actually being using in the script but are to
help anyone who wants to get access to a specific bodypart.
For example to just get a handle to tracking data for the head use:
myHead = vrpn.addTracker( 'Tracker0@localhost', HEAD).
"""
HEAD = 0
NECK = 1
TORSO = 2
WAIST = 3
LEFTCOLLOR = 4
LEFTSHOULDER = 5
LEFTELBOW = 6
LEFTWRIST = 7
LEFTHAND = 8
LEFTFINGERTIP = 9
RIGHTCOLLAR = 10
RIGHTSHOULDER = 11
RIGHTELBOW = 12
RIGHTWRIST = 13
RIGHTHAND = 14
RIGHTFINGERTIP = 15
LEFTHIP = 16
LEFTKNEE = 17
LEFTANGLE = 18
LEFTFOOT = 19
RIGHTHIP = 20
RIGHTKNEE = 21
RIGHTANKLE = 22
RIGHTFOOT = 23
#store trackers, links, and vizshape objects
trackers = []
links = []
shapes = []
#start vrpn
vrpn = viz.addExtension('vrpn7.dle')
#now add all trackers and link a shape to it
for i in range(0, 24):
t = vrpn.addTracker( 'Tracker0@localhost',i )
s = vizshape.addSphere(radius=.1)
l = viz.link(t,s)
trackers.append(t)
links.append(l)
shapes.append(s)
