FreeTrack
FreeTrack is a general-purpose optical motion tracking application for Microsoft Windows, released under the GNU General Public License, that can be used with common inexpensive cameras. Its primary focus is head tracking with uses in virtual reality, simulation, video games, 3D modeling, computer aided design and general hands-free computing to improve computer accessibility. Tracking can be made sensitive enough that only small head movements are required so that the user's eyes never leave the screen.
A camera is positioned to observe a rigid point model worn by the user, the points of which need to be isolated from background light by means of physical and software filtering. Motion is tracked with up to six degrees of freedom : yaw, pitch, roll, left/right, up/down and forward/back. Windows-compatible video devices like webcams are supported, as well as special Nintendo Wii Remote camera and NaturalPoint cameras.
FreeTrack can output head-tracking data to programs directly using its own open interface, as well as TrackIR, SimConnect and FSUIPC interfaces. Programs that support these interfaces are regarded as being FreeTrack-compatible. FreeTrack can also emulate mouse, keyboard, and joystick if a program does not support a direct interface.
FreeTrack is coded in Delphi 7 and uses DirectShow and DirectX. Head tracking is achieved using implementations of DeMenthon's four-point iterative pose estimation algorithm and Alter's three point geometric algorithm.
Software
FreeTrack uses a camera to collect real-time information on the point model worn by the user. Specifically the image coordinates of the model points, which are either received directly from the camera or extracted from a video stream. These coordinates are used to generate an estimate of the real head pose, which can be transformed by the user in a number of ways to create a virtual pose. One of the most fundamental transformations involves amplifying rotation so that only small head movements are required. Finally, the virtual pose is sent to the user's choice of outputs. This is all done in the background, with tracking status displayed in the system tray.A 3D preview is available that shows the virtual head position and orientation for a given real head pose and can be viewed from multiple perspectives, including first-person. This greatly assists with testing and makes it easier to experiment with different settings.
Each degree of freedom has a response curve that can be modified to change the way the virtual head moves for a given real head movement. This is commonly used to create a central deadzone region so that the user’s head can be more relaxed there.
Keyboard, mouse and joystick buttons can be used to toggle tracking settings, including the virtual centre location and individually toggle axes and outputs.
For NaturalPoint cameras, FreeTrack can provide advanced features and a level of customization that is not available with official software.
Camera
| Camera | Sensor resolution | FPS | Sensor | Angle | Output | CPU usage | Subpixel precision | IR LEDs | Approx. price |
| Ideal webcam | 640×480 | ≥60 | monochrome | 42 | highly compressed | small | Software-dependent | Yes | ? |
| OEM IR webcam | 640×480 | 30 | color | 42 | JPEG compressed | small | Software-dependent | Yes | $5 |
| Sony PlayStation EyeToy | 640×480 | 30 | color | 56 | JPEG compressed | small | Software-dependent | No | $16 |
| Sony PlayStation 3 Eye | 640×480 | 187@320x240, 125@320x240, 75@640x480 | color | 75, 56 | JPEG compressed, raw | small | Software-dependent | No | $24 |
| Microsoft Xbox Live Vision | 640×480 | 60@320×240, 30@640×480 | color | ? | JPEG compressed, raw | small | Software-dependent | No | $14 |
| Nintendo Wii Remote | 128×96 | 100, 250 | color | 41 | point coordinates | none | 1/8 | No | $23 |
| NaturalPoint TrackIR 1 | 60k pixels | 60 | monochrome | 33 | binary threshold | minimal | Yes | retired | |
| NaturalPoint TrackIR 2 | 60k pixels | 100 | monochrome | 33 | binary threshold | minimal | Yes | retired | |
| NaturalPoint TrackIR 3 | 355×288 | 80 | monochrome | 33 | binary threshold | minimal | Yes | retired | |
| NaturalPoint TrackIR 3 Pro | 355x288 | 120 | monochrome | 33 | binary threshold | minimal | Yes | retired | |
| NaturalPoint TrackIR 4 Pro | 355×288 | 120 | monochrome | 46 | binary threshold | minimal | 1/20th | Yes | $99.95 |
| NaturalPoint TrackIR 5 | 640×480 | 120 | monochrome | 51.7 | grayscale threshold | minimal | 1/150th | Yes | $149.95 |
| NaturalPoint SmartNav 1/2 | 60k pixels | 60 | monochrome | 33 | binary threshold | minimal | Yes | retired | |
| NaturalPoint SmartNav 3 | 355×288 | 120 | monochrome | 33 | binary threshold | minimal | 1/20th | Yes | retired |
| NaturalPoint SmartNav 4 | 640×480 | 100 | monochrome | 41 | grayscale threshold | minimal | 1/150th | Yes | $400 to $500 |
Resolution
In most cases a resolution of 320×240 is sufficient, this is capable of producing a much higher sub-pixel resolution, enough to allow accurate cursor control on a high-resolution monitor. Resolutions 640×480 and above have diminishing returns and correspond to an exponential increase in CPU usage when not sufficiently compressed before reaching the computer. Higher resolutions become more important at greater distances from the camera. The Wii utilizes a low-resolution 128×96 sensor, which is found by some to produce jittery tracking and may require smoothing to improve stability at the cost of decreased responsiveness.Sensor
For the same resolution, monochrome sensors can resolve finer details much better than color sensors due to the lack of a color filter array.Frame rate
FreeTrack uses interpolation with low-frame-rate video devices to improve panning smoothness. However, responsiveness is fundamentally limited to the frame rate; a 30 frame/s webcam has a maximal response delay of 33.3 milliseconds compared with 8.33 milliseconds for a 120 frame/s camera. To put this into perspective, a human’s reaction time to visual stimulus is typically around 200 ms; 30 ms can be regarded as a competitive ping in online reflex-based games, and an LCD monitor refresh rate is typically 17 ms.Higher responsiveness gives a greater feeling of control, but since virtual head motion is amplified, it can also cause it to move unrealistically fast. For this reason, some programs limit head movement speed, wasting some of the responsiveness of higher-frame-rate cameras.
Angle
A wider viewing angle allows a larger tracking region when in close proximity to the camera. At further distances a wide angle is not desirable, more of the frame is unused and the effective resolution drops more rapidly. More peripheral light can also be seen, which can interfere with tracking. Viewing angle can be reduced by using digital zoom at the cost of resolution.CPU usage
The Nintendo Wii Remote effectively uses no CPU, NaturalPoint cameras use a small amount, and general video devices can use a significant amount, depending on the brand and the specific camera settings in use. A PlayStation Eye running at the same resolution and frame rate as a TrackIR 4 would be very demanding on a single-core CPU. However, modern multi-core CPUs are making this less of an issue. Resolution and frame rate can always be reduced to conserve CPU resources.Filters
FreeTrack requires the tracking points to be isolated from all other light; this is best done using infrared LEDs and a visible-light blocking filter in front of the camera. Photographic film or the magnetic storage medium inside floppy disks can be used as inexpensive visible-light filters. Further filtering can be done in software by adjusting exposure and threshold.All video devices like webcams have a built-in infrared-blocking filter, which can be removed to improve sensitivity to infrared light, allowing better point isolation and the possibility of retroreflective tracking. This is normally a straightforward and reversible procedure for most webcams.
Wii Remotes and NaturalPoint cameras are designed for infrared point tracking, so they already have visible-light-blocking filters.
Point model
Model configurations
- Single point: a single tracking point, for example attached on a baseball cap or wire that extends forward of the head. Can be easily achieved with a small light source like a penlight. Unlike other configurations, this is limited to two degrees of freedom and can be more sensitive to translation than rotation.
- Three point clip: asymmetrical triangular arrangement of points that can be clipped to headphones and positioned to the side of the head.
- Three point cap: symmetrical triangular arrangement of points, typically attached to the rim of a baseball cap.
- Four point cap: symmetrical non-coplanar arrangement of points, also attached to the rim of a baseball cap.
Active points
An active point model uses visible or infrared LEDs to represent the tracking points, powered by battery, transformer or USB. The electric circuit is very basic and can be made by someone with little or no experience with electronics.Common LEDs, like those found in remote controls, have a narrow, highly focused beam which is not suitable for optical motion tracking. They can be easily turned into wide angle LEDs by filing their lens tips down flat. Alternatively, wide angle LEDs can be purchased from specialist electronics retailers, like the infrared Siemens/Osram SFH485P, with a half-angle of 40 degrees.
Reflective points
Retroreflective material can be used to represent the tracking points by illumination with an infrared light source. This configuration doesn’t require wires or batteries connected to the user but is more susceptible to interference by background light. In most cases a webcam’s internal infrared blocking filter needs to be removed to increase sensitivity enough that the infrared light reflected by the tracking points can be seen.FreeTrack interface
FreeTrack has a simple interface that can be freely used by third party programs to access 6DOF tracking data, both real raw measurements and virtual. It is hardware agnostic, so is not dependent on a specific brand or version of hardware and can be used without restriction. Bohemia Interactive's ARMA 2 is the first game to support the FreeTrack interface and is the first to have exclusive support.TrackIR interface
FreeTrack is compatible with the unencrypted version of NaturalPoint's head tracking TrackIR interface that has widespread support in simulation games. NaturalPoint have been supplying game developers with an encrypted version of the interface for more popular titles since late 2008, these can be identified as requiring TrackIR software version 4.1.036 or higher and are incompatible with FreeTrack.The developers of the first game affected, DCS: Black Shark, tried to release their own head tracking interface but soon after canceled it at NaturalPoint's request. FreeTrack compatibility is still possible using to remove the encryption requirement in games.
TIRViews.dll is a dynamic-link library file distributed with TrackIR software that provides tailored support for a small number of mostly older games, using special interfaces or memory hacks to facilitate view control.
Though a violation of the TrackIR software's EULA,
it is possible to use it with FreeTrack.
NaturalPoint's TrackIR interface SDK is only available under a signed license agreement
and is covered by a NDA, so while FreeTrack is free software, the TrackIR interface component is required to be closed source.