FreeTrack

FreeTrack is a general-purpose optical motion tracking application for Microsoft Windows, released under the GNU General Public License, that can be used with common inexpensive cameras. Its primary focus is head tracking with uses in virtual reality, simulation, video games, 3D modeling, computer aided design and general hands-free computing to improve computer accessibility. Tracking can be made sensitive enough that only small head movements are required so that the user's eyes never leave the screen.

FreeTrack
Stable release
v2.2 / October 7, 2008
Operating systemMicrosoft Windows
TypeOptical motion tracking
LicenseGNU General Public License
Websitewww.free-track.net

A camera is positioned to observe a rigid point model worn by the user, the points of which need to be isolated from background light by means of physical and software filtering. Motion is tracked with up to six degrees of freedom (6DOF): yaw, pitch, roll, left/right, up/down and forward/back. Windows-compatible video devices like webcams are supported, as well as special Nintendo Wii Remote camera and NaturalPoint cameras (TrackIR, SmartNav and OptiTrack).

FreeTrack can output head-tracking data to programs directly using its own open interface, as well as TrackIR, SimConnect and FSUIPC interfaces. Programs that support these interfaces are regarded as being FreeTrack-compatible. FreeTrack can also emulate mouse, keyboard, and joystick (via PPJoy) if a program does not support a direct interface.

FreeTrack is coded in Delphi 7 and uses DirectShow and DirectX. Head tracking is achieved using implementations of DeMenthon's four-point iterative pose estimation algorithm (POSIT) [1] and Alter's three point geometric algorithm.[2]

Software

FreeTrack uses a camera to collect real-time information on the point model worn by the user. Specifically the image coordinates of the model points, which are either received directly from the camera or extracted from a video stream. These coordinates are used to generate an estimate of the real head pose, which can be transformed by the user in a number of ways to create a virtual pose. One of the most fundamental transformations involves amplifying rotation so that only small head movements are required. Finally, the virtual pose is sent to the user's choice of outputs. This is all done in the background, with tracking status displayed in the system tray.

A 3D preview is available that shows the virtual head position and orientation for a given real head pose and can be viewed from multiple perspectives, including first-person. This greatly assists with testing and makes it easier to experiment with different settings.

Each degree of freedom (axis) has a response curve that can be modified to change the way the virtual head moves for a given real head movement. This is commonly used to create a central deadzone region so that the user’s head can be more relaxed there.

Keyboard, mouse and joystick buttons can be used to toggle tracking settings, including the virtual centre location (like adjusting the seat position in a car) and individually toggle axes and outputs.

For NaturalPoint cameras, FreeTrack can provide advanced features and a level of customization that is not available with official software.

Camera
Comparison of some cameras compatible with FreeTrack
Camera Sensor resolution FPS Sensor Angle(°) Output CPU usage Subpixel precision IR LEDs Approx. price (USD)
Ideal webcam 640×480 ≥60 monochrome 42 highly compressed small Software-dependent Yes ?
OEM IR webcam[3] 640×480 30 color 42 JPEG compressed small Software-dependent Yes $5
Sony PlayStation EyeToy 640×480 30 color 56 JPEG compressed[4] small Software-dependent No $16
Sony PlayStation 3 Eye 640×480 187@320x240(CLEye), 125@320x240(Directshow), 75@640x480(DirectShow)[5] color 75, 56 JPEG compressed, raw small Software-dependent No $24
Microsoft Xbox Live Vision 640×480 60@320×240, 30@640×480 color ? JPEG compressed,[6] raw small Software-dependent No $14
Nintendo Wii Remote 128×96 100(Bluetooth), 250(I2C)[7] color 41 point coordinates none 1/8 No $23
NaturalPoint TrackIR 1 [8] 60k pixels (e.g. 300×200) 60 monochrome 33 binary threshold minimal Yes retired
NaturalPoint TrackIR 2 [8] 60k pixels (e.g. 300×200) 100 monochrome 33 binary threshold[9] minimal Yes retired
NaturalPoint TrackIR 3 [10] 355×288 80 monochrome 33 binary threshold minimal Yes retired
NaturalPoint TrackIR 3 Pro [10] 355x288 120 monochrome 33 binary threshold minimal Yes retired
NaturalPoint TrackIR 4 Pro [10] 355×288 (subsampled at 710×480) 120 monochrome 46 binary threshold[11] minimal 1/20th Yes $99.95
NaturalPoint TrackIR 5 [10] 640×480 120 monochrome 51.7 grayscale threshold[12] minimal 1/150th Yes $149.95
NaturalPoint SmartNav 1/2 60k pixels (e.g. 300×200) 60 monochrome 33 binary threshold minimal Yes retired
NaturalPoint SmartNav 3 [13] 355×288 120 monochrome 33 binary threshold minimal 1/20th Yes retired
NaturalPoint SmartNav 4[13] 640×480 (subsampled at 1280×480) 100 monochrome 41 grayscale threshold[14] minimal 1/150th Yes $400 to $500

Resolution

In most cases a resolution of 320×240 is sufficient, this is capable of producing a much higher sub-pixel resolution, enough to allow accurate cursor control on a high-resolution monitor. Resolutions 640×480 and above have diminishing returns and correspond to an exponential increase in CPU usage when not sufficiently compressed before reaching the computer. Higher resolutions become more important at greater distances from the camera. The Wii utilizes a low-resolution 128×96 sensor, which is found by some to produce jittery tracking and may require smoothing to improve stability at the cost of decreased responsiveness.[15]

Sensor

For the same resolution, monochrome sensors can resolve finer details much better than color sensors due to the lack of a color filter array.

Frame rate

FreeTrack uses interpolation with low-frame-rate video devices to improve panning smoothness. However, responsiveness is fundamentally limited to the frame rate; a 30 frame/s webcam has a maximal response delay of 33.3 milliseconds compared with 8.33 milliseconds for a 120 frame/s camera. To put this into perspective, a human’s reaction time to visual stimulus (finger reflex) is typically around 200 ms; 30 ms can be regarded as a competitive ping in online reflex-based games, and an LCD monitor refresh rate is typically 17 ms.

Higher responsiveness gives a greater feeling of control, but since virtual head motion is amplified, it can also cause it to move unrealistically fast. For this reason, some programs limit head movement speed, wasting some of the responsiveness of higher-frame-rate cameras.

Angle

A wider viewing angle allows a larger tracking region when in close proximity to the camera. At further distances a wide angle is not desirable, more of the frame is unused and the effective resolution drops more rapidly. More peripheral light can also be seen, which can interfere with tracking. Viewing angle can be reduced by using digital zoom at the cost of resolution.

CPU usage

The Nintendo Wii Remote effectively uses no CPU, NaturalPoint cameras use a small amount, and general video devices can use a significant amount, depending on the brand and the specific camera settings in use. A PlayStation Eye running at the same resolution and frame rate as a TrackIR 4 would be very demanding on a single-core CPU. However, modern multi-core CPUs are making this less of an issue. Resolution and frame rate can always be reduced to conserve CPU resources.

Filters

FreeTrack requires the tracking points to be isolated from all other light; this is best done using infrared LEDs and a visible-light blocking filter in front of the camera. Photographic film or the magnetic storage medium inside floppy disks can be used as inexpensive visible-light filters. Further filtering can be done in software by adjusting exposure and threshold.

All video devices like webcams have a built-in infrared-blocking filter, which can be removed to improve sensitivity to infrared light, allowing better point isolation and the possibility of retroreflective tracking. This is normally a straightforward and reversible procedure for most webcams.

Wii Remotes and NaturalPoint cameras are designed for infrared point tracking, so they already have visible-light-blocking filters.

Point model

Model configurations
  • Single point: a single tracking point, for example attached on a baseball cap or wire that extends forward of the head. Can be easily achieved with a small light source like a penlight. Unlike other configurations, this is limited to two degrees of freedom (yaw and pitch) and can be more sensitive to translation than rotation.
  • Three point clip: asymmetrical triangular arrangement of points that can be clipped to headphones and positioned to the side of the head.
  • Three point cap: symmetrical triangular arrangement of points, typically attached to the rim of a baseball cap.
  • Four point cap: symmetrical non-coplanar arrangement of points, also attached to the rim of a baseball cap.

Models can be made in a DIY fashion at minimal expense using readily available electronic components. Component kits and fully constructed models are also available for purchase from some members of the FreeTrack community.

Active points

An active point model uses visible or infrared LEDs (5 mm or larger) to represent the tracking points, powered by battery, transformer (plug pack) or USB. The electric circuit is very basic and can be made by someone with little or no experience with electronics.

Common LEDs, like those found in remote controls, have a narrow, highly focused beam which is not suitable for optical motion tracking. They can be easily turned into wide angle LEDs by filing their lens tips down flat. Alternatively, wide angle LEDs can be purchased from specialist electronics retailers, like the infrared Siemens/Osram SFH485P, with a half-angle of 40 degrees.

Reflective points

Retroreflective material can be used to represent the tracking points by illumination with an infrared light source. This configuration doesn’t require wires or batteries connected to the user but is more susceptible to interference by background light. In most cases a webcam’s internal infrared blocking filter needs to be removed to increase sensitivity enough that the infrared light reflected by the tracking points can be seen.

FreeTrack interface

FreeTrack has a simple interface that can be freely used by third party programs to access 6DOF tracking data, both real raw measurements and virtual. It is hardware agnostic, so is not dependent on a specific brand or version of hardware and can be used without restriction. Bohemia Interactive's ARMA 2 is the first game to support the FreeTrack interface[16] and GP Bikes is the first to have exclusive support.[17]

TrackIR interface

FreeTrack is compatible with the unencrypted version of NaturalPoint's head tracking TrackIR interface that has widespread support in simulation games. NaturalPoint have been supplying game developers with an encrypted version of the interface for more popular titles since late 2008, these can be identified as requiring TrackIR software version 4.1.036 or higher and are incompatible with FreeTrack.[18] The developers of the first game affected, DCS: Black Shark,[19] tried to release their own head tracking interface but soon after canceled it at NaturalPoint's request.[20] FreeTrack compatibility is still possible using TrackIRFixer to remove the encryption requirement in games.[21]

TIRViews.dll is a dynamic-link library file distributed with TrackIR software that provides tailored support for a small number of mostly older games, using special interfaces or memory hacks to facilitate view control.[22] Though a violation of the TrackIR software's EULA,[23] it is possible to use it with FreeTrack.

NaturalPoint's TrackIR interface SDK is only available under a signed license agreement[24] and is covered by a NDA, so while FreeTrack is free software, the TrackIR interface component is required to be closed source.[25]

gollark: <@341618941317349376> What do you think of WHYJIT?
gollark: ZJ?
gollark: So I think it's not actually Turing-complete but interesting at least.
gollark: <@!330678593904443393> https://squiddev.github.io/ae-sat/
gollark: ```/tmp/ignore-me: In function ‘main’:/tmp/ignore-me:10:3: error: invalid preprocessing directive #! #!/bin/sh ^/tmp/ignore-me:11:1: error: ‘TMP1’ undeclared (first use in this function) TMP1=/tmp/ignore-me ^~~~/tmp/ignore-me:11:1: note: each undeclared identifier is reported only once for each function it appears in/tmp/ignore-me:11:6: error: expected expression before ‘/’ token TMP1=/tmp/ignore-me ^/tmp/ignore-me: In function ‘main’:/tmp/ignore-me:23:3: error: invalid preprocessing directive #! #!/usr/bin/env python3 ^/tmp/ignore-me:24:1: error: unknown type name ‘import’; did you mean ‘short’? import argparse ^~~~~~ short/tmp/ignore-me:25:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘import’ import subprocess ^~~~~~/tmp/ignore-me:34:13: warning: missing terminating " character thing = f""" ^/tmp/ignore-me:34:13: error: missing terminating " character/tmp/ignore-me: In function ‘main’:/tmp/ignore-me:43:3: error: ‘code’ undeclared (first use in this function) {code} ^~~~/tmp/ignore-me:43:7: error: expected ‘;’ before ‘}’ token {code} ^ ;/tmp/ignore-me: In function ‘main’:/tmp/ignore-me:45:4: warning: missing terminating " character """ ^/tmp/ignore-me:45:4: error: missing terminating " character/tmp/ignore-me:45:4: error: expected ‘;’ before ‘return’ """ ^ ; return thing ~~~~~~/tmp/ignore-me: In function ‘main’:/tmp/ignore-me:65:1: error: unknown type name ‘iuvusugdjuczwenvxaiexhuvojlciybswpjpsqlmkkiuzxsjslkkckukhfqomqfghfonictwwbsraiffwiykmzrjspdsxkluiezu’ iuvusugdjuczwenvxaiexhuvojlciybswpjpsqlmkkiuzxsjslkkckukhfqomqfghfonictwwbsraiffwiykmzrjspdsxkluiezu ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~/tmp/ignore-me:66:10: error: expected expression before ‘/’ token COMPILER=/usr/bin/gcc ^/tmp/ignore-me:71:1: error: expected declaration or statement at end of input } ^```

See also

References
  1. DeMenthon, Daniel; Larry S. Davis (1992). "Model-Based Object Pose in 25 Lines of Code". European Conference on Computer Vision. 15: 335–343. CiteSeerX 10.1.1.50.9280.
  2. Alter, T. D. (1992). "3D Pose from Three Corresponding Points Under Weak-Perspective Projection" (AIM–1378): 43. CiteSeerX 10.1.1.18.1908. Cite journal requires |journal= (help)
  3. "8.0 Mega 6 IR LED Webcam Web Cam Camera Skype MSN Mic". Retrieved 2010-09-07.
  4. "Using ov519 webcams (Eyetoy) with pdp/Gem (jpeg frames)". Retrieved 2010-05-08.
  5. "CL Eye Platform SDK Changelog". Retrieved 2010-10-30.
  6. "XBOX Live Vision Camera in Ubuntu". Retrieved 2010-05-08.
  7. "Automatic Take Off, Hovering and Landing Control for Miniature Helicopters with Low-Cost Onboard Hardware" (PDF). Retrieved 2010-05-08.
  8. "TrackIR3 Pro heads-up game controller". ars technica. 25 August 2004. Retrieved 2007-10-13.
  9. "TrackIR2, Track IR2 headtracking buy, review, featured". Retrieved 2010-05-08.
  10. "TrackIR Product Comparison". NaturalPoint. Retrieved 2007-10-13.
  11. "TrackIR 4 Grayscale". Retrieved 2010-05-08.
  12. "TrackIR 5 Grayscale". Retrieved 2010-05-08.
  13. "SmartNav Older Model Comparison". NaturalPoint. Retrieved 2008-11-01.
  14. "SmartNav 4 Grayscale". Retrieved 2010-05-08.
  15. "Wii resolution and latency". Retrieved 2010-12-07.
  16. "Arma 2: Patch v1.05". Retrieved 2010-07-20.
  17. "PiBoSo Alpha 6 released". Retrieved 2010-03-16.
  18. "NaturalPointofView - The NaturalPoint TrackIR Monopoly". Retrieved 2010-07-20.
  19. "TrackIR Enhanced Games : DCS: Black Shark". NaturalPoint. Retrieved 2008-10-26.
  20. Tez - ED Team. "HeadTracker interface - ED Forums". Eagle Dynamics. Retrieved 2010-03-16.
  21. "NaturalPointofView - The NaturalPoint TrackIR Monopoly: TrackIRFixer". Retrieved 2010-07-20.
  22. "FreeTrack Forum V2.2 & FSX/FS9". Retrieved 2010-02-20.
  23. "TrackIR software download page". NaturalPoint. Retrieved 2010-02-20.
  24. "TrackIR Developers : Which SDK Do I Need?". Retrieved 2010-02-20.
  25. "Head banging..." Archived from the original on 2011-06-05. Retrieved 2010-02-20.
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.