Tracking Competition

Many tracking technologies for Augmented Reality have been proposed so far. Some have made their way into commercial products, some are freely available as open-source software, and others are still in the development phase. To allow a fair comparison of the state-of-the art a tracking competition is being organized at ISMAR’08 in Cambridge.

Tracking quality is to be evaluated on a concrete and reasonably realistic industrial task. The task is inspired by a logistics scenario where a worker is given a list of items to pick in a storage room. These items are only described by their 3D-coordinate in a global reference coordinate frame. The worker has to be guided to the objects by visually overlaying the object location onto the worker’s view.

As the objects to be picked decrease in size, and the picking areas are in different corners of a large room, both locally accurate and wide-area tracking is needed. We make no restrictions on the type of tracking equipment used, and the competition is open to both academic and commercial participants.

Participants must bring their own hardware and we will not provide software for the visualization. However, we estimate that a talented undergraduate student could write an appropriate visualization in a short time, provided the hardware, the tracking system and registration tools are available.

The competition runs will start at the beginning of the last hour of each slot, i.e. 13:00 or 18:00. All conference attendees are invited to come and watch.

The winner will receive the sum of EUR 1.000. The money is sponsored by Intel, EADS and HDW.

The results of the competition will be presented at the end of the main conference.

We will also write a report on the results of the competition that will be published. All participants are encouraged to contribute a section describing the approach they have taken.

The tracking competition takes place during the main conference (September 16—18th). Each team has about half a day for setup, object picking and removal of their equipment. Depending on participation, we will also use the workshop day (Monday, September 15th).

In order to register, participating teams have to submit a one-page summary of their approach. The summary should cover the following topics:

• What tracking technology will be used
• What needs to be installed in the room
• A short description of the mobile setup
• Contact details

The deadline for proposals is July, 14th. There are still some slots available. The remaining slots will be allocated on a first-come-first-served basis, so please register as soon as possible.

The summaries are to be set to: ismar-trackingcompetition@ismar08.org

For further information, we have created a mailing list at

http://mailnavab.in.tum.de/mailman/listinfo/ismar-trackingcompetition

Please use this mailing list when asking questions about the general contest rules, so that everybody can profit from the answers. The mailing list also has a publicly accessible archive. We will also post updates on the competition to this list.

The tracking competition is organized by Daniel Pustka (Technische Universität München), Joseph Newman (University of Cambridge) and Gudrun Klinker (Technische Universität München).

1. The competition takes place in a large room (approximately 15m x 8m x 3.5m). In order to simulate a storage room environment, the room will be separated into different areas by moveable walls, shelves or similar objects. In particular, there will be no single place from which the whole room can be observed.
2. The organizers will define a global coordinate frame in the room using geodetic equipment. For registration with the participant’s tracking system, a number of reference points, spread throughout the whole room, will be provided. The coordinates of these reference points are made available to the participants before the setup phase. The shape of a reference point is shown in the following picture:
   
3. The unit used for all coordinates is meters.
4. Participants are not allowed to make permanent modifications (e.g. drilling holes) to the room or its equipment.
5. The room will contain a number of designated “picking areas” from which the objects are to be picked. The extent of these areas will be clearly marked. Participants are not allowed to place markers or any other objects into these areas. Examples of picking areas are shown in the following images:
   
6. In order to provide different levels of difficulty, the objects to pick will become increasingly smaller.
7. Participants can assume the availability of reference points near the picking areas so that precise local coordinate frame alignment should be possible.

1. The organizers will not provide binaries or source code for visualization.
2. The location of the object to pick must be visualized on a tracked mobile display by highlighting it in a video image or by otherwise overlaying it onto the user’s view.
3. The location of the object must be visualized by showing a simple 3D (e.g. sphere) or 2D (e.g. crosshair, circle) object registered at the given coordinates.
4. If the object to pick is outside the user’s view, a 2D arrow must be displayed to indicate the direction to look to find the object.
5. Any other type of output (e.g. textual) is not allowed. In particular, the visualization must not make use of a map of the environment.
6. The system must provide a mechanism to select the next object, such as a button or speech input.

1. The display hardware must be mobile such that it can be carried by a single user while performing the picking task. It is not permitted to install display devices (e.g. projectors) in the environment.
2. There are no other restrictions on the type of tracking or display equipment used.

1. In order to install equipment and perform registration with the room coordinate frame, each participant has exclusive access to the room for a maximum of 5 hours.
2. The setup phase is supervised by the organizers who will also measure the actual time used.
3. The setup phase is directly followed by the actual contest run (see section 5) takes place.
4. After the contest run, participants have to remove any equipment they installed in the room within 1 hour.
5. For a dry run of their systems, participants will be provided with coordinates of a few known objects.
6. The system will be tested by the organizers after setup, to make sure that it complies with the rules described in section 2.

1. At the beginning of the contest run, participants will receive the list of objects on a USB memory stick. The memory stick holds a Unix-format text file named “Contest.dat” which contains the 3D coordinates of the objects. For each object, the file contains a line with three floating point numbers, separated by spaces. An example file is provided upon request. Participants are not allowed to display the contents of the file in any other way than described in section 2.
2. The contest run is performed by a single person(“the user”) selected by the participant.
3. Only the mobile equipment carried by the user may be moved during the contest run. All other equipment must stay in place.
4. During the contest run, the user has to pick the objects in the given order.
5. All users start the contest run at the same location and have to return to it after having picked all objects.
6. The contest run is supervised by the organizers who will note the objects that were picked and measure the required time.

1. The timing starts when the location of the first object is displayed at the starting location and ends when the user has returned to the starting location.
2. The organizers will videotape the contest run to present the results to the ISMAR audience at the end of the conference.

1. The winner will be determined by evaluating the following criteria (in this order)
   • Number of correctly picked objects
   • Time needed to pick the objects
   • Time needed for setup