Field Trip: GestureTek

On Friday morning, October 12, 2007, I along with my classmates and my teacher Dan Zen visited GestureTek at downtown Toronto.

At GestureTek we met Mr. Vincent John Vincent, president of GestureTek Inc. Almost twenty years ago from now he, in collaboration with his business partner Francis Macdougall, pioneered the idea of human computer interaction using video cameras. Today GestureTek is world leader in gesture recognition technology. GestureTek has developed a system where you can use your hands and body instead of a mouse, keyboard, joystick and touchpad to control the interaction on the computer like: accessing information, controlling interactive interface, and immersing yourself in interactive 3D games and controlling the movement, for instance we can say like in Wii, but it is technically different from Wii. GestureTek’s system is designed to work with any computer devices like Desktops, Laptops, Mobile Phones, Consoles, and Public Displays etc. and can work on surfaces like floors, walls, panels, screens etc.

The moment I entered GestureTek I saw an interactive screen display right in front of the entrance which had GestureTek’s logo on a white background. As I started moving my hands in front of the screen the logo’s characters started bouncing with my gestures and that white background started wiping off with the movement of the hands and I could see my self in the screen for that moment, like clearing off the vapour from a mirror.

The first thing Mr. Vincent introduced is called HoloPoint, which can track the movement of the finger by placing the finger in the control frame that has two cameras on left and right side of the frame. This was very interesting and it was my first time to see anything like that in real. We don’t need no mouse or pointing device to control the interface. It is going to change the way we use computers or interactive environments.

Then he took us system called GestureXtreme. It was a green screen where we could virtually immerse ourselves into the 3D environment. It transports our image into a computer generated landscape without use of any gear or interactive device like joystick etc. I was fascinated when I saw myself on screen and interacted with onscreen characters in real time. My body motion controlled the program. I tried the soccer game where I was the goalie and I had to stop the ball coming into the goal by the movement of my hands, head or legs. The best part of this type of gaming environment is that we exercise physically also. This system of GestureTek is already being used by the news channels for the display of weather forecast.

The next thing we saw was HoloFrame, which was a finger tip control 60” projection screen which creates a sense of floating image in a dark room that allows the user to control and interact with the display just by pointing and moving fingers on it. Initially I thought it was a conventional touch screen, but then I asked Mr. Vincent how it works. Two tiny cameras imbedded in the screen track the movement of the hand or where we point. He mentioned the usage of this technology for Ford. The most interesting part was when Mr. Vincent mentioned about the possibility of having to try the clothes in store without actually wearing them.

Afterwards we saw an interactive floor which had eye-catching realistic visuals which intelligently responded to the body’s movements. I saw a couple of ads where we could play with the interactive floor. Three of my classmates tried the interactive racing game on the floor where they had to continuously shake their feet over the steps. Karen was the winner every time. I was impressed how it has or it is going to revolutionalize the advertising and entertainment in public space. Just use your hand and feet to control and interact in real time with dynamic ads, games, and images etc.

Next we saw a multi touch interactive table which looked like Microsoft Surface . I’m still wondering If GestureTek has licensed Microsoft for the use of their technology or Microsoft has created it on their own. Technically the interactive table was similar to the interactive floor, but in this case the floor was the glass on the table. I got a chance to play a ping-pong game. Though it was not very accurate but it was interesting. I also tried the interactive table where we had a virtual photo album on the screen and we could move, place, and change the size of the pictures in the way we like. It would be a good idea of having such a thing in restaurants or clubs.

After that Mr. Vincent showed us the interactive wall which was based on their GestureFX System Series like the interactive floor and interactive table. This is a very effective way to utilize walls for advertising and interactive games.

At last we saw the GestureTek’s AirPoint System which is from the GestPoint family of products like HoloPoint and HoloFrame. It has a similar architecture like HoloPoint; having two cameras and a couple of infrared capturing devices which capture motion of hands in a matrix. It is designed for public installations like conference rooms and can handle any lighting conditions.

GestureTek has created so many possibilities with this technology. DoCoMo, a Japanese company, in collaboration with GestureTek has started providing this innovative technology on mobile phones, which lets the user to browse maps and play videogames by tracing a finger or by tilting or waiving the handset etc. Examples can be seen here, here and here

GestureTek has also designed a system called IREX which will help the healthcare professionals use the gesture recognition technology for their patients in rehabilitation and exercise and may help them recover from disabilities. Also, it will help physically impaired people to interact with computer by the interpreting sign language

As far as technology and ideas are concerned, it was all very interesting and I can’t wait for the day when it will be implicated in our daily lives. No more of being couch potatoes and those old computer gaming interfaces because now we will be able to exercise physically. I don’t know if this will interest people in terms of health, but everyone has a different vision to it. The tour was a demo of how technology is going to impact our future lives.

Related Links:

● GestureTek Home
● About Computer Vision Technology
● GestureTek Mobile™ in Use
● Motion Sensing in Camera Phones
● Microsoft Surface
● Rehabilitation in Action
● Xbox 360 using GestureTek’s technology
● Working of Multi-Touch System
● Play Station to sense user emotions
● EyeToy using computer vision technology
● Replacing Keyboard and Mouse