Virtual Reality Draws Closer with Haptics 11/22/2000

With cameras flashing around him, Oussama Khatib moved his hand on a small joystick-like knob, causing a large robotic arm to emulate the motion of his hand.

Here on the first floor of the Gates Computer Science Building, Khatib, a professor of computer science and head of the Stanford Robotic Manipulation Group, was demonstrating the latest advances in robotics to a group of visiting high school teachers from Bermuda.

The visitors were awed time after time as Khatib gave demonstrations of his newest toys. Even Computer Science Prof. Eric Roberts, who was leading the teachers around and who had probably seen many of the demonstrations before, was fascinated.

After the demonstration, Khatib discussed recent innovations in robotics, notably in the are of haptics.

Haptics is a sub-field of robotics that deals with mechanically simulated physical feedback that will cause the user of a robotic system to "feel." For example, in one demonstration on the first floor of Gates, the user can use a metal knob to move a ball around several polygons on a computer screen. The user feels resistance from the metal tip that simulates how it would feel if the user were physically moving his fingers around polygons. When the user moves the computer ball around a cylinder, the user's fingers can feel the smooth surface of the cylinder. When he moves the ball around an octagon, he feels the ridge at each corner of the octagon.

According to Khatib, advances in the three-dimensional computer graphics environment, as well as the natural maturity of research in the field of haptics, have come together to create some promising developments within the last five years.

Kenneth Salisbury, a professor in computer science and co-director of the Center for Advanced Technology in Surgery at Stanford, noted, “The advent of haptic interface technologies in the last five years have inspired people [to come up with new ways to use this technology].”

Khatib envisions the day when haptics technology will be advanced enough for astronauts to be able to train doing construction work in space. Astronauts will be using simulated tools and materials that feel as if they were being used in space. Or, for that matter, what if an astronaut could control a robot by feeling the same forces as the robot?

While such sophisticated systems may not exist for a long while, haptic technology will soon be used in medicine. Groups at Stanford have been working to create haptic simulations so that doctors can train by doing mock surgeries.

Already, companies such as HT Medical Systems, located in Gaithersburg, Maryland, have developed haptic systems where nurses can train for putting catheters into arms for intravenous fluid injections. A trainee applies a catheter to a rubber arm. The catheter is attached to a mechanical system that simulates the pressure that the trainee would feel if he were indeed injecting the catheter into a real arm. On a computer screen, she can also see the arm as it would look with the catheter being applied.

Stanford’s Center for Advanced Technology in Surgery is also attempting to simulate the feel of soft tissue using haptics. Eventually, doctors will be able to simulate performing endoscopic surgeries. Endoscopic surgeries are surgeries performed non-invasively; instead, small holes are drilled into the site of the surgery. This type of surgery minimizes patient recovery time.

Salisbury described two models for emulating the feel of soft tissue currently being perfected.

One is the “spring and math” model. Soft tissue is simulated on the computer using several virtual weighted balls that are connected to each other through a series of springs. Depending on the tissue being simulated, the weight of the balls and the elasticities of the springs are adjusted to account for differences in each human organ.

Another model is the “volumetric” model, where human tissue and organs are simulated by many little virtual blobs put together. Each blob is like Jell-O, and has its elasticity and weight preset by the computer so that it acts like a little piece of the appropriate organ.

There are several problems with applying these techniques, however. François Conti, a first year doctoral student in computer science and a member of Khatib's group, explained that it was difficult to compute what elasticities these little springs or blobs should be set at without testing them with a real organ.

While organs removed from humans have been used for testing, Conti noted, “A dead liver isn't like a real liver.” He said that, when touched or manipulated, dead samples react differently from live organs.

Accurately simulate the feeling of soft tissue in real time is another problem that researchers face. Simulating the feel of polygon edges is one thing; simulating live tissue is quite another.

Nevertheless, Salisbury is confident that these obstacles are only temporary, and cautiously predicted that an evaluation-stage endoscopic surgery trainer will be possible in four or five years.

Khatib is also excited about future prospects in this field. “Haptics is one of those projects that bring people in from medicine, computer science, biomechanics.” With the development of the Bio-X program at Stanford, he sees progress as accelerating in the next few years.

The Bio-X program is a developing interdisciplinary program that brings together the fields of biology, chemistry, physics, engineering and medicine. The program was largely made possible by a $150 million donation by Netscape founder and former Computer Science Prof. Jim Clark.

And as for the day when doctors will be able to perform real surgeries using haptics and robot arms? Conti said that that would be really useful, as the computer would then be able to filter certain erroneous surgeon movements such as hand shaking during surgery.

Unfortunately, such applications are a long way off. Salisbury said that new algorithms and ways are needed before robot arms will be able to send the sensation of live tissue back to a physician's arm. However, Salisbury, Khatib and other researchers working on haptic technology enjoy the challenge. “it’s very fun,” Salisbury said, “And very scary.”