The first step toward creating the avatars is to build a motion capture studio equipped with digital cameras. The cameras will capture the body and facial movements of real people. Virtual characters can then be built using the motion capture data and special modeling software.
The UIC team will pay special attention to improving the lip motions of the animated characters they create, so that spoken words are closely synched with mouth movements. By the end of the third year, they hope to have facial recognition technology at a point where it can analyze a person's expression and provide an appropriate response.
But gaming technology and artificial intelligence may not be enough to achieve such natural results, said Michael Zyda, director of the University of Southern California's GamePipe Laboratory. Emotions need to be folded into the model too, he said.
"It's a separate thing from artificial intelligence. What can we do in modeling to give you the illusion that there is an emotional state?" said Zyda.
"We need sensing technology that allows us to sense real humans' emotional state, and then we can build models for the virtual state," he said.
After all, an empathetic Newton makes for a better teacher.
