Scientists are one step closer to turning science fiction into reality. A new 3-D display system developed by researchers in Canada, is able to transmit a full-size, 360-degree image of a human that can be seen without any special gadgets like headsets or fast-moving mirrors.
The futuristic technology could usher in a new era of ultra-realistic "telepresence," ranging from remote participation in meetings and conferences to virtual on-stage appearances at concerts.
The projected 3-D image looks real from any angle, and to multiple viewers from different angles, said Roel Vertegaal, a professor of human-computer interaction at Queen’s University in Ontario, Canada, and lead researcher of a new study about the technology. If you walk around to the side or back of the cylindrical screen, he added, you’ll see a side view or back view of the person “inside” it.
“It is quite beautiful,” Vertegaal said. “The screen is not perfect, but the human appears to be standing in the cylinder”
Vertegaal explained that the display system, dubbed “TeleHuman 2,” uses three stereoscopic cameras — which record information about the physical shape of a subject, as well as what it looks like — to take live video of the front, back and sides of a human subject. The information about the image and its 3-D shape, known as a “lightfield,” is then transmitted as data over a network or Internet connection to the 6-foot-high cylindrical screen, or “telepod,” where the virtual human is displayed.
A long time coming
Researchers have been trying for decades to create the sort of 3-D holographic images seen in many sci-fi movies, such as when Princess Leia makes a holographic plea for help to Obi-Wan Kenobi in the original "Star Wars" film, or the fictional "Holodeck" used to create virtual environments on the "Star Trek" TV series. But 3-D projections have been difficult to create in real life because they typically need to be projected onto a surface and can’t just be seen in thin air.
Recently, scientists tried to trap tiny particles with lasers to create 3-D images that appear to float in thin air, with some limited success. Other approaches create this effect using light bouncing off a rapidly rotating mirror. The mirror spins so fast that it appears invisible, while the light reflected from its moving surface results in a 3-D shape.
Some virtual reality systems can also create a 3-D presence, but viewers have to wear VR headsets to see it.
And the so-called “holograms” seen at some stadium shows, like dead rapper Tupac Shakur’s 2012 “appearance” at the Coachella Valley Music and Arts Festival, used a variation of a two-dimensional illusion projected onto a transparent screen, known as “Pepper’s Ghost,” after the 19th-century scientist who invented it.
The new TeleHuman 2 system uses a special reflective screen and an exterior hoop of more than 40 projectors to achieve a 3-D effect from any angle.
Practical applications
Vertegaal thinks the new display system will initially be used for teleconferencing. Thanks to its relatively low Internet bandwidth and computer processing requirements, a conversation between two 3-D “telehumans” would need just six times the bandwidth of a modern 2-D video call, he said.
Conference venues and corporate meeting rooms are places that could be equipped with cylindrical “telepods” for displaying a remote human presence, he added.
The tech could also be used for entertainment purposes, such as at stadium music shows, where remote singers — or even dead stars, like Tupac — can make virtual appearances on stage.
“We expect this to be a practical solution, but our research is always some years ahead of the marketplace,” Vertegaal said. “That's the whole point of research.”
Vertegaal and his team aren’t alone in the race to create 3-D virtual humans.
Related
At the University of Southern California, researchers have been working to create 3-D human images with a variety of systems for more than a decade. In 2009, scientists at the school’s Institute of Creative Technologies demonstrated a 3-D teleconferencing display that uses a fast-spinning mirror as a projection surface.
Paul Debevec, a research professor with the USC group, said their most advanced work on virtual 3-D humans involves an array of more than 200 projectors that project the image from behind onto a special screen.
The system was used to create a life-sized 3-D virtual image of the documentary filmmaker Morgan Spurlock that appeared in a television broadcast. The New York Times also used the projectors to record the testimony of Holocaust survivors for its 2017 documentary "116 Cameras."
Benefits and drawbacks
Debevec thinks the Queen’s University display may struggle to show an accurate 3-D image to viewers at different heights, or who change their height while viewing the display.
“That's why we design our systems so that the image of the person can be very close to the surface of the screen and still provide the range of views we need to project,” he said. “Sadly, that doesn't work for a cylindrical reflective screen.”
But David Brady, a professor with the imaging and spectroscopy program at Duke University’s School of Engineering in Durham, North Carolina, sees the TeleHuman 2 display as progress towards the sort of 3-D virtual worlds seen in science fiction shows.
“While it is limited to a view of just one person, it is the best approximation we have to the natural full-color 3-D world envisioned in Star Trek’s Holodeck,” Brady said.