Computer scientists bring polar region to life using virtual reality

Scott Sorensen’s experiences as a computer science student at the University of Delaware were hardly typical.

While he spent his share of time in windowless rooms staring at computer screens, he also made five trips to the Arctic Ocean as part of his doctoral research. And those travels put Sorensen, who received his Ph.D. from UD this past May, on a different career course than he originally imagined for himself.

Sorensen was part of a research group led by Prof. Chandra Kambhamettu, who directs the Video/Image Modeling and Synthesis (VIMS) Lab in UD’s Department of Computer and Information Sciences. Kambhamettu is particularly interested in applying big data computer vision and graphics to problems that impact life on earth, including climate change and the assessment and quantification of marine mammal habitat.

Computer vision uses signals from cameras for a broad range of applications, including scene reconstruction, obstacle detection, panorama stitching, facial recognition and position tracking. Various techniques are employed to fill in gaps and improve the clarity and quality of the collected images.

But despite the improvements afforded by some of the image enhancement techniques developed and used in the VIMS Lab, Sorensen was disappointed when he looked at the results on a computer screen.

“Three-dimensional images lose something on a two-dimensional screen,” he says.

So to provide a more realistic view of the Arctic—which is home to polar bears, walrus, and other wildlife whose habitats are changing with global warming — Sorensen turned to the virtual reality technology used by gamers.

With the setup he developed, the user can put on a headset and instantly be on an icebreaker, a special type of ship designed to break ice and make way in extremely cold regions. The system currently incorporates three prototype applications, which Sorensen eventually hopes to combine into one coordinated package.

The first provides an overall perspective of what the Arctic looks like, while the second offers a view into ice patterns and wildlife activity through thermal and visible images taken from a ship. Sorensen’s latest addition literally takes things to a global scale. With a pair of special controllers, the user can “grab” a globe and move it around on a virtual timeline to see how the Arctic region has changed during the past decade.

“We took consumer hardware and leveraged it not only for research but also for education and training,” says Sorensen, who is now working for Vision Systems Inc., a computer vision company based in Providence, Rhode Island. “Polar regions pose unique challenges to the hardware and personnel working in these places, and this approach provides a more tangible and realistic view of that environment.”

“With virtual reality, people are so interested in the medium that they listen to the message,” he adds.

Sorensen’s work, which was funded by the National Science Foundation, won the best student paper award at last year’s IEEE Applied Imagery Pattern Recognition Conference on Imaging and Artificial Intelligence. It is part of a chain of discovery, building on research done by others in Kambhamettu’s group. For example, in 2012, in a different NSF study, they reported the development of a novel camera system to map the surface topography of Arctic sea ice, which is a challenge because ice is white and has no visual texture, making it difficult to reconstruct in 3-D.

Just as Sorensen’s virtual reality system took advantage of the enhanced images generated using this novel camera apparatus, others in the VIMS lab will undoubtedly now enhance and expand the capabilities of his technology.

“Each step in the research is helping us to develop new tools that can help shed light on the polar regions and the life there,” says Kambhamettu. “Our overall goal is to build a database of habitat and other information for use by scientists and engineers, as well as develop new approaches to educate the public about how the topography and animal habitats are changing over time.”