Multiple Input & Multiple Output - 3-D Multimedia Contents Sharing among multiple devices

Development of a system for sharing 3-D virtual environment using multiple devices

Main purpose of the system is generating interactive 3-D virtual environment using multiple devices. Initial idea comes from making use of multiple device, which we can find in our daily lives, for generating immersive virtual environment, not using serious 3-D VR equipment, such as HMDs or CAVE. Consider their poses in 3-D environment, they act like conceptual windows which use can see VR world through them. Among the multiple devices, most important component is mobile device. According to user's movement, it senses poses of devices with its camera sensor and sends the information to main server. In the main server, it generates proper images to each device.

Multiple Input & Multiple Output - 2-D Multimedia Contents Sharing among multiple devices

Information Sharing with multiple device

Considering current poses of each of device, it shows one 2-D multimedia contents over the screens. Main problems of this research are automatic detection of devices's pose and synchronization of each screen image. For the first, to acquire knowledge of poses of devices, we use vision-based approaches. Each device shows reference image on the screen before it begins and system estimates poses with those images. After getting pose information, system server starts streaming video to each of devices. Considering current bandwidth of the network and capability of devices, in main server, it adaptively resize the image for each device.  

IEEE VR 2015 Conference "Multiple Devices as Windows for Virtual Environment"

Multiple Devices as Windows for Virtual Environment

Abstract

In our research, we introduce a method for using multiple devices as windows for observing 3-D virtual environment. Motivation of our work has come from generating collaborative workspace with multiple devices which can be found in our daily lives, like desktop PC and mobile devices. Provided with life size virtual environment, each device shows a scene of 3-D virtual space on its position and direction, and users would be able to perceive virtual space in more immersive way with it. By adopting mobile device to our system, users not only see outer space of stationary screen by relocating their mobile device, but also have personalized view in working space. To acquiring knowledge of device’s pose and orientation, we adopt vision-based approaches. For the last, we introduce an implementation of a system for managing multiple device and letting them have synchronized performance.