NYU is developing 3D video streaming technology with the help of its dance department

NYU is launching a project to stimulate the development of immersive 3D video for dance education and perhaps other fields. The commitment, supported by a four-year $1.2 million grant from the National Science Foundation, will work to make Point-Cloud Video (PCV) technology viable for streaming.

A point cloud is a collection of data points in a 3D space that represents the surface of an object or environment. Point-Cloud Video, which stitches together point-cloud frames into a moving scene, has been in development for the past decade, NYU says. However, it is usually too data intensive for practical purposes and requires bandwidth far beyond the capabilities of today’s connected devices.

According to an NYU engineering blog post published Monday, the researchers plan to address these obstacles by reducing bandwidth consumption and delivery latency, and increasing energy efficiency so that PCVs can be deployed more easily. Project leader Yong Liu, professor of electrical and computer engineering at New York University, believes that modern developments make this possible. With recent advances in key enabling technologies, we are now on the verge of completing the puzzle of transmitting holograms of real-world humans, creatures and objects over the global Internet, Liu wrote on Monday.

ChatGPT developer OpenAI launched a model last year that can create 3D point clouds from text messages. Engadget reached out to the project leader to find out if it or other AI-generating tools are part of the process, and will update this article if we hear back.

The team will test the technology with the NYU Tisch School of the Arts and the Mark Morris Dance Center for Dance Ensembles. Dancers from both organizations will perform on a volume recording stage. The team streams their moves live and on-demand, providing educational content for aspiring dancers looking to learn from high-level performers, allowing engineers to test and modify their PCV technology.

Researchers envision this opening the door to more advanced VR and mixed reality streaming content. The success of the proposed research will contribute to the widespread deployment of high-quality and robust PCV flow systems that facilitate augmented, virtual and mixed reality experiences and create new opportunities in many fields, including education, business, healthcare and entertainment, Liu said. .

“Point-cloud video has tremendous potential to transform a wide range of industries, and I’m thrilled that the NYU Tandon research team prioritized dance education to reap these benefits early,” said Jelena Kovaevi, NYU’s Tandon Dean. .