by: Anais Painchault and Thomas Burnichon
The Apple Vision Pro promises to change the world of entertainment forever – and the value of each viewer. How can you make sure you are part of the revolution and reap the benefits of higher ARPU? Read on to find out how joining this innovation can propel your business – and what you need to be part of it (hint: MV-HEVC encoding is part of it).
Unlike previous headsets Apple is putting media back in the spotlight. With the ARPU of an Apple Vision Pro user having the potential to reach levels that are several orders of magnitude higher than those of a typical streaming service subscriber, it’s important not to be left behind. In fact, dismissing the potential of what is happening today could mean ignoring a breakthrough in the video industry that hasn’t been seen since the move to color TV. Instead of becoming a dinosaur struggling with non-profitable video service while the world changes without you, you can embrace colors and be one of the pioneers to propel your business into the new era of Spatial Computing.
What is Spatial Computing?
Spatial Computing and the Apple Vision Pro enable new ways of experiencing content, where viewers are immersed into the action thanks to 3D stereoscopic video recreating true-to-life experiences by delivering superb looking 3D high-resolution video to the left and right eyes simultaneously. Beyond the private 3D cinema experience, viewers feel transported within the content itself thanks to the up to 180-degree video angle, the life-size characters and the immersive audio bouncing off the objects in the virtual scenes.
How to Stream 3D Stereoscopic Video to the Apple Vision Pro with MV-HEVC Encoding
To stream 3D movies to the Apple Vision Pro, you first need the content to be encoded using Multiview High Efficiency Video Coding (MV-HEVC). This is an HEVC extension focused on efficiently coding spatially related images such as the left eye and right eye views of 3D stereoscopic content. The base layer is backward compatible with HEVC and corresponds to the left eye view, while the second layer can be used to recreate the right eye view from the base layer and deliver the 3D rendition on a compatible decoder such as the Apple Vision Pro. You will of course need to follow Apple guidelines for encoding and packaging to ensure compatibility. For a life-like experience you might also want to use supported high dynamic range technologies such as Dolby Vision profile 20 and Next Generation Audio such as Dolby Atmos.
Challenges for Encoding Video in MV-HEVC for Spatial Computing
One of the main challenges when encoding 3D video is to ensure user comfort by delivering video in quality that is perceived as consistent between the two eyes for each area of the image. A traditional constant bitrate approach is not sufficient here, and it is fundamental to use a more advanced rate-control scheme to deliver constant visual quality based on a very accurate quality model, such as the Technology & Engineering Emmy®-award winning and Hollywood-vetted Ateme Constant Quality Rate.
Another challenge coming with encoding video for Spatial Computing is bitrate efficiency: to deliver realistic, mesmerizing experiences, the amount of data to be streamed is massive. Most media content available on the Apple Vision Pro features a video resolution of at least 4K per eye – if not 8K – and a frame rate that can be twice as high as typical movie content. To avoid buffering and saturating the network it is therefore critical to use an efficient encoder that can reach the 30% bitrate savings recommended by Apple for MV-HEVC. Thanks to an extensive codec expertise and a long experience with multi-layered HEVC – with SHVC support for ATSC3.0 since 2018 – Ateme’s Gen 7 compression core is able to efficiently use of left and right eye redundancies and deliver up to 50% bitrate savings in MV-HEVC.
What’s next for Spatial Computing?
The overwhelming majority of video streaming services offered at launch on the Apple Vision Pro were based on cinematic content. Sports and music are, however, at the top of everyone’s mind. Apple announced Major League Soccer highlights in 8K 3D with a 180-degree field of view and Spatial Audio to transport audiences into matches. Such highlights, captured Live but prepared off-line for on-demand consumption, can be considered an intermediate step toward full end-to-end live transmission chains. They enable new user experiences, with immersion taken one step further: With a larger field of view, viewers can decide where to look, rather than following the director’s decision. This involves significant technical changes for content production, distribution and consumption. However, it will also open up new monetization avenues, for example by making the best stadium seats available for sale to a much larger audience. Concerts could also be enjoyed at home from various perspectives, and more accurately recorded for posterity.
For further insight please see Ateme’s website: https://www.ateme.com/immersive-experiences/
Anais Painchault
Anais Painchault is the VP Product, Compression & Processing and Head of Titan Product Management at Ateme.
Thomas Burnichon
Thomas Burnichon is the director of technology at ATEME. He focuses on the video transcoding workflows for live, video on demand (VOD), and over-the-top (OTT) streaming applications. He is in close contact with content and service providers all over the world to identify best practices and help drive innovations such as HEVC/H.265, AOMedia Video Codec (AV1), versatile video coding (VVC), UHD-1/4K, UHD-2/8K, high dynamic range (HDR), and next generation audio (NGA).