Not long ago, virtual reality was little more than the stuff of science fiction books and movies. Today, virtual reality is not only making inroads in actual high-end science and technology, but also in ways that affect the life of everyday people from interactive gaming and data-driven sports broadcasting, to video conferencing, education & training, and live music & concerts.
Many music and concert lovers may have had the experience of been shoved around in the audience or had a very limited view of their favorite artists as they struggle to get the best possible view of the stage from the audience. Some people prefer to avoid the front crowds and prefer to sit back in the lawn area or back bench seats where the rowdy crowd behavior is minimal. But choosing this option means there is a risk of missing out on the details of the artists performance or the expressions conveyed to listeners within the proximity of eye contact with the performer. Such moments add to the live feel and reality of a concert experience.
In addition, since the quasi-collapse of the music industry where revenues for physical music tumbled drastically between 2001 and 2018, the industry has turned to live music as its main source of income. With several hundreds of live concerts taking place globally each year, it is impossible for fans to be physically present at all times.
Another challenge for live concerts is that they place a natural limit on the number of people who can attend them. This again, has made it harder for music fans to see their favorite artists. To meet these challenges, virtual reality has stepped in and is now playing a key role in bringing live concert experiences from the front rows to the living rooms of fans and audiences worldwide.
Virtual reality concerts are a win-win situation for the music industry and the music fans. In addition to those who pay to actually be present at a concert, the music industry can monetize everyone who couldn't obtain tickets, or who didn’t always feel like going out to see their favorite musicians perform. Virtual reality lets the music industry combine the best of both worlds: the apparent spontaneity and singularity of live music with the reproducibility and accessibility of recorded music.
Camera technology plays a key role in enabling virtual reality in live concerts. This is because the concerts are captured live from various angles using high-end cameras. The live images are then processed in almost real time to enable remotely-located audiences to choose and view the concert from positions of their choice (e.g., viewed from the front rows, viewed from different acompanists such as percussionsts, guitarists or pianists, different views of the crowds, etc.) all while delivering an immersive experience that goes far beyond that provided by a traditional concert DVD.
From a display perspective, like in sports imaging, the horizontal pixel resolution plays an important role in the quality of virtual reality. This resolution can either be actual or interpolated from a higher or lower raw image format.
4K horizontal resolution for VR has been around for quite some time. 4K, also known as Ultra HD, has a pixel resolution of 4,096 x 2,160 pixels. When compressing video streams from 4K to HD-streamable video the images are clearer, sharpner and cleaner. Shooting at such a high resolution gives editors and image processing engineers an opportunity to zoom far into images and reframe without losing information.
Using an 8K horizontal resolution, also known as Full Ultra HD, allows the user to zoom in twice as much and still get a 4K image. However, achieving real 8K horizontal resolution for virtual reality applications is difficult even if the cameras support 8K horizontal resolutions. This is because most virtual reality installations prefer each camera to have an ultra-wide field of view to give viewers a panoramic or hemispherical view while reducing the equipment handling complexities during live concerts.
The only way to achieve such ultra-wide fields of view is by using fish-eye lenses. Combining a fish-eye lens with a camera using a rectangular sensor is only possible by having an image circle that is smaller than the sensor. Today, sensors with 8K horizontal resolution are able to achieve 5324 pixels in real horizontal resolution when paired with a fish-eye lens of 4.3 mm focal length. This helps to achieve an angle of view of 250° with 21 pixels per degree, which is a good number of pixels for high quality image processing and enhancement. Interpolation can then be used to achieve an 8K horizontal screen resolution.
Obviously, the higher the camera resolution, the closer one can get to achieving real 8K horizontal resolution. But it is important to remember that these are live action events. Higher VR resolution is only useful if a camera speed of at least 30 FPS can be maintained. This limits the choice of cameras that can be used for VR applications.
One final requirement for cameras used in virtual reality applications is reliable data transmission at low noise levels over long distances. Concert venues are typically quite large, and cameras may need to be placed at locations far away from the crowds. CXP and optical interfaces (e.g., SFP+) are reliable and well-known interfaces to handle both the distances and the environmental conditions of outdoor applications.
More information:
Watch webcast: A look at the key imaging features of the new Spark Series 45-megapixel camera.
Read more about the Spark SP-45000-CXP4 area scan camera for virtual reality, motion picture capture, sports imaging and other entertainment systems + download a datasheet.
Questions?
For questions about machine vision cameras for virtual reality applications or for additional information about JAI’s high speed, high resolution cameras, please contact JAI.
