The Technology Inside PanaCast® 2 – World’s First Panoramic-4K Camera
Building a good real time video camera is a complex task. Building a great real time panoramic camera with multi imagers is a much more complex task and requires deep technical expertise in video imaging technology. That is why we at Altia Systems are proud of the technology we have developed in-house to produce the world’s first Panoramic-4K video camera that delivers a state-of-the-art 180º panoramic real time video stream. It delivers a “being there” experience in applications like web collaboration, education and telemedicine.
To capture true panoramic video, you need multiple imagers in a camera. Trying to stretch the capabilities of single lens cameras produces distortions such as fisheye – a convex type of distortion shown in figure one. Some of these distortions can be compensated, but at the expense of a significant loss of resolution and quality. Single lens panoramic cameras cannot be used in situations where scene accurate panoramic video is required and their use is confined to mostly consumer applications.
Figure 1 Distorted image from a fish eye lens
Multi-imagers are better suited for generating panoramic video because they don’t have distortions that can’t be compensated for like in single lens cameras.
However, using multi-imagers brings its own set of challenges. One problem to be solved is parallax. Parallex is the difference in the apparent position of an object viewed along two different lines of sight. It is measured by the angle of inclination between those two lines.
Figure 2 shows the angle of inclination between the left (#1) and right (#3) imager in the 3 imager PanaCast 2 Panoramic-4K video camera.
Figure 2 Angle of inclination between imager #1 and #3
The result is the raw image from each imager as seen in figure 3, Any image from one imager is divergent from the image of the other imager at a similar angle to what the imagers/sensors are mounted at. Ideally the distance between two imagers should be minimal to reduce parallax errors.
Figure 3 View from multi-imagers
So how do we adjust these angles so that the images can be integrated together? Geometric corrections is applied by bending the divergent lines and projecting them and their associated image on a virtual cylinder. This process turns the sharp angle into a shallow smooth surface. You can see correction profile in figure 4.
But that is not enough to insure a continuous panoramic image. In the PanaCast 2 camera,images from the three different imagers are overlapped to ensure proper transition from one to another. Every frame in the overlapping region is analyzed and a transition path for stitching the images together is calculated for that region by an on-board high-speed pipeline processor.
Figure 4. Geometric correction to eliminate parallax distortion
Figure 5 Geometric corrections applied on the images
Altia Systems’ Dynamic Stitching is a patent pending technology that does an analysis of the overlapping image. Typically images have foreground and background, each with their own characteristics. Foreground carries many details (chairs, people, etc.) whereas background is typically plain (walls, whiteboards, windows etc.) Once the geometric correction algorithm has finished correcting the sharp angles, the dynamic stitching algorithm creates an energy cost function of the entire overlap region and comes up with stitching paths of least energy. They typically lie in the background. The computation is done in real time on a frame-by-frame basis to create the panoramic video that you see. Each video frame is 1600 x1200 pixels from each Imager. Joining those frames from three imagers together creates a 4800 x 1200 image.
To insure the clearest and most accurate panoramic image, each PanaCast 2 camera is taken through Altia Systems patented calibration tool after it is manufactured. The tool calculates the details of back projection (projection of the image on a virtual cylindrical wall). The image is focused before projection onto the cylindrical surface. Each lens within an imager has unique variation and distortion characteristics and therefore the details of each projection needs to be calculated separately. Any vertical offset between the images from different imagers is corrected. The horizontal overlap between the imagers is almost 25% with the adjacent imager. Each pixel is evaluated and is run through the calibration tool algorithm which generates a calibration table for each camera, The result is a pleasing panoramic video as seen in Figure 5.
All image sensors inside the imagers are 3 megapixel resolution and can go up to 30 fps. A table below gives resolution rates that are supported with the PanaCast2 cameras with USB 3.0 and USB 2.0.
Table 1 Supported resolution, frame rate and video formats on PanaCast 2
What Altia Systems have achieved with PanaCast2 is a truly Plug-and-Play USB Panoramic-4K resolution camera that integrates seamlessly with any UVC application right out of the box and delivers stunning panoramic video quality to web collaboration and a host of other applications.
|About Javed Tufail:|
Javed has over 16 years of industry experience related to unified communications collaboration. In the past Javed held many significant roles at Cisco and Microsoft. He
has spoken at various Industry conferences including Cisco Live and Microsoft TechEd both nationally and internationally. He holds CCIE, MCP and VCP certifications currently.
|About Yashket Gupta:|
Yash has over 8 years of experience in the field of video imaging and video compression. He worked at Nethra Imaging (acquired by Imagination Technologies) building the camera SOCs for the high-end, ultra high-resolution and ultra fast digital video cameras used for making Hollywood movies. The cameras represent the state of the art in digital movie making and are used in well-known Hollywood Blockbusters.