Banuba, a leading augmented reality and computer vision technology vendor, was able to massively increase the top frame rate of the live videos using virtual backgrounds. This will be especially noticeable for clients that use Banuba Face AR SDK for their video communication and live streaming software. Potentially, they will be able to get up to 10 times more frames per second.
Virtual backgrounds are a core feature for social media, live streaming, and video calling applications. They help people deal with anxiety that comes from the online meetings, protect their personal information by hiding everything that might give out their address, and save them from embarrassing situations that could potentially harm their reputation. Now that remote work has become the norm in many companies, upgrading virtual backgrounds is at the top of Banuba’s priority list. You can see it for yourself in a video.
The results were achieved thanks to three main additions:
- Improved neural network architecture with optimized utilization of CoreML Neural Engine (on Apple devices with Bionic processors);
- Algorithm streamlining on Windows and Web, allowing the neural network to monitor every other frame instead of each of them;
- Less resource-intensive anti-jitter algorithms.
Read More: Experlogix Digital Commerce Expands into North America
Banuba offers two products with virtual backgrounds. The first is Face AR SDK, a multipurpose module that works with live video feed. The second is Video Editor SDK, which processes prerecorded footage. They can work independently or integrated to cover more use cases.
Other recent updates include:
- Facial feature editing – a new functionality that allows changing size and shape of any part of the face;
- Optimized hand tracking and gesture recognition;
- Better lips segmentation. This is especially noticeable on the corners of the mouth and near the philtrum;
- Acne removal for photos and an option to change the size of the area affected by the effect.
Read More: SalesTechStar Interview with Rob Seaman, SVP, Enterprise Product at Slack