DaVinci Resolve’s AI Voice Isolation has changed the video editing game. Here’s what you need to know.
Back in 2017, I found myself immersed in the production of my web series called “The Vagabond.” As an independent filmmaker working solo, I encountered a major setback during one of our filming sessions. Regrettably, I had failed to pay close attention to the gain knob on my field mixer, resulting in extremely low captured audio. When I tried to increase the volume during post-production, it also amplified unwanted noise from a distant road and the chirping of nearby birds.
Seeking a solution, I turned to a friend who happened to be an audio specialist, and he took a stab at fixing the file. At the time, back in 2017, that was the final outcome we had to settle for. It was passable, but there was a discernible drop in overall audio clarity.
Fast forward to 2023, and we have the incredible power of DaVinci Resolve’s AI Voice isolation. When I applied this technology to the same problematic audio file, the result was mind-blowing, as you can hear in the video.
If you’re unfamiliar with DaVinci Resolve’s Voice Isolation tool, it was introduced in Resolve 18, and the voice isolation plug-in can eliminate loud and undesirable sounds from pre-existing voice recordings. It’s powered by AI technology, so you’ll need a good GPU to get smooth playback.
The results are truly impressive, allowing you to isolate dialogue from background noise in the recording. This includes a wide range of sounds such as air conditioning, fans, jackhammers, restaurant ambiance, or even simultaneous music playing alongside the speaker’s voice. In fact, our resident Instagram Reel maker used it on a video where I had already deleted the clean audio. Which reel? Well, you’d never tell 😉.
So, witnessing this remarkable progress got me thinking. Considering the extensive preparations usually involved in isolating sound, such as silencing appliances, utilizing sound-dampening materials, and scheduling shoots away from aircraft noise, I began to question whether we still need to be as overly cautious when it comes to sound. Think about it. Over the last few years, I’ve slowly removed tools that would have been in my gear bag back in 2010.
- Grey cards; gone.
- ND filters; gone.
Hell, even the in body-stabilization has replaced a trusty old shoulder rig on some shoots.
With this curiosity in mind, I embarked on a journey today to explore precisely that question.
While our articles strive to provide comprehensive information, watching the tutorial video to grasp these auditory examples is crucial.
Audio Blimp vs. AI Sound Removal
Now, let’s dive into the topic of wind protection versus open mic. To put the AI voice isolation to the test, we ventured to a section of the local coastline on a windy day. Our goal was to determine whether this technology could eliminate the need for a blimp — a microphone cover resembling a large, hollow, pill-shaped cage. This specialized cover creates a still-air chamber around the microphone, safeguarding it against vibrations and proving especially valuable in challenging weather conditions.
Without the blimp, the wind’s force is undeniably overpowering, rendering the audio unusable. However, once the blimp is employed, it’s almost as if the wind ceases to exist. It’s as if there’s no wind at all. The audio becomes remarkably clear and undisturbed.
Curious to witness the AI’s capabilities in action, we imported the audio into DaVinci Resolve to see if the wind could be completely nullified from the take without the blimp. Unfortunately, the results were less than satisfactory. The AI algorithm struggled to distinguish the wind from the voice due to the sheer power of the wind in this scenario. Consequently, when filming in windy locations, AI will likely offer little assistance.
In such circumstances, it’s advisable to rely on other sound protection measures rather than solely relying on AI technology.
Background Noise vs. AI Sound Removal
Continuing our journey at the same location, just a few minutes down the path, there’s a less windy location; however, in the background, we can hear a farmer engrossed in their work, likely operating machinery such as a tractor. The noise generated by the machinery is quite prominent, and the microphone picks it up.
Given that we’re using a shotgun microphone with a narrow pick-up pattern, a simple solution presents itself: we can ask the subject to reposition themselves so that the microphone faces away from the background noise. By doing so, we can effectively minimize the noise interference. And indeed, we can hear that it works — the tractor noise becomes sufficiently diminished.
However, what if we’re in a position where this isn’t possible? Perhaps we really need to see the scenery behind the subject in the direction of the tractor. Or perhaps we need the shot at this specific time because of the lighting conditions.
Well, let’s explore how AI handles the removal of the tractor noise in this case. Remarkably, it performs quite well. In fact, it’s difficult to detect the tractor noise at all in the background. Therefore, in situations like this, where time is of the essence, we now have the option to rectify such issues in post-production. And as we saw in the introduction, it’s even fantastic at removing unwanted nature sounds.
If you’re creating a horror short and your subject is set in the wild, the beautiful sounds of the morning birds will need to bode better for the tone of your scene. However, instead of initiating an ADR session and then a load of Foley, you can now remove those elements using this tool.
However, there is sometimes a trade-off when removing background noise from voice audio. The voice may become slightly compressed and sound hollow. To shed light on this phenomenon, we consulted our resident audio expert, who shared the following insight: In the video, listen to the examples.
When removing noise from voice audio, sound quality can be adversely affected due to two primary reasons. Firstly, overprocessing occurs when noise removal settings are overly aggressive, resulting in the removal of noise and portions of the original voice signal. This excessive processing can lead to audio that sounds muffled, hollow, or thin. Secondly, the complexity and variability of real-world audio present inherent limitations.
Different types of noise, such as constant background hum, sudden bursts, or overlapping speech, pose challenges in accurately isolating the noise from the desired voice signal. Consequently, achieving an optimal balance between noise reduction and preserving the original audio quality becomes a challenge for algorithms.
Understanding these factors helps us navigate the delicate balance between noise reduction and maintaining the desired audio quality.
Considering the points mentioned earlier, investing in quality equipment or positioning oneself away from intrusive noises is undoubtedly practical. However, in a pinch, the AI voice isolation tool certainly provides a great alternative to ADR. This ensures that the fixed audio captures the full range of sound frequencies.
Interior Noise vs. AI Sound Removal
So, we’ve explored some challenges we encountered in an outdoor setting, but what about interior locations?
In this scene, our subject is situated near a running fridge, causing an audible electronic hum in the background. Ideally, we would either turn off the fridge or position the actress away from its source to mitigate the noise. When we turn off the fridge, the audio becomes perfect — devoid of any unwanted hum.
However, the distinct hum persists in the background when the fridge is on. At this point, we would typically employ the DeHum tool, commonly found in various non-linear editing software. While the results obtained from this tool are generally decent, they require adjusting multiple parameters to achieve a somewhat satisfactory outcome.
Now, here comes the remarkable part. The hum is almost instantly removed by using DaVinci Resolve’s AI Voice Isolation. The AI technology proves highly effective in this case, successfully eliminating unwanted noise and producing clean audio.
Nonetheless, if we add the oven’s extractor fan to the scene while the AI can remove the unwanted noise, the dialogue appears as if it’s processed too much, giving us that hollow sound again.
Conclusion
In recent months, there has been a surge of AI filmmaking and photography tools hitting the market. Unfortunately, the results from most of these tools have been underwhelming or simply unsatisfactory. However, one standout tool in Resolve deserves special mention — the voice isolation feature. Its capabilities are extraordinary as it effectively removes many unwanted background sounds. I believe it will soon become a standard component in the workflow of amateur filmmakers.
Nonetheless, it’s important to note that this tool does have its limitations. Suppose your subject is in a windy environment or positioned near a blaring amplifier at a rock concert. In that case, it cannot perform miracles by isolating the voice signal from the surrounding cacophony.
Moreover, depending on the severity of the unwanted noise, the processing applied to the speaker’s voice may introduce undesirable effects, necessitating dubbing or extensive post-production work to restore quality.
Therefore, I recommend taking proactive measures to minimize nuisance noise as much as possible and using appropriate audio equipment before recording.
Looking for filmmaking tips and tricks? Check out our YouTube channel for tutorials like this . . .
