AI IN FILM AND TV EDITING
Can machine learning change the workflow for the film and tv editor?
It is widely thought that AI will make mundane tasks disappear with its machine learning capabilities, leaving more time for the film and tv editor to focus on other more important aspects of the craft.
While it’s doubtful that an AI bot will replace the actual film and tv editor, it is claimed that 2023 is the year that AI will become more visible in our working lives and that visibility also will reveal itself in the film cutting room.
So I have discovered a few AI tools that should reduce time in the edit, while potentially allow me as a documentary editor more time to focus on shaping the story and finessing the soundscape.
1) AI IN AUDIO EDITING/MIXING
For a new way to audio mix check out Neutron 4 from Izotope. With this tool you can treat all problem audio tracks. You can apply effects and modify audio with the track assistant. Then for the overall mix a film and tv editor can use mix assistant to edit all tracks. I have watched the demo and I am impressed. I have yet to test it on a documentary edit but it’s on my to-do list for spring 2023. It is currently priced at $199.
2) AI IN VOICE-OVERS
Respeecher – who needs to wait for the voice-over session before you can lay down the voice track when you can ‘clone voices for films and tv shows’ with a software that allows you to change a voice to a specific one. Not sure what the copyright implications are, but this seems to be a game changer for the voice-over world. It also allows a documentary editor to test out accents, a broad range and tone of voices early on in the edit workflow.
3) AI IN VIDEO EFFECTS
Mask and rotoscope in minutes not hours with Runway.
Runway holds a basket of AI magic tools. You can switch that background and rotoscope that image with a few clicks. I have to admit this will save a lot of time in the edit room. Masking that takes hours will be a memory now with this plug-in.
You can test it with the freeplan or then opt into the Pro Plan at $144 per year:- 4k exports, 7 editors per workspace, 500GB assets, 125 free image generations per month and more AI Magic Tools export options or the Team Plan at $336 per year:– ITB of assets with up to 15 editors per workspace, full AI Magic Tools export options and 450 free image generations per month.
The tools that are the most impressive in Runway are remove background, erase and replace. Also to add to this list are backdrop remix, inpainting where you can remove people or objects from the video, blur faces so easily and motion tracking that is efficient and straightforward to achieve.
You will need to run the latest version of Chrome to use Runway.
4) AI IN DUBBING
Dubbing and subtitling could become a bit dated due to this piece of software - True Sync from Flawless.
Described as “… the world's first system that uses Artificial Intelligence to create perfectly lip-synced visualisations in multiple languages. At the heart of the system is a performance preservation engine which captures all the nuance and emotions of the original material.”
I will watch this closely as it will have huge implications if it is remarkably effective.
5) Papercup and its AI-powered dubbing tool which is currently available in English, LATAM Spanish, Brazilian Portuguese and Italian.
This is an actual dubbing tool where you upload your video to Papercup and then the AI transcribes, translates and creates a human-sounding voiceover. The team of QAs verify and quality check the dubbed video and you should then receive the newly dubbed video for use. Speedy speedy.
6) AI IN COLOUR GRADING
Colourlab.AI – grade in seconds with this tool from the Los Angeles-based company Color Intelligence. With three products on offer – Colourlab AI, Look Designer and Grainlab they claim these products can balance shots and match scenes with only a single click (amongst other colouring abilities). Prices are available at monthly/yearly or perpetual rates. Again as a documentary editor this will prove invaluable when testing out potential looks and tones.
7) AI IN TRANSCRIPTION
Sonix another game-changer in the AI transcription world. They state that their product is able to transcribe in 35 languages. However not only does it provide transcriptions in fairly fast time, you can upload your own transcript and it can merge it with the one it has transcribed, you can export out SRTs & VTTs for subtitles, while word by word timestamps and speaker labelling are just a few other features. Plus you have a free 30-minute trial to test it out. This is again a game-changer for a documentary editor as transcripts are always an essential part of the editing process and receiving them as soon as possible smooths out the editing workflow.
