Audio object separation from mixture signal using object-specific time/frequency resolutions

Simple SummaryContent extracted from patent full text and abstract with AI.

This patent describes an advanced audio encoding and decoding method that enables the separation of individual audio objects (like vocals, instruments, or speakers) from a combined audio mixture. It introduces object-specific time/frequency resolution for each audio object, meaning the characteristics of each sound source (such as tonal or transient properties) are taken into account to optimize how each object is encoded and separated. The method involves transmitting both a downmix audio signal and side information that includes instructions on how to best extract each audio object at a suitable time and frequency resolution, leading to much improved audio quality and object separation in the playback or post-processing stages.

Use CasesContent extracted from patent full text and abstract with AI.

• Music production and remixing, allowing isolation or adjustment of specific instrument or vocal tracks.
• Broadcast and live audio, enabling selective enhancement or suppression of speakers or sound sources.
• Video gaming and virtual reality, providing more realistic and interactive audio experiences by adapting sound source placement and clarity.
• Teleconferencing or hearing aids, where certain voices can be made clearer by separating and enhancing individual speakers from a room mix.
• Karaoke systems, allowing removal or isolation of vocal or instrumental tracks for practice or performance.
• Post-production for film and TV, enabling detailed manipulation of soundscapes by isolating individual sound sources.

BenefitsContent extracted from patent full text and abstract with AI.

• Much improved separation of audio objects, yielding less crosstalk and higher fidelity for each sound source.
• Enables fine control over individual sound elements in a mix, allowing user-customization and interactivity.
• Can lower bitrate requirements compared to transmitting separate channels for all objects while still allowing reconstruction of separated objects upon playback.
• Adaptively optimizes time/frequency analysis for each sound object, improving extraction quality for both tonal and transient sounds.
• Backward compatibility: can work with existing Spatial Audio Object Coding (SAOC) systems, providing basic or enhanced quality depending on decoder capability.
• Suitable for both consumer and professional audio applications, including real-time and offline processing.

Technical Classifications (CPCs)

Inventors & Applicants

Patent Abstract

An audio decoder is proposed for decoding a multi-object audio signal consisting of a downmix signal X and side information PSI. The side information comprises object-specific side information PSI i for an audio object s i in a time/frequency region R(t R ,f R ), and object-specific time/frequency resolution information TFRI i indicative of an object-specific time/frequency resolution TFR h of the object-specific side information for the audio object s i in the time/frequency region R(t R ,f R ). The audio decoder comprises an object-specific time/frequency resolution determiner 110 configured to determine the object-specific time/frequency resolution information TFRI i from the side information PSI for the audio object s i . The audio decoder further comprises an object separator 120 configured to separate the audio object s i from the downmix signal X using the object-specific side information in accordance with the object-specific time/frequency resolution TFRI i . A corresponding encoder and corresponding methods for decoding or encoding are also described.