Post-Training Latent Dimension Reduction in Neural Audio Coding

Thomas Muller (1, 2, 3, 4, 5) , Stéphane Ragot (1) , Pierrick Philippe (1) , Pascal Scalart (2, 3, 4, 5)

1 Orange Innovation
2 IRISA - Institut de Recherche en Informatique et Systèmes Aléatoires
3 UR - Université de Rennes
4 ENSSAT - École Nationale Supérieure des Sciences Appliquées et de Technologie
5 GRANIT - Algorithmes et architectures adaptatifs pour les systèmes sans-fils efficaces en énergie

This work addresses the problem of latent space quantization in neural audio coding. A covariance analysis of latent space is performed on several pre-trained audio coding models (Lyra V2, EnCodec, AudioDec). It is proposed to truncate latent space dimension using a fixed linear transform. The Karhunen-Lo`eve transform (KLT) is applied on learned residual vector quantization (RVQ) codebooks. The proposed method is applied in a backward-compatible way to EnCodec, and we show that quantization complexity and codebook storage are reduced (by 43.4%), with no noticeable difference in subjective AB tests.

Mots clés: Neural audio coding; Vector quantization; Latent space; Karhunen-Loève transform

Domaines: Traitement du signal et de l'image [eess.SP]

Thomas Muller, Stéphane Ragot, Pierrick Philippe, Pascal Scalart. Post-Training Latent Dimension Reduction in Neural Audio Coding. 2024. ⟨hal-04488929⟩