In this series, I will take you on a deep dive into the architecture of the Tortoise-TTS model and explain in detail how the Tortoise-TTS model works. This will not only be done theoretically, but will also be accompanied by code. In part 1, we will take a look at the overall architecture. In Part 2, we will use an autoregressive model (GPT-2 model) to generate Mel tokens, which are encoded representations of Mel spectrograms. In Part 3, we will compute a score for all generated Mel token sequences and evaluate how well they match our input text. In Part 4, we will then take the Mel token sequences with the highest scores and generate a Mel spectrogram for each sequence using a diffusion model. Finally, in Part 5, we will convert the Mel spectrograms into waveform audio using a vocoder (UnivNet model). Code Used in This Video https://colab.research.google.com/drive/1Bl_N1KtUSzjp9qEMUBu9jGdfNMG4rR-O?usp=sharing My Medium Article for This Video: https://medium.com/p/a3ac41a5d8f7 My Workstation GPU: NVIDIA RTX 6000 Ada https://nvda.ws/47U7wmA CPU: Intel Core i9-13900K https://amzn.to/47qDQgp RAM: Corsair Vengeance 64 GB https://amzn.to/47o4S8e Motherboard: ASRock Z790M PG https://amzn.to/3SxvtLS Storage: Samsung 980 PRO 2 TB https://amzn.to/3u8X23Y PSU: Corsair RM 850x https://amzn.to/3uhTNXS Case: Fractal Design Meshify 2 Mini https://www.fractal-design.com/products/cases/meshify/meshify-2-mini/ CPU Cooler: Noctua NH-U12A https://amzn.to/3Qpv4IM Case Fan: Noctua NF-A12x25 https://amzn.to/3srf1lE 00:00:00 Intro 00:00:20 Mel Spectrogram to Waveform Audio 00:02:32 UnivNet Paper 00:03:00 EVA-GAN Paper 00:03:32 BigVGAN Repository 00:04:02 Generate & Play Speech 00:05:56 Outro Stay in Touch Medium https://medium.com/@martin-thissen LinkedIn https://linkedin.com/in/mthissen135/ YouTube Of course, feel free to subscribe to my channel! :-) Of course, financial support is completely voluntary, but I was asked for it: https://patreon.com/MartinThissen https://ko-fi.com/martinthissen