In this series, I will take you on a deep dive into the architecture of the Tortoise-TTS model and explain in detail how the Tortoise-TTS model works. This will not only be done theoretically, but will also be accompanied by code. In part 1, we will take a look at the overall architecture. In Part 2, we will use an autoregressive model (GPT-2 model) to generate Mel tokens, which are encoded representations of Mel spectrograms. In Part 3, we will compute a score for all generated Mel token sequences and evaluate how well they match our input text. In Part 4, we will then take the Mel token sequences with the highest scores and generate a Mel spectrogram for each sequence using a diffusion model. Finally, in Part 5, we will convert the Mel spectrograms into waveform audio using a vocoder (UnivNet model). Register for GTC 2024 and win an NVIDIA RTX 3080 Ti (Deadline XY): https://nvda.ws/3Sastoe Send Your Proof of Attendance: https://forms.gle/hmXuJhvmFBp4hzrYA GTC sessions mentioned in the video: What’s Next in Generative AI https://www.nvidia.com/gtc/session-catalog/?tab.allsessions=1700692987788001F1cG#/session/1696214901866001OP1b?ncid=ref-inor-138326 The Fastest Stable Diffusion in the World https://www.nvidia.com/gtc/session-catalog/?tab.allsessions=1700692987788001F1cG&search.pgenerativeaip=1699468419333008hG6l#/session/1696220880767001P947?ncid=ref-inor-138326 Human-Like AI Voices: Exploring the Evolution of Voice Technology https://www.nvidia.com/gtc/session-catalog/?tab.allsessions=1700692987788001F1cG&search.pgenerativeaip=1699468419333009hkfR#/session/1698239323599001AWl4?ncid=ref-inor-138326 Code Used in This Video https://colab.research.google.com/drive/1Bl_N1KtUSzjp9qEMUBu9jGdfNMG4rR-O?usp=sharing My Medium Article for This Video: https://medium.com/p/92412ecc5d8d My Workstation GPU: NVIDIA RTX 6000 Ada https://nvda.ws/47U7wmA CPU: Intel Core i9-13900K https://amzn.to/47qDQgp RAM: Corsair Vengeance 64 GB https://amzn.to/47o4S8e Motherboard: ASRock Z790M PG https://amzn.to/3SxvtLS Storage: Samsung 980 PRO 2 TB https://amzn.to/3u8X23Y PSU: Corsair RM 850x https://amzn.to/3uhTNXS Case: Fractal Design Meshify 2 Mini https://www.fractal-design.com/products/cases/meshify/meshify-2-mini/ CPU Cooler: Noctua NH-U12A https://amzn.to/3Qpv4IM Case Fan: Noctua NF-A12x25 https://amzn.to/3srf1lE 00:00:00 Intro 00:00:33 Promo: Win an NVIDIA RTX 3080 Ti GPU 00:01:40 Pick Top-K Mel Token Sequences 00:05:40 Calculate Latent Representations 00:07:26 Pre-process Latent Representations 00:08:52 Outro Stay in Touch Medium https://medium.com/@martin-thissen LinkedIn https://linkedin.com/in/mthissen135/ YouTube Of course, feel free to subscribe to my channel! :-) Of course, financial support is completely voluntary, but I was asked for it: https://patreon.com/MartinThissen https://ko-fi.com/martinthissen