172: Transformers and Large Language ModelsIntro topic: Is WFH actually WFC?News/Links:<ul><li>Falsehoods Junior Developers Believe about Becoming Senior<ul><li><a href="https://vadimkravcenko.com/shorts/falsehoods-junior-developers-believe-about-becoming-senior/">https://vadimkravcenko.com/shorts/falsehoods-junior-developers-believe-about-becoming-senior/</a></li></ul></li><li>Pure Pursuit<ul><li>Tutorial with python code: <a href="https://wiki.purduesigbots.com/software/control-algorithms/basic-pure-pursuit">https://wiki.purduesigbots.com/software/control-algorithms/basic-pure-pursuit</a> </li><li>Video example: <a href="https://www.youtube.com/watch?v=qYR7mmcwT2w">https://www.youtube.com/watch?v=qYR7mmcwT2w</a> </li></ul></li><li>PID without a PHD<ul><li><a href="https://www.wescottdesign.com/articles/pid/pidWithoutAPhd.pdf">https://www.wescottdesign.com/articles/pid/pidWithoutAPhd.pdf</a></li></ul></li><li>Google releases Gemma<ul><li><a href="https://blog.google/technology/developers/gemma-open-models/">https://blog.google/technology/developers/gemma-open-models/</a></li></ul></li></ul>Book of the Show<ul><li>Patrick: The Eye of the World by Robert Jordan (Wheel of Time)<ul><li><a href="https://amzn.to/3uEhg6v">https://amzn.to/3uEhg6v</a></li></ul></li><li>Jason: How to Make a Video Game All By Yourself<ul><li><a href="https://amzn.to/3UZtP7b">https://amzn.to/3UZtP7b</a></li></ul></li></ul>Patreon Plug <a href="https://www.patreon.com/programmingthrowdown?ty=h">https://www.patreon.com/programmingthrowdown?ty=h</a>Tool of the Show<ul><li>Patrick: Stadia Controller Wifi to Bluetooth Unlock<ul><li><a href="https://stadia.google.com/controller/index_en_US.html">https://stadia.google.com/controller/index_en_US.html</a></li></ul></li><li>Jason: FUSE and SSHFS<ul><li><a href="https://www.digitalocean.com/community/tutorials/how-to-use-sshfs-to-mount-remote-file-systems-over-ssh">https://www.digitalocean.com/community/tutorials/how-to-use-sshfs-to-mount-remote-file-systems-over-ssh</a></li></ul></li></ul>Topic: Transformers and Large Language Models<ul><li>How neural networks store information<ul><li>Latent variables</li></ul></li><li>Transformers<ul><li>Encoders & Decoders</li></ul></li><li>Attention Layers<ul><li>History<ul><li>RNN<ul><li>Vanishing Gradient Problem</li></ul></li><li>LSTM<ul><li>Short term (gradient explodes), Long term (gradient vanishes)</li></ul></li></ul></li><li>Differentiable algebra</li><li>Key-Query-Value</li><li>Self Attention</li></ul></li><li>Self-Supervised Learning & Forward Models</li><li>Human Feedback<ul><li>Reinforcement Learning from Human Feedback</li><li>Direct Policy Optimization (Pairwise Ranking)</li></ul></li></ul>

 <a href="https://www.patreon.com/programmingthrowdown" rel="payment" title="★ Support this podcast on Patreon ★">★ Support this podcast on Patreon ★</a>

Podurama

Home

Trending

Personal

Your Podcasts

New Episodes

My Files

Favourite Episodes

Tags (Folders)

Recently Played

Bookmarks

Followed Playlists

172: Transformers and Large Language Models

Programming Throwdown

AI Chapters

AI Insights

Reviews