Docker launched "Docker Model Runner" to run LLMs through llama.cpp with a single "docker model" command. In this episode Bret details examples and some useful use cases for using this way to run LLMs. He breaks down the internals. How it works, when you should use it or not use it; and, how to get started using Open WebUI for a private ChatGPT-like experience.★Topics★<a href="https://docs.docker.com/desktop/features/model-runner">Model Runner Docs</a><a href="https://hub.docker.com/u/ai">Hub Models</a><a href="https://www.bretfisher.com/oci-artifacts">OCI Artifacts</a><a href="https://github.com/open-webui/open-webui">Open WebUI</a><a href="https://gist.github.com/BretFisher/aafd46eeb7acef2f5ef7d1ea70abe2ad">My Open WebUI Compose file</a>Creators & Guests
<ul>
 <li><a href="https://podcast.bretfisher.com/people/cristi-cotovan">Cristi Cotovan</a> - Editor</li>
 <li><a href="https://podcast.bretfisher.com/people/beth-fisher">Beth Fisher</a> - Producer</li>
 <li><a href="https://podcast.bretfisher.com/people/bret-fisher">Bret Fisher</a> - Host</li>
</ul>
<ul><li>(00:00) - Intro</li>
<li>(00:46) - Model Runner Elevator Pitch</li>
<li>(01:28) - Enabling Docker Model Runner</li>
<li>(04:28) - Self Promotion! Is that an ad? For me?</li>
<li>(05:03) - Downloading Models</li>
<li>(07:11) - Architectrure of Model Runner</li>
<li>(10:49) - ORAS</li>
<li>(11:09) - What's next for Model Runner? </li>
<li>(12:13) - Troubleshooting</li>
</ul>
You can also support my free material by subscribing to my <a href="https://www.youtube.com/@BretFisher">YouTube</a> channel and my weekly newsletter at <a href="https://bret.news/">bret.news</a>!Grab the best coupons for my <a href="https://www.bretfisher.com/courses">Docker and Kubernetes courses</a>.Join my cloud native DevOps community <a href="https://devops.fan/">on Discord</a>.Grab some merch at <a href="https://bretfisher.myspreadshop.com/">Bret's Loot Box</a>Homepage <a href="https://bretfisher.com/">bretfisher.com</a>

Podurama

Home

Trending

Personal

Your Podcasts

New Episodes

My Files

Favourite Episodes

Tags (Folders)

Recently Played

Bookmarks

Followed Playlists

Docker Model Runner

DevOps and Docker Talk: Cloud Native Interviews and Tooling

AI Chapters

AI Insights

Reviews