Links
<a href="https://www.linkedin.com/in/jahuang/" rel="nofollow">James on LinkedIn</a>
<a href="https://www.linkedin.com/in/dominucco/" rel="nofollow">Mike on LinkedIn</a>
<a href="https://dominickm.com" rel="nofollow">Mike&#39;s Blog</a>
<a href="https://discord.com/invite/k8e7gKUpEp" rel="nofollow">Show on Discord</a>

<a href="https://go.alice.dev/data-migration-offer-hands-on" rel="nofollow">Alice Promo</a>

<ol>
<li>AI on Red Hat Enterprise Linux (RHEL)</li>
</ol>

Trust and Stability: RHEL provides the mission-critical foundation needed for workloads where security and reliability cannot be compromised.

Predictive vs. Generative: Acknowledging the hype of GenAI while maintaining support for traditional machine learning algorithms.

Determinism: The challenge of bringing consistency and security to emerging AI technologies in production environments.

<ol>
<li>Rama-Llama & Containerization</li>
</ol>

Developer Simplicity: Rama-Llama helps developers run local LLMs easily without being "locked in" to specific engines; it supports Podman, Docker, and various inference engines like Llama.cpp and Whisper.cpp.

Production Path: The tool is designed to "fade away" after helping package the model and stack into a container that can be deployed directly to Kubernetes.

Behind the Firewall: Addressing the needs of industries (like aircraft maintenance) that require AI to stay strictly on-premises.

<ol>
<li>Enterprise AI Infrastructure</li>
</ol>

Red Hat AI: A commercial product offering tools for model customization, including pre-training, fine-tuning, and RAG (Retrieval-Augmented Generation).

Inference Engines: James highlights the difference between Llama.cpp (for smaller/edge hardware) and vLLM, which has become the enterprise standard for multi-GPU data center inferencing.

Podurama

Home

Trending

Personal

Your Podcasts

New Episodes

My Files

Favourite Episodes

Tags (Folders)

Recently Played

Bookmarks

Followed Playlists

636: Red Hat's James Haung

Coder Radio

AI Chapters

AI Insights

Reviews