On-Device AI Agents in Production: Privacy, Performance, and Scale // MLOps Podcast #340 with NimbleEdge&#39;s Varun Khare, Founder/CEO and Neeraj Poddar, Co-founder & CTO.Join the Community: <a href="https://go.mlops.community/YTJoinIn" target="_blank" rel="ugc noopener noreferrer">https://go.mlops.community/YTJoinIn</a>Get the newsletter: <a href="https://go.mlops.community/YTNewsletter" target="_blank" rel="ugc noopener noreferrer">https://go.mlops.community/YTNewsletter</a>// AbstractAI agents are transitioning from experimental stages to performing real work in production; however, they have largely been limited to backend task automation. A critical frontier in this evolution is the on-device AI agent, enabling sophisticated, AI-native experiences directly on mobile and embedded devices. While cloud-based AI faces challenges like constant connectivity demands, increased latency, privacy risks, and high operational costs, on-device breaks through these trade-offs.We&#39;ll delve into the practical side of building and deploying AI agents with “DeliteAI”, an open-source on-device AI agentic framework. We&#39;ll explore how lightweight Python runtimes facilitate the seamless orchestration of end-to-end workflows directly on devices, allowing AI/ML teams to define data preprocessing, feature computation, model execution, and post-processing logic independently of frontend code. This architecture empowers agents to adapt to varying tasks and user contexts through an ecosystem of tools natively supported on Android/iOS platforms, handling all the permissions, model lifecycles, and many more.// BioVarun KhareVarun is the Founder and CEO of NimbleEdge, an AI startup pioneering privacy-first, on-device intelligence. With an academic foundation in AI and neuroscience from UC Berkeley, MPI Frankfurt, and IIT Kanpur, Varun brings deep expertise at the intersection of technology and science. Before founding NimbleEdge, Varun led open-source projects at OpenMined, focusing on privacy-aware AI, and published research in computer vision.Neeraj PoddarNeeraj Poddar is the Co-founder and CTO at NimbleEdge. Prior to NimbleEdge, he was the Co-founder of Aspen Mesh, VP of Engineering at Solo.io, and led the Istio open source community. He has worked on various aspects of AI, networking, security, and distributed systems over the span of his career. Neeraj focuses on the application of open source technologies across different industries in terms of scalability and security. When not working on AI, you can find him playing racquetball and gaining back the calories spent playing by trying out new restaurants. // Related LinksWebsite: <a href="https://www.nimbleedge.com/" target="_blank" rel="ugc noopener noreferrer">https://www.nimbleedge.com/</a><a href="https://www.nimbleedge.com/blog/why-ai-is-not-working-for-you" target="_blank" rel="ugc noopener noreferrer">https://www.nimbleedge.com/blog/why-ai-is-not-working-for-you</a><a href="https://www.nimbleedge.com/blog/state-of-on-device-ai" target="_blank" rel="ugc noopener noreferrer">https://www.nimbleedge.com/blog/state-of-on-device-ai</a><a href="https://www.youtube.com/watch?v=Qqj_Nl2MihE" target="_blank" rel="ugc noopener noreferrer">https://www.youtube.com/watch?v=Qqj_Nl2MihE</a><a href="https://www.linkedin.com/events/7343237917982527488/comments/" target="_blank" rel="ugc noopener noreferrer">https://www.linkedin.com/events/7343237917982527488/comments/</a>~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~Catch all episodes, blogs, newsletters, and more: <a href="https://go.mlops.community/TYExplore" target="_blank" rel="ugc noopener noreferrer">https://go.mlops.community/TYExplore</a>Join our Slack community [<a href="https://go.mlops.community/slack" target="_blank" rel="ugc noopener noreferrer">https://go.mlops.community/slack</a>]Follow us on X/Twitter [@mlopscommunity](<a href="https://x.com/mlopscommunity" target="_blank" rel="ugc noopener noreferrer">https://x.com/mlopscommunity</a>) or [LinkedIn](<a href="https://go.mlops.community/linkedin" target="_blank" rel="ugc noopener noreferrer">https://go.mlops.community/linkedin</a>)] Sign up for the next meetup: [<a href="https://go.mlops.community/register" target="_blank" rel="ugc noopener noreferrer">https://go.mlops.community/register</a>]MLOps Swag/Merch: [<a href="https://shop.mlops.community/" target="_blank" rel="ugc noopener noreferrer">https://shop.mlops.community/</a>]Connect with Demetrios on LinkedIn: <a href="https://www.linkedin.com/in/dpbrinkm/" target="_blank" rel="ugc noopener noreferrer">/dpbrinkm</a>Connect with Varun on LinkedIn: <a href="https://www.linkedin.com/in/vkkhare/" target="_blank" rel="ugc noopener noreferrer">/vkkhare/</a>Connect with Neeraj on LinkedIn: <a href="https://www.linkedin.com/in/nrjpoddar/" target="_blank" rel="ugc noopener noreferrer">/nrjpoddar/</a>Timestamps:[00:00] On-device AI skepticism[02:47] Word suggestion for AI[06:40] Optimizing unique challenges[13:39] LLM on-device challenges[20:34] Agent overlord tension[23:56] AI app constraints[29:23] Siri limitations and trust gap[32:01] Voice-driven app privacy[35:49] Platform lock-in vs aggregation[42:26] On-device AI optimizations[45:38] Wrap up

Podurama

Home

Trending

Personal

Your Podcasts

New Episodes

My Files

Favourite Episodes

Tags (Folders)

Recently Played

Bookmarks

Followed Playlists

On-Device AI Agents in Production: Privacy, Performance, and Scale // Varun Khare & Neeraj Poddar // #340

MLOps.community

AI Chapters

AI Insights

Reviews