From PyTorch to Fireworks AI: Lin Qiao on Building AI Infrastructure

High Agency: The Podcast for AI Builders

This week we’re talking to Lin Qiao, former PyTorch lead at Meta and current CEO of Fireworks AI. We discuss the evolution of AI frameworks, the challenges of optimizing inference for generative AI, the future of AI hardware, and open-source models. Lin shares insights on PyTorch design philosophy, how to achieve low latency, and the potential for AI to become as ubiquitous as electricity in our daily lives.

Chapters:
00:00 - Introduction and PyTorch Background
04:28 - PyTorch's Success and Design Philosophy
08:20 - Lessons from PyTorch and Transition to Fireworks AI
14:52 - Challenges in Gen AI Application Development
22:03 - Fireworks AI's Approach
24:24 - Technical Deep Dive: How to Achieve Low Latency
29:32 - Hardware Competition and Future Outlook
31:21 - Open Source vs. Proprietary Models
37:54 - Future of AI and Conclusion

I hope you enjoy the conversation and if you do, please subscribe!

--------------------------------------------------------------------------------------------------------------------------------------------------
Humanloop is an Integrated Development Environment for Large Language Models. It enables product teams to develop LLM-based applications that are reliable and scalable. To find out more go to humanloop.com