Concerto for Java & AI - Building Production-Ready LLM Applications

Imagine you're a music composer struggling to find inspiration for a pivotal movie scene. Then you remember you're also a software engineer, and the solution becomes obvious. In this session, I'll share how I enhanced my music composition process by harnessing the power of Java and AI. You’ll learn how to enhance existing applications with AI capabilities such as text classification, structured data extraction, semantic search, and agentic tools. Throughout the session, I’ll build a "composer assistant" application using Spring AI to demonstrate how to make an LLM application production-ready. What’s the developer experience like when working with models? Is observability different? What strategies should you use for deploying LLM applications? In a final twist, you’ll choose a movie scene to score, and I’ll compose and perform the music live, supported by AI. Will it hit the mark? There’s only one way to find out: join me in this musical journey with Java. Aaaaand action! Presented by Thomas Vitale at JavaOne 2025 (CA, March 2025). All JavaOne talks ➤ https://www.youtube.com/playlist?list=PLX8CzqL3ArzVV1xRJkRbcM2tOgVwytJAi ➤ https://github.com/ThomasVitale/llm-apps-java-spring-ai ➤ https://github.com/ThomasVitale/concerto-for-java-and-ai ➤ https://www.thomasvitale.com ━━ Chapters ━━ • 0:00 Intro • 2:10 The WHY Factor • 4:35 Machine Learning • 6:38 Model Inference via HTTP APIs • 8:28 Building a Chatbot • 13:37 Protecting Against Prompt Injection • 16:40 Semantic Searching • 24:05 Retrieval Augmented Generation • 28:05 AI Model Observability • 30:45 Structured Data Extraction • 33:07 Tools/Function Calling • 35:19 Developing a Composition Plan • 37:00 Writing a Composition Plan • 44:30 Conclusion Tags: #Java #JavaOne #OpenJDK #ai SES1515