What is multimodal RAG, and can we build a village with it?
Conference (INTERMEDIATE level)
Mimosa 1
Multimodal Retrieval-Augmented Generation (RAG) is the art of using your own textual, visual and audio data to enhance prompts before they’re processed by a generative AI. Imagine users asking a question, the system querying your own data pool for answers, and then using generative AI to craft a coherent, relevant response… which is perfect for non-player characters (NPCs)! These are characters in video games that are not controlled by the player. Typically, NPCs have static and hardcoded responses, but with this approach we could make them more lifelike and fun to interact with!
We can take it even further. Imagine NPCs interacting with each other, having goals, and moving around dynamically, all driven by complex RAG workflows. This idea is exactly what Alexander has explored and wants to share with you!
Whether you’re a seasoned backend engineer or an adventurous developer seeking fresh ideas, Alexander will guide you through setting up basic to advanced RAG workflows. He'll demonstrate the challenges and benefits of using tools like Langchain4j to create RAG systems that provide up to date and factual answers.
Alexander’s goal is to offer an engaging and fun look at RAG, inspiring you to dive in and explore its potential yourself. Get ready for a session that mixes education and absurd fun to discover the inner workings of multimodal RAG!
We can take it even further. Imagine NPCs interacting with each other, having goals, and moving around dynamically, all driven by complex RAG workflows. This idea is exactly what Alexander has explored and wants to share with you!
Whether you’re a seasoned backend engineer or an adventurous developer seeking fresh ideas, Alexander will guide you through setting up basic to advanced RAG workflows. He'll demonstrate the challenges and benefits of using tools like Langchain4j to create RAG systems that provide up to date and factual answers.
Alexander’s goal is to offer an engaging and fun look at RAG, inspiring you to dive in and explore its potential yourself. Get ready for a session that mixes education and absurd fun to discover the inner workings of multimodal RAG!
Alexander Chatzizacharias
JDriven
Alexander, a 34-year-old Software Engineer at JDriven, holds dual Dutch and Greek nationality. He earned his master’s degree in Game Studies from the University of Amsterdam, where he discovered his passion for gamification and software engineering. Alexander aims to bridge the gap between game development and software engineering, believing that both industries have much to learn from each other. He is dedicated to integrating technologies and methodologies from both fields. Additionally, he enjoys experimenting with new technologies and cutting-edge sdk's.