11-03, 13:30–14:10 (America/New_York), Central Park West (Room 6501)
As we descend from the peak of the hype cycle around Large Language Models (LLMs), chat-based document interrogation systems have emerged as a high value practical use case. The ability to ask natural language questions and get relevant answers from a large corpus of documents has the potential to fundamentally transform organizations and make institutional knowledge accessible.
Retrieval-augmented generation (RAG) is a technique to make foundational LLMs more powerful and accurate, and a leading way to implement a personal or company-level chat-based document interrogation system. In this talk, we’ll understand RAG by creating a personal chat application. We’ll use a new OSS project called Ragna that provides a friendly Python and REST API, designed for this particular case. We’ll also demonstrate a web application that leverages the REST API built with Panel–a powerful OSS Python application development framework.
By the end of this talk, you will have an understanding of the fundamental components that form a RAG model as well as exposure to open source tools that can help you or your organization explore and build on your own applications.
Foundational LLM models like OpenAI’s GPT4 provide powerful capabilities but using them directly to answer questions accurately about a collection of documents is hard. Retrieval-augmented generation (RAG) is the leading approach to enhance the capabilities and usability of Large Language Models. It can provide more accurate results to questions by augmenting the LLM with relevant external information retrieved from vector storage.
This talk hopes to enable you to quickly understand core concepts of this approach by building RAG-powered projects with a new OSS tool called Ragna.
We will:
-
Understand RAGs and how they interact with LLMs
-
Discuss OSS projects in this area align with their capabilities and challenges
-
Experiment with generating answers based on your uploaded documents, in a familiar Jupyter Notebook and using a new project called Ragna
-
Explore how choice of RAG components can affect the accuracy of answers
-
Create a UI to build on the experiment, with Panel and Ragna’s REST API
Beginner-level understanding of Python is sufficient to follow along, and some prior knowledge of Panel is nice to have. This talk is designed for enthusiasts in our community, to explore an interesting topic using some beginner-friendly tools.
No previous knowledge expected
Pavithra Eswaramoorthy is a Developer Advocate at Quansight, where she works to improve the developer experience and community engagement for several open source projects in the PyData community. Currently, she maintains the Bokeh visualization library, and contributes to the Nebari (adjacent to the Jupyter community) and conda-store (part of the conda ecosystem) projects. Pavithra has been involved in the open source community for over 5 years, notable as a maintainer of the Dask library and an administrator for Wikimedia’s OSS programs. In her spare time, she enjoys a good book and hot coffee. :)
Dharhas Pothina is the CTO at Quansight where he helps clients wrangle their data using the pydata stack. He leads the development teams for the Nebari, Conda-Store and Ragna open source projects.
His background includes expertise in computational modeling, big data/high performance computing, visualization and geospatial analysis. Prior to his current position he worked for 15 years in state and federal research labs where he led large multi-disciplinary, multi-agency research projects.
He holds a PhD in Civil Engineering and an MS in Aerospace Engineering from the University of Texas at Austin and a BTech in Aerospace Engineering from the Indian Institute of Technology Madras.
Dharhas is passionate about enabling scientists and engineers with tools that let them scale as well as share their analyses, he loves woodworking, photography and teaching his daughters to love science.