Continuing the Conversation: Building Open-Source Semantic Discovery Tools
Northwestern University was recently awarded an IMLS Leadership Grant to extract an installable toolkit based on its recently launched open-source chat-based semantic search for Digital Collections https://www.library.northwestern.edu/about/news/library-news/2024/grant-supports-northwestern-libraries-launch-of-generative-ai-based-chat-search.html. We’ll discuss our work with Retrieval Augmented Generation (RAG) architectures, generating embedding vectors at scale in production, and the challenges of grounding AI responses in data managed by experts at the library. We will also discuss developing a toolkit that will allow other institutions to explore this impactful technology with their own data. Learn about opportunities, challenges, and discoveries we’ve made so far.
- Learn the fundamentals of semantic search and retrieval and integrations with genAI tools
- Learn about the semantic discovery tool developed as part of the IMLS Leadership grant
- Hear about the complexities of testing and evaluating generative AI pipelines and semantic retrieval systems
- Understand how Northwestern is leveraging AI in Digital Collections and expanding to other contexts