How to Befriend Generalist LLM’s – Lessons Learned Applying Generative AI to Build Better Data Products

Since the “attention” revolution, SOTA Large Language Models have grown 5000x bigger with unimagined capabilities and rapidly declining cost. The year 2023 saw the flourishing of commercial applications and the LLMops ecosystem; with enterprises rushing to ride the Gen AI tailwind. However, the “AI magic” that wowed ChatGPT users does not always easily translate into better products elsewhere. At CB Insights, we process millions of signals monthly to empower tech decision-makers and researchers. Our holy grail of serving ever-better insights from data is also boosted by LLMs, using both in-house specialist models and generalist LLMs like GPT. In this talk, we’ll share challenges and lessons learned applying Gen AI to build better data products. Part I will walk through the path of evolution for LLMs. Part II will discuss different ways to integrate generalist LLMs into your data products: LLM as a user interface, data interface, information extractor, and auto grader. Part III will take a stab at where we’re headed.