In the era of data-driven decision-making, companies are investing heavily in data platforms to structure and harness their domain knowledge. However, a challenge arises when working with Large Language Models (LLMs), as they are capable of processing only a few thousand words at once. This talk aims to explore the delicate balance between big data and small context, discussing how to effectively couple existing data platforms with the power of LLMs while considering their limitations.