Sounds pretty close to where reporting was before data storage and data mastering? So, should we borrow some ideas from data management and adjust them to the AI era? I think we should start creating shared, governed context storages that can be shared between multiple AI use cases.
Five lessons learned from building AI context storages
I want to share some things I have learned the hard way while doing this a few times now in customer projects. Hopefully, it can help someone trying to achieve this.
- Plan the context storage around your AI policy. If you don't have one yet, create it before planning the storage. You will need to know what data can be used with which AIs and for which use cases. Access for both the AI and the user should be built into the core of the context storage to make sure data access is restricted based on both AI services and user groups.
- Add enough metadata to be able to search efficiently. One search per question in vector storage is not the solution for organization-level context. Your agents will need to be able to search the context storage in multiple ways to iteratively find what is needed. Hierarchical documentation, direct metadata and documentation guidelines help AI find relevant documentation or other context from the storage.
- Make it easy to combine data from different source systems. When possible, have common keys between data from different systems in the context storage so that AI can easily fetch data based on different keys. Having to use mapping info to find related data just makes things slower for no reason.
- Use layers. It is OK to let agents search the raw data as well, but having a real distilled knowledge layer on top of that makes things much quicker and more predictable. Agents don't always have to comb through a large amount of raw data to understand the basics. They can use the higher-level documents to understand your business and interpret raw data based on that.
- Manage the context lifecycle. Context that is not up to date may be far worse than not having context at all. So, plan how the context will be updated and eventually removed.
