Building a private and secure Gen AI for Enterprises with RAG

By now, you would have experienced the capabilities of Generative AI in multiple ways, maybe you asked ChatGPT to write a blog or provided the enterprise version of it to your teams; but they come with limitations of being trained with past data past and cannot provide contextually relevant information or answers that are highly specific to your organization.

The P in GPT stands for ‘pre-trained’; which means it was trained on available set of inputs parameters at the time of its development, and this often leads to few known sets of problems including:

• AI Hallucinations / Factually incorrect response
• Contextual limitations
• Lower accuracy
• Overfitting or underfitting
• Bais and fairness issue

Hallucination / incorrect responses is when AI generates plausible sounding but factually incorrect, irrelevant, inappropriate, or nonsensical responses, and this is the biggest pain point among the list, I’ll tell you why. Imagine you are walking in a new country and ask a passerby for directions to the nearest bus stop, the person answering you has no idea where’s the bus stop but wants to be polite and points you in some random direction! What happens if your peers in office are relying on Gen AI to ‘get directions’ at work and are misled?

Additionally, most Gen AI models are trained to provide ‘some answer’ than to tell you ‘I don’t know’. This makes it dangerous if a manager is using a hallucinated answer from the AI to make mission critical decisions.

On the other hand, we have improved how we search and retrieve information from the web or file servers inside an organization, but with all known refinements in search,
conventional methods often fall short in the domain of enterprise knowledge management (KMS) for several reasons:

• Lack of contextual understanding
• Inability to handle unstructured data
• Inefficient query handling
• Lack of knowledge graph
• Doesn’t always support semantic search

Beyond all these limitations, they provide an overwhelming set of options and links as search results, which leads a knowledge worker to mine & sift through exhaustive number of responses to seek the right answers.

According to a McKinsey report, white collar employees spend 1.8 hours every day; 9.3 hours per week, on average searching and gathering information.

Be it a query on company policy or process, a single piece of information from an entire quarter worth of sales data, or a single value from the entire repertoire of accounting books of records…You know what information you seek is available, but it is safely, securely stored inside ‘some PPT’ or ‘some excel’ sheet in a vast file server.

So, Gen AI hallucinates, conventional search is inefficient. What can be done?

Enter RAG. Which stands for Retrieval Augmented Generation, an advanced AI technique that combines the strengths of information retrieval from your own data sources and generative models to produce highly accurate and contextually relevant responses.

It’s a hybrid approach that leverages the strengths of both LLM and your own data to address the limitations of purely generative or purely retrieval-based methods. A shift from static to interactive, from overwhelming to precise; KMS built with RAG overcomes siloes in knowledge management by integrating various trusted information sources you own in a secure manner.

Imagine the following use case, Paisa Bank India has created a pilot program which gives access to its employees to use a popular Gen AI that is publicly available for enterprises. While people in the company can use it for writing memo drafts or marketing team can use it to draft blogs, the model doesn’t know nuances that is specific to the bank beyond what is available publicly on the web, and it is also not updated real time. in sum, the responses generated will be generic.

But if PaisaBank India was to build a RAG powered KMS or Secure GPT as we call it, the responses will be much sharper, contextually relevant with a very high utility rate. It can even provide source documents on-demand for further continuance of the task at hand.

With a rightly implemented RAG, enterprises can enable high amount of cross functional collaboration among peers, help save hours of time spent in information gathering, provide reliable and accurate information on-demand; all this while keeping the source data of the enterprise secure and private.

FAQs & Caveats

Wait, will my private enterprise data be used by the LLM to train their models?
Short answer, no.
There’s a way of running a secure GPT model inside your VPC while keeping your private data securely stored elsewhere. Agrahyah can help you implement a secure GPT / KMS with RAG on a turnkey basis.

Will everyone in my organization have access to everything?
That doesn’t sound right. We have devised combination of techniques to configure tier wise information access matrix that is beyond hard coded rules, system assigned IAM roles or active directory. Give us a hoot, and we’ll tell you how to provide the right information to the right member in your team.