Private Docs (RAG/Tools) - 45 minutes

Red Hat Developer Hub integrates with Red Hat Single Sign On (RH-SSO) to enhance security and user experience for developers within an organization. By integrating with RH-SSO, developers only need to log in once to access both RHDH and other applications secured by RH-SSO. This eliminates the need to manage multiple login credentials for different developer tools and services. RH-SSO centralizes user authentication and authorization to strengthen security by ensuring only authorized users can access RHDH and other protected resources. In addition, The platform engineering team can manage user access and permissions centrally through RH-SSO, simplifying administration and reducing the risk of unauthorized access.

To get started, access Red Hat Developer Hub Dashboard to authenticate by GitLab.

Choose the GitLab option.

Enter the following credential in the Gitlab login page.

Click Authorize to allow the authentication to proceed.

Find more information about the Red Hat Developer Hub here.

In Red Hat Developer Hub, a software template serves as a pre-configured blueprint for creating development environments and infrastructure. Software templates streamline the process of setting up development environments by encapsulating pre-defined configurations. This saves developers time and effort compared to manually configuring everything from scratch.

Click on Create…​ on the left menu. Then, type Parasol in the search bar. Then, Choose the Parasol Insurance for Red Hat AI App Development (RAD) Workshop template.

Follow the next steps to create a component based on the pre-defined Software Templates:

When all of the initialization steps are successful (green check mark), click Open Component in catalog.

Dev Hub will open a new tab with the component information. Take a few moments to review the Deployment summary on the overview page.

Click on VIEW SOURCE to access the new source code repository created.

Go back to your Parasol component on Red Hat Developer Hub: Red Hat Developer Hub UI.

Click on CI tab to review your pipeline information. The pipeline will be automatically triggered and run. After a few minutes, you will see the pipeline finished as Succeeded.

Click on CD to review ArgoCD History.

Click on TOPOLOGY to review your deployment status. Then, you will see the deployment parasol-app-user1 in the Topology.

Access the Parasol web page to verify everything is working as expected.

You’ll be building new features in this application, based on Quarkus and the LangChain4j library. The Quarkus LangChain4j extensions bridge the gap between your Quarkus application and LangChain4j, a library that allows interaction with various LLMs like OpenAI, Hugging Face, or Ollama. It has the following key features and benefits:

Before examining the application’s code we need to first ensure you are starting from a clean state. Whether this is the first module you are completing or not, this module assumes starting from scratch.

Open a terminal window within Dev Spaces by clicking on the hamburger menu button in the top left, then click on Terminal and finally on New Terminal as seen below:

In the terminal window that has appeared at the bottom of the editor, run the following command:

This Quarkus application is a customer service processing tool that handles customer claim emails for Parasol insurance. The team has recently improved the application with a chatbot to interact with the agent and to generate responses based on the email content. In the VS Code environment, navigate to the src/main/java/org/parasol directory, which contains the main source code of the application.

In the src/main/java/org/parasol/ai/ClaimService.java file, you’ll find the main AI chatbot class that processes the customer claim emails with a @SystemMessage and @UserMessage annotation, and a chat method. The chat method processes the claim details and question, then generates a response based on the claim and references provided.

You can also explore the src/main/resources/application.properties file to review the application configuration, including the Quarkus configuration and the LangChain4j extension settings.

Then on the chat icon in the bottom right corner to open the chat interface.

You should notice that the chatbot can answer these questions based on the claim context we provided. Now, let’s ask a more complex question that requires knowledge of Parasol’s policies:

The answer you will get back might be different from the one in the screenshot above, but you’ll likely observe that the chatbot struggles with this question, as it doesn’t have access to Parasol’s specific policies. It might even respond with an incorrect decision! If only there was a way to automatically retrieve this information and provide it to the chatbot..

Use Retrieval-Augmented Generation (RAG) when you need to access a dynamic knowledge base in real-time, especially for tasks that involve varied or constantly updating information. RAG is ideal if you require scalability, need to handle out-of-domain queries, or want to deploy quickly without the resource demands of fine-tuning - which often requires specialized knowledge of working with AI models, something application developers often don’t possess.

On the other hand, choose fine-tuning when your task is specialized and you need precise control over the model’s behavior. Fine-tuning is better suited for homogeneous data, offline or low-latency applications, and situations where security or compliance requires all data to be embedded within the model. It’s also preferable when you have a well-defined use case with specific task requirements.

While we could use a vector database to store vectorized data to use with RAG, we can also simply specify a directory where text-based files are stored. We can then parse this data, retrieve similar data, vectorize it, and pass it along to our LLM calls. This process is made very easy with the aptly named "Easy RAG" extension in Quarkus.

Return to Dev Spaces and open a new terminal window as you did before (click on the hamburger menu button in the top left, then click on Terminal and finally on New Terminal), or click the + button on the top-right of your current terminal.

In the new terminal, add this extension by running the following command:

You will see that a new extension was added to your application:

We have already added a policy document (policy-info.pdf) in the src/main/resources/policies folder. {gitlab_url}/development/parasol-app-{gitlab_user}/-/blob/master/src/main/resources/policies/policy-info.pdf[Click here^] to view it.

We can tell the Easy Rag extension where to find it by adding the following lines in your src/main/resources/application.properties:

That’s all there is to it! When Quarkus restarts it will read the src/main/resources/policies/policy-info.pdf file and compute vectorized embeddings for all the text in the document. It will then store these embeddings into an in-memory vector store. When chatting with the chatbot the application will first query the vector store for relevant information, which it will then pass to the LLM with the user’s query.

Reload the browser window to make sure Quarkus Dev mode picks up the changes.

Now ask the same question in the chat window:

This time the bot should be able to answer the questions with accurate information based on the policies document!

In Dev Spaces, switch to the second terminal you opened previously (NOT the one currently running Quarkus dev mode).

Add the mailer and mailpit extensions to the Quarkus application by issuing the following command in this terminal.

Open src/main/resources/application.properties and add the mail configuration:

Now let’s create a new NotificationService class that will have the functionality to update claim status and send an email. Create a new file NotificationService.java in the src/main/java/org/parasol/ai folder.

Overwrite all the content in the file with the following:

There is a fair bit of code in this file, but the main thing you should pay attention to is the @Tool annotation with the instruction "update claim status". This piece of natural language text you add to the annotation is interpreted by the LLM, which will now know that if you tell it to "update the claim status of my case", it should call the updateClaimStatus method with given parameters.

To wire everything up, we will need to tell the AI Service about this tool. In the src/main/java/org/parasol/ai folder, open the ClaimService.java class again.

Find the @RegisterAiService annotation. We need to register the NotificationService class as a tool. Go ahead and replace the existing line with the following:

By doing this we have now registered the NotificationService file as class that contains one or more @Tool annotated methods.

We now need to make a few more changes. Currently when we chat with the assistant the responses stream back to us one word at a time. Unfortunately when using tools you can’t use this feature. Instead, we need to make a request and wait for the entire response from the LLM.

Still in src/main/java/org/parasol/ai/ClaimService.java, replace the line

with

Then open src/main/java/org/parasol/resources/ClaimWebsocketChatBot.java and replace

with

The last thing we need to do is to switch to a different LLM for our chat bot because the parasol-chat model does not currently support tools integration.

Open src/main/resources/application.properties and overwrite these lines:

with:

Let’s try it out! Go back to the browser and refresh the claim page for claim 195501 (Marty McFly). Open the chat interface again and this time tell the chat assistant:

After a few moment you should see that the chatbot answers, telling you that it updated the status and sent an email to the customer.

If you refresh the chat page you should also see that the claim’s status has now been changed to denied:

Since we’re running this exercise in Quarkus Dev Mode, we can actually test if the email functionality actually works, thanks to the MailPit extension which provides a Quarkus Dev Service with a mock email server.

Go to the Quarkus Dev UI by opening the Endpoints view within Dev Spaces and finding quarkus-devui, then click the Open in a new tab button.

Find the "Mailpit" section and click on the link next to "Mailpit UI" and you will see a new email that was sent thanks to our newly added functionality!

Click on the email to open it

You can now return to the terminal where Quarkus dev mode is running and terminate the application by using the q key or hitting CTRL-C.