Prompting Basics - 60 minutes

Podman Desktop is an open source tool by Red Hat that enables you to work with containerized applications in an easy to use graphical interface local to your laptop or workstation. AI Lab is an extension to Podman Desktop that provides additional functionality for interacting with Large Language Models using a variety of experiences or recipes (including chat and document summarization). Its Playground feature enables you to easily select and download a model and then have an interactive conversation with it using a system prompt that you provide. The most common parameters, such as temperature and token count, can be customized directly within the playground UI itself.

If you would like to install Podman Desktop in your own environment, visit podman-desktop.io and click the "Download Now" button in the middle of the page. Once installed, click the download button next to AI Lab on either the home screen or search for it in the Extensions view.

We are using Quarkus and LangChain4j extensively throughout this workshop, and LangChain4j provides a Dev UI extension for testing system and user prompts. While it is a simpler experience compared to other tools, Dev UI is pre-configured to communicate with the same model and settings that our Parasol application is using. That model is exposed with an OpenAI-compliant API through OpenShift AI and has already been trained with Parasol Insurance business rules and content.

The system prompt is entered with new chat sessions using the System text field. To change the default value, click within the System text box and replace the existing value with your new system prompt. Once chat sessions begin when the first user message is submitted, system prompts cannot be changed until a new chat session is started.

User Messages are entered in the Message text box at the bottom of the screen.

Temperature and Max Tokens can be changed in the Dev UI Configuration view. If necessary, navigate to the configuration screen and then use the Search bar to find and change the default values. (Please note that the default settings are sufficient for completing today’s exercises.)

For the remainder of the lab, you will be using the Quarkus Dev UI with the LangChain4j Chat extension as your prompt engineering playground.

Red Hat Developer Hub integrates with Red Hat Single Sign On (RH-SSO) to enhance security and user experience for developers within an organization. By integrating with RH-SSO, developers only need to log in once to access both RHDH and other applications secured by RH-SSO. This eliminates the need to manage multiple login credentials for different developer tools and services. RH-SSO centralizes user authentication and authorization to strengthen security by ensuring only authorized users can access RHDH and other protected resources. In addition, The platform engineering team can manage user access and permissions centrally through RH-SSO, simplifying administration and reducing the risk of unauthorized access.

To get started, access Red Hat Developer Hub Dashboard to authenticate by GitLab.

Choose the GitLab option.

Enter the following credential in the Gitlab login page.

Click Authorize Keycloak to allow the authentication to proceed.

Find more information about the Red Hat Developer Hub here.

In Red Hat Developer Hub, a software template serves as a pre-configured blueprint for creating development environments and infrastructure. Software templates streamline the process of setting up development environments by encapsulating pre-defined configurations. This saves developers time and effort compared to manually configuring everything from scratch.

Click on Create…​ on the left menu. Then, type Parasol in the search bar. Then, Choose the Parasol Insurance for Red Hat AI App Development (RAD) Workshop template.

Follow the next steps to create a component based on the pre-defined Software Templates:

When all of the initialization steps are successful (green check mark), click Open Component in catalog.

Dev Hub will open a new tab with the component information. Take a few moments to review the Deployment summary on the overview page.

Click on VIEW SOURCE to access the new source code repository created.

Go back to your Parasol component on Red Hat Developer Hub: Red Hat Developer Hub UI.

Open your workspace in Dev Spaces per the instructions in the prior section.

Spawn a terminal window within the IDE by clicking on the icon with three parallel bars in the upper left corner of the screen. Choose "Terminal" and then "New Terminal" menu entry from the list.

Run the following command from the cli using the terminal window just created.

In a minute or so, you will be prompted that a new application was detected with an open web port. Disregard this confirmation dialog. Instead, let’s use the "Endpoints" section at the bottom of the "Explorer" window on the left-hand side of the screen. In that view, click the middle icon adacent to "quarkus-devui" that opens Dev UI in a new tab. See the following screenshot for guidance.

Or you can also access the Quarkus Dev UI directly.

Click on Chat. This is highlighted with a red box in the screenshot below.

As we reviewed before, prompting is as important to getting value from a model as training. Prompting begins with a system prompt, which is where we recommend personas, context and guardrails be established. The System Prompt is a required field in AI Lab. Since our initial project for Parasol Insurance is focused on policy rules and regulatory requirements, let’s go ahead and set that context here.

Now let’s use a simple prompt that we know uses data that was included in the previous fine tuning exercises.

Copy the following text into the Message text box at the very bottom of the page.

Click the Send button.

The response will be negative or a direct No.

Chat playgrounds carry your previous AI interactions from that same session forward when interacting with the model. The LLM itself is essentially stateless from that perspective and the client application or service is responsible for accumulating and transmitting this history. Within the context of that session, this provides additional context to the LLM and is as influential to its response as the system prompt. How a user responded to its previous response helps it provide more meaningful responses or continue with similar levels of implied context in future inquiries.

Now ask the following question in the same chat session:

The LLM will now respond with what factors influence the cost of adding rental car insurance. Notice that the system was able to infer the meaning of "it". Also notice that it didn’t provide a specific dollar value as it relates to the customer inquiry. Numbers, calculations, math and data specific context (like that associated with a specific customer) are specific challenges we will look at later. For now, we are focusing on more general usage of the LLM.

Let’s see if the LLM is still adhering to our guidance in the system prompt. Ask it the following question about tennis shoes and confirm that it declines to comment:

The LLM should decline to answer the question.

Take some time to ask the LLM various questions and see how it responds. Remember, chat history influences the next response so start new playgrounds as needed for a fresh start.

Click the Start new conversation button again. Then, change the System Prompt to the following.

Copy the following question into the Message text box and click the Send button.

Analyze the response. Since we are giving the LLM very little information to go on, the results are somewhat unpredictable.

Try starting a new chat and using the same System Prompt and Message. Observe how the results vary each session.

From your perspective as an AI application developer, this is not helpful since the results are not in a format that the system can parse.

Click the Start new conversation button again. Then, change the System Prompt to the following.

Enter the following into the Message text box and click the Send button.

Notice how the LLM’s response more naturally conforms to a format a consuming application can consume.

Customers frequently reach out to Parasol Insurance’s customer service team to discuss their insurance claims. These emails are very diverse and routing them amongst teams is currently a time consuming manual task. This is where you come in - Parasol Insurance leadership is eager to pilot Generative AI in the context of email routing and responses. It hopes this automation will allow it to move these claims support resources to the claims analysis team, who is getting behind in their processes and causing customer impacting delays.

As the AI application developer, you have been given access to a robust LLM that has already been trained on Parasol Insurance’s business rules. We will use this model to enhance the Parasol application with customer email analysis, routing, and response. The robust analysis must be performed by the LLM and will require solid prompt to produce quality results.

Your Functional Requirements:

Required LLM Input/Output Specifications:

"customer email body"

{ "address": "new.claim@parasol.com", "subject": "<Email Subject>", "message": "<Customer Comment>" }

{ "address": "existing.claim@parasol.com", "subject": "<Email Subject>", "message": "<Customer Comment>" }

{ "address": "customer.service@parasol.com", "subject": "<Email Subject>", "message": "<Customer Comment>" }

{ "address": "discard@parasol.com", "subject": "<Email Subject>", "message": "<Customer Comment>" }

Think about what you’ve learned so far with LLMs and ground yourself in clear communication vs. a traditional programmatic solution to this problem. How would you communicate this use case to a new employee with no background at all in insurance or customer service? How would you communicate what good looks like in a way that is descriptive of a pattern?

We recommend that you create a new text file in the VS Code and then paste into playground for testing, as this will be an iterative process and you will need to restart conversations many times as you evolve your system prompt.

You can write your own sample emails and paste them into the user prompt text box or you can use the ones below that were copied from the Parasol Insurance application.

If you run into challenges while drafting the prompt, ask your instructor for assistance.

Now that you’ve developed a working prompt that can help satisfy leadership objectives with its first Generative AI initiative, you need to incorporate this logic into the Parasol Insurance application so that ongoing business value may be realized. For this exercise that involves updating the Email Response REST service and UI to provide an recommended recipient, email subject, and automated response based on a given customer email.

Now that we’ve developed a robust prompt that helps address the Parasol Insurance use case for email handling, we must integrate that prompt into the Parasol application.

During a prior workshop activity, email response generation using LangChain4j was implemented into the web UI. This functionality includes a new menu option on the left hand side of the screen that, when clicked, opens a web form for providing the customer email. We are going to replace the prompt used in that module with the one you created above that includes more functionality and data elements.

The Parasol Insurance application is invoking the LLM using LangChain4j’s AI Service framework. This approach leverages Java Interfaces created by the user with annotations that define the prompt using a String. Let’s open the AI Service that was previously created for Email Response Generation:

src/main/java/org/parasol/ai/EmailService.java

Now, change the current prompt to the one we created together in the previous section.

Before:

Replace the text in the red rectangle with the following revised system prompt.

After:

Assuming your Quarkus environment is still running from the prior steps, the updated prompt is automatically compiled and applied. Let’s try out the email form and see a subset of the new response:

Now, you’ll notice that this does not look drastically different from before the enhancement. Let’s now add the Forward-To Email Address to the form.

Open the src/main/webui/src/app/components/EmailGenerate/EmailGenerate.tsx typescript file for the Generate Email view.

At the top of the source file there is a data structured called EmailResponse. Add address of type string to the end of the list.

Add the following html to the email-response div block:

Additionally, we must add the address field to the REST Service’s JSON Response.

Open the src/main/java/org/parasol/model/EmailResponse.java file and replace the Java record signature with the following code.

Now let’s revisit the web form and test out the new AI-generated attribute:

Stop the Quarkus dev mode environment by Ctrl+C.

Congratulations! You have successfully modernized the Parasol Insurance application and improved the enterprise business processes surrounding customer email handling! We look forward to celebrating similar successes in your own applications!