AI Model Fine-Tuning - 45 minutes

1. Goals of this lab

As Parasol Insurance embraces the era of AI, we face a unique challenge: leveraging cutting-edge AI technology while maintaining strict control over our proprietary data and processes. This journey requires us to develop in-house AI capabilities that are as powerful as they are secure. The goal of this lab is to show you as a developer how to use open source AI tooling to fine-tune a foundation model using Parasol’s proprietary data, producing an optimized and efficient LLM that can be used across a variety of use cases at Parasol. In this lab, you will:

  • Explore fine-tuning techniques for AI models, incorporating Parasol’s unique insurance expertise into LLMs

  • Learn how to create and utilize a custom knowledge base for training AI models with organization-specific scenarios and regulations

  • Apply the LAB methodology to generate synthetic training data and specialize our model while keeping data in-house

  • Gain hands-on experience in training and serving a customized AI model

  • Understand the benefits and limitations of fine-tuning compared to other AI customization methods

1.1. Enhancing AI Models: RAG vs Fine-Tuning

When customizing AI models, two techniques stand out: Retrieval-Augmented Generation (RAG) and fine-tuning.

  • Fine-tuning excels when you need precise, controlled outputs for specialized tasks using curated data. It’s ideal for applications where security and compliance require embedding all data within the model. Fine-tuning is also suitable when you have clear use cases with specific task requirements.

RAG vs Fine-Tuning

  • In contrast, Retrieval-Augmented Generation (RAG) is best when you need real-time access to dynamic knowledge bases, especially in environments with constantly updating information. RAG allows for scalability and handles out-of-domain queries effectively without the need for retraining, making it quick solution for improving model output. However, by providing contextual information to the model for each prompt, your expenses and computer resources may be higher than "baking-in" information into the model through fine-tuning.

RAG vs Fine-Tuning

2. Get Started with Fine-Tuning

Now that we’ve outlined Parasol’s AI needs, let’s dive into the process of meeting them. In this section, we’ll explore how to use the InstructLab project to tailor a foundation language model to Parasol’s specific requirements, focusing on the key areas we’ve identified.

2.1. What is InstructLab?

InstructLab is an open-source project designed to enhance large language models (LLMs) for use in generative AI applications. It provides a novel approach to model alignment and fine-tuning, allowing developers and domain experts to add new knowledge and skills to pre-trained models with minimal data and computational resources. Key features of InstructLab include:

  • A taxonomy-driven approach to curating training data

  • Large-scale synthetic data generation

  • Iterative alignment tuning for continuous model improvement

InstructLab Overview

InstructLab is particularly useful for organizations that want to leverage private AI and keep their data in-house while still benefiting from state-of-the-art language models, without needing AI/ML or data science expertise.

As you work with InstructLab, you will see the terms Skills and Knowledge. What is the difference between Skills and Knowledge? A simple analogy is to think of a skill as teaching someone how to fish. Knowledge, on the other hand, is knowing that the best place to catch a Bass is when the sun is setting while casting your line near the trunk of a tree along the bank.

2.2. Access your Virtual Environment

To begin working with InstructLab, you’ll need to access the provided noNVC virtual environment. This environment comes pre-configured with all necessary tools and dependencies. Connect to noVNC server using the password openshift.

2.2.1. Click the Activities label in the top-left corner of the screen

2.2.2. Click the Show Applications icon to show all the applications

2.2.3. Click the Visual Studio Code icon to launch Visual Studio Code

Launch Podman Desktop

2.2.4. InstructLab mainly relies on the ilab command-line interface (CLI) to interact with the project

To access the ilab CLI, open a new terminal in Visual Studio Code by clicking on the Terminal menu and selecting New Terminal.

Open new VSCode terminal

2.2.5. Copy and Paste in your VNC environment

In the noVNC environment, you cannot directly copy content from your local machine and paste into the remote desktop. Instead, there is an intermediate clipboard which you can use for this purpose. Anytime you need to copy/paste from a block of text or code from these instructions into the noVNC desktop, just click on the icon to copy it to your local clipboard:

Copy Paste instructions

Next, click the small arrow at the far left to open the noVNC control panel, and click the Clipboard icon. This will show the current content of the clipboard, which should be the content you just copied.

To paste the content, position the cursor in the field in which you wish to paste into, and press the paste command (usually CTRL+V on Windows/Linux and CMD+V on Mac):

The following screenshots are examples, to detail the process of copying and pasting text from the instructions into the noVNC environment. The actual content may vary.
Copy Paste instructions

In some apps (e.g. VSCode), you can position the cursor at the paste point, and right-click in the field and select Paste:

Copy Paste instructions

Another option is to open these instructions in a browser within the VNC environment, which does not require using the intermediate clipboard. To do this, Click on Activities and open the Firefox browser, and type in red.ht/rad-instructions into the browser.

Copy Paste instructions

With this option, you simply click the same button at the upper right of the text you wish to copy, and then move the cursor to where you wish to paste, and press Ctrl+V (or in some apps such as a Linux terminal, Ctrl+Shift+V).

3. Model Fine-Tuning for the Insurance Organization

In the next few sections, we’ll walk through the process of fine-tuning an AI model using InstructLab. We’ll start by setting up our environment, generating synthetic training data, training the model, and then interacting with it. We will build upon that and delve further into the biggest insurance company in North America, Parasol, which has the most extensive customer base. Parasol Insurance gets many requests to process claims, questions about different products, etc. These requests are not just internal but also external.

Parasol Insurance’s primary concern is ensuring that its staff is capable of handling such requests and has access to this information through a single interface rather than going through multiple systems to scrape documents and internal portal pages. To this end, you have been tasked with adding knowledge that will aid the following use cases.

  • Products and coverage (e.g., providing comprehensive policy and product information)

  • Basic knowledge of the Insurance rules (e.g., offering insights on relevant local regulations)

  • Responses to general claim questions and remedies (e.g., generating product-specific email templates)

Parasol Insurance

3.1. Preparing the Parasol Insurance Knowledge Base

The approach of fine-tuning a model allows us to shape a language model to better understand a specific domain, and fill in the gaps in its knowledge. The InstructLab taxonomy provides a structured way to guide the model fine-tuning process, enabling us to add domain-specific knowledge to the model in a heirarchical manner, similar to the example below:

InstructLab Taxonomy

Your role is crucial in this process. You’ll be adding a knowledge domain to the LLM, using the organization’s specific information, knowledge that the LLM doesn’t have and is specific to Parasol Insurance.

3.2. Understanding the Knowledge Structure

Knowledge consists of data and facts and is backed by documents. When you create knowledge for a model, you’re giving it additional data to more accurately answer questions. Knowledge contributions in this project contain a few things:

  • A file in a Git repository that holds your information. For example, these repositories can include markdown versions of information on: Parasol products, insurance domain knowledge, claims processing etc.

  • A qna.yaml file that asks and answers questions about the information in the git repository, with desirable responses.

  • An attribution.txt that includes the sources for the information used in the qna.yaml, which aids in transparency and accountability.

LLMs have inherent limitations that make certain tasks extremely difficult, like doing math problems. They’re great at other tasks, like creative writing. And they could be better at things like logical reasoning. However, these limitations can be overcome by providing them with the right knowledge (and skills, which InstructLab can also help with). An LLM with knowledge helps it create a basis of information that it can learn from, then you can teach it to use this knowledge via the qna.yaml files.

In our case we want the LLM to learn more about Parasol Insurance like so:

version: 2(1)
task_description: "Teach a model more details about Parasol Insurance"(2)
created_by: sshaaf(3)
domain: insurance(4)
seed_examples:(5)
  - question: What is class imbalance in the context of Parasol insurance claims datasets?
    answer: |
      Class imbalance refers to the situation where the number of non-claims instances far exceeds that of actual claims, posing challenges for predictive modeling.

document:(6)
  repo: https://github.com/sshaaf/parasol_knowledge.git
  commit: b87677d
  patterns: (7)
    - Insurance_claims_data.md

Each qna.yaml file requires a minimum of five question-answer pairs. The qna.yaml format must include the following fields:

1 version: Defines the InstructLab taxonomy schema version
2 task_description: An optional description of the knowledge for easily understanding the specific knowledge contribution
3 created_by: The author of the contribution, typically a GitHub username
4 domain: Category of the knowledge
5 seed_examples: Five or more examples sourced from the provided knowledge documents, representing a question for the model and desired response
6 document: The source of your knowledge contribution, consisting of a repo URL pointing to the knowlege markdown files, and commit SHA that contains the specific files
7 patterns: A list of glob patterns specifying the markdown files in your repository. Any glob pattern that starts with , such as .md, must be quoted due to YAML rules. For example, "*.md". In our case we have placed all the knowledge documents in the parasol-knoledge repository.

3.3. Creating the Parasol Insurance Knowledge Base

Now that we understand the constructs of the taxonomy’s knowledge, let’s go ahead and create our knowledge base, which we will then feed into the LLM to train. This will help our applications that utilize the LLM, and agents directly chatting with the model. Furthermore, it will help with claims processing, fraud detection, or anyone who would like to ask the LLM about products, coverage, laws, and some information about Parasol itself. Let’s get started!

You can open VSCode by following the instructions below:

3.3.1. Open the instructlab directory in Visual Studio Code through the terminal:

cd ~/instructlab && code . --reuse-window

3.3.2. With the new window open, you can select Yes, I trust the authors to proceed

You’ll be prompted to enter your password, openshift, to confirm the action.

Trust Authors

3.3.3. Navigate to the taxonomy/knowledge/ folder

This file will contain the questions and answers that will be used to train the model.

3.3.4. Create a structure for Parasol insurance knowledge

To do that create folders by first right clicking on the Knowledge folder, and then pressing New Folder, as shown in the screen shot below

New Folder

We should create a knowledge folder structure that we can add to later as we add more knowledge and for our peers to also understand how its structured. Lets create a structure like this knowledge > economy > finance > insurance. In VSCode this is quite easy.

3.3.5. In the field as shown in the image below type economy/finance/insurance

New folder

Perfect, now we have the basic working structure to add in specific knowledge about our organization.

4. Curating the Data for our AI Model

Let’s now start adding the following taxonomy knowledge files for Parasol Insurance, consisting of the following (you can expand each section to see the instructions):

4.1. Knowledge File #1: Driving Age

Details

There are specific driving age rules for New Hampshire and Alaska for Teens driving. Lets add this information to the Model.

Create a new folder driving_age under knowledge > economy > finance > insurance as shown in the image below

Copy the following and add it as a new file called qna.yaml in the folder driving_age as shown in the image above. qna.yaml file ("qna" is short for "questions and answers")

New folder 
version: 2
task_description: "Driving age in New hampshire and Alaska"
created_by: sshaaf
domain: insurance
seed_examples:
  - question: Whats the legal driving age for teens in Alasks?
    answer: |
      Drivers must be at least 14 years old to obtain an instruction permit in Alaska.
  - question: Whats the legal driving age for teens in New Hampshire?
    answer: |
      Teenage drivers in New Hampshire are allowed to practice driving beginning at age 16 years and 6 months.
  - question: When can teen drivers get a Youth Operator License?
    answer: |
      Beginning at age 16, teen drivers may apply for a Youth Operator license in New Hampshire,
      assuming the teen has taken an approved driver education course.
  - question: When does the youth operator license expire?
    answer: |
      It expires on the individuals 21st birthday.
  - question: What are the rules for teens driver license in Alaska?
    answer: |
      Must hold permit for 6 months, and complete 40 hours driving, 10 of which should be at night.
  - question: Can teen drive with passengers?
    answer: |
      Teen drivers can not drive with passengers. Restrcitons are lifted at 18 years in Alaska.
document:
  repo: https://github.com/sshaaf/parasol_knowledge.git
  commit: b22cd9c
  patterns:
    - alaska_teen_driving_laws.md
    - new_hampshire_driving_laws.md

And now lets also create an attribution.txt file for citing sources. Copy the following and create a new file attribution.txt in the folder driving_age

Title of work: Parasol Insurance
Link to work: https://huggingface.co/rh-rad-ai-roadshow
License of the work: CC-BY-SA-4.0
Creator names: Syed M Shaaf, Philip Hayes

A qna.yaml that contains a set of key/value entries with the following keys. Each qna.yaml file requires a minimum of five question and answer pairs. An attribution.txt that includes the sources for the information used in the qna.yaml

4.2. Knowledge File #2: Parasol Claims Data

Details

Like any insurance company on the planet, data is stored into multiple systems, files etc. Employees at Parasol Insurance either using the system for the first time or using it for e.g. detecting fraud, tyring to understand the glossary, acronyms etc. A good example is Policy ID, a unique ID for policy in our database systems. The LLM does not know about this. By adding this, we can ensure that once a claims agent or an application asks about a policy ID, the LLM can give reasonable answers and suggestions.

Create a new folder parasol_claims_data under knowledge > economy > finance > insurance as shown in the image below

New folder

Copy the following and add it as a new file called qna.yaml in the folder parasol_claims_data as shown in the image above.

version: 2
task_description: "Teach a model more details about Parasol Insurance"
created_by: sshaaf
domain: humanities
seed_examples:
  - question: What is a claim status?
    answer: |
      Indicates the status of a cliam. 1 if a claim was made and 0 if it was not.
  - question: Whats the application of the claims data?
    answer: |
      It can be used to determine Risk Assesment, Claim history, Policy detials,
      Risk factors or external factors influencing a cliam.
  - question: What is a policy id?
    answer: |
      It is the unique identifier for a Parasol insurance policy.
      Everytime one needs to find a policy they can use this number.
  - question: Does the number of cylinders and max_power or max_torque effect a claim?
    answer: |
      It does not effect the claim directly but there can be external factors such as DUI or over speeding.
  - question: Can automatic systems, sensor, airbags reduce number of claims?
    answer: |
      Yes the likely hood of reducing claims increases using systems that aid the drivers.
document:
  repo: https://github.com/sshaaf/parasol_knowledge.git
  commit: b22cd9c
  patterns:
    - Insurance_claims_data.md

And now lets also create an attribution.txt file for citing sources. Copy the following and create a new file attribution.txt in the folder parasol_claims_data

Title of work: Parasol Insurance
Link to work: https://huggingface.co/rh-rad-ai-roadshow
License of the work: CC-BY-SA-4.0
Creator names: Syed M Shaaf, Philip Hayes

4.3. Knowledge File #3: Parasol Insurance Overview

Details

Here, we are adding some basic information about Parasol Insurance, an overview of product details. This will enable the LLM to give answers on a high level about the different offerings, fomulate a context about Parasol Insurance, history etc.

Create a new folder parasol_insurance under knowledge > economy > finance > insurance as shown in the image below

New folder

Copy the following and add it as a new file called qna.yaml in the folder parasol_insurance as shown in the image above.

version: 2
task_description: "Teach a model more details about Parasol Insurance"
created_by: sshaaf
domain: humanities
seed_examples:
  - question: What is Parasol Auto Insurance?
    answer: |
      Parasol is an Insurance company providing Car Insurance coverage to customers all over the United States.
  - question: What products does Parasol Insurance offer?
    answer: |
      Bodily injury cover upto $250,000.
      Property damage upto $100,000.
      Medical coverage upto $5,000 per person.
      Uninsured Motorist bodily injury upto $250,000 per person.
      Loss of use rental cover is $50 per day.
      The comprehensive deductable is $500.
  - question: What is Loss of Income?
    answer: |
      If you are at fault, to be able to claim for your own lost wages from Parasol Insurance
  - question: What is Uninsured and Underinsured Motorist Coverage?
    answer: |
      If an at-fault driver is uninsured or doesnt have enough coverage to pay for your injuries or car damage.
  - question: What is Car Rental Insurance?
    answer: |
      Parasol will pay the daily rental charge incurred when an insured rents a car from a car business
      while your car or newly acquired car is not driveable.
  - question: What is Apex plus package?
    answer: |
      The Parasol insurance Apex plus package provides additional coverage to motorist on top of their insurance policy
document:
  repo: https://github.com/sshaaf/parasol_knowledge.git
  commit: b22cd9c
  patterns:
    - Parasol_Auto_Insurance_Products.md
    - Parasol_auto_insurance.md
    - claims_cost_data.md
    - vehcile_insurance_in_the_United_States.md

And now lets also create an attribution.txt file for citing sources. Copy the following and create a new file attribution.txt in the folder parasol_insurance

Title of work: Parasol Insurance
Link to work: https://huggingface.co/rh-rad-ai-roadshow
License of the work: CC-BY-SA-4.0
Creator names: Syed M Shaaf, Philip Hayes

4.4. Knowledge File #4: Parasol Policies

Details

Here, we are adding information specific to policies in relation to the different products. This will help our claims processing agents to ask questions about specific cases and scenarios to the LLM. The LLM should be able to suggest remedies or further knowledge to look into.

Create a new folder parasol_policies under knowledge > economy > finance > insurance as shown in the image below

New folder

Copy the following and add it as a new file called qna.yaml in the folder parasol_policies as shown in the image above.

created_by: Philip Hayes
domain: insurance
seed_examples:
  - answer: Parasol will pay the daily rental charge incurred when an insured rents a car from a car business while your car or newly acquired car is not driveable or being repaired as a result of loss which would be payeable under comprehensive coverage.\n Parasol will pay this daily rental charge during the period that starts on the date the vehicle is not driveable as a result of the loss or the vehicle is left at a repair facility if the vehicle is driveable.\nPayment for car rental from Parasol insurance will end on the earliest of the date the vehicle has been repaired or replaced, the date Parasol insurance offers to pay for the loss of the vehicle if you choose to delay repairs, or seven days after Parasol insurance offers to pay for the loss if the vehicle is stolen and not recovered.\nThe amount of the car rental must be reported to Parasol insutance before we will pay such an amount.
    question: What is the car rental policy from Parasol Insurance?
  - answer: Parasol will pay the daily rental charge incurred when an insured rents a car from a car business while your car or newly acquired car is not driveable or being repaired as a result of loss which would be payeable under comprehensive coverage.
    question: When will Parasol Insurance pay daily rental charge incurred?
  - answer: Parasol will pay this daily rental charge during the period that starts on the date the vehicle is not driveable as a result of the loss or the vehicle is left at a repair facility if the vehicle is driveable.
    question: When will Parasol Insurance start paying daily car rental charges?
  - answer: Payment for car rental from Parasol insurance will end on the earliest of the date the vehicle has been repaired or replaced, the date Parasol insurance offers to pay for the loss of the vehicle if you choose to delay repairs, or seven days after Parasol insurance offers to pay for the loss if the vehicle is stolen and not recovered.
    question: When will payments for Car Insurance from Parasol Insurance end?
  - answer: The amount of the car rental must be reported to Parasol insutance before we will pay such an amount.
    question: What needs to happen before Parasol Insurance will pay car rental charges?
  - answer: If you are at fault, to be able to claim for your own lost wages from Parasol Insurance, you need to have the optional Apex plus package which provides coverage of loss of income insurance as part of your policy. If another driver is at fault, the at-fault driver’s auto insurance company is typically responsible for covering your lost income.
    question: Does Parasol insurance cover loss of income due to a car collision?
  - answer: The Parasol insrurance Apex plus package provides additional coverage for loss of income and uninsured driver insurance on top of their insurance policy.
    question: What is the Parasol insurance Apex plus package?


task_description: ""
document:
  repo: https://github.com/rh-rad-ai-roadshow/parasol_knowledge.git
  commit: b22cd9c
  patterns:
    - Parasol_auto_insurance.md

And now lets also create an attribution.txt file for citing sources. Copy the following and create a new file attribution.txt in the folder parasol_policies

Title of work: Parasol Insurance
Link to work: https://huggingface.co/rh-rad-ai-roadshow
License of the work: CC-BY-SA-4.0
Creator names: Syed M Shaaf, Philip Hayes

4.5. Check that the Taxonomy is Recognized by InstructLab

Now that we’ve added data, let’s check that the taxonomy is recognized by InstructLab. This will help us ensure that the data we’ve added is valid and can be used to generate synthetic training data.

We’re going to run some commands from the terminal, so from the Terminal menu, select New Terminal to open a new terminal window (or use the terminal you already have open).

Open new VSCode terminal

Let’s navigate to the instructlab directory:

cd ~/instructlab

Now, activate the Python virtual environment:

source venv/bin/activate

Run the following command to check the validity of the taxonomy:

ilab taxonomy diff

After running the above command you should be able to see the following output.

└─> ilab diff                                                                                                                                                                                                                                                                                    (base)
knowledge/economy/finance/insurance/parasol_claims_data/qna.yaml
knowledge/economy/finance/insurance/driving_age/qna.yaml
knowledge/economy/finance/insurance/parasol_insurance/qna.yaml
knowledge/economy/finance/insurance/parasol_policies/qna.yaml
Taxonomy in taxonomy is valid :)

If you do not see output similar to above, you may not have added in all of the Q&A files. This is important as the model will use these files to generate synthetic data in the next section.

5. Generating Synthetic Training Data & Training the New Model

Now that we’ve added some initial data, let’s use InstructLab to generate synthetic training data. Large Language Models inherently require a large amount of data to be effective, and it can be difficult to find enough real-world data for a niche domain. However, by using InstructLab, we can generate synthetic data that can be used to train the model.

5.1. Open a terminal in Visual Studio Code

Open new VSCode terminal

5.2. Run the following command to generate synthetic training data

cd ~/instructlab

5.3. Activate the Python virtual environment

source venv/bin/activate

5.4. Run the data generation command:

ilab data generate  --num-instructions 5

To reduce the amount of time the generation process takes, we are setting the "--num-instructions" flag to "5", the default for this value is "100". If we were generating data for a production deployment we would likely set this value even higher.

This process may take some time, depending on the amount of data and the computational resources available. Once the generation stage is complete, you should see something like this:

$ ilab data generate --num-instructions 5
Generating synthetic data using 'models/merlinite-7b-lab-Q4_K_M.gguf' model, taxonomy:'taxonomy' against http://127.0.0.1:60385/v1 server
Cannot find prompt.txt. Using default prompt depending on model-family.
  Synthesizing new instructions. If you aren't satisfied with the generated instructions, interrupt training (Ctrl-C) and try adjusting your YAML files. Adding more examples may help.
INFO 2024-09-09 15:32:37,674 generate_data.py:505: generate_data Selected taxonomy path knowledge->economy->finance->insurance->parasol_insurance
  INFO 2024-09-09 15:32:45,955 generate_data.py:505: generate_data Selected taxonomy path knowledge->economy->finance->insurance->driving_age

Q> How long must supervised driving practice be before applying for a license?
I>
A> 10 hours of nighttime driving are required.

Q> What happens to the license when a driver reaches 21 years old?
I>
A> It expires on the individual's 21st birthday, at which time they can obtain a horizontally-oriented driver’s license.

INFO 2024-09-09 15:33:15,263 generate_data.py:505: generate_data Selected taxonomy path knowledge->economy->finance->insurance->parasol_claims_data
Q> What does the engine type mean?
I>
A> It can refer to either a fuel source or type of engine that powers a vehicle.
This engine type can affect how powerful a car is,
or how much it costs to run it.

INFO 2024-09-09 15:33:34,454 generate_data.py:505: generate_data Selected taxonomy path knowledge->economy->finance->insurance->parasol_policies

Q> How long does it take to accelerate to 88 miles per hour in a DeLorean DMC-12?
I>
A> Approximately 8.8 seconds is needed to reach 88 mph in a Delorean DMC-12.

A> If you are at fault, to be able to claim for your own lost wages from Parasol Insurance, you need to have the optional Apex plus package which provides coverage of loss of income insurance as part of your policy. If another driver is at fault, the at-fault driver’s auto insurance company is typically responsible for covering your lost income. But if an at-fault driver is uninsured or doesn’t have enough coverage to pay for your injuries or car damage, Parasol underinsured motorist coverage can help pay the difference. Uninsured motorist coverage is provided as part of the optional Apex plus package.

Q> What is Bodily Injury Limit?
I>
A> It means how much a company will cover in case you or someone else was hurt, not including property damage.

100%|██████████████████████████████████████████████████| 5/5 [02:03<00:00, 24.77s/it]
INFO 2024-09-09 15:34:41,519 generate_data.py:609: generate_data 5 instructions generated, 8 discarded due to format (see generated/discarded_merlinite-7b-lab-Q4_K_M_2024-09-09T15_32_37.log), 0 discarded due to rouge score
INFO 2024-09-09 15:34:41,519 generate_data.py:613: generate_data Generation took 132.15s

We can examine the output of the generation stage by taking a look at the "generated" folder in ~/instructlab. Within this folder you’ll see 4 different files:

  • discarded_merlinite-7b-lab-Q4_K_M_DATETIME.log: This contains a log of any generated instructions which were discarded during the generation process for any reason e.g. "didn’t match expected format"

  • generated_merlinite-7b-lab-Q4_K_M_DATETIME.json: This contains the list of generated instructions plus context with which the model training instructions will be created

  • test_merlinite-7b-lab-Q4_K_M_DATETIME.jsonl: This file contains test prompts and responses which are used at the end of the training process

  • train_merlinite-7b-lab-Q4_K_M_DATETIME.jsonl: This file contains the instructions used during the train process to train the model.

5.5. Training the Model with New Data

With our synthetic data generated, we should now be in a position to train the model. Because we only created 5 sample instructions and due to time constraints we’re not going to perform the actual training in this lab.

If we were to do this, we would again use the "ilab" CLI with the "model train" command. Something like ilab model train --iters 10 --device cuda. Depending on the hardware available, this can take anywhere from several minutes to several hours or days. Once this process was finished we would then have a model we can serve locally with ilab to test our results.

5.6. Serve the new the Model

We have provisioned a trained model in the folder ~/instructlab/models called parasol-model.gguf. Serve this model by running the following command in the terminal:

ilab model serve --model-path ~/instructlab/models/parasol-model.gguf

It may take a minute to start, but you should see the following:

(venv) [instruct@bastion instructlab]$ ilab model serve --model-path ~/instructlab/models/parasol-model.gguf
INFO 2024-09-09 16:01:01,652 serve.py:51: serve Using model '/home/instruct/instructlab/models/parasol-model.gguf' with -1 gpu-layers and 4096 max context size.
INFO 2024-09-09 16:02:14,664 server.py:218: server Starting server process, press CTRL+C to shutdown server...
INFO 2024-09-09 16:02:14,664 server.py:219: server After application startup complete see http://127.0.0.1:8000/docs for API.

5.7. Interacting with the Model

To chat, open a new terminal using the + button above the terminal:

Launch Activities

In the new terminal, run the following command to begin a chat session with the model:

cd ~/instructlab
source venv/bin/activate
ilab chat

We can now ask the trained model some parasol specific questions such as:

  • Does Parasol insurance policies include loss of income cover if the insured driver is at fault?

  • Will Parasol insurance cover the cost of car rental if my car is undriveable as a result of an accident?

  • What is Apex plus from parasol insurance?

Launch Activities

The answers are specific to Parasol’s policies and demonstrate the ability for organizations to fine-tune models with their own data, to improve the accuracy of responses, which can be used in many use cases.

6. Conclusion

This exercise showed how organizations can leverage fine tuning with InstructLab to improve the accuracy of LLM responses. Here’s a quick summary of what we have learned:

  • You Learned about fine-tuning techniques for AI models, incorporating Parasol’s unique insurance expertise into LLMs

  • Your also learned how to create and utilize a custom knowledge base for training AI models with organization-specific scenarios and regulations

  • Apply the LAB methodology to generate synthetic training data and specialize our model while keeping data in-house

  • You gained hands-on experience in training and serving a customized AI model

  • You understand the benefits and limitations of fine-tuning compared to other AI customization methods

These skills and tools will be invaluable as you continue to develop AI-enabled applications at Parasol Insurance, allowing you to quickly iterate on ideas and integrate powerful AI capabilities into your workflow.

7. Appendix

These commands install VSCode and a few other needed tools. You should not have to run these, it’s only here for reference.

curl -L -o files/vscode.rpm 'https://code.visualstudio.com/sha/download?build=stable&os=linux-rpm-x64'
sudo dnf install -y xdg-utils
sudo rpm -ivh files/vscode.rpm
sudo dnf -y install zip gcc glibc-devel zlib-devel firefox
curl -s "https://get.sdkman.io" | bash
source "/home/instruct/.sdkman/bin/sdkman-init.sh"
sdk install java 21.0.3-tem
pip install 'numpy<2.0'

cat <<EOF >> /home/instruct/instructlab/config.yaml
chat:
  context: default
  greedy_mode: false
  logs_dir: data/chatlogs
  max_tokens: null
  model: models/merlinite-7b-lab-Q4_K_M.gguf
  session: null
  vi_mode: false
  visible_overflow: true
general:
  log_level: INFO
generate:
  chunk_word_count: 1000
  model: models/merlinite-7b-lab-Q4_K_M.gguf
  num_cpus: 10
  num_instructions: 100
  output_dir: generated
  prompt_file: prompt.txt
  seed_file: seed_tasks.json
  taxonomy_base: origin/main
  taxonomy_path: taxonomy
serve:
  gpu_layers: -1
  host_port: 127.0.0.1:8000
  max_ctx_size: 4096
  model_path: models/merlinite-7b-lab-Q4_K_M.gguf
EOF

curl -Lo ~/instructlab/models/parasol-model.gguf <a href="https://huggingface.co/rh-rad-ai-roadshow/parasol-merlanite-trained-GGUF/resolve/main/parasol-model-0715-sdg400.gguf?download=true" class="bare">https://huggingface.co/rh-rad-ai-roadshow/parasol-merlanite-trained-GGUF/resolve/main/parasol-model-0715-sdg400.gguf?download=true</a>


curl -Lo ~/Documents/marty-mcfly-auto.pdf <a href="https://raw.githubusercontent.com/rh-rad-ai-roadshow/parasol-insurance/main/app/src/main/resources/claims/marty-mcfly-auto.pdf" class="bare">https://raw.githubusercontent.com/rh-rad-ai-roadshow/parasol-insurance/main/app/src/main/resources/claims/marty-mcfly-auto.pdf</a>

cd ~
git clone <a href="https://github.com/rh-rad-ai-roadshow/parasol-insurance.git" class="bare">https://github.com/rh-rad-ai-roadshow/parasol-insurance.git</a>
cd parasol-insurance/app
./mvnw clean package -DskipTests
nohup java -jar -Dquarkus.langchain4j.openai.parasol-chat.base-url=http://localhost:8000/v1 target/quarkus-app/quarkus-run.jar > ~/quarkus.out 2>&1 &