# AI use cases for Litgpt (Lightning-AI/litgpt)

Repo: [Lightning-AI/litgpt](https://github.com/Lightning-AI/litgpt) | Use cases: 3 | Page: https://dedupedai.com/use-cases/lightning-ai-litgpt/

LitGPT is a Python library and command line tool to pretrain, finetune, evaluate, chat with and serve 20+ open LLMs. One `litgpt` command covers each action and downloads model weights on first use.

This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL.

- Finetune a small LLM on your own JSON dataset: https://dedupedai.com/use-cases/lightning-ai-litgpt/finetune-a-small-llm-on-your-own-json-data-with-litgpt/
- Serve an open LLM as a local HTTP API: https://dedupedai.com/use-cases/lightning-ai-litgpt/serve-an-open-llm-as-a-local-http-api-with-litgpt/
- Benchmark a model on MMLU and TruthfulQA: https://dedupedai.com/use-cases/lightning-ai-litgpt/benchmark-a-model-on-mmlu-and-truthfulqa-with-litgpt/

## How to finetune a small LLM on your own JSON dataset with Litgpt

Repo: [Lightning-AI/litgpt](https://github.com/Lightning-AI/litgpt) | Difficulty: Intermediate | Time: 1 to 3 hours | Basis: From the README

Use the Litgpt finetune command to train microsoft/phi-2 on a JSON file of examples, then chat with and serve the result.

**Who it is for:** A developer with a few thousand question and answer pairs from their own domain.

**The problem:** Generic models answer domain questions vaguely, and writing a training loop takes days.

**The outcome:** A finetuned checkpoint in out/custom-model/final that you can chat with and serve.

### Steps

1. Install with `pip install 'litgpt[extra]'`
2. Download the example dataset with `curl -L https://huggingface.co/datasets/ksaw008/finance_alpaca/resolve/main/finance_alpaca.json -o my_custom_dataset.json` or point to your own JSON file
3. Run `litgpt finetune microsoft/phi-2` with `--data JSON`, `--data.json_path my_custom_dataset.json`, `--data.val_split_fraction 0.1` and `--out_dir out/custom-model`
4. Test the result with `litgpt chat out/custom-model/final`
5. Deploy it with `litgpt serve out/custom-model/final`

### Prompt to paste into your agent

```text
Using Lightning-AI/litgpt, install it with `pip install 'litgpt[extra]'`, finetune microsoft/phi-2 on <my JSON dataset> with `litgpt finetune microsoft/phi-2 --data JSON --data.json_path my_custom_dataset.json --data.val_split_fraction 0.1 --out_dir out/custom-model`, then run `litgpt chat out/custom-model/final` and report five sample answers.
```

### Caveat

Finetuning needs a GPU; training time depends on the dataset size and the card.

Risk, download: The first run downloads the base model weights, which can be several gigabytes.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/lightning-ai-litgpt/finetune-a-small-llm-on-your-own-json-data-with-litgpt/

## How to serve an open LLM as a local HTTP API with Litgpt

Repo: [Lightning-AI/litgpt](https://github.com/Lightning-AI/litgpt) | Difficulty: Beginner | Time: 20 minutes | Basis: From the README

Start Litgpt serve on an off the shelf model and call its /predict endpoint from Python.

**Who it is for:** A developer who wants a private model endpoint for an internal tool.

**The problem:** Hosted APIs send data off the machine and cost per call.

**The outcome:** A model server on 127.0.0.1:8000 that answers prompts posted to /predict.

### Steps

1. Install with `pip install 'litgpt[extra]'`
2. List supported models with `litgpt download list`
3. Start the server with `litgpt serve microsoft/phi-2`
4. From a separate Python session, POST a JSON body with a prompt to http://127.0.0.1:8000/predict using requests
5. Read the reply from the output field of the JSON response

### Prompt to paste into your agent

```text
With Lightning-AI/litgpt, run `litgpt serve microsoft/phi-2`, then write a small Python script that posts {"prompt": "Fix typos in the following sentence: Example input"} to http://127.0.0.1:8000/predict and prints the output field.
```

### Caveat

The server listens on localhost only; add your own auth before exposing it.

Risk, download: Model weights download on first use.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/lightning-ai-litgpt/serve-an-open-llm-as-a-local-http-api-with-litgpt/

## How to benchmark a model on MMLU and TruthfulQA with Litgpt

Repo: [Lightning-AI/litgpt](https://github.com/Lightning-AI/litgpt) | Difficulty: Intermediate | Time: 1 hour | Basis: From the README

Run Litgpt evaluate with the truthfulqa_mc2 and mmlu tasks to score a base or finetuned model.

**Who it is for:** An ML engineer comparing a finetuned model with its base model.

**The problem:** Without a standard benchmark it is hard to tell if finetuning helped or hurt.

**The outcome:** Benchmark scores for the tasks you chose, ready to compare across checkpoints.

### Steps

1. Install with `pip install 'litgpt[extra]'`
2. Run `litgpt evaluate microsoft/phi-2 --tasks 'truthfulqa_mc2,mmlu'` for the base model
3. Run the same evaluate command on your finetuned checkpoint folder
4. Compare the scores per task

### Prompt to paste into your agent

```text
Using Lightning-AI/litgpt, run `litgpt evaluate microsoft/phi-2 --tasks 'truthfulqa_mc2,mmlu'` and the same command on <my checkpoint folder>, then put both sets of scores in one table.
```

### Caveat

Full MMLU runs take a long time on small GPUs.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/lightning-ai-litgpt/benchmark-a-model-on-mmlu-and-truthfulqa-with-litgpt/
