# AI use cases for Open EBFM (datamllab/OpenEBFM)

Repo: [datamllab/OpenEBFM](https://github.com/datamllab/OpenEBFM) | Use cases: 4 | Page: https://dedupedai.com/use-cases/datamllab-openebfm/

OpenEBFM is open-source Python code for training, scaling, evaluating, and chatting with energy-based foundation language models.

This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL.

- Pretrain an EBFM model to a chosen token budget: https://dedupedai.com/use-cases/datamllab-openebfm/pretrain-ebfm-at-token-budget/
- Chat with the published EBFM SFT model in a browser: https://dedupedai.com/use-cases/datamllab-openebfm/chat-with-published-sft-model/
- Fine-tune a published EBFM checkpoint with SFT: https://dedupedai.com/use-cases/datamllab-openebfm/fine-tune-published-checkpoint-with-sft/
- Measure BPB and QA scores for an EBFM checkpoint: https://dedupedai.com/use-cases/datamllab-openebfm/evaluate-checkpoint-bpb-and-qa/

## How to pretrain an EBFM model to a chosen token budget with Open EBFM

Repo: [datamllab/OpenEBFM](https://github.com/datamllab/OpenEBFM) | Difficulty: Advanced | Time: 3 hours | Basis: From the README

Prepare DCLM data and run a full pretraining job with a set token count.

**Who it is for:** An ML researcher who wants to reproduce or extend EBFM pretraining on a GPU machine and needs a working end-to-end run.

**The problem:** Setting up energy-based language model pretraining from scratch involves dependencies, data preparation, and a training budget that must be set correctly.

**The outcome:** A pretraining run that writes checkpoints, a clean Hugging Face export, logs, and results under the standard outputs layout.

### Steps

1. Run `uv sync --extra gpu --extra hf --extra web` to install the GPU, Hugging Face, and web-chat dependencies.
2. Run `source .venv/bin/activate` to activate the environment.
3. Run `bash runs/prepare_data.sh --datasets dclm` on a machine with network access to download and tokenize DCLM.
4. Run `TARGET_TOTAL_TOKENS=100000000 bash runs/train.sh` to start pretraining with a 100M token budget.
5. Check `outputs/checkpoints_hf/` for the exported model containing config.json and model.safetensors.

### Prompt to paste into your agent

```text
Using the OpenEBFM repo (datamllab/OpenEBFM), prepare the dclm dataset and run pretraining with TARGET_TOTAL_TOKENS set to <token_budget> via runs/train.sh. Report the path of the final checkpoints_hf/ export and confirm it contains config.json and model.safetensors.
```

### Caveat

Data preparation needs network access and training must run where the prepared data is available, and the README does not state hardware needs or run duration.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/datamllab-openebfm/pretrain-ebfm-at-token-budget/

## How to chat with the published EBFM SFT model in a browser with Open EBFM

Repo: [datamllab/OpenEBFM](https://github.com/datamllab/OpenEBFM) | Difficulty: Intermediate | Time: 1 hour | Basis: From the README

Serve the EBFM-FineWeb-1B-CHAT model through a local web chat interface.

**Who it is for:** A researcher or hobbyist with a GPU server who wants to try a published energy-based language model interactively.

**The problem:** Trying a new language model usually means writing inference code, and it is not obvious how to serve it to a browser.

**The outcome:** A running web chat at port 8000 that talks to the published SFT checkpoint.

### Steps

1. Run `uv sync --extra gpu --extra hf --extra web` and `source .venv/bin/activate` to set up the environment.
2. Run `bash runs/chat.sh --web --checkpoint guan-wang/EBFM-FineWeb-1B-CHAT --host 0.0.0.0 --port 8000` to start the server. (inferred, check the README)
3. Open http://localhost:8000 in a browser on the server machine.
4. If the server is a remote GPU node, use your cluster's port forwarding to expose port 8000 on your local machine.

### Prompt to paste into your agent

```text
In the datamllab/OpenEBFM repo, start the web chat with runs/chat.sh using guan-wang/EBFM-FineWeb-1B-CHAT on host 0.0.0.0 and port <port>. Report the URL to open and confirm the server responds on that port.
```

### Caveat

The README says the chat runs on a GPU and the published checkpoint is downloaded on first use, so the first launch depends on network speed and GPU availability.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/datamllab-openebfm/chat-with-published-sft-model/

## How to fine-tune a published EBFM checkpoint with SFT with Open EBFM

Repo: [datamllab/OpenEBFM](https://github.com/datamllab/OpenEBFM) | Difficulty: Advanced | Time: 2 hours | Basis: From the README

Start supervised fine-tuning from a Hugging Face EBFM model using the SFT launcher.

**Who it is for:** An ML practitioner who wants to adapt a published EBFM base model to a conversation or instruction dataset.

**The problem:** Adapting a pretrained base model requires the right entry point, checkpoint reference, and dataset name.

**The outcome:** A supervised fine-tuning run that starts from the chosen Hugging Face model and writes outputs to the standard layout.

### Steps

1. Run `uv sync --extra gpu --extra hf --extra web` and `source .venv/bin/activate` to set up the environment.
2. Run `bash runs/prepare_data.sh --datasets dclm` or otherwise make sure your SFT data is available (the README says data is not included in the repository).
3. Run `TARGET_TOTAL_TOKENS=100000000 bash runs/sft.sh --dataset_name ebfm_sft --finetuning_model_ckpt guan-wang/EBFM-DCLM-1B` to start fine-tuning. (inferred, check the README)
4. Check `outputs/checkpoints_hf/` after the run for the exported fine-tuned model.

### Prompt to paste into your agent

```text
In the datamllab/OpenEBFM repo, run runs/sft.sh with --finetuning_model_ckpt set to <hf_model_id> and --dataset_name set to <dataset_name>. Report the output directory and confirm checkpoints_hf/ was created.
```

### Caveat

The README names ebfm_sft as an example dataset name but does not describe how to prepare that dataset, so the data setup step may need extra work.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/datamllab-openebfm/fine-tune-published-checkpoint-with-sft/

## How to measure BPB and QA scores for an EBFM checkpoint with Open EBFM

Repo: [datamllab/OpenEBFM](https://github.com/datamllab/OpenEBFM) | Difficulty: Intermediate | Time: 2 hours | Basis: From the README

Run position-wise BPB and QA evaluation on a published model and collect results.

**Who it is for:** A researcher comparing EBFM checkpoints who needs consistent benchmark numbers.

**The problem:** Comparing language models needs the same metrics computed the same way across checkpoints.

**The outcome:** Position-wise BPB and QA result files written under results/ for the chosen checkpoint.

### Steps

1. Run `uv sync --extra gpu --extra hf --extra web` and `source .venv/bin/activate` to set up the environment.
2. Download the local snapshot with `MODEL_DIR="$(python -c 'from huggingface_hub import snapshot_download; print(snapshot_download(repo_id="guan-wang/EBFM-DCLM-1B"))')"` so the BPB metric has token_bytes.pt.
3. Run `bash runs/eval.sh --checkpoint guan-wang/EBFM-DCLM-1B --dataset dclm --data-dir data/dclm --tokenizer-path "${MODEL_DIR}" --output_root . --run_name eval-dclm` for BPB. (inferred, check the README)
4. Run `bash runs/qa.sh --checkpoint guan-wang/EBFM-DCLM-1B --tokenizer-path "${MODEL_DIR}" --eval-bundle data/eval_bundle --output_root . --run_name qa-dclm` for QA. (inferred, check the README)

### Prompt to paste into your agent

```text
In the datamllab/OpenEBFM repo, run runs/eval.sh for BPB and runs/qa.sh for QA on the checkpoint <hf_model_id>, using the local snapshot directory for the tokenizer path. Report the results/ paths and confirm token_bytes.pt was found in the snapshot.
```

### Caveat

BPB evaluation needs the local snapshot directory, and the eval bundle at data/eval_bundle must be prepared separately because the README says data is not included.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/datamllab-openebfm/evaluate-checkpoint-bpb-and-qa/