# AI use cases for Whallm (yanun0323/Whallm) Repo: [yanun0323/Whallm](https://github.com/yanun0323/Whallm) | Use cases: 4 | Page: https://dedupedai.com/use-cases/yanun0323-whallm/ Whallm is a macOS app that runs large language models on Apple Silicon Macs by keeping common weights in memory and reading experts from SSD, with a chat interface and an OpenAI-compatible API server. This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL. - Point Codex at a local model on your Mac: https://dedupedai.com/use-cases/yanun0323-whallm/local-codex-backend-on-mac/ - Chat with a large model privately on your Mac: https://dedupedai.com/use-cases/yanun0323-whallm/private-chat-with-large-model/ - Measure which local model runs well on your Mac: https://dedupedai.com/use-cases/yanun0323-whallm/measure-model-speed-on-your-mac/ - Call the local model from a script through its API: https://dedupedai.com/use-cases/yanun0323-whallm/script-local-model-via-openai-api/ ## How to point Codex at a local model on your Mac with Whallm Repo: [yanun0323/Whallm](https://github.com/yanun0323/Whallm) | Difficulty: Intermediate | Time: 3 hours | Basis: From the README Run Whallm's server on your Mac and send Codex requests to it instead of a hosted API. **Who it is for:** A developer on an Apple Silicon Mac who wants Codex to use a model that runs on their own machine, for example to keep code prompts local. **The problem:** Codex is set up to call a hosted model, and the developer wants inference to stay on the Mac while still using the Codex client. **The outcome:** Codex sends its requests to Whallm at http://127.0.0.1:11434/v1 and receives answers from a model loaded on the Mac. ### Steps 1. Download Whallm-macOS-arm64.zip from GitHub Releases, unzip it and open Whallm.app. 2. Open Model, pick a model and select Download Model. Wait for the download to finish, resuming it if it stops. 3. Open Server and select Start Server. The default address is http://127.0.0.1:11434. 4. Add the provider block from the README to ~/.codex/config.toml, setting model to an API model ID such as deepseek-v4-flash-0731 or an Alias you set in Model to Advanced Settings. 5. Restart Codex and send a short test prompt. ### Prompt to paste into your agent ```text Work in the Whallm repo at yanun0323/Whallm. Confirm the Whallm server is running at http://127.0.0.1:11434 and that GET /v1/models lists the model ID . Then check that ~/.codex/config.toml sets model_provider to deepseek-v4-ssd with base_url http://127.0.0.1:11434/v1 and wire_api set to responses, and report whether the file matches the README example. ``` ### Caveat The first model download is very large, and the server keeps one model loaded and handles one request at a time, so Codex may wait on long prompts. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/yanun0323-whallm/local-codex-backend-on-mac/ ## How to chat with a large model privately on your Mac with Whallm Repo: [yanun0323/Whallm](https://github.com/yanun0323/Whallm) | Difficulty: Beginner | Time: 2 hours | Basis: From the README Use Whallm's built-in Chat window to ask a large local model questions without a hosted service. **Who it is for:** A non-developer Mac owner who wants to chat with a large language model and keep their prompts on their own computer. **The problem:** Hosted chat tools send prompts to a company's servers, and the user wants a local option that does not require writing code. **The outcome:** The user gets answers in the Chat window from a model that runs on the Mac, including image input for Qwen models. ### Steps 1. Download Whallm-macOS-arm64.zip from GitHub Releases, unzip it and open Whallm.app. 2. Open Model, pick Qwen3.8 or another model that fits your memory and disk space, and select Download Model. 3. Open Server and select Start Server. 4. Open Chat, pick your model and type your first question. The model loads the first time you use it. ### Prompt to paste into your agent ```text In the Whallm repo at yanun0323/Whallm, check the README's Supported models table and the Requirements section against this Mac's chip, macOS version, memory and free disk space for . Report whether the model fits and which steps from the Quick start remain. ``` ### Caveat Whallm recommends 64 GiB of memory, and the README says actual use depends on the model and settings, so a smaller Mac may be slow or unable to run the larger models. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/yanun0323-whallm/private-chat-with-large-model/ ## How to measure which local model runs well on your Mac with Whallm Repo: [yanun0323/Whallm](https://github.com/yanun0323/Whallm) | Difficulty: Intermediate | Time: 2 hours | Basis: From the README Use the Throughput tool to test prefill and decode speed on your own hardware before committing to a large download. **Who it is for:** A Mac owner deciding between Whallm models or cache settings who wants real numbers for their own chip and SSD. **The problem:** Benchmark numbers from other machines do not show how a given model, cache mode and SSD will perform on this Mac. **The outcome:** The user has prefill and decode speeds and peak memory for their own Mac, to compare against the README benchmark table. ### Steps 1. Install Whallm and download a model from the Model page, as described in the Quick start. 2. Open Model and check Advanced Settings for the cache mode and slot count you want to test. 3. Open the Throughput tool in the app and run it on your Mac using the 16K-token input and 512-token answer from the README benchmark, if the tool allows it. 4. Compare your prefill tok/s, decode tok/s and peak memory with the BENCHMARK.md results and note the cache mode used. ### Prompt to paste into your agent ```text In the yanun0323/Whallm repo, read the Benchmark summary and the BENCHMARK.md file and list the metrics and settings a user must record to compare a run with the published results. Output a short checklist with the chip, cache mode, slots, prefill, decode and peak memory, and note that the README calls these numbers reference only. ``` ### Caveat The README says the published benchmarks are user-provided and not a controlled comparison, and peak memory is process memory rather than total Mac memory. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/yanun0323-whallm/measure-model-speed-on-your-mac/ ## How to call the local model from a script through its API with Whallm Repo: [yanun0323/Whallm](https://github.com/yanun0323/Whallm) | Difficulty: Advanced | Time: 2 hours | Basis: From the README Send requests from your own code to Whallm's OpenAI-compatible endpoint running on your Mac. **Who it is for:** A developer or automation builder who wants a script or agent to call a local model instead of a hosted API. **The problem:** A script needs a model endpoint that is free of per-request hosting and keeps data on the local machine. **The outcome:** A script that lists available model IDs and sends a chat request to the local server, returning the model's answer. ### Steps 1. Start Whallm and select Start Server in the Server page. The default address is http://127.0.0.1:11434. 2. Request GET /v1/models from http://127.0.0.1:11434 to list the API model IDs. 3. Use the model ID from that list in your script, not the display name, and set base_url to http://127.0.0.1:11434/v1. 4. If you set an API key in Whallm, send the same key from your script; the README example uses no key. ### Prompt to paste into your agent ```text In the yanun0323/Whallm repo, read the README sections on the API and Connect Codex, then write a short script that calls GET /v1/models at http://127.0.0.1:11434 and prints each API model ID. Run it against a running Whallm server, and verify that the printed IDs match the model ID table in docs/usage.md. ``` ### Caveat The server handles one request at a time with one loaded model, so concurrent requests from a script will queue, and the README says tool calls are executed by the client, not by Whallm. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/yanun0323-whallm/script-local-model-via-openai-api/