# AI use cases for LocalAI (mudler/LocalAI) Repo: [mudler/LocalAI](https://github.com/mudler/LocalAI) | Use cases: 3 | Page: https://dedupedai.com/use-cases/mudler-localai/ LocalAI is an open-source AI engine that runs LLMs, vision, voice, image and video models on any hardware, with no GPU required. Each backend (llama.cpp, vLLM, whisper.cpp, stable-diffusion, MLX and more) is pulled only when a model needs it, and every backend sits behind drop-in OpenAI, Anthropic and ElevenLabs compatible APIs, with multi-user auth, quotas and built-in agents with tools, RAG and MCP. This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL. - Self-host an OpenAI-compatible API on your own server: https://dedupedai.com/use-cases/mudler-localai/self-host-an-openai-compatible-api/ - Chat with local models from the terminal: https://dedupedai.com/use-cases/mudler-localai/chat-with-local-models-in-the-terminal/ - Run any model from Hugging Face, Ollama or an OCI registry: https://dedupedai.com/use-cases/mudler-localai/run-models-from-hugging-face-or-ollama/ ## How to self-host an OpenAI-compatible API on your own server with LocalAI Repo: [mudler/LocalAI](https://github.com/mudler/LocalAI) | Difficulty: Beginner | Time: 30 minutes | Basis: From the README Run LocalAI in Docker, load a model and point your existing OpenAI client code at it. **Who it is for:** A developer or small team that wants to stop sending prompts and data to a cloud API. **The problem:** Cloud AI APIs cost money per call and your data leaves your infrastructure. **The outcome:** A local endpoint on port 8080 that existing OpenAI or Anthropic clients can call, running on CPU or GPU. ### Steps 1. Start the CPU image: `docker run -ti --name local-ai -p 8080:8080 localai/localai:latest` 2. Or with an NVIDIA GPU: `docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12` 3. Load a model, for example `local-ai run llama-3.2-1b-instruct:q4_k_m` 4. Point your app's OpenAI base URL at http://localhost:8080 5. Restart later with `docker start -i local-ai` ### Prompt to paste into your agent ```text Set up mudler/LocalAI in Docker on this machine (GPU image if an NVIDIA GPU is present, CPU image otherwise), load a small instruct model, and change 's OpenAI client to use it. Run one test request and report latency. ``` ### Caveat Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/mudler-localai/self-host-an-openai-compatible-api/ ## How to chat with local models from the terminal with LocalAI Repo: [mudler/LocalAI](https://github.com/mudler/LocalAI) | Difficulty: Beginner | Time: 20 minutes | Basis: From the README Run a model in one terminal and chat with it in another, switching models inside the session. **Who it is for:** A developer who wants quick private answers without opening a browser or paying per token. **The problem:** Trying different local models usually means juggling several tools and configs. **The outcome:** A terminal chat against any installed model, with commands to list and switch models. ### Steps 1. Install LocalAI (Docker or the macOS app) 2. In terminal 1 run `local-ai run llama-3.2-1b-instruct:q4_k_m` 3. In terminal 2 run `local-ai chat --model llama-3.2-1b-instruct:q4_k_m` 4. Inside the session use /models to list installed models and /model to switch 5. Measure latency and throughput with `local-ai benchmark` ### Prompt to paste into your agent ```text Using mudler/LocalAI, run two different small models, chat with each using local-ai chat on the same three questions, run local-ai benchmark, and recommend which model to keep for . ``` ### Caveat Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/mudler-localai/chat-with-local-models-in-the-terminal/ ## How to run any model from Hugging Face, Ollama or an OCI registry with LocalAI Repo: [mudler/LocalAI](https://github.com/mudler/LocalAI) | Difficulty: Intermediate | Time: 20 minutes | Basis: From the README Load models straight from Hugging Face, Ollama or OCI references with one command. **Who it is for:** A tinkerer who wants to try a specific GGUF or Ollama model through one consistent API. **The problem:** Each model source has its own tooling and serving setup. **The outcome:** Models from different sources running behind the same LocalAI API. ### Steps 1. Install LocalAI 2. Run a Hugging Face file: `local-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf` 3. Run an Ollama model: `local-ai run ollama://gemma:2b` 4. Or an OCI image: `local-ai run oci://localai/phi-2:latest` 5. List what is installed with `local-ai models list` ### Prompt to paste into your agent ```text With mudler/LocalAI, load from , confirm it appears in local-ai models list, and call it through the OpenAI-compatible endpoint with a test prompt. ``` ### Caveat Check each model's license before using it in a product. Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/ URL: https://dedupedai.com/use-cases/mudler-localai/run-models-from-hugging-face-or-ollama/