DEDUPED Hot AI repos, turned into use cases Get 5 a day

How to get typed LLM decisions from text on your own machine with Rizzo Flow

Run a local decision API and get yes/no, choice and score answers as JSON with probabilities.

Who it is forA developer who wants an LLM to classify or score messages, tickets or records, and does not want the data to leave their machine.

The problemLLM answers are free text that must be parsed, and hosted decision APIs send your data to a third party.

The outcomeA running local server at http://127.0.0.1:8017 that returns typed decisions with probabilities for any state you send.

What you need first

  • Python 3.11 or newer
  • git
  • uv (https://docs.astral.sh/uv/)
  • About 4.4 GB of free disk space for the default 4B model, plus the llama.cpp runtime (about 11 MB on a Mac, 30 MB with Vulkan, about 570 MB with CUDA)
  • A machine that can run llama.cpp: an Apple, NVIDIA, AMD or Intel GPU, or no GPU at all

Steps

  1. Install Python 3.11 or newer, git and uv.
  2. Run git clone https://github.com/Rizzo-AI-Academy/rizzo-flow && cd rizzo-flow
  3. Run uv sync --locked
  4. Run uv run rizzo download and wait for the model and runtime to finish.
  5. Run uv run rizzo serve, then open http://127.0.0.1:8017/playground to try a state with questions.
  6. Send the curl example from the Quickstart to http://127.0.0.1:8017/v1/systemone with your own state and questions.

Prompt for your agent

In the rizzo-flow repo, start the server with `uv run rizzo serve`, then POST a JSON body with the state from <path-to-text-file> and one yes/no question to http://127.0.0.1:8017/v1/systemone using model rizzo-latest. Return the typed decision and its probabilities. Verify that the response contains a probability for each option.

Caveat. Probabilities are uncalibrated unless you calibrate them on your own data, and the fine-tuned model is still below TypeSafe Jev in accuracy (0.648 against 0.727 on the typed-decisions benchmark in the README).

Share: X LinkedIn Hacker News Email Copy link

Disclaimer

Deduped lists open-source repositories that belong to other people. We do not own, maintain, audit, or vouch for any of them, and a repo appearing here is not an endorsement or a safety check. The use cases, steps, and prompts are written by AI and may be incomplete, outdated, or just wrong. Where a use case is marked "Seen in the wild", we link to public posts where someone described something similar, but we have not verified those posts and the use case itself is still our AI's own write-up. Use all of this at your own risk. Only run code, commands, or agents on systems and accounts you are authorized to use, and read the code and every command before you run it, especially anything that touches credentials, files, or the network. Nothing here is professional, security, legal, or financial advice. All trademarks and project names belong to their respective owners, and their use here does not imply affiliation. Content is provided as is, without warranties of any kind. Found a mistake, a safety problem, or something you want removed? Email [email protected] and we will look at it promptly. Full disclaimer page