DEDUPED Hot AI repos, turned into use cases Get 5 a day

How to run decisions offline on a Mac with MLX with LLM2 Jev

Score choices in Python on Apple Silicon without a server, using the MLX backend.

Who it is forA developer on an Apple Silicon Mac who wants local decisions without an NVIDIA GPU.

The problemMany local decision setups assume an NVIDIA GPU and a running server.

The outcomeChoice or Score results from an offline Python call on the MLX backend.

What you need first

  • A local model directory supported by SGLang, Transformers, or MLX (Apple Silicon)
  • For the SGLang path: Linux with a supported NVIDIA GPU
  • Python with uv or pip, as described in the Installation guide
  • A text or image input to decide on

Steps

  1. git clone https://github.com/Yinsongxu/LLM2Jev.git
  2. cd LLM2Jev
  3. Install the MLX dependencies using the Installation guide in docs/en/installation.md, then uv sync in the repo (inferred, check the README)
  4. source .venv/bin/activate
  5. Open the Offline: Python API section of docs/en/usage.md and run its example with your model path (inferred, check the README)

Prompt for your agent

Using the LLM2Jev repo on Apple Silicon, run the offline Python API with the MLX backend on the model at <model-path>, asking a Choice question over <options> for the input <input-text>. Return the option probabilities as JSON. Verify that the probabilities sum to about 1 before reporting.

Caveat. The README names MLX for Apple Silicon, but the exact MLX install steps and model requirements are only in the linked Installation and Usage guides.

Share: X LinkedIn Hacker News Email Copy link

Disclaimer

Deduped lists open-source repositories that belong to other people. We do not own, maintain, audit, or vouch for any of them, and a repo appearing here is not an endorsement or a safety check. The use cases, steps, and prompts are written by AI and may be incomplete, outdated, or just wrong. Where a use case is marked "Seen in the wild", we link to public posts where someone described something similar, but we have not verified those posts and the use case itself is still our AI's own write-up. Use all of this at your own risk. Only run code, commands, or agents on systems and accounts you are authorized to use, and read the code and every command before you run it, especially anything that touches credentials, files, or the network. Nothing here is professional, security, legal, or financial advice. All trademarks and project names belong to their respective owners, and their use here does not imply affiliation. Content is provided as is, without warranties of any kind. Found a mistake, a safety problem, or something you want removed? Email [email protected] and we will look at it promptly. Full disclaimer page