# AI use cases for Respan Gateway (respanai/respan-gateway)

Repo: [respanai/respan-gateway](https://github.com/respanai/respan-gateway) | Use cases: 5 | Page: https://dedupedai.com/use-cases/respanai-respan-gateway/

Respan Gateway is a single Rust binary that sits between your code and AI vendor APIs, letting any supported SDK call many models and tools through one endpoint.

This file holds every use case of this repo. Each one starts with a "## " heading and ends with its URL.

- Call any model from the SDK you already use: https://dedupedai.com/use-cases/respanai-respan-gateway/call-any-model-from-your-existing-sdk/
- Route one model name across providers with fallback: https://dedupedai.com/use-cases/respanai-respan-gateway/add-fallback-and-hedged-routes/
- Give any chat model web search and MCP tools: https://dedupedai.com/use-cases/respanai-respan-gateway/give-any-model-web-search-and-mcp-tools/
- Trace every model call and read cost and latency: https://dedupedai.com/use-cases/respanai-respan-gateway/trace-every-model-call-in-respan/
- Write a custom guardrail or PII masking plugin: https://dedupedai.com/use-cases/respanai-respan-gateway/build-a-guardrail-or-pii-plugin/

## How to call any model from the SDK you already use with Respan Gateway

Repo: [respanai/respan-gateway](https://github.com/respanai/respan-gateway) | Difficulty: Intermediate | Time: 15 minutes | Basis: From the README

Point an existing OpenAI, Anthropic or Gemini SDK at the gateway to reach other vendors' models.

**Who it is for:** A developer whose app is built on one vendor's SDK and who wants to try or switch to a model from another vendor without rewriting the calls.

**The problem:** Each vendor has its own API format, so switching models usually means changing client code.

**The outcome:** The existing SDK keeps working unchanged and reaches a model from another vendor through the gateway's local endpoint.

### Steps

1. Install the gateway with `pip install respan-gateway`.
2. Set a vendor key in your shell with `export OPENAI_API_KEY=sk-...`.
3. Start the gateway by running `respan-gateway`.
4. In your app, create the client with base_url set to http://localhost:8080 and call a model such as gpt-6-luna, as in the README example.
5. Open http://localhost:8080/ui/ to confirm the request appears in the live log.

### Prompt to paste into your agent

```text
In the respan-gateway repo, start the gateway with the OPENAI_API_KEY from the environment, then send one chat request through http://localhost:8080 using the Anthropic SDK with model <target-model>. Return the reply text, and verify that the request appears in the live log at localhost:8080/ui/.
```

### Caveat

The gateway answers only on loopback unless client keys are configured, and it refuses to start on a public address without them.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/respanai-respan-gateway/call-any-model-from-your-existing-sdk/

## How to route one model name across providers with fallback with Respan Gateway

Repo: [respanai/respan-gateway](https://github.com/respanai/respan-gateway) | Difficulty: Intermediate | Time: 30 minutes | Basis: From the README

Define a named route that falls back to a second vendor and races a slow call.

**Who it is for:** A backend engineer whose service fails or slows down when one model provider has an outage or latency spike.

**The problem:** A single provider outage or slow response breaks requests or hurts tail latency.

**The outcome:** Clients call one route name, and the gateway switches or hedges to another provider automatically.

### Steps

1. Create a starter config with `respan-gateway init`.
2. Add a [routes.<name>] table whose `to` list names two providers and whose `hedge` value sets the delay, following the README example.
3. Validate the file with `respan-gateway check`.
4. Reload the config with --watch or SIGHUP, as the README describes, without restarting the gateway.
5. Send a request with "model" set to your route name and confirm which provider answered in the live log.

### Prompt to paste into your agent

```text
In the respan-gateway repo, edit gateway.toml to add a route that lists two providers in `to` with a hedge delay, then run the check command and send one request using the route name <route-name>. Return which provider served the reply, and verify the check command reports no errors.
```

### Caveat

The README example pairs an Anthropic and an OpenAI model, so each provider needs its own key set before the route can answer.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/respanai-respan-gateway/add-fallback-and-hedged-routes/

## How to give any chat model web search and MCP tools with Respan Gateway

Repo: [respanai/respan-gateway](https://github.com/respanai/respan-gateway) | Difficulty: Advanced | Time: 45 minutes | Basis: From the README

Attach a tool vendor or MCP server so a plain chat call can use it.

**Who it is for:** A developer building an assistant who needs web search or other tools available to whichever model the app calls.

**The problem:** Tool support differs by model and client, so tools must be wired separately for each combination.

**The outcome:** A plain chat request through the gateway can call search tools, whatever the client or model.

### Steps

1. Run `respan-gateway init` to create gateway.toml.
2. Add a [tools.search] table with the vendor and key settings shown in the README, for example vendor = "linkup" with key = "env:LINKUP_API_KEY".
3. Add a [plugins.tool_loop] table whose `servers` list includes "search".
4. Run `respan-gateway check` to validate the file, then start the gateway with `respan-gateway`.
5. Send a chat request that asks a question needing current web data and check the reply for tool use in the live log.

### Prompt to paste into your agent

```text
In the respan-gateway repo, add a search tool vendor and the tool_loop plugin to gateway.toml, validate it, then send a chat request to the gateway that needs web search and report the answer. Verify that the live log at localhost:8080/ui/ shows the tool call.
```

### Caveat

The README example uses the linkup tool vendor, so you need a key for that vendor before search tools will work.

Risk, credentials: Tool vendor keys are stored in gateway.toml through env references, so keep the environment variables out of shared logs and version control.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/respanai-respan-gateway/give-any-model-web-search-and-mcp-tools/

## How to trace every model call and read cost and latency with Respan Gateway

Repo: [respanai/respan-gateway](https://github.com/respanai/respan-gateway) | Difficulty: Intermediate | Time: 20 minutes | Basis: From the README

Send all gateway traffic to Respan and check cost, latency and tokens per call.

**Who it is for:** A team lead or engineer who needs to see what each AI call costs and how long it takes across providers.

**The problem:** Model spend and latency are spread across several vendor dashboards, with no single view of each call.

**The outcome:** Every call through the gateway is traced in Respan, with cost, latency and tokens visible in one place.

### Steps

1. Create gateway.toml with `respan-gateway init`.
2. Add a [plugins.respan] table with key = "env:RESPAN_API_KEY", as in the README example.
3. Export your Respan key with `export RESPAN_API_KEY=...`. (inferred, check the README)
4. Validate with `respan-gateway check`, then start the gateway with `respan-gateway`.
5. Send a few requests and open http://localhost:8080/ui/ for the live log, then check traces in Respan.

### Prompt to paste into your agent

```text
In the respan-gateway repo, configure the respan plugin in gateway.toml using RESPAN_API_KEY from the environment, validate the config, and send <number> test requests through the gateway. Return the cost, latency and token count for one call, and verify that a matching trace appears in Respan.
```

### Caveat

Traces only reach Respan when a valid RESPAN_API_KEY is set, and the README does not describe what data is included in each trace beyond cost, latency and tokens.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/respanai-respan-gateway/trace-every-model-call-in-respan/

## How to write a custom guardrail or PII masking plugin with Respan Gateway

Repo: [respanai/respan-gateway](https://github.com/respanai/respan-gateway) | Difficulty: Advanced | Time: 90 minutes | Basis: From the README

Use the public plugin API to add masking or policy checks to every call.

**Who it is for:** A Rust developer on a platform team who must enforce a rule, such as masking personal data, on all model traffic.

**The problem:** Model traffic may contain personal data or policy-breaking content that must be filtered before it leaves the company.

**The outcome:** A custom plugin built on the gateway's one-trait, four-hook API runs on every request.

### Steps

1. Read the plugin guide at docs/plugins.md.
2. Open the examples index at examples/README.md and pick the example closest to your rule, such as guardrails or PII masking.
3. Implement the plugin trait's four hooks in Rust, following the example.
4. Build the gateway with your plugin mounted, using the embed option in docs/plugins.md, then run the check command against your config. (inferred, check the README)
5. Send a request containing test personal data and confirm the masked output in the live log.

### Prompt to paste into your agent

```text
In the respan-gateway repo, read docs/plugins.md and examples/README.md, then write a Rust plugin that implements the four hooks for <rule>. Return the plugin file and the test request result, and verify that the masked value does not appear in the live log.
```

### Caveat

The README describes the plugin API only at a high level, so the exact hook names and signatures must be taken from docs/plugins.md rather than assumed.

Disclaimer: Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk. Full text: https://dedupedai.com/disclaimer/

URL: https://dedupedai.com/use-cases/respanai-respan-gateway/build-a-guardrail-or-pii-plugin/