What I'm building this week: Route your Gemini API calls on your terms

A little router I'm building and improving to use different Gemini (and soon other) models. A question I’ve received a few times recently is how companies can route Gemini API calls based on their own rules and specs. This usually includes a few things.

Cover image from the original post.

A question I’ve received a few times recently is how companies can route Gemini API calls based on their own rules and specs. This usually includes a few things:

How to route calls based on more of a central governance model than letting any developer pick any model for their app
How you could dynamically route based on some complexity map
or How to control some level of app throttle (including dealing with 429’s from specific models)

So, I built the smartrouter app over the past two weeks, and I’m intending to extend it further.

What is smartrouter and what does it do?

Very simply, smartrouter is compliant with a subset of the Gemini SDK. It allows you to specify a custom endpoint and model type. Then, the call is routed to smartrouter, which runs on Google Cloud Run. It’s just a container with some services though, so you could run this elsewhere if you wanted.

Here’s the architecture:

For Gemini models, you can specify the Google Cloud region/location you want to run smartrouter in. Then you dynamically detect all of the available models in that region, the multi region of your region, and globally. You can turn models on and off for users to use.

For requests, these are routed via a gemini-dynamic model. Based on your app you configure, the requests are sent to the models you choose. All requests and metrics are logged via Google Cloud Monitoring and Logging so you can see usage based metrics and track costs in near real time.

Features coming soon

I’m a little rusty on my AWS and Azure skills, so I’m planning to extend this to other clouds and model types as somewhat of a forcing function for me to learn about the updates that have happened since I last used them a few years back. This includes extending the endpoint capabilities to other SDK types.

I also want to explore more dynamic router methods and I’m planning to make this more advanced and configurable.

I’ll respond to community requests as they come up and are within reason :).

How I used AI to do the work

Since Antigravity was updated, I've mostly used it to help with the code work. I’ll work with Antigravity on the feature I want, let it do some coding, and then validate or correct the work. A few quick things I’ve found work well:

I keep sessions pretty short. I define a small feature, work on it, get the tests done, and then open a new session / conversation.
If I need a planning discussion, the only artifact I’m looking for is a task list to handoff to subsequent sessions to keep the context window small.
There were a few features that the agent just struggled with. When this happens, I’ll usually try a few different approaches. Then I will go back to the old methods, which include Google search. I get a few pages, give these to the agent, and then ask it to correct the work. This pattern works pretty well.
As I go along, I ask the agent to update itself. This was part of the inspiration for the session journal, which I intend to extend.

Next steps

The link to get started with smartrouter is: https://github.com/davenportjw/smartrouter

Try it out, find bugs or enjoy it, and let me know your thoughts!

Code for this essay

smartrouter

Routes agent prompts dynamically between Google Cloud Vertex AI (Gemini 3.8 Flash / Gemini 3.5 Flash-Lite) and local Gemma 4 runners based on complexity score, context token budget, and data residency policy.

Open in Code

Sample output from a local run

$ uv run python -m smartrouter.cli --prompt 'Refactor AST graph lock lease expiry' --slo-ms 850
ROUTE DECISION: vertex-ai/gemini-3.8-flash (project=davenport-boutique, location=us-central1)
REASON: multi-symbol code mutation (complexity=0.82 > threshold=0.65)
AUTH MODE: google-cloud-adc (zero API key) · EST. LATENCY: 410 ms

Social card and structured data

The share image, feeds and Schema.org JSON-LD that search engines and AI tools read for this page.

Social card

Card URL: /og/what-im-building-this-week-route.svg

Structured data

{
  "@context": "https://schema.org",
  "@graph": [
    {
      "@type": "Person",
      "@id": "https://davenporter.com/#jason-davenport",
      "name": "Jason Davenport",
      "alternateName": "@davenportjw",
      "jobTitle": "AI & Distributed Systems Architect",
      "description": "Exploring agent-first source control (cosm), trajectory-to-lineage observability, Spanner-backed agent memory consolidation (Electric Sheep), hybrid local/cloud LLM routing, and real-time multiplayer browser simulations on Google Cloud Run.",
      "url": "https://davenporter.com",
      "sameAs": [
        "https://davenporter.substack.com",
        "https://www.linkedin.com/in/davenportjw",
        "https://github.com/davenportjw",
        "https://x.com/davenportjw"
      ]
    },
    {
      "@type": "WebSite",
      "@id": "https://davenporter.com/#website",
      "url": "https://davenporter.com",
      "name": "The Davenporter",
      "description": "Learnings about software, technology, agentic systems, and occasionally home renovation.",
      "publisher": {
        "@id": "https://davenporter.com/#jason-davenport"
      },
      "inLanguage": "en-US"
    },
    {
      "@type": "ImageObject",
      "@id": "https://davenporter.com/writings/what-im-building-this-week-route#primaryimage",
      "url": "https://davenporter.com/og/what-im-building-this-week-route.png",
      "contentUrl": "https://davenporter.com/og/what-im-building-this-week-route.png",
      "width": 1200,
      "height": 630,
      "caption": "Title card for What I'm building this week: Route your Gemini API calls on your terms",
      "representativeOfPage": true,
      "creator": {
        "@id": "https://davenporter.com/#jason-davenport"
      },
      "license": "https://creativecommons.org/licenses/by/4.0/",
      "acquireLicensePage": "https://davenporter.com/writings"
    },
    {
      "@type": "WebPage",
      "@id": "https://davenporter.com/writings/what-im-building-this-week-route#webpage",
      "url": "https://davenporter.substack.com/p/what-im-building-this-week-route",
      "name": "What I'm building this week: Route your Gemini API calls on your terms | The Davenporter",
      "description": "A little router I'm building and improving to use different Gemini (and soon other) models. A question I\u2019ve received a few times recently is how companies can route Gemini API calls based on their own rules and specs. This usually includes a few things.",
      "isPartOf": {
        "@id": "https://davenporter.com/#website"
      },
      "about": {
        "@id": "https://davenporter.com/#jason-davenport"
      },
      "primaryImageOfPage": {
        "@id": "https://davenporter.com/writings/what-im-building-this-week-route#primaryimage"
      },
      "inLanguage": "en-US"
    },
    {
      "@type": "FAQPage",
      "@id": "https://davenporter.com/writings/what-im-building-this-week-route#faq",
      "mainEntity": [
        {
          "@type": "Question",
          "name": "What is \u201cWhat I'm building this week: Route your Gemini API calls on your terms\u201d about?",
          "acceptedAnswer": {
            "@type": "Answer",
            "text": "A little router I'm building and improving to use different Gemini (and soon other) models. A question I\u2019ve received a few times recently is how companies can route Gemini API calls based on their own rules and specs. This usually includes a few things."
          }
        }
      ]
    },
    {
      "@type": "BlogPosting",
      "@id": "https://davenporter.com/writings/what-im-building-this-week-route#article",
      "mainEntityOfPage": {
        "@id": "https://davenporter.com/writings/what-im-building-this-week-route#webpage",
        "url": "https://davenporter.substack.com/p/what-im-building-this-week-route"
      },
      "headline": "What I'm building this week: Route your Gemini API calls on your terms",
      "alternativeHeadline": "A little router I'm building and improving to use different Gemini (and soon other) models.",
      "description": "A little router I'm building and improving to use different Gemini (and soon other) models. A question I\u2019ve received a few times recently is how companies can route Gemini API calls based on their own rules and specs. This usually includes a few things.",
      "datePublished": "2026-05-28T16:02:33Z",
      "dateModified": "2026-05-28T16:02:33Z",
      "wordCount": 577,
      "keywords": "WIBTM, Source Control, Model Routing",
      "author": {
        "@id": "https://davenporter.com/#jason-davenport"
      },
      "publisher": {
        "@id": "https://davenporter.com/#jason-davenport"
      },
      "image": {
        "@id": "https://davenporter.com/writings/what-im-building-this-week-route#primaryimage"
      },
      "url": "https://davenporter.substack.com/p/what-im-building-this-week-route"
    },
    {
      "@type": "BreadcrumbList",
      "@id": "https://davenporter.com/writings/what-im-building-this-week-route#breadcrumb",
      "itemListElement": [
        {
          "@type": "ListItem",
          "position": 1,
          "name": "Home",
          "item": "https://davenporter.com/"
        },
        {
          "@type": "ListItem",
          "position": 2,
          "name": "Essays",
          "item": "https://davenporter.com/writings"
        },
        {
          "@type": "ListItem",
          "position": 3,
          "name": "What I'm building this week: Route your Gemini API calls on your terms",
          "item": "https://davenporter.com/writings/what-im-building-this-week-route"
        }
      ]
    }
  ]
}