HN

Comments (57)

• neilly
  Given this app seems to expose itself via REST calls, why would anyone care that it’s written in Go? I guess it matters to potential contributors but the majority of interest would be from users.
• Menu

  hgo
  Hey, this looks super nice. I do like the 'compact' feel of this. Reminds me of Traefik. It seems very promising indeed!One problem I have is that yes, LiteLLM key creation is easier than creating it directly at the providers and managing it there for team members and test environments, but if I had a way of generating keys via vault, it would be perfect and such a relief in many ways.I see what I need on your roadmap, but miss integration with service where I can inspect and debug completion traffic, and I don't see if I would be able to track usage from individual end-users through a header.Thank you and godspeed!
• Menu

  nzoschke
  Looks nice, thanks for open sourcing and sharing.I'm all in on Go and integrating AI up and down our systems for https://housecat.com/ and am currently familiar and happy with:https://github.com/boldsoftware/shelley -- full Go-based coding agent with LLM gateway.https://github.com/maragudk/gai -- provides Go interfaces around Anthropic / OpenAI / Google.Adding this to the list as well as bifrost to look into.Any other Go-based AI / LLM tools folks are happy with?I'll second the request to add support for harnesses with subscriptions, specifically Claude Code, into the mix.
• Menu

  pizzafeelsright
  I have written and maintained AI proxies. They are not terribly complex except the inconsistent structure of input and output that changes on each model and provider release. I figure that if there is a not a < 24 hour turn around for new model integration the project is not properly maintained.Governance is the biggest concern at this point - with proper logging, and integration to 3rd party services that provide inspection and DLP type threat mitigation.
• Menu

  crawdog
  I wrote a similar golang gateway, with the understanding that having solid API gateway features is important.https://sbproxy.dev - engine is fully open source.Another reason golang is interesting for the gateway is having clear control of the supply chain at compile time. Tools like LiteLLM the supply chain attacks can have more impact at runtime, where the compiled binary helps.
• Menu

  mosselman
  Does this have a unified API? In playing around with some of these, including unified libraries to work with various providers, I've found you are, at some point, still forced to do provider-specific works for things such as setting temperatures, setting reasoning effort, setting tool choice modes, etc.What I'd like is for a proxy or library to provide a truly unified API where it will really let me integrate once and then never have to bother with provider quirks myself.Also, are you also planning on doing an open-source rug pull like so many projects out there, including litellm?
• Menu

  sowbug
  Are these kinds of libraries a temporary phenomenon? It strikes me as weird that providers haven't settled on a single API by now. Of course they aren't interested in making it easier for customers to switch away from them, but if a proprietary API was a critical part of your business plan, you probably weren't going to make it anyway.(I'm asking only about the compatibility layer; the other tracking features would be useful even if there were only one cloud LLM API.)
• Menu

  glerk
  This is awesome work, thanks for sharing!How do you plan on keeping up with upstream changes from the API providers? I have implemented something similar, and the biggest issue I have faced with go is that providers don’t usually have sdk’s (compared to javascript and python), and there is work involved in staying up to date at each release.
• Menu

  pjmlp
  Expectable, given that LiteLLM seems to be implemented in Python.However kudos for the project, we need more alternatives in compiled languages.
• Menu

  Talderigi
  Curious how the semantic caching layer works.. are you embedding requests on the gateway side and doing a vector similarity lookup before proxying? And if so, how do you handle cache invalidation when the underlying model changes or gets updated?
• Menu

  driese
  Nice one! Let's say I'm serving local models via vllm (because ollama comes with huge performance hits), how would I implement that in gomodel?
• Menu

  indigodaddy
  Any plans for AI provider subscription compatibility? Eg ChatGPT, GH Copilot etc ? (Ala opencode)
• Menu

  tahosin
  This is really useful. I've been building an AI platform (HOCKS AI) where I route different tasks to different providers — free OpenRouter models for chat/code gen, Gemini for vision tasks. The biggest pain point has been exactly what you describe: switching models without changing app code.One thing I'd love to see is built-in cost tracking per model/route. When you're mixing free and paid models, knowing exactly where your spend goes is critical. Do you have plans for that in the dashboard?
• immanuwell
  it's nice that it supports different providers
• phoenixranger
  looks interesting, will defo give it a try. thanks for open-sourcing it!
• Menu

  rvz
  I don't see any significant advantage over mature routers like Bifrost.Are there even any benchmarks?
• Menu

  anilgulecha
  how does this compare to bifrost - another golang router?
• rpdaiml
  [dead]
• pukaworks
  [dead]