HN

Need help?
<- Back
Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code

Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code

vbtechguy

Comments (67)

  • seifbenayed1992
    Local models are finally starting to feel pleasant instead of just "possible." The headless LM Studio flow is especially nice because it makes local inference usable from real tools instead of as a demo.Related note from someone building in this space: I've been working on cloclo (https://www.npmjs.com/package/cloclo), an open-source coding agent CLI, and this is exactly the direction I'm excited about. It natively supports LM Studio, Ollama, vLLM, Jan, and llama.cpp as providers alongside cloud models, so you can swap between local and hosted backends without changing how you work.Feels like we're getting closer to a good default setup where local models are private/cheap enough to use daily, and cloud models are still there when you need the extra capability.
  • edinetdb
    Claude Code has become my primary interface for iterating on data pipeline work — specifically, normalizing government regulatory filings (XBRL across three different accounting standards) and exposing them via REST and MCP.The MCP piece is where the workflow gets interesting. Instead of building a client that calls endpoints, you describe tools declaratively and the model decides when to invoke them. For financial data this is surprisingly effective — a query like "compare this company's leverage trend to sector peers over 10 years" gets decomposed automatically into the right sequence of tool calls without you hardcoding that logic.One thing I haven't seen discussed much: tool latency sensitivity is much higher in conversational MCP use than in batch pipelines. A 2s tool response feels fine in a script but breaks conversational flow. We ended up caching frequently accessed tables in-memory (~26MB) to get sub-100ms responses. Have you noticed similar thresholds where latency starts affecting the quality of the model's reasoning chain?
  • trvz
    ollama launch claude --model gemma4:26b
  • hackerman70000
    The real story here isn't Gemma 4 specifically, it's that the harness and the model are now fully decoupled. Claude Code, OpenCode, Pi, Codex all work with any backend. The coding agent is becoming a commodity layer and the competition is moving to model quality and cost. Good for users, bad for anyone whose moat was the harness
  • janalsncm
    Qwen3-coder has been better for coding in my experience and has similar sizes. Either way, after a bunch of frustration with the quality and price of CC lately I’m happy there are local options.
  • vbtechguy
    Here is how I set up Gemma 4 26B for local inference on macOS that can be used with Claude Code.
  • martinald
    Just FYI, MoE doesn't really save (V)RAM. You still need all weights loaded in memory, it just means you consult less per forward pass. So it improves tok/s but not vram usage.
  • jonplackett
    So wait what is the interaction between Gemma and Claude?
  • asymmetric
    Is a framework desktop with >48GB of RAM a good machine to try this out?
  • AbuAssar
    omlx gives better performance than ollama on apple silicon
  • Someone1234
    Using Claude Code seems like a popular frontend currently, I wonder how long until Anthropic releases an update to make it a little to a lot less turn-key? They've been very clear that they aren't exactly champions of this stuff being used outside of very specific ways.
  • NamlchakKhandro
    I don't know why people bother with Claude code.It's so jank, there are far superior cli coding harness out there
  • aetherspawn
    Can you use the smaller Gemma 4B model as speculative decoding for the larger 31B model?Why/why not?
  • maxbeech
    [dead]
  • techpulselab
    [dead]
  • meidad_g
    [dead]
  • aplomb1026
    [dead]
  • inzlab
    awesome, the lighter the hardware running big softwares the more novelty.