Running Gemma 4 and OpenCode locally on Mac

A technical guide details a functional local pipeline for Gemma 4 26B and OpenCode on macOS Apple Silicon. The recipe works around Ollama’s tool calling bugs and local provider limitations.

A guide published in April 2026 describes a local setup to run Gemma 4 26B with OpenCode on macOS Apple Silicon, bypassing Ollama’s tool calling bugs and limitations of local OpenAI-compatible providers. The proposed stack relies on llama.cpp compiled from source and a rebuilt version of OpenCode with a compatibility fix. The goal: to get reliable tool calls on Mac without relying on a cloud service. The topic matters for developers who want to keep their data local, but also for teams testing agents on personal machines without spending their evening chasing parsing errors.

Why Ollama blocks Gemma 4

According to the note, Ollama v0.20.0 causes three issues with Gemma 4: the tool call parser crashes, streaming loses tool calls en route, and the tokenizer produces noise in output. The document refers to ollama/ollama#15241 for the parsing crash and specifies that tool calls can be sent in the reasoning field with empty content, preventing the client from retrieving them. In this context, the recommended llama.cpp version includes the template fix linked to PR #21326 and the tokenizer fix from PR #21343.

The same applies to OpenCode: the standard version cannot properly retrieve responses from a local model when it returns tools in raw JSON, uses finish_reason: "stop" instead of tool_calls, or still relies on the old function_call format. PR #16531 adds a compatibility layer to absorb these discrepancies. In other words, the model may speak a somewhat too creative dialect, and the wrapper must act as translator.

A local stack designed for Apple Silicon

The guide was tested on a Mac Apple Silicon M1 Max with 32 GB RAM, on April 2, 2026. To run the model, it recommends cloning llama.cpp from source, applying the tokenizer fix, then compiling with -DGGML_METAL=ON to enable Metal, Apple’s graphics API. The server is then launched on port 8089, with a minimum context of 32,768 tokens for OpenCode’s tool definitions, and the --jinja option, deemed essential for Gemma 4’s tool calling.

The proposed configuration loads the Gemma 4 26B A4B IT GGUF model in Q4_K_M quantization. According to the note, the model consumes about 17 GB of memory for a typical session, leaving little margin when Chrome decides to have its breakfast on the Mac’s RAM. For more capable machines, the guide goes up to 128k context, but warns that memory pressure increases significantly.

OpenCode, the link that prevents breaking tools

OpenCode is then installed via its official script, then replaced by a version rebuilt from the anomalyco/opencode repository with PR #16531. The opencode.json configuration file points to http://127.0.0.1:8089/v1 and enables two parsers: raw-function-call and json. This duo serves to retrieve tool calls even when the model returns them in a shaky form. The model definition also includes tool_call: true, to signal OpenCode that the local backend can speak tools.

The guide also emphasizes AGENTS.md, an instruction file that gives the model the exact names of parameters expected by each tool. Schemas are detailed for bash, write, read, edit, glob, grep, and todowrite. The idea is simple: if Gemma 4 gets a field name wrong, compatibility collapses. The guide’s author therefore recommends strict parameters, without creative improvisation.

Key points

  • Ollama v0.20.0 breaks Gemma 4 according to the April 2026 note.
  • llama.cpp uses fixes from PR #21326 and #21343.
  • OpenCode relies on PR #16531 for tool calls.
  • The cited test was done on M1 Max, 32 GB, April 2, 2026.
  • The recommended local port is 8089, not 11434.
  • The indicated minimum context is 32,768 tokens.

By the numbers

  • 26B — recommended Gemma 4 model size for macOS Apple Silicon.
  • 17 GB — approximate memory required by the model, according to the guide.
  • 32 GB — RAM of the test Mac, M1 Max, April 2, 2026.
  • 8089 — local port used by llama-server.
  • 128k tokens — maximum context announced for Gemma 4 26B.

To read