Give Your Local Model a Memory (Ollama, LM Studio)

The problem: local models forget everything the moment you close the window

People run models locally for good reasons — privacy, cost, working offline, or simply because a 9 GB model on a laptop is now genuinely useful. But every local setup shares one weakness the hosted assistants have quietly solved: there is no memory. Ollama starts each conversation from zero. LM Studio keeps a chat history you can scroll, but the model does not read it. Switch from your local model to Claude on your phone and nothing you told the first one exists for the second.

So the workflow everyone ends up with is copy-paste: a notes file of "things the model should know", pasted into the first message, growing until it no longer fits in the context window. That works for a week. It does not work for a project.

What a local model needs is the same thing a hosted one needs: a memory that lives outside any single conversation, that the model can search when a question calls for it, and that follows you to whatever tool you open next. That is what this connection adds. The model keeps running on your machine; the memory is a service it can reach.

What stays local and what does not — stated plainly

Before the setup, the trade-off, because a page about local models that hides it would be selling you something else.

Stays on your machine: the model weights, the generation, the full text of every conversation you have with the model. None of that is sent anywhere by this connection.

Leaves your machine: the memory operations. When your model decides a question needs context, it sends a search query to the memory server and receives the matching notes back. When you tell it to remember something, that note is stored on our servers, encrypted at rest, in your workspace. If your requirement is that nothing whatsoever leaves the machine, stop here — a fully offline memory is a different product, and we would rather say so than let you find out later.

If the requirement is "my model runs locally and I keep control of what it remembers", read on. You can see, edit and delete every stored memory from the web app at any time, and the model only ever sees what a search returns.

Path 1 — LM Studio (native MCP, the cleanest option)

LM Studio added MCP support in 0.3.17. It reads servers from its own mcp.json, which follows Cursor's notation: a mcpServers object with a url and optional headers per server. That is exactly the shape a remote memory server needs.

1. Get your workspace token. Open the guided installer at /install/lm-studio — it mints a token for you and shows the filled-in block. (The public example on the lander pages keeps a placeholder, because a page cannot mint a credential.)
2. Open mcp.json. In LM Studio, open the Program tab in the right sidebar, click Install, then Edit mcp.json.
3. Merge the block. Paste the great-arrow entry into the mcpServers object, preserving any servers you already have, and save. It looks like this, with your token in place of the placeholder:

{
  "mcpServers": {
    "great-arrow": {
      "url": "https://www.greatarrow.ai/api/mcp",
      "headers": {
        "Authorization": "Bearer gad_PASTE_YOUR_TOKEN",
        "X-MCP-Client": "lm-studio"
      }
    }
  }
}

1. Toggle it on. The server appears in LM Studio's Integrations list; switch it on for the chat. Pick any model that supports tool calling — Llama 3.1, Qwen 2.5, Gemma, Mistral all do.
2. Ask it what it remembers. A first question like "search my memory for anything about the Q3 plan" makes the model call the search tool, and you will see the tool call and its result inline. Then "remember that we decided to ship the pricing change on the 30th" stores a note you can find later from any client.

That is the whole setup. From here on every conversation in LM Studio can reach the same memory, and so can every other client you connect.

Path 2 — Ollama through Open WebUI, Jan or Continue

Ollama itself is a model server, not an MCP client — it has no place to put a tool configuration. The clients people put in front of it do. Three that work today:

• Open WebUI attaches MCP servers through its tools settings (it bridges MCP to its own tool protocol). Add the server URL and the Authorization header there.
• Jan has an MCP servers panel in settings; it accepts the same url + headers block as LM Studio.
• Continue (the VS Code and JetBrains extension) takes MCP servers in its config.yaml, and can run against an Ollama backend, which gives a local coding model the memory your Cursor or Claude Code sessions already use.

In each case the block is the one shown above; only where you paste it differs. If your client of choice is not on this list, the general rule holds: any client that can attach a remote MCP server with a bearer header can attach this one. Pick your client on the install page for the exact steps, or use the generic remote-server instructions there.

Path 3 — the terminal, with no client at all

If you live in a terminal and already have Ollama or LM Studio running, the gad CLI can ask your local model a question and hand it the memory tools directly:

npm install -g @great-arrow/cli   # or: see /install/cli/npm
gad login
gad local check                  # finds Ollama / LM Studio, lists loaded models
gad local ask "what do we know about the onboarding rewrite?"

gad local check looks for a runtime on localhost and tells you which models are loaded. gad local ask sends your question to the local model with a read-only tool loop: it can search memories, read documents, list tasks and upcoming meetings, and check which integrations are connected — and it cannot change anything. That restriction is deliberate for a path that runs unattended in a shell; if you want the model to save memories too, use a full MCP client like LM Studio.

One thing to know about cold starts: loading a large model for the first time can take longer than the first-word deadline, so the CLI extends its patience only when it can positively see that the runtime is still loading the model. An unreachable runtime is reported as unreachable, not waited on forever.

What the model can do once it has memory

The tools a local client receives are the same ones every other client gets, so the behaviour is consistent across your setup:

• Search — semantic search over everything in your workspace: notes you saved, documents you uploaded, and whatever you connected (email, calendar, GitHub, Slack, Drive) with citations back to the source.
• Remember — save a fact, a decision or a preference with tags, from inside the conversation.
• Read — pull a specific document or memory in full when the search result is not enough.
• Look around — upcoming meetings, open tasks, which integrations are live.

A local model with these four verbs stops being a stateless text generator and becomes something closer to an assistant that knows your work. And because the memory is shared, the division of labour becomes natural: draft and iterate on the local model where it is free and private, then open Claude or ChatGPT for the hard question, and both have read the same notes.

Cost, honestly

Inference on your own hardware costs us nothing, and we book it that way: local calls are recorded at $0 and never appear in any cost figure. That sounds obvious, but it is a real trap in cost accounting — local runtimes expose model names shaped exactly like cloud catalog names, and a naive price lookup happily bills a free local call at cloud rates. Ours zeroes local usage before it reaches any price table, which is the only order that produces a correct number.

What does count is the memory service: storage, embeddings, and the search calls. A trial starts on sign-up with no card; after it, the free floor keeps memory sync and search working, and the paid plans add the hosted chat and agents that a local-model user may never open. If you only want the memory, you will not be paying for the rest.

Where to start

• Install for LM Studio — the guided version of Path 1, with your token filled in.
• Pick any other client — Open WebUI, Jan, Continue, or the generic remote-server instructions.
• One memory across every AI client — how the shared memory behaves once more than one tool is connected.
• Give Cursor persistent memory — the same idea for a coding editor, which pairs well with a local model in Continue.

The short version: your model stays yours, and now it remembers.