Run a local AI coding agent with Ollama and Andromity

Andromity can use an Ollama model running on your computer as the model behind its coding agent. You get the same repository workflow in VS Code or terminal: give the agent a task, review its proposed approach, and inspect the resulting changes. Local inference avoids cloud model API charges, though model downloads, hardware, and electricity still have costs.

What you need before starting

• Ollama installed and its local service running.
• Andromity installed through the VS Code extension or terminal CLI.
• A downloaded model with tool-calling support and enough available memory for its context window.
• A small repository task with a test you can run locally.

A model that answers coding questions may still struggle to call tools or finish a multi-file task. Check the model’s capabilities rather than selecting it only by size. Ollama’s tool-calling documentation uses Qwen3 in its examples; the example below uses that family without promising a particular task success rate.

1. Download a model and verify the service

ollama pull qwen3:8b
ollama list
curl http://localhost:11434/api/tags

The tags endpoint should return the models installed on your machine. If the connection is refused, start Ollama or run ollama serve. On Windows PowerShell, use Invoke-RestMethod http://localhost:11434/api/tags to inspect the response.

2. Select Ollama in Andromity

Open Andromity’s model picker in the sidebar or use /model in the terminal. Choose Ollama and the exact model tag shown by ollama list. The default local endpoint is http://localhost:11434. You do not need a cloud API key for local Ollama inference.

For a persistent default, merge these values into your existing configuration rather than replacing the file. The file is at ~/.andromity/config.toml on macOS/Linux or %APPDATA%\andromity\config.toml on Windows, unless you set ANDROMITY_CONFIG_DIR.

[default]
provider = "ollama"
model = "qwen3:8b"
profile = "builder"
permission_mode = "safe"

[[providers]]
name = "ollama"
type = "ollama"
base_url = "http://localhost:11434"

3. Test one bounded repository task

Open a repository you control and grant workspace trust before allowing repository tools. Keep the permission mode on SAFE for your first run. Try a request with a clear limit:

Find the function that validates email input. Propose a change for the empty-string case, add one regression test, and run the relevant test file. Ask before adding dependencies.

Review the plan, watch the tool calls, and read the diff. Verify the test output yourself. Successful text generation alone does not establish that the model can use repository tools correctly.

Troubleshoot local inference

The model is missing

Compare the selected tag with ollama list. Tags are exact identifiers; download the selected tag or select the one already installed.

The model replies but does not use tools

Check its tool-calling support. Reduce the task to one file and one test. If it repeatedly returns prose instead of tool calls, use a model that supports tools; increasing the permission mode will not fix model capability.

Generation is slow or runs out of memory

Try a smaller model, reduce the files included in the task, and check memory usage. Context length and quantization change memory requirements, so a single VRAM estimate cannot cover every setup.

What does “local” mean here?

With Ollama selected at a local endpoint, model inference runs there. Downloads need internet access initially. Web tools, remote MCP servers, package installations, telemetry, or another selected model can still use the network. Review configuration and telemetry settings before treating a workflow as offline. Local inference is one part of a private workflow.

For installation and provider reference, continue with the Ollama setup documentation. To try the editor workflow first, read your first AI coding task in VS Code.