Local AI Coding with Ollama

Use Ollama as Andromity’s model provider to run inference on your computer. Local inference avoids cloud model API charges. Hardware, model downloads, and electricity still have costs, and task performance depends on the model and available memory.

Install Ollama and download a modelCopy Anchor Link

Install Ollama, then download a model with tool-calling support. This example uses the Qwen3 family shown in Ollama’s tool-calling documentation:

ollama pull qwen3:8b
ollama list
curl http://localhost:11434/api/tags

On Windows PowerShell, check the service with Invoke-RestMethod http://localhost:11434/api/tags. Start the Ollama application or run ollama serve if the endpoint refuses connections.

Choose a model your hardware can run comfortably. Context length and quantization affect memory usage; a model’s size alone does not guarantee that it can complete repository tasks.

Select the model in AndromityCopy Anchor Link

Open the model picker using /model in the terminal or the model control in VS Code. Select Ollama and the exact downloaded tag from ollama list. The default endpoint is http://localhost:11434.

To persist the choice, merge these settings into your existing configuration:

[default]
provider = "ollama"
model = "qwen3:8b"
profile = "builder"
permission_mode = "safe"

[[providers]]
name = "ollama"
type = "ollama"
base_url = "http://localhost:11434"

The configuration file is ~/.andromity/config.toml on macOS/Linux or %APPDATA%\andromity\config.toml on Windows, unless ANDROMITY_CONFIG_DIR overrides the directory. Keep existing provider entries when editing the file.

Verify a small coding taskCopy Anchor Link

Trust a repository you control before enabling repository tools. Start in SAFE mode with one bug, one file, and one test:

Find the function that handles an empty search query. Propose a minimal fix, add one regression test, and run the relevant test. Ask before adding dependencies.

Review the proposed approach, tool output, and diff. A model that answers code questions may still struggle with tools. Treat a passing test as evidence for the specific behavior it checks, not proof of the entire patch.

TroubleshootingCopy Anchor Link

• Connection refused: confirm Ollama is running and the endpoint is reachable.
• Model not found: compare the selected tag with ollama list, then download or select the correct tag.
• No tool calls: check the model’s tool-calling capability and simplify the task. Permission changes do not add model capabilities.
• Slow output or memory exhaustion: use a smaller model, reduce context, and check available system memory.

Local inference and network accessCopy Anchor Link

Selecting a local Ollama endpoint keeps model inference on that endpoint. Initial downloads need internet access. Web tools, remote MCP servers, package installations, telemetry, or a different selected provider can still use the network. Review configuration and telemetry before treating a complete workflow as offline.

Read the local coding agent walkthrough for a complete first-task workflow, or return to the quickstart.