--- title: Local AI Models url: https://www.tines.com/stories/docs/self-hosted/configuring-tines/local-ai-models/ updated: 2026-10-31T00:01:46+00:00 --- *[tines.com](https://www.tines.com/llms.txt) › [Docs](https://www.tines.com/stories/docs/llms.txt) › [Self-Hosted](https://www.tines.com/llm/stories/docs/self-hosted.md) › [Configuring Tines](https://www.tines.com/llm/stories/docs/self-hosted/configuring-tines.md)* # Local AI Models *[View on tines.com](https://www.tines.com/stories/docs/self-hosted/configuring-tines/local-ai-models/)* This page is for running Tines Stories with a local model: a model you host on your own hardware and serve with a tool such as llama.cpp, vLLM, or Ollama. Teams often do this to keep AI traffic inside their own network, or when an air-gapped deployment can't reach a hosted provider. Tines AI features, including AI Agent actions, automatic mode, and formula generation, rely on the model you connect. With a local model, you control the settings that decide how well those features work. This page covers what to check before you connect one. If you use a hosted provider such as OpenAI, Anthropic, or Amazon Bedrock, see [AI provider configuration](https://www.tines.com/docs/self-hosted/configuring-tines/ai-provider-configuration/) instead. Check these five settings for a local model: - Model capability. On the [Artificial Analysis coding index](https://artificialanalysis.ai/?intelligence=coding-index), aim for about 25 or higher for the smart model. - Context size. Set at least 200,000 tokens in the server launch command. - Reasoning. Turn thinking on for the smart model. Some servers leave it off. - Effort. Higher effort takes longer. `none` turns thinking off. - Quantization. `Q4_K_XL` or `Q4_K_M` is the smallest size we recommend. Use `Q8_0` when the GPU has room. ## Model capability We compare models with the [Artificial Analysis coding index](https://artificialanalysis.ai/?intelligence=coding-index). The [open-weights view](https://artificialanalysis.ai/models/open-source) lists models you can run yourself. Read the score on the day you choose. The index is updated, and older cutoffs stop matching the chart. The smart model is used for AI Agent actions with tools, formula generation, and automatic mode. On Tines Cloud it also powers Workbench. Set it in [AI settings](https://www.tines.com/docs/admin/ai/). A model selected on a Workbench conversation or an AI Agent action overrides that default. The fast model is used for automatic transform, simple AI Agent runs, and generated names. It can score lower. If you run one model, use it as the smart model. If the model is not on the chart, test it on a story you know before you roll it out. ## Context size Set the server context length to at least 200,000 tokens. Use the value in the launch command. A model started at 8,000 or 16,000 tokens runs at that smaller size. Confirm the context fits in GPU memory at the quantization you serve. The context cache uses more memory than the model weights. ## Reasoning and effort Turn thinking on for the smart model. On llama.cpp, set `enable_thinking` in `--chat-template-kwargs` when the model uses that switch. An effort of `none` turns thinking off. If the server rejects `reasoning_effort`, check Omit reasoning effort under Extra options and leave thinking on at the server. ## Quantization We recommend `Q4_K_XL` or `Q4_K_M` at a minimum. Use `Q8_0` when it fits. Below 4-bit, the coding-index score no longer describes the model you are running. A 4-bit copy on your own hardware scores a little below the published number. ## Connect the server Point Tines Stories at an OpenAI-compatible endpoint that supports streaming and tool use. Set API type under Extra options. See [AI](https://www.tines.com/docs/admin/ai/) for providers, custom models, and credentials. - The server must complete a chain of tool calls. If Tines Stories does not recognize the format, set Function calling mode to Hermes. - The model id must match the name the server advertises. Add it as a custom model if the server has no models list. - Set the context length in the launch command. Check whether a model fits on your GPU at [llmrun.dev](https://llmrun.dev/) or with the [APXML VRAM calculator](https://apxml.com/tools/vram-calculator). Run the strongest model that fits at `Q4_K_XL` or better with a 200,000-token context.