Use local models
By default, AI Assistant provides access to a predefined set of cloud-based models that are used for AI-related features. However, you can configure it to use custom local models provided by third parties.
The following third-party providers are supported:
Ollama – a command-line tool and runtime environment for running and managing local language models. Explore available models in the Ollama model library.
LM Studio – a desktop application that simplifies running and interacting with open-source LLMs locally. Explore available models in the LM Studio model catalog.
Other OpenAI-compatible endpoints like:
llama.cpp – a C++ framework for running large language models locally. Explore available models in the Hugging Face model catalog.
LiteLLM – a lightweight tool that lets you connect to many different AI models using a single OpenAI-compatible API.
Connect to a third-party provider
If you want to utilize models from a third-party provider, you need to connect to it first. To do this, you need to enable the corresponding provider and specify the URL where it is accessible.
Go to .
In the Third-party AI providers section, select your LLM provider and specify the URL where it can be accessed. Click Test Connection to check whether the connection is established successfully.
Click Apply to save changes.
Once the connection is established, the configured models will become available for use. You can select them to process your requests in the chat.
Use custom models in AI Assistant features
Custom models can also be used for other AI Assistant features, such as code completion, in-editor code generation, commit message generation, and more.
To configure custom models to be used in AI features:
Go to .
Configure your third-party provider.
In the Local models section, specify the models that you want to use for core, lightweight, and code completion features.
Core features – this model will be used for in-editor code generation, commit message generation, chat responses, and other core features.
Instant helpers – this model will be used for lightweight features, such as name suggestion.
Completion model – this model will be used for inline code completion.
Click Apply to save changes.
Configure the model context window
AI Assistant allows you to configure the model context window for local models. A larger window lets the model handle more context in a request, while a smaller one reduces memory usage and may improve performance. This helps balance context length with system resources.
To configure the model context window:
Go to .
Configure your third-party provider, so the local models become available for use.
In the Local models section, adjust the Context window setting as needed. The default value is 64 000 tokens.
Click Apply to save changes.
Switch to offline mode
If you want to restrict calls to remote models and only use the local ones, you can enable the offline mode. In this mode, most cloud model calls will be blocked, and all AI-related features will rely on the local models instead.
To enable offline mode:
Once you have finished the setup, you can toggle offline mode on and off whenever applicable:
Click the
JetBrains AI widget located in the toolbar in the window header.Hover over the Offline Mode option and click Enable or Disable.
