8/26/2024

Setting Up Environment Variables for Ollama

The world of machine learning models is vast & complex, but with Ollama, you can get started with Large Language Models (LLMs) right from your local machine. However, to make the most of Ollama's capabilities, especially in terms of customization & networking, setting up environment variables is crucial. In this blog post, we’ll dive deep into how to configure environment variables for Ollama effectively, making your journey smoother and your experience richer.

What Are Environment Variables?

Before we jump into the setup, let's clarify what environment variables are. Simply put, they are key-value pairs that define various settings & configurations for software applications. Environment variables can dictate where your models get stored, how they operate, or even how they communicate over networks.

Whether you're running Ollama on macOS, Linux, or Windows, the process might differ slightly, but the core concepts remain the same. Ready to dive in? Let’s get started!

Why Set Environment Variables?

Setting environment variables in Ollama can help you:

Customize settings for model storage locations.
Manage service configurations like host addresses.
Optimize performance by setting up GPU usage options.
Control the behavior of Ollama server when it communicates across networks.
Ensure that your environment is secure & efficient for running LLMs.

If you’re looking to utilize Ollama effectively in your projects, understanding how to set & implement these variables become vital.

Key Environment Variables for Ollama

Let's look at some of the most important environment variables you should configure:

1OLLAMA_HOST
: This variable specifies which IP address Ollama server will bind to. By default, it's set to
1127.0.0.1
, which means it will only listen to local requests. If you want it accessible over your whole network, you must set this to
10.0.0.0
.
1OLLAMA_MODELS
: Designates where your models will be stored on disk. By default, Ollama saves them in a user-specific directory (like
1~/.ollama/models
on Linux or
1C:\Users\<Username>\.ollama\models
on Windows).
1OLLAMA_KEEP_ALIVE
: If you'd like your models to remain loaded in memory longer than the default time, this variable can be adjusted to control the model's lifespan in memory.
1OLLAMA_MAX_QUEUE
: Controls how many concurrent requests the Ollama server can handle at once, preventing your server from getting overloaded.
1OLLAMA_NUM_PARALLEL
: Defines how many requests can be processed at the same time, letting you squeeze out more performance when you have available resources.

Setting Environment Variables on Different Platforms

Now that you have the goods on the important environment variables, let’s check out how to set them up on various Operating Systems.

On macOS

Setting up environment variables on macOS can be done using the

launchctl

command:

Open your terminal.
Use the following command to set a variable:
1 2bash launchctl setenv OLLAMA_HOST "0.0.0.0"
Restart the Ollama application to apply changes.

If you want these changes to persist across reboots, consider adding them to your shell configuration file (like

.bash_profile

.zshrc

) in the form of:

1
2

bash
export OLLAMA_HOST="0.0.0.0"

Make sure to source your file afterward:
1 2bash source ~/.bash_profile

On Linux

For Linux, you can use

systemd

services to set environment variables for the Ollama service:

Edit the
1ollama.service
file:
1 2bash sudo systemctl edit ollama.service
Add the following to define environment variables:
1 2 3ini [Service] Environment="OLLAMA_HOST=0.0.0.0"
Save & exit, then reload the systemd configuration:
1 2 3bash sudo systemctl daemon-reload sudo systemctl restart ollama

You can also set environment variables for individual terminal sessions using:

1
2

bash
export OLLAMA_MODELS=/path/to/models

Make sure to verify your settings using:
1 2bash echo $OLLAMA_HOST

On Windows

For Windows, you can set environment variables via the Control Panel or Command Prompt:

Quit Ollama by right-clicking the application icon in the taskbar.
Go to Settings (Windows 11) or Control Panel (Windows 10), search for environment variables and click Edit environment variables for your account.
Here, you can add new variables like:
- Variable Name:
  1OLLAMA_HOST
- Variable Value:
  10.0.0.0
Alternatively, you can use the Command Prompt to set it like this:
1 2cmd setx OLLAMA_HOST "0.0.0.0"
Restart Ollama application and verify settings.

Using Docker

When you’re working with Docker, you may need to pass environment variables when starting your container:

1
2

bash
sudo docker run -e OLLAMA_HOST="0.0.0.0" ollama/ollama

This command will launch an Ollama container with the specified environment settings.

Best Practices for Managing Environment Variables

Setting up environment variables may sound simple, but there are a few best practices to consider:

Documentation: Keep a record of what each environment variable does for future reference. This will help if you later need to update or debug your configurations.
Separation of Concerns: If you're working on multiple projects, try to keep your environment variables organized by project. This can prevent conflicts and confusion later on.
Security: Ensure sensitive information (like API keys) is not hardcoded in your applications but passed as environment variables instead.
Testing: After setting the variables, always run tests to confirm they’re configured as expected. You can use scripts to automate testing.

Ollama also ensures no conversation data leaves your machine, as it runs completely locally. This is a significant feature that enhances your data privacy.

Conclusion

In summary, setting up environment variables in Ollama enables you to optimize local model settings, manage configurations for better performance, and control how your platform interacts with external requests. Follow the steps outlined above based on your operating system, and you’ll be well on your way to harnessing the full potential of Ollama.

As you explore the capabilities of Ollama, consider enhancing your engagement with your audience using Arsturn. Arsturn is an effortless, no-code AI chatbot builder that allows you to create conversational chatbots tailored to your needs. By integrating Arsturn, you can significantly boost engagement & conversions, helping you forge meaningful connections across your digital channels. Plus, you can do all this without the need for coding expertise! 🌟

Join thousands of users & unlock the power of conversational AI today at Arsturn! No credit card is required!

Become part of the Arsturn community now, & elevate your brand’s conversational capabilities!

Happy modeling with Ollama!