Setting Up Ollama with Azure Functions

What is Ollama?

Why Choose Azure Functions?

Prerequisites

1. Azure account: If you don’t have one, you can easily create one for free.
2. Ollama installed: You need the Ollama CLI tool which you can download here.
3. Knowledge of basic command-line operations: This will be helpful throughout the setup process.
4. Azure CLI: Install Azure CLI to interact & manage resources— follow this guide.

Step 1: Set Up Azure Functions

1. Open the Azure portal & log in.
2. Click on Create a resource.
3. Select Compute > Function App.
4. Fill in the basic function details:
   
   • Subscription: Choose your Azure subscription.
   • Resource Group: Create a new one or use an existing one.
   • Function App name: Create a unique name that’s easy to remember.
   • Runtime Stack: Choose your preferred runtime (Node.js, Python, etc.).
   • Region: Choose a region close to your users.
5. Click Review + Create, and once validated, select Create.

Step 2: Deploying Ollama with Azure Functions

1. Navigate to your newly created Function App.
2. Click on Functions in the left sidebar & then select Create.
3. Choose the HTTP Trigger template.
4. Name your function and set the Authorization Level to Anonymous if you want it to be publicly accessible.
5. Click Create.

Step 3: Writing Code to Connect Ollama

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
const axios = require('axios');
module.exports = async function (context, req) {
    const prompt = req.body.prompt;
    try {
        const response = await axios.post('http://localhost:11434/v1/chat/completions', {
            model: 'llama2',
            messages: [{ role: 'user', content: prompt }]
        }); 
        context.res = {
            status: 200,
            body: response.data,
        };
    } catch (error) {
        context.res = {
            status: 500,
            body: error.message,
        };
    }
};

Step 4: Testing Your Setup

1
curl -X POST -H "Content-Type: application/json" -d '{"prompt": "Why's the sky blue?"}' https://<your-function-app-name>.azurewebsites.net/api/<your-function-name>

Step 5: Additional Configuration & Best Practices

• Ensure that your Ollama & functions are secured right, especially if dealing with sensitive data. Use Azure's built-in authorization features to provide security where needed.
• Monitor your function’s usage to ensure that you stay within the Azure Free Tier limits if you're using a trial.
• Customize the HTTP trigger to handle different types of requests, depending on how you plan to use Ollama.
• Use Azure Cosmos DB or Azure Storage for sustaining state or keeping track of user inputs and responses.

Conclusion: Join the AI Revolution with Arsturn

FAQs

1. What if I want to connect other models or APIs?
   You can easily modify the connection points in your Azure function code to use other AI services as needed.
2. Can Ollama work without an internet connection?
   Yes! Ollama can run locally, and your Azure Function will still be able to interact with it through localhost. However, it's best suited for development & testing in such scenarios.
3. Will I incur costs while using Azure Functions?
   As long as you stay within the Free Tier limits, you can experiment without worrying about costs. Always monitor your usage!