Local AI and Automation Docker Suite

OK, let's get you started! At the end of the day, you only really need Ollama and an IDE (either VS Code or JetBrains), so why does this docker setup have so many different services? The reason is that this setup is here to get you started, and it takes a modular approach which, over time, is going to allow you to upgrade parts of this install at a component level. As an example, if you love what this setup brings, but want to use bigger better LLM Models? You can buy a server just for LLM Hosting, put Ollama on it, and just re-point your Open WebUI container to the new server, and everything else continues to work as normal.

Docker Compose Contains

• Ollama (LLM Host)
• Open WebUI (Chat Interface)
• MCPO Server (Model Context Protocol relay server)
• N8N (Automation)
• PostgresDB (for N8N)
• Jupyter Notebook

Getting Started

1. Make sure you have the prerequisites installed (Git, Docker Desktop)
2. Clone this repo

git clone https://git.brianfertig.com/brianfertig/LocalAIChat.git LocalAIChat

Install Instructions

1. Copy sample.env to .env
2. Edit .env file to contain your ultimate secret passwords
3. Copy config/mcpo/sample.config.json to config/mcpo/config.json
4. Note -- there is no need to edit the config.json file right away
5. If you have an NVidia GPU, open docker-compose.yml and uncomment the lines from the Ollama container
6. From the root directory (of this repo) run docker compose up -d
7. If you experience any errors, run it again until it stands completely up

Next Steps

Once you're up and running in Docker, lets configure a few things...

1. Open WebUI http://127.0.0.1:8080
2. Click Get Started
3. Create a new account (This will be the admin account)
4. Open the sidebar (three lines on the left), click on your name, then click on Admin Panel
5. Browse to Settings then Models then click the down arrow icon (Manage Models)
6. In the Pull a model from Ollama.com field enter qwen3:4b
7. Wait for this model to completely download and successfully install
8. Close the Manage Models window, and click New Chat in the upper-left
9. Chat with the model to make sure it is functioning properly. (just a quick check is all that is needed to verify it's working)
10. Click Workspace from the left menu, then select Import Models
11. Browse to the /config/OpenWebUI directory in this repo, and import the software-assistant.json file.
12. Next, click your name in the lower left of the lefthand menu and then click Settings
13. Click Account then API Keys and then Create new secret key
    1. Save your API key for the next part.

Configure VS Code for AI

1. In VS Code, search for Continue and install the Continue - open-source AI code assistant extension
2. Once it is installed you should see Continue in VS Code's lower right corner with a check next to it.
3. Press CTRL+L to open a chat window. Just above the chat input you should see Local Assistant expand this and click on the Gear Icon next to Local Assistant.
4. Paste the following over the existing YAML, and edit as needed:

name: Local Assistant
version: 1.0.0
schema: v1
models: 
  - name: Software Assistant
    provider: openai
    model: software-assistant-32k-context
    env:
      useLegacyCompletionsEndpoint: false
    apiBase: http://127.0.0.1:8080/api
    apiKey: {{ your API key }}
    roles:
      - chat
      - edit

  - name: Software Assistant Autocomplete
    provider: openai
    model: software-assistant-32k-context
    env:
      useLegacyCompletionsEndpoint: false
    apiBase: http://127.0.0.1:8080/api
    apiKey: {{ your API key }}
    roles:
      - autocomplete

prompts:
  - name: test
    description: Write unit tests for highlighted code
    prompt: |
      Write a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.      

context:
  - provider: code
  - provider: docs
  - provider: diff
  - provider: terminal
  - provider: problems
  - provider: folder
  - provider: codebase

5. You should now be able to chat with AI in VS Code as well.