101 lines
4.4 KiB
Markdown
101 lines
4.4 KiB
Markdown
# Local AI and Automation Docker Suite
|
|
|
|
OK, let's get you started! At the end of the day, you only really need ```Ollama``` and an IDE (either ```VS Code``` or ```JetBrains```), so why does this docker setup have so many different services? The reason is that this setup is here to get you started, and it takes a modular approach which, over time, is going to allow you to upgrade parts of this install at a component level. As an example, if you love what this setup brings, but want to use bigger better LLM Models? You can buy a server just for LLM Hosting, put Ollama on it, and just re-point your Open WebUI container to the new server, and everything else continues to work as normal.
|
|
|
|
|
|
|
|
## Docker Compose Contains
|
|
|
|
- Ollama (LLM Host)
|
|
- Open WebUI (Chat Interface)
|
|
- MCPO Server (Model Context Protocol relay server)
|
|
- N8N (Automation)
|
|
- PostgresDB (for N8N)
|
|
- Jupyter Notebook
|
|
|
|
## Getting Started
|
|
|
|
1. Make sure you have the prerequisites installed (Git, Docker Desktop)
|
|
2. Clone this repo
|
|
```git
|
|
git clone https://git.brianfertig.com/brianfertig/LocalAIChat.git LocalAIChat
|
|
```
|
|
|
|
## Install Instructions
|
|
|
|
1. Copy ```sample.env``` to ```.env```
|
|
2. Edit ```.env``` file to contain your ultimate secret passwords
|
|
3. Copy ```config/mcpo/sample.config.json``` to ```config/mcpo/config.json```
|
|
4. Note -- there is no need to edit the config.json file right away
|
|
5. If you have an NVidia GPU, open ```docker-compose.yml``` and uncomment the lines from the Ollama container
|
|
6. From the root directory (of this repo) run ```docker compose up -d```
|
|
7. If you experience any errors, run it again until it stands completely up
|
|
|
|
## Next Steps
|
|
|
|
Once you're up and running in Docker, lets configure a few things...
|
|
|
|
1. Open WebUI [http://127.0.0.1:8080](http://127.0.0.1:8080)
|
|
2. Click ```Get Started```
|
|
3. Create a new account (This will be the admin account)
|
|
4. Open the sidebar (three lines on the left), click on your name, then click on ```Admin Panel```
|
|
5. Browse to ```Settings``` then ```Models``` then click the ```down arrow icon``` (Manage Models)
|
|
6. In the ```Pull a model from Ollama.com``` field enter ```qwen3:4b```
|
|
7. Wait for this model to completely download and successfully install
|
|
8. Close the ```Manage Models``` window, and click ```New Chat``` in the upper-left
|
|
9. Chat with the model to make sure it is functioning properly. (just a quick check is all that is needed to verify it's working)
|
|
10. Click ```Workspace``` from the left menu, then select ```Import Models```
|
|
11. Browse to the ```/config/OpenWebUI``` directory in this repo, and import the ```software-assistant.json``` file.
|
|
12. Next, click your name in the lower left of the lefthand menu and then click ```Settings```
|
|
13. Click ```Account``` then ```API Keys``` and then ```Create new secret key```
|
|
1. Save your API key for the next part.
|
|
|
|
## Configure VS Code for AI
|
|
|
|
1. In VS Code, search for ```Continue``` and install the ```Continue - open-source AI code assistant``` extension
|
|
2. Once it is installed you should see ```Continue``` in VS Code's lower right corner with a check next to it.
|
|
3. Press ```CTRL+L``` to open a chat window. Just above the chat input you should see ```Local Assistant``` expand this and click on the ```Gear Icon``` next to ```Local Assistant```.
|
|
4. Paste the following over the existing YAML, and edit as needed:
|
|
|
|
```YAML
|
|
name: Local Assistant
|
|
version: 1.0.0
|
|
schema: v1
|
|
models:
|
|
- name: Software Assistant
|
|
provider: openai
|
|
model: software-assistant-32k-context
|
|
env:
|
|
useLegacyCompletionsEndpoint: false
|
|
apiBase: http://127.0.0.1:8080/api
|
|
apiKey: {{ your API key }}
|
|
roles:
|
|
- chat
|
|
- edit
|
|
|
|
- name: Software Assistant Autocomplete
|
|
provider: openai
|
|
model: software-assistant-32k-context
|
|
env:
|
|
useLegacyCompletionsEndpoint: false
|
|
apiBase: http://127.0.0.1:8080/api
|
|
apiKey: {{ your API key }}
|
|
roles:
|
|
- autocomplete
|
|
|
|
prompts:
|
|
- name: test
|
|
description: Write unit tests for highlighted code
|
|
prompt: |
|
|
Write a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.
|
|
|
|
context:
|
|
- provider: code
|
|
- provider: docs
|
|
- provider: diff
|
|
- provider: terminal
|
|
- provider: problems
|
|
- provider: folder
|
|
- provider: codebase
|
|
```
|
|
5. You should now be able to chat with AI in VS Code as well. |