Wulf's Various Things

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== RagFlow ======

notes:

add to /etc/sysctl.conf:
<code>
vm.max_map_count=262144
sudo sysctl -w vm.max_map_count=262144
</code>

<code>
git clone https://github.com/infiniflow/ragflow.git

cd ragflow/docker
chmod +x ./entrypoint.sh
</code>

adjust docker-compose.yml, add
<code>
    deploy:
      resources:
        reservations:
          devices:
          - driver: nvidia
            device_ids: ['0']
            capabilities: [gpu]
</code>
for
change ports

<code>
docker compose up -d
</code>


Docker ENV Variables...:
user_default_llm

Newly signed-up users use LLM configured by this part. Otherwise, user need to configure his own LLM in setting.
factory

The LLM suppliers. "OpenAI"， "Tongyi-Qianwen", "ZHIPU-AI", "Moonshot", "DeepSeek", "Baichuan", and "VolcEngine" are supported.
api_key
The corresponding API key of your assigned LLM vendor.

ollama docker compose:
<code>
docker compose -f docker-ollama.yml up -d
</code>
run models (more on https://ollama.com/library ): 
preferred ones: mistral-openorca , llama3-chatqa? 
  llama3, qwen2, phi3, mistral, mixtral, codegemma, codellama, dolphin-mixtral, deepseek-coder, mistral-openorca , codestral, codeqwen, orca2, llama3-gradient,  llama3-chatqa (nvidia for RAG) and others...

download/update model:
<code>
docker exec -it ollama ollama pull codestral
</code>

list available models:
<code>
docker exec -it ollama ollama list
</code>

run model locally:
<code>
docker exec -it ollama ollama run llama3
</code>