mode Local LLM

ollama-docker-deploy-expert-mode

Production self-host Ollama in Docker/Compose with GPU passthrough, model preload, reverse proxy auth, and multi-GPU

View source on GitHub ↗ ← Back to search
KindMode
CategoryLocal LLM
Installnpx -y github:anubhavg-icpl/vibe add ollama-docker-deploy-expert-mode
LicenseCC BY-NC-SA 4.0

More in Local LLM

mode

exllama-awq-gptq-expert-mode

Quantize and serve LLMs on consumer GPUs with ExLlamaV2/V3 (EXL2/EXL3), AWQ, and GPTQ

View → mode

gguf-quantization-expert-mode

Convert HF safetensors to GGUF, run llama-imatrix, choose K-quants vs IQ-quants, and quantize models for llama.cpp

View → mode

jan-ai-expert-mode

Use Jan.ai open-source desktop assistant as a local LLM hub, OpenAI-compatible server on port 1337, and MCP host

View → mode

litellm-proxy-expert-mode

Run LiteLLM as a unified gateway over local + cloud LLMs with router config, virtual keys, budgets, fallbacks, and Redis caching

View → mode

llama-cpp-expert-mode

Build, run, and tune llama.cpp for local LLM inference across CUDA, ROCm, Metal, Vulkan, and SYCL

View → mode

llama-cpp-server-expert-mode

Run llama.cpp's HTTP server with OpenAI-compatible endpoints, slots, multimodal, and reverse proxies

View →