AnythingLLM and LM Studio: Step-by-Step Setup Guide

LM Studio is a desktop app that runs local language models. AnythingLLM is a RAG platform that uses those models for conversational AI with document context. Together, they form a private alternative to cloud-based AI without vendor lock-in. This guide walks you through installing both, connecting them, and deploying for teams.

What is LM Studio? Why Use It With AnythingLLM?

LM Studio is a local inference engine--a desktop application that downloads and runs open-source language models on your computer or server. AnythingLLM, by contrast, is a retrieval-augmented generation (RAG) platform that orchestrates conversations, manages document storage, and routes queries to a language model. They are not competitors; they are complementary layers of a private AI stack.

LM Studio provides the "brains"--the language model that generates text. AnythingLLM provides the "middleware"--the retrieval logic, conversation memory, document embedding, and workspace organization. You download a model in LM Studio, start its API server, and point AnythingLLM to that server. AnythingLLM then uses the model to answer questions about your documents.

Why choose this stack? Because everything runs locally on your infrastructure. No cloud vendor has access to your data. No API keys sent to third parties. No per-token pricing. You control the models, the embeddings, and the output. For teams handling sensitive documents--legal contracts, proprietary research, customer data--this is a major advantage.

LM Studio is free and open-source (via its Python SDK, though the desktop app is proprietary). AnythingLLM is MIT-licensed open-source with 68.1K GitHub stars, maintained by Mintplex Labs. Both are production-ready and actively developed as of late 2026.

Prerequisites & System Requirements

Before you download anything, make sure your machine meets the minimum specs.

For LM Studio alone: You need at least 8GB of RAM. If you're running AnythingLLM on the same machine, plan for 16GB. Storage is the big one--a single medium-sized model (Mistral 7B, Llama 2) takes 14-16GB. Download two or three models, and you'll need 40-50GB of free disk space. A GPU accelerates inference dramatically; if your GPU has 4GB VRAM or more, LM Studio will use it automatically.

For AnythingLLM: It requires Python 3.10 or Node.js 18+ depending on your installation method. Disk space is modest (a few GB for the app and embeddings database).

Operating system: Both tools support Windows, macOS, and Linux. On Windows, administrator privileges are needed for the LM Studio installer. On Linux, ensure your CUDA drivers are up-to-date if you have an NVIDIA GPU.

Network: If you're running them on separate machines (recommended for teams), both need to be on the same network or connected via a stable IP. LM Studio's API server listens on localhost:1234 by default; you'll expose this via your network IP for remote access.

Step 1--Install & Configure LM Studio

Download LM Studio from https://lmstudio.ai. The application is free and available for Windows, macOS, and Linux.

Installation is straightforward. On Windows, run the installer and follow the wizard. On macOS, download the DMG file and drag it to Applications. On Linux, extract the AppImage or use your package manager. The app will prompt you to choose a folder for model storage--this is where downloaded models live. If you have multiple drives, choose one with plenty of free space.

When you first launch LM Studio, it auto-detects your GPU. If you have an NVIDIA or AMD GPU, it will offer to enable acceleration. Accept this; it makes inference 5-10x faster. If you have no GPU, LM Studio will fall back to CPU inference, which is slower but still usable for testing.

Next, you'll see the Model Library. This is LM Studio's built-in model browser. Search for a starting model: Mistral 7B is a good first choice (14GB, fast, good quality). Llama 2 (13B) and Llama 3 are also solid. Click Download, select a quantization (Q4 is a good balance of speed and quality), and wait. Depending on your internet speed, this takes 5-15 minutes per model.

Once a model is downloaded, you'll see it in the Local Models tab. Click on it and select Load. The model will load into memory. On first load, LM Studio displays the context window size (typically 4,096 or 8,192 tokens).

Finally, start the Local Server. In the left sidebar, click Local Server. Ensure "Allow Remote Connections" is unchecked (for local testing). Click Start Server. The default port is 1234. You'll see a log: "Server listening on localhost:1234". Keep this window open or minimize it; the server runs in the background.

Step 2--Install AnythingLLM

AnythingLLM offers multiple deployment options: desktop app, Docker, or self-hosted Node.js.

For a quick start, download the desktop app from GitHub (https://github.com/mintplex-labs/anything-llm/releases). Choose the installer for your OS. Windows users get an MSI; macOS users get a DMG; Linux users get an AppImage or tarball. Installation is one-click, similar to LM Studio.

If you prefer Docker, run: docker pull mintplexlabs/anythingllm and docker run -p 3001:3001 mintplexlabs/anythingllm. AnythingLLM starts on localhost:3001. This is ideal if you're running it on a server separate from your local machine.

If you're deploying to managed hosting like Opsily's AnythingLLM hosting, follow their deployment guide; they handle Docker orchestration for you.

After installation, launch AnythingLLM. You'll see a setup wizard. Create a username and password. Agree to terms. You're now in the dashboard.

Create your first workspace. A workspace in AnythingLLM is a bounded context--documents, settings, and chat history are isolated per workspace. You might have a "Legal Documents" workspace and a "Technical Docs" workspace, each with separate models and settings.

Step 3--Connect AnythingLLM to LM Studio

This is the critical bridge. You're telling AnythingLLM: "When you need a language model, talk to LM Studio."

Open AnythingLLM Settings. In the left sidebar, click Settings (gear icon). Navigate to AI Providers.

In the AI Providers list, find LM Studio. If you don't see it, you may need to select an OpenAI-compatible provider (which LM Studio acts as). Click LM Studio. You'll see a form with one field: Base URL.

If LM Studio is running on the same machine, enter: localhost:1234/v1

If LM Studio is on a different machine, replace localhost with its IP address. For example, if LM Studio is on a server at 192.168.1.100, enter: 192.168.1.100:1234/v1

Click Test Connection. AnythingLLM will ping LM Studio and confirm the connection. If it fails, check: 1) Is LM Studio's Local Server running (see Step 1)? 2) Is the IP address correct? 3) Is the port correct (default 1234)? 4) If they're on different machines, is the firewall allowing port 1234?

Once the test passes, click Save Settings. AnythingLLM now knows where to find LM Studio.

Next, go to Workspace Settings within your workspace. Under Chat Providers, select LM Studio from the dropdown. You'll see a list of models available in your LM Studio instance. Select the model you loaded earlier (e.g., Mistral 7B). Set the context window (typically 4,096 or 8,192 tokens--more tokens mean more document context but slower responses). Save.

Open the Chat tab. Start typing. AnythingLLM will send your query to LM Studio, which processes it and returns a response. You're now running private AI.

Step 4--Load Models & Configure Embeddings

You can switch between models without restarting anything. Download additional models in LM Studio (Model Library > Search > Download > Load), and they'll immediately appear in AnythingLLM's model dropdown.

Model choice depends on your use case:

• Mistral 7B: Fast, good for real-time chat.
• Llama 2 13B or Llama 3 8B: Balanced; higher quality than 7B, still fast.
• Llama 3 70B: Highest quality but slow on consumer GPUs; best on dedicated servers.

Switch models in AnythingLLM's Workspace Settings > Chat Providers. No restart needed.

Embeddings are a separate concern. Embeddings convert documents and queries into numerical vectors for semantic search. By default, AnythingLLM uses Ollama's embedding model (if Ollama is installed) or a built-in model.

To use LM Studio for embeddings as well, go to Workspace Settings > Embedding Preferences. Select "LM Studio" if it appears in the dropdown. If not, use the OpenAI-compatible option and point to localhost:1234/v1. Choose an embedding model from the available list (e.g., nomic-embed-text).

Upload documents to your workspace. AnythingLLM will embed them using your chosen provider. When you ask a question, AnythingLLM embeds your query, searches the document embeddings for matches, and sends the matching context plus your query to LM Studio's chat model. The result is contextual, private, and fast.

Test with a simple query first: upload a short document, ask a specific question about it, and verify the answer cites the document. Then scale up to larger document sets.

LM Studio vs Ollama--Which Should You Choose?

Both LM Studio and Ollama are local inference engines. If you're choosing between them, here are the tradeoffs:

Choose LM Studio if: You value a graphical interface and ease of use. You're not an engineer and prefer clicking to typing. You want to test models quickly without learning command-line tools. Your team includes non-technical users who need a model browser.

Choose Ollama if: You prefer open-source software. You're comfortable with the command line. You need tight integrations with other tools (LlamaIndex, LangChain, Hugging Face). Your stack already uses Docker and containerization. You want the lightest-weight option.

For most AnythingLLM setups, either works equally well. LM Studio is easier for on-ramps; Ollama is lighter and more integrated. If you're unsure, start with LM Studio. If you hit API limitations, switch to Ollama--AnythingLLM supports both equally.

For Teams--Managed AnythingLLM + GPU-Hosted LM Studio

Self-hosting both tools works for individuals, but teams need a different topology.

The scalable pattern: Run AnythingLLM on managed hosting (such as Opsily), and run LM Studio on a dedicated GPU server.

Why separate them? AnythingLLM is stateless--you can run multiple instances behind a load balancer, and each handles chat requests independently. LM Studio is stateful--it holds a model in memory and processes one inference request at a time (or a few in parallel, depending on GPU memory). By separating layers, you can scale AnythingLLM horizontally (more replicas) without scaling model inference proportionally. You pay for compute efficiency.

Topology: 1) LM Studio on GPU server (internal IP 192.168.1.50, port 1234). 2) AnythingLLM on managed hosting (Opsily, for example) with outbound network access. 3) AnythingLLM's AI Provider settings point to 192.168.1.50:1234/v1. 4) Users connect to AnythingLLM's public URL; all chat routed through LM Studio internally.

For inter-datacenter setup (LM Studio in one region, AnythingLLM in another), you may expose LM Studio's API via reverse proxy (nginx) and configure AnythingLLM with that public URL. This adds latency but allows geographic distribution.

Scaling LM Studio: If model inference becomes a bottleneck, add multiple GPU servers, each running LM Studio with the same models, and use a load balancer (round-robin) to distribute requests. This is complex and rarely needed for teams under 100 concurrent users.

For AnythingLLM cost analysis and scaling decisions, Opsily's hosted platform includes automated backups, SSL termination, and scaling. You manage the LM Studio GPU server separately or use a cloud GPU provider (Vast.ai, Crusoe, Lambda).

Frequently Asked Questions

Can I run LM Studio and AnythingLLM on the same machine?

Yes, if you have 16GB+ RAM and 40GB+ free disk. Allocate at least 8GB to LM Studio's model, leaving 8GB for AnythingLLM, the OS, and other processes. For production, separate machines are safer.

What models work best with LM Studio?

Mistral 7B, Llama 2 13B, and Llama 3 8B are popular. LM Studio's model browser shows download size and performance characteristics. Smaller models are faster; larger models are more capable. Start with 7B and scale up if needed.

Does AnythingLLM support LM Studio's embedding models?

Yes. In Workspace Settings > Embeddings, select LM Studio and choose an embedding model from the dropdown (e.g., nomic-embed-text).

Can I run LM Studio on a headless server without a GUI?

Yes, use the LM Studio Python SDK (https://github.com/lmstudio-ai/lmstudio-python) or the REST API directly. The desktop app requires a display, but the server component runs headless via Docker or direct API calls.

What's the latency of local inference vs cloud APIs?

Latency depends on model size and GPU capabilities. Local inference on a modern GPU is typically faster than cloud APIs for repeated queries, but cold-start time varies. Cloud APIs (OpenAI, Anthropic) have optimized infrastructure; local setups have zero marginal cost after hardware investment.

Can I use AnythingLLM without LM Studio?

Yes. AnythingLLM supports OpenAI, Claude, Anthropic, Groq, and many other providers. LM Studio is one option; it's not required. If you prefer cloud models or different inference engines (Ollama, VLLM), AnythingLLM works with all of them.

The Bottom Line

LM Studio and AnythingLLM together form a powerful, private alternative to cloud AI. LM Studio runs the language models; AnythingLLM orchestrates the retrieval and conversation logic. Setup takes under 30 minutes on a capable machine.

For small teams, run both on the same server. For larger teams, separate the inference layer (LM Studio on a GPU box) from the application layer (AnythingLLM on managed hosting). This gives you independent scaling and fault tolerance.

If you're deploying AnythingLLM at scale, Opsily's managed hosting removes operational overhead, letting you focus on models and data.

Start with the basics: install both, load a model, ask a question about a document. Once you're comfortable, explore model switching, embedding configuration, and remote setups. The learning curve is shallow, and the privacy guarantees are real.