Appsmith AI Guide: Build AI Apps After the 2026 EOL

In September 2026, Appsmith deprecated its built-in AI datasource. You can still build AI-powered apps with Appsmith, but you now connect to external LLM providers like OpenAI, Anthropic, or Google AI instead. This guide walks you through the setup, from securing API keys to building a fully functional AI chat widget, no coding experience required.

What Happened to Appsmith's Built-In AI?

On September 30, 2026, Appsmith discontinued its native AI datasource, which meant users lost access to no-code AI query functionality without an API key. The company redirected users to external LLM providers, shifting responsibility to you to choose a provider and manage credentials. This is not a limitation; it is actually more flexible. You now pick the LLM that fits your use case: OpenAI for general text and reasoning, Anthropic for long context windows and accuracy, Google AI for multimodal work, or Cohere for cost-conscious workloads.

The EOL happened fast, with minimal transition period. If you were already using Appsmith's built-in AI for a production application, you had to migrate immediately. Most users moved to OpenAI, simply because it is mature, widely documented, and the default choice for Appsmith community examples. Anthropic Claude is gaining traction for tasks requiring reasoning or analysis. Google AI (Gemini) covers cases where you need image input or a lower cost tier.

This shift also opened up the ability to run private, self-hosted LLMs via APIs. You can now connect to Ollama, LocalAI, or vLLM running on your own infrastructure if you need full data privacy. Appsmith does not lock you into a specific provider; you are in complete control.

Choosing Your LLM Provider: OpenAI vs Anthropic vs Google AI

The choice of LLM provider depends on three factors: cost, capability, and privacy. Here is the honest breakdown.

OpenAI is the safest first choice. Its GPT-4 and GPT-4o models are the most familiar to developers and product teams. Most Appsmith documentation examples and community templates use OpenAI. Pricing is competitive at $0.03-$0.15 per 1,000 input tokens and $0.06-$0.60 per 1,000 output tokens, depending on the model. You get fast inference, reliable uptime, and a huge number of integrations. The downside: you send your data to OpenAI servers. If your company has strict data residency rules or compliance requirements, OpenAI may not be approved.

Anthropic's Claude excels at long-context reasoning and is harder to jailbreak. Claude 3.5 Sonnet costs $3 per 1M input tokens and $15 per 1M output tokens, cheaper than GPT-4 for large documents. The real win is context window: Claude handles up to 200,000 tokens in a single request, versus OpenAI's 128,000. If you are building a RAG system that grounds responses in large knowledge bases or long documents, Claude is often the better pick. Like OpenAI, Anthropic Cloud-hosted models send data to external servers.

Google AI (Gemini) is the multimodal option. Gemini 2.0 Flash costs $0.075 per 1M input tokens and $0.30 per 1M output tokens, fast and cheap for text. Gemini also processes images, PDFs, and video natively, powerful if you are building document analysis tools. The trade-off: fewer Appsmith examples exist for Gemini, so you may need to troubleshoot integration yourself.

Cohere serves cost-sensitive workloads and can be deployed on your own infrastructure. Command R is around $0.50 per 1M tokens. Use Cohere if you have massive token volume and need to reduce API spend by 80% compared to OpenAI.

For most teams building an internal AI chat tool or knowledge base assistant, OpenAI or Claude is the right call. Pick OpenAI if you want the most documentation. Pick Claude if you have long documents or need better privacy controls.

Step 1: Getting Your API Key

Every LLM provider requires an API key to authenticate requests. Here is how to set it up.

For OpenAI:

1. Go to platform.openai.com/account/api-keys.
2. Click "Create new secret key."
3. Name it (e.g., "Appsmith Production").
4. Copy the key and save it somewhere secure.
5. Set a monthly usage limit under "Billing" to prevent surprise charges.

For Anthropic:

1. Go to console.anthropic.com/account/keys.
2. Click "Create key."
3. Copy it and store it securely.
4. Set up a workspace budget limit.

For Google AI:

1. Go to aistudio.google.com/app/apikey.
2. Click "Create API key in a new Google Cloud project."
3. Copy the key.

Do not paste the key directly into an Appsmith text field. Appsmith stores values in plaintext by default, which is a security risk. Instead, use environment variables or Appsmith's Secrets feature to store the key separately. For Opsily-managed Appsmith, we handle this for you: store secrets in your instance's environment configuration, and Appsmith accesses them via process.env variables.

Step 2: Creating an LLM Query in Appsmith

In Appsmith, you now add an LLM query just like any other API query. Here is the workflow:

1. Go to your Appsmith app and click the "+" icon in the Queries panel (bottom left).

2. Select "API" as the query type.

3. Set the request type to POST.

4. Enter the API endpoint for your chosen provider (see examples below).

5. Set headers:

   • Content-Type: application/json
   • Authorization: Bearer YOUR_API_KEY

6. In the body, define the LLM parameters. For OpenAI:

{
  "model": "gpt-4o-mini",
  "messages": [
    {
      "role": "user",
      "content": "{{ userMessage.value }}"
    }
  ],
  "temperature": 0.7,
  "max_tokens": 500
}

7. Click Save and Run to test it.

If the query fails, check: Is the API key valid? (Try it in curl first.) Are the headers spelled correctly? Is the endpoint URL correct? Does your account have an active billing method and positive balance?

Step 3: Building Your Chat UI

You now design the user interface for the chat. Here is the minimal setup:

Components needed:

1. A Text Input widget for user messages. Name it userMessage.
2. A Button widget labeled "Send." Name it sendButton.
3. A List widget to display chat history. Name it chatHistory.

Configure the Text Input:

• Set Placeholder to "Ask me anything."
• Leave Clearable enabled so it clears after sending.

Configure the Button:

• Set it to trigger the LLM query on click.
• In the onClick handler, run: llmQuery.trigger(); then userMessage.clearValue();

Configure the List Widget:

• Set Data Source to: {{ chatHistory.value }}
• In the list template, add two Text widgets:
  • One for the role: {{ currentItem.role }}
  • One for the message: {{ currentItem.content }}
• Style them so user messages align right and assistant messages align left.

Test by clicking Send. You should see the text input clear and the LLM response appear in the list.

Step 4: Managing Chat History with JSObject

The raw response from the LLM is just one message. A chat widget requires a state that grows with each exchange. Use Appsmith's JSObject to manage this.

In Appsmith, create a JSObject named chatState with this code:

export default {
  messages: [],
  
  addUserMessage(text) {
    this.messages.push({
      role: "user",
      content: text
    });
  },
  
  addAssistantMessage(text) {
    this.messages.push({
      role: "assistant",
      content: text
    });
  },
  
  getHistory() {
    return this.messages;
  },
  
  clearHistory() {
    this.messages = [];
  }
};

Now bind the chatHistory widget to {{ chatState.getHistory() }}.

Modify the LLM query body to include the full message history:

{
  "model": "gpt-4o-mini",
  "messages": {{ chatState.messages }},
  "temperature": 0.7,
  "max_tokens": 500
}

Update the sendButton onClick handler:

chatState.addUserMessage(userMessage.value);
llmQuery.trigger();
chatState.addAssistantMessage(llmQuery.data.choices[0].message.content);
userMessage.clearValue();

Now each turn adds both user and assistant messages to the history, and the next request includes the full conversation. Multi-turn conversations work.

Real-World Example: Building a RAG Chat Widget

RAG stands for Retrieval-Augmented Generation. Instead of letting the LLM hallucinate, you ground its responses in your own documents.

Here is the pattern:

1. Upload your knowledge base (PDFs, markdown files, or text) to a database or object storage (S3, Google Cloud Storage).
2. Before sending a user query to the LLM, retrieve the most relevant documents using semantic search (embeddings).
3. Include those documents in the system prompt.
4. The LLM generates an answer based only on the retrieved documents.

Example system prompt:

You are a helpful assistant. Answer the user's question using only the context provided below. If the answer is not in the context, say "I don't have that information in my knowledge base."

Context:
{{ retrievedDocuments.map(doc => doc.text).join("\n\n") }}

User question:
{{ userMessage.value }}

To implement this in Appsmith:

1. Create a search query that retrieves documents matching the user input. This can be a database query or API call to a vector database (Pinecone, Weaviate, Milvus).
2. Modify the LLM query to include {{ retrievedDocuments }} in the system prompt.
3. Set the LLM query to only run after the search query completes.

For teams without a vector database, a simpler approach works: upload documents to an Appsmith database or Google Sheets, and use keyword search to retrieve the top 3 matching documents. Pair this with a good LLM system prompt, and you get 80% of the value of full semantic search. This pattern is how Appsmith community members have built employee handbook assistants, product documentation bots, and FAQ chatbots.

Cloud vs Self-Hosted: Where to Deploy Your AI App

Appsmith offers two deployment models: Cloud (managed) and self-hosted.

Appsmith Cloud is the managed SaaS version. You sign up, create an app, and it runs on Appsmith's servers. AI features work out of the box since Appsmith handles infrastructure. You do not manage servers, backups, or updates. However, you are subject to Appsmith's uptime SLAs and data storage policies. Appsmith Cloud is free to start and paid plans begin at $0 per month for personal projects.

Self-hosted Appsmith runs on your own infrastructure: Docker, Kubernetes, or a VPS. You get full control over data residency and compliance. Appsmith stores app definitions and query results on your servers. This is critical if you handle sensitive customer data or operate under compliance requirements. The trade-off is operational burden: you manage Docker and Kubernetes, SSL certificates, backups, database upgrades, and patches.

For AI-powered apps specifically, both Cloud and self-hosted support external LLM APIs equally well. The LLM API call goes from Appsmith to OpenAI or Anthropic servers, not to Appsmith's infrastructure. So your LLM data flows the same way in Cloud or self-hosted. The difference is where Appsmith itself runs and where your chat history is stored.

Start with Appsmith Cloud for prototyping. Once your AI app is production-ready and handling real customer data, migrate to self-hosted or a managed provider. Opsily offers managed self-hosted Appsmith: we handle the infrastructure, upgrades, backups, and monitoring. You keep full control over your data and can deploy on your own cloud or our private infrastructure. This is the middle ground: all the operational benefits of a managed service, all the control of self-hosted.

Troubleshooting Common AI Setup Issues

Problem: "Invalid API key" error.

• Double-check that you copied the key correctly and the quotation marks are not included.
• Verify the key is active (not deleted or revoked) in your provider dashboard.
• Check that the header format is correct: Authorization: Bearer sk-... (not "Bearer: sk-...").

Problem: Requests time out or fail intermittently.

• OpenAI and Anthropic have rate limits. If you are sending 100 concurrent requests, some will fail.
• Add a 1-2 second delay between requests or use exponential backoff.
• Check your account balance and billing method.

Problem: Chat widget shows old responses even after sending a new message.

• Verify that chatState.addAssistantMessage() is being called in the button handler.
• Check that the chatHistory widget is bound to {{ chatState.getHistory() }}, not a static value.
• In browser dev tools, console.log(chatState.messages) to see if it is actually growing.

Problem: Responses are cut off or seem incomplete.

• Raise the max_tokens parameter in your LLM query (default 500 tokens = ~375 words).
• For long responses, set max_tokens to 2000 or higher.
• Check the LLM provider's output token limits for your account tier.

Problem: Appsmith app is slow or freezes during AI queries.

• LLM API calls can take 3-10 seconds. During this time, the UI feels frozen.
• Add a loading indicator: enable a Skeleton widget or Spinner while llmQuery is running.
• Use Appsmith's showNotification() to confirm the user's message was sent immediately.

What is Next: Agents and Autonomous Workflows

The next evolution of AI in Appsmith is agents: LLMs with access to tools and the ability to chain actions. An agent can decide to retrieve a document, query a database, call a webhook, and generate a response, all autonomously.

Example: A customer support agent receives a question, looks up the customer in Salesforce, retrieves their support tickets from your database, searches your knowledge base, and generates a personalized response. All in one LLM call.

Appsmith does not have built-in agent scaffolding yet, but you can build this yourself:

1. Define a set of tools (functions) the LLM can call: retrieve_docs, query_database, send_email.
2. Send the LLM an initial prompt that lists these tools and their descriptions.
3. Parse the LLM response to see if it requested a tool call.
4. Execute that tool in Appsmith.
5. Send the tool result back to the LLM and let it decide the next step.

This pattern is called function calling or tool use. OpenAI and Anthropic both support it natively. It is more complex than a simple chat widget, but the payoff is huge: you can automate workflows, answer questions with live data, and build copilots that handle multi-step tasks.

If building an agent feels daunting, consider using a hosted agent platform like Langchain, CrewAI, or Anthropic's native agents API. These handle orchestration complexity. Appsmith can then call your agent as a webhook and display the result.

Frequently Asked Questions

Q: Do I need to write code to use AI in Appsmith? A: No. For basic chat widgets, you can build the entire UI using visual components. You only need JavaScript if you want custom state management, which is a copy-paste pattern we showed you above. For advanced features like function calling or agent loops, you will write some JavaScript, but it is usually under 50 lines.

Q: Which LLM provider should I choose for a production app? A: Start with OpenAI if you want the most documentation and examples. GPT-4o-mini is cheap and fast. Move to Anthropic if you have long documents, need better reasoning, or have strict data privacy rules. Google AI is best for multimodal work. For cost-sensitive workloads at scale, Cohere. Test with your actual data before committing.

Q: Is my chat data secure? A: Your chat history lives in Appsmith, which stores it according to your deployment: Cloud SaaS (on Appsmith servers) or self-hosted (on your servers). Each message you send to the LLM provider goes to that provider's API. Those providers log and may use your data unless you opt out. Check the privacy policy of your LLM provider. If you cannot send data to external APIs, use a self-hosted LLM like Ollama or vLLM.

Q: Can I use the same LLM query for multiple widgets? A: Yes. Define the query once and call it from multiple components: {{ myLlmQuery.trigger() }}. You can also pass different parameters each time. This saves you from duplicate queries and makes the app easier to update.

Q: What happens if the LLM API is down? A: Your Appsmith app will not break, but the query will fail. Add error handling: if (llmQuery.isLoading) { show loading }; if (llmQuery.data) { show response }. Display a friendly message to the user instead of a raw error.

Q: Can I use Appsmith for image generation (DALL-E, Midjourney)? A: Appsmith does not have built-in image generation. You can make API calls to DALL-E or Midjourney just like you do for chat, but you need to handle authentication and webhook polling yourself. OpenAI's DALL-E 3 API is the easiest: send a prompt, get back an image URL.

The Bottom Line

Appsmith's EOL of its native AI datasource was disruptive but ultimately liberating. You now have choice: pick OpenAI for simplicity, Claude for long documents, Google AI for images, or Cohere for cost. Setting up a basic AI chat widget takes 15 minutes. Building a RAG system that grounds answers in your own documents takes a few hours. Multi-turn conversations are native once you manage chat history with a JSObject. The real constraint is not Appsmith, it is your LLM choice. Get the API key, test it, then wire it into Appsmith. Ready to build an AI-powered internal tool? Deploy Appsmith on Opsily's managed hosting and let our team support your infrastructure.