AI News
All stories
Official

datasette-llm 0.1a11 adds support for OpenAI-compatible model endpoints

The datasette-llm plugin release lets Datasette use any OpenAI-compatible endpoint, including local servers such as LM Studio, Ollama and llama.cpp.

Open Source · · 2 min read

datasette-llm 0.1a11 lets Datasette use language models from any model server that speaks the OpenAI-style web protocol, including a model running on the user's own machine. The release was published on 14 November by Simon Willison. It is a pre-release of the plugin that configures language models for other Datasette plugins to depend on. Its headline change is that a model no longer has to be registered with a separate LLM plugin before Datasette can use it.

Willison names LM Studio, Ollama and llama.cpp as local runners for models. He notes that llama.cpp has just hit v1.0.0. The post describes the OpenAI protocol as a "pseudo-standard" web protocol. According to the release notes, an endpoint can be configured with a model reference that includes an api_base, pointing at a Chat Completions or Responses endpoint. Endpoint models support tools and streaming by default.

The example in the post runs Datasette through uvx with the datasette-agent plugin. It sets plugins.datasette-llm.default_model to a JSON object. That object gives a local ID, lmstudio-qwen, the model name qwen/qwen3.8-27b that the server expects, and an api_base of http://localhost:1234/v1. Willison's post says this produces "a local server running a Datasette Agent instance powered by your local model". The example is a configuration snippet. The post reports no test results for how the agent performs on a local model.

The release notes add several details for anyone who wants to set this up. An endpoint's local ID must not match an installed model ID or alias, and the model_name field carries the ID the remote server expects. Credentials go in a key field, which is looked up first in datasette-secrets and then in LLM's stored keys. The notes say an endpoint with no key is treated as unauthenticated and never receives a provider's default key. Users are also warned to put credentials in key rather than headers, because extra headers are visible in Datasette's /-/config.json.

The notes list several compatibility changes that could affect existing setups. Allowlists, blocklists and llm_filter_models hooks now apply to model() and group() as well as models(). As a result, selecting an excluded model raises ModelNotFoundError where it previously did not. Missing keys now fail when a model is selected, if require_keys is on, rather than when the first prompt runs. An explicit key no longer falls back to the provider's default credential. The release requires LLM 0.37.1 or newer. Willison's post does not mention any of these changes, so readers upgrading an existing configuration should read the release notes on GitHub first.

What changed

datasette-llm 0.1a11 lets Datasette use models from any OpenAI-compatible endpoint, including local servers, without first registering them with a separate LLM plugin.

Type: New capability · Applies to: datasette-llm 0.1a11, LM Studio, Ollama, llama.cpp

Why it matters

Teams that want to keep data and model calls on their own hardware can connect Datasette tools to self-hosted models without writing custom integration code.

What you can do

Teams already running Datasette with local models can test the new endpoint configuration before relying on it.

Original source

Where we saw it

1 source

Simon Willison

AI wrote this summary from the sources listed, and nobody reviewed it individually before publication. Check the original source before you rely on it. How it works