DSPy: An open-source framework for LLM-powered applications

The past year has seen explosive growth in generative AI and the tools for integrating generative AI models into applications. Developers are eager to harness large language models (LLMs) to build smarter applications, but doing so effectively remains challenging. New open-source projects are emerging to simplify this task. DSPy is one such project—a fresh framework that exemplifies current trends in making LLM app development more modular, reliable, and data-driven. This article provides an overview of DSPy, covering what it is, the problem it tackles, how it works, key use cases, and where it’s headed.

Project overview – DSPy

DSPy (short for Declarative Self-improving Python) is an open-source Python framework created by researchers at Stanford University. Described as a toolkit for “programming, rather than prompting, language models,” DSPy allows developers to build AI systems by writing compositional Python code instead of hard-coding fragile prompts. The project was open sourced in late 2023 alongside a research paper on self-improving LLM pipelines, and has quickly gained traction in the AI community.

As of this writing, the DSPy GitHub repository, which is hosted under the StanfordNLP organization, has accumulated nearly 23,000 stars and nearly 300 contributors—a strong indicator of developer interest. The project is under active development with frequent releases (version 2.6.14 was released in March 2025) and an expanding ecosystem. Notably, at least 500 projects on GitHub already use DSPy as a dependency, signaling early adoption in real-world LLM applications. In short, DSPy has rapidly moved from research prototype to one of the most-watched open-source frameworks for LLM-powered software.

What problem does DSPy solve?

Building applications with LLMs today involves a lot of prompt engineering and ad hoc orchestration. Developers using frameworks like LangChain or LlamaIndex must manually craft prompt templates and chain model calls together, introducing several pain points:

Brittle prompts and workflows. Writing prompts can be time-consuming and error-prone, and the prompts often break when you change models or inputs. Small differences in wording might yield inconsistent outputs, making maintenance a nightmare.

Lack of reusability. Prompt logic is typically embedded in code or configuration that’s hard to generalize. There’s no standardized way to reuse reasoning steps, retrieval, or other components across projects.

Scaling and optimization challenges. Improving the performance of an LLM app may require endless trial-and-error in writing prompts, providing examples, or configuring hyperparameters. Existing tools provide little automation for this process, so developers must rely on intuition and constant tweaking.

DSPy addresses these issues by shifting the paradigm from prompt hacking to high-level programming. Instead of writing one-off prompts, developers define the behavior of the AI in code (specifying model inputs, outputs, and constraints) and let DSPy handle the rest. Under the hood, DSPy will automatically optimize prompts and parameters for you, using algorithms to refine them based on feedback and desired metrics. Whenever you update your code, data, or evaluation criteria, you can recompile the DSPy program and it will re-tune the prompts to fit the changes. The framework essentially replaces manual prompt tuning with a smarter, iterative compilation process.

By replacing fragile prompts with declarative modules, DSPy makes LLM pipelines more robust to changes. It mitigates the “pipeline of sticks” problem where an update to the model or task requires rebuilding your prompt chain from scratch. In comparison to LangChain or LlamaIndex, which excel at connecting LLMs with tools and data but leave prompt crafting to the developer, DSPy provides a higher-level abstraction.

The value becomes apparent when integrating multiple steps or models: DSPy can coordinate the parts and optimize their interactions without extensive human fine-tuning. In summary, DSPy’s promise is to replace the painstaking, unscalable aspects of current LLM app development with a more systematic, maintainable approach to building AI applications.

A closer look at DSPy

How does DSPy achieve this shift from prompting to programming? The framework introduces an architecture inspired by software engineering principles and machine learning pipelines:

Modules and signatures. At the core of DSPy are modules—reusable components that encapsulate a particular strategy for invoking an LLM. You define a module by specifying its input and output interface (called a signature). For example, you might declare a module for question answering as question -> answer: text, or a math solver as question -> answer: float. DSPy expands these signatures into proper prompts and parses the model’s responses according to the expected output type. This design decouples your application logic from the raw prompt strings. You can compose multiple modules to form a pipeline, and each module can be updated or optimized independently.

Optimizers (self-improving pipelines). What differentiates DSPy is its optimizer subsystem. Optimizers are algorithms that DSPy uses to iteratively improve the prompts or even fine-tune smaller models behind the scenes. Using provided example data or even heuristic feedback, DSPy will generate variations of prompts, test them, and retain the ones that perform best on your defined success metrics. One of DSPy’s standout features is the ability to continuously refine prompts using feedback loops, leading to better model responses with each iteration. In practice, this means higher accuracy and consistency without the developer having to manually adjust prompt wording over and over.

Built-in strategies. Out of the box, DSPy provides a library of prebuilt modules for common patterns in advanced LLM usage. These include Chain of Thought (to guide the model in step-by-step reasoning), ReAct (for reasoning and acting with tools in an agent loop), and other primitives for few-shot examples, tool usage, and history/context management. The framework also integrates with external tools via its Tool and Example abstractions. For instance, you can incorporate a web search or database lookup as part of a pipeline, or enforce output validation using schemas.

Model and library support. DSPy is designed to be LLM-agnostic and work with a variety of model providers. It supports mainstream cloud APIs such as OpenAI GPT, Anthropic Claude, Databricks Dolly, etc., as well as local models running on your own hardware. Under the hood, DSPy uses a unified language model interface, and can leverage libraries like Hugging Face or the OpenAI SDK, so developers can plug in whatever model is appropriate.

Key use cases for DSPy

How can developers put DSPy into practice? Thanks to its flexible architecture, DSPy can be applied to a wide range of LLM-driven scenarios. Here are a few key use cases where the framework particularly shines:

Complex question answering with retrieval-augmented generation (RAG)

DSPy enables the creation of robust QA systems that retrieve relevant information before generating answers.

import dspy
# Configure the language model
dspy.configure(lm=dspy.LM('openai/gpt-4o-mini'))

# Define a retrieval function (e.g., search Wikipedia)
def search_wikipedia(query: str) -> list[str]:
results = dspy.ColBERTv2(url=' k=3)
return [x['text'] for x in results]

# Define the RAG module
class RAG(dspy.Module):
def __init__(self):
super().__init__()
self.retrieve = dspy.Retrieve(k=3)
self.generate = dspy.ChainOfThought('question, context -> answer')

def forward(self, question):
context = self.retrieve(question)
return self.generate(question=question, context=context)

# Instantiate the RAG module
rag = RAG()

# Example usage
question = 'What is the capital of France?'
answer = rag(question)
print(answer)

Text summarization

DSPy facilitates dynamic text summarization by defining modules that adapt to varying input lengths and styles.

import dspy

# Configure the language model
dspy.configure(lm=dspy.LM('openai/gpt-4o-mini'))

# Define the summarization module
class Summarizer(dspy.Module):
def __init__(self):
super().__init__()
self.summarize = dspy.ChainOfThought('document -> summary')

def forward(self, document):
return self.summarize(document=document)

# Instantiate the summarizer
summarizer = Summarizer()

# Example usage
document = 'DSPy is a framework for programming language models...'
summary = summarizer(document)
print(summary)

LLM agents with tool integration

DSPy supports building AI agents that can reason and interact with external tools to perform tasks.

import dspy

# Configure the language model
dspy.configure(lm=dspy.LM('openai/gpt-4o-mini'))

# Define a custom tool (e.g., a calculator)
def calculator(expression: str) -> float:
return eval(expression)

# Define the agent module
class Agent(dspy.Module):
def __init__(self):
super().__init__()
self.react = dspy.ReAct('question -> answer', tools=[calculator])

def forward(self, question):
return self.react(question=question)

# Instantiate the agent
agent = Agent()

# Example usage
question = 'What is 2 plus 2?'
answer = agent(question)
print(answer)

Bottom line – DSPy

DSPy is an ambitious framework that pushes the envelope of what development frameworks for LLMs can do. It addresses real pain points by making LLM applications more declarative, modular, and self-improving. By decoupling application logic from prompt strings, DSPy allows developers to combine modules into pipelines, update or optimize modules independently, and even continuously optimize modules using feedback loops. While still young, DSPy has a strong potential to become a go-to solution for building complex LLM-powered applications.