翻译自https://levelup.gitconnected.com/dspy-is-all-you-need-drop-langchain-now-1-5914ac4e31bb
提示工程的状态并不强大。只需看看我们现在必须包含在提示中的信息，以引出我们想要的回应。这既荒谬又令人担忧。拟人化的程度已经高得离谱。我们现在必须贿赂 LLMs？得了吧。
从一开始，我就发现提示工程有一些严重的局限性。同一个提示并不总是产生相似的或甚至是连贯的结果。改变一个词可能会对输出产生巨大的影响。提示工程往往太脆弱，太不可靠。我们不能在沙子上建造未来的系统，更不用说人工通用智能（AGI）了。顺便说一下，我最近提出，我们可能甚至无法建造任何 AGI。
我的核心信念一直是，我们越快消除对提示工程花招的过度依赖，我们就会越好。这就是为什么现在是摆脱 Langchain 的时候。LangChain 建立在严重依赖提示工程的想法上。那不是通往未来的路。
我最近开始使用 DSPy，这是一个美丽的框架，它通过为你生成和优化提示来减少你对提示工程的依赖。DSPy 非常强调编程而不是提示。那才是通往未来的路。每个 AI 工程师现在都应该放弃 LangChain，使用 DSPy！！！现在就行动！！！！！在接下来的几行中，我将解释为什么？
你准备好迎接一次令人震惊的体验了吗？
DSPy 集成了提示、微调和增强语言模型（LMs）的方法，通过推理和工具/检索增强。这是通过 Python 中的一系列流畅的操作实现的，这些操作是可组合的。
通过用户友好的 Pythonic 语法，DSPy 提供了模块化和声明式的组件来指导 LMs。此外，DSPy 还具有创新的自动编译器。这个编译器解释你程序中的声明式程序，使其能够为更大的 LMs 生成复杂的提示或为较小的 LMs 进行自动微调，有效地指导它们完成特定任务的过程。
如果您想更多了解 DSPy，可以在 www.lycee.ai 上找到惊人的资源，这是第一个专注于 AI 内容的学习管理系统。
DSPy Setup on Google Colab
在 Google Colab 上设置 DSPy
这里没有必要重新发明轮子，我只是采用了 DSPy 文档中介绍的那个。它非常完美。
脚本首先启用模块的自动重新加载，确保代码中的更改在不重启笔记本的情况下得到反映。然后，脚本检查是否在 Google Colab 中运行；如果是，它通过从现有存储库拉取更新或从指定的 GitHub URL 克隆来设置存储库 ‘dspy’。这确保始终使用 DSPy 的最新版本。
此外，脚本将存储库路径添加到系统路径，并为 DSPy 的缓存目录设置环境变量。它还检查是否安装了 ‘dspy-ai’ 包，如果没有，它会安装或更新它。这一步确保 DSPy 环境完全配备了所有必要的依赖项。最后，它导入了 ‘dspy’ 模块，表明设置过程已经完成，DSPy 已准备好在笔记本中使用。
对于我们的实验，我们需要访问一个语言模型和一个检索器。在这种情况下，我们将使用 OpenAI 和 Colbert。对于某些高级用例，可以使用自定义检索器模型。
For every task, DSPy workflow includes collecting data, writing a program, adding validation logic, compiling and finally iterating to improve the workflow.
对于每项任务，DSPy 工作流程包括收集数据、编写程序、添加验证逻辑、编译，最后迭代以改进工作流程。
在 Google Colab 上设置 DSPy
这里没有必要重新发明轮子，我只是采用了 DSPy 文档中介绍的那个。它非常完美。脚本首先启用模块的自动重新加载，确保代码中的更改在不重启笔记本的情况下得到反映。然后，脚本检查是否在 Google Colab 中运行；如果是，它通过从现有存储库拉取更新或从指定的 GitHub URL 克隆来设置存储库 ‘dspy’。这确保始终使用 DSPy 的最新版本。
此外，脚本将存储库路径添加到系统路径，并为 DSPy 的缓存目录设置环境变量。它还检查是否安装了 ‘dspy-ai’ 包，如果没有，它会安装或更新它。这一步确保 DSPy 环境完全配备了所有必要的依赖项。最后，它导入了 ‘dspy’ 模块，表明设置过程已经完成，DSPy 已准备好在笔记本中使用。
%load_ext autoreload
%autoreload 2
import sys
import os
try:
import google.colab
repo_path = 'dspy'
!git -C $repo_path pull origin || git clone https://github.com/standfordnlp/dspy $repo_path
except:
repo_path = '.'
if repo_path not in sys.path:
sys.path.append(repo_path)
os.environ["DSP_NOTEBOOK_CACHEDIR"] = os.path.join(repo_path, 'cache')
import pkg_resources
if not "dspy-ai" in {pkg.key for pkg in pkg_resources.working_set}:
!pip install -U pip
!pip install dspy-ai
import dspy
对于我们的实验，我们需要访问一个语言模型和一个检索器。在这种情况下，我们将使用 OpenAI 和 Colbert。对于某些高级用例，可以使用自定义检索器模型。
llm = dspy.OpenAI(model='gpt-3.5-turbo')
colbertv2_wiki = dspy.ColBERTv2(url='http://20.102.90.50:2017/wiki17_abstracts')
dspy.settings.configure(lm=llm, rm=colbertv2_wiki)
对于每项任务，DSPy 工作流程包括收集数据、编写程序、添加验证逻辑、编译，最后迭代以改进工作流程
使用 DSPy 进行简单的 RAG
DSPy 可以用于各种任务。在本文中，我将向您展示如何根据 DSPy 的文档实现一个简单的 RAG。这是代码。之后，可以找到详细的解释。
from dspy.datasets import HotPotQA
# Load the dataset
dataset = HotPotQA(train_seed=1, train_size=20, eval_seed=2024, dev_size=50, test_size=0)
# Tell DSPy that the 'question' field is the input. Any ohter fields are labels and/or metadata.
trainset = [x.with_inputs('question') for x in dataset.train]
devset = [x.with_inputs('question') for x in dataset.dev]
class GenerateAnswer(dspy.Signature):
"Answer with long and detailled answers"
context = dspy.InputField(desc="may content relevant facts")
question = dspy.InputField()
answer = dspy.OutputField(desc="often between 10 and 50 words")
class RAG(dspy.Module):
def __init__(self, num_passages=3):
super().__init__()
self.retrieve = dspy.Retrieve(k=num_passages)
self.generate_answer = dspy.ChainOfThought(GenerateAnswer)
def forward(self, question):
context = self.retrieve(question).passages
prediction = self.generate_answer(context=context, question=question)
return dspy.Prediction(context=context, answer=prediction.answer)
from dspy.teleprompt import BootstrapFewShot
def validate_context_and_answer(example, pred, trace=None):
answer_EM = dspy.evaluate.answer_exact_match(example, pred)
answer_PM = dspy.evaluate.answer_passage_match(example, pred)
return answer_EM and answer_PM
teleprompter = BootstrapFewShot(metric=validate_context_and_answer)
compiled_rag = teleprompter.compile(RAG(), trainset=trainset)
my_question = "What castle did David Gregory inherit"
pred = compiled_rag(my_question)
# Print the contexts and the answer.
print(f"Question: {my_question}")
print(f"Predicted Answer: {pred.answer}")
print(f"Retrieved Contexts (truncated): {[c[:200] + '...' for c in pred.context]}")
上面的脚本首先从 DSPy 的数据集模块导入 HotPotQA 数据集。然后，它使用指定的训练、评估和开发集参数初始化一个数据集对象。通过将‘问题’字段标记为输入，而其他字段被视为标签或元数据，来准备训练和发展集。
然后，脚本定义了两个类： GenerateAnswer 和 RAG 。 GenerateAnswer 是 DSPy 中的一个自定义签名，它指定了生成答案的输入和输出字段，以及详细的描述。 RAG （检索增强生成）是一个模块，它使用 DSPy 的 Retrieve 和 ChainOfThought 功能。它为给定的问题检索上下文，并使用指定的 GenerateAnswer 结构生成答案。
最后，脚本涉及使用 DSPy 的 BootstrapFewShot 类设置一个引导式的少量样本学习方法。这涉及将 RAG 模型与训练集编译在一起，然后使用这个编译好的模型来预测问题的答案。脚本包括一个自定义的验证函数，用于上下文和答案准确性。最后，它展示了如何使用模型预测给定问题的答案，以及如何显示检索到的上下文和预测的答案。
llm.inspect_history(n=1)
这是 DSPy 在后台自动创建的提示的一个示例：
Answer with long and detailled answers
---
Question: At My Window was released by which American singer-songwriter?
Answer: John Townes Van Zandt
Question: "Everything Has Changed" is a song from an album released under which record label ?
Answer: Big Machine Records
Question: The Victorians - Their Story In Pictures is a documentary series written by an author born in what year?
Answer: 1950
Question: Which Pakistani cricket umpire who won 3 consecutive ICC umpire of the year awards in 2009, 2010, and 2011 will be in the ICC World Twenty20?
Answer: Aleem Sarwar Dar
Question: Having the combination of excellent foot speed and bat speed helped Eric Davis, create what kind of outfield for the Los Angeles Dodgers?
Answer: "Outfield of Dreams"
Question: Who is older, Aleksandr Danilovich Aleksandrov or Anatoly Fomenko?
Answer: Aleksandr Danilovich Aleksandrov
Question: The Organisation that allows a community to influence their operation or use and to enjoy the benefits arisingwas founded in what year?
Answer: 2010
Question: Tombstone stared an actor born May 17, 1955 known as who?
Answer: Bill Paxton
Question: In what year was the club founded that played Manchester City in the 1972 FA Charity Shield
Answer: 1874
Question: which American actor was Candace Kita guest starred with
Answer: Bill Murray
Question: Which is taller, the Empire State Building or the Bank of America Tower?
Answer: The Empire State Building
Question: What is the code name for the German offensive that started this Second World War engagement on the Eastern Front (a few hundred kilometers from Moscow) between Soviet and German forces, which included 102nd Infantry Division?
Answer: Operation Citadel
---
Follow the following format.
Context: may content relevant facts
Question: ${question}
Reasoning: Let's think step by step in order to ${produce the answer}. We ...
Answer: often between 1 and 5 words
---
Context:
[1] «Tae Kwon Do Times | Tae Kwon Do Times is a magazine devoted to the martial art of taekwondo, and is published in the United States of America. While the title suggests that it focuses on taekwondo exclusively, the magazine also covers other Korean martial arts. "Tae Kwon Do Times" has published articles by a wide range of authors, including He-Young Kimm, Thomas Kurz, Scott Shaw, and Mark Van Schuyver.»
[2] «Kwon Tae-man | Kwon Tae-man (born 1941) was an early Korean hapkido practitioner and a pioneer of the art, first in Korea and then in the United States. He formed one of the earliest dojang's for hapkido in the United States in Torrance, California, and has been featured in many magazine articles promoting the art.»
[3] «Hee Il Cho | Cho Hee Il (born October 13, 1940) is a prominent Korean-American master of taekwondo, holding the rank of 9th "dan" in the martial art. He has written 11 martial art books, produced 70 martial art training videos, and has appeared on more than 70 martial arts magazine covers. Cho won several national and international competitions as a taekwondo competitor, and has appeared in several films, including "Fight to Win", "Best of the Best", "Bloodsport II", and "Bloodsport III". He founded the Action International Martial Arts Association (AIMAA) in 1980, and is its President. Cho is a member of both "Black Belt" magazine's Hall of Fame and "Tae Kwon Do Times" magazine's Hall of Fame.»
Question: Which magazine has published articles by Scott Shaw, Tae Kwon Do Times or Southwest Art?
Reasoning: Let's think step by step in order to produce the answer. We know from the context that "Tae Kwon Do Times" is a magazine that focuses on taekwondo and other Korean martial arts. It has published articles by authors such as Scott Shaw. On the other hand, we don't have any information about Southwest Art magazine and whether it has published articles by Scott Shaw or not.
Answer: Tae Kwon Do Times
---
Context:
[1] «Rosario Dawson | Rosario Isabel Dawson (born May 9, 1979) is an American actress, producer, singer, comic book writer, and political activist. She made her film debut in the 1995 teen drama "Kids". Her subsequent film roles include "He Got Game", "Men in Black II", "25th Hour", "Rent", "Sin City", "Death Proof", "Seven Pounds", "", and "Top Five". Dawson has also provided voice-over work for Disney and DC.»
[2] «Sarai Gonzalez | Sarai Isaura Gonzalez (born 2005) is an American Latina child actress who made her professional debut at the age of 11 on the Spanish-language ""Soy Yo"" ("That's Me") music video by Bomba Estéreo. Cast as a "nerdy" tween with a "sassy" and "confident" attitude, her performance turned her into a "Latina icon" for "female empowerment, identity and self-worth". She subsequently appeared in two get out the vote videos for Latinos in advance of the 2016 United States elections.»
[3] «Gabriela (2001 film) | Gabriela is a 2001 American romance film, starring Seidy Lopez in the title role alongside Jaime Gomez as her admirer Mike. The film has been cited as an inspiration behind the Premiere Weekend Club, which supports Latino film-making.»
Question: Which American actress who made their film debut in the 1995 teen drama "Kids" was the co-founder of Voto Latino?
Reasoning: Let's think step by step in order to produce the answer. We know that the actress we are looking for made her film debut in the 1995 teen drama "Kids". From the context, we also know that she is an American actress and that she co-founded Voto Latino.
Answer: Rosario Dawson
---
Context:
[1] «Kerry Condon | Kerry Condon (born 4 January 1983) is an Irish television and film actress, best known for her role as Octavia of the Julii in the HBO/BBC series "Rome," as Stacey Ehrmantraut in AMC's "Better Call Saul" and as the voice of F.R.I.D.A.Y. in various films in the Marvel Cinematic Universe. She is also the youngest actress ever to play Ophelia in a Royal Shakespeare Company production of "Hamlet."»
[2] «Corona Riccardo | Corona Riccardo (c. 1878October 15, 1917) was an Italian born American actress who had a brief Broadway stage career before leaving to become a wife and mother. Born in Naples she came to acting in 1894 playing a Mexican girl in a play at the Empire Theatre. Wilson Barrett engaged her for a role in his play "The Sign of the Cross" which he took on tour of the United States. Riccardo played the role of Ancaria and later played Berenice in the same play. Robert B. Mantell in 1898 who struck by her beauty also cast her in two Shakespeare plays, "Romeo and Juliet" and "Othello". Author Lewis Strang writing in 1899 said Riccardo was the most promising actress in America at the time. Towards the end of 1898 Mantell chose her for another Shakespeare part, Ophelia im Hamlet. Afterwards she was due to join Augustin Daly's Theatre Company but Daly died in 1899. In 1899 she gained her biggest fame by playing Iras in the first stage production of Ben-Hur.»
[3] «Judi Dench | Dame Judith Olivia "Judi" Dench, {'1': ", '2': ", '3': ", '4': "} (born 9 December 1934) is an English actress and author. Dench made her professional debut in 1957 with the Old Vic Company. Over the following few years, she performed in several of Shakespeare's plays in such roles as Ophelia in "Hamlet", Juliet in "Romeo and Juliet", and Lady Macbeth in "Macbeth". Although most of her work during this period was in theatre, she also branched into film work and won a BAFTA Award as Most Promising Newcomer. She drew strong reviews for her leading role in the musical "Cabaret" in 1968.»
Question: Who acted in the shot film The Shore and is also the youngest actress ever to play Ophelia in a Royal Shakespeare Company production of "Hamlet." ?
Reasoning: Let's think step by step in order to produce the answer. We know that the actress we are looking for is the youngest actress ever to play Ophelia in a Royal Shakespeare Company production of "Hamlet." We also know that she acted in the short film "The Shore."
Answer: Kerry Condon
---
Context:
[1] «Samantha Cristoforetti | Samantha Cristoforetti (] ; born 26 April 1977 in Milan) is an Italian European Space Agency astronaut, Italian Air Force pilot and engineer. She holds the record for the longest uninterrupted spaceflight of a European astronaut (199 days, 16 hours), and until June 2017 held the record for the longest single space flight by a woman until this was broken by Peggy Whitson. She is also the first Italian woman in space. Samantha Cristoforetti is also known as the first person who brewed an espresso coffee in space.»
[2] «ISSpresso | ISSpresso is the first espresso coffee machine designed for use in space, produced for the International Space Station by Argotec and Lavazza in a public-private partnership with the Italian Space Agency (ASI). The first espresso coffee was drunk in space by astronaut Samantha Cristoforetti on 3 May 2015. ISSpresso is one of nine experiments selected by the Italian Space Agency for the Futura mission.»
[3] «Mark Shuttleworth | Mark Richard Shuttleworth (born 18 September 1973) is a South African entrepreneur who is the founder and CEO of Canonical Ltd., the company behind the development of the Linux-based Ubuntu operating system. In 2002, he became the first citizen of an independent African country to travel to space as a space tourist. He currently lives on the Isle of Man and holds dual citizenship from South Africa and the United Kingdom.»
Question: Samantha Cristoforetti and Mark Shuttleworth are both best known for being first in their field to go where?
Reasoning: Let's think step by step in order to produce the answer. 1. Samantha Cristoforetti: She is an Italian astronaut who holds the record for the longest uninterrupted spaceflight of a European astronaut. She is also known for being the first Italian woman in space and for brewing the first espresso coffee in space. 2. Mark Shuttleworth: He is a South African entrepreneur and the founder of Canonical Ltd., the company behind the development of the Ubuntu operating system. In 2002, he became the first citizen of an independent African country to travel to space as a space tourist. Based on this information, both Samantha Cristoforetti and Mark Shuttleworth are best known for being the first in their field to go to space.
Answer: Space
---
Context:
[1] «David Gregory (physician) | David Gregory (20 December 1625 – 1720) was a Scottish physician and inventor. His surname is sometimes spelt as Gregorie, the original Scottish spelling. He inherited Kinnairdy Castle in 1664. Three of his twenty-nine children became mathematics professors. He is credited with inventing a military cannon that Isaac Newton described as "being destructive to the human species". Copies and details of the model no longer exist. Gregory's use of a barometer to predict farming-related weather conditions led him to be accused of witchcraft by Presbyterian ministers from Aberdeen, although he was never convicted.»
[2] «Gregory Tarchaneiotes | Gregory Tarchaneiotes (Greek: Γρηγόριος Ταρχανειώτης , Italian: "Gregorio Tracanioto" or "Tracamoto" ) was a "protospatharius" and the long-reigning catepan of Italy from 998 to 1006. In December 999, and again on February 2, 1002, he reinstituted and confirmed the possessions of the abbey and monks of Monte Cassino in Ascoli. In 1004, he fortified and expanded the castle of Dragonara on the Fortore. He gave it three circular towers and one square one. He also strengthened Lucera.»
[3] «Gregory of Gaeta | Gregory was the Duke of Gaeta from 963 until his death. He was the second son of Docibilis II of Gaeta and his wife Orania. He succeeded his brother John II, who had left only daughters. Gregory rapidly depleted the "publicum" (public land) of the Duchy of Gaeta by doling it out to family members as grants. Gregory disappears from the records in 964 and was succeeded by his younger brother Marinus of Fondi over the heads of his three sons. It is possible that there was an internal power struggle between factions of the Docibilan family and that Gregory was forced out. On the other hand, perhaps he died and his sons fought a losing battle for their inheritance to Gaeta.»
Question: What castle did David Gregory inherit
Reasoning: Let's think step by step in order to produce the answer. We know that David Gregory was a Scottish physician and inventor who inherited a castle. From the context, we also know that the castle he inherited is called Kinnairdy Castle.
Answer: Kinnairdy Castle
The truth about DSPy DSPy 的真相
现在，DSPy 的真相是它在后台仍然使用了一些提示工程。例如，当你定义一个签名类时，你必须包括任务或字段的文本描述。那仍然是提示工程。在下面的 GenerateAnswer 类中，“用长而详细的答案回答”，“可能包含相关事实”，和“通常在 10 到 50 个单词之间”仍然是提示。
class GenerateAnswer(dspy.Signature):
"Answer with long and detailled answers"
context = dspy.InputField(desc="may content relevant facts")
question = dspy.InputField()
answer = dspy.OutputField(desc="often between 10 and 50 words")
DSPy 只是使创建带有少量示例的提示自动化。否则，你将不得不编写自己的提示并选择一些示例包括在提示中以引出更好的响应。DSPy 使用一个根据你提供的示例数据更新的后台提示模板。
如果你查看 DSPy 的代码，你会发现他们仍然在底层使用文本提示，比如“推理：让我们一步步来思考”。我猜只要你使用语言模型，这就是不可避免的，对吧？
到目前为止，我的信念是，DSPy 只是基于这样一个想法：要使你的提示更有效，你必须在其中提供一些示例。所以 DSPy 似乎在任何地方都使用这个想法，因此专注于使使用这些示例生成提示更容易。你可以用 LangChain 做同样的事情，只是方式不同。我觉得 DSPy 有趣的地方是他们在检索领域的专业知识，和 Colbert 以及同事们一起。
DSPy 和 LangChain 之间的差异更多在于如何提示，而不是我们是否应该提示。DSPy 旨在保持最低限度的手动提示。DSPy 的优势在于，只要您有一些数据，它就会通过智能地添加少量示例来创建提示，从而提高效率。此外，DSPy 还简化了评估过程，即使这些评估有局限性（对于问答进行精确匹配对我来说并没有太大意义，但我们有更好的选择吗？）。LangChain 和 DSPy 之间的差异更多是语法上的，而不是根本上的。