Background Information

Project overview

Setup

Introduction

Prompt template

LCEL

Introduction to prompt engineering

Prompts

Memory

Conversation memory

Supabase

Restoring memory

Vectorstores

Supabase vector

Ingest

Retrieval

Generation

Retrieval Augmented Generation

Output Parsing

Tools

Agents

Langsmith

Farewell

( the primaryDescriptionMarkdown is long - several lines, include images, as long as you want )

In this course we <headline of what we do>. <why this technology is good> but <here's how learning it is problem>. 

In <the length of these lessons> we cover <three salient points about what we cover. Your BEST points>. Its <however long it is> and comes with <these bonus features, and reasons to buy. code, etc.>

It's taught by <author name> who <credibility, reason you should listen to them>. <social proof, if you have it yet>

<then you can list other supporting stuff you want to add, anything you want here>

-------
Example:
-------

We created a free course to create your first GraphQL server (from scratch) using Node and TypeScript. TypeScript and GraphQL are a powerful combo, but there's a lot of confusing edges if you're getting it setup the first time.

In this course we cover the basics of getting TypeScript setup with Node, how GraphQL works, and how to set up a GraphQL server using Apollo server. It's two hours long, it comes with complete code examples, and we walk through the code step-by-step.

It's taught by my friend, Shopify-engineer Hassan Djirdeh and he's integrated what he's learned using GraphQL in production as an engineer at Shopify. We've already had over 1,300 students enroll and it's been getting great reviews so far.

// ... and so on

In this course we <headline of what we do>. <why this technology is good> but <here's how learning it is problem>. 

( the summary is short - one or two lines )

Langchain.js Bootcamp

In this module we will learn about "Memory".

If you have used chat LLM applications then memory should be a familiar feature. Memory is often taken for granted. But it's a key part of conversational LLM systems and it's behavior is very important to the course.

The chains that we have built so far do not hold memory. Which means that the LLM does not see the past messages. We have been giving the model the most recent message with 

Conversational Memory

That kind of memory is stored in context and you can think of it as the "State" of a chain (the part of the prompt that changes during runtime). It is often called Conversational Memory. If you like analogies, you can think of it as data held in RAM. The prompt is a program. And the LLM is a computer.

Later in the module, we'll back up our Memory to a database and restore conversations. In our analogy, that is similar to a hard disk.

The array is typically a list of conversation history, uploaded documents, and functions that the LLM uses.

Before we implement memory, let's review why it is important and the technical details of "Chat". Remember that LLMs without finetuning do not care for conversations. They output the most likely completion for an input text. So, "Chat" is driven by a UI idea founded on the belief that humans like to interface with language entities through conversation. During finetuning, researchers often create different prompt abstractions and the model learns to respect a certain syntax. For example, let's look at the 

Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{instruction}

### Input:
{input}

### Response:

The Alpaca model was finetuned on special sequences like 

. Below is one of the lines of the training data.

If you used the model with not regards for the finetuning template it will confuse the model and negatively impact performance.

The outputs are often LLM generations that have been corrected by users. The contents, wording, and underlying semantics of the output are often very contentious.

 you will see that it doesn't contain any conversations. Let's examine a similar model called 

. The Vicuna researchers selected for high quality conversations that train the model to make use of previous messages.

Vicuna was finetuned from Llama 2. The total training cost was around $300.

Vicuna Dataset

Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena

 for Alpaca and the "Selected" Vicuna finetune dataset.

As an example of when that could be useful, say you introduce yourself to Vicuna. You expect the model to remember your name.

A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.
USER: "Hi, my name is Ayesha."
ASSISTANT: "Hi Ayesha!"
USER: "What is my name?"
ASSISTANT:

Naturally, you expect the model to output "Ayesha". We can divide this prompt into 3 message types: System, User, and Assistant. This is an abstraction shared across Chat models. The implementations are sometimes different, but thankfully Langchain will handle that for us. The previous user and assistant messages and how they update are related to Memory.

As an extended note, it's worth knowing that there is a gray line between things you can replicate through prompting and through finetuning. For example you can get a model to memorize names with a template like below. However, this approach does not generalize well to other conversations.

People used to hold conversations with base LLMs through clever prompting and stop sequences. You had to stop the LLM from generating more text after seeing 

 to enable "Human in the Loop". Otherwise the LLM would talk to itself and generate the entire conversation. A typical conversation would also start with a message that explains the background of the conversation and roles - similar to system messages today.

In this lesson we'll use a simple project to help us visualize memory and demonstrate how we can use it in an application. You can find the code, 

 runtime, so you will have to install it with 

. You will also need to edit proper environment variables if you plan on using any APIs.

If you have more issues, please refer to the repo's README or submit an issue.

For this section I'm using Cohere's command-light model. As of this writing, Cohere gives out free rate-limited tokens. They should have more than enough bandwidth for solo testing. If you want to follow along you can sign up, 

. Otherwise feel free to use your model of choice.

 for general knowledge and more specific tips for prompting their models. One thing that I learned was that you can use the AIDA Framework for prompting by including Attention, Interest, Desire, Action.

import { ChatCohere } from "@langchain/cohere"

const model = new ChatCohere({
  apiKey: process.env.COHERE_API_KEY,
  model: "command-light",
})

To use the System, User, Assistant message schema in Langchain, we can use 

. This is not associated with any message type, but it lets langchain know that we intend to use the "history" key as part of the prompt template. This is the key that will let us use chat history in the prompt. It can be any value as long as it lines up with the following code.

const prompt = ChatPromptTemplate.fromMessages([
  new MessagesPlaceholder("history"),
  ["human", "{input}"]
])

ChatCohere is currently limited to "human" and "ai" message types. So you should not use system prompt unless it is released.

---
code: https://gitlab.com/fullstackio/books/newline-course-apps/jeffery-wang-langchain.js-bootcamp-app/-/tree/main/Code/module_03/lesson_03.01/protected/src?ref_type=heads
---

# Memory

In this module we will learn about "Memory".

If you have used chat LLM applications then memory should be a familiar feature. Memory is often taken for granted. But it's a key part of conversational LLM systems and it's behavior is very important to the course.

The chains that we have built so far do not hold memory. Which means that the LLM does not see the past messages. We have been giving the model the most recent message with `currentMessage`.

## TODO change diagram

![Memory Diagram](https://d2uusema5elisf.cloudfront.net/courses/langchainjs-bootcamp/module_03/lesson_03.01/public/assets/memory.png "Conversational Memory")

That kind of memory is stored in context and you can think of it as the "State" of a chain (the part of the prompt that changes during runtime). It is often called Conversational Memory. If you like analogies, you can think of it as data held in RAM. The prompt is a program. And the LLM is a computer.

Later in the module, we'll back up our Memory to a database and restore conversations. In our analogy, that is similar to a hard disk.

The array is typically a list of conversation history, uploaded documents, and functions that the LLM uses.

## Background

Before we implement memory, let's review why it is important and the technical details of "Chat". Remember that LLMs without finetuning do not care for conversations. They output the most likely completion for an input text. So, "Chat" is driven by a UI idea founded on the belief that humans like to interface with language entities through conversation. During finetuning, researchers often create different prompt abstractions and the model learns to respect a certain syntax. For example, let's look at the [Alpaca](https://github.com/tatsu-lab/stanford_alpaca) format.

```text
Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{instruction}

### Input:
{input}

### Response:
```

The Alpaca model was finetuned on special sequences like `### Instruction:\n`. Below is one of the lines of the training data.

```json
{
  "instruction": "Who is the world's most famous painter?",
  "input": "",
  "output": "The world's most famous painter is generally considered to be the Italian renaissance artist Leonardo da Vinci. His work includes the Mona Lisa and The Last Supper, both of which are considered masterpieces. Da Vinci was also a master of science, inventing the helicopter and the tank, among other inventions. He was also a prolific musician, sculptor, and architect. His work has inspired generations of artists and continues to be revered today."
}
```

If you used the model with not regards for the finetuning template it will confuse the model and negatively impact performance.

:::important
The outputs are often LLM generations that have been corrected by users. The contents, wording, and underlying semantics of the output are often very contentious.
:::

### Chat memory

If you look at the [Alpaca dataset](https://raw.githubusercontent.com/tatsu-lab/stanford_alpaca/main/alpaca_data.json) you will see that it doesn't contain any conversations. Let's examine a similar model called [Vicuna](https://lmsys.org/blog/2023-03-30-vicuna/). The Vicuna researchers selected for high quality conversations that train the model to make use of previous messages.

:::note
Vicuna was finetuned from Llama 2. The total training cost was around $300.
:::

![Vicuna Dataset](https://d2uusema5elisf.cloudfront.net/courses/langchainjs-bootcamp/module_03/lesson_03.01/public/assets/vicunadataset.png "Vicuna Dataset")

> Diagram from the Vicuna paper, **_[Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://arxiv.org/abs/2306.05685)._** Note the **_"Avg. turns of conversation"_** for Alpaca and the "Selected" Vicuna finetune dataset.

As an example of when that could be useful, say you introduce yourself to Vicuna. You expect the model to remember your name.

```text
A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.
USER: "Hi, my name is Ayesha."
ASSISTANT: "Hi Ayesha!"
USER: "What is my name?"
ASSISTANT:
```

:::note
This is the [vicuna_v1.1](https://github.com/lm-sys/FastChat/blob/a47b8f9e93c8b5a85e81d1ae33e3a1106d8cdf80/fastchat/conversation.py#L255-L268) format.
:::

Naturally, you expect the model to output "Ayesha". We can divide this prompt into 3 message types: System, User, and Assistant. This is an abstraction shared across Chat models. The implementations are sometimes different, but thankfully Langchain will handle that for us. The previous user and assistant messages and how they update are related to Memory.

### Finetuning vs prompting

As an extended note, it's worth knowing that there is a gray line between things you can replicate through prompting and through finetuning. For example you can get a model to memorize names with a template like below. However, this approach does not generalize well to other conversations.

```text
User name: {name}
Message: {message}
```

People used to hold conversations with base LLMs through clever prompting and stop sequences. You had to stop the LLM from generating more text after seeing `USER:` or `ASSISTANT:` to enable "Human in the Loop". Otherwise the LLM would talk to itself and generate the entire conversation. A typical conversation would also start with a message that explains the background of the conversation and roles - similar to system messages today.

## Example project

In this lesson we'll use a simple project to help us visualize memory and demonstrate how we can use it in an application. You can find the code, [here](https://github.com/imjwang/chatprompt-sandbox).

The project uses a [Bun](https://bun.sh/) runtime, so you will have to install it with `npm install -g bun` if you do not have Bun.

Afterwards, you can run `./start.sh` and open up `http://localhost:5173/`. You will also need to edit proper environment variables if you plan on using any APIs.

If you have more issues, please refer to the repo's README or submit an issue.

### Cohere

For this section I'm using Cohere's command-light model. As of this writing, Cohere gives out free rate-limited tokens. They should have more than enough bandwidth for solo testing. If you want to follow along you can sign up, [here](https://cohere.com/chat). Otherwise feel free to use your model of choice.

Cohere has a free [LLM University](https://docs.cohere.com/docs/constructing-prompts) for general knowledge and more specific tips for prompting their models. One thing that I learned was that you can use the AIDA Framework for prompting by including Attention, Interest, Desire, Action.

```ts { actualFilename=./protected/src/memory.ts endLine=1 startLine=1 statedFilename=Import ChatCohere url=https&#x3A;//gitlab.com/fullstackio/books/langchainjs-bootcamp/module_03/lesson_03.01/protected/src/memory.ts }
import { ChatCohere } from "@langchain/cohere"
```

```ts { actualFilename=./protected/src/memory.ts endLine=11 startLine=8 statedFilename=ChatCohere url=https&#x3A;//gitlab.com/fullstackio/books/langchainjs-bootcamp/module_03/lesson_03.01/protected/src/memory.ts#L8 }
const model = new ChatCohere({
  apiKey: process.env.COHERE_API_KEY,
  model: "command-light",
})
```

### `ChatPromptTemplate`

To use the System, User, Assistant message schema in Langchain, we can use `ChatPromptTemplate`. We also need to add a `MessagesPlaceholder`. This is not associated with any message type, but it lets langchain know that we intend to use the "history" key as part of the prompt template. This is the key that will let us use chat history in the prompt. It can be any value as long as it lines up with the following code.