The Conversation Model

Before going any further, it is worth pausing to think about how we will store the conversation history in the graph.

The chatbot will use two methods to retrieve information from the graph, but both need to store memory consistently. As the particular implementation may change based on the data retrieval method, the chains themselves will be responsible for persisting the memory, keeping the logic close to the chain.

Let’s walk through an example conversation with a user asking for a recommendation.

  • Human: Can you recommend a movie about robots?

  • AI: Sure, iRobot is a 2004 sci-fi film where a homicide detective with a distrust of robots investigates a homicide with the help of a robopsychologist.

  • Human: That doesn’t sound very age-appropriate. Can you recommend a film suitable for children?

  • AI: Oh, sorry. WALL-E is a 2008 animated film where a waste-cleaning robot falls in love and saves the world. It has a 4.2-star rating.

  • Human: That sounds interesting, tell me more…

  • AI: The film is a Disney Pixar production directed by Andrew Stanton. Ben Burtt voices WALL-E, while Jeff Garlin plays Captain B. McCrea.

Retrieval Tool

When the human asks for a recommendation based on its plot, the chatbot will embed the rephrased question and search the vector index for similar embeddings. The text associated with those embeddings will be appended to the prompt, allowing the LLM to use the information to generate a natural language answer.

At the same time, the chain will be responsible for creating a :CONTEXT relationship from the (:Response) node to the node that contains the embedding.

The Retrieval Conversation History Data Model

Cypher Tool

As the human asks more complex questions, the chatbot must fall back to the Cypher tool, which will generate and execute a Cypher statement capable of answering the rephrased question.

The Cypher Conversation History Data Model

The Cypher generation prompt must include instructions to return the elementId of the node along with any relevant properties. This will provide the chain with the information needed to create the :CONTEXT relationship, which, in turn, will provide additional clarity on the response.

The (:Response) node should also store the generated Cypher statement.

Note also the addition of the .cypher property on the (: Response), which will clarify the Cypher statement used to query the database.

Saving the conversation history

The chatbot must create a (:Response) node that contains the original input, the rephrased question, and the output from the LLM.

As users receive one or more responses within a session, this forms a natural hierarchy of (:Session) nodes, each containing relationships to many (:Response) nodes.

Regardless of which chain creates it, the (:Response) node should hold the following properties:

  • .createdAt - The creation date of the node

  • .input - The original input that comes from the user

  • .rephrasedQuestion - The standalone question rephrased by the LLM using the conversation history

  • .output - The output subsequently generated by the LLM following the RAG stage

Creating a Response Chain

Creating a chain of :NEXT relationships between responses will provide an optimal way to find the conversation history.

When adding a new response, the chatbot must create an additional :LAST_RESPONSE relationship from the (:Session) node. This relationship offers a shortcut to locate the latest response in the session, from which one can follow the :NEXT relationship to trace the recent response history.

Using the :LAST_RESPONSE relationship to obtian recent message history will be more performant than collecting all responses in memory and ordering by the .createdAt timestamp. From there, you will follow the path of :NEXT relationships in a variable-length traversal to obtain a list of previous messages.

This :LAST_RESPONSE pointer will need to be actively maintained, removing any old relationships before creating a new one to ensure the pointer always points to the latest response.

Modelling a sequence of responses

Modeling Decisions

You can learn more about modeling nodes and relationships in the Graph Data Modelling Fundamentals course.

Saving responses

You will use the following Cypher statement to save conversation history to the database.

cypher
MERGE (session:Session { id: $sessionId }) // (1)

// <2> Create new response
CREATE (response:Response {
  id: randomUuid(),
  createdAt: datetime(),
  source: $source,
  input: $input,
  output: $output,
  rephrasedQuestion: $rephrasedQuestion,
  cypher: $cypher
})
CREATE (session)-[:HAS_RESPONSE]->(response)

WITH session, response

CALL {
  WITH session, response

  // <3> Remove existing :LAST_RESPONSE relationship if it exists
  MATCH (session)-[lrel:LAST_RESPONSE]->(last)
  DELETE lrel

  // <4? Create :NEXT relationship
  CREATE (last)-[:NEXT]->(response)
}

// <5> Create new :LAST_RESPONSE relationship
CREATE (session)-[:LAST_RESPONSE]->(response)

// <6> Create relationship to context nodes
WITH response

CALL {
  WITH response
  UNWIND $ids AS id
  MATCH (context)
  WHERE elementId(context) = id
  CREATE (response)-[:CONTEXT]->(context)

  RETURN count(*) AS count
}

RETURN DISTINCT response.id AS id

The statement performs the following actions:

  1. Find or create the user’s session

  2. Create a new (:Response) node with properties

  3. If it exists, remove any existing :LAST_RESPONSE relationship from the session

  4. Use the last node in the chain to create a :NEXT relationship to the newly created node

  5. Create a :LAST_RESPONSE relationship from the session to the new Response node

  6. Create :CONTEXT relationships to any element IDs used in the response

Rephasing Questions

In the next lesson, you will build a chain that utilizes the conversation history to rephrase a question into a standalone question.

Check Your Understanding

Contextual Relationships

What does the :CONTEXT relationship do in the conversation model?

  • ❏ It links the user’s session to the AI’s response

  • ✓ It creates a connection between the (:Response) node and the nodes which were used to construct the response

  • ❏ It stores the original input from the user

  • ❏ It provides a rating for the movie recommendation

Hint

In Retrieval Augmented Generation (RAG), contextual information is provided along with the prompt to improve the accuracy of the response.

Solution

The :CONTEXT relationship creates a connection between the (:Response) node and the nodes that were used to construct the response.

Chaining Responses

What does creating a chain of :NEXT relationships between responses accomplish?

  • ❏ It increases the complexity of the database

  • ❏ It decreases the efficiency of retrieving conversation history

  • ✓ It provides an optimal way to find the conversation history

  • ❏ It stores the movie recommendations more securely

Hint

Neo4j is designed to follow relationships in a connected dataset.

Solution

The :NEXT relationship provides an optimal way to find the conversation history.

Last Response Relationship

What is the purpose of the :LAST_RESPONSE relationship?

  • ❏ To point to the first response in the conversation

  • ✓ To provide a shortcut to find the latest response in the session

  • ❏ To link the user directly to the movie recommendation

  • ❏ To store the AI’s response for future reference

Hint

The last response in a relationship is the most contextually important in the conversation.

Solution

The purpose of the :LAST_RESPONSE relationship is to provide a shortcut to find the latest response in the session.

Summary

In this lesson, we explored the data model you will use to store conversation history in the graph.

In the next lesson, you will create a Neo4jGraph object for interacting with Neo4j.