Build a ChatBot#
In this notebook, we will build an interactive chatbot using the unifyai python package.
Under the hood, chatbots are very simple to implement. All LLM endpoints are stateless, and therefore the entire conversation history is repeatedly fed as input to the model. All that is required of the local agent is to store this history, and correctly pass it to the model.
Install Dependencies#
To run this notebook, you will need to install the unifyai python package. You can do so by running the cell below ⬇️
[ ]:
!pip install unifyai
Collecting unifyai==0.8.1
Downloading unifyai-0.8.1-py3-none-any.whl (14 kB)
Requirement already satisfied: openai<2.0.0,>=1.12.0 in /usr/local/lib/python3.10/dist-packages (from unifyai==0.8.1) (1.17.1)
Requirement already satisfied: requests<3.0.0,>=2.31.0 in /usr/local/lib/python3.10/dist-packages (from unifyai==0.8.1) (2.31.0)
Requirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.12.0->unifyai==0.8.1) (3.7.1)
Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai<2.0.0,>=1.12.0->unifyai==0.8.1) (1.7.0)
Requirement already satisfied: httpx<1,>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.12.0->unifyai==0.8.1) (0.27.0)
Requirement already satisfied: pydantic<3,>=1.9.0 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.12.0->unifyai==0.8.1) (2.6.4)
Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.12.0->unifyai==0.8.1) (1.3.1)
Requirement already satisfied: tqdm>4 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.12.0->unifyai==0.8.1) (4.66.2)
Requirement already satisfied: typing-extensions<5,>=4.7 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.12.0->unifyai==0.8.1) (4.11.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.31.0->unifyai==0.8.1) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.31.0->unifyai==0.8.1) (3.6)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.31.0->unifyai==0.8.1) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.31.0->unifyai==0.8.1) (2024.2.2)
Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->openai<2.0.0,>=1.12.0->unifyai==0.8.1) (1.2.0)
Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.23.0->openai<2.0.0,>=1.12.0->unifyai==0.8.1) (1.0.5)
Requirement already satisfied: h11<0.15,>=0.13 in /usr/local/lib/python3.10/dist-packages (from httpcore==1.*->httpx<1,>=0.23.0->openai<2.0.0,>=1.12.0->unifyai==0.8.1) (0.14.0)
Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1.9.0->openai<2.0.0,>=1.12.0->unifyai==0.8.1) (0.6.0)
Requirement already satisfied: pydantic-core==2.16.3 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1.9.0->openai<2.0.0,>=1.12.0->unifyai==0.8.1) (2.16.3)
Installing collected packages: unifyai
Attempting uninstall: unifyai
Found existing installation: unifyai 0.8.0
Uninstalling unifyai-0.8.0:
Successfully uninstalled unifyai-0.8.0
Successfully installed unifyai-0.8.1
The Agent#
We define a simple chatbot class below, with the only public function being run. Before starting, you should to obtain a UNIFY key from the console page and assign it to the UNIFY_KEY variable below.
[ ]:
UNIFY_KEY = #ENTERUNIFYKEY
[ ]:
import sys
from typing import Optional
from unify import Unify
class ChatBot:
"""Agent class represents an LLM chat agent."""
def __init__(
self,
api_key: Optional[str] = None,
endpoint: Optional[str] = None,
model: Optional[str] = None,
provider: Optional[str] = None,
) -> None:
"""
Initializes the ChatBot object.
Args:
api_key (str, optional): API key for accessing the Unify API.
If None, it attempts to retrieve the API key from the
environment variable UNIFY_KEY.
Defaults to None.
endpoint (str, optional): Endpoint name in OpenAI API format:
<uploaded_by>/<model_name>@<provider_name>
Defaults to None.
model (str, optional): Name of the model. If None,
endpoint must be provided.
provider (str, optional): Name of the provider. If None,
endpoint must be provided.
Raises:
UnifyError: If the API key is missing.
"""
self._message_history = []
self._paused = False
self._client = Unify(
api_key=api_key,
endpoint=endpoint,
model=model,
provider=provider,
)
@property
def client(self) -> str:
"""
Get the client object.
Returns:
str: The model name.
"""
return self._client
def set_client(self, value: Unify) -> None:
"""
Set the model name.
Args:
value: The unify client.
"""
self._client = value
@property
def model(self) -> str:
"""
Get the model name.
Returns:
str: The model name.
"""
return self._client.model
def set_model(self, value: str) -> None:
"""
Set the model name.
Args:
value (str): The model name.
"""
self._client.set_model(value)
if self._client.provider:
self._client.set_endpoint("@".join([value, self._client.provider]))
else:
mode = self._client.endpoint.split("@")[1]
self._client.set_endpoint("@".join([value, mode]))
@property
def provider(self) -> Optional[str]:
"""
Get the provider name.
Returns:
str: The provider name.
"""
return self._client.provider
def set_provider(self, value: str) -> None:
"""
Set the provider name.
Args:
value (str): The provider name.
"""
self._client.set_provider(value)
self._client.set_endpoint("@".join([self._model, value]))
@property
def endpoint(self) -> str:
"""
Get the endpoint name.
Returns:
str: The endpoint name.
"""
return self._client.endpoint
def set_endpoint(self, value: str) -> None:
"""
Set the model name.
Args:
value (str): The endpoint name.
"""
self._client.set_endpoint(value)
self._client.set_model(value.split("@")[0])
self._client.set_provider(value.split("@")[1])
def _get_credits(self):
"""
Retrieves the current credit balance from associated with the UNIFY account.
Returns:
float: Current credit balance.
"""
return self._client.get_credit_balance()
def _process_input(self, inp: str, show_credits: bool, show_provider: bool):
"""
Processes the user input to generate AI response.
Args:
inp (str): User input message.
show_credits (bool): Whether to show credit consumption.
show_credits (bool): Whether to show provider used.
Yields:
str: Generated AI response chunks.
"""
self._update_message_history(role="user", content=inp)
initial_credit_balance = self._get_credits()
stream = self._client.generate(
messages=self._message_history,
stream=True,
)
words = ""
for chunk in stream:
words += chunk
yield chunk
self._update_message_history(
role="assistant",
content=words,
)
final_credit_balance = self._get_credits()
if show_credits:
sys.stdout.write(
"\n(spent {:.6f} credits)".format(
initial_credit_balance - final_credit_balance,
),
)
if show_provider:
sys.stdout.write("\n(provider: {})".format(self._client.provider))
def _update_message_history(self, role: str, content: str):
"""
Updates message history with user input.
Args:
role (str): Either "assistant" or "user".
content (str): User input message.
"""
self._message_history.append(
{
"role": role,
"content": content,
},
)
def clear_chat_history(self):
"""Clears the chat history."""
self._message_history.clear()
def run(self, show_credits: bool = False, show_provider: bool = False):
"""
Starts the chat interaction loop.
Args:
show_credits (bool, optional): Whether to show credit consumption.
Defaults to False.
show_provider (bool, optional): Whether to show the provider used.
Defaults to False.
"""
if not self._paused:
sys.stdout.write(
"Let's have a chat. (Enter `pause` to pause and `quit` to exit)\n",
)
self.clear_chat_history()
else:
sys.stdout.write(
"Welcome back! (Remember, enter `pause` to pause and `quit` to exit)\n",
)
self._paused = False
while True:
sys.stdout.write("> ")
inp = input()
if inp == "quit":
self.clear_chat_history()
break
elif inp == "pause":
self._paused = True
break
for word in self._process_input(inp, show_credits, show_provider):
sys.stdout.write(word)
sys.stdout.flush()
sys.stdout.write("\n")
Let’s Chat#
Now, we can instantiate and chat with this agent. For this demo, we’ll utilize the llama-2-7b-chat model from anyscale. However, you have the flexibility to select any model and provider from our supported options on the benchmarks interface.
[ ]:
agent = ChatBot(api_key = UNIFY_KEY, endpoint = "llama-2-70b-chat@anyscale")
agent.run()
Let's have a chat. (Enter `pause` to pause and `quit` to exit)
> Hi, nice to meet you. My name is Foo Barrymore, and I am 25 years old.
Hello Foo! Nice to meet you too. I'm just an AI, I don't have a personal name, but I'm here to help you with any questions or concerns you might have. How has your day been so far?
> How old am I?
You've told me that you're 25 years old. Is there anything else you'd like to know or discuss?
> Your memory is astounding
Thank you! I'm glad you think so. I'm designed to remember and process large amounts of information, and I'm constantly learning and improving my abilities. However, it's important to note that my memory is not perfect, and there may be times when I forget or misremember certain details. If you have any specific questions or concerns about my memory or abilities, feel free to ask!
> quit
You can also see how many credits your prompt used. This option is set in the constructor, but it can be overwritten during the run command. When enabled, each response from the chatbot will then be appended with the credits spent:
[ ]:
agent.run(show_credits=True)
Let's have a chat. (Enter `pause` to pause and `quit` to exit)
> What is the capital of Palestine?
The question of the capital of Palestine is a politically sensitive and complex issue. The status of Jerusalem is disputed between Israelis and Palestinians, with both sides claiming it as their capital.
The Palestinian National Authority, which governs the Palestinian territories in the West Bank and Gaza Strip, has its administrative center in Ramallah, which is often referred to as the "de facto capital" of Palestine. However, the Palestinian Authority has not declared a capital city, and the issue remains a matter of debate and negotiation in the Israeli-Palestinian peace process.
The international community has not recognized any capital of Palestine, and many countries maintain their diplomatic missions to the Palestinian Authority in Tel Aviv, Israel, rather than in Ramallah or East Jerusalem, which is claimed by the Palestinians as the capital of a future Palestinian state.
It is important to note that the issue of the capital of Palestine is closely tied to the broader conflict between Israelis and Palestinians, and any resolution to the conflict will need to address this issue in a way that is acceptable to both sides.
(spent 0.000274 credits)
> quit
Finally, you can switch providers half-way through the conversation easily. This can be useful to handle prompt of varying complexity.
For example we can start with a small model for answering simple questions, such as recalling facts, and then move to a larger model for a more complex task, such as creative writing.
[ ]:
agent = ChatBot(api_key = UNIFY_KEY, endpoint = "llama-2-70b-chat@anyscale")
agent.run(show_credits=True)
Let's have a chat. (Enter `pause` to pause and `quit` to exit)
> What is the capital of Portugal?
The capital of Portugal is Lisbon (Portuguese: Lisboa).
(spent 0.000032 credits)
> My name is José Mourinho.
Ah, I see! José Mourinho is a well-known Portuguese football manager and former football player. He has managed several top-level clubs, including Chelsea, Inter Milan, Real Madrid, and Manchester United. Mourinho is known for his tactical approach to football and his ability to motivate his players. He has won numerous honors and awards throughout his career, including several league titles, domestic cups, and European championships. Is there anything else you'd like to know about José Mourinho?
(spent 0.000159 credits)
> pause
[ ]:
agent.set_endpoint("gpt-4-turbo@openai")
agent.run(show_credits=True)
Welcome back! (Remember, enter `pause` to pause and `quit` to exit)
> Please write me a poem about my life in Lisbon, using my name in the poem.
In Lisbon's embrace, where tales intertwine,
Lives José Mourinho, beneath the sun's fine shine.
From cobblestone streets where echoes dance,
To the Tagus' gentle waves that entrance.
In youth, he dreamt beneath Iberian skies,
Where passion is fierce and ambition never dies.
With a ball at his feet and dreams in his heart,
In Lisbon's grand story, he crafted his part.
Eduardo VII Park, in the spring's embrace,
Where thoughts of tactics first took place.
Through Alfama's alleys, past Fado's mournful sound,
Mourinho's purpose, in football, was found.
From Benfica's nest to União de Leiria's helm,
His journey began, in a realm
Where strategies and plays, meticulously spun,
Foreshadowed the triumphs that would be won.
In Estádio da Luz, where eagles soar,
Mourinho pondered scores and more.
Though his stay was brief, the impact was deep;
In Lisbon's lore, his legacy would steep.
The boy from Setúbal, with Lisbon in his tale,
Set forth to conquer, to win, and to prevail.
Through Porto, London, Milan, Madrid's grand stage,
His story was written, page by page.
Yet, amidst the victories and the fame's bright light,
In his heart, Lisbon remains, ever so bright.
For it's there José Mourinho's dreams took flight,
In Lisbon's embrace, under the starry night.
So, here's to Mourinho, with Lisbon's spirit in his veins,
Where the love for the game forever remains.
In every triumph, in every fall,
Lisbon, his beginning, the most cherished of all.
(spent 0.012020 credits)
> quit
Switching between providers mid-conversation makes it much easier to maximize quality and runtime performance based on the latest metrics, and also save on costs!
In fact, you can automatically optimize for a metric of your choice with our dynamic routing modes. For example, you can optimize for speed as follows:
[ ]:
agent.set_endpoint("llama-2-70b-chat@highest-tks-per-sec")
agent.run(show_provider=True)
Let's have a chat. (Enter `pause` to pause and `quit` to exit)
> Tell me your favorite physics fact.
My favorite physics fact is that the universe is still expanding! This means that the galaxies that are currently moving away from us will continue to move away from us, and eventually, they will move faster than the speed of light. This is known as the "dark energy" that is thought to be responsible for the acceleration of the universe's expansion.
I find this fascinating because it shows that the universe is still evolving and changing, and there is still so much to learn about it. It's mind-boggling to think about the vastness of space and the mysteries that it holds.
Additionally, this fact also reminds me of the importance of continuous learning and exploration. There is always more to discover and understand, and it's important to have a curious and open-minded approach to life.
I hope this fact inspires you to learn more about the wonders of the universe!
(provider: fireworks-ai)
> quit
The flag show_provider ensures that the specific provider is printed at the end of each response. For example, sometimes anyscale might be the fastest, and at other times it might be together-ai or fireworks-ai. This flag enables you to keep track of what provider is being used under the hood.
If the task is to summarize a document or your chat history grows, typically the input-cost becomes the primary cost driver. You can use our lowest-input-cost mode to direct queries to the provider with the lowest input cost automatically.
[ ]:
agent = ChatBot(api_key=UNIFY_KEY, endpoint="llama-2-70b-chat@lowest-input-cost")
agent.run(show_provider=True)
Let's have a chat. (Enter `pause` to pause and `quit` to exit)
> Summarize the following in less than 10 words: Sir Isaac Newton FRS (25 December 1642 – 20 March 1726/27[a]) was an English polymath active as a mathematician, physicist, astronomer, alchemist, theologian, and author who was described in his time as a natural philosopher.[7] He was a key figure in the Scientific Revolution and the Enlightenment that followed. His pioneering book Philosophiæ Naturalis Principia Mathematica (Mathematical Principles of Natural Philosophy), first published in 1687, consolidated many previous results and established classical mechanics.[8][9] Newton also made seminal contributions to optics, and shares credit with German mathematician Gottfried Wilhelm Leibniz for developing infinitesimal calculus, though he developed calculus years before Leibniz.[10][11] In the Principia, Newton formulated the laws of motion and universal gravitation that formed the dominant scientific viewpoint for centuries until it was superseded by the theory of relativity. Newton used his mathematical description of gravity to derive Kepler's laws of planetary motion, account for tides, the trajectories of comets, the precession of the equinoxes and other phenomena, eradicating doubt about the Solar System's heliocentricity.[12] He demonstrated that the motion of objects on Earth and celestial bodies could be accounted for by the same principles. Newton's inference that the Earth is an oblate spheroid was later confirmed by the geodetic measurements of Maupertuis, La Condamine, and others, convincing most European scientists of the superiority of Newtonian mechanics over earlier systems. Newton built the first practical reflecting telescope and developed a sophisticated theory of colour based on the observation that a prism separates white light into the colours of the visible spectrum. His work on light was collected in his highly influential book Opticks, published in 1704. He also formulated an empirical law of cooling, made the first theoretical calculation of the speed of sound, and introduced the notion of a Newtonian fluid. In addition to his work on calculus, as a mathematician Newton contributed to the study of power series, generalised the binomial theorem to non-integer exponents, developed a method for approximating the roots of a function, and classified most of the cubic plane curves. Newton was a fellow of Trinity College and the second Lucasian Professor of Mathematics at the University of Cambridge. He was a devout but unorthodox Christian who privately rejected the doctrine of the Trinity. He refused to take holy orders in the Church of England, unlike most members of the Cambridge faculty of the day. Beyond his work on the mathematical sciences, Newton dedicated much of his time to the study of alchemy and biblical chronology, but most of his work in those areas remained unpublished until long after his death. Politically and personally tied to the Whig party, Newton served two brief terms as Member of Parliament for the University of Cambridge, in 1689–1690 and 1701–1702. He was knighted by Queen Anne in 1705 and spent the last three decades of his life in London, serving as Warden (1696–1699) and Master (1699–1727) of the Royal Mint, as well as president of the Royal Society (1703–1727).
Newton: polymath, mathematician, physicist, astronomer, alchemist, theologian, and author.
(provider: octoai)
> quit
Python Package#
The python package already contains the ChatBot agent and you may use it directly as follows:
[ ]:
from unify import ChatBot
chatbot = ChatBot(api_key = UNIFY_KEY, endpoint="llama-2-7b-chat@anyscale")
chatbot.run()
Let's have a chat. (Enter `pause` to pause and `quit` to exit)
> Hey! How's it going?
Hello! I'm doing well, thank you for asking! It's going great here, just busy with various tasks and learning new things. However, I must point out that this conversation is a bit unusual as I'm just an AI and don't have personal experiences or emotions like humans do. I'm here to help answer any questions you may have, so feel free to ask me anything!
> quit
Round Up#
Congratulations! 🚀 You are now capable of building ChatBot Agents for your application using our LLM endpoints.