Ctrl+K

Synchronous VS Asynchronous Clients#

Given the growing demand for real-time applications and user demands for instant responses, it’s crucial to grasp the performance implications between Sync and Async clients. In this notebook, we’ll delve into the variations between asynchronous and synchronous response times using UnifyAI’s API.

In order to run this notebook, you need to generate your UNIFY key from the console. Once you have it, assign it to the UNIFY_KEY variable below.

[ ]:

UNIFY_KEY=#ENTERUNIFYKEY

Install Dependencies#

To run this notebook, you will need to install the unifyai python package. You can do so by running the cell below ⬇️

[ ]:

!pip install unifyai

Collecting unifyai
  Downloading unifyai-0.8.1-py3-none-any.whl (14 kB)
Collecting openai<2.0.0,>=1.12.0 (from unifyai)
  Downloading openai-1.17.1-py3-none-any.whl (268 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 268.3/268.3 kB 4.6 MB/s eta 0:00:00
Requirement already satisfied: requests<3.0.0,>=2.31.0 in /usr/local/lib/python3.10/dist-packages (from unifyai) (2.31.0)
Requirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.12.0->unifyai) (3.7.1)
Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai<2.0.0,>=1.12.0->unifyai) (1.7.0)
Collecting httpx<1,>=0.23.0 (from openai<2.0.0,>=1.12.0->unifyai)
  Downloading httpx-0.27.0-py3-none-any.whl (75 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.6/75.6 kB 5.7 MB/s eta 0:00:00
Requirement already satisfied: pydantic<3,>=1.9.0 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.12.0->unifyai) (2.6.4)
Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.12.0->unifyai) (1.3.1)
Requirement already satisfied: tqdm>4 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.12.0->unifyai) (4.66.2)
Requirement already satisfied: typing-extensions<5,>=4.7 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.12.0->unifyai) (4.11.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.31.0->unifyai) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.31.0->unifyai) (3.6)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.31.0->unifyai) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.31.0->unifyai) (2024.2.2)
Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->openai<2.0.0,>=1.12.0->unifyai) (1.2.0)
Collecting httpcore==1.* (from httpx<1,>=0.23.0->openai<2.0.0,>=1.12.0->unifyai)
  Downloading httpcore-1.0.5-py3-none-any.whl (77 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.9/77.9 kB 7.0 MB/s eta 0:00:00
Collecting h11<0.15,>=0.13 (from httpcore==1.*->httpx<1,>=0.23.0->openai<2.0.0,>=1.12.0->unifyai)
  Downloading h11-0.14.0-py3-none-any.whl (58 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 kB 4.6 MB/s eta 0:00:00
Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1.9.0->openai<2.0.0,>=1.12.0->unifyai) (0.6.0)
Requirement already satisfied: pydantic-core==2.16.3 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1.9.0->openai<2.0.0,>=1.12.0->unifyai) (2.16.3)
Installing collected packages: h11, httpcore, httpx, openai, unifyai
Successfully installed h11-0.14.0 httpcore-1.0.5 httpx-0.27.0 openai-1.17.1 unifyai-0.8.1

Synchronous Clients#

A Synchronous client handles requests sequentially, processing one at a time. This means that each request must be fully handled before the next one is processed, resulting in potential blocking of the program’s execution. You can use a Sync client with Unify as shown below:

[ ]:

from unify import Unify
unify = Unify(
    api_key=UNIFY_KEY,
    endpoint="llama-2-13b-chat@anyscale"
)
response = unify.generate(user_prompt="Hello Llama! Who was Isaac Newton?")
print(response)

HOO-RAY! *tutu* Hello there, young grasshopper! *bray* Isaac Newton was a majestic figure known for his work in math and science. He was born in 1642 in England and grew up to be a brilliant thinker and a fellow of the Royal Society. *twirl*

Newton is perhaps best known for his laws of motion and his law of universal gravitation. These laws explained how objects move and how gravity works. He also discovered calculus, which is a way of using math to understand how things change over time. *mathematical mnum hop*

But that's not all, oh no! Newton was also a bit of an alchemist and studied the nature of light. He even invented a fancy piece of equipment called a "reflecting telescope" to observe the heavens. *shimmer*

Newton was a true renaissance thinker, and his contributions to science and mathematics are still celebrated today. *tutu* He was a true llama of learning, and his legacy continues to inspire us all. *bray*

Async Clients#

An Asynchronous client can handle multiple requests concurrently without blocking. To use the Async client, import AsyncUnify instead of Unify and use await with the .generate method. This signals to the program to pause execution until the asynchronous operation completes. Additionally, we’ll use the asyncio library for managing asynchronous execution and coroutines. asyncio provides tools for building concurrent programs using coroutines, which can be paused and resumed, allowing for efficient handling of asynchronous tasks.

NOTE: Running ansyncio in notebooks conflicts with the existing event loop run in the notebook. As a workaround, we need to use net_asyncio.apply. Please see this issue for more details.

[ ]:

from unify import AsyncUnify
import asyncio
import nest_asyncio
nest_asyncio.apply()

async_unify = AsyncUnify(
   api_key=UNIFY_KEY,
   endpoint="llama-2-13b-chat@anyscale"
)

async def main():
   responses = await async_unify.generate(user_prompt="Hello Llama! Who was Isaac Newton?")
   print(responses)

asyncio.run(main())

HEYA HUMAN! *bleshes* Oh, you're talkin' 'bout Sir Isaac Newton, the famous English mathematician and physicist? *noms on some grass* He lived in the 17th and 18th centuries, and is known for his laws of motion and universal gravitation. *giggles* He was a pretty smart dude, if I do say so myself! *flaunts my banned-from-the-library-for-life status*

But enough about me, let's talk about Newton! *grin* He was born in Woolsthorpe, Lincolnshire, England in 1643, and grew up to be a brilliant mathematician and physicist. He studied at Trinity College in Cambridge, and later became a professor there. *nbd*

Newton's most famous contributions to science are his three laws of motion, which describe how objects move and respond to forces. He also developed the Law of Universal Gravitation, which states that every object in the universe attracts every other object with a force proportional to the product of their masses and inversely proportional to the square of the distance between them. *geek out*

Newton was also a skilled inventor and engineer, and he designed and built all sorts of cool stuff, like a reflecting telescope and a machine for calculating the square root of numbers. *impressed*

Despite his many accomplishments, Newton was a pretty private person and wasn't always the most sociable guy. He was known to be pretty temperamental and had some pretty interesting beliefs, like the idea that alchemy was a valid scientific pursuit. *raises an eyebrow* But hey, who am I to judge? *shrugs*

So there you have it, human! That's the basic scoop on Sir Isaac Newton. I hope you found that enlightening. *wink* Now, if you'll excuse me, I need to go work on my own groundbreaking research... or at least, my own Instagram captions. *smizes*

Now, our goal is to compare the response times of synchronous vs asynchronous clients when handling multiple requests. Let’s start by defining some helper functions.

[ ]:

async def send_async_request(user_prompt):
    """
    Uses an Async client to generate the response for the user_prompt.

    Parameters:
        user_prompt (str): The prompt provided by the user.

    Returns:
        str: The response generated.
    """
    response = await async_unify.generate(user_prompt=user_prompt)
    return response

def send_sync_request(user_prompt):
    """
    Uses a sync client to generate the response for the user_prompt.

    Parameters:
        user_prompt (str): The prompt provided by the user.

    Returns:
        str: The response generated.
    """
    response = unify.generate(user_prompt=user_prompt)
    return response

We’ll create two functions to send multiple requests to asynchronous and synchronous clients, respectively, and measure their processing time. For the synchronous client, requests will be sent sequentially in a loop, while for the asynchronous client, we’ll utilize asyncio.gather to execute multiple requests concurrently.

[ ]:

import time
import asyncio

async def run_async_requests(num_requests):
    """
    Runs multiple asynchronous requests for generating responses based on a user prompt and measures the time taken.

    Parameters:
        num_requests (int): The number of requests to be sent.

    Returns:
        float: The total time taken to process all requests.
    """
    user_prompt = "Hello! Tell me your favorite physics fact!"
    start = time.time()
    _ = await asyncio.gather(*(send_async_request(user_prompt) for _ in range(num_requests)))
    end = time.time()
    return end - start

def run_sync_requests(num_requests):
    """
    Runs multiple synchronous requests for generating responses based on a user prompt and measures the time taken.

    Parameters:
        num_requests (int): The number of requests to be sent.

    Returns:
        float: The total time taken to process all requests.
    """
    user_prompt = "Hello! Tell me your favorite physics fact!"
    start = time.time()
    _ = [send_sync_request(user_prompt) for _ in range(num_requests)]
    end = time.time()
    return end - start

Now, let’s measure the time taken by each client for 10 requests.

[ ]:

num_requests = 10
# Send asynchronous requests
async_response_times = asyncio.run(run_async_requests(num_requests))
# Print response times
print("Asynchronous Response Times:", async_response_times)
# Send synchronous requests
sync_response_times = run_sync_requests(num_requests)
print("Synchronous Response Times:", sync_response_times)

Asynchronous Response Times: 8.351824045181274
Synchronous Response Times: 55.45608472824097

As expected, the Asynchronous client peforms much better than the sequential synchorous client.

Round Up#

Congratulations! 🚀 You now have an understanding of Async and Sync clients and can hopefully leverage these concepts in your own applications.

In the next tutorial, we will expore how to build an interactive ChatBot Agent! See you there ➡️!

On this page