Table of Contents
Consultation Service
Releases

Introducing The Model Hub

Guillermo Sanchez-Brizuela
February 6, 2024
5 min read
thumbnail

We’re very excited to announce The Hub: a collection of LLM endpoints, with live runtime benchmarks all plotted across time 📈

Knowing which LLM to use is very complex 🤖, and even after deciding which model to use, it’s equally complex to choose the right provider.

We rigorously benchmark the runtime performance of each provider, making it much simpler to choose the right provider for any given application. Our unified API then makes it very easy to test and deploy the chosen endpoints in production, without needing to create several accounts 🔑

We test across different regions 🌏 (Asia, US, Europe), with varied concurrency 🔀 and sequence length 🔡. By plotting across time, our dashboard highlights the stability and variability of the different endpoints, and their ongoing evolution across API updates and system changes. This is essential in order to make informed decisions about the best provider to use, as we explain in our post: Static LLM Benchmarks Are Not Enough.

Before The Hub, choosing which LLM provider to use looked like this:


Now, with all LLMs and all providers in one place, with rigorous benchmarks, it looks like this:

Our Hub is a work in progress ⚒️, and we will be releasing new features every week 🚀

We’re very excited to see what the community does with it! 😊

We’re very excited to announce The Hub: a collection of LLM endpoints, with live runtime benchmarks all plotted across time 📈

About the Author
Guillermo Sanchez-Brizuela
Unify | ML Engineer, Head of Deployment‍

Guillermo has led predictive analytics and AI research projects, and earned a Master’s with a focus on Deep Learning, Big Data, and Machine Learning from UVa. His work bridges Deep Learning research and AI deployment.

More Reads

left button chevronright button chevron

Consultation Service