OpenAI Assistants API vs Chat Completion API - Here Are the Differences

OpenAI recently released their new Assistants API. Their purpose is to (quote) "help developers build powerful AI assistants capable of performing a variety of tasks".

But how do assistants differ from the chat completion APIs that we already had before? And which one should you use to build your AI apps?

Over the last two weeks, I've been building a Next.js app for a freelance client that makes heavy use of this new Assistants API. In this post, I want to give you a short overview of what assistants are and what they can and cannot do.

Table of Contents

• What are OpenAI Assistants?
• Limitations and use of OpenAI Assistants
• Using Assistants in your code with the Vercel AI SDK
• Conclusion

What are OpenAI Assistants?

Assistants are basically fully functional chatbots. They automatically maintain a history of chat messages which they can then use for context (i.e. to answer questions). Each assistant can start and maintain multiple independent message threads.

The normal chat completion API doesn't have such an inbuilt chat history. There, we have to manage our AI's memory ourselves using techniques like vector embeddings.

Furthermore, you can upload and attach files to assistants. Assistants can, for example, read from a PDF and answer questions based on that information. Again, in the past, we would have to build this functionality ourselves using vector embeddings.

Assistants can also do a bunch of other tasks like interpreting and running code. You can create assistants and play around with them in the assistants playground.

Limitations and use of OpenAI Assistants

Although they are powerful, assistants don't make the chat completion API obsolete.

One limitation of Assistants is that you can only attach a maximum of 20 files to them (with a maximum size of 512MB per file). This makes them not suited to work with user-specific data. For example, in my Next.js AI chatbot tutorial on Youtube, we build a chatbot that can retrieve and read the user's personal notes. This functionality can't be replaced by an assistant, because the AI needs access to each user's individual data.

The popular "Chat with PDF"-type apps are also not replaced by assistants. With the 20 files limitation, you can't have each user upload their individual PDFs. You could create a separate assistant for each user, but there is a total file limit of 100GB per organization in your OpenAI account, so this is not feasible either.

The real purpose of this new Assistants API is to create application-specific assistants that can work with a limited set of data. You could, for example, build an assistant for a documentation page, upload the whole documentation as PDFs, and then have the bot answer user questions based on it.

Using Assistants in your code with the Vercel AI SDK

The Vercel AI SDK makes it easier to use OpenAI API features in your Next.js apps. I teach how to use this library in my Next.js AI chatbot tutorial on Youtube. If you haven't watched it yet, I highly recommend that you do so. These are the most important skills to learn as a web developer right now.

To handle the new Assistants API, the Vercel AI SDK added a new AssistantResponse class and a useAssistant hook. They are both experimental at the time of this writing. The code is still a bit unwieldy and assistants also don't support streaming yet. These new APIs are not even mentioned in the documentation yet, but you can find a code example of how to use them here. I'm sure they will get easier to use soon.

Conclusion

To summarize, OpenAI Assistants provide an easy way to build fully functional AI chatbots with memory & knowledge retrieval for specific apps or websites. They are not suited to work with large amounts of individual, user-specific data (like "Chat with PDF" apps).

Once the APIs get stable, I'll make a detailed tutorial on the Assistants API on my Youtube channel. So subscribe to the channel if you haven't yet.

One more thing: I recently overhauled the Coding in Flow Discord server. It's a great place to ask programming questions and discuss with other developers. It's completely free and I would be happy if you would join me and thousands of other developers there!

Happy coding!

Florian