Prisma Pulse: Introducing Delivery Guarantees for Database Change Events

If you want to try Prisma Pulse yourself, jump straight to the example projects: a real-time chat, a live leaderboard, a server that sends onboarding emails for new users and a serverless cron job that syncs data into a search index!

Event-driven architectures enable scalability and real-time

Building robust and scalable apps with real-time features like chat, notifications, live collaboration, polls or quizzes can be complex. Polling your database for changes is not only resource-intensive, but it also gets complex very quickly when building more advanced scenarios.

We want to make this easy with Prisma Pulse by enabling you to integrate your database into an event-driven architecture!

What are event-driven architectures?

Event-driven architectures help build your real-time apps in a robust and scalable way! They also save resources when compared to other approaches like polling. Polling consumes many database connections and CPU, both on your application and the database servers:

On the other hand, in an event-driven architecture, the event is delivered to the consumer right when it happens:

In an event-driven architecture, you typically have (at least) two components:

• An event producer is responsible for publishing events and delivering them to the consumers.
• An event consumer subscribes to events and receives them after they're published by the producer.

In the illustration above, the database is the event producer and the application server is the event consumer.

Easily stream change events from your database with Prisma Pulse

Prisma Pulse enables event-driven architectures by easily letting you stream all the changes that are happening in your database into your application.

Pulse uses Change Data Capture (CDC) to capture write-events (meaning any create, update, delete operations) that happen in the database by monitoring the database's transaction log, such as the Write-Ahead Log in PostgreSQL.

Delivery guarantees with at least once semantics 🎉

Imagine you're building a health monitoring application and you use an event-driven architecture to deliver notifications to your users in case of an emergency. In these and many other scenarios, your application can't afford to lose any events because that may have serious implications for your business — or, even worse, a person's health!

Note: Prisma Pulse requires a PostreSQL database (> v12) with logical replication enabled. Let us know if you would like to see support for other databases.

What are delivery guarantees in event-driven architectures?

To be sure that your events actually arrive on the consumer-side, you need to understand the delivery semantics of your event-driven architecture. Delivery semantics define the guarantees your system can provide for the delivery of an event:

• At most once: The event is delivered either once or not at all.
• At least once: The event is delivered one or more times, ensuring it never goes undelivered.
• Exactly once: The event is delivered exactly once, avoiding duplicates.

Another dimension of delivery guarantees is the order in which an event arrives at the consumer. In general, there are two different options for this:

• Events are delivered in the same order in which they were emitted by the producer.
• Events may be delivered in the same or different order than they were emitted by the producer.

At least once and same order delivery in Prisma Pulse

As of today, Prisma Pulse supports at least once delivery in the same order and hence can guarantee that you'll never lose any of your database events.

Under the hood, Pulse is now able to persist the events that are produced by your database which enables the new delivery guarantees.

Note: Event persistence is available on all our plans, with free limits included to let you take stream() for a spin. Visit the pricing page to find the right plan for you, and learn more about database event, event read, and event storage costs.

To use the new delivery guarantees with Prisma Pulse, you need to:

1. Make sure Event persistence is enabled when setting up Pulse in the Console:

   

2. Use the new .stream() API offered by the Pulse extension:

   The stream() method returns an async iterable that will receive an event for every write operation that happens in the database. You can use it as follows:

   The console.log statement will run any time a change occurs on the User table. Here is a sample terminal output that will be printed when a record gets updated:

Note: The previous .subscribe() API still exists but isn't compatible with event persistence and thus doesn't provide any delivery guarantees. If you are using .subscribe() today but want to take advantage of delivery guarantees, you need to re-enable Prisma Pulse in your Platform project, update the Client extension for Pulse to use the latest version and adjust your code to use .stream().

Exactly once delivery in Prisma Pulse

While Pulse by default offers "at least once" semantics, it also provides the primitives for you to implement "exactly once" delivery guarantees yourself!

Each event produced by Prisma Pulse carries an identifier/idempotency key (which you can access via the id field). This key can be used to deduplicate during downstream processing of the event. This is best implemented by passing the id along to external services that support idempotency or using concepts like upsert when working with a database.

Never lose any events — even if your server goes down

If you use the new .stream() API, Pulse guarantees that all database events arrive in your application at least once in the right order.

But what happens if your server goes down? Are you going to lose the events that happen in the database in the meantime?

You don't have to: The new .stream() API enables you to "pick up where you left off" by providing a name argument to it:

Whenever you provide a name, Pulse associates a cursor with that particular stream to track which events have arrived at the consumer-side.

If your server (which is the consumer in this scenario) goes down, any events that happen in the database in the meantime are not going to be acknowledged by the consumer and the cursor won't move.

Once your server comes back online, the stream picks up again and delivers all the events that had been missed in the meantime. The stream resumes from the point where it dropped off:

If you omit the name option, your stream will not "replay" any events that have gotten lost while your server was down:

The case for serverless with Prisma Pulse

One major use case for Prisma Pulse is building real-time applications. If you want database events to be delivered to your application in real-time, you will need a long-running server on which Prisma Client calls the .stream() API.

However, with the new delivery guarantees, Prisma Pulse can be used in serverless environments for certain use cases. If it's not important when an event arrives, but only that it will arrive (eventually), Prisma Pulse with delivery guarantees will meet your needs.

Imagine you are building an e-commerce application and need to sync your product data into a secondary data store, like a search index: Instead of using a long-running server that constantly runs (and thereby consumes resources which you need to pay for), you could set up a cron job in a serverless function that runs every hour. If you use the new .stream() API with a name argument, all events that happened in the hour before the cron job runs the next time will be "replayed" by Pulse:

Check out use cases & example projects

To make it easy for you to try out Prisma Pulse, we have created a couple of example projects for you.

Live leaderboard

This live leaderboard example demonstrates how to build a fullstack app using Prisma Pulse where the frontend updates in real-time when a change is made to the database. It uses the following stack:

• Next.js (frontend) with a custom server (backend)
• React Flip Move for animating React components
• socket.io for the websocket connection between client and server
• Prisma Pulse to get real-time updates from the database
• PostgreSQL as the database

You can use the example by following these steps:

1. Run the following command:

2. Follow the steps in the README. Note that you will need to use a Pulse-ready database and generate a Pulse API key in the Platform Console for this to work.

More examples

Check out the prisma-examples repo for further examples with Prisma Pulse, such as:

• Real-time chat: Fullstack chat application using Next.js, Express and socket.io
• Onboarding emails: Server that sends onboarding emails to new users with Resend
• Syncing cron job: Serverless cron job that syncs data into Typesense to enable a product search

Try it out and share your feedback 💚

We're super excited about the new delivery guarantees in Prisma Pulse! They not only provide you with the confidence that you'll never lose any events that happen in your database, they also enable entirely new use cases like using Pulse in serverless environments (since events can be "resumed" any time a function is spawned).

Try it out and let us know how you like the new additions to Prisma Pulse. In case you get stuck or aren't sure where to start, you can always reach out to us on X or on Discord.