Concepts
Cache strategies
For all read queries in Prisma Client, you can define the cacheStrategy parameter that configures cache behavior. The cache strategy allows you to define two main characteristics of the cache:
- Time-to-live (TTL): Duration in seconds a cached response is considered fresh.
- Stale-while-Revalidating (SWR): Duration in seconds a stale cache response is considered acceptable while the cache is refreshed in the background
Time-to-live (TTL)
Time-to-Live (TTL) determines how long cached data is considered fresh. By specifying the ttl in seconds, you can control the duration for which data in the cache remains valid. When a read query is executed, if the cached response is within the ttl limit, Prisma Client retrieves the data from the cache without querying the database. If the cached data is not available or has expired, Prisma Client queries the database and stores the results in the cache for future requests.
Use ttl in cacheStrategy and specify the TTL of the query in seconds:
With a specified TTL of 60 seconds, the majority of requests will result in a cache hit throughout the TTL duration:
TTL is useful for reducing database load and latency for data that does not require frequent updates.
Stale-While-Revalidate (SWR)
Stale-While-Revalidate (SWR) allows you to control how long Accelerate can serve stale cache data while fetching fresh data in the background. When a read query is executed, Accelerate checks the age of the cached response against the swr duration. If the cache data is within the swr limit, Accelerate serves the stale data while simultaneously refreshing the cache by fetching the latest data from the database.
Use swr in cacheStrategy and specify the SWR of the query in seconds:
When specifying a SWR of 60 seconds, the cache serves stale data until the cache refreshes itself in the background after each request:
Selecting a cache strategy
Caching helps you improve query response times and reduce database load. However, it also means you might serve stale data to the client. Whether or not serving stale data is acceptable and to what extent depends on your use case. ttl and swr are parameters you can use the tweak the cache behavior.
Cache strategy using TTL
Use TTL to reduce database load when stale cached data is acceptable.
Use case: Product catalog in e-commerce applications
Consider an e-commerce application with a product catalog that doesn't frequently change. By setting a ttl of, let's say, 1 hour, Prisma Client can serve cached product data for subsequent user requests within that hour without hitting the database. This significantly reduces the database load and improves the response time for product listing pages.
Cache strategy using SWR
Use SWR to respond quickly to requests with minimal stale data. While it does not reduce database load, it can improve response times significantly.
Use case: User profile in social media platforms
Imagine a social media platform where user profiles are frequently accessed. By leveraging swr with a duration of, let's say, 5 minutes, Accelerate can serve the cached user profile information quickly, reducing the latency for profile pages. Meanwhile, in the background, it refreshes the cache after every request, ensuring that any updates made to the profile are eventually reflected for subsequent requests.
Cache strategy using TTL + SWR
For very fast response times and reduced database load, use both TTL and SWR. You can use this strategy to fine-tune your application’s tolerance for stale data.
Use ttl and swr in cacheStrategy and specify the TTL and SWR of the query in seconds:
When specifying a TTL of 30 seconds and SWR of 60 seconds, the cache serves fresh data for the initial 30 seconds. Subsequently, it serves stale data until the cache refreshes itself in the background after each request:
Use case: News articles
Consider a news application where articles are frequently accessed but don't require real-time updates. By setting a ttl of 2 hours and an swr duration of 5 minutes, Prisma Client can serve cached articles quickly, reducing latency for readers. As long as the articles are within the ttl, users get fast responses. After the ttl expires, Prisma Client serves the stale articles for an additional 5 minutes while fetching the latest news from the database, maintaining a balance between performance and freshness.
Connection pooling
A connection pool is a storage of database connections that can be reused for future requests to the database. When a new connection is requested, it is retrieved from the pool if one is available. Once the connection is no longer needed, it is returned to the pool for reuse.
Connection pooling is important as it allows you to reuse existing connections instead of creating new ones, which can be expensive. It also allows you to limit the number of connections to the database, which can help prevent the database from being overloaded.
Accelerate provides built-in connection pooling by default. By simply using Accelerate, you get the benefits of connection pooling without having to configure anything. However, you can also configure the connection pool to suit your needs.
For more information about connection pooling in Prisma, see the documentation here.
Default connection pool size
By default, Accelerate calculates a default connection pool size using the formula num_physical_cpus * 2 + 1.
For example, a machine with 2 physical CPUs will have a default connection pool size of 2 * 2 + 1 or 5.
This means that Accelerate will create a maximum of 5 connections to the database. If more than 5 connections are requested, Accelerate will queue the requests until a connection is available.
Configuring the connection pool size
The connection pool size can be configured to a value other than the default via the database connection string.
To adjust the connection pool size, you may add the connection_limit parameter to the database connection string. The value of connection_limit is the maximum number of connections that can be created to the database.
For example, here is how you can set a connection pool size of 10:
Configuring the connection pool timeout
The connection pool timeout is the duration, measured in seconds, during which the query engine must process a specific query; failing to do so within this timeframe results in an exception being thrown, and the system proceeds to the next query in the queue.
Similar to the connection pool size, you may also configure the connection pool timeout via the database connection string. To adjust this value, you may add the pool_timeout parameter to the database connection string.
For example:
The default value for pool_timeout is 10 seconds.
Default cache strategy
Accelerate defaults to no cache to avoid counterintuitive issues. Caching can be a powerful tool for improving performance but can also be dangerous if not used correctly.
For example, consider writing a query on a critical path without explicitly defining a cache strategy. If you run the code, you might receive incorrect data without explanation. This could be caused by someone forgetting to disable the default implicit cache behavior. Implicit caching allows these counterintuitive issues to arise, leading to undesirable results.
You must explicitly opt-in to caching if you want to use it. This makes it clear to developers that caching is not enabled by default and helps prevent counterintuitive issues from occurring.
When no cache strategy is specified or during a cache miss, a Prisma Client with the Accelerate extension routes all queries to the database through a connection pool instance near the database region.
