Skip to main content

Durable Promises

What is a Durable Promise?

Durable promises are like regular promises but they persist in storage as a REST resource. They have a unique identity that lives beyond the execution of the underlying function. By doing this, if something goes wrong (like a power outage or network hiccup), the application can look back at the saved results, quickly skip to where it left off, and keep going as if nothing ever happened, making your application resilient to unexpected failures and gracefully recover.

Storage​

Where are Durable Promises stored?

There are two places where a Durable Promise may be stored, locally or remotely.

Local storage​

Local storage, sometimes referred to as "Local mode" is when the promise is stored in local memory.

Local in memory promise storage diagram

This is in contrast to Remote storage (Remote mode), and provides function level recovery. In other words, if a called function fails (throws an error, or rejects a promise) in local mode then it is retried within the bounds of the timeout budget.

Local in memory promise storage with a retry

Use local storage when you require very low latency and are not concerned with the Application Node restarting the execution from scratch should it crash but you still want individual functions to be transparently retried if they fail.

Part 1 of the quickstart tutorials showcases this functionality:

Remote storage​

Remote storage, sometimes referred to as "Remote mode", is when the promise is stored in the Resonate Server (the supervisor service).

Remote promise storage diagram

Using Remote storage enables the "distribution" aspect of Distributed Async Await and provides application-level (also known as "process-level") recovery.

First, let's look at application-level recovery, for example when your (Application Node) crashes. Remote storage can ensure that when your process comes back up, any "in-progress" executions continue from where they left off.

Part 2 of the quickstart tutorials showcases this functionality:

Using the same sequential function execution from the quickstart tutorial a sequential execution diagram might look this:

Remote promise storage diagram with retries

In the diagram above, notice how function 1 gets the result of function 2 from the Durable Promise after the process is restarted. That is because the result of function 2 was stored in promise 2 before the process crashed. This effectively resumes the execution of function 1 from where it left off.

Now let's look at the distribution aspect that is enabled by Remote storage.

The remote use case involves a durable promise that is created by one process and completed by another distinct process. The primary purpose is to facilitate coordination between different processes or services, serving as the foundation for features like:

  • Task framework: Durable promises allow you to distribute tasks across multiple machines for parallel execution and collect the operations’ results.

  • Notifications: When a durable promise is created or completed, it can trigger notifications to other processes or services that are interested in the result, enabling efficient communication and coordination.

  • Human in the Loop: Durable promises can seamlessly integrate human input into your automated workflows, allowing for manual intervention or approval steps when needed.

Identity​

In distributed async/await applications, durable promises must have a unique identifier (ID). This ID is used to store the computational progress and final result of the execution, allowing it to resume seamlessly from where it left off if interrupted.

purchase.ts
// UID uniquely identifies the purchase.
const uid = `purchase/user/${user.id}/song/${song.id}`;

// Run the registered 'purchase' function with the above uid and the following function arguments.
const val = await resonate.run("purchase", uid, user, song);

When designing the naming scheme for your durable promise IDs, keep the following considerations in mind:

  1. Uniqueness: The naming scheme should guarantee uniqueness to avoid conflicts between executions.
  2. Readability: Choose a naming scheme that is easy to understand and interpret, making it easier to debug and manage executions.
  3. Relevance: Incorporate relevant information into the naming scheme, such as the purpose or context of the execution.
note

If an ID is accidentally reused for a different execution, it will result in retrieving the stored result of the previous execution instead of starting a new one. This behavior differs from regular executions and can lead to confusion if not handled properly.

Common ID naming schemes​

There are several approaches to ensure your Durable Promise IDs are unique but also readable and relevant.

Date-Based​

One very common approach is to use the date as part of the naming scheme to. For example, if you have a durable promise that fetches and aggregates news articles on a daily basis, you could include the date in the ID format to ensure uniqueness and provide clear indication of when the execution occurred.

news_feed_YYYY-MM-DD

Hierarchical​

You can use a hierarchical naming scheme similar to file system paths to represent the identity of a durable promise. The naming scheme can include information such as the environment, service, and specific execution details. For example:

staging/analytics/monthly-report/2023-05

Platform-specific​

If your durable promises are running on a specific platform or orchestrator, you can incorporate the platform's identity concepts into the naming scheme. For example, if you are using Kubernetes, you can include the namespace, pod, and other relevant information:

k8s/staging/namespace/analytics/gpu/h100/monthly-report-2023-05

Opaque with Metadata​

In this case, the durable promise ID is a randomly generated unique identifier, and you would store the associated metadata (such as environment, service, execution details) in a separate database that can be queried using the ID.

executions/a7b89c3d-f012-4e78-9a7d-89a3f6b2e1c7