Merge pull request #251 from dahlia/mq-native-retrial

To be released.

 -  Added optional `MessageQueue.nativeRetrial` property to indicate whether

    the message queue backend provides native retry mechanisms.  When `true`,

    Fedify will skip its own retry logic and rely on the backend to handle

    retries.  When `false` or omitted, Fedify will handle retries using its

    own retry policies.  [[#250], [#251]]

     -  `DenoKvMessageQueue.nativeRetrial` is `true`.

     -  `WorkersMessageQueue.nativeRetrial` is `true`.

     -  `InProcessMessageQueue.nativeRetrial` is `false`.

     -  `ParallelMessageQueue.nativeRetrial` inherits from the wrapped queue.

[#250]: https://github.com/fedify-dev/fedify/issues/250

[#251]: https://github.com/fedify-dev/fedify/pull/251

Version 1.6.1

-------------

the `createExponentialBackoffRetryPolicy()` function, which is a default

implementation of the retry policy.

> [!NOTE]

> This policy is ignored when using message queue backends that provide native

> retry mechanisms (`MessageQueue.nativeRetrial` is `true`).  In such cases,

> the backend's native retry logic takes precedence.

### `inboxRetryPolicy`

*This API is available since Fedify 0.12.0.*

type.  Or you can adjust the parameters of the built-in

`createExponentialBackoffRetryPolicy()` function.

> [!NOTE]

> This policy is ignored when using message queue backends that provide native

> retry mechanisms (`MessageQueue.nativeRetrial` is `true`).  In such cases,

> the backend's native retry logic takes precedence.

### `activityTransformers`

*This API is available since Fedify 1.4.0.*

incoming activities.

With the `queue` enabled, the failed activities are automatically retried

after a certain period of time.  The default retry strategy is exponential

backoff with a maximum of 10 retries, but you can customize it by providing

an [`inboxRetryPolicy`](./federation.md#inboxretrypolicy) option to

the `createFederation()` function.

after a certain period of time.  By default, Fedify handles retries using

exponential backoff with a maximum of 10 retries, but you can customize it

by providing an [`inboxRetryPolicy`](./federation.md#inboxretrypolicy) option

to the `createFederation()` function.

However, if your message queue backend provides native retry mechanisms

(indicated by `MessageQueue.nativeRetrial` being `true`), Fedify will skip

its own retry logic and rely on the backend to handle retries.  This avoids

duplicate retry mechanisms and leverages the backend's optimized retry features.

> [!NOTE]

> Activities with invalid signatures/proofs are silently ignored and not queued.

:   Simple, no external dependencies.

Cons

:   Not suitable for production, doesn't persist messages between restarts.

:   Not suitable for production, doesn't persist messages between restarts,

    no native retry mechanism.

~~~~ typescript twoslash

import type { KvStore } from "@fedify/fedify";

:   Production use in Deno environments.

Pros

:   Persistent, scalable, easy to set up.

:   Persistent, scalable, easy to set up, native retry with exponential backoff.

Cons

:   Only available in Deno runtime.

:   Production use in Cloudflare Workers environments.

Pros

:   Persistent, reliable, scalable, easy to set up.

:   Persistent, reliable, scalable, easy to set up, native retry with exponential

    backoff and dead-letter queues.

Cons

:   Only available in Cloudflare Workers runtime.

} from "@fedify/fedify";

class CustomMessageQueue implements MessageQueue {

  // Set to true if your backend provides native retry mechanisms

  readonly nativeRetrial = false;

  async enqueue(

    message: any,

    options?: MessageQueueEnqueueOptions,

    messages are processed only once.

However, you don't need to implement retry logic yourself, as Fedify handles

retrying failed messages automatically.

retrying failed messages automatically.  If your message queue backend provides

native retry mechanisms (like exponential backoff, dead-letter queues, etc.),

you can set the `nativeRetrial` property to `true` to indicate this.

When this property is `true`, Fedify will skip its own retry logic and rely

on your backend to handle retries, avoiding duplicate retry mechanisms.

Parallel message processing

`ParallelMessageQueue`, a special implementation of the `MessageQueue` interface

designed to process messages in parallel.  It acts as a decorator for another

`MessageQueue` implementation, allowing for concurrent processing of messages

up to a specified number of workers:

up to a specified number of workers.  The `ParallelMessageQueue` inherits

the `nativeRetrial` property from the wrapped queue:

~~~~ typescript twoslash

import type { KvStore } from "@fedify/fedify";

> you should not place the `NODE_TYPE=worker` nodes behind a load balancer.

Native retry mechanisms

-----------------------

*This API is available since Fedify 1.7.0.*

Some message queue backends provide their own retry mechanisms with features

like exponential backoff, dead-letter queues, and automatic failure handling.

To avoid duplicate retry logic and improve efficiency, Fedify supports

the `~MessageQueue.nativeRetrial` property on `MessageQueue` implementations.

When `MessageQueue.nativeRetrial` is `true`, Fedify will skip its own retry

logic and rely entirely on the backend's native retry mechanisms.

When `false` or omitted, Fedify handles retries using its own retry policies.

### Current implementations

The following implementations currently support native retry:

`DenoKvMessageQueue`

:   Deno KV provides automatic retry with exponential backoff

    (`~MessageQueue.nativeRetrial` is `true`).

`WorkersMessageQueue`

:   Cloudflare Queues provide automatic retry with exponential backoff and

    dead-letter queues (`~MessageQueue.nativeRetrial` is `true`).

The following implementations do not yet support native retry:

`InProcessMessageQueue`

:   No native retry support (`~MessageQueue.nativeRetrial` is `false`).

[`RedisMessageQueue`]

:   Native retry support planned for future release.

[`PostgresMessageQueue`]

:   Native retry support planned for future release.

[`AmqpMessageQueue`]

:   Native retry support planned for future release.

`ParallelMessageQueue` inherits the `~MessageQueue.nativeRetrial` value from

the wrapped queue.

### Benefits of native retry

Using native retry mechanisms provides several advantages:

Reduced overhead

:   Eliminates duplicate retry logic between Fedify and the message queue

    backend.

Better reliability

:   Leverages proven retry mechanisms from established queue backends.

Improved observability

:   Backend-native retry mechanisms often provide better monitoring and

    debugging capabilities.

Optimized performance

:   Backend-specific optimizations for retry logic.

Using different message queues for different tasks

--------------------------------------------------

> For further information, see the [*Message queue* section](./mq.md).

The failed activities are automatically retried after a certain period of time.

The default retry strategy is exponential backoff with a maximum of 10 retries,

but you can customize it by providing

By default, Fedify handles retries using exponential backoff with a maximum of

10 retries, but you can customize it by providing

an [`outboxRetryPolicy`](./federation.md#outboxretrypolicy) option to

the `createFederation()` function.

However, if your message queue backend provides native retry mechanisms

(indicated by `MessageQueue.nativeRetrial` being `true`), Fedify will skip

its own retry logic and rely on the backend to handle retries.  This avoids

duplicate retry mechanisms and leverages the backend's optimized retry features.

If the `queue` is not set, the `~Context.sendActivity()` method immediately

sends the activity to the recipient's inbox.  If the delivery fails, it throws

an error and does not retry the delivery.