Building a Scalable Bulk Messaging System with Fastify and AWS SNS - code-examples -

Building a Scalable Bulk Messaging System with Fastify and AWS SNS

Leverage the power and scalability of Amazon Simple Notification Service (SNS) combined with the speed of the Fastify web framework to build a robust system capable of broadcasting messages to a large number of subscribers efficiently. This guide provides a complete walkthrough, from initial project setup to deployment considerations, focusing on implementing bulk message publishing using the AWS SDK v3's PublishBatch API for optimal performance and cost-effectiveness.

This system solves the common challenge of needing to send notifications, alerts, or updates to potentially millions of endpoints (like mobile devices, email addresses, or other backend services) without overwhelming the sending application or incurring excessive costs from individual API calls. We'll build a simple Fastify API that accepts a list of messages and uses SNS topics and the PublishBatch operation to distribute them.

Technologies Used:

  • Node.js: The JavaScript runtime environment.
  • Fastify: A high-performance, low-overhead web framework for Node.js.
  • AWS SNS: A fully managed messaging service for both application-to-application (A2A) and application-to-person (A2P) communication.
  • AWS SDK v3 for JavaScript: Used to interact with AWS services, specifically SNS. We will primarily use @aws-sdk/client-sns.
  • dotenv: To manage environment variables for configuration.

System Architecture:

graph LR
    Client[Client Application] -->|HTTP POST Request| FastifyAPI[Fastify API Server];
    FastifyAPI -->|PublishBatchCommand| AWSSNS[AWS SNS Topic];
    AWSSNS -->|Delivers Messages| SubscriberA[Subscriber A (e.g., Lambda)];
    AWSSNS -->|Delivers Messages| SubscriberB[Subscriber B (e.g., SQS Queue)];
    AWSSNS -->|Delivers Messages| SubscriberC[Subscriber C (e.g., HTTPS Endpoint)];
    AWSSNS -->|Delivers Messages| SubscriberD[... other subscribers];

Prerequisites:

  • An AWS account with permissions to manage SNS and IAM.
  • Node.js (v18 or later recommended) and npm installed.
  • Basic understanding of Node.js, Fastify, and AWS concepts.
  • AWS Access Key ID and Secret Access Key configured for programmatic access OR an environment configured to use IAM Roles (recommended for deployed applications).
  • A preferred code editor (like VS Code).
  • Tools like curl or Postman for testing the API.

By the end of this guide, you will have a functional Fastify application capable of accepting bulk message requests via an API endpoint and efficiently publishing them to an AWS SNS topic using the PublishBatch method.

1. Setting Up the Project

Let's initialize our Node.js project and install the necessary dependencies.

  1. Create Project Directory: Open your terminal and create a new directory for the project, then navigate into it.

    mkdir fastify-sns-bulk-sender
cd fastify-sns-bulk-sender

  2. Initialize Node.js Project:

    npm init -y

    This creates a package.json file with default settings.

  3. Install Dependencies: We need Fastify, the AWS SDK v3 SNS client, and dotenv for managing environment variables.

    npm install fastify @aws-sdk/client-sns dotenv

  4. Install Development Dependencies (Optional but Recommended): We'll use nodemon for automatic server restarts during development.

    npm install --save-dev nodemon

  5. Create Project Structure: Set up a basic directory structure:

    fastify-sns-bulk-sender/
├── src/
│   ├── server.js       # Main Fastify application logic
│   └── routes/
│       └── sns.js        # Routes related to SNS operations
├── .env.example      # Example environment variables
├── .gitignore        # Files/folders to ignore in Git
└── package.json

  6. Configure .gitignore: Create a .gitignore file in the root directory and add the following lines to prevent sensitive information and unnecessary files from being committed:

    # Dependencies
node_modules/

# Environment Variables
.env

# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*

# Optional editor directories
.vscode/
.idea/

# OS generated files
.DS_Store
Thumbs.db

  7. Set up Environment Variables: Create a file named .env in the root directory. Do not commit this file to Git. Populate it with your AWS credentials (if using keys locally), the desired AWS region, and a placeholder for your SNS Topic ARN (we'll create this next). Also include a simple API key for basic security.

    • AWS_ACCESS_KEY_ID: Your AWS Access Key ID. Obtainable from the IAM console (suitable for local development).
    • AWS_SECRET_ACCESS_KEY: Your AWS Secret Access Key. Obtainable from the IAM console (suitable for local development).
    • AWS_REGION: The AWS region where your SNS topic will reside (e.g., us-east-1).
    • SNS_TOPIC_ARN: The Amazon Resource Name (ARN) of the SNS topic. We will get this in the next step.
    • API_KEY: A secret key clients must provide to use the API. Generate a strong random string.
    • PORT: The port the Fastify server will listen on (default: 3000).

    Important: While using Access Keys from the IAM console is possible for local development, it is strongly recommended to use IAM Roles or temporary credentials (e.g., via AWS SSO) for applications deployed to AWS environments (like EC2, ECS, Lambda). Avoid using long-lived user keys in production.

    Create .env (replace placeholders with actual values if using keys locally, otherwise ensure your environment provides credentials):

    # .env
AWS_ACCESS_KEY_ID=YOUR_AWS_ACCESS_KEY_ID_IF_NEEDED
AWS_SECRET_ACCESS_KEY=YOUR_AWS_SECRET_ACCESS_KEY_IF_NEEDED
AWS_REGION=us-east-1
SNS_TOPIC_ARN=YOUR_SNS_TOPIC_ARN_HERE
API_KEY=YOUR_SUPER_SECRET_API_KEY
PORT=3000

    Also, create a .env.example file to track required variables (without actual secrets):

    # .env.example
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_REGION=
SNS_TOPIC_ARN=
API_KEY=
PORT=3000

  8. Add Run Scripts to package.json: Modify the scripts section in your package.json for easier development and starting the server:

    {
  ""scripts"": {
    ""start"": ""node src/server.js"",
    ""dev"": ""nodemon src/server.js"",
    ""test"": ""echo \""Error: no test specified\"" && exit 1""
  }
}

2. AWS SNS Setup and IAM Configuration

Before writing code, we need to create the SNS topic and ensure our application has the necessary permissions.

  1. Create an SNS Topic:

    • Navigate to the AWS Management Console and go to the Simple Notification Service (SNS).
    • In the left navigation pane, click on Topics.
    • Click Create topic.
    • Choose the Standard type. FIFO topics have different constraints and use cases not covered here.
    • Enter a meaningful Name for your topic (e.g., bulk-broadcast-topic).
    • Scroll down and click Create topic.
    • Once created, copy the ARN (Amazon Resource Name) displayed on the topic details page. It will look like arn:aws:sns:us-east-1:123456789012:bulk-broadcast-topic.
    • Update the SNS_TOPIC_ARN variable in your .env file with this value.

  2. Configure IAM Permissions: Your AWS credentials (whether keys or an IAM Role) need permission to publish messages to the SNS topic. Create an IAM policy and attach it to the IAM user or role associated with your credentials.

    • Navigate to the IAM service in the AWS Console.
    • Go to Policies and click Create policy.
    • Switch to the JSON editor tab.
    • Paste the following policy document:
    {
    ""Version"": ""2012-10-17"",
    ""Statement"": [
        {
            ""Sid"": ""AllowSnsPublishAndPublishBatch"",
            ""Effect"": ""Allow"",
            ""Action"": [
                ""sns:Publish"",
                ""sns:PublishBatch""
            ],
            ""Resource"": ""arn:aws:sns:<your-region&gt;:<your-account-id&gt;:<your-topic-name&gt;""
        }
    ]
}

    Crucially, you must replace the entire string value for the Resource key (i.e., ""arn:aws:sns:<your-region&gt;:<your-account-id&gt;:<your-topic-name&gt;"") with the specific ARN of the SNS topic you created in the previous step.

    Why PublishBatch? We explicitly include sns:PublishBatch because our core functionality relies on this efficient batch operation. sns:Publish is included for potential fallback or single-message sending.

    • Click Next: Tags, then Next: Review.
    • Give the policy a descriptive Name (e.g., FastifySnsBulkPublishPolicy).
    • Add an optional Description.
    • Click Create policy.
    • Finally, attach this newly created policy to the IAM user (whose keys might be in .env for local dev) or, preferably for deployed applications, attach it to the IAM role your compute environment (EC2 instance, ECS task, Lambda function) uses.

3. Implementing Core Functionality (Fastify Server and SNS Integration)

Now, let's build the Fastify server and integrate the AWS SDK to send messages.

  1. Basic Fastify Server (src/server.js): Create the main server file. It will load environment variables, initialize Fastify, configure the SNS client, register routes, and start listening.

    // src/server.js
'use strict';

// Load environment variables (primarily for local development)
require('dotenv').config();

const Fastify = require('fastify');
const { SNSClient } = require('@aws-sdk/client-sns');
const snsRoutes = require('./routes/sns');

// Basic configuration validation
const requiredEnv = ['AWS_REGION', 'SNS_TOPIC_ARN', 'API_KEY'];
for (const envVar of requiredEnv) {
    if (!process.env[envVar]) {
        console.error(`Error: Missing required environment variable ${envVar}`);
        process.exit(1);
    }
}

// Initialize Fastify
const fastify = Fastify({
    logger: true, // Enable built-in Pino logger
});

// Initialize AWS SNS Client
// The SDK automatically picks up credentials from environment variables
// (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN),
// shared credential file (~/.aws/credentials), or an IAM role if
// running on EC2/ECS/Lambda etc. Using IAM roles is the recommended approach for deployed applications.
try {
    const snsClient = new SNSClient({
        region: process.env.AWS_REGION,
        // Add retry strategy customization here if needed
    });

    // Decorate Fastify instance with the SNS client and topic ARN for easy access in routes
    fastify.decorate('snsClient', snsClient);
    fastify.decorate('snsTopicArn', process.env.SNS_TOPIC_ARN);
    fastify.decorate('apiKey', process.env.API_KEY);

} catch (error) {
    fastify.log.error('Failed to initialize AWS SNS Client:', error);
    process.exit(1);
}


// Register routes
fastify.register(snsRoutes, { prefix: '/api/sns' });

// Basic root route
fastify.get('/', async (request, reply) =&gt; {
    return { hello: 'world' };
});

// Start the server
const start = async () =&gt; {
    try {
        const port = process.env.PORT || 3000;
        await fastify.listen({ port: port, host: '0.0.0.0' }); // Listen on all network interfaces
        fastify.log.info(`Server listening on port ${port}`);
    } catch (err) {
        fastify.log.error(err);
        process.exit(1);
    }
};

start();

    Why Decorate? Decorating the Fastify instance (fastify.decorate) makes the configured snsClient, snsTopicArn, and apiKey readily available within route handlers via the fastify object passed to the route registration function or the request.server object inside handlers, avoiding the need to re-import or re-initialize them everywhere.

  2. SNS Routes (src/routes/sns.js): Create the file to handle SNS-related API endpoints. We'll implement the /send-batch endpoint here.

    // src/routes/sns.js
'use strict';

const { PublishBatchCommand } = require('@aws-sdk/client-sns');
const { randomUUID } = require('crypto'); // For generating unique batch entry IDs

// Define the maximum number of messages allowed in a single PublishBatch request
const MAX_SNS_BATCH_SIZE = 10;
// Define the maximum payload size per PublishBatch request (slightly under 256KB to provide a buffer)
const MAX_SNS_BATCH_PAYLOAD_SIZE = 262000; // 256 KiB = 262144 bytes. Using 262000 provides a small safety margin.

async function snsRoutes(fastify, options) {

    // --- Pre-handler for API Key Authentication ---
    // This hook runs before the main route handler for all routes defined in this plugin
    fastify.addHook('preHandler', async (request, reply) =&gt; {
        const apiKey = request.headers['x-api-key'];
        if (!apiKey || apiKey !== fastify.apiKey) {
            fastify.log.warn('Unauthorized attempt blocked.');
            reply.code(401).send({ error: 'Unauthorized: Invalid or missing API Key' });
            return; // Stop further processing
        }
    });

    // --- Schema for /send-batch endpoint ---
    const sendBatchSchema = {
        description: 'Sends multiple messages in batches to the configured SNS topic.',
        tags: ['SNS'],
        headers: {
            type: 'object',
            properties: {
                'x-api-key': { type: 'string', description: 'Your assigned API Key.' }
            },
            required: ['x-api-key']
        },
        body: {
            type: 'object',
            required: ['messages'],
            properties: {
                messages: {
                    type: 'array',
                    minItems: 1,
                    items: {
                        type: 'object',
                        required: ['body'],
                        properties: {
                            body: { type: 'string', description: 'The content of the message.' },
                            // Example structure for message attributes (optional)
                            attributes: {
                                type: 'object',
                                additionalProperties: {
                                    type: 'object',
                                    required: ['DataType', 'StringValue'], // Or BinaryValue
                                    properties: {
                                        DataType: { type: 'string', enum: ['String', 'Number', 'Binary', 'String.Array'] },
                                        StringValue: { type: 'string' },
                                        BinaryValue: { type: 'string' } // base64 encoded binary
                                    }
                                },
                                description: 'Optional SNS message attributes.'
                            }
                        },
                    },
                    description: 'An array of message objects to send.'
                },
            },
        },
        response: {
            200: {
                description: 'Batch processing summary.',
                type: 'object',
                properties: {
                    totalMessagesAttempted: { type: 'integer' },
                    totalBatchesSent: { type: 'integer' },
                    successfulMessages: { type: 'integer' },
                    failedMessages: { type: 'integer' },
                    details: {
                        type: 'array',
                        items: {
                            type: 'object',
                            properties: {
                                batchId: { type: 'integer' },
                                successful: { type: 'array', items: { type: 'object' } },
                                failed: { type: 'array', items: { type: 'object' } }
                            }
                        }
                    }
                }
            },
            400: { // Bad Request
                description: 'Invalid input data.',
                type: 'object',
                properties: {
                    statusCode: { type: 'integer' },
                    error: { type: 'string' },
                    message: { type: 'string' }
                }
            },
            401: { // Unauthorized
                description: 'Invalid or missing API Key.',
                type: 'object',
                properties: { error: { type: 'string' } }
            },
            500: { // Internal Server Error
                description: 'Error processing batch request.',
                type: 'object',
                properties: { error: { type: 'string' }, message: { type: 'string' } }
            }
        }
    };

    // --- /send-batch Route ---
    fastify.post('/send-batch', { schema: sendBatchSchema }, async (request, reply) =&gt; {
        const messages = request.body.messages;
        const { snsClient, snsTopicArn, log } = fastify; // Destructure decorated items
        let batches = [];
        let currentBatch = [];
        let currentBatchSize = 0;

        // --- Batching Logic ---
        // Split messages into batches respecting AWS SNS limits (size and count)
        log.info(`Starting batching for ${messages.length} messages.`);
        for (const message of messages) {
            // Caveat: This size calculation is an approximation. It only considers the message body's
            // byte length and does NOT account for the overhead of the message ID, message attributes,
            // or the JSON structure of the PublishBatchRequestEntries itself. For messages with many
            // attributes or complex structures, this could underestimate the actual size.
            // Be conservative with MAX_SNS_BATCH_PAYLOAD_SIZE or implement more precise calculation if needed.
            const messageString = JSON.stringify(message.body);
            const messageSizeApproximation = Buffer.byteLength(messageString, 'utf8'); // Approximation!
            const entryId = randomUUID(); // Unique ID for each message within the batch

            // Check if adding the message exceeds size or count limits
            if (currentBatch.length &gt; 0 && (currentBatch.length &gt;= MAX_SNS_BATCH_SIZE || currentBatchSize + messageSizeApproximation &gt; MAX_SNS_BATCH_PAYLOAD_SIZE)) {
                batches.push(currentBatch); // Finalize the current batch
                log.debug(`Finalized batch ${batches.length} with ${currentBatch.length} messages, size ~${currentBatchSize} bytes.`);
                currentBatch = [];        // Start a new batch
                currentBatchSize = 0;
            }

            // Construct the batch entry
            const batchEntry = {
                Id: entryId, // Must be unique within the batch (up to 80 chars)
                Message: message.body,
                // Add MessageAttributes if present in the input message object
                ...(message.attributes && { MessageAttributes: message.attributes })
                // Example structure if constructing manually:
                // MessageAttributes: {
                //   'attributeName': { DataType: 'String', StringValue: 'attributeValue' },
                //   'anotherAttr': { DataType: 'Number', StringValue: '123.45' }
                // }
            };

            // Add message to the current batch
            currentBatch.push(batchEntry);
            currentBatchSize += messageSizeApproximation; // Add approximate size
        }

        // Add the last batch if it's not empty
        if (currentBatch.length &gt; 0) {
            batches.push(currentBatch);
            log.debug(`Finalized last batch ${batches.length} with ${currentBatch.length} messages, size ~${currentBatchSize} bytes.`);
        }

        log.info(`Split ${messages.length} messages into ${batches.length} batches.`);

        // --- Sending Batches to SNS ---
        let successfulMessages = 0;
        let failedMessages = 0;
        const results = [];

        for (let i = 0; i &lt; batches.length; i++) {
            const batchEntries = batches[i];
            const command = new PublishBatchCommand({
                TopicArn: snsTopicArn_
                PublishBatchRequestEntries: batchEntries_
            });

            log.debug(`Sending batch ${i + 1}/${batches.length} with ${batchEntries.length} entries.`);

            try {
                const response = await snsClient.send(command);
                log.info(`Batch ${i + 1}/${batches.length} sent. Success: ${response.Successful?.length || 0}_ Failed: ${response.Failed?.length || 0}`);

                successfulMessages += response.Successful?.length || 0;
                failedMessages += response.Failed?.length || 0;

                results.push({
                    batchId: i + 1_
                    successful: response.Successful || []_
                    failed: response.Failed || []_
                });
                if (response.Failed && response.Failed.length &gt; 0) {
                    log.warn(`Failures reported in batch ${i + 1}: ${JSON.stringify(response.Failed)}`);
                }

            } catch (error) {
                log.error(`Error sending batch ${i + 1}: ${error.name} - ${error.message}`, error);
                // Simplification: Assume all messages in this batch failed if the API call itself errors
                // (e.g., network issue, throttling). This doesn't distinguish from partial failures
                // that might theoretically occur or require different handling strategies.
                failedMessages += batchEntries.length;
                results.push({
                    batchId: i + 1,
                    successful: [],
                    failed: batchEntries.map(entry =&gt; ({ // Construct basic failure info
                        Id: entry.Id,
                        Code: error.name || 'SendCommandError',
                        Message: error.message || 'Failed to send batch command',
                        SenderFault: true // Assume client-side, SDK, or network error
                    })),
                });
                 // Depending on the error type (e.g., ThrottlingException), you might want to break,
                 // implement retries with backoff, or adjust sending rate.
                 // For simplicity here, we continue to the next batch.
            }
             // Optional: Uncomment to add a small delay between batches if encountering SNS rate limits
             // await new Promise(resolve =&gt; setTimeout(resolve, 100));
        }

        log.info(`Batch processing complete. Total Successful: ${successfulMessages}, Total Failed: ${failedMessages}`);

        // --- Respond to Client ---
        reply.code(200).send({
            totalMessagesAttempted: messages.length,
            totalBatchesSent: batches.length,
            successfulMessages: successfulMessages,
            failedMessages: failedMessages,
            details: results,
        });
    });

    // Add other SNS-related routes here if needed (e.g., single send, topic management)
}

module.exports = snsRoutes;

    Why PublishBatchCommand Directly? While plugins like fastify-aws-sns (found in research) simplify some SNS operations, they often don't expose specialized APIs like PublishBatch. Using the @aws-sdk/client-sns directly gives us full control and access to all SNS API actions, including the crucial batch operation for efficiency. Why Batching Logic? The PublishBatch API has strict limits: max 10 messages per batch and max 256KB total payload size per batch. The code iterates through the input messages, creating batches that respect these limits before sending them. Each message within a batch needs a unique Id. A caveat regarding the accuracy of the payload size calculation has been added. Why Schema Validation? Fastify's built-in schema support (using JSON Schema) automatically validates incoming request bodies, headers, query parameters, etc. This eliminates boilerplate validation code, improves security by rejecting malformed requests early, and automatically generates documentation (if using plugins like fastify-swagger).

4. API Layer Details and Testing

We've defined the API endpoint /api/sns/send-batch with request validation and basic API key authentication.

  • Authentication: A simple API key check is implemented using a preHandler hook. Clients must include the correct API key (from your .env) in the x-api-key HTTP header. Note: This method is basic. For production systems, implement more robust authentication/authorization mechanisms like JWT, OAuth2, or platform-specific identity solutions, potentially leveraging Fastify plugins like fastify-jwt or fastify-oauth2.
  • Request Validation: The sendBatchSchema ensures the request body contains a non-empty array named messages, where each item is an object with at least a body property (string). Invalid requests will receive a 400 Bad Request response.
  • API Endpoint: POST /api/sns/send-batch
  • Headers:
    • Content-Type: application/json
    • x-api-key: YOUR_SUPER_SECRET_API_KEY (Replace with the value from .env)
  • Request Body (JSON): 
    {
  ""messages"": [
    { ""body"": ""This is the first message content."" },
    { ""body"": ""This is another message for the batch."" },
    { ""body"": ""A third update!"" }
  ]
}
    Optional with attributes: 
    {
  ""messages"": [
    {
      ""body"": ""Message with attributes"",
      ""attributes"": {
        ""category"": { ""DataType"": ""String"", ""StringValue"": ""updates"" },
        ""priority"": { ""DataType"": ""Number"", ""StringValue"": ""1"" }
      }
    },
    { ""body"": ""Another message"" }
  ]
}
  • Successful Response (200 OK - JSON): 
    {
  ""totalMessagesAttempted"": 3,
  ""totalBatchesSent"": 1,
  ""successfulMessages"": 3,
  ""failedMessages"": 0,
  ""details"": [
    {
      ""batchId"": 1,
      ""successful"": [
        { ""Id"": ""uuid-1"", ""MessageId"": ""sns-message-id-1"", ""SequenceNumber"": null },
        { ""Id"": ""uuid-2"", ""MessageId"": ""sns-message-id-2"", ""SequenceNumber"": null },
        { ""Id"": ""uuid-3"", ""MessageId"": ""sns-message-id-3"", ""SequenceNumber"": null }
      ],
      ""failed"": []
    }
  ]
}
  • Example curl Test: Replace placeholders with your actual API key and local server address/port. Using single quotes around the -d payload is generally more robust across different shells. 
    curl -X POST http://localhost:3000/api/sns/send-batch \
-H 'Content-Type: application/json' \
-H 'x-api-key: YOUR_SUPER_SECRET_API_KEY' \
-d '{
  ""messages"": [
    { ""body"": ""Test message 1 via curl"" },
    { ""body"": ""Another test message from curl command"" },
    { ""body"": ""Message number three for batch testing"" },
    { ""body"": ""Four"" },
    { ""body"": ""Five"" },
    { ""body"": ""Six"" },
    { ""body"": ""Seven"" },
    { ""body"": ""Eight"" },
    { ""body"": ""Nine"" },
    { ""body"": ""Ten"" },
    { ""body"": ""This one starts the second batch!"" }
  ]
}'
    You should see a response indicating 11 messages attempted, 2 batches sent, and details about success/failure for each batch.

5. Error Handling, Logging, and Retries

  • Logging: Fastify's default logger (pino) is enabled (logger: true). We use fastify.log.info, fastify.log.warn, and fastify.log.error to record significant events and errors. Logs will output to the console by default. In production, configure log destinations (e.g., files, CloudWatch Logs) and log levels appropriately.
  • Error Handling:
    • Schema Validation: Fastify handles invalid request formats automatically, returning 400 errors.
    • API Key: The preHandler hook returns a 401 Unauthorized error.
    • SNS Errors: The try...catch block around snsClient.send(command) catches errors during the PublishBatch API call (e.g., network issues, throttling, authentication problems). The code logs these errors. Note: This catch block currently assumes the entire batch failed if the send command throws an error, which is a simplification. A network error might occur after some messages were processed by SNS, although the API typically succeeds or fails atomically.
    • Partial Batch Failures: The response from a successful PublishBatch API call contains Successful and Failed arrays detailing the outcome for each individual message within the batch. The code logs these details, including warnings for messages in the Failed array.
    • Unhandled Errors: Fastify has default error handlers, but you can customize them using fastify.setErrorHandler for more specific responses or reporting.
  • Retry Mechanisms:
    • AWS SDK Retries: The AWS SDK v3 has built-in retry logic with exponential backoff for transient network errors and certain server-side errors (like ThrottlingException). You can customize the default retry strategy (e.g., number of retries) when initializing the SNSClient.
    • Application-Level Retries: For failures reported within the Failed array of a PublishBatch response (e.g., due to InternalError reported by SNS for a specific message, though less common for standard topics), you might implement application-level retries:
      1. Identify retryable error codes reported in the Failed array (e.g., InternalError).
      2. Collect the original message data corresponding to the failed entry IDs (entry.Id).
      3. Re-batch these specific messages and attempt to send them again after a delay (using exponential backoff).
      4. Consider using a library like async-retry to manage this retry loop.
    • Throttling Retries: If ThrottlingException is caught frequently, besides relying on SDK retries, consider adding a delay between sending batches (see commented code in sns.js) or implementing a more sophisticated rate-limiting mechanism on the sending side.
    • Current Implementation: The provided code relies primarily on SDK retries and logs other failures but doesn't implement sophisticated application-level retries for individual message failures within a batch. Add this based on your specific reliability requirements.

6. Database Schema and Data Layer

This specific guide focuses on the messaging transport layer (Fastify API to SNS) and doesn't require its own database.

However, in a real-world application, you would likely need a database for:

  • User/Subscriber Management: Storing user profiles, device tokens, email addresses, or other endpoint information needed to target messages (though SNS subscriptions handle the delivery mechanism itself).
  • Message Tracking/Auditing: Recording which messages were sent, when, to which logical groups of users, and potentially the success/failure status reported back from SNS (if you implement feedback loops, e.g., via SNS Delivery Status logging).
  • Content Management: Storing message templates or content snippets if messages aren't entirely generated on the fly.
  • API Key/Client Management: If you have multiple clients using the API, storing their credentials and permissions.

The choice of database (SQL like PostgreSQL, NoSQL like DynamoDB or MongoDB) depends heavily on the specific requirements of these related features. Integrating a database would typically involve adding an ORM (like Prisma, Sequelize) or a database client library to the Fastify application and creating data access logic separate from the route handlers.