Next JS API and AWS S3

So one thing I love about using NextJS is that on top of using React as it's frontend it also has a built in API so you can create internal endpoints for your website. In this example we are going to create a file uploader tool, that first sends the file to our API, then the API uploads the file to our S3 Bucket.

Arguably you could just upload the file directly from the frontend to S3 using Signed URL or through Cognito (that's generally how it should be done in production anyhow) but I wanted to demonstrate how simple it is to do it using the NextJS API directly with IAM credentials. This method is useful if you're building your own internal tooling systems.

Create an S3 Bucket

First, sign into your AWS account, and lets create a new bucket, in this example we are using all of the default settings, so that means it'll be a "non-public" bucket by default.

So, simply go to S3, click the Create Bucket button, type a bucket name, and use all the default settings.

Setup your IAM Policy and Create a user.

Now within AWS navigate to the IAM setup, click "policies" on the left had side, and click Create Policy, choose a name for your policy, and then paste in the following:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:ListAllMyBuckets",
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::YOUR_BUCKET_NAME",
                "arn:aws:s3:::YOUR_BUCKET_NAME/*"
            ]
        }
    ]
}

...make sure you change "YOUR_BUCKET_NAME" to the name of your actual bucket from the first step.

Now follow the steps to finish creating your policy (you can simply use the default options) and now lets create a new user, so go to Users on the left menu, click on Add User, choose a name for the User and we only need to click the checkbox for "Access key - Programmatic access", now click "Next:Permissions" and click the "Attach existing policies directly" button, search for your recently created policy and click the checkbox, then click "Next:Tags", then click "Next:Review" and then "Create User".

Add your IAM user and Bucket name to .env.local

In order for the API to be able to use our credentials, we need to add them to our .env.local file, so create a new file called .env.local in the root of your project, and add the following:

API_S3_UPLOADER_USER=[YOUR_ACCESS_KEY_ID]
API_S3_UPLOADER_SECRET=[YOUR_SECRET_ACCESS_KEY]
API_S3_UPLOADER_BUCKET=[YOUR_BUCKET_NAME]

...adding these to our .env.local file means that our NextJS API can read them, but they are not visible to the public.

Create the frontend code and API endpoints

Now we have our S3 bucket and IAM user setup, we can create our API and Frontend within our Next JS site, see below for the code, I've kept these as simple as possible, to try and show you whats going on. NextJS API by default also has a 4.5 Mb upload limit as well, so we have to be mindful of this.

Front End Component:

import { useState } from "react";

export default function UploadFilesToDiskTool(){
    const [ uploadVisible, setUploadVisible ] = useState(false);
    const [image, setImage] = useState<null | File>(null);
    
    const uploadToClient = (event : React.ChangeEvent<HTMLInputElement>) => {
        setUploadVisible( event.target.files && event.target.files.length > 0 ? true : false);
        if (event.target.files && event.target.files[0]) {
        const imageData = event.target.files[0];

        setImage(imageData);
        }
    };
    const uploadToServer = async () => {
        const body = new FormData();
        // Very basic error checking
        if( !image ) { console.log('No image data!! Halting Upload'); return;}
        if (image.size > 4.5e6) { console.log("File size is too large. Max size is 4.5MB");return;}
        body.append("file", image);
        const data = await fetch("/api/upload-files-to-disk", {
            method: "POST",
            body
        });
        const response = await data.json();
        console.log(response)
    };

    return (
        <>
        <div>
            <input
                accept={`image/*`}
                multiple={false}
                name={'theFiles'}
                onChange={uploadToClient}
                type="file"
            />
            <button 
                onClick={uploadToServer} 
                disabled={!uploadVisible}
                className={ (uploadVisible ? 'bg-green-500' : 'bg-gray-500') + (' text-white font-bold py-2 px-4 rounded')}
            >Upload</button>
        </div>    
        </>
    )
}

API Code:

This API uses the formidable plugin, and we could locate it in /pages/api/my-uploader.ts for example

import formidable from "formidable";
import fs from "fs";
import aws from 'aws-sdk';
import { NextApiResponse } from "next";

export const config = {
  api: {
    bodyParser: false
  }
};

const post = async (req : any , res : NextApiResponse) => {
  const form = new formidable.IncomingForm();
  form.parse(req, async function (err, fields, files :any) {
    await saveFile(files.file);
    return res.status(201).send({ data: "success" });
  });
};

const { API_S3_UPLOADER_BUCKET , API_S3_UPLOADER_USER, API_S3_UPLOADER_SECRET } = process.env

aws.config.update({
  accessKeyId: API_S3_UPLOADER_USER,
  secretAccessKey: API_S3_UPLOADER_SECRET,
  region: 'sa-east-1',
  signatureVersion: 'v4',
});

interface IFileInput  {
  filepath : string;
  originalFilename : string;
}

const saveFile = async (file : IFileInput) => {
  const data = fs.createReadStream(file.filepath);

    const s3 = new aws.S3();
    const post = await s3.upload({
      Bucket: API_S3_UPLOADER_BUCKET,
      Key: file.originalFilename,
      Body: data,
    }, function (err : any, data : any) {
      if (err) {
          console.log("Error", err);
      } if (data) {
          console.log("Upload Success", data.Location);
      }
  });
    await fs.unlinkSync(file.filepath);
    return 'UPLOADED';
};

interface IReqInput  {
  method: string;
}

const apiFunc = (req : IReqInput, res : NextApiResponse) => {
  req.method === "POST"
    ? post(req, res)
    : res.status(404).send("");
};

export default apiFunc;

File Uploader using NextJS API and AWS S3