Loading

Amazon S3 with Express

Uploading Files to Amazon S3 with Node.js and Express

Barnabas Nomo | Thu, September 28th 2017


What is Amazon S3?

According to its website:

Amazon Simple Storage Service is storage for the Internet designed to make web-scale computing easier for developers by using simple web services interface that can store and retrieve any amount of data, at any time, from anywhere on the web; giving any developer access to the same highly scalable, reliable, fast, inexpensive data storage infrastructure that Amazon uses to run its own global network of web sites. The service aims to maximize benefits of scale and to pass those benefits on to developers.

We'll look at a simple example on how to upload a file to Amazon S3 from an express server instance. But first, let's make sure we have the following resources.

NOTE: I'm assuming you already have Node.js as well as Express Generator installed.
  • Amazon Web Services Account
  • Amazon S3 Bucket ( very easy to setup after signing in.)
  • The following npm packages for the project scope dotenv connect-multiparty aws-sdk
  • Amazon API Credentials

First, let's create our express application skeleton:

express --view=ejs test-s3

Now let's navigate into our test-s3 directory and install the modules

$ cd test-s3
$ npm install

Next, let's install the other modules

$ npm install --save dotenv connect-multiparty aws-sdk

In our root directory, let's create a file called .env which will store our  environment variables for this project. and let's fill it up.

S3_KEY_ID=pUtYourKeYHeRe
S3_SECRET_ACCESS_KEY=putSecretAccessHere
S3_BUCKET_NAME=bucketNameHere

On the first line of your app.js file, put the following:

require('dotenv').config()

Most of the edits will be in routes/index.js which at the moment has just the following content:

var express = require('express');
var router = express.Router();

/* GET home page. */
router.get('/', function(req, res, next) {
  res.render('index', { title: 'Express' });
});

module.exports = router;

let's make it upload files to our Amazon S3 Bucket when we send a post request to /upload

routes/index.js

var express = require('express');
var router = express.Router();
var fs = require('fs');

// require multiparty and aws-sdk
var multipart = require('connect-multiparty');
var AWS = require('aws-sdk');

router.use('/upload', multipart());

// AWS config
AWS.config.update(
    {
        accessKeyId: process.env.S3_KEY_ID,
        secretAccessKey: process.env.S3_SECRET_ACCESS_KEY
    });


/* GET home page. */
router.get('/', function(req, res, next) {
  res.render('index', { title: 'Express' });
});

router.post('/upload', function(req, res, next) {
    fs.readFile(req.files['images'].path, function (err, data) {
        var options = {partSize: 10 * 1024 * 1024, queueSize: 1};
        var base64data = new Buffer.from(data, 'binary');
        var s3 = new AWS.S3();
        s3.upload(
            {
                Bucket: process.env.S3_BUCKET_NAME,
                Key: "images/"+req.files['images'].name,
                Body: base64data,
                ACL: 'public-read'
            },
            options,
            function (err, data) {
                if (err) {
                    console.log(err);
                    return res.send(err)
                }
                console.log(data);
                res.send(data);
            });
    });
});


module.exports = router;

In views/index.ejs


  <%= title %>
    
  
  <%= title %>Welcome to <%= title %>
  

 

Our Awesome Form is Here!!!!

Let's add our form. It is very important to set enctype="multipart/form-data" since we are trying to upload files.

views/index.ejs


<form action="/upload" enctype="multipart/form-data" method="post">
    <input class="btn btn-default btn-sm" name="images" type="file">
    <input class="btn btn-success" type="submit" value="Submit" >
</form>

 

Voila!!!

If everything is successful, you should receive a JSON Response of the file's details.

{
    "ETag":"1a132d3286ead6ddc496e240964ac0a7",
    "Location":"https://{your-bucket-name}.s3.amazonaws.com/images/54ae31ac2abd65612e5f9f650368573a.680x680x1.jpg",
    "key":"images/54ae31ac2abd65612e5f9f650368573a.680x680x1.jpg",
    "Key":"images/54ae31ac2abd65612e5f9f650368573a.680x680x1.jpg",
    "Bucket":"{your-bucket-name}"
}