Brad Clonan Code Notebook

Overview

...

Table of Contents

Express Deep Dive

Folder Use

Overview

...

Table of Contents

...

Express Lingo

Term Definition
Api Application programming interface. Spell out the abbreviation when it is first used.
Libuv A multi-platform support library which focuses on asynchronous I/O, primarily developed for use by Node.js.
Middleware A function that is invoked by the Express routing layer before the final request handler, and thus sits in the middle between a raw request and the final intended route. A few fine points of terminology around middleware:
request An HTTP request. A client submits an HTTP request message to a server, which returns a response. The request must use one of several request methods such as GET, POST, and so on.
response An HTTP response. A server returns an HTTP response message to the client. The response contains completion status information about the request and might also contain requested content in its message body.
route Part of a URL that identifies a resource. For example, in http://foo.com/products/id, “/products/id” is the route.
Templating Engines JavaScript template engines are often used when writing thick JS-clients or as "normal" template engines when using server-side js like Node.js.

Instead of cluttering the code by generating HTML using string concatenating etc., you instead use a template and interpolate the variable in them.

For instance in the example bellow you pass in an array foo to the template engine which then interpolates (replaces the placeholder ${data}) with item i from the array foo:|

Middleware

Middleware

Quick facts

  • middleware is called sequentially (in order)
  • Generally you should put each middleware function in seperate folders

Middleware examples

//requiring or using a Node.js module
var foo = require('middleware') 

// Then the statement typically returns the middleware.
var mw = foo() 

// Add the middleware to the global processing stack.
app.use(mw) 

// Add middleware to the “GET /foo” processing stack.
app.get('/foo', mw, function (req, res) { ... })
1
2
3
4
5
6
7
8
9
10
11
12

Built in middleware(express)

express ships with some middleware built in already a few of them are listed bellow

function use
express.json() parse body request as json
express.urlencoded() key value pair parsing ex forms

Examples

const express = require('express');
const app = express();

// parses request object as json
app.use(express.json()); // json object

// parses request body so that the request looks similar to
app.use(express.urlencoded()); // key=value&key=value&key=value
// USECASE : forms for example

// Serve static files
app.use(express.static('public')); 
// USECASE : we can serve all the static assets in the 'public folder for example'
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

to use urlencoded you have to change app.use(express.urlencoded()) to app.use(express.urlencoded({ extended: true}))

Creating Custom middleware functions

To create custom middleware functions you generally follow a few general steps

  1. define middleware
  2. attach it to a variable
  3. use in processing stacks

Arguments

-req

  • the requested resource
  • res
    • result of middleware function
  • next
    • points to next step after the middleware is complete

Example 1: Custom middleware

const Joi = require('joi');
const express = require('express');
const app = express();

app.use(express.json());

app.use(function(req, res, next) {
  console.log('Logging...');
  next();
});
1
2
3
4
5
6
7
8
9
10
11

It is important to call the 'next();' function if you do not the request will end up hanging.

Example 2: Custom Authenticating Middleware

const Joi = require('joi');
const express = require('express');
const app = express();

app.use(express.json());

app.use(function(req, res, next) {
  console.log('Logging...');
  next();
});

app.use(function(req, res, next) {
  console.log('Authenticating...');
  next();
});
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

Organizing middleware

You may notice our code is quickly becoming cluttered. You can solve this problem by saving each piece of middleware in seperate files.

Bellow we will restructure our middleware into seperate files

  • index.js -- logger.js -- auth.js

After you have moved the function out of index.js it is important to export the function so that index.js may later import it.

Example 3: Moving middleware to multiple folders

  1. Create the new middleware files
|-index.js
|_ /middleware/
          |- logger.js
          |_ auth.js
1
2
3
4
  1. Copy the logging function from index.js and move it into logger.js

Before

// index.js
const Joi = require('joi');
const express = require('express');
const app = express();

app.use(express.json());

app.use(function(req, res, next) {
  console.log('Logging...');
  next();
});

app.use(function(req, res, next) {
  console.log('Authenticating...');
  next();
});
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
// logger.js
1

After

// index.js
const Joi = require('joi');
const express = require('express');
const app = express();

app.use(express.json());

app.use(function(req, res, next) {
  console.log('Authenticating...');
  next();
});
1
2
3
4
5
6
7
8
9
10
11
// logger.js
app.use(function(req, res, next) {
  console.log('Logging...');
  next();
});
1
2
3
4
5

Currently your app wont work, because you still need to change the code within logger.js

Before

  1. Changing logging function -- convert to function -- export
// logger.js (broken)
app.use(function(req, res, next) {
  console.log('Logging...');
  next();
});
1
2
3
4
5
//logger.js (fixed)

// NOTE: we named our function 'log'
function log(req, res, next) {
  console.log('Logging...');
  next();
}

// Export 'log'
module.exports = log;
1
2
3
4
5
6
7
8
9
10
  1. Import 'log' into index.js
// index.js
const Joi = require('joi');
// Note : You can name your constant whatever you want so long as it valid js syntax
const logger = require('./middleware/logger');
const express = require('express');
const app = express();

app.use(express.json());

// Use our 'logger' function
app.use(logger);

app.use(function(req, res, next) {
  console.log('Authenticating...');
  next();
});
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
  1. Repeat for authentication function

Final Result

// logger.js
function log(req, res, next) {
  console.log('Logging...');
  next();
}

module.exports = log;
1
2
3
4
5
6
7
// auth.js
function auth(req, res, next) {
  console.log('Authenticating...');
  next();
}

module.exports = auth;
1
2
3
4
5
6
7
// index.js
const Joi = require('joi');
const logger = require('./middleware/logger');
const authme = require('./middleware/auth');
const express = require('express');
const app = express();

app.use(express.json());

app.use(logger);
app.use(authme);

...
1
2
3
4
5
6
7
8
9
10
11
12
13
14

Some third party middleware

There are many middleware packages available. Here is an example of adding a few of them. For now we'll just take a look at the ones on the express website Express middleware

  1. Find a package you want to install
  2. install the package to your package.json -- i.e npm install package --save -- npm i package
  3. This may change from package to package though generally you would import or require x package in your app now.

Some suggested middleware packages

  • helmet
    • Helps secure your apps by setting various HTTP headers.
  • morgan
    • HTTP request logger.
  • multer
    • Handle multi-part form data.

** other popular **

  • formible

Package install refresher

  • find the package on npmjs
  • require (or import in certain situations)
  • use in app
  1. install 'x' i.e. helmet
npm i helmet
1
  1. require 'x' i.e. helmet
const express = require('express')
const helmet = require('helmet')

const app = express()
//...
1
2
3
4
5
  1. use the package as express middleware i.e. helmet
const express = require('express')
const helmet = require('helmet')

const app = express()

app.use(helmet());

// ...
1
2
3
4
5
6
7
8

More info on helmet helmet - npm

More info on morgan - npm

Changing environment variables

Changing the environment in which your application is running is common practice to accomplish various tasks such as

  • debugging
  • deffining different variables
  • linting
  • building
  • etc..

an example of changing the environment bellow throuch accessing the process.env variable

standard node environment variable process.env.NODE_ENV

if you dont define the environment the default will be 'development'

General environment changing options

  • edit package.json file under scripts
  • in console set environment
    • Windows cmd
      • set
    • Mac Cmd
      • export

Command line option

windows

set NODE_ENV=production
1

mac

export NODE_ENV=production
1

Change back to development

windows

set NODE_ENV=development
1

mac

export NODE_ENV=development
1

Package.json option

{
  "name": "xyz-tuts",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "dev": "NODE_ENV=development",
    "build": "NODE_ENV=production"
  },
  "keywords": [],
  "author": "BClonan",
  "license": "MIT",
  "dependencies": {}
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14

Now by running the following commands you can change your environment variable

Set the evironment to production

npm run build
1

Set the evironment to development

npm run dev
1

there are plenty of ways to 'skin a cat' in this example and consulting the npm or yarn package managers will help you understand the full build cycle

config | npm Documentation yarn run | Yarn

Example 1 : Logging the Environment

const express = require('express');
const helmet = require('helmet');

const app = express();

//Option 1 : log our current environment to the console
console.log(`NODE_ENV: ${process.env.NODE_ENV}`);
// Option 2 : app.get('env') 
//our app object holds information about our current environment
console.log(`app: ${app.get('env')}`);
app.use(helmet());

//...
1
2
3
4
5
6
7
8
9
10
11
12
13

Example 2 : Enabling 'x' only on developement

in the example bellow we only want to run our logger function when the environment is set to developement

const express = require('express');
const helmet = require('helmet');
const morgan = require('morgan')

const app = express();

console.log(`NODE_ENV: ${process.env.NODE_ENV}`);
console.log(`app: ${app.get('env')}`);
app.use(helmet());

//Option 1 : Run a conditional statement
if (app.get('env') === 'development') {
  app.use(morgan('tiny'));
  // debuging console.log
  console.log('Morgan Enabled...')
}
//...
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

Configuration

Storing configuration variables

Now that we can set our environment variables you can expand further by storing your configuration files based on your different environments

there are several node libraries you can use to manage your environment variables a few listed bellow it really just depends on what works best for you and is a personal preference

Example : using the config package

config flow

  • install config
  • create config directory
    • create config.json settings

Bellow we will set different config files

  • default.json
  • development.json
  • production.json

Our app will behave differently depending on the variables we supply within our config files you can set multiple configuration options in the files.

// ./config/default.json
{
  "name": "My super cool app"
}
1
2
3
4
5
// ./config/development.json
{
  "name": "My super cool app - Development",
  "mail": {
    "host": "dev-mail-server"
  }
}
1
2
3
4
5
6
7
// ./config/production.json
{
  "name": "My super cool app - Production",
  "mail": {
    "host": "prod-mail-server"
  }
}
1
2
3
4
5
6
7

Example : import configuration file in the app

Now we can import our configuration file to our app by following our normal procedure.

const config = require('config');
const express = require('express');
const helmet = require('helmet');
const morgan = require('morgan')

const app = express();

console.log(`NODE_ENV: ${process.env.NODE_ENV}`);
console.log(`app: ${app.get('env')}`);
app.use(helmet());

// Configuration

// access the name value we've set in config
console.log('Application Name :' + config.get('name'));

// access the email host we've set in config 
// NOTE: we are using js dot notation to access the value within the object
console.log('Mail Server :' + config.get('mail.host'));


if (app.get('env') === 'development') {
  app.use(morgan('tiny'));
  console.log('Morgan Enabled...')
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

Dont store sensitive values such as passwords or access tokens in environmental variables

Solution: store sensitive data in .env files

In the terminal we will export our example password

Note: in the example bellow we are pretending the name of our app is 'app'

export app_password=1234
1

Alternatively create a new file in your /config/ folder named : custom-environment-variables.json

abbridged app folder structure

|-/config/
|       |-custom-environment-variables.json
|       |- default.json
|       |- development.json
|       |_ production.json
|_index.js
1
2
3
4
5
6

custom-environment-variables.json We only have the mapping of the password in this configuration file.

{
  "mail": {
    "password": "app_password"
  }
}
1
2
3
4
5

Now to use it in index.js

const config = require('config');
const express = require('express');
const helmet = require('helmet');
const morgan = require('morgan')

const app = express();

console.log(`NODE_ENV: ${process.env.NODE_ENV}`);
console.log(`app: ${app.get('env')}`);
app.use(helmet());

// Configuration
console.log('Application Name :' + config.get('name'));
console.log('Mail Server :' + config.get('mail.host'));
// NOTE: you shouldnt be logging this
console.log('Mail Password :' + config.get('mail.password'));

if (app.get('env') === 'development') {
  app.use(morgan('tiny'));
  console.log('Morgan Enabled...')
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

if you find yourself console.loging a ton you can use the debug package, this allows you to console.log depending on your environment debug - npm

Using debug package

flow

  • npm i debug
  • use in app

The debug package returns a namespace for instance we named our namespace 'startup'

require('debug')('app:startup');

index.js

const startupDebugger =require('debug')('app:startup');
//debugger for database modules
const dbDebugger =require('debug')('app:db');
const config = require('config');
const express = require('express');
const helmet = require('helmet');
const morgan = require('morgan')

const app = express();

console.log(`NODE_ENV: ${process.env.NODE_ENV}`);
console.log(`app: ${app.get('env')}`);
app.use(helmet());

// Configuration
startupDebugger('Application Name :' + config.get('name'));
startupDebugger('Mail Server :' + config.get('mail.host'));
// replace console.log with our startupDebugger
startupDebugger('Mail Password :' + config.get('mail.password'));

if (app.get('env') === 'development') {
  app.use(morgan('tiny'));
  // replace console.log with our startupDebugger
  dbDebugger('Morgan Enabled...')
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

Now set a debugging environment either in package.json or through terminal

Example : setup debugging environments

windows

set DEBUG=app:startup
1

mac

export DEBUG=app:startup
1

Turn off or reset debugging

windows

set DEBUG=
1

mac

export DEBUG=
1

Show all debugging

windows

set DEBUG=app:*
1

mac

export DEBUG=app:*
1

package.json

{
  "name": "xyz-tuts",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "dev": "NODE_ENV=development",
    "build": "NODE_ENV=production",
    "debugStartup": "DEBUG=app:startup node index.js",
    "debugDb": "DEBUG=app:db node index.js",
    "debugReset": "DEBUG= node index.js",
    "debugAll": "DEBUG=* node index.js",
  },
  "keywords": [],
  "author": "BClonan",
  "license": "MIT",
  "dependencies": {}
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

Now by running the following commands you can run your debugging configurations

Debug startup

npm run debugStartup
1

Debug db

npm run debugDb
1

Debug Reset

npm run debugReset
1

Debug All

npm run debugAll
1

Debugging

Templating Engines

Sometimes you need to return html markup rather than just json. In this case you can use a templating engine. Each templating engine has different syntax, speeds and setups and is a matter of your applications needs.

Popular templating engines for express

  • Pug
    • Pug is a high performance template engine heavily influenced by Haml and implemented with JavaScript for Node.js and browsers
    • pug - npm
  • mustache
  • EJS

Using Pug

In the following example we'll go over how to use the pug templating engine with express

Templating Engine with express flow

  • install your templating engine
  • set your view engine within express

index.js

const express = require('express');

const app = express();

//Set our view engine
app.set('view engine', 'pug');
//Optional : You can provide a default path that holds all of your templates i.e. ./views
app.set('views', './views'); //default 
// now you would put all of your templates in the ./views directory for example   ./views/index.pug .....   ./views/hello.pug
1
2
3
4
5
6
7
8
9
10

Example: index.pug

Here is a very basic pug template where our api will send both a dynamic title and message as a response object.

To define dynamic variables in pug we use "=" if we wanted our api to render a dynamic message in a html h1 tag we would add h1= message

in the bellow example we are telling pug to render both a dynamic title and dynamic message based off of the response from our api.

./views/index.pug

html
      head
              title= title
      body
            h1= message
1
2
3
4
5

index.js

const debug = require('debug')('app:startup');
const config = require('config');
const morgan = require('morgan');
const helmet = require('helmet');
const Joi = require('joi');
const logger = require('./middleware/logger');
const express = require('express');
const app = express();

app.set('view engine', 'pug');
app.set('views', './views'); // default


app.get('/', (req, res) => {
  // index = name of our view
  // title = dynamically passed title
  // message = dynamically passed h1 data
  res.render('index', { title: 'My Express App', message: 'Hello'});
});

//...
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

Database Integration

There a ton of database integrations available for express and you are also allowed to define your own. You can find a few of them here . Express database integration

Propperly structure express applications

in order to clean up the application and seperate different route endpoints we can move each route's logic into its own file.

To do this we create the files housing each route's logic than use the express.Router() function

routes example

  • require express (mandatory)
  • change app constant (mandatory)
  • change const app to const routes (optional)
  • remove long routes from origional functions
    • i.e. in courses.js we can change
      • /api/courses/:id
      • to
      • /:id
      • This is because we told express in index.js that any route requested from /api/courses use the courses module
  • export route data (mandatory)

so far in index.js we have been defining an app constant

index.js

index.js
const app = express();
1
2

To create a seperate js file to house a certain routes logic need to structure the file differently

// ./router/courses.js

//Mandatory import express
const express = require('express');
// here we use the express.Router() function rather than just express()
// for organization purposes we will also change the const app to router
/*old
const app = express();
*/
const router = express.Router();

/*old 
app.get('/', (req, res) => {
  res.send(courses);
});
*/

// New : app converted to router
router.get('/', (req, res) => {
  res.send(courses);
});


// export the courses module
module.exports = router;
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

Load module in index.js

//index.js
const app = express();
const courses = require('./routes/courses');

// tell the application to use the courses module
// first argument is the api endpoint, second tells express for any api request to /api/courses use the courses module
app.use('/api/courses', courses);
1
2
3
4
5
6
7
8
Last Updated: 8/11/2019, 3:34:12 AM

←