Code Quality

In this worksheet you will be applying a range of techniques to improve the quality of your code. Specifically we will cover:

How to split your code into separate files (modules).
How to enforce good coding practices using a linter.
Generating documentation for your code using special comment blocks.
Improving the readability of your async code blocks using promises and async functions.

Before you start you need to pull any upstream changes. Detailed instructions can be found in the Setup lab.

1 Modularisation

The first step you will need to do is to split your code up to make it easier to understand. Take a look at the end of the index.js routes file.

Start by updating your routes file by copying over the modules/accounts.js file from the lab materials and making sure you import it into your index.js file by adding the following statement just below where all the other modules are imported:

const accounts = require('modules/accounts')

This loads the module into a constant called accounts.

Now locate the second router.post('/login') route (this is currently commented out). Comment out the one you have been using and uncomment this new shorter version. If you run your server and test it you will find the functionality is identical. Its time to understand how this new version works:

We start by storing the request.body data (the HTTP POST request body data) in an immutable variable called body.
Next we call the checkCredentials() function that we imported from the accounts.js module passing the username and password as parameters.
If this function does not throw an exception it means the credentials are valid so we set the cookie and redirect to the home page.
If either the username or password are invalid, the checkCredentials() function will throw an exception which will be handled by the catch() block. The error message will explain what us wrong so we pass this back to the login page.

Now we need to look at the accounts.js module. This implements some very important concepts that you will need to understand and apply to your assignment.

The accounts module contains two types of function:
1. The function declarations such as runSQL() have a private scope and are not visible outside the module.
2. Any function expressions stored as keys in the module.exports object such as module.exports.checkCredentials are available to any code that imports this module.
All the code in a function is wrapped in a try-catch block to handle any exceptions.
The catch block simple propagates the Error object to the calling code.
Any non-valid code flow is handled by throwing an Error object which forces the code flow to jump directly to the catch block.
This means that if the program flow reaches the end of the try block everything was successful and data can be returned.

1.1 Test Your Understanding

To check you understand how to use modules you are now expected to move more of the functionality from the index.js file into this separate module. To help you with this you will find stub functions that only require the functionality to be added! You will be modifying the functions below the /* --- STUB FUNCTIONS --- */ line.

Implement the checkNoDuplicateUsername(username) function to comply with the JSDoc comments, it should throw an exception if a duplicate user is found or true if not.
Implement the saveImage() function. This should check that the image is of the correct type then save it to the avatars/ directory.
Now implement the addUser() function. This should make use of the functions you created in the first two tasks. It should check for duplicates before saving the image then it should encrypting the password and then saving the new record.
The final step is to comment out the router.post('register') route in index.js then create a replacement that makes use of the functions you added to the accounts.js module.

Now much of the business logic has been moved to the separate module, are there any module imports in index.js that are no longer needed? Locate these and delete.

2 Linting

When using a language as flexible as JavaScript which contains so many legal (but terrible) features, it is important to use a linter. This will check your code against a set of rules. These ensure:

You are not using what are considered bad language features.
You are implementing optional syntax (such as indentation and semicolons) in a consistent manner.
You are writing code that is easy to maintain.

If you look over both your index.js and accounts.js files you should be feeling pretty comfortable that you are already writing clean, consistent and maintainable code, lets see how good your code really is!

You should start by ensuring you have installed eslint which is considered the industry standard and that you have a copy of the approved configuration file .eslintrc.json in the root directory of your project. You can find this in the TEACHING-MATERIALS repository but make sure you take a copy of the latest version from the master repository!

Try running the linter on your index.js routes file:

$ node_modules/.bin/eslint index.js

You will see a list of issues that the linter has flagged in your code. Notice that some of these are flagged as errors (serious) and some as warnings (recommendations). Each message includes:

The line and comumn number where the error was found.
A description of the error.
The rule that is being broken.

The latter can be used to quickly look up the rules in the comprehensive documentation.

Instead of running separate checks on every file, we can specify the directory we want to check and it will automatically scan all the subdirectories. For example to scan all the files in the modules/ directory we could run:

$ node_modules/.bin/eslint modules/

2.1 Test Your Understanding

How could you run the linter to scan all the files in your project (HINT: you need to start scanning in the current directory)?
Now you should locate and fix all the errors and warnings in your code.
If you are using VS Code, install eslint globally and then install the eslint extension. After restarting your editor you should see any errors and warnings flagged in the editor.

3 Documentation

In this third and last topic we will be using the JSDoc tool to build a detailed code documentation website by extracting special comments inserted into our source code.

The default set of documentation tools provided in JSDoc are not suitable for documenting Koa routes and so we will be using a plugin called jsdoc-route-plugin. This should have been installed by the package manifest however you should check that you are using the current version of the package.json file and update if needed, rerunning the npm install command to ensure all packages are installed. You should also check that you have the latest version of the jsdoc.conf configuration file.

Now everything is installed we can run the jsdoc tool to generate our documentation.

$ node_modules/.bin/jsdoc

If you run this command you should see a new directory called docs/ which will contain a jsdoc/ directory. Inside this you will see some website files, opening the index.html file in your browser you should see the documentation pages for your website!

3.1 Test Your Understanding

You will probably have noticed that only a couple of the functions include complete JSDoc comments and so the documentation website is incomplete. Your task is to use the existing comments for guidance and complete the task of documenting your code. You will find the JSDoc and jsdoc-route-plugin documentation helpful.

4 Improved Async Code

Since NodeJS has a single thread that handles all incoming requests it is vital that we push long-running tasks into their own threads, typically through the use of callback functions. In this section of the lab you will learn about the limitations of callbacks and explore more powerful ways to handle multi-threading.

4.1 Nested Callbacks

Because the code to be run after a callback is run needs to be inside the callback code it is very challenging to build a script that contains several long-running tasks you get into a situation where you nest callbacks inside callbacks (inside callbacks) which makes the code very difficult to write, debug and read and means its very difficult to split into separate functions, a situation commonly known as Callback Hell.

Open the file nestedCallbacks.js which asks for a base currency code then prints out all the exchange rates against other currencies. Notice that there are four functions defined, three of which include a callback. Our script is designed to capture user input using stdin (needing a callback), identify whether a currency code is valid (requiring a second callback) and then getting the currency conversion rates for the specified currency (requiring a third callback).

Notice that the checkValidCurrencyCode() function is called by the callback for the getInput() function and the getData() function is called by the callback for the checkValidCurrencyCode() function.
Each callback takes two parameters as normal. The first contains the error (if any) and this needs to be handled in each callback.
The data from the first callback is needed when calling the third function so needs to be stored in an immutable variable (constant).
The fourth, and final, function does not have a callback.

Callbacks are the simplest possible mechanism for asynchronous code in JavaScript. Unfortunately, raw callbacks sacrifice the control flow, exception handling, and function semantics familiar from synchronous code.

4.1 Test Your Knowledge

The callbacks are already nested 3 deep. To test your knowledge of deeply nested callbacks you are going to create a script that has 6 levels of nested callbacks!

modify the script to ask for the currency to convert to and display only the one conversion rate.
instead of printing the exchange rate, ask for the amount to be converted and them return the equivalent in the chosen currency
use the OpenExchangeRates API to display the full name of the chosen currency.

Even though the script is still simple you are probably already getting in a tangle! Imagine a more complex script with conditions, it would quickly get out of hand and become practically impossible to debug.

Thankfully there are a number of advance features in NodeJS that are designed to flatten out these callbacks and to treat asynchronous code in a more synchronous manner. These care called Generators, Promises and Async Functions and are described below. Even though you don't technically need to know these, its worth learning them to keep your code manageable.

6 Promises

A promise is an object that proxies for the return value thrown by a function that has to do some asynchronous processing (Kris Kowal).

A promise represents the result of an asynchronous operation. As such it can be in one of three possible states:

pending - the initial state of a promise.
fulfilled - the asynchronous operation was successful.
rejected - the asynchronous operation failed.

6.1 Creating a Promise

Promises are created using the new keyword. This function is called immediately with two arguments. The first argument resolves the promise and the second one rejects it. Once the appropriate argument is called the promise state changes.

const getData = url => new Promise( (resolve, reject) => {
  request(url, (err, res, body) => {
    if (err) reject(new Error('invalid API call'))
    resolve(body)
  })
})

This example creates a Promise that wraps a standard callback used to handle an API call. Notice that there are two possible cases handled here.

If the API call throws an error we set the promise state to rejected.
If the API call succeeds we set the promise state to fulfilled.

As you can see it it simple to wrap any async callbacks in promises but how are these called?

6.2 Consuming a Promise

To use promises we need a mechanism that gets triggered as soon as a promise changes state. A promise includes a then() method which gets called if the state changes to fulfilled and a catch() method that gets called if the state changes to rejected.

const aPromise = getData('http://api.fixer.io/latest?base=GBP')

aPromise.then( data => console.log(data))

aPromise.catch( err => console.error(`error: ${err.message}`) )

In this example we create a new Promise and store it in a variable. It get executed immediately. The second line calls its then() method which will get executed if the promise state becomes fulfilled (the API call is successful). The parameter will be assigned the value passed when the resolve() function is called in the promise, in this case it will contain the JSON data returned by the API call.

If the state of the promise changes to rejected, the catch() method is called. The parameter will be set to the value passed to the reject() function inside the promise. In this example it will contain an Error object.

This code can be written in a more concise way by chaining the promise methods.

getData('http://api.fixer.io/latest?base=GBP')
  .then( data => console.log(data))
  .catch( err => console.error(`error: ${err.message}`))

Because the Promise is executed immediately we don't need to store it in a variable. The .then() and .catch() methods are simply chained onto the promise. This form is much more compact and allows us to chain multiple promises together to solve more complex tasks.

6.3 Chaining Promises

The real power of promises comes from their ability to be chained. This allows the results from a promise to be passed to another promise. All you need to do is pass another promise to the next() method.

const getData = url => new Promise( (resolve, reject) => {
  request(url, (err, res, body) => {
    if (err) reject(new Error('invalid API call'))
    resolve(body)
  })
})

const printObject = data => new Promise( resolve => {
  const indent = 2
  data = JSON.parse(data)
  const str = JSON.stringify(data, null, indent)
  console.log(str)
  resolve()
})

const exit = () => new Promise( () => {
  process.exit()
})

getData('http://api.fixer.io/latest?base=GBP')
  .then( data => printObject(data))
  .then( () => exit())
  .catch(err => console.error(`error: ${err.message}`))
  .then( () => exit())

Notice that we pass the printObject promise to the then() method. The data passed back from the getData promise is passed to the printObject promise.

Because we can chain then() and catch() methods in any order we can add additional steps after the error has been handled. In the example above we want to exit the script whether or not an error has occurred.

Despite the code in the printObject promise being synchronous it is better to wrap this in a promise object to allow the steps to be chained.

If a promise only takes a single parameter and this matches the data passed back when the previous promise fulfills there is a more concise way to write this.

getData('http://api.fixer.io/latest?base=GBP')
  .then(printObject)
  .then(exit)
  .catch(err => console.error(`error: ${err.message}`))
  .then(exit)

There are some situations where you can't simply pass the output from one promise to the input of the next one. Sometimes you need to store data for use further down the promise chain. This can be achieved by storing the data in the this object.

getData('http://api.fixer.io/latest?base=GBP')
  .then( data => this.jsonData = data)
  .then( () => printObject(this.jsonData))
  .then(exit)
  .catch(err => console.error(`error: ${err.message}`))
  .then(exit)

In the example above we store the data returned from the getData promise in the this object. This is then used when we call the printObject promise.

6.4 Test Your Knowledge

Run the promises.js script, its functionality should be familiar to the currency.js script you worked with in chapter 3.

Open the promises.js script and study the code carefully. Notice that it defines 5 promises and chains them together. You are going to extend the functionality by defining some additional promises and adding them to the promise chain.

modify the script to ask for the currency to convert to and display only the one conversion rate.
instead of printing the exchange rate, ask for the amount to be converted and them return the equivalent in the chosen currency
use the OpenExchangeRates API to display the full name of the chosen currency

6.5 Executing Code Concurrently

In the async examples we have seen so far, each async function needs to complete before the next async call is run. The diagram below shows how this looks.

         1      2      3        
      ───⬤─────⬤─────⬤

The program flow is.

The first async call getData is executed.
Once this has completed, printObject is executed.
Only when this has completed will the exit step execute.

There are many situations where two steps can run at the same time. This would be impossible to build using standard callbacks but this can be written using promises.

The first stage is to create an array of promises. Typically this is done by looping through an array of data and using this to return an array of promises.

const dataArray = ['USD', 'EUR']
const promiseArray = []
dataArray.forEach( curr => {
	promiseArray.push(new Promise( (resolve, reject) => {
		const url = `http://api.fixer.io/latest?base=GBP&symbols=${curr}`
		request.get(url, (err, res, body) => {
			if (err) reject(new Error(`could not get conversion rate for ${curr}`))
			resolve(body)
		})
	}))
})

In the example above we loop through the dataArray, creating a new promise object that we push onto our promiseArray.

Once we have an array of promises there are two possible scenarios.

We want all the promises in the array to be fulfilled before continuing the promise chain.
We want one of the promises to be fulfilled but we don't care which one.

6.5.1 Promises All

In the first scenario we want all the promises to be fulfilled before continuing and for this we use the Promises.all() method.

Promise.all(itemPromises)
  .then( results => results.forEach( item => console.log(item)))
  .catch( err => console.log(`error: ${err.message}`))

When the Promise.all() method fulfills it returns an array of results. In the example above we loop through these and print each to the terminal.

6.5.2 Promises Race

The alternative is that once one of the promises in the array has fulfilled we want to take its returned value and continue the promise chain. In this scenario we use Promise.race().

Promise.race(promiseArray)
	.then( result => console.log(result))
	.catch( err => console.log(`error: ${err.message}`))

As you can see, only a single value is returned by Promise.race(). In the example above you won't be able to predict which conversion rate will be returned but you will only get the one. A good application of this would be if you can get your data from multiple APIs but you don't know which ones are working.

7 Async Functions

In the previous sections we have covered the use of generators which allow the use of synchronous-style code to handle async code but the syntax is far from intuitive.

We then looked at the use of promises which allows you to wrap async code as a series of promises which can be chained together and implements exception handling. The price we pay for this is non-intuitive syntax which can become over complex. Async functions combine the benefits of promises with a clean synchronous-style syntax, avoiding the complex syntax used in promise chains. They are designed to simplify the behaviour of using promises in a synchronous manner.

Whenever we execute a function there is some implicit behaviour we expect. One behaviour is that, once invoked, a function will run until it gets to the end. Async functions break this behaviour, they can pause at any point and resume at a later point on the script. This enables us to write asynchronous code that looks and feels synchronous, it can even use standard try-catch execption handling.

We can chain promises together in a cleaner way with full exception handling.
We can substitute a promise with an async function without needing to change any other part of the script.

7.1 Simplifying Promise Chains

Here is a simple example.

const getData = url => new Promise( (resolve, reject) => {
  request(url, (err, res, body) => {
    if (err) reject(new Error('invalid API call'))
    resolve(body)
  })
})

const printObject = data => new Promise( resolve => {
  console.log(JSON.stringify(JSON.parse(data), null, 2))
  resolve()
})

async function main() {
  try {
    const data = await getData('http://api.fixer.io/latest?base=GBP')
    await printObject(data)
    process.exit()
  } catch (err) {
    console.log(`error: ${err.message}`)
    process.exit()
  }
}
main()

Async functions are declared using the async keyword in the function declaration, all errors are handled using the standard try-catch block. Because the main block of code needs to be in an async function, this has to be explicitly executed at the end of the script.

The getData() function returns a promise. it is called using the await keyword, this pauses the execution of the main() function until getData() is either fulfilled or rejected. If it is fulfilled, the data returned is stored in the data variable and control moves to the next line, if it is rejected code execution jumps to the catch() block.

7.2 Simplified Promises

Async functions are implicitly wrapped in a Promise.resolve() and any uncaught errors are wrapped in a Promise.reject(). This means that an async function can be substituted for a promise. let's look at a simple example.

const printObjectPromise = data => new Promise( (resolve) => {
  const indent = 2
  data = JSON.parse(data)
  const str = JSON.stringify(data, null, indent)
  console.log(str)
  resolve()
})

const printObjectAsync = async data => {
  const indent = 2
  data = JSON.parse(data)
  const str = JSON.stringify(data, null, indent)
  console.log(str)
}

both printObjectPromise and printObjectAsync behave in exactly the same manner. They both return a Promise.resolve() and so can be used in either a promise chain or an async function.

7.3 Test Your Knowledge

Run the asyncFunctions.js script. Note that it works in the same way as the previous ones. Open the script and study it carefully.

modify the script to ask for the currency to convert to and display only the one conversion rate.
instead of printing the exchange rate, ask for the amount to be converted and them return the equivalent in the chosen currency
use the OpenExchangeRates API to display the full name of the chosen currency
rewrite the printObject promise as an async function.
rewrite another promise as an async function.

foundation/06 Code Quality.md