01 Intro to NodeJS.md

Introduction to NodeJS

In this chapter we will begin learning about programming in NodeJS. Note that we will only be covering the parts of the language that relate directly to building our APIs, there are lots of books that cover the language in more depth.

NodeJS is based on the JavaScript language which is used extensively in the browser to handle such tasks as form validation and animations. The latest version of the language (known as ECMA6) is not fully supported by the different browsers and, as a result, it is not recommended that you use the more up to date features.

NodeJS is built on Chrome's V8 JavaScript engine which powers the latest version of the Chrome web browser and supports the majority of the latest ECMA6 features. It has been modified to allow it to run on a server.

Because our scripts will run on the server on which we have installed a known version of the ECMA6 runtime, we can guarantee that our scripts will run correctly which means we can focus on teaching the absolute latest programming concepts and features.

1 Benefits of NodeJS

There are a lot of languages that can be used to write server-side script ranging from Perl, one of the earliest CGI Script languages through PHP, one of most popular languages and through to Java and C#. So why is NodeJS becoming so popular in recent years (and why have I chosen it for this book?

The first, and most obvious answer, is that, as a web developer, you are going to have to learn JavaScript anyway because its the only language that will run natively in a web browser so why not use the same language for both client and server?

The second benefit, and one we will return to in chapter 3, is that, unlike other scripting languages which create a new process for each connected user, NodeJS runs a single process shared between all users. Since processes are expensive in computing resources it means that a NodeJS deployment is far more scalable. In chapter 3 we will learn how it handles concurrency through the use of threads.

It is assumed that you can already program in a modern language such as Python or Java and have had exposure to 'old school' JavaScript. This chapter will therefore expose you to some of key new features of the language and their impact on the way you program.

Make sure you have cloned the repository containing the sample code by following the instructions in the previous chapter.

1 Node Packages

NodeJS can be extended through the use of packages and these are installed using the Node Package Manager. This was installed when we installed Node.

$ npm -v
  4.2.10

One of the most valuable features of NodeJS is its package manager which allows you to install additional functionality in the form of packages. There are thousands of these to choose from and, in a later chapter, you will be shown how you can publish your own packages.

In the examples in this chapter we will be using a node package called readline-sync to capture user input. The documentation for all published packages can be found on https://www.npmjs.com so open this page and search for the documentation for readline-sync.

1.1 Installing Packages

Packages can be installed either locally or globally.

• Local packages are installed in a node_modules/ directory within the directory containing the NodeJS scripts. This is the way we install most of the modules. These are only available within that directory.
• Global packages on the other hand are installed system-wide and can be accessed by all the scripts. Normally these need to be installed using root privileges. We will be using global packages in a later chapter.

Open the Terminal and navigate to the nodejs/ directory containing the sample code. If you are using Visual Studio Code you can use File > Open to open the nodejs/ directory then open the Integrated Terminal, accessed from the View menu, this will automatically open in the project root.

Navigate to the 01\ Introduction\ to\ NodeJS/ directory and then install the package.

npm install readline-sync

This will create a new directory called node_modules/ which will contains the scripts from the readline-sync/ package plus any dependencies.

1.2 Listing and Uninstalling Packages

There are two ways to see what packages are currently installed. The quickest is to locate the node_modules/ directory. Alternatively you can use the npm ls subcommand which will print this information to the shell.

$ npm ls
/Users/.../01 Introduction to ECMAScript 6
└── readline-sync@1.4.7

To uninstall a local package you can use the npm uninstall subcommand and pass it the name of the package you want to uninstall, this will uninstall both the named package and any dependencies installed by it.

npm uninstall readline-sync

1.3 Useful Modules

Although there are a lot of modules available through the package manager you will only need a few of these to complete the exercises in this book.

• Request: an HTTP client written in JavaScript, for accessing web resources such as APIs
• Simple-Storage: a wrapper to store data in the filesystem
• Mongoose: a MongoDB object modeling tool.
• FS: a module giving direct access to the host file system, for reading and writing files
• Sentiment: a module that uses the AFINN-165 wordlist and Emoji Sentiment Ranking to perform sentiment analysis on arbitrary blocks of input text.

2 Variables and Scope

If you have ever worked with JavaScript you will have declared variables with the var keyword. This creates a hoisted function-scoped variable which has several issues:

• The variable is function-scoped meaning that it is only visible inside the enclosing function. This means that if you want to hide this from the rest of your code (considered good practice) you need to keep it inside a function which is tricky to implement.
• The variable is also hoisted which means that it can be accessed before the line on which it is declared! Effectively all variable declarations are moved to the top of the function block. This can have some nasty side effects.

Until the release of ECMA6, programmers in JavaScript have had to work with these issues but the release of ECMA6 provided two more options and the use of var is now deprecated and should no longer be used.

2.1 Block-Level Variables

Now we can declare block-level variables using the let keyword. These behave much like variables in other languages such as Python or Java in that they are only visible in the block they are declared in (such as a loop or branch).

2.2 Block-Level Constants

Until ECMA6, you could not declare immutable variables (otherwise known as constants). ECMA introduced the const keyword that can be used to declare block-level constants.

2.3 Worked Example

Now we have learned some theory it's time to see this in practice. Load up the todo.js script and read through it to understand how it works. There are lots of code comments to assist you.

2.3.1 Strict Mode

Notice the first line contains a Directive. This is a feature from ECMA5 telling the JavaScript runtime to run the script in strict mode.

'use strict'

This:

• Prevents the declaration of global variables.
• Exceptions are thrown rather than the script fail silently.
• Prevents duplicate property and parameter names (more on this later).

For the above reasone

Use the terminal to run the script by entering node todo.js. Once running you can use the add command to add new items to the list and the list command to print out the list items. The final command will terminate the application.

2.3.2 Importing a Package or Module

Since we will be using the functionality in the readline-sync package it needs to be imported using the require function.

const readline = require('readline-sync')

This can be used to import an installed package (as we are doing here) but can also be used to import other scripts we have created locally (this is covered in a later chapter).

2.3.3 Conditionals

JavaScript includes the standard set of conditionals (if, if...else and switch) and the syntax is similar to other modern programming languages. The todo.js script makes use of a number of if statements to identify which option the user has entered, for example:

if (input.indexOf('list') === 0) {
  // the user has chosen the 'list' command.
}

Later in this chapter you will be required to implement a switch conditional. These share the same syntax as most modern languages and require a break command to exit and take an optional default clause.

const name = String(readline.question('your name: ')).trim()
switch(name) {
  case 'John':
    console.log('your name is John')
    break
  case 'Jane':
    console.log('your name is Jane')
    break
  default:
    console.log('unknown name')
}

2.3.4 Loops

Javascript also supports a wide number of loop constructs:

• for
• while...do
• do while

In the todo.js script you can see the run-loop has been implemented using a do...while loop.

do {
  // this is the run loop
} while (input !== 'exit')

It also uses a traditional for loop with loop variable using a syntax similar to C++ and Java. It uses the Array length property to iterate through it.

for (let i=0; i< items.length; i++) {
      /* Here we reference the array index. */
			console.log(`${i}. ${items[i]}`)
		}

2.3.5 Strings

In JavaScript, all strings are objects and have a number of useful methods. In the todo.js example there is a line:

const input = String(readline.question('enter command: ')).trim()

1. The String() function takes the expression entered by the user and turns it into a String object
2. we remove any whitespace from the beginning and end of the user-entered string by calling the trim() method which is part of the String object.

Later in the script we use another method indexOf() which returns the index of the first instance of the string parameter. This is used in an if statement to see what is at the start of the string.

if (input.indexOf('add ') === 0) {
  // the string starts with 'add '
}

Notice the use of === rather then the standard == equality operator. JavaScript supports both but they work in slightly different ways:

• === is used for Strict Equality Comparison where the result is only true if both the value and data types match. This is the preferred choice in all situations.
• == is used for Abstract Equality Comparison and works by automatically converting both values to a common type. This can lead to obscure bugs and so should be avoided.

The script uses a second if statement locate the index of the first space in a string to allow it to be split into two. The substring() method takes a parameter and returns the string after the supplied index. We need to use trim() to remove the space from the start.

const space = input.indexOf(' ')
const item = input.substring(space).trim()

It's worth taking a few moments to learn about some of the useful string methods.

2.3.6 String Concatenation

There are two ways to concatenate (join) strings. Prior to ECMA6 the + operator was used to join string literals and string variables.

const name = 'John Doe'
console.log('my name is '+ name)

ECMA6 introduces the concept of template literals which are string literals allowing embedded expressions. The string literal needs to be encased in backticks rather than quotes and the variables enclosed in ${}. The previous example would look line this.

const name = 'John Doe'
console.log(`my name is ${name}`)

By using template literals your strings become much easier to read and so you should get into the habit of always using these.

2.3.7 Arrays

Our todo.js script declares an array near the start to hold the items in our todo list. Notice that the array is declared as immutable using the const keyword.

const items = []

Arrays are objects and have a number of built-in methods. Later in the script we use the built-in push() method to add an element to the end of the array. You should take a moment to look through the list of built-in array methods and familiarise yourself with them.

items.push(item)

2.4 Running a NodeJS Script

Now you have an understanding of the code features used in the script it is time to run it. Unlike client-side JavaScript, you don't run NodeJS scripts in a web browser, instead you need to run them using the Terminal.

To run your script you need to run the node command and pass it the name of your script. You don't need to pass the file extension, so these commands are equivalent:

node todo
node todo.js

When the script is running you will be prompted to enter a command. Try adding three items and listing them all. Finally typing exit to return to the shell prompt:

enter command: add bread
adding "bread"
enter command: add butter
adding "butter"
enter command: add cheese
adding "cheese"
list
0. bread
1. butter
3. cheese
exit

2.4.1 Executing NodeJS Files

There is an alternative way to execute a NodeJS script which works on Linux systems. it works because we have a shebang, otherwise known as a processor directive as the first line of our script. This tells the operating system where to find the command to run the script.

#!/usr/bin/env node

This tells the operating system to use the node command that appears in the environment path variable. You will also need to set the execute flag on the file.

chmod +x todo.js
./todo.js

The last line above tells the OS to run the todo.js file in the current directory.

2.2 Test Your Knowledge

Now you are familiar with the basics of the ECMA6 language its time to put this to the test. Make sure you successfully complete all six tasks before continuing to the next section.

1. locate the input variable declaration (just inside the do loop)

• define it as a block-scoped variable by replacing the var with let, what effect does this have?
• modify the script so that it still works (keep the let variable declaration).
• substitute a constant by substituting const for let, what effect does this have?

2. the array at the top of the script is defined using var. What happens if you make this immutable (use const)?
3. Items are added to the array using its push() method.

• substute the unshift method. How does this change the script?

4. modify the code to prevent duplicate items being added. You will need to use the Array.indexOf() method.
5. create a remove option so an item such as cheese can be removed using the syntax remove cheese. You may need to use the Array.splice() method.
6. The current version is case sensitive. Modify the code so that items are converted to lowercase before being added or searched for. You will need to use the String.toLowerCase() method.

3 Exception Handling

When JavaScript executes code errors and exceptions may occur. These may be due to incorrect user input or a broken network connection for example. JavaScript includes a rich set of tools for handling these, based on the Error object.

1. Errors are serious problems that normally mean the application will terminate
2. Exceptions on the other hand are problems that can be handled by the program logic and thus prevent the application from terminating. In this task we will be focussing on exception handling.

Open the contact.js script and study it carefully, as before, the code includes detailed comments to explain how it works.

1. All code that could throw an exception must be in a try{} block.
2. If an exception is thrown the execution moves to the catch{} block.

• the error object thrown will be passed as the parameter.
• the error object contains three properties: the name of the error, the message passed and the stack trace.
• the stack trace is a list of the method calls that the application was in the middle of when an Exception was thrown and can help identify some of the more insidious errors. You should learn to read and understand what information it contains.

3.1 Test Your Knowledge

Implement the validateEmail() function and thoroughly test it, you should avoid using regular expressions at this stage:

1. Check that the string is at least 5 character long
2. Check that there is a @ character and that it is not at the start of the string (HINT: use the indexOf String prototype method.
3. Check that there is a period (.) character after the @ character but before the end of the string.

4 Passing Parameters on the Command Line

You have probably seen some commands that take startup parameters. Each time you run a script, everything you type at the shell is made available through the process object which contains an argv array.

This is useful since it means we can pass extra data to a script when we invoke it from the shell. We will be using this feature to pass in a sentence and measure its sentiment (positive or negative).

Open the script sentiment.js and make sure you understand how it works.

Try running the script without any extra parameters.

$ node sentiment
  [ '/Users/.../bin/node',
    '/Users/.../sentiment.js' ]
  missing parameters

Notice that it prints out an array which contains two indexes corresponding to the node command and the script name (sentiment.js). There is also a message that there are missing parameters. Where is this message coming from?

const minParam = 3
console.log(process.argv)
if (process.argv.length < minParam) {
  throw new Error('missing parameters')
}

In the code above you can see that the array printed to the shell is the contents of process.argv. The error was because the script expected an array of at least 3 indexes.

Now let's run the command and pass it a sentence.

node sentiment happy to meet you
$ node sentiment happy to meet you again
  [ '/Users/.../bin/node',
    '/Users/.../sentiment',
    'happy',
    'to',
    'meet',
    'you',
    'again' ]
  happy to meet you again
  { score: 3,
    comparative: 0.6,
    tokens: [ 'happy', 'to', 'meet', 'you', 'again' ],
    words: [ 'happy' ],
    positive: [ 'happy' ],
    negative: []
  }

So what's happened here? Well now the process.argv array contains the additional words we typed. because there were more than 2 indexes the error is not thrown.

Now we have some words to process we need to combine them into a single string.

const words = process.argv.slice(minParam-1).join(' ')
console.log(words)

The built-in Array.slice() method returns the section of array between the specified index and the end of the array. The join() method converts an array into a string using the parameter as separator. It is standard practice when programming in JavaScript to use method chaining whereby several methods are called on the same data.

The final step is to pass this string to the sentiment tool which returns the sentiment of the sentence.