diff --git a/01 Lab 3 Async Code.md b/01 Lab 3 Async Code.md index 54c0a31..aa54d2d 100644 --- a/01 Lab 3 Async Code.md +++ b/01 Lab 3 Async Code.md @@ -54,7 +54,8 @@ Because callbacks are such a fundamental part of NodeJS you need to spend time t Lets improve the currency exchange tool. You will need to refer to the API [documentation](http://fixer.io) as you work through the tasks. Its often helpful to see the complete response body. This allows you to see precisely what data was returned. Add a line to print out the entire JSON object. This needs to be immediately after the body was parsed as a JavaScript object. -``` + +```javascript console.log(JSON.stringify(json, null, 2)) ``` @@ -91,17 +92,17 @@ Lets recap a little about JavaScript functions. Functions are _first-class objec In the previous examples you passed a function as an argument to another function. By passing a function argument we can execute it when we wish, for example after a network operation to retrieve data. In this context, the function is called a **callback function**. -In this topic you will learn how to create your own functions that take a callback function argument and how to store these in CommonJS modules, importing them where needed. +In this topic you will learn how to create your own functions that take a callback function argument and how to store these in CommonJS modules, importing them where needed. Locate the `directions/` directory then open and run the `index.js` script. You will be prompted to enter a start and finish address, the script will return the driving distance between them. Test the exception handling by using both valid and invalid data. 1. The `directions` module is imported. 2. The `getDistance()` function it contains is called: - 1. This takes two string parameters - 2. The third parameter is a callback function + 1. This takes two string parameters + 2. The third parameter is a callback function 3. The callback function takes two arguments: - 1. The first should always be an error object, this will be `null` if no error occurred. - 2. The second argument is the data returned. + 1. The first should always be an error object, this will be `null` if no error occurred. + 2. The second argument is the data returned. 4. Exceptions are handled _inside_ the callback function. 5. The final line in the script executes _before the callback function_ 6. The callback function executes once the data has been retrieved, _without blocking the thread_. @@ -112,11 +113,11 @@ Open the `directions.js` file and study it carefully. 2. The `getDistance()` function is exported. 3. The third argument is the _callback function_ which has two arguments, the error and the data. This is the recommended callback argument pattern sequence. 4. The `getDistance()` function makes an aynchronous call to the `request.get()` function. - - by isolating the API call in its own private function we won't need to duplicate this code when we add more functionality (the DRY principle). + - by isolating the API call in its own private function we won't need to duplicate this code when we add more functionality (the DRY principle). 5. Its third parameter is a _callback function_. 6. In the callback function we check for a non-null first parameter which would indicate an error has occurred. - - If there has been an error we call our callback function and pass an Error object as its first parameter. - - If no error has occurred we return null for the first parameter and the data as the second one. + - If there has been an error we call our callback function and pass an Error object as its first parameter. + - If no error has occurred we return null for the first parameter and the data as the second one. ### 3.1 Test Your Knowledge @@ -190,6 +191,7 @@ A promise represents the result of an asynchronous operation. As such it can be ### 6.1 Creating a Promise Promises are created using the `new` keyword. This function is called immediately with two arguments. The first argument resolves the promise and the second one rejects it. Once the appropriate argument is called the promise state changes. + ```javascript const getData = url => new Promise( (resolve, reject) => { request(url, (err, res, body) => { @@ -198,6 +200,7 @@ const getData = url => new Promise( (resolve, reject) => { }) }) ``` + This example creates a `Promise` that wraps a standard callback used to handle an API call. Notice that there are two possible cases handled here. 1. If the API call throws an error we set the promise state to _rejected_. @@ -207,7 +210,8 @@ As you can see it it simple to wrap any async callbacks in promises but how are ### 6.2 Consuming a Promise -To use promises we need a mechanism that gets triggered as soon as a promise changes state. A promise includes a `then()` method which gets called if the state changes to _fulfilled_ and a `catch()` method that gets called if the state changes to _rejected_. +To use promises we need a mechanism that gets triggered as soon as a promise changes state. A promise includes a `then()` method which gets called if the state changes to _fulfilled_ and a `catch()` method that gets called if the state changes to _rejected_. + ```javascript const aPromise = getData('http://api.fixer.io/latest?base=GBP') @@ -215,21 +219,25 @@ aPromise.then( data => console.log(data)) aPromise.catch( err => console.error(`error: ${err.message}`) ) ``` + In this example we create a _new Promise_ and store it in a variable. It get executed _immediately_. The second line calls its `then()` method which will get executed if the promise state becomes _fulfilled_ (the API call is successful). The parameter will be assigned the value passed when the `resolve()` function is called in the promise, in this case it will contain the JSON data returned by the API call. If the state of the promise changes to _rejected_, the `catch()` method is called. The parameter will be set to the value passed to the `reject()` function inside the promise. In this example it will contain an `Error` object. This code can be written in a more concise way by _chaining_ the promise methods. + ```javascript getData('http://api.fixer.io/latest?base=GBP') .then( data => console.log(data)) .catch( err => console.error(`error: ${err.message}`)) ``` + Because the Promise is executed immediately we don't need to store it in a variable. The `.then()` and `.catch()` methods are simply chained onto the promise. This form is much more compact and allows us to chain multiple promises together to solve more complex tasks. ### 6.3 Chaining Promises The real power of promises comes from their ability to be _chained_. This allows the results from a promise to be passed to another promise. All you need to do is pass another promise to the `next()` method. + ```javascript const getData = url => new Promise( (resolve, reject) => { request(url, (err, res, body) => { @@ -256,11 +264,12 @@ getData('http://api.fixer.io/latest?base=GBP') .catch(err => console.error(`error: ${err.message}`)) .then( () => exit()) ``` + Notice that we pass the `printObject` promise to the `then()` method. The data passed back from the `getData` promise is passed to the `printObject` promise. Because we can chain `then()` and `catch()` methods in any order we can add additional steps after the error has been handled. In the example above we want to exit the script whether or not an error has occurred. -Despite the code in the `printObject` promise being _synchronous_ it is better to wrap this in a promise object to allow the steps to be chained. +Despite the code in the `printObject` promise being _synchronous_ it is better to wrap this in a promise object to allow the steps to be chained. If a promise only takes a single parameter and this matches the data passed back when the previous promise _fulfills_ there is a more concise way to write this. @@ -272,8 +281,8 @@ getData('http://api.fixer.io/latest?base=GBP') .then(exit) ``` - There are some situations where you can't simply pass the output from one promise to the input of the next one. Sometimes you need to store data for use further down the promise chain. This can be achieved by storing the data in the `this` object. + ```javascript getData('http://api.fixer.io/latest?base=GBP') .then( data => this.jsonData = data) @@ -282,6 +291,7 @@ getData('http://api.fixer.io/latest?base=GBP') .catch(err => console.error(`error: ${err.message}`)) .then(exit) ``` + In the example above we store the data returned from the `getData` promise in the `this` object. This is then used when we call the `printObject` promise. ### 6.4 Test Your Knowledge @@ -297,10 +307,12 @@ Open the `promises.js` script and study the code carefully. Notice that it defin ### 6.5 Executing Code Concurrently In the async examples we have seen so far, each async function needs to complete before the next async call is run. The diagram below shows how this looks. + ``` - 1 2 3 + 1 2 3 ───⬤─────⬤─────⬤ ``` + The program flow is. 1. The first async call `getData` is executed. @@ -310,19 +322,21 @@ The program flow is. There are many situations where two steps can run at the _same time_. This would be impossible to build using standard callbacks but this can be written using promises. The first stage is to create an array of promises. Typically this is done by looping through an array of data and using this to return an array of promises. + ```javascript const dataArray = ['USD', 'EUR'] const promiseArray = [] dataArray.forEach( curr => { - promiseArray.push(new Promise( (resolve, reject) => { - const url = `http://api.fixer.io/latest?base=GBP&symbols=${curr}` - request.get(url, (err, res, body) => { - if (err) reject(new Error(`could not get conversion rate for ${curr}`)) - resolve(body) - }) - })) + promiseArray.push(new Promise( (resolve, reject) => { + const url = `http://api.fixer.io/latest?base=GBP&symbols=${curr}` + request.get(url, (err, res, body) => { + if (err) reject(new Error(`could not get conversion rate for ${curr}`)) + resolve(body) + }) + })) }) ``` + In the example above we loop through the `dataArray`, creating a new promise object that we push onto our `promiseArray`. Once we have an array of promises there are two possible scenarios. @@ -339,16 +353,19 @@ Promise.all(itemPromises) .then( results => results.forEach( item => console.log(item))) .catch( err => console.log(`error: ${err.message}`)) ``` + When the `Promise.all()` method fulfills it returns an array of results. In the example above we loop through these and print each to the terminal. #### 6.5.2 Promises Race The alternative is that once one of the promises in the array has fulfilled we want to take its returned value and continue the promise chain. In this scenario we use `Promise.race()`. + ```javascript Promise.race(promiseArray) - .then( result => console.log(result)) - .catch( err => console.log(`error: ${err.message}`)) + .then( result => console.log(result)) + .catch( err => console.log(`error: ${err.message}`)) ``` + As you can see, only a single value is returned by `Promise.race()`. In the example above you won't be able to predict which conversion rate will be returned but you will only get the one. A good application of this would be if you can get your data from multiple APIs but you don't know which ones are working. ## 7 Async Functions @@ -391,6 +408,7 @@ async function main() { } main() ``` + Async functions are declared using the `async` keyword in the function declaration, all errors are handled using the standard `try-catch` block. Because the main block of code needs to be in an _async function_, this has to be explicitly executed at the end of the script. The `getData()` function returns a _promise_. it is called using the `await` keyword, this pauses the execution of the `main()` function until `getData()` is either _fulfilled_ or _rejected_. If it is _fulfilled_, the data returned is stored in the `data` variable and control moves to the next line, if it is _rejected_ code execution jumps to the `catch()` block. @@ -398,6 +416,7 @@ The `getData()` function returns a _promise_. it is called using the `await` key ### 7.2 Simplified Promises Async functions are implicitly wrapped in a `Promise.resolve()` and any uncaught errors are wrapped in a `Promise.reject()`. This means that an _async function_ can be substituted for a _promise_. let's look at a simple example. + ```javascript const printObjectPromise = data => new Promise( (resolve) => { const indent = 2 @@ -414,6 +433,7 @@ const printObjectAsync = async data => { console.log(str) } ``` + both `printObjectPromise` and `printObjectAsync` behave in exactly the same manner. They both return a `Promise.resolve()` and so can be used in either a _promise chain_ or an _async function_. ### 7.3 Test Your Knowledge @@ -437,7 +457,6 @@ In this task you will learn how to extract data from HTML web pages, a technique Despite these issues sometimes this approach is the only way to get the information you need. - Open the `quotes/index.js` file and notice that it imports a custom `quotes` module, the `./` indicates that it is in the current directory. Because the parsing code can get quite complex it is best practice to place this in a custom module. There is only one function in this module, called `getQuotes()` which takes two parameters, the author name plus a callback. The callback follows best practice by passing an error as the first parameter followed by the data. @@ -458,30 +477,34 @@ Use this to expand the DOM until you can highlight the first quote in the list. In the `scraper()` function: 1. The supplied parameters are used to create a **unique url**. It is absolutely vital that: - - each resource have a unique URL. - - the URL for each resource can be calculated based on the supplied parameters. + - each resource have a unique URL. + - the URL for each resource can be calculated based on the supplied parameters. 2. The url is logged to the console so that it can be pasted into a browser to check for validity. 3. The `request` module is used to grab the web page html which is available i the `body` parameter of the callback. 4. The `cheerio` module loads the page DOM into a constant which can be parsed using JQuery syntax. 5. We then check the DOM for particular elements. - - If there is a `

` tag containing the text `No quotations found` we know the search has returned no quotations so an error is returned. - - The number of quotes is extracted from the DOM and stored as a property of the data object. - - An empty `quotes[]` array is added to the `data` object. - - [JQuery.each()](http://api.jquery.com/jquery.each/) is used to loop through each of the tags of interest. - - Each quote is then pushed onto the `quotes[]` array. + - If there is a `

` tag containing the text `No quotations found` we know the search has returned no quotations so an error is returned. + - The number of quotes is extracted from the DOM and stored as a property of the data object. + - An empty `quotes[]` array is added to the `data` object. + - [JQuery.each()](http://api.jquery.com/jquery.each/) is used to loop through each of the tags of interest. + - Each quote is then pushed onto the `quotes[]` array. 6. Once all the data has been extracted from the DOM and added to the `data` object this is passed to the callback function. -### 4.1 Test Your Knowledge +### 8.1 Test Your Knowledge The best way to learn about screen scraping is to have a go. In this task you will be writing a script to search for books based on ISBN numbers and returning useful data. You will be using the Amazon website, start by searching for a specific ISBN such as `1449336361`, this will give you a URL to parse. +``` https://www.amazon.co.uk/JS-Next-ECMAScript-6-Aaron-Frost/dp/1449336361/ref=sr_1_1?ie=UTF8&qid=1475398158&sr=8-1&keywords=1449336361 +``` The next step is to remove the unnecessary parts of the URL until you are left with something you can work with. This is a process of trial and error but you need to be able to construct this URL using only the ISBN number. +``` https://www.amazon.co.uk/dp/1449336361 +``` Have a go at writing a `books` screen scraper and try to return: @@ -514,8 +537,8 @@ Outcomes Waiting for IO to complete is big waste of resources Three solutions: synchronous -processes Apache -threads Node +processes: Apache +threads: Node ## NodeJS Threading Model @@ -524,19 +547,17 @@ JavaScript supports lambda / callbacks Callbacks run in their own threads After callback thread is destroyed - - - - Using Request. Main methods correspond to HTTP verbs: -``` + +```javascript request.get(url, callback) request.put(url, data, callback) request.post(url, data, callback) request.del(url, callback) ``` + Be careful, because callbacks are asynchronous ## Callbacks @@ -570,6 +591,7 @@ The callback is a function that should be run after the operation is complete. While it is processing, control is passed back to the main event loop. Simple GET request with callback: + ```javascript 'use strict' const request = require('request') @@ -595,12 +617,13 @@ Needs to communicate both the data and its structure. Common data exchange formats -- XML (Extensible Markup Language) -- JSON (JavaScript Object Notation) -- YAML (Yet Another Markup Language) -- CSV (Comma-Separated Values) +- XML - (Extensible Markup Language) +- JSON – (JavaScript Object Notation) +- YAML - (Yet Another Markup Language) +- CSV - (Comma-Separated Values) XML Example + ```xml

Coventry University @@ -610,7 +633,9 @@ XML Example CV1 5FB
``` + JSON Example + ```json address { "org": "Coventry University", @@ -620,7 +645,9 @@ address { "postcode": "CV1 5FB", } ``` + YAML Example + ```yaml address: org: "Coventry University" @@ -629,11 +656,14 @@ address: country: "United Kingdom" postcode: "CV1 5FB" ``` + CSV Example + ```csv -"org", "street", "city", "country", "postcode" +"org", "street", "city", "country", "postcode" "Coventry University", "4 Gulson Road", "Coventry", "United Kingdom", "CV1 5FB" ``` + Why do we prefer the JSON format? - Text-based @@ -642,6 +672,7 @@ Why do we prefer the JSON format? - Interoperable with JavaScript Objects Converting to and from JSON + ```javascript const jsObj = { firstname: 'John', @@ -686,16 +717,21 @@ To access resources: Here are some examples: Amazon Book Search URL (javascript) + ``` https://www.amazon.co.uk/s/ref=nb_sb_noss_2?url=search-alias%3Dstripbooks&field-keywords=javascript https://www.amazon.co.uk/s/?url=search-alias%3Dstripbooks&field-keywords=javascript ``` + Guardian Bookstore + ``` http://bookshop.theguardian.com/catalogsearch/result/?q=javascript&order=relevance&dir=desc http://bookshop.theguardian.com/catalogsearch/result/?q=javascript ``` + BBC iPlayer search for history + ``` http://www.bbc.co.uk/iplayer/search?q=history ``` @@ -703,6 +739,7 @@ http://www.bbc.co.uk/iplayer/search?q=history Accessing resources. Amazon Books + ``` https://www.amazon.co.uk/JavaScript-Definitive-Guide-Guides/dp/0596805527/ref=sr_1_2?s=books&ie=UTF8&qid=1476384737&sr=1-2&keywords=javascript https://www.amazon.co.uk/dp/0596805527 diff --git a/02 Lab 2 TDD API.md b/02 Lab 2 TDD API.md index 202605f..31f3786 100644 --- a/02 Lab 2 TDD API.md +++ b/02 Lab 2 TDD API.md @@ -20,7 +20,6 @@ Navigate to the `exercises/02_tdd/web_api/` directory which contains a simple we 3. the `package.json` file contains the project metadata. 4. The `__tests__/` directory contains a test scripts for the `module.js` script. - Open the `package.json` file. Notice that there are certain modules listed as dependencies and others labelled as devDependencies. Use the `npm install` command to install all of these. ## Using the API @@ -117,13 +116,13 @@ The first step is to define the functionality we want in the form of a test. Cle ```javascript test('throws error if body data is missing', () => { - expect.assertions(1) - try { - const request = {body: undefined} - list.extractBodyData(request) - } catch(err) { - expect(err.message).toBe('missing request body') - } + expect.assertions(1) + try { + const request = {body: undefined} + list.extractBodyData(request) + } catch(err) { + expect(err.message).toBe('missing request body') + } }) ``` @@ -181,10 +180,10 @@ The next step in the TDD process is to modify the `extractBodyData()` function u ```javascript module.exports.extractBodyData = request => { - if(request.body === undefined) { - throw new Error('missing request body') - } - return {item: request.body.item, qty: request.body.qty} + if(request.body === undefined) { + throw new Error('missing request body') + } + return {item: request.body.item, qty: request.body.qty} } ``` @@ -196,8 +195,8 @@ The third and final step in TDD is to clean up the code to make it easier to rea ```javascript module.exports.extractBodyData = request => { - if(!request.body) throw new Error('missing request body') - return {item: request.body.item, qty: request.body.qty} + if(!request.body) throw new Error('missing request body') + return {item: request.body.item, qty: request.body.qty} } ``` diff --git a/02 Lab 4 MQTT JavaScript.md b/02 Lab 4 MQTT JavaScript.md index c1074d7..f3eee8b 100644 --- a/02 Lab 4 MQTT JavaScript.md +++ b/02 Lab 4 MQTT JavaScript.md @@ -42,12 +42,12 @@ The first step is to define the functionality we want in the form of a test. Cle ```javascript it('empty string parameter should throw an error', () => { - try { - messages.extractData('') - expect(1).toBe(0) // this line should not be run! - } catch(err) { - expect(err.message).toBe('empty string parameter') - } + try { + messages.extractData('') + expect(1).toBe(0) // this line should not be run! + } catch(err) { + expect(err.message).toBe('empty string parameter') + } }) ``` @@ -156,7 +156,7 @@ extractData: payloadString => { if(err.message === 'Unexpected token h in JSON at position 0') { throw new Error('parameter is not a json string') } - throw new Error(err) // we need to propagate all other errors + throw new Error(err) // we need to propagate all other errors } } ```