Merged files

00 Local Setup.md

@@ -0,0 +1,14 @@
+
+# Local Setup
+
+If you are planning on using your own laptop you will need to install some key pieces of software. Obviously its impossible to cover the installation process in detail for every operating system but there are plenty of guides online. You should install:
+
+1. Visual Studio Code (Not Visual Studio!)
+2. The latest version of NodeJS. For Linux follow the instructions to install nvm, for other platforms download and install the latest version from the [official site](https://nodejs.org/en/), make sure you install the **latest** and not the LTS version.
+3. You also need to install Git if it is not already installed. Mac and Windows users can download the installer from the [official website](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git).
+
+Once everything is installed you can clone the module repository using the same command you used in the Goorm IDE lab (make sure you clone it to a sensible location).
+
+Once the repository is cloned you should open VS Code and use the File > Open Folder option to select and open the folder containing the repository (On some systems you choose File > Open).
+
+Once the web server is up and running you access the page by pointing the browser to `localhost:8080`.

01 Lab 1 Templates and Forms.md

@@ -9,11 +9,17 @@ We will start by producing dynamic web pages that combine a static layout with d
 
 There are a number of _templating view engines_ that are compatible with Express however in this worksheet we will be using one of the more popular ones, called [Handlebars](https://www.npmjs.com/package/handlebars). This needs to be imported into your script and set the default _layout page_.
 
-### 1.1 Basic Templating
+For this part of the lab you will be working with the website found in the `01_nodejs/01_templates_forms/simple_templating/` directory, the same one you used in the previous setup exercise.
+
+1. As you have already seen, this displays a simple message together with the picture of a clock.
+2. Add `/date` to the end of the URL, notice that this now displays a page showing the current date.
+3. Finally replace the `/date` with `/food`, again notice you are viewing a different page, this time listing some different foods.
 
-Locate the files in the `01_nodejs/01_templates_forms/simple_templating/` directory, install the dependencies and start the server.
+By changing the URL you tell the server to send you different web pages. This is how the World-Wide-Web works. The bit of text after the server name and port is known as the **route**. Open the `index.js` file and study the script, can you see how it is serving up the three different pages?
+
+### 1.1 Basic Templating
 
-Access the base route `/`, notice that you are seeing a basic html page. Open the script:
+The URL without any additions is called the **base** route, and is shown with a `/`. Locate the `app.get()` function that handles this route.
 
 1. We start by importing the Handlebars package and create a default layout called main. This defines the `main.handlebars` page as the one to use as the default website layout.
 	1. Open the `views/layouts/main.handlebars` file.
@@ -67,12 +73,10 @@ To understand what happens to this data we need to understand the _template_. Lo
 
 #### 1.2.1 Test Your Understanding
 
-1. Use suitable properties of the [`Date` object](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date) to display the date in a variety of different formats in a series of paragraph elements:
-    1. dd/mm/yyyy
+1. Use suitable properties of the [`Date` object](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date) to add paragraph elements to display the following:
+    1. The current time in the 24 hour time format: HH:MM:SS
     2. a Unix timestamp (number of seconds since 1st Jan 1970)
-2. Add a table to display some information about the client computer (using the `req.connection` object).
-3. Extend the table to display the header information (using the `req.headers` object).
-4. Insert a picture of a calendar.
+2. Insert a picture of a calendar.
 
 ### 1.3 Repeating Data
 
@@ -102,31 +106,6 @@ This allows the handlebars template view engine to handle repeated data.
 3. Add a table header to display column headings.
 4. Without adding any more html, colour every other row of the table in light grey.
 
-### 1.4 Putting it Together
-
-You have covered a lot of topics over the first few weeks of the module. Before you continue, complete the challenges listed below. These will help you revise all the content you have covered.
-
-The `repeating_data/` directory contains a 2-page template-driven dynamic website based on the data you used in the **Databases** lab. Install the dependencies, start the server and access the base route `/` and the `/details/1` route then study the script `index.js`.
-
-#### 1.4.1 Test Your Understanding
-
-Now try to complete the following challenges:
-
-1. Add a route called `/about` that displays information about the fictional bookshop.
-2. Add a footer that appears on all pages using the correct html5 element.
-3. Convert the list of books into a table.
-4. Add a column to display the ISBN number
-5. Add a hyperlink to the book titles to jump to the correct book details page.
-6. Add a column that displays links to take you to the [Amazon product page](https://www.amazon.co.uk/gp/search/ref=sr_adv_b/?search-alias=stripbooks&field-isbn=9781491943120) (hint: use the ISBN number and study this link carefully!).
-7. Create and link a stylesheet to improve the page appearance:
-    1. Style the header.
-    2. Style the footer.
-    3. Make the table easier to read.
-8. Add all the database fields to the product details page.
-9. Modify the stylesheet to improve the appearance.
-
-If you can complete them all successfully, well done! You are ready to move onto the next section.
-
 ## 2 Forms
 
 In this final part of the worksheet you will be building forms that can send data to a web server. Locate the `simple_forms/` directory, install the necessary modules and start the server.
@@ -141,14 +120,20 @@ Make sure the web server is running and access the `/postform` route and open th
 2. Click on the the **Submit** button.
 3. Examine the URL carefully:
     1. Notice that it points to the base route `/` with no additional data in the URL.
-4. Open the _Chrome developer tools_ and look at the _http request_:
+4. Open the _Chrome developer tools_ and look at the _http request_ (select the **Network** tab and click on the URL in the **Name** column):
     1. Notice that the request uses the `POST` method. This corresponds to the `method` attribute in the `<form>` element.
     2. The request header includes a `Content-Type` header which contains the value `application/x-www-form-urlencoded`.
 5. There is a _request body_ which contains the form data:
     1. This uses the `application/x-www-form-urlencoded` encoding.
     2. Notice that it contains 2 query parameters in a querystring.
     3. The names of the query parameters correspond to the values in the `name` attributes in the `<input>` elements.
 
+#### 2.1.1 Test Your Understanding
+
+1. Currently all the form data is processed in a loop and added to the table:
+    1. Modify the `home.handlebars` template to display the full name as a second level heading.
+2. Add a new field to store an email address
+
 ### 2.2 Submitting Data Using GET
 
 Make sure the web server us running and access the `/getform` route and open the corresponding html file.
@@ -186,10 +171,14 @@ locate and open the `complex_forms/` directory. Install the required modules and
 
 #### 2.4.1 Test Your Understanding
 
-Complete the following tasks. After each, complete and submit the form to ensure all data is avaialable:
+Complete the following tasks. After each, complete and submit the form to ensure all data is available:
 
 1. Add a box for the user's email address in the personal detail section.
 2. Create a new fieldset for the user's address and add input boxes to capture this. Make sure each box has a unique, descriptive `name` attribute.
 3. You will find a json file containing a longer list of courses. Replace the static list with the data contained in this file (you will need to pass some repeating data to the template).
+    1. To load the data from a file you need to use the [`fs.readfile()`](https://nodejs.org/api/fs.html#fs_fs_readfile_path_options_callback) function.
+    2. You are loading a JSON-formatted string from the file. To turn this into an object you need to use [`JSON.parse()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/parse).
+    3. You then need to extract the array from the object (try using [`console.log()`](https://developer.mozilla.org/en-US/docs/Web/API/Console) to visualise the data.
+    4. You then need to pass this array to the handlebars template and insert a helper in this template to insert the courses into the dropdown list.
 4. Display the form data on the page rather than just the JSON string (you will need to pass each item through to the template and insert into individual placeholders).
 5. Add a link to the data page to return to the form.

01 Lab 2 Dynamic Websites.md

@@ -17,7 +17,7 @@ Start by installing the tools on your computer:
     2. `$ sudo apt-get install sqlite3 libsqlite3-dev`
 3. It a bit of a pain to install on Windows 10 but there is a good [YouTube video](https://youtu.be/zOJWL3oXDO8) that covers the process.
 
-If you use the terminal/command prompt to navigate to the `exercises/13_website/bookshop/` directory you will find a prebuilt database called `bookshop.db`. To open a database you use the `sqlite3 bookshop.db` command. Note: if the file you specify does not exist, a new database will be created. Open the `bookshop.db` database.
+If you use the terminal/command prompt to navigate to the `exercises/01_nodejs/02_dynamic_website/bookshop/` directory you will find a prebuilt database called `bookshop.db`. To open a database you use the `sqlite3 bookshop.db` command. Note: if the file you specify does not exist, a new database will be created. Open the `bookshop.db` database.
 
 Notice that the prompt changes to `sqlite>`, this means you are interacting with the `sqlite3` program. There are a number of special [commands](https://www.sqlite.org/cli.html) that include standard SQL statements as well as special sqlite commands (these start with a period/dot and are sometimes called _dotcommands_. Try the following commands (note: the up arrow recalls previous commands):
 
@@ -96,7 +96,7 @@ Using the data in the `books.csv` file, add the authors to each of your book rec
 
 ## 2 Building a Dynamic Web Page
 
-Now you have a working database containing some useful data you need to display this in a web page. Start by locating and running the `index.js` script in the `bookshop/` directory (you will need to install the dependencies first). Notice that there is a message in the terminal to let you know the script has connected to the database. Now view the web page, notice it displays the book titles from your database (including the ones you added).
+Now you have a working database containing some useful data you need to display this in a web page. Start by locating and running the `index.js` script in the same `bookshop/` directory (you will need to install the dependencies first). Notice that there is a message in the terminal to let you know the script has connected to the database. Now view the web page, notice it displays the book titles from your database (including the ones you added).
 
 This is known as a dynamic (or data-driven) website. By the end of this section you will have a clear understanding of how this can be created using NodeJS. Lets look at how this works. Open the `index.js` script.
 
@@ -115,7 +115,7 @@ This is known as a dynamic (or data-driven) website. By the end of this section
 	2. We loop through the array and for each index we take the data, wrap it in html tags and append it to the variable.
 	3. Finally we append the closing list element.
 8. We print this to the terminal on line 37 to check the html is correct.
-9. Finally we pass this string to the `index.html` template and render it in the browser.
+9. Finally we pass this string to the `home.handlebars` template and render it in the browser.
 
 ### 2.1 Test Your Understanding
 
@@ -127,10 +127,10 @@ This is known as a dynamic (or data-driven) website. By the end of this section
 
 ## 3 Adding Search Functionality
 
-There are not many books in our database so displaying them all is not a problem however once we increase the number of books significantly we need to find a way to filter them. In this section we will be implementing a simple search functionality. Start by opening the `index.html` template file.
+There are not many books in our database so displaying them all is not a problem however once we increase the number of books significantly we need to find a way to filter them. In this section we will be implementing a simple search functionality.
 
 1. If you look directly under the first route you will see a second route commented out (lines 42-63). Comment out the route you have been using in section 2 and uncomment this one. Restart the script.
-2. Notice the route uses a different template (`newindex.html`) which contains an html form. This will display a search box and button.
+2. Notice the route uses a different template (`newindex.handlebars`) which contains an html form. This will display a search box and button.
 3. Type in your search term `sqlite` and click on the search button, this reloads the page. You will see the search term `sqlite` remains in the text box and the page displays the books that match your search term.
 	1. Click in the _address bar_ in your browser and look carefully at the URL.
 	2. It ends with the string `?q=hello`.
@@ -170,7 +170,7 @@ Make sure the script is running and try accessing the `/details/1` route. This d
 2. Now we call the `all()` function that is part of the database (`db`) object. This returns a JavaScript object containing the first matching record.
 	1. The callback runs after the query completes or an error occurs.
 	2. If no error occurs, the entire object is displayed in the terminal and passed to the html template.
-3. If you open the `details.html` template you can see the placeholder names match the database fields.
+3. If you open the `details.handlebars` template you can see the placeholder names match the database fields.
 
 ### 4.1 Linking the Pages
 
@@ -203,8 +203,8 @@ To create a working form you need to have two routes:
 
 There is already a working form. Access the `/form` route in your browser, this will display a simple html form where you can enter a book title, isbn and description. Try adding a book using this (there are some extra examples in the `books.csv` file). Notice that when you click on the add button you end up back at the list of books and the new one has been added to the end. Lets look under the bonnet to see how this has been achieved. Open the `index.js` script.
 
-1. Towards the end of the script there is a `/form` route which sends the contents of the `form.html` file to the browser. This is how we display the form.
-2. In the `form.html` file you will see that we have created a simple html form. Note:
+1. Towards the end of the script there is a `/form` route which sends the contents of the `form.handlebars` template to the browser. This is how we display the form.
+2. In the `form.handlebars` file you will see that we have created a simple html form. Note:
 	1. The `form` element contains a couple of important attributes:
 		1. The `action="/add"` attribute directs the form to send its data to the `/add` route.
 		2. The `method="post"` attribute directs the form to send its data in the message body and not in the URL.

02 Lab 3 MQTT.md

@@ -0,0 +1,60 @@
+# Publish Subscribe
+
+Up to this point all the activities have been using the **HTTP Protocol**, which uses a _request-response_ process (the client requests a resource and the server responds with this resource). If this seems unfamiliar you should work through the HTTP Protocol worksheet.
+
+Whilst this approach works fine for delivering content to a web browser it is not a useful approach for certain applications. Imagine a chat room where you had to refresh the page to view new messages.
+
+In this worksheet you will learn how to use a new HTML5 **websocket** protocol that allows a full duplex (2 way) communication over a single TCP connection. We will then explore the **MQTT** protocol which can be run over websockets and is used to implement a _push message_ system, technically called _publish-subscribe_.
+
+## 2 Set Up
+
+Start by installing the [Mosquitto Tools](https://www.eclipse.org/mosquitto/download/).
+
+If you are using MacOS you should install the [Brew Package Manager](https://brew.sh) and use this to install Mosquitto using `brew install mosquitto`.
+
+If you are using Ubuntu you can install using`sudo apt install mosquitto`.
+
+```
+mosquitto_sub -h test.mosquitto.org -t "#" -v
+```
+
+If you are using Windows 10 you can download the 64 bit Binary exe and install.
+
+## 1 The MQTT Protocol
+
+Now we have the tools installed we can start using the protocol. You have installed 2 tools, `mosquitto_pub` is used to publish messages and `mosquitto_sub` subscribes to a channel. We will use the `test.mosquitto.org` broker.
+
+Start by opening _two_ terminal windows:
+
+In the first window we will run the `mosquitto_sub` command  and subscribe to a _topic_ called `302CEM/XXX` where `XXX` is your university username.
+
+```shell
+$ mosquitto_sub -h test.mosquitto.org -t 302CEM/XXX
+```
+
+1. The `-h` flag allows us to specify the _host_, in this case `test.mosquitto.org`.
+2. The `-t` flag allows us to specify the _topic_, in this case `205CDE/XXX` (remember to substitute your username)
+
+In the second terminal we will run the `mosquitto_pub` command to publish messages to our topic.
+
+```shell
+$ mosquitto_pub -h test.mosquitto.org -t 302CEM/XXX  -m 'hello world'
+```
+
+1. The `-h` flag allows us to specify the _host_, in this case `test.mosquitto.org`.
+2. The `-t` flag allows us to specify the _topic_, in this case `205CDE/XXX` (remember to substitute your username)
+3. The `-m` flag allows us to specify the _message_, in this case `hello world`.
+
+If you look at the first terminal window (running `mosquitto_sub`) you should see your message displayed.
+
+### 2.1 Test Your Understanding
+
+Working in small groups of between 2 and 4 people:
+
+1. Decide on the _topic name_ you will use.
+2. Everyone runs the `mosquitto_sub` tool and subscribes to this same topic.
+3. Each person launches a new terminal in a new pane (so you can see both terminal windows).
+4. use the `mosquitto_pub` tool to send a message to your chosen _topic name_.
+5. Look at the output of your `mosquitto_sub` command (in the first terminal window).
+
+What have you produced? Can you think of any application for this...