03 Acceptance Testing.md

Acceptance Testing

If you have completed the foundation lab exercises (and you really should do so) you have already learned about the concept of Unit Tests being used to validate the code in your modules. These tests called the functions you defined and made sure they were returning valid data. During this process you used a process called Test-Driven Development where you wrote tests to define the functionality _before_writing the code to implement these.

Whilst this form of testing is useful to a development team to help them implement the code, it does not describe the system from the end user's point of view. This will be addressed in this lab.

Acceptance testing (also known as Functional Testing, UI Testing and Customer Testing) is testing the system from the end-user's perspective, identifying whether its behaviour meets the customer's needs. Since we are developing software that runs in the web browser it naturally follows that our acceptance tests will need to run in a web browser.

There are many benefits of (automated) acceptance testing:

1. The tests define the correct behaviour of the system in a clear, unambiguous manner.
2. The tests run automatically, meaning the development team don't need to manually test the system.
3. The entire suite of tests are run each time meaning both new defects and regressions get picked up quickly.
4. The tests are run in precisely the same way every time they are run.

1 Testing With Jest and Puppeteer

Our first version of acceptance tests will be defined using the same Jest testing framework that we used when writing our unit tests but at this point the similarity ends. We will need to run our tests in a web browser environment and the Puppeteer API provides us with a version of the Chrome browser.

Start by locating the exercises/03_acceptance/todo/ directory. You can see that it contains the now familiar todo website. Try installing all the dependencies and running the server. You should be able to access the web page in the usual manner localhost:8080.

Stop the server and close the browser.

1.1 Running the Acceptance Tests

We can run the acceptance tests by using the command:

$ npm run acceptance
  listening on port 8080
    PASS  acceptance tests/ui.test.js
      todo list
        ✓ adding one item (1616ms)

  Test Suites: 1 passed, 1 total
  Tests:       1 passed, 1 total
  Snapshots:   1 passed, 1 total
  Time:        3.95s, estimated 4s

Looking at the console output it is clear that there was a single test in a test suite that has executed and the test has passed, but how? The test must have run is a browser but where was the browser?

The tests run in a headless browser. This only exists in memory and does not have a graphical representation. There are several benefits:

1. The tests run really quickly as there is no GUI to render.
2. You can run the tests on a computer without a GUI such as a server.

Lets take this process a step at a time:

1. The npm run xxx command runs the script alias in the package.json file.
   1. If you look in the scripts key in this file you will find an alias called acceptance.
   2. This alias runs a shell script called test.sh.
2. You should be able to find the test.sh script in the project directory. If we open this we will find:
   1. The top line is a shebang line that tells your computer to run the command using the bash shell.
   2. The script then starts the koa server, the & means run it as a background process (so we don't see the output of this in stdout).
   3. Next it runs the jest command, pointing it to the acceptance tests directory.
   4. Finally, once the tests are finished it kills the background process (the koa server).

Lets take a look at the test suite in the ui.test.js file. This contains the test that is being executed.

1. On line 4 we are importing the puppeteer package. This is used to control the Chrome web browser.
2. On line 5 we import a module that allows us to take snapshots (more on this later).
3. We then define variables to hold the browser and page objects, we will assign values to these later.
4. On lines 15-19 we configure the snapshot tool that will be used as part of the test.
5. The test suite starts with a call to the beforeAll() function that runs once before the tests start:
   1. Line 22 launches a _headless_browser with the specified dimensions. Headless means there is no GUI displayed (its hidden). The sloMo key defines how many milliseconds delay should be added between keystrokes.
   2. Next we create a new tab (page) in the browser and make sure the page dimensions are correct.
6. Line 27 closes the browser after all the tests have completed.

The rest of the file contains the test (lines 29-82). There is a lot to understand here but the code is aplit into the three steps we have already covered in unit testing, Arrange, Act and Assert:

1. Arrange
   1. We point the browser at the localhost server on the correct port then refresh the browser.
   2. We then take a screenshot and save in in the screenshots/ directory. This allows us to see what is in the headless browser.
2. Act
   1. We enter data into the form (the item name and the quantity).
   2. We click on the submit button.
   3. We wait for the next page to load (can we see the top level heading?).
3. Assert
   1. We check the page title (in the tab) is correct (lines 52-53).
   2. We capture the top level heading and check it is correct (lines 56-60).
   3. We grab an array that contains the text in the first column of the table (lines 66-70).
   4. We check that there is only a single row in the table (line 75).
   5. We check that the first row of the table contains the text "bread".
   6. We grab a second screenshot.

1.1.1 Test Your Understanding

You have seen how to use Puppeteer to write a test to see that we can add a single item. We will now write some more tests. Unlike unit tests, we will be changing the state of the system in each test. The tests will run sequentially (one after the other).

1. Write a second test to check that we can add more than one item.
2. Write another test to check that if we add the same item twice we increment the quantity.

1.2 Snapshots

1.2.1 Test Your Understanding

xxx

2 Profiling

When the tests were running we included two profiling tools:

1. A Performance tick file that uses the Chrome Devtools Protocol.
2. A HTTP Archive that tracks information between the web browser and the server. This is used to:
   1. identify performance issues such as slow load times and,
   2. identify page rendering problems.

Both of these profilers save their log files to the trace/ directory. The files contains a lot of information stored in JSON format. To visualise this data we need to load it into visualisation tools.

The tick file can be loaded into the Performance tab of the Chrome Dev Tools using the up arrow button.

This shows the timing of the various steps and the CPU and memory load. Try hovering over the different elements and scrubbing along the grey bar.

The HAR files can be visualised using the GSuite Toolbox HAR Analyser. Use this to open the .har file and you will see something like the following:

There is a lot of information generated. Hover over the different icons to find out more.

An alternative is to use the network tab of the Chrome DevTools. Clicking on the up arrow as shown allows you to load the har file.

There are many more settings you can experiment with to help understand the performance of your web server. Start by looking at this tutorial.

2.1 Test Your Understanding

xxx

2.2 Flame Graphs

Another question often asked by software developers is how the software is consuming resources, what exactly is consuming how much, and how did this change since the last software version? These questions can be answered using software profilers, tools that help direct developers to optimize their code and operators to tune their environment. Flame graphs are generated from perf output.

The output of profilers can be verbose, however, making it laborious to study and comprehend. The flame graph provides a new visualization for profiler output and can make for much faster comprehension, reducing the time for root cause analysis.

Flame graphs are a way of visualizing CPU time spent in functions. They can help you pin down where you spend too much time doing synchronous operations. In this exercise we will use a package called 0x which has already been added to the devDependencies in the todo project. You can start the profiler using the script alias in the package manifest or by running the command directly:

$ npm run profiler

> todo@1.0.0 profiler /dynamic-websites/exercises/03_acceptance/todo
> 0x -o index.js

🔥  Profilinglistening on port 8080

You should now launch the app in the web browser and add and remove items from the list. This will capture the perf output in a directory ending in .0x with each run generating a new directory. Once you have made use of the todo list, press ctrl+c to quit.

🔥  Process exited, generating flamegraph
🔥  Flamegraph generated in
file://dynamic-websites/exercises/03_acceptance/todo/24526.0x/flamegraph.html

It will then try to open the specified html document in your default browser which will look something like this:

Hovering over a box will reveal more information.

1. Each box represents a function in the stack (a "stack frame").
2. The y-axis shows stack depth (number of frames on the stack).
   1. The top box shows the function that was on-CPU. Everything beneath that is ancestry.
   2. . The function beneath a function is its parent, just like the stack traces shown earlier.
3. The x-axis spans the sample population.
   1. It does not show the passing of time from left to right, as most graphs do.
   2. The left to right ordering has no meaning (it's sorted alphabetically to maximize frame merging).
   3. The width of the box shows the total time it was on-CPU or part of an ancestry that was on-CPU (based on sample count).
   4. Functions with wide boxes may consume more CPU per execution than those with narrow boxes, or, they may simply be called more often. The call count is not shown (or known via sampling).

2.2.1 Test Your Understanding

xxx

3 Domain-Specific Languages

Earlier in this lab you wrote a series of acceptance tests to ensure that your software (web application) met the user requirements. These tests were written in JavaScript using the Jest testing frameworks.

3.1 Gherkin

Gherkin is the format for cucumber specifications. It is a domain specific language which helps you to describe business behavior without the need to go into detail of implementation. This text acts as documentation and skeleton of your automated tests. Gherkin is based on TreeTop Grammar which exists in 37+ languages. Therefore you can write your gherkin in 37+ spoken languages.

This script serves two primary purposes:

1. Documents user scenarios
2. Writing an automated test (BDD)