Skip to content

Audit Methods

One way we can look at the security of our software is perform an Audit of the code. Like unit-testing helps us find bugs in the implementation of the code, security audit can help us spot and fix security flaws.

In this article we will get an overview of Audit Methods, and how they can be used to check our code.

Note

While you shuold do your own code audits, also consider getting a second (external) opinion.

Internal audit can pick up errors in the code, but it also has some disadvantages.

The Audit may be performed by the people who wrote the code, and therefore has the same set of assumptions. obviously we wont write insecure code, so we know what we have written is fine.

You can have an analogy with unit testing here. We know we should write tests, and we can test for the issues we know about (or think we know about). However, no test can deal with the "Unknown Unknowns"1

Types of Testing: Static Analysis

Static analysis is a set of techniques that can analyse the correctness of the software code without executing it. While a lot of these methods may be more applicable to branches of computer science such as formal methods2 where we aim to prove our code is correct before implementing it.

Some forms of Static analysis involve low-level techniques like Code Inspection or Review, Syntax Checking and Standards, which can be used manually or through a simple automated tool.

There are also more complex methods of Static analysis require more advanced technology, and use automated tools for the reasoning and examination of software. Such cases are the Automated analysis of programs (e.g. Data Flow Analysis) and the Finite state verification (e.g. Model checking).

Data Flow Analysis

This form of testing involves examining all data flows in the system, and checking the logic behind processing. This can help us to see how data is manipulated before being used.

For example, Consider a website login page, We know that the data should:

  1. Be transmitted from the site to the server in a secure way
  2. Have input sanitised (remove potential bad input) before processing
  3. Input password should be hashed to allow it to be checked in the database
  4. Input should be used for validation in the database.

We can then go through each stage of our program and map these requirements against what actually happens in the code. For Example

  • Step 1: (secure transport) happens as part of HTTP when the user connects
  • Step 2: (sanitisation) May happen:
    • on the client side, IE check format of emails etc. This can help filter some bad input and improve user experience.
    • As we KNOW we cannot trust client side validation It also needs to occur on the server side after the input is received.

By documenting each of the requirements, and mapping it to a code component that performs this task, we can demonstrate the requirements are met.

Code Review

Code review can also be used at this stage. Here we look through the code manually, searching for potential vulnerabilities.

We will talk about this in more detail in the Code Review Article

Types Of Testing: Dynamic Analysis

Is all about running the code and checking if it works. You should be familiar with this approach through the standard Unit Testing.

Dynamic analysis allows us to check for flaws in the way data is handled and input. We will discuss it in more detail in Automated Audit

Tools for Dynamic Analysis include:

  • Unit Testing
  • Fuzzing
  • Vulnerability Scanning.

"White Box" and "Black Box" Testing

As well as the testing tools above, there are also two main approaches to the testing process.

White Box Testing

In this approach we have full access to all parts of the system, and its source code. This means that white box testing will tend to focus on the source code and the structure of the application. (We can discover what the program does based on an input)

Black Box Testing

Black box testing is used to examine the behaviour of a system without any implementation details. It is so called because the system acts like a black box which cannot be accessed by the tested.

Black box testing is useful for evaluating the functionality of a system compared to its specification. IE We can compare the output of the system, for a given input.

Summary

In this section we introduced common audit methods. We will examine both Code audit and automated testing in detail in the next sections.

Grey Box Testing #greybox

You may also have come actoss the term "Grey Box Testing" (if not Google it).

  • What do you think are the advantages of the grey box approach
  • Are there any extra complications compared to the traditional white and black box testing?

  1. Donald Rumsfeld on Unknown Unknowns ↩

  2. Why don't people use Formal Methods ↩

Back to top