Space Complexity¶

Welcome to the final lecture, Week 11.

We have finally made it to the end of this module, and the last topic will look at the storage cost of computation.

Learning Objectives for this week

Represent numbers using unary and positional systems (binary, etc.).
Define space complexity.
Define SPACE and NSPACE.
Define the space complexity classes L and NL.
Define the classes EXPTIME and EXPSPACE.
State the Savitch Theorem.
Know that PSPACE=NPSPACE by Savitch Theorem.
Know the Complexity Onion. (The subset shown in this lecture.)

Pre-class Activity¶

Imagine that we decide to represent all numbers using a unary alphabet, so the first numbers are written as follows:

▪
▪▪
▪▪▪
▪▪▪▪
▪▪▪▪▪
▪▪▪▪▪▪
▪▪▪▪▪▪▪
▪▪▪▪▪▪▪▪
▪▪▪▪▪▪▪▪▪
▪▪▪▪▪▪▪▪▪▪

etc.

Such a system would make addition very easy: just a concatenation!

Subtraction is marginally harder, and multiplication is a succession of concatenations, while division can be done via a (long) succession of subtractions.

This is a perfectly working system, and similar system were used in ancient number systems. So why do we use the familiar positional system despite it being trickier for e.g. addition?

Answer

The positional system is economical in the number of symbols needed to write a given number.

Imagine having to write the number "one million":

Using the unary system we would need a million symbols, that is a thousand pages of paper where each page has a thousand symbols!
Using the familiar decimal positional system we only need 7 symbols! 1000000.

Let us generalise and work out the values for arbitrary numbers.

Question

Given a number \(n\), how many symbols do we need to write it in unary and binary number systems?

Answer

In unary we need \(n\) symbols.
In binary we need about \(\log_2 n\) symbols.

Justification

The number 111....11, with \(k\) many 1's, is the largest number of length \(k\) symbols.

It represents the number \(2^k-1\).

We want to find the 111....11 that is just long enough to represent a number equal to \(n\) or bigger than it.

The smallest \(k\) that makes \(2^k-1\) bigger than \(n\) is the solution to \(2^k-1\geq n\), i.e. \(2^k\geq n+1\), i.e. \(k\geq \log_2 (n+1)\).

Lecture videos¶

The PDF slides are available for download below.

Download Slides

Space Complexity¶

Following the same approach that we used to study time complexity, we can define what it means for a TM to "run in space \(m(n)\)" (= the storage required for an input of size \(n\)). And we can also define classes for deterministic and non-deterministic space, SPACE and NSPACE (similar to TIME and NTIME).

Savitch Theorem¶

In the same way that we defined the time complexity classes P and NP, we can define two space complexity classes: PSPACE and NPSPACE depending on whether the TM is deterministic or non-deterministic.

The surprising result is that contrary to the P vs NP, which is still open, we know that PSPACE=NPSPACE (by Savitch Theorem).

L and NL¶

For time complexity, we usually need at least enough time to read the input, so usually the time is \(\geq n\).

For space complexity, we often care about how much more space is needed beyond the space required to store the input. This extra space can be as small as constant or logarithmic in terms of \(n\).

The extra space can be logarithmic often because we encode the numbers using the familiar positional system.

Complexity Onion¶

There are a lot of complexity classes, and drawing their Venn diagram gives a shape that looks like an onion...

Here we introduce another two important classes: EXPTIME and EXPSPACE, and we draw the picture with all the classes that we have seen so far.

Ready for the lab exercises?

Attempt Lab 11 exercises

Ask & Answer Questions

Don't forget: you can use Teams or the Aula's feed to raise questions for me and the rest of the class.

Add your questions to the Community Feed and take a look at others' questions. If you see a question which you'd like to answer then just go ahead and participate in the discussion!