Reading and Writing Files

Finding files on your system

Before we go into detail about how to interact with a file it's worth noting the ways in which you can find files on your system. Python has a couple of ways to do it but you must also understand the difference between absolute paths and relative paths in programming.

Absolute paths

Absolute paths are the way in which your system stores files for you to find later. It is the full path directory in which the file is located, although it is different between Linux and Windows systems. In a Linux system, an absolute path starts with '/root/...' and in a Windows system 'C:\...'.

Absolute paths are very long sometimes and are also different from system to system. Regardless of the host operating system, it's smart to assume that someone can move or rename files on their system, that their username directories are different, or that they have chosen a different location to install the malware you have so carefully crafted. Because of this, absolute paths have limited uses in programming directly, it's better to force the program to work out where it's own absolute path is than type it directly in. Consider this code:

example-1.py from the beginning to the end

#!/usr/bin/env python3
# small program to show how absolute values can be found in python

import os 
from pathlib import Path

# the os way is considered the older way
# for anything less than python 3.4
# the __file__ will use this files name
print(f"os.path = {os.path.abspath(__file__)}")


# The modern way is to use pathlib
# this is only good for versions of 
# python above 3.4
# the __file__ will use this files name
print(f"pathlib = {Path(__file__)}") 

Example

For this example, all we are really doing is printing out the paths that have been calculated by different methods. It's worth noting the __file__ function, also known as a 'dunder' because it's surrounded in double underscores, is a way to find the name of the file that you are currently running in python. But what happens if you need to access the file next to the one that is running?? That's where relative paths come in. These methods also only find the path and not the name of the actual file, the filename is 'example-1.py' and the program thinks it's called 'main.py', try it for yourself!!

Relative paths

Because absolute paths can't always easily find a file relative to the one running the code, relative paths were also created. To find a relative path you need it to be relative to something. Take a look at this example:

example-2.py from the beginning to the end

#!/usr/bin/env python3
# A small program to illustrate relative paths in python

# library set-up
import os 
from pathlib import Path


# The os version again is the old way. 
# it's still a perfectly valid way to find 
# files and relative paths in your operating 
# system though. 
print(f"os.path = {os.path.join(os.path.dirname(os.path.abspath(__file__)), 'file.txt')}")


# the pathlib way is a the better and more 
# modern way of doing paths in python. You will 
# need python 3.4 or above for this to work
# however. 
p = Path(__file__)
file_extension = '\\file.txt'
print(f"pathlib = {str(p.parent) + file_extension}") 

Example

In this example you can start to see how these things can cascade.

If you guys want more information or another opinion on working with paths in Mac, Windows, and Linux you can find one through Medium here.

Opening files

Opening files for reading and writing is something every programmer must master at some point. In a lot of programming languages you can encounter memory faults and broken programs just by failing to open a file. If the clean up after a file being opened, such as clearing the extraneous data from memory, then you can quickly run into buffer issues and data overwriting itself if you're not careful.

Python on the other hand does a lot of this hard work for us. It has a lot of built-in cleanup functions and a lot of memory protections already there so that there is less chance of these kinds of errors and issues. The only thing is you have to make sure you use them!!

Take a look at this code and see if you can work out what's going on:

example-3.py from the beginning to the end

#!/usr/bin/env python3
# A small program to illustrate the ways that 
# reading files can be done. 

# some setup to find the file relative path location
from pathlib import Path
file_to_read = Path(__file__).parent / "file.txt"


# this is the old way of doing opening
# you can run into the troubles we mentioned
# with this style as it doesn't automatically 
# do cleanup.
f = open(file_to_read, "r")

for x in f:
    print(f.read())

f.close()


# This is the newer way to write file reading
# it takes care of the closing of the file
# and if an exception is thrown then there is
# still cleanup done. 
with open(file_to_read) as f:
    read_data = f.read()
    print(read_data)

# we can check if the file has already been closed too
if f.closed:
    print(f"The file {f.name} is already closed")

Example

In this example we can see two ways in which you can open files and interact with files in python. The first way is the older way and doesn't include any inherent error catching, and if an exception is thrown during the file opening it doesn't close the file and can leave it in a vulnerable state. The other way using the "with" statement, ensures that the file will close regardless of what happens to it. We even go so far as to prove it with a test.

When writing your own file opening and closing then make sure that you use the with statements in your code. It's also shorter than the other ways when you include the error checking try/except blocks too, so good for the lazier/efficient programmer.

If you would like more examples you can search for them pretty easily on our old pal google or find some here

Reading, writing, appending.

You may have also noticed that the open statements have a flag that is associated with them. This flag is generally used to open the file in a specific way so as to allow reading, writing, appending and such to the file. Here is a complete table of the flags:

Mode	description
r	Opens a file for reading only.
	The file pointer is placed at the beginning of the file. This is the default mode.
rb	Opens a file for reading only in binary format.
	The file pointer is placed at the beginning of the file. This is the default mode.
r+	Opens a file for both reading and writing.
	The file pointer placed at the beginning of the file.
rb+	Opens a file for both reading and writing in binary format.
	The file pointer placed at the beginning of the file.
w	Opens a file for writing only.
	Overwrites the file if the file exists. If the file does not exist, creates a new file for writing.
wb	Opens a file for writing only in binary format.
	Overwrites the file if the file exists. If the file does not exist, creates a new file for writing.
w+	Opens a file for both writing and reading.
	Overwrites the existing file if the file exists. If the file does not exist, creates a new file for reading and writing.
wb+	Opens a file for both writing and reading in binary format.
	Overwrites the existing file if the file exists. If the file does not exist, creates a new file for reading and writing.
a	Opens a file for appending. The file pointer is at the end of the file if the file exists.
	That is, the file is in the append mode. If the file does not exist, it creates a new file for writing.
ab	Opens a file for appending in binary format. The file pointer is at the end of the file if the file exists.
	That is, the file is in the append mode. If the file does not exist, it creates a new file for writing.
a+	Opens a file for both appending and reading. The file pointer is at the end of the file if the file exists.
	The file opens in the append mode. If the file does not exist, it creates a new file for reading and writing.
ab+	Opens a file for both appending and reading in binary format.
	The file pointer is at the end of the file if the file exists. The file opens in the append mode.
	If the file does not exist, it creates a new file for reading and writing.

Note

If you want another article to look at file includes, paths and working with files in general, you can find realpython's one here