Welcome to day 19 of the 30 Days of Python series! Today we're going to be looking at the very important topic of exception handling.
We're also going to talk a little bit about how exceptions are used in Python, and the concept of asking for forgiveness vs. asking for permission in our code.
What are exceptions?
At this point in the series, I'm sure we've all encountered our fair share of exceptions.
TypeError are probably starting to feel like co-workers that you see every time you sit down to code.
At the moment, encountering one of these exceptions when running our code is fatal to the application. If we try to turn user input into an integer, and the user enters
"Five", the program is going to terminate, and we're going to get some lovely red text in the console about how
"Five" isn't a base-10 integer.
Traceback (most recent call last): File "main.py", line 1, in <module> int("Five") ValueError: invalid literal for int() with base 10: 'Five'
Looking at an example like this, it's fair to assume that exceptions are just errors, and for the most part that assumption holds true. Most exceptions are indeed errors. However, there are some exceptions which don't really indicate that something went wrong. They're more like notifications that a given event occurred.
An example is
StopIteration, which is probably the most common exception being raised in your applications right now: you've just never seen it, because it never terminated your programs! More on that in a little bit.
One place we see
StopIteration is when we iterate over some iterable in a
for loop. It's used to indicate that the all of the iterable's values have been used, and it has no new values to give us. This is how Python knows when to terminate
for loops, and the same happens when we use destructuring to assign to several variables.
Using an exception as a signal like this is very common in Python, and this is something we're going to return to throughout this series.
Asking for permission vs asking for forgiveness
Let's return to the example from earlier where the we're trying to get an integer from the user, and they enter
"Five" instead of the numeral we expect.
Our code may look something like this:
number = int(input("Please enter a whole number: "))
This type of code is nothing new to us, but it's good for us to be clear about what we're discussing.
Perhaps you disagree, but I think terminating the application in this case is a bit of a strong response to the user entering an invalid value. Even if we do want to end the program, giving the user a big red error message is probably not the best way to go. It makes it sound like something is broken, rather than there being an issue with their input. After all, we never had any plans to accept numbers written as words.
Let's think about how to solve this problem with the things we know so far.
I think a sensible approach here would be to put the prompt inside a
while loop. We can then check if the user input is a valid integer, and if it is, we'll use that assign that value to a variable and break out of the loop. If it isn't valid, we'll move onto the next iteration, and the user will be prompted again.
The question is, how do we check if something is a valid integer value when we just have a string?
This is a bit tricky for us to do manually, but if you've been looking at the methods we have available for strings, you may have found the
isnumeric method. We can therefore write something like this:
while True: user_number = input("Please enter a whole number: ") if user_number.isnumeric(): number = int(user_number) break else: print("You didn't enter a valid integer!")
If we give it a try, it seems to work. We can enter
4834854, and it doesn't accept
3.141592, which is good, because would cause problems for our
However, if you tested any negative numbers, you're going to uncover an issue.
False for negative numbers, because not all of the digits are numerals.
That's a problem, but it's not insurmountable. We can just strip off any initial
- symbol, since we know that
int can handle those. For this, we might use
lstrip, rather than normal
lstrip only removes the character from the left side of the string.
while True: user_number = input("Please enter a whole number: ") if user_number.lstrip("-").isnumeric(): number = int(user_number) break else: print("You didn't enter a valid integer!")
Let's try again!
-1 works no problem now, so that's great. Positive numbers still work as well, so we haven't broken anything there. However, we do have a new problem:
lstrip is a bit too good, and it will strip off many
- characters if it finds them. That's an issue for us, because while
-3 is a valid number as far as
int is concerned,
That means that our if statement will accept
--3 as valid, but then converting it to an integer will give us an error.
If we try
--3, we get a
Traceback (most recent call last): File "main.py", line 5, in <module> number = int(user_number) ValueError: invalid literal for int() with base 10: '--3'
At this point, I think it's starting to become clear that we're on the wrong path. Even for this very simple case, we're having to manually deal with lots of edge cases, and it can be difficult to know if we're missing something.
This kind of approach is called "asking for permission". We're checking if something can be done in advance, and then we proceed if we determine that there aren't going to be any problems. As we've seen, this approach can be very messy, and can get extremely complicated.
This is not the approach to exception handling that we take in Python. In Python, the preferred approach is to simply attempt what we think may fail, and then to recover from an exception if one occurs. This turns the problem into a much simpler one: knowing what exceptions might occur. In the case above, we only need to worry about one exception:
This alternative pattern is known as "asking for forgiveness", because we're attempting something that could go wrong, and then we're doing something to make amends if something does go wrong.
Let's take a look at a new piece of syntax that will allow us to use this asking for forgiveness pattern: the
try statement can get very long and detailed, but we actually only need two parts to get going. We need a
try clause, which houses the code we expect to fail, and then usually we need at lease one
except clause that will describe what to do when a certain type of failure occurs.
Let's look at our number example again to see a
try statement in action:
while True: try: number = int(input("Please enter a whole number: ")) break except ValueError: print("You didn't enter a valid integer!")
Here we have two lines of code we want to try. We would like to take in some user input and convert it to an integer, and then we would like to break out of the
while loop if nothing goes wrong.
except clause is waiting to see if any
ValueError is raised while we're running these operations in the
try clause. If a
ValueError is raised, we abandon the code in the
try clause and we perform the actions listed in this
except clause instead.
In this example, we simply print,
"You didn't enter a valid integer!", and then we run out of code in the loop body, so a new iteration of the loop begins.
Note that we don't get an error message in the console when the
We "handled" the exception with our
except clause, and we provided an alternative course of action. Only unhandled exceptions terminate the application, because in those cases, we haven't provided a viable alternative. Terminating the application is really Python's last resort.
Another thing to keep in mind is that we've only handled the one type of exception here. What happens if we somehow get a
except clause is doing nothing at all to handle a
TypeError so we aren't providing some alternative course of action in this instance. That means a
TypeError is still going to terminate our program, and that can sometimes be exactly what we want. Sometimes there's just nothing we can do to correct a problem, and in those cases letting an exception terminate the program is perfectly acceptable.
An important thing to keep in mind is that the
try block is going to stop running as soon as an exception occurs. If something goes wrong, it's as if none of the code in the
try block ever ran.
Handling multiple possible exceptions
There are two ways we can handle multiple exceptions using a single
try statement, and they have different use cases.
Let's first imagine we have two different exceptions that might occur, and we want to do different things depending on what happens. In this case we should have two
except clauses, and each
except clause should describe the course of action we want to take when that exception occurs.
As an example, let's create a function that calculates the mean average from some collection of numbers. We don't know what the user is going to pass into this function, so a few things can go wrong.
Here is what we're going to start off with:
import math def average(numbers): mean = math.fsum(numbers) / len(numbers) print(mean)
fsum here, just because the numbers are likely to be floats rather than integers in many cases.
Okay, so let's think about potential issues:
- The user may pass in an empty collection, so then we're going to get
len. That's going to lead to division by
0, which is not allowed. In this case, we get a
- The user may pass in something which isn't a collection. This is going to give us a
fsumexpects an iterable, and
lenexpects a collection.
- The user may pass in a collection which contains things which aren't numbers. This is also going to be a
That gives us two exceptions we need to take care of:
If you're not sure what kind of error is going to occur, a good approach is just to try the test case and see what happens. The documentation is also quite good at describing what exceptions occur for various operations and functions.
Now that we know what can go wrong, we can write our
import math def average(numbers): try: mean = math.fsum(numbers) / len(numbers) print(mean) except ZeroDivisionError: print(0) except TypeError: print("You provided invalid values!")
Now we're handling situations where we get
ZeroDivisionError differently from those where we get
TypeError, so we're able to provide more specific feedback on what went wrong. In the case of
ZeroDivisionError, we're not even informing the user of an issue: we've decided that the average of nothing is
0 for the purposes of our function.
If we instead want to catch both exceptions and do the same thing, we don't need two
except clauses. We can catch multiple exceptions with the same
except clause like so:
import math def average(numbers): try: mean = math.fsum(numbers) / len(numbers) print(mean) except (ZeroDivisionError, TypeError): print("An average cannot be calculated for the values you provided.")
At some point you may run across code which reads like this:
import math def average(numbers): try: mean = math.fsum(numbers) / len(numbers) print(mean) except: print("An average cannot be calculated for the values you provided.")
You'll notice that we don't have any exceptions listed after the
except clause. This is called a bare
except clause, and it will catch any exceptions that occur.
While this has its uses, this is generally a really bad thing to use in your code. It opens up the possibility of catching many exceptions we didn't anticipate, and it can mask serious implementation problems.
In addition to the
except clauses, we can also use an
else clause with our
try statements. The code under the
else clause only runs if no exceptions occur while executing the code in the
When finding out about
else I know a lot of students think it's totally useless. Can't we just put more code in the
We can, but here are a couple of reasons why this can be a really bad idea.
- We may accidentally catch exceptions we didn't anticipate. The more code that ends up in the
tryclause, the more likely this is to happen. This may make our handling of the exception inappropriate, because we're left dealing with a situation which didn't actually occur.
- It harms readability. The
tryclause expresses what we expect to fail, and the
exceptclauses express the ways that we plan to handle specific failures in that code. The more code that gets added to the
tryclause, the less clear it is what we're actually trying, and that can make the whole structure more difficult to understand.
Does that means we always need an
else clause? No.
Use your judgement. For very simple examples, it can be overkill, like in this example:
import math def average(numbers): try: mean = math.fsum(numbers) / len(numbers) except ZeroDivisionError: print(0) except TypeError: print("You provided invalid values!") else: print(mean)
We're not likely to fall prey to any issues when printing a number, and it's clear that this is not the thing we're concerned with testing.
Just make sure you don't forget about
else when it comes to more complicated operations.
In addition to
else, we have one more important clause available to us for
finally is very special, because it will always run.
If an unhandled exception occurs, it doesn't matter.
finally will still run its code before that exception terminates the program.
If we return from a function inside the
finally will interrupt that return to run its own code first. You can see an example by running this code:
def finally_flex(): try: return finally: print("You return when I say you can return...") finally_flex()
This property is extremely useful for any situations where vital clean up is required after an operation. An example is when working with files. What happens if we encounter some problem while processing data in a file? We still want to close the file when we're done, and with
finally we can make sure that this happens.
The context manager we've been using for working with files actually uses something very similar to this
finally clause behind the scenes, to ensure that files are closed no matter what. It's so similar in fact that there are ways to make our own context managers using
try statements with
The main use cases for
finally are generally found in more advanced code, but it's still an important tool to know about at this stage.
1) Create a short program that prompts the user for a list of grades separated by commas. Split the string into individual grades and use a list comprehension to convert each string to an integer. You should use a
try statement to inform the user when the values they entered cannot be converted.
2) Investigate what happens when there is a
return statement in both the
try clause and
finally clause of a
3) Imagine you have a file named
data.txt with this content:
There is some data here!
Open it for reading using Python, but make sure to use a
try block to catch an exception that arises if the file doesn't exist. Once you've verified your solution works with an actual file, delete the file and see if your
try block is able to handle it.
When files don't exist when you try to open them, the exception raised is
You can find our solutions to the exercises here.
You can find more information about all of the built in exceptions here.