Error Handling and Exceptions

[sdfgeoff]

In another topic I posted the line:

The end two options assume you don’t put big try-except statements through your code. If you are, you should probably reconsider why your code is generating exceptions. <rant>For some reasons, throwing and catching exceptions is considered ‘pythonic’, but in my mind, exceptions should be reserved for when there is unexpected behaviour. Pretty much: if you can check first, do, and don’t be afraid to throw exceptions yourself.</rant>

To which there was the response

It is the opposite, you catch expected errors.

Unexpected errors are errors you do not catch = your code is not ready to deal with them.

In other words: When you catch an error it means the error is expected behavior of the surrounded code. The surrounding code knows how to deal with it.

Finally it comes to the same point: do not be afraid to throw exceptions when your code does not follow the “good case”. It is sort of a return value too.

And

Throwing exceptions is sin!!

So let’s discuss it.
***We’ll start by me talking about how and why I use exceptions.

Where do I use Try-Except
I use try except to handle things that cannot be controlled, such as the input and output to files:

        try:
            new_data = json.load(open(self._path))
        except:
            try:
                json.dump(
                    self._defaults, 
                    open(self._path, 'w'), 
                    indent=2, 
                    sort_keys=True
                )
            except:
               print("Unable to write file, check permissions")
            new_data = self._defaults.copy()

It’s a bit of safety around loading JSON files. If it can’t load them, it will attempt to save the defaults and then use them.
Why do I use them here:
It adds robustness to my program, because it can now deal with unusual/exceptional situations such as missing/corrupted data files or a read only file system, and still carry on working.
A note on the word ‘unexpected’: Clearly I have anticipated the code failing here, but this is still ‘unexpected.’ Why? I would expect that the file is valid as I saved to it earlier - something odd must have happened here, and we can recover from it.

To me this is what try-except’s are about: keeping your code running when odd things are happening. It’s for the cases where the program can operate with a little less functionality in a situation where it formerly couldn’t - because of the error handling.

Where do I not use Try-Except

Code that runs every frame:

try:
     property_value = cont.owner['property_name']
except IndexError:
    cont.owner['property_name'] = 0
    property_value = 0

Instead I would ensure it exists by creating it before I attempt to access it (through an initialisation routine).
Likewise, I would not use cont.owner.get(‘property_name’, 0) here either - unless it is reading configuration set by the user. making this possibly invalid input.

Why do I not use them here?
A game object does not have the properties by default. Thus, it is expected that this code will fail! So why even try the first time? Set up a system so that your code does not ‘fail’ in an expected way.

When do I raise errors:
How about this

def from_dict(source):
    if source['type'] not in TYPE_STRINGS:
        raise ValueError("Unknown source type {} requested",.format(source['type']))

Again this is part of parsing external input. It is when there are no syntax issues with the input, but rather the operation cannot be completed because the argument value is wrong. In this case it is non-recoverable, so we raise an error. Why not let python raise it’s own error when we do the indexing? Because we want the error to explain a bit more about the problem. A generic IndexError which would be thrown by default would not tell the user what is actually wrong, but by raising it ourself we can add more information.

If the string not being in the dictionary was recoverable, I would not raise the error, but rather log a warning to the file, and continue (presumably using a default value).

When do I use custom errors
When there are issues that need to be conveyed that cannot be encapsulated by one of the default error types. In a non-bge project, I was working with a protocol for talking over serial. A python parser and GUI was written around this protocol, and it used a whole heap of custom errors to deal with malformed headers, invalid checksums etc. These are all ‘unexpected’ behaviour that prevent the code from working, and provide much more information to a person trying to get your code to work.

An interesting use
In a scheduler I have the code:

        errors = list()
        for event in self.events:
            if event.time_until_next_run <= 0:
                try:
                    event(self)
                except Exception as err:
                    errors.append(err)
        for err in errors:
            raise err

Now why would I do this? Catch every single error, and then a little later, raise them all (or at least, raise the first one)!
The reason is quite simple. A scheduler runs multiple independent bits of code. Just because one part has failed doesn’t mean all of it has. Imagine one AI in your game hits an error - it enters an invalid internal state. Now a whole heap of your other code is not even running, even though it may deal with other things completely, such as player input and GUI handling. So it adds robustness, keeping as much of your code running even though some parts have failed. If this scheduler weren’t in the BGE (which starts again next frame), I’d likely have a system that just logs the errors and then continues.

---

Summary

It’s much easier to not ‘bug’ in the first place than it is to ‘debug’ later

• My cosc lecturer

If you just code it right from the start, there won’t be exceptions, and handling them won’t be necessary!

• This anecdote

A try-except that is expected to be used in non-exceptional situations is a bug. In C, an inappropriate list access will crash your program with a segfault. In python, it generates an IndexError. In C, it’s considered a bug, in python, well, you can catch and ignore it. DON’T. Make sure you aren’t providing your code with bad data internally intentionally.

• Use exception handling when it an error is unexpected but recoverable.
• Don’t use exception handling on internal, recoverable things.
• Raise exceptions wen something unrecoverable happens and providing more data will help debugging.

---

So when do you use exceptions and error handling?

[BluePrintRandom]

try:
(tab)command = Dictionary[str(input)]
(tab)command(own)
except:
(Tab) pass

This allows me to avoid almost all branching in many cases

[Monster]

Unspecified error handling should rarely be used:

try:
   ...
except:
   ...

It catches ALL errors. This construct is viable when you want to explicitly encapsulate the surrounded code. You have to be aware you need to deal with ALL these different errors - even errors you do not think about. In most cases it means you have to treat them all the same. There are use cases to it, e.g. to close a connection or roll back a transaction. It should not be uses just because you can.

The “pass” in the unspecified except clause is evil!

Why? Because it hides any error without dealing with it. This makes it nearly impossible to maintain the surrounded code as there is no information that errors happened or not. Imagine there is an AttributeError. You would not notice that you mistyped the attribute.

Better specify the error:

try:
   ...
except KeyError:
   pass # explicitly do not care if the key exists or not

It is valid to use “pass” on specific errors e.g. when KeyError. The code explicitly says: “yes, I know this error. When it happens I will not do anything and just go on”. The

In general I recommend to mention any expected error you expect. This way any unexpected error will be passed to the code layer above. Additional the code tells what you expect can happen beside the “good case”.

[Smoking_mirror]

Why is it important to identify errors?
Because we need to know what went wrong and try to fix it.

Usually the game works fine on my computer, but if someone else is running it and it doesn’t work I need to know why.

I try to raise errors and exit the game right away when I’m working on it, but when the game is finished I’d like to raise an error in game, maybe send the player to a special screen:
“Permission denied when writing save game.”
…rather than having the game crash.

Sometime I expect the possibility of failure. When connecting to online content for example, or saving a file to disk, you can’t do an “if” statement, all you can do is try and handle the possibility of failure.

[BluePrintRandom]

@Monster

Unspecified error handling should rarely be used:

try:
   ...
except:
   ...

It catches ALL errors. This construct is viable when you want to explicitly encapsulate the surrounded code. You have to be aware you need to deal with ALL these different errors - even errors you do not think about. In most cases it means you have to treat them all the same. There are use cases to it, e.g. to close a connection or roll back a transaction. It should not be uses just because you can.

The “pass” in the unspecified except clause is evil!

Why? Because it hides any error without dealing with it. This makes it nearly impossible to maintain the surrounded code as there is no information that errors happened or not. Imagine there is an AttributeError. You would not notice that you mistyped the attribute.

Better specify the error:

try:
   ...
except KeyError:
   pass # explicitly do not care if the key exists or not

It is valid to use “pass” on specific errors e.g. when KeyError. The code explicitly says: “yes, I know this error. When it happens I will not do anything and just go on”. The

In general I recommend to mention any expected error you expect. This way any unexpected error will be passed to the code layer above. Additional the code tells what you expect can happen beside the “good case”.

I set the ‘input’ as a debug property, and I have the ‘command’ set a state debug property,
if it’s faling inside the command, I fall back to print

I am counting on it failing lots,

I am checking to see if something has a entry, if it does not, don’t do anything

input == “”

dict[ “” ] = Idle

input == str([“Forward”])
dict[input]= walkForward()

input == str([“Forward”, “Right”])
dict[input] = Turn right while walking

avoiding a big list of nested if statments.

My keymapper does the same thing to convert keypresses and joystick presses to serialized output

(joystick up, and W, and up arrow all check to see if forward is on the list in the first postion on the list)

dict[key] = [ ‘Entry’ , sortOrderInt]

after entries are put on list the list is sorted

[Monster]

@BluePrintRandom: how is this related to error handling?

[Smoking_mirror]

@BluePrintRandom

try:
(tab)command = Dictionary[str(input)]
(tab)command(own)
except:
(Tab) pass

This allows me to avoid almost all branching in many cases

try:
    something()
except:
    pass

This kind of error handling isn’t recommended. You need to specify a little further which kind of errors you are expecting to crop up. In this case the input might not be in the dictionary (a KeyError) or the command function might not exist. In that case you could miss an important error because it is passed.

You could do:

command = command_dictionary.get(str(input))
if command:
    command(own)

Having a dictionary of commands is a good idea though, I use that in my profiler.

[agoose77]

@sdfgeoff"

In another topic I posted the line:

To which there was the response

And

So let’s discuss it.
***We’ll start by me talking about how and why I use exceptions.

Where do I use Try-Except
I use try except to handle things that cannot be controlled, such as the input and output to files:

        try:
            new_data = json.load(open(self._path))
        except:
            try:
                json.dump(
                    self._defaults, 
                    open(self._path, 'w'), 
                    indent=2, 
                    sort_keys=True
                )
            except:
               print("Unable to write file, check permissions")
            new_data = self._defaults.copy()

It’s a bit of safety around loading JSON files. If it can’t load them, it will attempt to save the defaults and then use them.
Why do I use them here:
It adds robustness to my program, because it can now deal with unusual/exceptional situations such as missing/corrupted data files or a read only file system, and still carry on working.
A note on the word ‘unexpected’: Clearly I have anticipated the code failing here, but this is still ‘unexpected.’ Why? I would expect that the file is valid as I saved to it earlier - something odd must have happened here, and we can recover from it.

To me this is what try-except’s are about: keeping your code running when odd things are happening. It’s for the cases where the program can operate with a little less functionality in a situation where it formerly couldn’t - because of the error handling.

Where do I not use Try-Except

Code that runs every frame:

try:
     property_value = cont.owner['property_name']
except IndexError:
    cont.owner['property_name'] = 0
    property_value = 0

Instead I would ensure it exists by creating it before I attempt to access it (through an initialisation routine).
Likewise, I would not use cont.owner.get(‘property_name’, 0) here either - unless it is reading configuration set by the user. making this possibly invalid input.

Why do I not use them here?
A game object does not have the properties by default. Thus, it is expected that this code will fail! So why even try the first time? Set up a system so that your code does not ‘fail’ in an expected way.

When do I raise errors:
How about this

def from_dict(source):
    if source['type'] not in TYPE_STRINGS:
        raise ValueError("Unknown source type {} requested",.format(source['type']))

Again this is part of parsing external input. It is when there are no syntax issues with the input, but rather the operation cannot be completed because the argument value is wrong. In this case it is non-recoverable, so we raise an error. Why not let python raise it’s own error when we do the indexing? Because we want the error to explain a bit more about the problem. A generic IndexError which would be thrown by default would not tell the user what is actually wrong, but by raising it ourself we can add more information.

If the string not being in the dictionary was recoverable, I would not raise the error, but rather log a warning to the file, and continue (presumably using a default value).

When do I use custom errors
When there are issues that need to be conveyed that cannot be encapsulated by one of the default error types. In a non-bge project, I was working with a protocol for talking over serial. A python parser and GUI was written around this protocol, and it used a whole heap of custom errors to deal with malformed headers, invalid checksums etc. These are all ‘unexpected’ behaviour that prevent the code from working, and provide much more information to a person trying to get your code to work.

An interesting use
In a scheduler I have the code:

        errors = list()
        for event in self.events:
            if event.time_until_next_run <= 0:
                try:
                    event(self)
                except Exception as err:
                    errors.append(err)
        for err in errors:
            raise err

Now why would I do this? Catch every single error, and then a little later, raise them all (or at least, raise the first one)!
The reason is quite simple. A scheduler runs multiple independent bits of code. Just because one part has failed doesn’t mean all of it has. Imagine one AI in your game hits an error - it enters an invalid internal state. Now a whole heap of your other code is not even running, even though it may deal with other things completely, such as player input and GUI handling. So it adds robustness, keeping as much of your code running even though some parts have failed. If this scheduler weren’t in the BGE (which starts again next frame), I’d likely have a system that just logs the errors and then continues.

---

Summary

• My cosc lecturer

• This anecdote

A try-except that is expected to be used in non-exceptional situations is a bug. In C, an inappropriate list access will crash your program with a segfault. In python, it generates an IndexError. In C, it’s considered a bug, in python, well, you can catch and ignore it. DON’T. Make sure you aren’t providing your code with bad data internally intentionally.

• Use exception handling when it an error is unexpected but recoverable.
• Don’t use exception handling on internal, recoverable things.
• Raise exceptions wen something unrecoverable happens and providing more data will help debugging.

---

So when do you use exceptions and error handling?

Python is based around EAFP. From the docs:

Easier to ask for forgiveness than permission. This common Python coding style assumes the existence of valid keys or attributes and catches exceptions if the assumption proves false. This clean and fast style is characterized by the presence of many try and except statements. The technique contrasts with the LBYL style common to many other languages such as C.

Consquently, one is usually encouraged to use exception handling than conditional pre-checks. There are times when this isn’t relevant (such as in performant code where the exception case is common, or raising the exception is expensive), but most of the time we’re not writing this code.

[sdfgeoff]

@Smoking_Mirror

Sometime I expect the possibility of failure. When connecting to online content for example, or saving a file to disk, you can’t do an “if” statement, all you can do is try and handle the possibility of failure.

And to me, that is exactly when exceptions should be used. In nearly all other cases the situation should be checked first.

@BPR

I am counting on it failing lots,

I am checking to see if something has a entry, if it does not, don’t do anything
… ship …
avoiding a big list of nested if statments.

I just had a look through your control mapper and … uh!!!
It makes a nice case study of how not to do a lot of things.

Since we’re talking about error handling here, I’ll post a snippet:

    for event in L:  # Note from sdfgeoff: L is bge.logic.keyboard.events
        #Gather Movement keys
        try:
            event = dictM[bge.events.EventToString(event[0])]
            if event not in listM:
                listM.append(event)
                
        except:
            pass
        

It’s clear you’re expecting something to fail here. I’m guessing it’s that dictM lookup (dictM is {‘WKEY’:‘Forwards’, ‘SKEY’:Backwards} and so on.
Yes, trying it and catching the exception (which is an IndexError, and should be caught specifically) does work, but there is a better way.

A much simpler setup that doesn’t need error handling at all is to go:

for event_str in dictM:
    # event_str is something like 'WKEY'
    event_code = bge.events.__dict__[event_str]  # Convert it into a character code - this is a neat little trick.
    event_status = bge.logic.keyboard.events[event_code]
    listM.append({event_code:event_status})

And iterate through the things you are looking for rather than iterate through all of them. I believe the above code is functionally equivalent. (but haven’t tested).
Actually, I can’t figure out why you’re making listM at all - because later on (in another file admittedly), you do a lookup in it using dictM again, recovering data that you already had and discarded.

@BPR: Have you ever looked at using module mode instead of script mode? It will make some things a lot simpler

[BluePrintRandom]

@sdfgeoff

@Smoking_Mirror

And to me, that is exactly when exceptions should be used. In nearly all other cases the situation should be checked first.

@BPR

I just had a look through your control mapper and … uh!!!
It makes a nice case study of how not to do a lot of things.

Since we’re talking about error handling here, I’ll post a snippet:

    for event in L:  # Note from sdfgeoff: L is bge.logic.keyboard.events
        #Gather Movement keys
        try:
            event = dictM[bge.events.EventToString(event[0])]
            if event not in listM:
                listM.append(event)
                
        except:
            pass
        

It’s clear you’re expecting something to fail here. I’m guessing it’s that dictM lookup (dictM is {‘WKEY’:‘Forwards’, ‘SKEY’:Backwards} and so on.
Yes, trying it and catching the exception (which is an IndexError, and should be caught specifically) does work, but there is a better way.

A much simpler setup that doesn’t need error handling at all is to go:

for event_str in dictM:
    # event_str is something like 'WKEY'
    event_code = bge.events.__dict__[event_str]  # Convert it into a character code - this is a neat little trick.
    event_status = bge.logic.keyboard.events[event_code]
    listM.append({event_code:event_status})

And iterate through the things you are looking for rather than iterate through all of them. I believe the above code is functionally equivalent. (but haven’t tested).
Actually, I can’t figure out why you’re making listM at all - because later on (in another file admittedly), you do a lookup in it using dictM again, recovering data that you already had and discarded.

@BPR: Have you ever looked at using module mode instead of script mode? It will make some things a lot simpler

No, the other dictionary is not Keypresses, it’s game Commands

[Controller (gather input) -> send input to targeted actor -> actor calls local command]

W, UP_arrow, Joytstick_up = all add ‘Forward’ to the list if it does not exist

the input list is checked against a dictionary local to each actor*

[‘Forward’] called by car, is different than [‘Forward’] called by actor

this also allows drag and drop actors without linking any python scripts
(they are attached to the actor)
I just re-wrote this today*
(still a very heavy wip)

removing Try: entierly

also - this is to allow multiusers on 1 pc or even potentially networking

Attachments

Serilizer_2.blend (470 KB)