7 replies to this topic

[denarced]

Hello,

I've been trying to write code that prepares for errors of all kinds.
What I haven't yet done, is create sabotage; to directly cause the
errors in order to test the error behavior. Certain parts of the task
are clear; you have a OS provided function that could fail? Create
an interface for it and then when testing use the implementation
that throws the exception that could happen. This part is clear to me.

What I don't yet know is how to implement this in a clean way.
How to do it in a way that unit tests can simply activate each
error at a time in order to test the results.

All input is welcome.

Cheers

[WingedPanther]

It really depends on the code you're trying to test. Can you give an example of code you want to test?

[denarced]

WingedPanther said:

It really depends on the code you're trying to test. Can you give an example of code you want to test?

void parseReports(const string& dir)
{
    wxDir reports(dir);
    if (reports.isOpen() == false) {
        throw FailedToOpenDirException("Dir open failed");
    }

    while ( /* loop through each report */ )
    {
        ofstream report(filename);
        if(report.is_open() == false || report.good() == false)
        {
            throw FileError("Unable to open the file.", filename);
        }
    }

}

[WingedPanther]

Okay, here's what I would try:
1) pass it a directory that doesn't exist
2) pass it a directory that does exist
3) pass it a file instead of a directory
4) pass it a directory the user doesn't have access to
5) pass it an empty directory
6) pass it it a directory that only has subdirectories.

[denarced]

WingedPanther said:

Okay, here's what I would try:
1) pass it a directory that doesn't exist
2) pass it a directory that does exist
3) pass it a file instead of a directory
4) pass it a directory the user doesn't have access to
5) pass it an empty directory
6) pass it it a directory that only has subdirectories.

Thanks for that rather complete unit test :)
But that's not what I was going for, I should've been more precise (once again).
If it's only about testing a single method, class or a function, the answer is clearly a simple unit test.
What I'm thinking is how do I launch those errors from way up the call stack.
Consider that I have a legacy app, in which, starting from main, the call stack has seven levels
before reaching the method to test. The purpose is to the test system's error handling and how
the entire program reacts to certain errors/exceptions.

How would you then directly cause the error?

[WingedPanther]

When doing unit testing, you don't. You test the bottom methods that don't call anything else. Then you test the methods that only rely on those already tested. If you find a method is to hard to test, break it into smaller functions (refactor) so that you can.

[denarced]

Ok, then let's widen the net :); not unit tests.
They're then, what, integration tests ?

Today it occurred to me that I could make each failure point
read a file in the same directory and if the file contained something
relevant, it'd fail the test. When the file doesn't exist, do nothing
special. This solution feels a little heavy thou. But should work.
Or a central point of the program would keep tabs on which points
should fail and only it would read a conf file. The central solution
would be easy to exclude by making it an empty function call.

WingedPanther said:

If you find a method is to hard to test, break it into smaller functions (refactor) so that you can.

I agree on this wider point but I'm working on a legacy app and sometimes management just won't let
me refactor all that much. I'll refactor if I'm changing something anyway.

[WingedPanther]

There's a few types of testing:
1) Unit testing.
2) Integration testing: create "dummy" classes/functions with simple behavior or create driver classes that will exercise interface failure modes.
3) Functional testing: defining sequences of actions for a user/test program to implement to ensure expected interface behavior.