Programming assignments and blind men and the elephant

Evaluating a tough programming elephant is like the story of the blind men and the elephant. Let me explain.

The assignments that I’ve handed out as part of my Spreadsheet Modelling for Business Decision Problems course at IIMB involve fairly complex spreadsheet modelling (as the name of the course suggests). Thus, while it is a lot of effort on behalf of the student to do the assignment, it is also a lot of effort on my behalf if I’ve to go through the code line by line (these guys code using VBA macros), understand it and evaluate them.

Instead, I have come up with a set of “tests” – specific inputs that I give to the program (I’ve specified what the “front sheet” should look like so this is easy), and then see if the program gives out the desired outputs. Either way, I dig a little deeper and see if they’ve done it right, and based on that I grade the assignment.

If the assignments that they’ve turned in are elephants, it’s too much of an effort for me to open my eyes and actually see that they are elephants. Hence, I feel around, and check for a few different components to make sure they’ve submitted elephants. So for this assignment I might check if the trunk is like a snake, and if so, they’ve passed. For another assignment, I might check if the legs are like trees, and if they are, pass them. And so forth.

Now, this is evidently not perfect. For example, if you know that I’ll only check for the trunk to be like a snake, you’ll just submit a trunk that’s like a snake rather than submitting a full assignment! But if you don’t know what I’m going to check for, then it might be possible that you’ve only submitted a snake, I look for treetrunks and not finding them, give you a failing grade! There is a little bit of luck involved on both sides, but that’s how things work!

Extending this analogy to software testing, you can think of that too as an exercise of blind men learning about an elephant. The testers are the blind men of Indostan, trying to find out if the piece of code they’ve been given is an elephant. Each tester pokes around at a different part of the beast, trying to confirm if it fits what they’re looking for. And if the beast has a knife, a snake, a fan, a wall, a tree and a rope as part of it, it is declared as an elephant!

Speaking of software testing, I came across this brilliant video of a class in Hyderabad where software testing is being taught. Enjoy (HT: V Vinay).

3 thoughts on “Programming assignments and blind men and the elephant”

  1. Another option is to use “crowd-testing” approach. Give the teams a chance to evaluate and ‘break’ the code by other teams. Identifying mistakes and what to do to fix them will fetch extra credits. After all, that is part of spreadsheet modeling work too (You need to figure out other people’s excel models).

Put Comment