No Open Blockers

Grading sucks.

Don’t get me wrong, I love teaching. And I love getting to see how students approach problems. I tend to leave my projects open-ended enough where they can express a bit of their own creativity into the solution. I like having small classes since I can spend a good amount of time offering suggestions on their implementation in addition to their results (my students can vouch for this, I usually provided 2-3 pages worth of comments that were unrelated to a particular grade item).

But coming up with an actual consistent, subjective, quantified number grade is a pain in the ass. I end up with a big checklist of things to look for and I am constantly having to go back and recheck projects to make sure I ran a particular test or to see how I scored a certain shortcoming.

I thought about writing my own test cases in one of the existing test frameworks (such as JUnit) to automate some of the process. The concept is sound, but in the end I was trying to shove the square peg into the circle hole; there would still be a lot of manual intervention to interpret the results and turn it into a grade.

I looked around for a solution. From what I saw, there really isn’t a solid application to facilitate what I wanted. So, I decided to write one.

The idea started out small: Point my application at a bunch of student projects and have it run tests and return me a grade. That would give me a very solid and consistent basis on which to derive the grade, allowing me to spend more time providing manual comments on the code or approach itself. I would just need a pretty flexible API to support the types of tests and introspection I wanted so it could be used to grade multiple projects easily.

Not surprisingly, once I started writing it more and more ideas started to take shape. I realized I could have my application, which I named CodeTurtle (more on that later), generate the report to return to the student as to which tests passed or failed. If I was going to have CodeTurtle generate the report, I needed a way to add my comments in, so comment functionality was written. Since most of my comments were based on a code review, I wanted to be able to view the students’ source files as well and then tie that into the comments subsystem to skip the copy/paste need when commenting on particular code snippets. And so on…

Also not surprisingly, I open sourced it at SourceForge. Ideally, I’d love to get other teachers, both at Villanova and eventually other schools, involved with both usage and development. But I also wanted to give some of my former students an opportunity to get some solid, real-world programming experience before they graduate.

I built the 0.1 version and made it available to download from the SourceForge site for people to start playing around with. Being the first release there’s obviously a ton of more work to do, but I included a sample project to demonstrate the types of student submission validation I’m going after (screenshots available as well). I also included some starting documentation, which will continue to get flushed out as I get the time. I’ve also got the interest of at least one friend of mine, so getting it off my laptop into a public area soon was important.

Feel free to contact me through SourceForge (mailing list or forums) or e-mail me directly if anyone is interested in either learning more about it or wanting to contribute.

Oh, and since I’ve been asked this a dozen times already and don’t have a FAQ up yet, I’ll just toss this out now. The name “CodeTurtle” came from the fact that I really didn’t have a good idea for a name. One night while working on it I looked up from my laptop and asked my 18 month daughter what I should call the project. The newest word at the time was “turtle” which sounded as good as any for a project name. And so, CodeTurtle was born. It’s not the most descriptive name out there, but it’s also not the worst named open source project I’ve ever heard of.