Open Ended Assignments

A few weeks ago, I got an email from a student complaining about how I structure assignments.  He was bothered by the fact that I require some decision making that is beyond the scope of the assignments.  He complained that he was spending more time trying to decide what to do for the assignments than he was actually doing them.  Now, I want to be clear: This student is very intelligent.  I teach upper division undergrad Computer Science courses.  I don't get students that are not very intelligent, because they would not have passed the prerequisites otherwise.  Sometimes I get students that struggle with creativity though.

Why do highly intelligent students struggle with creativity?  How can I get students in an upper division Computer Science course, which is part of a field where intelligence and creativity are both  critical, who struggle with the creativity element?  As with most problems I have discussed, the problem is not with the students.  The problem is with our education system.  We do a poor job of teaching creativity.  Yes, many schools are trying to focus on creativity more now.  They are adding art courses, music classes are starting to come back in some places, and there is more emphasis on artistic expression in things like English.  Creativity does not mean art though.  Computers can make art out of randomness or through complex algorithms, and computers are literally incapable of creativity.  They can make art that appears to be creative, but the creativity is artificial.  "Creative" classes in public school tend to be the same way.  Music classes are all about singing.  Singing is not terribly creative (it can be, but you won't find that below the college level outside of private lessons).  Kindergarten art tends to be creative, but it also tends to mostly be scribbling.  Much past kindergarten, art classes are generally highly structured, squeezing out creativity.  There are some reasons for this.  Gaining skill takes practice, and purely creative practice tends to miss important skills.  Structured assignments have value in improving skill, but when it is almost all structured assignments, creativity is lost.  Despite many attempts public schools do a poor job of encouraging creativity.  They prefer to focus heavily on technical skills and memorization over creativity.  And frankly, colleges are not much better.  There is more creativity in liberal arts courses, but when you get to more technical courses, there are a lot of structured assignments with very little creative flexibility.  Being required to think for themselves is something students are generally not used to, because creativity is marginalized in most of our education system.

Why do I encourage creativity?  In Computer Science, creativity is a critical skill.  CS is all about advanced problem solving.  Creativity is essential to effective problem solving.  Now, most of my students have decent basic problem solving skills.  Given a structured programming assignment, they can figure out how to solve the problem presented.  This is a great start.  So what happens when a boss says, "Hey, someone suggested using this new piece of software.  Can you experiment and compare it to what we are currently using?"  The first thing the employee has to do is devise an application that the software can be tested against.  If the software is a programming language, this is literal.  The employee will probably have to write a program twice, once for each language, and it is going to have to highlight the most important aspects of what the company will use the software for.  This is precisely the kind of creativity my open ended assignments require.  A more likely scenario is that an employee is put on the design team.  Solving a programming problem when you have a design document in front of you is totally different from creating the design document in the first place.  This kind of creativity cannot be learned from highly structured assignments.  It can only be learned with a potentially uncomfortable level of flexibility in assignments.

Here are some examples of assignments I have given:

• Create a simple multi-threaded program in Python.  Keep track of issues you run into.
• Create a multi-threaded C++11 program that uses mutual exclusion.
• Write a program that demonstrates that you understand [some assembly language principle].

You don't need to understand what these mean to see that they are fairly open ended.  Compare this to assignments in other CS  classes, where students are provided with a design document or are even given specific input/output requirements.  Little creativity goes into writing a program where the inputs it should expect and the outputs required for each input are handed to you.  Sometimes there is reasonable value in doing it this way, but I find that more often it is better for everyone to tell students what you want to see them demonstrate and let them figure out how.

A major part of the reason I use open ended assignments so heavily is that I cannot design good assignments.  I am not suggesting I have a problem or am inferior to other teachers.  It is just a fact that teachers cannot design good assignments for 10 to 30 students.  There is no good assignment for that many students.  Public school curriculum tends to shoot for the lowest reasonable expectations.  This strategy allows all students with a reasonable understanding to get a good grade, but it cheats the more skilled and more advanced students out of additional opportunities for growth.  In addition to that, I can only design assignments that fit my experience, and I am prone to designing assignments that fit my own interests, because that is what I know the most about.  In short, I will end up training my students poorly in topics that are of personal interest if I design assignments completely on my own, and they will miss out on things that might interest them and that will likely be more relevant to them in the long run.  So instead, I leave the assignments open ended, so that my students can pick topics that interest them and are relevant to them.

How well does this work out?  Honestly, I think it works out great.  One fear of open ended assignments is that students will be lazy, doing the minimum possible, and learning the minimum possible.  This is a valid fear, but the evidence suggests it is not worth worrying about.  Those three assignments I listed above have resulted in some excellent work from students.  The first one could be done trivially.  A student who pays attention in class could complete the assignment in less than 10 minutes.  And I would give the student full credit, because it fulfills the requirements.  This semester not a single student did this!  Out of 16 students, not one took the easy path.  Most wrote programs designed to highlight the issues they ran into, which made for a very valuable discussion when they presented their work.  The second one they went even further!  Again, not a single student took the easy path.  Some students struggled.  They could have given up and done it the easy way, but they did not.  In my assembly language course, I have seen a broad range of strategies.  Some students divide the work into a lot of tiny programs, each demonstrating one thing.  Others write larger, more complex programs covering a range of things in one shot.  Some students take the easy route first, then they use the time they saved to write larger, more complex programs that don't technically pass off  anything, because they want to.  And honestly, it does not matter terribly if they do take the easy route, because most of the assignments are only intended to demonstrate one proficiency and then give students the opportunity to learn.  The place I expect them to show full ability is in the final project.  I love that most of my students are motivated to take advantage of the additional learning opportunities open ended assignments provide, but even if they don't, they are still proving that they can do the things I am teaching them to do.

One of the most valuable parts of open ended assignments is motivation.  If I give students a strictly defined assignment, they don't own it.  It was not their idea.  Odds are they are not terribly interested in the topic.  Their commitment to the assignment is only implicit in the fact that they chose to take the course.  When they design their own assignments around a small set of requirements though, it is their assignment.  In designing it, they are committing to it.  They own it.  Multiple times this semester I have had students present their work, when it is complete enough to get credit, but there are still things they don't like about it.  They go home after class and keep working on it, because they are not satisfied with their work.  They have already gotten full credit, but they are committed to the work, and they own the assignment, so they care that it is done right.  A few days later, I get an email, or they pull me aside in class, to show me what they have accomplished.  These students are not just designing the minimal assignment, and they are not just stopping once they have the grade.  They are choosing to work hard when they don't have to, because I am allowing and encouraging them create the assignments they want to do, instead of making them do assignments that I like.  Not only is this producing better work, but my students are also learning more, gaining more experience, and gaining a love for the topics of my courses.  They are also learning to learn on their own, as many choose to go beyond what I am teaching them.  Open ended assignments allow and encourage students to do work that they are motivated to do.  This results in better work overall, superior learning, and it allows more advanced students to do more challenging work and less advanced students to do work more consistent with their level of experience.

Open ended assignments can result in more grading work.  When all assignments are identical, it is easy to produce a rubric that exhaustively defines how various elements of the assignment will affect the grade.  It would take an absurdly complex rubric to cover all possibilities for an open ended assignment.  Instead, I use the proficiency based method I have discussed in the past.  Instead of expecting a perfect program from students, I distill the requirements down to only the proficiencies I want the students to demonstrate in that particular assignment.  Each assignment has one, two, or maybe three bullet points.  If a student can demonstrate that all of those bullet points have been satisfied, the assignment is given full credit.  Otherwise it gets no credit.  Multiple attempts at demonstrating proficiency are allowed, within reason, and each failure receives feedback.  This is how we learn in real life.  This system eliminates the burden of carefully reviewing every assignment for every students to count up a score.  I makes it much easier to deal with open ended assignments.  This semester, I am trying something new as well.  Instead of turning in work for me to grade, and instead of meeting with me one-on-one to demonstrate work, students present their work to the class.  This allows me to easily grade work during class time, and it also provides discussion material and examples to other students.  If a student makes a common mistake or uses a bad practice, I can (politely and without judgement) point it out to the class and explain how it might be improved upon.  This allows all students to learn from each student's mistakes.  It also gives students the opportunity to provide other students with feedback, and it allows less advanced students to learn from the work of more advanced students.  In addition to all of this, it reduces the grading burden, by putting most of the grading in class.  All of this makes open ended assignments have a far more reasonable grading burden, despite the additional complexity it adds.  (It also happens to give students some experience with code reviews, which are commonly used in larger companies in industry, as a quality assurance strategy.  In artistic courses, it would teach students to give and take constructive criticism better.)

Open ended assignments help to teach creativity by providing more opportunity and encouragement to be creative.  Many students will likely struggle with it initially, because they are used to being spoon fed assignments, without any need to exercise creativity.  Creativity cannot be taught using normal teaching techniques.  It is developed through practice.  Open ended assignments provide this practice.

Ineffective Teaching Methods

Earlier today, I decided two write two articles.  One was on mandatory group work, and the other was on controlling the pace of learning.  Now I have a third topic, which is withholding feedback.  So, I decided to just write a single article on commonly used ineffective teaching methods.

Mandatory Group Work

I was once a student, and I was required to work in groups in some of my classes.  I am something of an introvert, I am a very fast learner, and thus far, I have taken only a few classes where I did not already have a strong background in the subject being taught.  (Welcome to an education system where the paper matters more than the actual learning.)  In a few classes, the mandatory group work was legitimately necessary due to the subject matter.  In most, however, it was an artificial construct based on the misapplication of school guidelines.  The college I graduated from and now teach at has formal guidelines teachers are very highly encouraged to follow.  One of those guidelines is that students should teach each other.  This is based on very solid research indicating that when one person teaches something to another, the teacher will learn the subject better as well as the learner.  In other words, when students teach each other, everyone benefits.  In addition, this leaves more time for professors to spend helping students that need help specifically from the professor.  In other words, the school's policy is backed by solid science.

Most classes I have taken with mandatory group work did not actually need the group work to be effective.  In all, I recall only one where the group work was actually beneficial to me, and that was partially because there was more work involved than was reasonable for a single person to do, and the other person was a fast enough learner to catch up with me (the other person was a woman; I have not had a similar experience with male students).  There were maybe two other instances where working as a group helped the other members of the group, but, ironically, I knew the material so well that teaching them did not actually benefit me significantly.  All of the other instances of mandatory group work I was required to do for my college classes actually made things more difficult for everyone.  In one case, where we got to choose our own topics, the other members of my group thought they understood the topic I suggested, and in the end, because they did not, one member of the group left in a fit a rage, because myself and the third member would not let him present incorrect information in the group presentation.  The group ultimately fell apart, because neither of the other members understood the topic that I found interesting, but they did not realize that until it was too late.  I could easily have gone solo on both the research paper and the presentation, but that was not allowed, due to a misapplication of the school's teaching policy.

The big problem with mandatory group work is that it is almost impossible to group people in a way that works well for everyone.  Some people just work better on their own (I am one of these people).  Some people need to work with people who have complimentary personalities.  Some people work best when they are ahead of others in the group, and some work better with those who are ahead of them.  And of course, some people work better with those close to the same level as them.  This does not even account for differences in personality or those students who are lazy and freeload off of the rest of the group.  The fact is, for most topics, group work is not the best arrangement for everyone.  Yes, I understand that most work in the real world is group work, and I even point this out to my students when I encourage them to work in groups.  Don't forget though, school is not the real world workplace, and it is not appropriate to treat it like it is.  Students recognize this, which is why many complain about mandatory group work.  When group work interferes with effective learning, it is a bad idea, no matter how common group work is in the workplace.  School is not the place to learn to get along with others.  To be totally frank, I think the workplace is a better place to learn this skill, for those who have not already learned it.

There are some places where group work is valuable or even necessary.  Certain types of creative design benefit significantly from collaboration.  I teach video game design, which is one of these types of creative design.  The value of collaboration is so high that collaboration is an essential skill for good video game design.  Even then though, I don't require group work.  I strongly encourage it, and I require students to present their ideas and listen to feedback, even if they are not part of a group.  This is not always a good solution, but it is a way of getting some of the benefits of groups without having formal groups.  Another case I have seen where group work is appropriate is when the class size is too big to be reasonable to grade students individually for all of the work.  Ideally, this should be a temporary situation, where additional sections will be added to distribute some of the grading burden.  The last place where group work is appropriate is when a project is necessarily too large for a single student (interestingly, this is the reason group work is necessary in the real world as well).

The takeaway here is that group work is not necessary for students to teach each other, and it is often a hindrance to effective learning.  When considering requiring students to work in groups, serious thought should be given to the value of doing this, as the cost to students is almost certainly going to be high.  There are places where it is appropriate, but there are perhaps more where it is not.  My personal solution is to encourage students to self organize into groups but allow them to work solo if they prefer.  I find this works better on two levels.  One is that when students self organize, they are more likely to form groups that work better.  The other is that some of the students who work solo do so because they learn so much faster than everyone else.  The benefit to this is that no one is holding them back, and they are not pulling anyone else in deeper than they can handle.  I also find that these solo students are more likely to and more capable of helping people in other groups.  Mandatory group work not only hinders effective learning, it also tends to hinder the ability of students to teach each other, because they feel partitioned.

Pace of Learning

It was recently suggested to me that allowing students to work at their own pace hinders their ability to teach each other.  Unfortunately, this is an attitude I have seen a lot.  Most classes I have taken or otherwise been involved with limit the dispensation of knowledge, in the attempt to enforce  a particular pace of learning.  The theory is, if everyone is at the same place, they will be more able to help each other.  To me, this is totally counter-intuitive.  It does not make any sense to me that a collection of people who all have the same level of knowledge would be able to effectively teach each other, because if they all know the same stuff, what is there to teach?  There are, of course, some teaching opportunities, as some students catch on to the subject at hand faster and are able to help those who catch on slower, but in a traditional setting, there is not much time for this, because the class just keeps progressing.  If a fast student for a topic spends time teaching a slower one, they both get behind on whatever is taught next.

In my experience, a collection of students where some are further ahead than others is a good thing, because those who are ahead are better equipped to help those who need it.  In fact, before learning was industrialized into an assembly line system (and let me point out that it got significantly worse when this happened), heterogeneous classrooms were the norm, and it was not unusual for more advanced (and often older) students to help less advanced ones.  In my mind, this is what teaching each other in a learning environment is all about, and as I mentioned, my experience supports this.

Artificially controlling the pace of learning comes with a plethora of other problems.  It destroys motivation.  It is hard to stay motivated to learn something when it is dispensed at a slow rate over a long period of time.  The best time to teach someone something is when they want to learn it, and that is generally right after they have learned what comes directly before it.  Some people can handle the "weekly 1 hour episode" model, but the reason things like Netflix and Hulu are becoming so much more popular than regular television is that people can binge watch.  More and more data is showing that people prefer to watch an entire season of a show, one episode right after another, over the course of a few days to a week, and many people have cited this specifically as a reason for dumping cable TV in favor of these online media providers.  This is a form of learning, though not as useful, as the learning is almost exclusively about fictional people in varying degrees of fictional settings.  Imagine what would happen if we just let students binge learn.  Interestingly, it turns out that natural learning favors binge learning over regulated learning.  It is common knowledge where I teach that motivated students learn more between semesters than they do during semesters from their classes.

Regulating the pace of learning destroys motivation to learn, it makes learning frustrating, and the end result of that is often that students dislike or even hate learning.  Further though, not only is regulating the pace of learning unnecessary, it is actually harmful to the goal of students teaching each other.  In both of the courses I teach, students teaching each other happens the most and most effectively when students are not at the same place in the course content.  Students who are significantly ahead are almost always better teachers than those who are about at the same level as the students needing help.  Allowing students to work at their own pace allows them to stay motivated and help each other more effectively.

Withholding Feedback

I hope this sounds like it is as bad of an idea as I think it is.  Withholding feedback from students is a terrible idea.  A great way to make sure students cannot learn from their mistakes is to avoid telling them what those mistakes were.  If the goal is to teach students, this is a horrible strategy.  Just today, I had a discussion with a student on this very topic.  She told me that her teacher would not provide test results aside from the final grade of the test.  This meant that she would have no idea what questions she got wrong, which further means that she is unable to study and improve her understanding of those things that she is weak on.  This is horribly stupid for a course that is supposed to teach students.  Now, there is a justification for this.  The justification is that this helps to prevent cheating.  If test results were provided to the students, they could forward those results to other students to help them cheat on the test.  In my opinion, the fact that this is even a concern is a major flaw in the assessment setup for the course.  If the course design makes it seem like a good idea to withhold feedback, then the course is poorly designed, and it should not be taught until it is fixed.

When I first took physics, my professor had an excellent policy (which I have mentioned before).  If a student scored higher on the comprehensive final than the composite grade for the course, then the grade for the final would be the course grade.  Otherwise the composite grade of all assignments would be used.  I was the first, and thus far only, student to ever benefit from this policy.  The reason I was able to do this, despite years and hundreds of students failing to score better on their final, was that I took advantage of feedback on my tests.  When I took the tests, I generally got a good idea of where I was weak, and looking at the test results afterwards (results being what I got right and wrong, not just the grade for the test) gave me an even better feel for where I was weak.  Then, I studied those things.  By the end of the semester, I had significantly strengthened my understanding in the things I had initially been weak in.  The result was that, with a composite grade of B in the course, I was able to earn a solid A on the final, which was my grade for the course.  According to a brother who took the same course from the same professor many years later, he now tells students that getting a higher grade on the final has only ever been done once, instead of never.  Without good feedback, I could not have done that.  Without being able to see where I made mistakes on the tests, I would have gotten a B in the course, because I would not have had the opportunity to learn the material as well as I did.  Withholding feedback in the way that was described to me by the student mentioned earlier hinders effective learning!

Good feedback is essential to effective learning.  Yes, I understand that teachers have to worry about cheating and making sure that assessments are effective and accurate.  If this requires withholding feedback though, there is something wrong.  Frankly, there are plenty of better alternatives.  The easiest is to have a large pool of questions from which test questions are randomly chosen.  This is a good enough strategy for the Ham radio license tests, which have a few hundred questions, where the questions and the answers are public knowledge, because trying to memorize 200 answers for a 30 question test is more work than actually learning the material.  Another is to just come up with unique questions each time the class is taught (this is actually a good strategy for building a question pool to use once there are enough).  Some kinds of questions are just especially hard to cheat, for example any kind of essay question.  For math questions, adding a variable that is randomly changed each semester is an effective way of generating unique questions at very low expense.  Alternatively, requiring students to show their work or describe the process they are using can be effective for math and other logical subjects.  Personally, I prefer to eliminate tests entirely, and instead grade based on specific proficiencies as well as a demonstration of application.  This won't work with every subject, but it will work for a lot.

Courses and assessments should be designed with feedback in mind.  Even the final for a class that has one should be made so that it won't compromise security to provide students with the full results, once it is over.  Students that fail may choose to retake the class, and knowing where they need to focus their study may be very valuable to them.  If good feedback cannot be provided for a class or assessment without compromising the security of the course, then it is poorly designed and should not be used at all.  The ability to provide feedback should be a quality metric in course design and teaching.  If we cannot provide students with the knowledge required for them to learn and improve, then we have no business telling them that we can teach them.

Proficiency Based Grading

Sometimes also called Competency Based Grading, Proficiency Based Grading strives to grade students exclusively on how proficient they are in the subject.  I started teaching college level computer science courses in Fall 2015.  I designed my own courses using many of the techniques I have written about in the past.  With over a year of experience (a total of 5 semester long classes taught), I now want to discuss how I taught and graded, the outcomes of my classes, and what I have learned.

From the beginning, I was determined to use a proficiency based grading approach.  My first course was Video Game Design.  The course focuses on the design aspect of games, but it also must spend some time on the development aspect of video games, since the students are expected to actually produce one.  I began the course design by enumerating topics I expected students to learn.  The course is an introductory course, so the topics were broad, and I designed the course to provide and assess exposure rather than deep understanding.  Each topic could easily be a course by itself.  The list was composed of five broad topics, one on general game design (including tabletop games), a couple on video game design and development, and a couple on the value and creation of games beyond mere entertainment.  Each of these five topics had two to four subtopics.  These subtopics are what I expect students to know about.  Of course, any video game course should also include actually making a video game, for the students to show that they can apply the knowledge.  I also included a simple table top game project, for learning basic game mechanics design.  The first iteration of the course required students to write a short essay or give a presentation on each of the various topics, including some research.  In addition to this, each student was expected to design and develop a video game, and groups of up to 4 students was encouraged.  About a third of the class time was spent on lectures, another third was spent on ungraded in-class group assignments, and the last third was workdays for students to work on projects.  This worked fairly well, but neither myself nor the students liked the time spent on lectures much.

The next few semesters saw significant changes to this course.  I reduced the number of lectures, and then I removed them from the classroom entirely.  I wrote a series of articles, an informal textbook of sorts, and posted them on my technical blog.  Then I provided students with links to the articles and a recommended reading schedule.  This freed up an enormous amount of class time, which I filled with in-class coding demonstrations.  This became most of the development aspect of the course.  Near the end of the first semester doing this, I discussed it with some of my students, and they told me it really helped them.  The coding demonstrations were unscripted, which means that I made mistakes and had to find and fix them as the students watched.  This helped them learn two incredibly valuable lessons.  One is that even seasoned programmers make mistakes.  The other was the processes used by seasoned programmers to fix mistakes.  In the typical scripted classroom style, students don't see these things, and it makes them underestimate their abilities and struggle with one of the most common activities in software development (debugging).  My students expressed their appreciation for exposing them to a real software development process.

The current state of the class, as taught last semester, still needs some work, but it has made some wonderful progress.  Instead of assigning point values to each individual assignment, I assign grade percentages to the major topics.  The application part of the assessment has been broken up into its own five sections.  The grading rubric is divided into two parts: knowledge and application.  Each part has five sections, and each section represents one proficiency and 10% of the overall grade for the course.  The five knowledge sections each have the same subtopics they had in the original course design.  The two sections that have four subtopics only require three to be checked off to get credit (exposure, not depth).  The application section is largely composed of things students should be able to demonstrate in the two projects.  This fits on one sheet of paper, and each student is given one when the semester starts.  This way, students know exactly what they need to learn up front, and with each of the topics discussed in my technical blog, the resources are all readily available.  Since the knowledge proficiencies are supposed to be broad and shallow, I encourage students to pass off multiples at a time.  Students may choose to pass them off in essays, with up to three topics (which do not have to be from the same sections), or in presentations with up to two topics.  The rule here is, one paragraph discussing each topic and one discussing how they are all related.  For presentations, students should fit them within 10 to 15 minutes, discuss each subject, and discuss their relationship.  I have mentioned cost of failure as having a major influence in quality of learning.  To reduce cost of failure, the only formal due date is a week before the end of the semester, to give me time for grading, and if a student submits an essay that does not sufficiently prove an understanding of something, I give the student some feedback and ask the student to answer some specific questions to prove that proficiency.  There is no permanent failure in the course until final grades are posted.  This means the grades students have at the end of the semester reflect all of the proficiency they have demonstrated to me up to that point.  Some students do struggle with things earlier in the course, but if they figure them out by the end, and show me that they have, then their grade reflects that they understand those things.

As I said, some work is still needed.  This last semester, I asked my students for their feedback.  Several told me that they would have learned better if we had discussed some of the reading (originally the in-class lectures) in class.  One student said he would have liked more in-class group activities, which I had reduced significantly that semester.  So, next time around, I am going to have a more solid reading schedule, and we are going to start some classes with discussions on the readings.  I am also going add back in some of the in-class group activities.

Quality course design requires a level of balance that can be difficult to achieve, and when I first designed this course, I did it with that in mind.  In my experience, this is not something a lot of teachers do, which may be part of the problem with our education system.  Of course, in my case, an essential element of getting the feedback I need to improve my course is a good relationship with my students.  The college I teach at has had student evaluations for my courses pretty much every semester, but I don't get to see the results if too few students respond.  I have never gotten enough responses to see the results, but that does not bother me, because my relationship with my students helps encourage them to share their opinions with me frankly and in person.  I expect to be making adjustments to this course after every semester I teach it, and I hope it gets better every time.

Last semester I added a new course.  The course is ARM Assembly Programming.  I won't go into details on the subject, but it is essentially programming on the lowest level that is reasonable for humans to do (one step away from machine language).  In some ways it is simpler and in some ways it is more complex than programming in higher level languages.  It is widely regarded as significantly harder than programming in higher level languages.  It turns out that it is very suitable to a proficiency based approach.  As with the game class, I started by enumerating what I expected students to learn from the course.  Instead of using these as proficiency categories though, they were used only to determine what proficiencies should go into the class.  This is the kind of class that, in the tech industry, an employer might look at specifically (for some context, in the tech and engineering industries, employers do look at college transcripts as well as outcomes for specific courses related to their particular work).  I wrote up a list of proficiencies, and then I divided them into lettered sections, A through D.  In the D section, I put proficiencies that an employer might reasonably expect a student who got a D in my class to have.  In the C section, I put proficiencies that an employer might expect for a C grade, and so on.  The proficiency checklist included 10 to 15 proficiencies per section, but most of them were fairly small.  I expected students to mostly write programs that would each check off multiple proficiencies.  From there, I had to determine what learning materials to provide my students with.  Again, I looked at how things are done in industry.  When an employee is asked to learn something new in this industry, it does not generally come with a manual or some nice little course.  A manager says, "I need you to learn this language," and the employee is expected to figure out how to do it.  So, I encouraged my students to find online resources on their own.  I did provide a few myself though, including a nice tutorial that covered most of the D and C proficiency sections, and some reference materials.  I also provided data sheets, like what might be provided on the off chance that an employer did provide anything.  Students with questions were encouraged to ask Google first, and then ask their class mates.  Only if these had failed, were they encouraged to ask me (in practice, I did not turn down requests for help though).  The last part of the grading was a final project, which would hopefully be of some value as part of a portfolio.

I have only taught this course once so far, but it went very well.  I gave a few lectures in class.  The only things I spoon fed students were things that were required for their work but which were not integral parts of the topic of the course.  I also did live coding demos, as my experience with the other class taught me that those are very valuable.  Most classes started with a QA session, where students asked me questions, and we discussed the answers.  Around half of the class periods for the course were workdays, since students were encouraged to learn primarily on their own in groups, instead of listening to me talk a lot.  During workdays, students would also pass off proficiencies with me.  I would sit down with them, with my master sheet with the proficiencies for the entire class, and they would show me their code, explain what it was doing, and then show me that it actually worked.  I would check them off for any proficiency their code demonstrated.  For a class of 18 students, this worked very well, though as the end of the semester approached, I had to start accepting email submissions due to a lack of time in class.  Working with students one on one during this grading time also allowed me to give feedback and provide some advice and help when students failed to correctly demonstrate a proficiency.  As with the video game class, the cost of failure was low.  If a student failed to demonstrate a proficiency, the only consequence was not getting a check mark on the sheet at that time.  The student could demonstrate it correctly later to get the check, with no penalty aside, perhaps, from lost time.

Using a proficiency checklist, instead of regular formal assignments was a significant benefit to students' ability to learn.  Some students wrote one small program for every proficiency.  Others devised larger assignments for themselves that would cover up to 10 proficiencies all in one program.  Students tended to pick the strategy that worked the best for them.  The ones that wrote one program per proficiency learned best by doing one thing at a time, in small pieces.  The ones that devised larger assignments learned best by applying lots of things at once to a single larger project.  Some students worked together on some things, while others went solo.  All of this allowed students to learn the different things required at the pace that worked best for them, and aside from a few students that got way ahead and a few that struggled a bit more than the rest, everyone had about the same average pace, which frankly was pretty fast as far as the learning went.

The student feedback on this course was also good.  One student who struggled for nearly all of the semester, barely making a passing grade during the last two weeks, told me afterwards that she really liked the design of the course.  She said the low cost of failure reduced the stress that she had from struggling and getting behind, allowing her to continue to function and progress in the course even though she was struggling.  In any other class, she would likely have given up halfway through the semester, because her grade would have already been established as failing.  As it was, she was able to pass the course when another student helped her understand the things she was struggling with, by passing off a large number of proficiencies in the last two weeks of the semester.  When the semester ended, she legitimately had a good enough grasp on the topic to pass the course, and she understood it well enough that an employer might want to hire her in a position that requires it.  Using a traditional grading system, she would have failed the class, even though she was reasonably proficient by the end.  With my grading system, she was able to get the grade that accurately reflected her proficiency at the end of the semester.


I have learned a number of interesting and valuable things from my experience thus far.  One is that low cost of failure is better for encouraging students to learn than high cost of failure.  High cost of failure might motivate them to do the work, but they will only do what is necessary, and if they get behind, they will give up and stop learning.  The outcome of my assembly programming class taught me some things that I did not expect.  Most of my students who passed the class went way beyond the scope of the class.  One student learned to do networking in assembly, which is not trivial even in higher level languages.  Several students learned how to manipulate text colors and position in the terminal, which is not horribly difficult, but which is also not trivial in assembly.  A few wrote their own memory managers, entirely for the challenge, and again, this is not an easy task.  Every student who got an A in the class went way beyond the scope of the class (this was not required for an A either).  I believe the reason for this is that reducing the expectations of the course to a list of proficiencies allowed students to optimize their work, giving them more time to learn things that they found more interesting.  Otherwise stated, the course had absolutely no busy work, which freed up time for other things.  In addition, the low cost of failure reduced stress, which allowed students to feel more comfortable pursuing knowledge beyond the scope of the class.  Lastly, allowing students to select their own final projects (with only a few requirements) let them use their imaginations, and they were able to come up with ideas that required them to learn more to implement.

I noticed an interesting thing in my courses this semester.  The grades in my courses are not well distributed.  In fact, in my experience, traditional course grades tend to take on something of a bell curve.  Often the peak is around C or a low B.  Fewer students get As and Fs.  This does not happen in my courses.  The middle range tends to be fairly empty, while most students get high Bs to As, a few get lower Bs and Cs, very few get Ds, and a few get Fs.  I don't think my classes get more Ds and Fs than normal, but they tend to get more As and Bs, and fewer Cs.  Now, perhaps this could be explained by the classes being too easy, except that I would expect to see fewer Fs and Ds.  In fact, I would expect to see the same bell curve, just shifted upwards.  I don't see that, and I can honestly tell you that my classes are not too easy.  I am an extremely fast learner.  Almost all of the courses I took in college were easy for me (including some that were extremely hard for others).  If anything, my classes are harder than average, because I tend to underestimate how hard things are for others.  One benefit I have is that my classes are not required, so every student taking one of my classes is doing it voluntarily, because he or she wants to take the class and learn the topic.  This combined with my low stress grading system allows motivated students to learn quickly and get good grades.  Students who struggle early but then figure it out are not getting penalized for struggling at first, which probably also increase the grade of motivated students.  Overall, this teaching and grading style seems to provide motivated students with a better learning environment than more traditional styles.

In closing,  I would like to point out a few other interesting things that my experiences support.  First, live demonstrations are good.  Teaching students theory and then expecting them to figure out how to apply it themselves in assessments is a common teaching method, but walking students through practical application a few times is extremely valuable in helping students learn.  Second, when the focus of the class is learning instead of getting the answers right on the test, students actually learn.  A course with 4 tests distributed throughout the semester, where test scores are permanent, only encourages students to study for the tests.  When evaluation is about learning instead of memorizing some things temporarily, students seem to learn far better.  Third, taking the lectures out of the classroom, and putting the homework in the classroom really works.  I got this idea from Salman Khan of Khan Academy, when he worked with an Arizona school, using the Khan Academy videos for homework and doing the actual course work in class.  When I moved the initial learning out of class and the coursework into the classroom, my students progressed much faster, because they were able to get help when they needed it (while doing the work), instead of having to wait for the next class period.

This has been a great experience for me.  I won't be teaching my own course this semester, but I will be working with another professor, so perhaps I will learn some things I might have missed.  This will also give me a bit more experience with courses where students may not be as invested, as his courses are required for the major.  I do plan on teaching more of my own courses in the future though, and I will try to report back on what new things I learn as I do this.