I just wrote an article criticizing some decisions made by Khan Academy staff, as well as the underlying motives for making those decisions. I explained in detail why the decisions were bad, and how the decisions actually undermined Khan Academy's stated goals, a well as failing to accomplish their intended purposes. I did not offer any solutions to these problems, nor did I make any suggestions on how to improve things. As a companion to that article, I am now writing this, to be published at the same time, just in case Khan Academy wants to improve. Of course, anyone may use these ideas. If Khan Academy staff is not willing to accept constructive criticism, adopt a learning mindset, and be more diligent in their stated goal of providing a free, world class education, perhaps someone else will.
First, to be clear, I really like Khan Academy. I want it to succeed. I would very much like to see Khan Academy improve its mindset and get back to doing what it was made to do than see someone else take up the torch, but if Khan Academy will not take advantage of modern technology to revolutionize learning, I sincerely hope someone will, because we need it.
The main problem I discussed in the previous article was the evaluation system, specifically unit tests. Unit test can be over 30 questions long. The questions are graded, one by one, as soon as they are submitted. The problem is that if you miss a question, there is no way to restart the test. If you exit and reenter, it will just start you right where you left off. There is no reset button. So, if you miss say the second question in a 31 question test (as I did), you cannot pass the test without doing 60 more problems. That is the remaining 29 problems and then the 31 problems in your next attempt. I am not going to recap all of the ways this is horrifically wrong and detrimental both to students and to Khan Academy's goal of providing a world class education, but it is really bad design. I have come up with several solutions to this problem, from the easiest, to the one that requires the smallest changes in pedagogical dogma, to the one that will provide the best learning experience.
Obviously the easiest solution is to just add a reset button that lets students start over, without having to complete the test. And to be clear, this is already better than the current situation, regardless of your pedagogical dogma, because most people quickly discover that they can just enter random (wrong) answers for the rest of the questions to get through fast. This is a waste of their time, which they could be using to study the topic they missed, and even if they do not, skipping the rest of the test faster is not making things any worse for them, though it does interfere with Khan Academy's ability to keep track of where students need more work. The fact is, there is no way adding a reset button can make things worse for anyone, and for many students it will improve their learning.
Perhaps the best solution that maintains the pedagogical dogma as much as possible, would merely be to leave things as they are by default, but giving students and teachers options they can change in the settings to change the behavior. By default, there is no way to restart and students will just be forced to put in random answers to get through and start over. If this is what Khan Academy's pedagogical perspective prefers, then go ahead and make it the default. However, also add an option so students or their coaches can have a reset option. This will allow teachers to control the behavior of the platform for their students, and it will allow adults working independently to chose what works best for them, if they become frustrated enough to go looking for a solution.
The best solution, however, is to lose the dogma and make students restart when they fail a problem! Perhaps this sounds a little extreme, but allow me to explain. Part of the whole testing thing is to figure out where students need more work. The way tests currently work, they force students to complete them before trying again, allowing the platform to determine all of the places they need work, all in one shot. This is unnecessary though. We can do this far more dynamically, in a way that is also more natural. This will result not only in less frustration but also in better learning, because the best time to learn is right after failure has primed your brain to want to learn. (Actually, sometimes the best solution for repeated failure is taking a nap, so your brain can have some time to build new connections, but this is not about repeated failure.) So, imagine you start the test. You get a few problems in, and you make a mistake. Instead of forcing you to go on, the test immediately terminates and presents you with the suggestion that now is a good time to review the video or do a practice set. We do not care about the rest of the topics right now. We can deal with them later. The student may then ignore the recommendation and start a new test, or the student can heed the recommendation, practice the subject again or watch the video, and then test again. Learning coaches like teachers or parents should probably have an option to require additional practice instead of making it optional, but this probably should not be the default. This will encourage students to approach failure by putting some more time into learning, before trying again. It will remove the barrier to effective learning, by reducing the repetition on subjects students are already proficient at and by reducing the frustration of having to waste time on problems they already know, before they are allowed to test again.
That solution could still work reasonably well watered down a bit. What if instead, students are allowed to complete the test, however the act of practicing will reset a pending test. In this scenario, when I miss a problem, the test gives me the option to continue or quit. If I chose to quit, the test will resume where I left off if I get back into it, but, if I practice all of the topics I missed questions on, the test will automatically reset (or at least give me the option to reset). This is not as good for people who have made mistakes that are not their fault (for example, I ended up having to work through 60 problems, because my wireless keyboard missed a keystroke), but it would still be better than the current situation (I would have been able to restart after 4 practice problems instead of 29 test problems).
The ideal solution has one thing the others do not: They encourage students to strive for perfection. Without a reset button, students are encourage to give up, once they get a passing grade, because trying to get a higher grade results in significant amounts of punishment, frustration, and misery. A reset button would largely eliminate that discouragement, making perfection more accessible but not actually encouraging it. Encouraging students to go back and study as soon as they miss a problem will encourage them to stop testing as soon as a weakness is identified, to fix that weakness. It will provide some motivation for students to fix gaps in their skill immediately, instead of rewarding them for aiming for "good enough". My personal opinion is that no education program has any excuse for graduating students without high proficiency, given our current technology and teaching capacity. There is no reason that Khan Academy could not just keep teaching students until they get 100%. In public schools, this is impossible, because you have one teacher and many students, and the teacher just does not have the time to strive for perfect proficiency. Khan Academy is scalable though. It can afford to at least encourage students to strive for perfection. 75% does not need to be good enough anymore, because if one student needs a few more hours of practice than another, Khan Academy does not have to split its time between that student and others. Every student who is struggling can go back, review the lecture, and practice, and it will not cut into lesson time for anyone else. But, we cannot do this with the current system, because it harshly punishes students for failure. The only way teaching effectiveness can be maximized is by eliminating punishment as much as possible and providing a positive response to failure as immediately as possible. Ideally that means building resetting right into the learning flow, so that when students fail they are immediately given the opportunity to improve and allowed to restart evaluation as soon as they have taken the opportunity to improve.
Khan Academy is already pretty good, but as I have said before, the very fact that the team has a pedagogical perspective suggests they are locked into a dogma that cannot provide the flexibility required to produce a good, evidence based (or world class) learning system. Above are some good options for improving some specific features, within whatever pedagogical dogma the team may subscribe to, but the better long term solution would be to leave the pedagogy and dogma behind and instead take a more scientific, evidence based approach to learning. Evidence based learning is not well suited to pedagogy in general, because it needs more flexibility than strictly defined pedagogy can provide or allow, to rapidly adapt to new discoveries in education research. Pedagogy is inherently dogmatic, and dogma has no place in evidence based anything.
I hope these ideas will find some good use in helping Khan Academy become what it desires to be, but if they do not, I hope someone eventually uses them to produce an electronic learning system that is truly world class and ideally free.
No comments:
Post a Comment