Judge Fatigue by Chris Theis

This weekend at Harvard between the round robin and the tournament I judged a total of 23 debates. I did not have a round off until the semi finals. By the end I was exhausted mentally and physically. I bring this up not to complain, though it was terrible, but rather to bring up a dynamic that I believe is under-appreciated by tournament directors and debaters alike.

Judging rounds can be mentally taxing, especially in more competitive, high-powered debates, i.e. the most important ones.  Over the course of a tournament the judging takes its toll, particular taking into account that most judges do not maintain a healthy lifestyle over the course of the tournament.  As a result my ability to effectively evaluate arguments in quarters was nowhere near my ability to do so round one (luckily those late rounds were not very difficult to resolve). I was not alone. Three times in outrounds alone I heard other judges tell warn debaters of their fatigue before the round, or preface their RFD with that information. Also, we have all heard anecdotes about judges falling asleep during prep, or even worse, during speeches.   This is clearly problematic in certain respects.  When fatigued a judge is more likely to misunderstand, misinterpret, or simply miss, arguments. As the tournament drags on and the stakes go up, judging gets worse and bad decisions become more likely.

I think there are a couple of things worth discussing about this.

First, is there anything that tournaments can do to lessen the extent of the problem? Off the top of my head there are a couple of workable solutions:

a. Tournaments could mandate the number of rounds a single judge could be obligated for. For example, if a team needed to cover 6 rounds the tournament could require that two different judges take those rounds. Tournaments would not have to change the number of rounds required per debater just the maximum amount that could be judged by a single person. This would ensure that every judge had a least a few rounds of rest over the course of a tournament and that there would be more judges obligated to cover elimination debates, meaning all else being equal a single judge fewer of those debates.  The obvious issue here is that it would require teams and tournaments to supply more judges. Given that so many teams have trouble finding qualified judges already, this could be a serious problem. Also, the quality of those additional judges is almost certainly going to be worse than the judges they are replacing. Would you rather have a rested B or an exhausted A? I think there is a fair debate to be had on that point. This bug could be somewhat mitigated by trying to give the most highly preferred judges more rounds off earlier in the tournament to ensure they are rested and able to judge more critical debates.

b. Tournaments could more equitably distribute judging assignments. At every tournament there are judges who judge nearly every debate on hand, and on the other hand there are judges who go almost entirely unused because they are not as preferred. Indeed, some teams even take advantage of this fact by purposely bring judges who will not be preferred so that they can cover their obligation while not having to actually give up coaching time to judge.  If tournaments were more willing to use less preferred judges in more circumstances the judges that are normally worked to death would be able to get a least a few rounds off. Again, the problem with this solution is that there is a trade off with judge quality.  The quality problem can be dealt with by packing those judges into meaningless rounds. This is already done at most tournaments to varying degrees. However, at every tournament there are preferred judges judging debates that do not matter, like Sam Duby’s annual 0-4 double 30 showcase at Greenhill. Tournaments know that some of the judges in those meaningless rounds will have students who break so they will be obligated later on, why not replace them with the judge who will not fit anywhere else?

The second thing that should be discussed is if they are aware of the problem what debaters can do to deal with it. Judge fatigue is something that debaters seem to be absolutely oblivious to. Below are a couple of tips to make life easier for your exhausted judge and raise your chances of winning their ballot:

a. Watch the judge. Are they alert and engaged or do they look like all they want in the world is to crawl back to their hotel room and sleep?  Does is look they did not get much sleep or perhaps had a few too many drinks the night before? Take note. The first step in finding a solution is recognizing that there is a problem. If there is a problem do not stop watching the judge. Make sure they are following you. If they look confused, stop and explain. If they look lost, give them more sign posting or time to catch up.

b. Simplify. Instead of going for 6 off or making 12 responses to the standard, how about going for 1 off and making 3 more developed responses?  A tired is less able to sort through an overload of arguments particularly when there are complicated interactions occurring. If you provide a simpler, easier to understand story it is more likely that the judge will be able to construct a coherent ballot story for you. The more complicated the round becomes, the more random the decision of a fatigued judge becomes. Randomness is the enemy of the good debater.

c. Slow down. Fatigue decreases reaction time meaning a judge who is tired will flow and comprehend arguments more slowly than a fully rested judge. If you absolutely must maintain your breakneck pace, then at the very least, make sure that your arguments do not come in rapid-fire succession. A tired judge will not flow or understand blips delivered at full speed, so make arguments longer and increase the time you pause between arguments.

d. Tell a story. I know what you’re thinking. What is this? 1992? Nope this is still important. Debate is a game of persuasion; judges are simply persuaded by different things. While a dynamic speaker compels some judges, others enjoy a substantive or novel argument. However, no matter who they are judges need to be able to explain to themselves why it is they are voting before they make a decision. So over-explain, give status updates and crystallize. I know its not the cool thing to do but the tired judge is a lazy judge, so make sure the reason to vote for you is clearer, even if it is not better than your opponents reasons.

So, what do you think? Is this just the ravings of an exhausted judge? Or is this a real problem? If there is a problem what should tournaments and debaters do to deal with it?

  • Anonymous

    Solution: Save highly preferred judges for outrounds. Since they will probably have kids in outrounds anyway, might as well. On the other hand, use middle range prefed judges for prelims. I don't think it makes sense to have Chris Theis judge the 1st round of prelims.

  • This is a really, really good point that debaters should be aware of. We'd all like to hope our judges are at 100%, but even well-intentioned judges won't be. Most judges are (optimistically) working on less than 5 hours of sleep, and when a tournaments stretches on after dinner, those judges are going to be exhausted.None of this is going to change any time soon, and debaters are doing themselves a *huge* strategic disservice when they assume a judge will be fully engaged in these kind of scenarios.I appreciate the various sentiments suggesting that judges should be well-rested, sober, etc… but, the reality is that most judges are tired for perfectly good reasons. Do they have a fiduciary responsibility to the tournament? Sure. But, they also have similar (if not greater) obligations to the debaters they coach late into the previous night. And, frankly, no one is getting rich judging tournaments. The pay is good for what you have to do, but the time commitment is still incredibly significant, and most judges have other things justifiably going on in their lives, and those things rarely go on pause for a debate tournament.Regardless, as Chris said… it's a reality debaters must face.

  • As a tournament director and tab room official, I understand what you're saying. However, here is the reality of the situation:Teams and debaters demand (and I don't mean ask nicely… they're downright rude and intolerant about it at times) that tournaments run on time, that the decisions and pairings in the tab room be accurate, and that the best judges are selected for each of the rounds. This is a lot to do in a very short period of time. Add to the mix the fact that many judges will hold ballots for 30, 45, even 60 minutes post round at some tournaments, giving long oral critiques and then agonizing over a decision (or having coffee with their fellow judges and talking about next week's social agenda rather than turning ballots, but I digress), and what you have a very limited window in which to tab a round. Now this article is proposing adding yet another level of complexity to the tabulation equation. While I empathize with your plight, I can't help but think that some, if not all, of this proposal will be rejected out of hand by tab room directors as unnecessary work – that the coaches and judges need to manage their own fatigue levels and it is not the responsibility of the tournament to manage that.But, criticism should never come without a solution. I propose that tournament directors… especially those at large tournaments, designate one or two "Assigners" whose sole job is to manage the pool of judges. Judges could check in and out with the assigners, who would keep track of each team's obligations and how many rounds each judge in the pool has covered. This person/team would be in the judge lounge, away from tab, and would be trusted by tab to assign judges that are not conflicted or less preferred. And at the end, the assigner's desk could report back to tab what teams did or did not meet their judging obligations. Actively managing your judge pool works wonders to improve morale of the judges and keep the tournament running on time. I have done this – had a coach specifically tasked with handing out judge assignments – for the last two tournaments I have tabbed, and it has worked beautifully. At our last meet, only one judge had to work all 6 prelim rounds. The rest got at least one round off.In doing this, by segmenting the judge pairing from the round pairing, you can reduce the stress on tab staff while simultaneously getting a better judge management experience.

  • Jonathan Horowitz

    I do think something that is very easily done and helpful is to guarantee one round off for every judge. I feel like that should be there as a matter of courtesy as well as helping to stop fatigued judges. For more preferred judges making it Round 3 or 4 (of a 6 round tournament) would be a great time as the rounds aren't that important and it keeps them fresh for outrounds.I definitely agree with the debaters part.

  • First, judges need to show up to tournaments sober and rested. If you're covering a judging obligation, that's no less a part of your responsibility than coaching – and it's not fair to the students to show up strung out, hungover (or still drunk), or exhausted. If you can't tolerate an evening without ingesting some substance to reduce your level of consciousness, there are alternatives that don't impact your morning rounds the next day.Second, whether judge fatique is the reason or not, debaters need to make judges' lives easier and the community norms should not punish judges who expect debaters to make their lives easier. Debaters: tag your cards with the claim the card is supposed to warrant, provide the citation, read the card, and repeat – preferably with clear enumeration ("I value life. 1. Life is conceptually and actually prerequisite to achieving any other values, Sciabarra, 1995." or "and… D. Dehumanization is the ultimate impact – worse than nuclear war. Berube, 1997."). If your case consists of that, flowing constructives is easy (flow tags and things we expect to be important out of cards, call for cards if necessary). Our community should not expect judges to catch and flow blippy analytical spikes stuck in the middle of nowhere. Of course you can't focus for 10 rounds in a day if you're expected to do it at a level where the debaters feel justified in complaining because you didn't catch the difference between affect and effect at 500wpm and understand the implications of that difference for strength of link.

    • Oh geez. Yes, PLEASE don't come to tournaments wasted. Party after the tournament. So many judges on the circuit demand respect (from everyone), but won't respect the kids, the coaches, or the tournament officials to show up ready to work and in a proper state of mind and well being.

      • To be clear I am not saying this is ok. I am just saying it is a fact of life that debaters should be aware of.