Question

A dumb person's idea of a smart question

When I was a student at the Command and General Staff Officer Course, I had to participate in a program known as Project ATHENA. According to the program’s website:

“Project Athena is an Army leader development program designed to inform and motivate Soldiers to embrace personal and professional development. Adding to the Army’s culture of assessments, Project Athena uses batteries of assessments to increase a Soldier’s self-awareness of leadership skills and behaviors, cognitive abilities, and personal traits and attributes. Assessment batteries are strategically selected to compliment the leadership skills being developed at a number of Army schools.”

There are so many problems with this program that it’s hard to know where to begin. But this essay isn’t focused on the program as a whole. There is one question on one of the assessments that, to me, essentially, invalidates the entire program.

One of the questions on one of the Project ATHENA tests was a restated version of the classic “Monty Hall Problem.”

1. Monty Hall (from the old TV Show Let’s Make a Deal) shows you three doors.

2. Behind one door is a new car, and behind the other two doors are goats.

3. Monty asks you to pick a door. Let’s say you pick door 1.

4. Monty then opens up one of the other two doors, let’s say he opens up door 2, and shows you a goat.

5. Monty then asks you if you want to stay with door 1 or if you want to switch to door 3.

6. The correct answer, mathematically speaking, is that you should switch doors. Let’s not get into the math, you can google it if you want. The point is, when faced with a pure Monty Hall problem, the answer is to switch doors.

The restated version for the ATHENA test substituted the doors for patrol routes and the goats with enemy artillery fire. So it went something like this, “You are a patrol leader and you have the choice of three routes. You choose Route 1, but right before you leave for the patrol, Route 3 receives a huge barrage of enemy artillery fire. Should you stay on Route 1 or switch to Route 2?

I wanted to throw the damn computer across the room when I read that question because I was so furious. I knew the answer to the Monty Hall Problem so I knew that the answer they wanted was for me to switch routes. But the fact that they posed the Monty Hall Problem in that way demonstrates to me that whoever wrote the question has no business assessing the skills of military leaders or anyone for that matter.

Deriving the “answer” to the Monty Hall problem without any prior knowledge requires a pretty firm grasp of probability theory. When the problem was first posed, professional mathematicians thought that it was ridiculous to say that switching your answer makes any difference. You can watch this video about it. But to save you hours agonizing over this problem, let me give you the easiest explanation that I learned from Naval Ravikant.

Rather than three doors, imagine Monty Hall shows you 1 million doors and asks you to pick one. You choose door #1. Monty then opens up 999,998 doors and shows that they all have goats behind them, leaving only door #1 (the door you chose) and another door (let’s call it door #117,856). He then asks you if you want to switch. What are the chances that you picked the right door out of 1 million? You didn’t. The car is obviously behind door #117,856.

So, why did the restated question on the Project ATHENA assessment anger me so much?

I have planned and executed hundreds of patrols in training and in combat and, to me, route selection is not an academic exercise based on probability theory. When it comes to route selection there are dozens, if not hundreds, of variables at play. Here are a few:

Is this a mounted patrol, dismounted patrol, or mixed? What are the most recent enemy SIGACTS? Where are the likely ambush points? What is the trafficability of the routes? Who else is taking these routes? Should the route facilitate speed or security? Am I going over hills? Do I need to cross bridges? Are there significant choke points? Am I close to another unit’s boundary? Which route is more likely to be under enemy observation? Where are the air corridors in relation to these routes? What assets are available to support me if I get into contact?

And there are other factors as well, these are just a few. Many other factors are probably known unconsciously from doing this kind of thing pretty often.

So when you reduce my route selection process to a pure matter of probability theory I am going to get a little agitated.

Route selection is not a Monty Hall Problem. For one thing, the enemy doesn’t just target routes with artillery; he can target anything in range at any time. He can also target all of the damn routes at the same time if he wants! In the Monty Hall Problem, there’s a new car behind one of the doors for sure. In combat, the enemy can reign down rocket and artillery fire wherever he can range it. There’s not one guaranteed route! The fact that he hit one route with artillery has zero bearing on the probability of other routes being hit. In fact, the first question I would ask is, “did our counter-fire radar acquire a like point of origin and did we fire a counter-battery mission?” The fact that the enemy just fired artillery means that he is either being hit with counter-fire or he is displacing and can’t shoot. Better to get moving on the planned route right away than try to re-plan and re-issue march orders.

Obviously, I am speculating here, but I can’t help but think the question’s designer was in some way influenced by Kevin Spacey’s character in the movie 21.

I imagine the question designer speaking to a panel of General Officers, explaining to them the purpose of the assessment, and how this particular question supports that purpose.

“You see, General, this question will help us identify officers that are driven by paranoia, fear, and emotions, and identify officers who can keep emotions aside and make reasonable judgments based on analysis.”

But of course, this question does no such thing. Almost no student who shows up at a place like The Command and General Staff Officer Course will have enough math background to be able to derive the answer to the Monty Hall Problem on the spot. You either know the answer, as I did, or you don’t, or you guess correctly. The ability to answer the question tells you absolutely nothing about the person being tested. Maybe they are dumb as a box of rocks and just watched 21, or maybe they are super smart and thought, “well if I picked a route (based on all the variables above) I see no reason why enemy artillery on another route would affect my choice.” Which, of course, is the most reasonable answer. They might as well have asked the tested student to list Pi out to seven digits. In fact, that is quite a good parallel because you either have that memorized or you don’t; no one is going to derive it on the spot.

[Game show voice] But Wait! There’s More!

The Monty Hall Problem itself is completely useless other than as a silly math brain teaser. In his book Risk Savvy, Psychologist Gerd Gigerenzer explains why understanding the Monty Hall Problem (and others like them) doesn’t help us make decisions in the Real World.

[game show voice again] Tune in next week to learn more!