Your argument proves that PhD qualifying exams are crap because it’s a generalised argument against testing, not a specific one against multiple choice tests.

It’s perfectly possible to write good tests, ones with good reliability and validity. If the test is good teaching to the test is not a problem.

> Standardized testing leads to teaching to the test, and the tests need to be easily gradable, so we teach kids how to take tests that represents content in a mannor that is a mile wide but an inch deep.

Not true, the test could be arranged in other ways depending on the goals. I've taken one standardized suite of test, after 12 years of schooling, and the questions were quite deep enough.

The limited number of questions you can cram into six hours per subject meant that the math test could contain only one task in statistics, one in geometry, and so on; with luck you could do well while lacking in some areas. That would be a gamble though.

As a former math teacher in a disadvantaged area, I'm going to have to stick to my original statement. It could be different in other regions of the world, but, in the US, because funding is based on test scores, and teachers are required to teach the test concepts, whether students are ready for them or not. Students have to cover so much material in a given time period. For many, the time to truly "get" the material is not there. They can memorize something, test on it, move on. The fact that some students are capable of getting through this and learn or retain information does not mean it is a good method of doing things.

As a practical example why multiple choice can be unfair: a student could understand that solving for the roots of a parabolic curve (quadratic equation) could model the landing point of a ballistic projectile. They could understand the difference of squares method to solving quadratics, factoring, solving linear equations, etc. But a simple arithmetic mess up (they add instead of subtracting on one step even though they wrote a subtraction sign) leads to the wrong answer. Because it is a multiple choice test and because test makers have seen this same error before, the wrong answer the student found is one of the multiple choice solutions. Boom. Zero credit on that question. This failed to measure the student's understanding. How do you get around that? More questions with more nuances.

Can you get signal on student understanding from multiple choice tests? Sure. Is it good? Depends on the test and it is hard to write good tests.

I tried to suggest that different goals could mean a different test setup. Instead of a multiple choice test after each course you could have a proper test after a longer period, where grading would consider in the full solution / essay.

I understand that it may be incompatible with the current US practice, but I don't see why each test should be standardized. Locally arranged course exams with a common state/nationwide final exam with proper grading resources should be enough as far as I see.

I do think it is worth drawing a distinction between college and K-12 though. In college students are there by their own choice, and colleges choose who they admit, and if students are paid when they succeed on tests they have a strong and immediate incentive that doesn't exist for them in K-12.

There is no necessary reason that tests have to be on a strict time schedule. Imagine instead that testing is offered continuously throughout the year, and a small fee is collected from both the school and the student for taking the test, which is refunded along with the payout if the student succeeds. Then there would be a reason to avoid taking testing before the student has a good handle on the material.

I think that testing serves different functions depending on the context, the binary signal of whether someone has a degree or not from some random university doesn't really measure deep understanding either. It makes sense for a teacher to measure the fined-grained understanding of their students it order to figure out what they most need help with, but maybe it is less important for determining when their level of skill and knowledge is above the threshold where they should be given a degree or credential.