Imagine you were an alien come to visit the earth to find out what makes humans tick. The purpose of your mission is determine what the humans will do in the future and, in particular, if in time they are likely to become a threat to your civilization. If so, your civilization will put an end to the human one before it becomes a threat.

Now, if you were a sociologist alien interested in what humans want and how they will act, you might organize a grand survey of the entire population, carefully tabulating everything people say on what they want or how they would act in any given situation.

After the survey you return home with the happy news: These humans are a marvelous lot and wouldn't hurt a fly. In fact, once they have mastered space travel, they are sure to become our best friends. The acclaim for you and your mission is enormous. Your grandchildren, however, are disappointed when, a hundred years later, the humans show up at your home world and nuke it from orbit.

Fortunately, however, you are of a rather more clever type than the sociologist alien—the economist alien. As such you know that what people say they want or would do is of little consequence. For people habitually deceive themselves and others about such things. So a grand survey would yield results little better than worthless.

That is not to say that people do not have motivations or that these motivations, if known, could not effectively be used to predict behavior. It is just that you cannot find out about motivations by asking. The only way to discover motivations is by observing the behavior of people in many situations.

So you set out, not on a grand survey, but on a massive series of experiments. You are an alien of refined taste, so you do not go for the cliché kidnapping and probing. Rather you subtly create situations in which your subject (who remains completely unaware of being experimented on) has to choose various outcomes you have selected; let's call them A, B, C, and so on.

After a while you will have learned a great deal about the commonalities and diversity of human motivation. In particular, for any given subject, you will have learned a complete ordering of their preferences. Just present all possible outcomes to the subject and see which one is chosen; let's say A. Then you repeat the experiment—mind-wiping the subject first if necessary—with the option A removed. Let's call the one picked then, B. By this procedure you arrive a complete ordering from A to Z and will be able to predict how the subject will act when presented with any set of options.^{1}

But there are some important things you will still not know. For if the subject greatly prefers A to B, the subject will pick A over B every time. But if the subject merely slightly prefers A to B, the subject will still A over B every time. So none of your experiments can discover the strength of the subjects preference. You might say that you only have an ordinal understanding of the subject's utility function, not a cardinal one.

This is a major omission because in decision-making under uncertainty, as most is, knowing the ordinal utility is not enough. If the subject faced some combination of A, B, and C, knowing the ordinal utility is enough to predict a pick of A over B over C. But if the subject faced instead a choice between the certain outcome B or a 50/50 chance at A and C, ordinal utility will not give you a prediction. For if the subject greatly preferred A over B and B only slightly over C, the subject would take the 50/50 A/C chance. But if the subject only slightly preferred A over B, but greatly preferred B over C, the subject would pick the certain B.

But the same circumstances which reveal the limits of ordinal utility also show how to overcome it: A lot more experiments in which the subject has to choose not only between certain outcomes, but also bundles of outcomes, each with a known probability to occur if the bundle is picked. With enough of these you will fully understand the subject's motivations and will know the subject's cardinal utility—an exact number—for every possible outcome. With that knowledge you can predict its action in any situation, certain or uncertain.

^{1} It is true that a particularly perverse subject might when presented with the choices of A, B, and C, pick A, but when presented with the choices of only A and B, pick B. Fortunately that type is fairly rare, particularly where important matters are concerned. But, when encountered, you'll have just perform a lot more experiments, presenting not only all outcomes, but all possible sets of outcomes.^{2}

^{2} If the number of outcomes you need to evaluate is finite, so will all their possible sets. Admittedly, if your number of outcomes was infinite then the set of possible sets of outcomes would be a strictly larger infinity. But you wouldn't have time even for one of the smaller infinities anyway.

^{3} Yeah, yeah, up to affine transformations. But those wouldn't change your predictions of the subject's behavior.