Position Paper to be presented at the AAAI Fall Symposium on Rational Agency, November 1995. Comments welcome.
Revised August 1995
But often we find it helpful to view this product at a higher level, say, as the solution to some well-posed problem. Indeed, every subfield of computer science defines its own abstractions above the level of computing functions. For example, database specialists might prefer to view the input of their machine as a query, the product as data, and the machine's activity as retrieval. Of course we can always reduce this abstract database machine to the more concrete computing machine, but in doing so we may increase the description complexity and make it more difficult to analyze, predict, or design the behavior of database machines. In this case and most others, the reduction is perfect, in the sense that the lower-level machine exactly realizes the behavior of the abstract machine, and the relationship between them is typically described with complete precision.
Fifteen years ago, Newell proposed that a central characteristic of AI practice is a particular abstraction level at which we interpret the behavior of computing machines. Viewing a system at Newell's knowledge level entails attributing to the system knowledge, goals, and available actions, and predicting its behavior based on a principle of rationality that specifies how these elements dictate action selection. Rationality as applied here is a matter of coherence, defining a relation in which the knowledge, goals, and actions must stand. This is exactly the Bayesian view of rationality, in which knowledge and goals (or beliefs and preferences) are subjective notions, constrained only by self-coherence (consistency) and coherence with resulting behavior.
Newell acknowledged that the knowledge level is more radical than most other abstractions, in that we recognize that it is at best an approximation to the actual behavior described at a mechanistic (directly functional) level. That is, we employ the knowledge-level abstraction even though we lack a perfect reduction to systems described in purely computational terms. In fact, we know that such a reduction cannot exist, as the knowledge level (at least in its pure form) idealizes away the computational process itself.[1] Nevertheless, the knowledge-level perspective can be quite powerful, enabling us to predict the behavior of a system too complex or opaque to be described at a mechanistic level.
In introducing the knowledge-level idea, Newell proposed a particular basic rationality principle (along with several auxiliary principles):
If an agent[2] has knowledge that one of its actions will lead to one of its goals, then the agent will select that action.This formulation can be criticized on several grounds, foremost of which is that it embodies, logical rather than economic rationality (Doyle, 1992). But let us put aside such concerns and consider the perspective that a rationality principle of this general form takes on computing machines. The relevant output identified by this principle is an action selection. Thus, action selections, or in effect, actions, are the products of a knowledge-level machine.
But exactly what are actions? Newell posits as a simplifying assumption that the agent's "body" (that is, its range of behavior) consists of a set of actions it can choose, acknowledging that in actuality the mode of interaction between an agent and its environment can be arbitrarily complex. Ultimately, actions are in the interpretation of a beholder. The true output of a computer program is a stream of physical encodings, which are then transduced by some external effectors into activity affecting the environment. But neither this activity nor the information states inducing it are inherently divisible in some particular way into discretely selectable action units (Rosenschein and Kaelbling, 1995).
There are at least two standard technical meanings of the term action in AI and the decision sciences. In one, typical of foundational treatments of decision theory, an act or action denotes the entire behavior of the agent in question. In the other, typical of AI planning discussions, an action is an atomic component (or step) of a more composite activity description, called a plan. An action by this latter interpretation is not satisfactory as the subject of a rationality criterion, however, because it is generally not possible to determine whether an individual action is rational separate from consideration of the entire plan of which it is part.[3] If it were, planning would be a trivial exercise.
An action by the former interpretation--or a plan by the latter--could be legitimately evaluated for rationality. This interpretation, however, does not yield a very practical rationality principle, because its unit of analysis is the entire behavior of the system. At a fundamental level, this is Savage's (1972) problem of small worlds. In Savage's words, the ideal rational agent
...has only one decision to make in his whole life. He must, namely, decide how to live, and this he might in principle do once and for all.Choice of this lifelong policy is the agent's grand-world decision problem. Decision scientists following (and including) Savage have rightfully deemed grand-world planning unrealistic, and instead focus on episodes which they assume can be reasonably isolated from the remainder of the agent's lifetime of activity. This is at best an approximation, and guaranteeing an approximation of acceptable quality may require considering episodes of impractically coarse granularity.
What we need is a more flexible unit of analysis--one that recognizes that the ultimate object for evaluation is the grand-world plan, but admits scrutiny of more localized computational products. Let [[Pi]] be the plan space, the set of all grand-world plans available to the agent. We define a decision to be a commitment to a subset, [[Pi]]', of [[Pi]]. The choice of [[Pi]]' over its complement constrains the activity of the agent, in any of a variety of ways.
For example, selection of an action (plan step) is a special case of a decision, in this case commitment to the class of plans containing that action, excluding those that do not. But decisions can also involve more complex patterns of activity, including conditional commitments, combinations of actions, or any expressible distinction in plan space, for that matter. Indeed, something of this nature is necessary to incorporate notions of intention (Pollack, 1992), since intentions are rendered superfluous if either individual actions or entire grand-world plans are adopted as the unit of analysis.
Since the most comprehensive rationality criteria are expressed in terms of grand-world plans, assessing the rationality of a decision is not entirely straightforward. To even entertain the possibility of rationality, we must attribute to an agent choice, that is, an ability to behave other than it actually behaves. In other words, we must view the computation as nondeterministic, or uncertain, at least from the observer's perspective. As a computation proceeds, uncertainty is progressively resolved, and its possible end results are narrowed. This is equivalent, according to the definition here, to saying that the agent makes decisions, committing to progressively smaller sets of possible plans.
In previous work I have proposed a general dominance-based criterion for assessing grand-world distinctions (Wellman, 1990), but more specialized approaches are appropriate for particular formulations of plan space. And of course we can always revert to the special case of episodic decision making when that approximation is acceptable.
For present purposes, I wish to emphasize the flexibility of the concept of decision as defined here, and propose that decisions are the appropriate unit from which to formulate a comprehensive principle of rationality. By viewing our artifacts as decision machines, we focus on the relevant product--patterns of activity--at whatever granularity is appropriate.
Hence the question "why adopt a rationality principle?" reduces to "why do we care to attribute knowledge and other so-called agent attitudes to our machines?". This latter question has been addressed by Newell, McCarthy, Dennett, Shoham, and others, and I will not attempt to reproduce their eloquence. Suffice it to say that the arguments all boil down to effectiveness of the rationality abstraction. Empirically, it appears that we can often effectively predict and explain the behavior of exceedingly complex or mechanistically ill-understood systems (such as people) by attributing to them agent attitudes such as beliefs, preferences, etc., ... and rationality. The premise is that the complexity of sufficiently capable computing machines will also lie beyond the scope of program-level abstractions, and that the most effective way to deal with our decision machines will be in terms of beliefs, desires, intentions, and the like.
Even though the primary motivation for rationality is coherence, it is without a doubt advisable to consider competence when deciding what rationality principle to adopt. All else being equal, we would naturally prefer a rationality principle that is normatively compelling and comprehensive (such as Bayesian decision theory, in my view). The descriptive validity of alternative rationality principles for various classes of decision machines is an interesting and largely unexplored question. It would seem that absent countervailing evidence, we should aim to design machines adhering to the highest standards of rationality we can imagine. Relaxation of the standards should be entertained only for specifically motivated compromises.
If one is seeking ideas about collections of rational agents, my summary suggestion is to look toward microeconomics (Wellman, 1995). Indeed, its pervasive use of the rationality abstraction is one of the main features distinguishing economics from other social science disciplines. In that sense economics and artificial intelligence are natural allies, and we could gain much by adapting the concepts, mechanisms, and theories of economics for the design of decision factories. For some approaches that do this, see Clearwater (1995), Rosenschein and Zlotkin (1994), and Wellman (1993).
Doyle, J. (1992). Rationality and its roles in reasoning. Computational Intelligence 8: 376-409.
Newell, A. (1982). The knowledge level. Artificial Intelligence 18: 87-127.
Pollack, M. E. (1992). The uses of plans. Artificial Intelligence 57: 43-68.
Pollock, J. L. (1992). New foundations for practical reasoning. Minds and Machines 2: 113-144.
Rosenschein, J. S., and G. Zlotkin (1994). Rules of Encounter: Designing Conventions for Automated Negotiation among Computers. MIT Press.
Rosenschein, S. J., and L. P. Kaelbling (1995). A situated view of representation and control. Artificial Intelligence 73: 149-173.
Russell, S. J., and P. Norvig (1995). Artificial Intelligence: A Modern Approach. Prentice-Hall.
Russell, S. J., and D. Subramanian (1995). Provably bounded-optimal agents. Journal of Artificial Intelligence Research 2: 575-609.
Savage, L. J. (1972). The Foundations of Statistics. Second edition. New York, Dover Publications.
Wellman, M. P. (1990). Formulation of Tradeoffs in Planning Under Uncertainty. London, Pitman.
Wellman, M. P. (1993). A market-oriented programming environment and its application to distributed multicommodity flow problems. Journal of Artificial Intelligence Research 1: 1-23.
Wellman, M. P. (1995). The economic approach to artificial intelligence. ACM Computing Surveys September 1995.