o Langley, P., & Laird, J. E. (2002). Cognitive architectures: Research issues and
challenges (Technical Report). Institute for the Study of
Learning and Expertise,
Papers and books on Soar: The Soar Papers: Readings on Integrated Intelligence, Rosenbloom, Laird, and Newell (1993), and Unified Theories of Cognition, Newell (1990). These web pages and the Soar FAQ also provide answers to many common questions about Soar.
Soar continues to evolve, even after 15 years of development. A review of its evolution up through Soar 6 is in
o Laird, J.E., & Rosenbloom, P.S. (1996) The evolution of the Soar cognitive architecture. In T. Mitchell (ed.) Mind Matters.
At the 23rd Soar workshop I gave a talk based on that paper, but updated to 2003 (and Soar 8).
o John Laird, 2003, The Evolution of Soar.
Soar 7 incorporates TCL into Soar to enhance our ability to easily interface other systems to Soar and to create new task environments for Soar. It also changed the underlying problem space computational model (PSCM) so that only operators are proposed, compared and selected (no longer problem spaces and states).
Soar 8 incorporates changes to the semantics of Soar to enhance its ability to maintain consistency in its reasoning as the world changes. Some of this has been described in the following papers:
o Robert E. Wray and John E. Laird. An architectural approach to consistency in hierarchical execution. Journal of Artificial Intelligence Research. 19. 355--398. 2003.
o Wray,
R. E. (1998). Ensuring Reasoning Consistency in Hierarchical Architectures. Ph.
D. Thesis.
o
Wray, R. E., and Laird, J. (1998). Maintaining
consistency in hierarchical reasoning. Fifteenth National Conference on
Artificial Intelligence. 928-935.
o
Wray, R. E., Laird, J., and Jones, R. M. (1996).
Compilation of
non-contemporaneous constraints. In Proceedings of the Thirteenth
National Conference on Artificial Intelligence, 771-778.
Soar 9 is “under development” and will include reinforcement learning, episodic memory, semantic memory and emotion, as well as activation. We are developing these pieces independently, but plan on working on integration in fall 2005. Some aspects may be available for release in spring 2006.
o Nason, S. and Laird, J. E., Soar-RL, Integrating Reinforcement Learning with Soar, Cognitive Systems Research, 6 (1), 2005, pp. 51-59. Also in International Conference on Cognitive Modeling, 2004.
o Nuxoll, A., Laird, J., A Cognitive Model of Episodic Memory Integrated With a General Cognitive Architecture, International Conference on Cognitive Modeling 2004.
o
Nuxoll, A., Laird, J., James, M. Comprehensive
Working Memory Activation in Soar.
International Conference on Cognitive Modeling, Poster, 2004.
o Marinier, R., Laird, J. Toward a Comprehensive Computational Model of Emotions and Feelings, International Conference on Cognitive Modeling 2004.
I'm also very interested developing methodologies for evaluating and comparing agent architectures. Some the issues in doing this include identifying the set of desired architectural capabilities.
o Wallace, S., Laird, J. E., Coulter, K. Examining the Resource Requirements of Artificial Intelligence Architectures. Conference on Computer Generated Forces and Behavior Representation, May 2000.
o Bhattacharyya, S. & Laird, J. E., Lessons for Empirical AI in Plan Execution. Accepted to the IJCAI-99 workshop on Empirical AI.
o Wallace, S. & Laird, J. E., Toward a Methodology for AI Architecture Evaluate: Comparing Soar and CLIPS. ATAL-99, July, 1999.
o
Laird, J., Pearson, D. J., Jones, R. M., and
Wray, R. E. (1996). Dynamic Knowledge Integration During
Plan Execution. In Papers from the 1996 AAAI Fall Symposium on Plan
Execution: Problems and Issues, 92-98.
The students in a class of mine created an extensive web document that attempts to classify and analyze many existing AI Agent Architectures.
· Robert
E. Wray, Ronald Chong,
o Wallace,
S. Validating
Complex Agent Behavior, Ph.D. Thesis
o Wallace, S. and Laird, J. E. Behavior Bounding: Toward Effective Comparisons of Agents & Human Behavior, International Joint Conference on Artificial Intelligence, 2003.
o Wallace, S., and Laird, J. E. Toward Automatic Knowledge Validation. In Proceedings of the Eleventh Conference on Computer Generated Forces and Behavioral Representation. pp. 447-456. May 2002.
o Wallace,
S., and Laird, J. E. Intelligence
and Behavioral Boundaries. NIST Workshop on Performance Metrics for
Intelligent Systems (PerMIS 2002).
My interests in learning center on integrating learning with performance (planning and execution) in architectures for general intelligent agents who interact with complex environments. Over the years, my students have looked at different aspects of this problem, always trying to understand the integration and architectural issues. We have tried to build up more and more learning capabilities, within a single architecture. One cut at what we've done is that we've continually looked at different sources of knowledge (experience, instruction, examples) and how they can improve different aspects of performance.
This material is based upon work supported by the National Science Foundation under Grant No. 0413013
We are in the process of adding reinforcement learning to Soar. This paper is the first publication on the work.
Nason, S. and Laird, J. E., Soar-RL, Integrating Reinforcement Learning with Soar, International Conference on Cognitive Modeling, 2004.
This paper presents a model of rat learning using Soar’s RL mechanism and compares it to an ACT-R model. Wang, Y., and Laird, J.E. 2007. The Importance of Action History in Decision Making and Reinforcement Learning. Proceedings of the Eighth International Conference on Cognitive Modeling. Ann Arbor, MI. http://www-personal.umich.edu/~yongjiaw/publications/ICCM_2007.pdf
We are also adding an episodic memory to Soar.
Nuxoll, A. M. and Laird, J. E. (2007). Extending Cognitive Architecture with Episodic Memory. In Proceedings of the 21st National Conference on Artificial Intelligence (AAAI). http://ai.eecs.umich.edu/soar/sitemaker/docs/pubs/AAAI2007_NuxollLaird_ver14(final).pdf
Nuxoll, A., Laird, J., A Cognitive Model of Episodic Memory Integrated With a General Cognitive Architecture, International Conference on Cognitive Modeling 2004.
The first integration of chunking in Soar led to the study of learning search control knowledge and is described in the AAAI-84 paper below. This was followed by looking at learning macro-operators using chunking. We also did a comparison of chunking in Soar to Explanation-based Generalization/Learning, concluding that chunking is one form of EBL.
Although chunking has always been integrated with performance in Soar, the following papers exam that issue directly.
o Laird,
J. E. and Rosenbloom, P. S. Integrating Execution, Planning, and Learning in
Soar for External Environment. In Proceedings of National Conference of
Artificial Intelligence, 1022-1029, July 1990,
o Laird,
J. E., Hucka, M., Yager, E.
S., and Tuck, C. M. Robo-Soar: An integration of
external interaction, planning and learning using Soar. Robotics and
Autonomous Systems, Vol. 8, 1991, pp 113-129. This also appears as a
chapter in Toward Learning Robots, W. Van de Velde
(Editor), MIT Press,
In this work, we've looked at inductive learning of concepts using symbolic mechanisms, while still preserving many of the typicality and graded performance behaviors seen in humans. Craig Miller developed an approach to inductive learning, called SCA, that does not require any modifications to chunking. This research demonstrates that it is possible to use analytic methods, such as chunking (or EBL) to do more than just speed-up learning. SCA is notable because it is incremental, noise tolerant, can make use of many different sources of knowledge (not just examples), and matches human typicality data for concept learning. SCA has been integrated into many of the follow-on learning systems listed below.
o Miller, C. S., and Laird, J. E. A Constraint-Motivated Lexical Acquisition Model. In Proceedings of the Eighth International Workshop on Machine Learning, 95-99, 1991.
o Miller, C. S. and Laird, J. E., Accounting for graded performance within a discrete search framework, Cognitive Science, 20 (4), 1996, pp.499-537.
To further demonstrate the ability to integrate multiple sources of knowledge in learning, as well as demonstrate the integration of learning and performance, we took a page from Winograd's book and explored how to integrate instruction with performance. This work was done by Scott Huffman for his thesis and resulted in a system called Instructo-Soar, which had the ability to dynamically request instruction whenever it was unable to work on a problem. It could accept a variety of types of instructions, perform the task, and learn from the instructions so that in future similar situations, the agent would perform the task without the need of instructions.
o S. B. Huffman and J. E. Laird, Learning procedures from interactive natural language instructions, in P. E. Utgoff, ed., Machine Learning: Proceedings of the Tenth International Conference (ML-93), 1993.
o S. B. Huffman, The requirements of instructability, in Working notes of the 1994 AAAI Spring Symposium on Active Natural Language Processing, ed. C. Martin, J. Lehman, and K. Eiselt, March 1994.
o S. B. Huffman and J. E. Laird. Learning from highly flexible tutorial instruction, in Proceedings of the Twelfth National Conference on Artificial Intelligence (AAAI-94).
o S. B. Huffman and J. E. Laird. Flexibly Instructable Agents. In Journal of Artificial Intelligence Research , Volume 3, Pages 271-324, 1995.
This work started out with the goal of just showing that it is possible to correct knowledge using EBL-like techniques if the architecture supports the right kind of deliberation mechanism for selecting operators. It then expanded as Doug Pearson perused it for his thesis. In the end, his system, IMPROV, is pretty amazing in that it is able to detect and correct errors in its planning knowledge through interactions with dynamic environments in which actions take time and where there may be noise in the sensors. This uses SCA extensively.
o
Laird, J. E. Recovery from Incorrect Knowledge
in Soar. In Proceedings of AAAI-88, American Association for Artificial
Intelligence,
o S. B. Huffman, D. J. Pearson, and J. E. Laird, Correcting Imperfect Domain Theories: A Knowledge-Level Analysis. in Machine Learning: Induction, Analogy and Discovery, edited by Susan Chipman and Alan Meyrowitz, Kluwer Academic Press, 1993. This is about the types of errors you can have in an agent's knowledge of the world.
o D. J. Pearson, Learning Procedural Planning Knowledge in Complex Environments. Ph.D. Thesis, 1996.
o Active Learning in Correcting Domain Theories: Help or Hindrance? in AAAI Symposium on Active Learning (1995). This is about the relative merits of learning by doing, rather than learning by being given carefully chosen sample problems.
o Toward Incremental Knowledge Correction for Agents in Complex Environments. In Machine Intelligence 15 (1998). This is a general overview of Doug Pearson's system that learns to correct mistakes as it solves problem. The system is called IMPROV.
o Dynamic Knowledge Integration during Plan Execution. in AAAI-96 Fall Symposium on Plan Execution: Problems and Issues (1996). This is about how tasks and environments constrain the way plans can be built and used.
Instructo-Soar and IMPROV have been integrated together. Here are two similar papers that cover their integration. The second tries to take a broader view of adaptation in intelligent agents.
o Pearson, D. J ., Laird, J. E., “Incremental Learning of Procedural Planning Knowledge in Challenging Environments,” Computational Intelligence, 2005, 21:4, 414
o D. J. Pearson and S. B. Huffman, "Combining learning from instruction with recovery from incorrect knowledge." ML-95 workshop on Agents that learn from other agents,, July 1995.
o
Laird, J. E., Pearson, D. J., Huffman, S. B., Knowledge-Directed
Adaptation in Intelligent Agents. AAAI Workshop on Intelligent
Adaptive Agents, August 1996. Published in Imam, I.F., and Kodratoff, Y., Intelligent Adaptive Agents: A Highlight
on the Field and A Report on the AAAI-96 Workshop, A Technical Report of
the Machine Learning and Inference Laboratory,
This work was done by Seth Rogers (now of ISLE at Stanford). It investigates using SCA style symbolic learning within the context of continuous environments.
o Increasing Learning Rate via Active Goal Selection (84KB), in the 1995 AAAI Symposium on Active Learning.
o New Results on Learning from Experience in Continuous Domains (136KB), unpublished technical update.
o Symbolic Performance & Learning in Complex Environments (43KB), AAAI 1996 National Conference on Artificial Intelligence. Student Abstract and Poster Program. (new version (161KB))
o Symbolic Performance & Learning in Complex Environments (485KB), unpublished longer version of AAAI abstract.
Seth's thesis is available as a Technical Report
from EECS Department,
We have started a research project on learning from observation. The basic idea is to learn procedural knowledge based on observations of humans performing the same task. We are building on work done on "behavioral cloning" by Claude Sammut and his colleagues. One extension we have added is the ability of the human to add annotations in terms of current goals. This simplifies the parsing of the behavior into relevant segments. This work is being done by Michael van Lent. His web site is more likely to have current descriptions of the research. Click here for more information.
o
van Lent, and Laird,
Learning by Observation in a Tactical Air Combat Domain, Proceedings of the
Seventh Conference on Computer Generated Forces and Behavioral Representation.
o
van Lent, M. & Laird, J. E., Learning Procedural
Knowledge by Observation. Proceedings of the First International Conference
on Knowledge Capture (K-CAP 2001),
Air-Soar was a system that flew the SGI flight simulator. It demonstrated that it was possible to have a symbolic rule-based system act as a controller in a dynamic environment. Air-Soar designed directly used Soar's hierarchical operators to represent a hierarchy of goals and action.
o D. J. Pearson, S. B. Huffman, M. B. Willis, J. E. Laird, and R. M. Jones, "Intelligent multi-level control in a highly reactive domain," in Proceedings of the Third International Conference on Intelligent Autonomous Systems, Pittsburgh, PA, February 1993.
o D. J. Pearson, S. B. Huffman, M. B. Willis, J. E. Laird and R. M. Jones, A symbolic solution to intelligent real-time control. in Robotics and Autonomous Systems 11 (1993) (or try Elsevier Publishers). This is about designing a rule-based system to fly a simulated plane in real-time.
Over the last eight years, we have been developing intelligent agents for simulated battlefields. This is the Soar/IFOR component of the WISSARD/IFOR project (funded by DARPA/ISO). The goal of Soar/IFOR is the development of autonomous computer agents whose behavior is tactically indistinguishable from humans. These synthetic agents must not only be lifelike, they must be humanlike with many of the capabilities we commonly associate with intelligent human behavior: real-time reactivity, goal-directed problem solving and planning, large bodies of knowledge, adaptation to changing situations, and interaction and coordination with other intelligent entities. The Soar/IFOR consortium, involving the University of Michigan, University of Southern California's Information Sciences Institute, and Carnegie Mellon University , is developing such agents for air missions: air to air combat, air to ground attacks, helicopter missions. A long-term goal of this research is to extend this technology to education, training, and entertainment where humans can interact with humanlike intelligent agents in a variety of synthetic environments.
A number of papers have been published about our project. Below are some of the UM papers:
The research and development in this area continues at Soar Technology, Inc.
We are now working hard on developing AI systems for Computer Games.
Below are papers on integrating Soar with a real-time strategy game engine (ORTS)
Wintermute, S., Xu, J., Irizarry, J., Laird, J.E. 2007. SORTS Tech Report. http://ai.eecs.umich.edu/soar/sitemaker/docs/pubs/sorts_report.pdf
Wintermute, S., Xu, J., and Laird, J.E. SORTS: A Human-Level Approach to Real-Time Strategy AI. Proceedings of the Third Artificial Intelligence and Interactive Digital Entertainment Conference (AIIDE-07), Stanford, California http://www.eecs.umich.edu/~swinterm/papers/AIIDE07-SORTS.pdf
Doug Pearson and I have been working on a new was for doing knowledge acquisition that involves scenarios described as diagrams by an expert.
·
Douglas Pearson, John E. Laird, Redux: Example-Driven Diagrammatic Tools for Rapid
Knowledge Acquisition, Proceedings of Behavior Representation in Modeling
and Simulation, 2004,