AI Seminar
    -------------------------------
    Tuesday, March 23rd, 2004
    4:00 pm - 5:30 pm
    175 ATL (Large Conference Room)

    "Adaptive Cognitive Orthotics: Combining Reinforcement Learning and
    Constraint-Based Temporal Reasoning"
    and
    "A Nonlinear Predictive Representation of State"

    Matthew Rudary
    Department of Electrical Engineering and Computer Science
    University of Michigan




----------------------------------
I will be covering two topics of ongoing research. First, I will
discuss the integration of Autominder and a
reinforcement-learning-based reminder policy. Autominder is a reminder
system that supports people with impaired cognitive ability by
providing them with reminders of their functional activities. The
integration with RL allows Autominder to personalize to a user and
adapt to both short- and long-term changes. In addition to advancing
the application domain, our integrated algorithm includes a novel
method of combining reinforcement learning with existing
constraint-based temporal reasoning methods. I will present our
methods and results of experiments with simulated users.

Next, I will present a nonlinear predictive state representation
(PSR). PSRs use predictions of a set of tests to represent the state
of controlled dynamical systems. PSR models of systems may be much
more compact than POMDP models. Empirical work on PSRs to date has
focused on linear PSRs, which have not allowed for compression
relative to POMDPs. I will introduce a new notion of tests that allows
us to define a new type of PSR that is nonlinear in general and
allows for exponential compression in some deterministic dynamical
systems.

This research has been conducted with Satinder Singh and Martha
Pollack.