Emotive Character Agents for Python Universe Builder
Terry Hancock, 2005-8/8
Project services hosted by:
Emerald will be a class-libreary for creating character agents for use with Python Universe Builder
to create richer, more believeble characters. It is inspired by a reading of the papers from
the Carnegie Mellon University's "Oz Project" (see my brief review below,
with links to the various papers), combined with a simple, four-parameter
that has some computational advantages over the word-based model used in "Em" (the emotion engine
used in the Oz project).
Whereas the Oz project was implemented in a combination of C and Lisp, and never produced
published code, Emerald will use Python with the possibility of extension with Pyrex (in case
the emotion math turns out to require it), and will of course be open source.
This page is pretty incomplete at this time (2005-8/8), and I'm not going to spend too much
time on it just yet. For me, this is in support of
The Light Princess, and it isn't critical for
some time. But I wanted to express my intent, and publish some of the ideas that I've been
working on. I'll periodically update this page as I continue to develop the concept. At
this time, what I have is an experimental emotion model, which needs to be brought up to
date (I wrote it when I was just learning Python, and it's apparently rather poorly
written). The original can be found in the CVS for The Light Princess until I get the
update loaded here.
Review of the Oz Project at Carnegie Mellon University
In the 1990's, a group at CMU developed a game engine very much
like what we wanted to have for "The Light Princess". It incorporated
relatively simple, but believeable character agents, and a drama
planning mechanism that made for much more "believeable" and
entertaining (if not necessarily more realistic) game play.
Oz Architecture -- Interactive Fiction Engine
The Oz architecture is illustrated below (after the illustration
in the Hap architecture paper):
The "physical world" is a very simple topological model, almost
exactly like what we find in Interactive Fiction games, so it
seems that Python Universe Builder (PUB)
which is based on the same kind of model, should be a good replacement.
This is good, because the world model was said to constitute about 85% of
the code in the Oz Architecture.
For more detail and context, please see:
An Oz-Centric Review of Interactive Drama and Believable Agents,
Michael Mateas, June 1997, CMU-CS-97-156 (HTML).
Tok Architecture -- Character Agents
The agent behavior represents most of the functionality of Tok. Here is a diagram of the
Tok architecture, showing how it relates to the two main components which
comprise it, namely "Em" and "Hap" (from the Tok Architecture paper):
For more detail, see the original paper:
An Architecture for Action, Emotion, and Social Behavior,
Joseph Bates, A. Bryan Loyall, W. Scott Reilly, May 1992, CMU-CS-92-144 (Gzipped Postscript).
Hap Architecture -- Goal Seeking
Hap is the side of the Tok architecture that attempts to implement goals
and determine success or failure of plans to achieve its goals. It is
composed of two primary structures: an "Active Plan Tree" and a
"Plan Memory" and interacts with its environment (represented by
an Interactive Fiction world) and with the companion "Em" module,
which represents emotional and social behavior. Here is a
diagram of the Hap architecture, based on the Oz papers:
Details on the architecture can be found in the original paper,
which I highly recommend reading:
Hap, A Reactive, Adaptive Architecture for Agents,
A. Bryan Loyall, Joseph Bates, June 1991, CMU-CS-91-147 (Gzipped Postscript).
Em Architecture -- Emotions
Em is the emotional subsystem of Tok. I'm less sure of the maturity
of this system, and, given the need to re-implement the code,
I'm considering using my original emotive behavior architecture
instead (documented in the existing CVS for The Light Princess),
but Em should be adequate for our game, so it's possible
that we should simply implement it as it was. This may be simpler
in the end.
To give an idea of the level of complexity in Em, here's a list of
the emotions understood by the version of Em used to implement the
"Lyotard" game (in which the agent modeled a cat):
For more detail, see the original paper:
Building Emotional Agents, W. Scott Reilly, Joseph Bates, May 1992, CMU-CS-92-143 (Gzipped Postscript).
The Oz architecture seems to be an excellent fit to our needs.
It demonstrates the feasibility of this approach, and has adequate
documentation to completely re-implement the architecture in
Python, which should be well-adapted to representing the structures
and algorithms used. Tok originally consumed about 7500 lines
of code (mostly in Lisp), and I think our implementation will
probably be no larger.
The world model for Oz was implemented in Lisp, and is described
in a way that sounds almost exactly like the model found in
interactive fiction games made with Inform or TADS. Since the
Python Universe Builder (PUB)
was also built to follow this model, it should be an excellent
fit. Thus we should be able to implement almost the entire
engine in Python.
The Oz Project produced a lot of papers and online information
as well. I have selected a few key papers to illustrate the
architecture as concisely as possible. However, a visit to
Oz Project Website
will be well worth the trouble, if you are interested in
this project. There are many more papers available from
there (and also the uncompressed forms of the papers above,
in case you have trouble uncompressing these).
Review of WorldForge's Cyphesis Mind System
Another influence on our design should probably come from the (working) NPC agents in the
WorldForge system. These agents are not emotive, but are purely heuristic, based on
knowledge available to them, and innate drives. They might be described as "behavior-based",
but whether this is truly a "subsumption architecture" is not immediately clear to me from
reading the source. Readers coming from PUB may be interested to know that the Cyphesis
mind system is written in Python, even though the Cyphesis framework is in C/C++.
The main idea of interest here is the concept of "Knowledge" as a subjective model of
the objective game model world, to be maintained by the character's "mind". The idea
that characters should collecte this information from the world seems like a good one,
although most of the characters in "The Light Princess" will already be highly embedded
in their environments, so it's unclear whether I'll gain anything specific for that
project over just using the objective model (which is equivalent to assuming that the
characters have already "seen everything" in the model).
The emotion model was adapted from psychological literature on understanding and modelling
real emotions, in contrast to Em's approach, which is based on the use of emotional words.
So, my model is more mathematical and abstract, using an ordered tuple of four numbers to
represent an emotional state. It might therefore be considered part of the "color-model"
class of emotion representations.
Emotion theory is full of highly conflicting constructs and terminology, because it is so
immature, and researchers' ideas do not fit into any well-established system of theory. Nor
is it easy to make empirical measurements of which system is more closely related to the
way emotion really works. Worse, the jargon words in this field tend to be words we associate
with normal English usage. Returning to the CMU Oz project's philosophy, though, I want to
reiterate that we are only trying to achieve "believeability" in our characters, not trying
to accurately create emotional machine intelligence!
In order to communicate, we need some standards for terminology (for extra
clarity, Emerald and PUB jargon words are boldface to emphasize that we mean them
in their technical sense):
The active-ness or aggressiveness of an emotional affect. A tendency to act rather
than remain passive. First component of affect. Symbol: A.
In Emerald, the term "affect" will mean two things: most definitively it will mean a four-tuple
representing the emotional reaction that a character feels in response to a certain object
(which may be a concrete or abstract noun of any kind). In a broader sense, it will also mean
the same four-tuple in any context. The four parameters in this model are:
valence, and intensity.
An agent is just a game object in PUB, based on existing character objects,
but with the ability to make decisions and moves. At each turn of the game, all
agents are allowed to think and make one move, just like the player character can.
In the context of Python Universe Builder, we now have three major types of character:
"player" (meaning that decisions are made according to input from the user), "state-machine"
(meaning the character is a simple state machine as NPCs are in older IF engines), and
"agent", meaning that the character's actions are controlled by an intelligent agent as
we are trying to build with Emerald.
A constant property of a character agent which determines a prefered
for that character. E.g. a brooding character will tend to be sad or unhappy, while
an optimistic character will tend towards happy regardless of stimuli.
Property of a character that defines how much their
expression is driven by mach
Emotion is the name of the emotion model package as a whole, as
distinguished from the Planner.
- Emotive Sum
Ideally, a sum over objects, weighted by
salience, of affect. Less,
ideally, an incomplete sum we think approximates this well, which is limited to objects
in the room and objects of high-saliency from previous computations. I thought
seriously about calling this the jargon meaning of "Emotion", but I think it's wise to
avoid using the term for anything so specific.
Expression is a sum of affect that drives the emotion the
character communicates to others or expresses outwardly. This may be used to generate
inflected text by choosing emotional adjectives and adverbs, or to drive a visual
avatar, as will be done by
for "The Light Princess".
Strength of the affect. How much does this affect tend to outweigh others, gain
the characters' attention, weigh against other affects in determining
etc. Fourth component of affect. Symbol: I.
An item is just an inanimate object represented in the game, like a rock or
a fork, or a rose.
"Mach" or "Machiavellian quotient" is a term from pop psychology literature referring
to how manipulative a person is. Do you smile because you are happy, or because you want
the person you're smiling at to give you a job? Infants are low-mach, politicians are
high-mach. Everybody else is somewhere in between. In Emerald, mach is an emotional
state preferred by the planning engine to achieve a given goal, by contrast with
mood. Which rules the character's expression is determined by the
Term from mathematics for the formula used to compute distance. For example, we
are used to the "space metric" r = sqrt((x-x0)**2 + (y-y0)**2 + (z-z0)**2), but there
is also the "taxicab metric" r = (abs(x-x0) + abs(y-y0) + abs(z-z0)). A metric
is usually constructed to model the amount of work required to get from one place to another,
so a taxicab metric measures the time/distance a car will actually have to travel through a
grid of streets. Choosing an accurate metric can be complicated. In Emerald, we use three
major metrics: physical, sensual,
and topical, with the additional point that we are usually interested in
the subjective metric, not the objective one.
Emotive sum, moderated by time-based relaxation to agent's
Any model object about which the character might have an emotional reaction or
affect (probably all game objects plus some abstract pseudo-objects).
The game model of Python Universe Builder is the objective universe that our characters
live in. When they sense things, information passes from this objective world into the
character's subjective one.
- Physical Metric
Distance to an object. If the object is in the current room, this is a number in the
range [0.-1), with the exact number possibly being determined by a spatial stage to the
game engine (as in Universe/"The Light Princess"), or it may be predetermined. Integer
values, 1,2,3... indicate how many rooms (or containers) away the object
is. Or to put it another way, how many turns are required to be in a position to "look"
at the object?
Emerald's planning engine, based on the ideas in Hap and the Cyphesis Mind system from
The person playing the game does so through controlling a special player character,
which may be thought of as a special case of a character agent, in which the human interface
replaces the agent.
Positive proximation means an impulse to move towards a thing, negative means to
move away. By "move towards" or "move away", we mean to decrease or increase the
metric distance between the agent and
object in physical,
sensual, or topical metrics. Called
"approach" in the literature. Second component of affect.
Distance as determined by the physical metric only.
Physical and/or sensual nearness of a object to an
agent. How much does the
object impinge on the character's senses? Is it an invasive/obvious sensation that the
agent receives immediately, or is it something the character must actively seek out? Is it
in this room or the next? Or somewhere even further away?
The degree that a character's mood will respond to stimuli. Expressed as a set of three
values, associated with proximal, visible,
and topical reactivity. A calm, level-headed character is very
unreactive, while a flighty character may be highly reactive. Not to be confused with
In interactive fiction / text adventure game models such as Python Universe Builder,
the world is model as a set of rooms with connections between them. Sometimes these
represent real-world rooms with the connections being "doors", but sometimes they are
just outside places and the doors simply represent travelling across an imaginary
boundary. A room is modelled as a fairly basic "container" object which holds other
objects including items and characters.
A property of an object with respect to an
agent, which may be regarded as
its "psychological proximity". Saliency is a combination of physical,
sensual, and topical metrics using a
formula determined by the agent's reactivity vector.
Saliency is used as weight and as a culling-metric in summing agent states
to compute mood.
Distance as determined by the sensual metric. How easy is the
thing to sense?
- Sensual Metric
I've been using "look" to indicate sensing the object, but of course, characters may "look",
"listen", "smell", "feel", and "taste". The sensual metric indicates how difficult (or
easy) it is to sense the object. In addition to various modes of sensing, there is also the
matter of size, intensity, loudness, etc. Some things are easier to see than others.
Characters are not omniscient. When a character learns more about the game world, it acquires
subjective knowledge about it (such as what room to find an
- Topical Metric
Sentences and conversations have a topical metric relating the importance of words in the
present utterance and previous utterances. Values in [0.-1) refer to the
last utterance, with exact value weighted according to the sentence structure,
while [1,2,3,..) refer to previous utterances.
Distance as determined by the topical metric. How pertinent is
the object to the present conversation or utterance.
Throughout the Emerald design, we'll assume turn-based character play: agents (and the player)
are allowed to make one "move" (change of the topological world model) per turn. Just as the
player is allowed time to think about their move, the agents in the game each get a chance to
"think" on each turn. Additionally, a character in the same room as the player may have some
additional real-time behavior, possibly initiating a turn-event just as the character can do.
Positive or negative feeling about the object. Is it good or bad?
Third component of affect. Symbol: V.
XML File Formats
Another change we need to apply is using an XML-based
unified language for describing characters. The Oz Project
apparently used a custom language grammar for this. We
want ours to be XML compliant and be embedable with the
rest of the game design files.
The DTD for this block (character) must encompass at least:
Emotion (for Em analog).
- attitudes (towards other characters and objects)
- standards (conduct standards applied to self and others
- ...? (review Em architecture further)
Planning (for Hap) analog).
- plan library (contains all possible plans?)
- actions (primitive actions)
- lines or expressions (for output to PUB/AutoManga layers)
Resources (for AutoManga output layers).
- Walk Cels (for sprite walk cycle, etc.)
- Expression Cels (for close-ups / reaction shots)
- Cutaways (various scripted inserts, selected to show actions).