Emotive Character Agents for Python Universe Builder

Terry Hancock, 2005-8/8
Project services hosted by:
SourceForge.net Logo


Emerald will be a class-libreary for creating character agents for use with Python Universe Builder to create richer, more believeble characters. It is inspired by a reading of the papers from the Carnegie Mellon University's "Oz Project" (see my brief review below, with links to the various papers), combined with a simple, four-parameter emotion model that has some computational advantages over the word-based model used in "Em" (the emotion engine used in the Oz project).

Whereas the Oz project was implemented in a combination of C and Lisp, and never produced published code, Emerald will use Python with the possibility of extension with Pyrex (in case the emotion math turns out to require it), and will of course be open source.

This page is pretty incomplete at this time (2005-8/8), and I'm not going to spend too much time on it just yet. For me, this is in support of The Light Princess, and it isn't critical for some time. But I wanted to express my intent, and publish some of the ideas that I've been working on. I'll periodically update this page as I continue to develop the concept. At this time, what I have is an experimental emotion model, which needs to be brought up to date (I wrote it when I was just learning Python, and it's apparently rather poorly written). The original can be found in the CVS for The Light Princess until I get the update loaded here.

Oz Project Review

Review of the Oz Project at Carnegie Mellon University

In the 1990's, a group at CMU developed a game engine very much like what we wanted to have for "The Light Princess". It incorporated relatively simple, but believeable character agents, and a drama planning mechanism that made for much more "believeable" and entertaining (if not necessarily more realistic) game play.
Oz Architecture -- Interactive Fiction Engine
The Oz architecture is illustrated below (after the illustration in the Hap architecture paper):

The "physical world" is a very simple topological model, almost exactly like what we find in Interactive Fiction games, so it seems that Python Universe Builder (PUB) which is based on the same kind of model, should be a good replacement. This is good, because the world model was said to constitute about 85% of the code in the Oz Architecture.

For more detail and context, please see:
An Oz-Centric Review of Interactive Drama and Believable Agents, Michael Mateas, June 1997, CMU-CS-97-156 (HTML).

Tok Architecture -- Character Agents
The agent behavior represents most of the functionality of Tok. Here is a diagram of the Tok architecture, showing how it relates to the two main components which comprise it, namely "Em" and "Hap" (from the Tok Architecture paper):

For more detail, see the original paper:
An Architecture for Action, Emotion, and Social Behavior, Joseph Bates, A. Bryan Loyall, W. Scott Reilly, May 1992, CMU-CS-92-144 (Gzipped Postscript).

Hap Architecture -- Goal Seeking
Hap is the side of the Tok architecture that attempts to implement goals and determine success or failure of plans to achieve its goals. It is composed of two primary structures: an "Active Plan Tree" and a "Plan Memory" and interacts with its environment (represented by an Interactive Fiction world) and with the companion "Em" module, which represents emotional and social behavior. Here is a diagram of the Hap architecture, based on the Oz papers:

Details on the architecture can be found in the original paper, which I highly recommend reading:
Hap, A Reactive, Adaptive Architecture for Agents, A. Bryan Loyall, Joseph Bates, June 1991, CMU-CS-91-147 (Gzipped Postscript).

Em Architecture -- Emotions
Em is the emotional subsystem of Tok. I'm less sure of the maturity of this system, and, given the need to re-implement the code, I'm considering using my original emotive behavior architecture instead (documented in the existing CVS for The Light Princess), but Em should be adequate for our game, so it's possible that we should simply implement it as it was. This may be simpler in the end.

To give an idea of the level of complexity in Em, here's a list of the emotions understood by the version of Em used to implement the "Lyotard" game (in which the agent modeled a cat):

For more detail, see the original paper:
Building Emotional Agents, W. Scott Reilly, Joseph Bates, May 1992, CMU-CS-92-143 (Gzipped Postscript).
The Oz architecture seems to be an excellent fit to our needs. It demonstrates the feasibility of this approach, and has adequate documentation to completely re-implement the architecture in Python, which should be well-adapted to representing the structures and algorithms used. Tok originally consumed about 7500 lines of code (mostly in Lisp), and I think our implementation will probably be no larger.

The world model for Oz was implemented in Lisp, and is described in a way that sounds almost exactly like the model found in interactive fiction games made with Inform or TADS. Since the Python Universe Builder (PUB) was also built to follow this model, it should be an excellent fit. Thus we should be able to implement almost the entire engine in Python.

The Oz Project produced a lot of papers and online information as well. I have selected a few key papers to illustrate the architecture as concisely as possible. However, a visit to the original Oz Project Website will be well worth the trouble, if you are interested in this project. There are many more papers available from there (and also the uncompressed forms of the papers above, in case you have trouble uncompressing these).

Review of WorldForge's Cyphesis Mind System

Another influence on our design should probably come from the (working) NPC agents in the WorldForge system. These agents are not emotive, but are purely heuristic, based on knowledge available to them, and innate drives. They might be described as "behavior-based", but whether this is truly a "subsumption architecture" is not immediately clear to me from reading the source. Readers coming from PUB may be interested to know that the Cyphesis mind system is written in Python, even though the Cyphesis framework is in C/C++.

The main idea of interest here is the concept of "Knowledge" as a subjective model of the objective game model world, to be maintained by the character's "mind". The idea that characters should collecte this information from the world seems like a good one, although most of the characters in "The Light Princess" will already be highly embedded in their environments, so it's unclear whether I'll gain anything specific for that project over just using the objective model (which is equivalent to assuming that the characters have already "seen everything" in the model).

Emotion Model

The emotion model was adapted from psychological literature on understanding and modelling real emotions, in contrast to Em's approach, which is based on the use of emotional words. So, my model is more mathematical and abstract, using an ordered tuple of four numbers to represent an emotional state. It might therefore be considered part of the "color-model" class of emotion representations.

Emotion theory is full of highly conflicting constructs and terminology, because it is so immature, and researchers' ideas do not fit into any well-established system of theory. Nor is it easy to make empirical measurements of which system is more closely related to the way emotion really works. Worse, the jargon words in this field tend to be words we associate with normal English usage. Returning to the CMU Oz project's philosophy, though, I want to reiterate that we are only trying to achieve "believeability" in our characters, not trying to accurately create emotional machine intelligence!

In order to communicate, we need some standards for terminology (for extra clarity, Emerald and PUB jargon words are boldface to emphasize that we mean them in their technical sense):

The active-ness or aggressiveness of an emotional affect. A tendency to act rather than remain passive. First component of affect. Symbol: A.

In Emerald, the term "affect" will mean two things: most definitively it will mean a four-tuple representing the emotional reaction that a character feels in response to a certain object (which may be a concrete or abstract noun of any kind). In a broader sense, it will also mean the same four-tuple in any context. The four parameters in this model are: arousal, proximation, valence, and intensity.

An agent is just a game object in PUB, based on existing character objects, but with the ability to make decisions and moves. At each turn of the game, all agents are allowed to think and make one move, just like the player character can.

In the context of Python Universe Builder, we now have three major types of character: "player" (meaning that decisions are made according to input from the user), "state-machine" (meaning the character is a simple state machine as NPCs are in older IF engines), and "agent", meaning that the character's actions are controlled by an intelligent agent as we are trying to build with Emerald.

A constant property of a character agent which determines a prefered mood for that character. E.g. a brooding character will tend to be sad or unhappy, while an optimistic character will tend towards happy regardless of stimuli.

Property of a character that defines how much their expression is driven by mach versus mood.

Emotion is the name of the emotion model package as a whole, as distinguished from the Planner.

Emotive Sum
Ideally, a sum over objects, weighted by salience, of affect. Less, ideally, an incomplete sum we think approximates this well, which is limited to objects in the room and objects of high-saliency from previous computations. I thought seriously about calling this the jargon meaning of "Emotion", but I think it's wise to avoid using the term for anything so specific.

Expression is a sum of affect that drives the emotion the character communicates to others or expresses outwardly. This may be used to generate inflected text by choosing emotional adjectives and adverbs, or to drive a visual avatar, as will be done by Universe for "The Light Princess".
Strength of the affect. How much does this affect tend to outweigh others, gain the characters' attention, weigh against other affects in determining emotive sum, etc. Fourth component of affect. Symbol: I.

An item is just an inanimate object represented in the game, like a rock or a fork, or a rose.

"Mach" or "Machiavellian quotient" is a term from pop psychology literature referring to how manipulative a person is. Do you smile because you are happy, or because you want the person you're smiling at to give you a job? Infants are low-mach, politicians are high-mach. Everybody else is somewhere in between. In Emerald, mach is an emotional state preferred by the planning engine to achieve a given goal, by contrast with mood. Which rules the character's expression is determined by the character's duplicity.
Term from mathematics for the formula used to compute distance. For example, we are used to the "space metric" r = sqrt((x-x0)**2 + (y-y0)**2 + (z-z0)**2), but there is also the "taxicab metric" r = (abs(x-x0) + abs(y-y0) + abs(z-z0)). A metric is usually constructed to model the amount of work required to get from one place to another, so a taxicab metric measures the time/distance a car will actually have to travel through a grid of streets. Choosing an accurate metric can be complicated. In Emerald, we use three major metrics: physical, sensual, and topical, with the additional point that we are usually interested in the subjective metric, not the objective one.

Emotive sum, moderated by time-based relaxation to agent's disposition.

Any model object about which the character might have an emotional reaction or affect (probably all game objects plus some abstract pseudo-objects).

The game model of Python Universe Builder is the objective universe that our characters live in. When they sense things, information passes from this objective world into the character's subjective one.

Physical Metric
Distance to an object. If the object is in the current room, this is a number in the range [0.-1), with the exact number possibly being determined by a spatial stage to the game engine (as in Universe/"The Light Princess"), or it may be predetermined. Integer values, 1,2,3... indicate how many rooms (or containers) away the object is. Or to put it another way, how many turns are required to be in a position to "look" at the object?

Emerald's planning engine, based on the ideas in Hap and the Cyphesis Mind system from World Forge.

The person playing the game does so through controlling a special player character, which may be thought of as a special case of a character agent, in which the human interface replaces the agent.

Positive proximation means an impulse to move towards a thing, negative means to move away. By "move towards" or "move away", we mean to decrease or increase the metric distance between the agent and object in physical, sensual, or topical metrics. Called "approach" in the literature. Second component of affect. Symbol: P.

Distance as determined by the physical metric only. Physical and/or sensual nearness of a object to an agent. How much does the object impinge on the character's senses? Is it an invasive/obvious sensation that the agent receives immediately, or is it something the character must actively seek out? Is it in this room or the next? Or somewhere even further away?

The degree that a character's mood will respond to stimuli. Expressed as a set of three values, associated with proximal, visible, and topical reactivity. A calm, level-headed character is very unreactive, while a flighty character may be highly reactive. Not to be confused with duplicity, though!

In interactive fiction / text adventure game models such as Python Universe Builder, the world is model as a set of rooms with connections between them. Sometimes these represent real-world rooms with the connections being "doors", but sometimes they are just outside places and the doors simply represent travelling across an imaginary boundary. A room is modelled as a fairly basic "container" object which holds other objects including items and characters.

A property of an object with respect to an agent, which may be regarded as its "psychological proximity". Saliency is a combination of physical, sensual, and topical metrics using a formula determined by the agent's reactivity vector. Saliency is used as weight and as a culling-metric in summing agent states to compute mood.

Distance as determined by the sensual metric. How easy is the thing to sense?

Sensual Metric
I've been using "look" to indicate sensing the object, but of course, characters may "look", "listen", "smell", "feel", and "taste". The sensual metric indicates how difficult (or easy) it is to sense the object. In addition to various modes of sensing, there is also the matter of size, intensity, loudness, etc. Some things are easier to see than others.

Characters are not omniscient. When a character learns more about the game world, it acquires subjective knowledge about it (such as what room to find an item in.

Topical Metric
Sentences and conversations have a topical metric relating the importance of words in the present utterance and previous utterances. Values in [0.-1) refer to the last utterance, with exact value weighted according to the sentence structure, while [1,2,3,..) refer to previous utterances.

Distance as determined by the topical metric. How pertinent is the object to the present conversation or utterance.

Throughout the Emerald design, we'll assume turn-based character play: agents (and the player) are allowed to make one "move" (change of the topological world model) per turn. Just as the player is allowed time to think about their move, the agents in the game each get a chance to "think" on each turn. Additionally, a character in the same room as the player may have some additional real-time behavior, possibly initiating a turn-event just as the character can do.

Positive or negative feeling about the object. Is it good or bad? Third component of affect. Symbol: V.

XML File Formats

Another change we need to apply is using an XML-based unified language for describing characters. The Oz Project apparently used a custom language grammar for this. We want ours to be XML compliant and be embedable with the rest of the game design files.

The DTD for this block (character) must encompass at least: