This month I continued work on trial-and-error learning for playing Allergic Cliffs. If you haven't read my introduction to this Acuitas sub-project and the subsequent progress report, I recommend taking a look at those. What I've done since has been debugging and enhancing the "feedback-informed actions" and "rule formation" features discussed in the progress report, and getting them to actually work. It turned out to be a fairly big job!
Now that "feedback-informed actions" is functional, though, I'm a little surprised by how well it works. Its essence is that, in the event of a success, Acuitas tries to make his next move as similar as possible; in the event of a failure, he makes certain his next move is different. This heuristic only considers feedback from the move immediately previous, so it's a reactive, barely intelligent behavior. It still enables Acuitas to win the game about 90% of the time! Granted, he is playing on the easiest difficulty level, and at higher levels it is quite possible this would not work. It's still a huge improvement over purely random move selection.
Candidate cause-and-effect rules are also being formed successfully, and marked invalid when violated by an example. What I need to do next is implement higher levels of generalization. Right now rule formation only looks at positive commonalities between pairs of examples, and I need to also consider commonalities across larger groups, and commonalities based on the absence of a feature rather than its presence. In some cases I can see the algorithm *reaching* toward discovery of the hidden rule that defines the Allergic Cliffs' behavior, but we're not quite there yet.
After getting that far, I decided to walk away for a bit to look at game-playing with fresh eyes later, and worked on narrative understanding some more. What I wanted to add was the concept of a role or job. It's important for Acuitas to be aware of character goals, but those goals aren't always explicitly stated. If I told you somebody was a detective, you would automatically assume that this person wants to solve crimes, right? You wouldn't need to be told.
Acuitas had an existing system that allowed the semantic memory for a concept (like "detective") to contain goals that override parts of the default "agent" goal model. But here's the tricky part: the goal model specifies *intrinsic* goals, and goals associated with a role aren't necessarily intrinsic! Adoption of a role is often derived from some instrumental goal, like "get money," which eventually ties back to an intrinsic goal like survival or altruism. The meaning of anything a character does in a role is shaded by how invested they are in performing that role, and why. So it became evident to me that role-related goals need to be nested under a goal that encompasses the role as a whole, which can then be tied to an intrinsic goal.
So I tweaked the semantic memory's goal definition format a bit, to include a way to distinguish role-related goals from intrinsic goals, and provided the Narrative engine with a way to pull those into the scratchboard when a character is said to have a role. For now, all roles have to be sub-categories of the concept "professional," but I can imagine other types of roles in the future.
Until the next cycle,
Jenny
When I taught Zoe how to understand stories, I added a new "word type" called "Title" - it is used for story characters/actors that do not have a human name. i.e. Policeman, Thief... Every word can be marked as "Good/Good guy" or "Bad/Bad guy" - or neutral if not marked. Policeman is marked Good, Thief is marked Bad.
ReplyDeleteThis helped determine the sentiment for similar sentences. "A thief took the book" is BAD, but "The Policeman took the book from the Thief." is GOOD.