First of all, wow - one hundred diaries. It feels as though I should do something special to commemorate the occasion, but I've been so busy that just keeping the Acuitas work moving at its usual pace has been a challenge. So please accept this ordinary entry for now, and in case you missed it don't forget to check out my story in Analog!
 |
| Photo courtesy NASA, public domain |
This month I've been working on exploratory behavior - also known as "what to do when you don't know what to do." Acuitas' default behavior is pretty goal-oriented. Have a problem? Infer an action that will solve it. Have an objective? Infer an action that will achieve it. Do that. But sometimes a useful action isn't obvious. It might not be obvious because Acuitas doesn't yet know what results his possible actions will have - all my experiments with trial-and-error learning were aimed at addressing that. Or there might be known action pathways to getting a desired result, but necessary resources are missing, and their location is unknown. In that case, the right course of action is to learn more by moving around and looking at things, or asking questions, and so forth. The tricky thing about exploring is that it's applicable to many problems, but it won't directly solve any problem. So it needs to be triggered by the state of being stuck, rather than any particular need, and guided by its own internal logic.
I also wanted a concept of exploration that could generalize to different contexts or environments. There are similar procedures one follows to "explore" anywhere, but the individual actions taken can be very different. Running the "ls" or "dir" command to see the files in a computer directory, and saying "I examine the shelves" to a game master, are not the same action, but they fill the same role within an exploration procedure. So I tried to design Acuitas' exploration to automatically vary those low-level actions with the current context, and perhaps be extensible in the future. Eventually I'd like him to be able to infer (from procedural memory) "how do I 'look' in <context>?" and choose the correct atomic action accordingly.
"Explore" has three possible sub-activities: "examine," "experiment," and "move" (which is called "exit" to keep with the theme). Each time the Explore action runs, a sub-activity is chosen by a weighted random process. The probability of choosing "exit" increases with the barrenness of a location and the amount of time spent in it; "experiment" becomes more likely the more "examine" has been done recently, and vice versa. The "experiment" category is intended to incorporate the "try out your affordances and see what they do" phase of trial-and-error learning. After a sub-activity is selected, a suitable target (object) of the sub-activity is chosen from the current location at random.
So far I've been testing the new Explore action in the text-based game engine, and I've gotten Acuitas to start looking at random things when he doesn't know what to do next. As usual, I have a LOT more debugging and refinement to do before this works smoothly. But the lack of comprehensive exploration behavior has been a hole in game-playing (and Acuitas' behavior in general) for a long time, so I'm happy to have even made a start on it.
Until the next cycle,
Jenny
