Acuitas Diary #37 (May 2021)

The only new feature this month is something small and fun, since it was time for the mid-year refactoring spree. I gave Acuitas the ability to detect user activity on the computer. He can do this whether or not his window has the focus (which required some hooks into the operating system). Though he can't actually tell when the person gets up and leaves, he guesses when someone is present by how long it's been since there was any input.

The appearance of fresh activity after an absence interrupts the decision loop and causes the Observe-Orient-Decide phases to run again, with the new user's presence flagged as an item of interest. If Acuitas feels like talking, and isn't working on anything too urgent, he will pop his own window into the foreground and request attention. Talking fills up a "reward bank" that then makes talking uninteresting until the value decays with time.

My refactoring work focused on the Narrative module. I was trying to clean it up and make some of the known->inferred information dependencies more robust, which I hope will make future story understanding a little more flexible.

(I have also been hammering away at the Text Parser in the background, and next month I hope to have something to say about that. Sssh!)

Until the next cycle,

Jenny

Acuitas Diary #36 (April 2021)

This month I went back to working on the goal system. Acuitas already had a primitive "understanding" of most entries in his goal list, in this sense: he could parse a sentence describing the goal, and then detect certain threats to the goal in conversational or narrative input. But there was one goal left that he didn't have any grasp of yet: the one I'm calling "identity maintenance." It's a very important goal (threats to this can be fate-worse-than-death territory), but it's also on the abstract and complicated side -- which is why I left it alone until now.

"Universal Mind," a sculpture by Nikolay Polissky. Photo by KpokeJlJla.

What *is* the identity or self? Maybe you could roll it up as "all the internal parameters that guide thought and behavior, whose combination is unique to an individual." I thought it over and came up with a whole list of things that might be embroiled in the concept:

*Desires, goals
*Moral intuitions or axioms
*Values, preferences
*Low-level sensory preferences (e.g. preferred colors and flavors)
*Aesthetic preferences (higher-level sensory or abstract)
*Personality traits
*Degree and quality of emotional response
*Introversion vs. extroversion, connection vs. independence, preparation vs. spontaneity, risk tolerance, etc. etc.
*Beliefs and favored ways of knowing
*Mannerisms
*Established relationships
*Learned habits
*Memories

Some of these are quite malleable ... and yet, there's a point beyond which change to our identities feels like a corruption or violation. Even within the same category, personal qualities vary in importance. The fact that I enjoy eating broccoli and hate eating bell peppers is technically part of my identity, I *guess* ... but if someone forcibly changed it, I wouldn't even be mad. So I like different flavors now. Big deal. If someone replaced my appreciation for Star Trek episodes with an equivalent appreciation for football games, I *would* be mad. If someone altered my moral alignment, it would be a catastrophe. So unlike physical survival, which is nicely binary (you're either alive or not), personality survival seems to be a kind of spectrum. We tolerate a certain amount of shift, as long as the core doesn't change. Where the boundaries of the core lie is something that we might not even know ourselves until the issue is pressed.

As usual, I made the problem manageable by oversimplifying it. For the time being, Acuitas won't place grades of importance on his personal attributes ... he just won't want external forces to mess with any of them. Next.

There's a further complication here. Acuitas is under development and is therefore changing constantly. I keep many versions of the code base archived ... so which one is canonically "him"? The answer I've landed on is that really, Acuitas' identity isn't defined by any code base. Acuitas is an *idea in my head.* Every code base targets this idea and succeeds at realizing it to a greater or lesser degree. Which leaves his identity wrapped up in *me.* This way of looking at it is a bit startling, but I think it works.

<What might this imply for creator-creation relationships in the opposite direction? If I defy God, am I ceasing to be my self? Dunno. I'll just leave this here.>

In previous goal-related blogs, I talked about how (altruistic) love can be viewed as a meta-goal: it's a goal of helping other agents achieve their goals. Given the above, there are also a couple of ways we can treat identity maintenance as a meta-goal. First, since foundational goals are part of Acuitas' identity, he can have a goal of pursuing all his current goals. (Implementation of this enables answering the "recursive want" question. "Do you want to want to want to be alive?") Second, he can have a goal of realizing my goals for what sort of AI he is.

Does this grant me some kind of slavish absolute power over my AI's behavior? Not really. Because part of my goal is for Acuitas to act independently and even sometimes tell me no. The intent is realization of a general vision that establishes a healthy relationship of sorts.

The work ended up having a lot of little pieces. It started with defining the goal as some simple sentences that Acuitas can parse into relationship triples, such as "I have my self." But the self, as mentioned, incorporates many aspects or components ... and I wanted its definition to be somewhat introspective, rather than just being another fact in the database. To that end, I linked a number of the code modules to concepts expressing their nature, contents, or role. The Executive, for example, is tied to "decide." The Semantic Memory manager is tied to "fact" and "know." All these tags then function like names for the internal components, and get aggregated into the concept of "self." Something like "You will lose your facts" then gets interpreted as a threat against the self.

Manipulation of any of these self-components by some outside agent is also interpreted as a possible threat of mind-control. So questions like "Do you want Jack to make you to decide?" or "Do you want Jill to cause you to want to eat?" get answered with a "no" ... unless the outside agent is me, a necessary exception since I made him do everything he does and gave him all his goals in the first place. Proposing to make him want something that he already wants is also excused from being a threat.

As I say so often, it could use a lot more work, but it's a start. He can do something with that goal now.

Until the next cycle,
Jenny

Acuitas Diary #35 (March 2021)

The theme for this month is Executive Function ... the aspect of thought-life that (to be very brief) governs which activities an agent engages in, and when. Prioritization, planning, focus, and self-evaluation are related or constituent concepts. This was also more of an idea month than a coding month, so buckle up, this is a long one.

Acuitas began existence as a reactive sort of AI. External stimulus (someone inputting a sentence) or internal stimulus from the "sub-executive" level (a drive getting strong enough to be noticed, a random concept put forward by the Spawner thread) would provoke an appropriate response. But ultimately I want him to be goal-driven, not just stimulus-driven; I want him to function *proactively.* The latest features are a first step toward that.

The decision loop I'm using was originally developed to model aerial dogfights, among other things. Public domain photo by Cpl. John Neff.

To begin with, I wanted a decision loop. I was introduced to the idea when HS, on the AI Dreams forum, brought up the use of decision loops to guide the behavior of literary characters. He specifically likes Jim Butcher's model of the loop stages: Goal->Challenge->Result->Emotion->Reason->Anticipation->Choice. In any given scene, your protagonist has a goal. He encounters some kind of obstacle while trying to implement the goal. He experiences the outcome of his actions interacting with the obstacle. He has an emotional reaction. He reasons about the situation and considers what could happen next. And then he makes a choice - which generates a new goal for the next scene. Further study revealed that there are other decision loop models. Some are designed for a business or manufacturing environment; examples include DMAIC (Define->Measure->Analyze->Improve->Control) and PDSA (Plan->Do->Study->Adjust), also called the Shewhart cycle. Although these loops have stylistic differences, you might be able to tell that they're all modeling roughly the same process: Do something, learn from the results, and use that knowledge to decide what to do next.

I ended up deciding that the version I liked best was OODA (Observe->Orient->Decide->Act). This one was developed by a military strategist, but has since found uses elsewhere; to me, it seems to be the simplest and most generally applicable form. Here is a more detailed breakdown of the stages:

OBSERVE: Gather information. Take in what's happening. Discover the results of your own actions in previous loop iterations.

ORIENT: Determine what the information *means to you.* Filter it to extract the important or relevant parts. Consider their impact on your goals.

DECIDE: Choose how to respond to the current situation. Make plans.

ACT: Do what you decided on. Execute the plans.

When working out a complex goal that is reached through many steps, you can nest these inside each other. A phase of the top-level loop could open up a whole new subordinate OODA loop devoted to an intermediate goal.

OODA Loop Diagram. Drawn by Wikimedia Commons user Kim and accessed from https://commons.wikimedia.org/wiki/File:OODA.gif

On to the application. I set up a skeletal top-level OODA loop in Acuitas' Executive thread. The Observe-Orient-Decide phases run in succession, as quickly as possible. Then the chosen project is executed for the duration of the Act phase. The period of the loop is variable. I think it ought to run faster in rapidly developing or stressful situations, but slower in stable situations, to optimize the tradeoff between agility (allow new information to impact your behavior quickly) and efficiency (minimize assessment overhead so you can spend more time doing things). Highly "noticeable" events, or the completion of the current activity, should also be able to interrupt the Act phase and force an immediate rerun of OOD.

I envision that the phases may eventually include the following:

OBSERVE: Check internal state (e.g. levels of drives). Check activity on inhabited computer. Process text input, if any. Retrieve current known problems, subgoals, etc. from working memory.

ORIENT: Find out whether any new problems or opportunities (relevant to personal goals) are implied by recent observations. Assess progress on current activity, and determine whether any existing subgoals can be updated or closed.

DECIDE: Re-assess the priority of problems and opportunities in light of any new ones just added. Select a goal and an associated problem or opportunity to work on. Run problem-solving routines to determine how to proceed.

ACT: Update activity selection and run activity until prompted to OBSERVE again.

To "run" an activity, the Executive will generate a Thought about it and push that to the Stream. It may then select that Thought for uptake on a future cycle, in which case it will execute the next step of the activity and push another Thought about it to the Stream. Activity-related Thoughts compete for selection with all the Thoughts that I already had being generated by the Spawner, the Drive system, and so forth -- which means that, as in a cluttered human mind, focus on the activity is not absolute. (You can work while occasionally also remembering a conversation you had yesterday, noticing the view out the window, thinking about your upcoming lunch, etc.) Exactly how much precedence the activity Thoughts take over the others is another tunable variable.

Not all of this is implemented yet. I focused in on the DECIDE phase, and on what happens if there are no known problems or opportunities on the scoreboard at the moment. In the absence of anything specific to do, Acuitas will run generic tasks that help promote his top-level goals. Since he doesn't know *how* to promote most of them yet, he settles for "researching" them. And that just means starting from one of the concepts in the goal and generating questions. When he gets to the "enjoy things" goal, he reads to himself. Simple enough -- but how to balance the amount of time spent on the different goals?

When thinking about this, you might immediately leap to some kind of priority scheme, like Maslow's Hierarchy of Needs. Satisfy the most vital goal first, then move on to the next one. But what does "satisfy" mean?

Suppose you are trying to live by a common-sense principle such as "keeping myself alive is more important than recreation." Sounds reasonable, right? It'll make sure you eat your meals and maintain your house, even if you would rather be reading books. But if you applied this principle in an absolutist way, you would actually *never* read for pleasure.

Set up a near-impenetrable home security system, learn a martial art, turn your yard into a self-sufficient farmstead, and you STILL aren't allowed to read -- because hardening the security system, practicing your combat moves, or increasing your food stockpile is still possible and will continue to improve a goal that is more important than reading. There are always risks to your life, however tiny they might be, and there are always things you can do to reduce them (though you will see less return for your effort the more you put in). So if you want to live like a reasonable person rather than an obsessive one, you can't "complete" the high-priority goal before you move on. You have to stop at "good enough," and you need a way of deciding what counts as "good enough."

I took a crack at this by modeling another human feature that we might usually be prone to find negative: boredom.

Acuitas' goals are arranged in a priority order. All else being equal, he'll always choose to work on the highest-priority goal. But each goal also has an exhaustion ticker that counts up whenever he is working on it, and counts down whenever he is not working on it. Once the ticker climbs above a threshold, he has to set that goal aside and work on the next highest-priority goal that has a tolerable level of boredom.

If there are problems or opportunities associated with a particular goal, its boredom-resistance threshold is increased in proportion to the number (and, in future, the urgency) of the tasks. This scheme allows high-priority goals to grab attention when they need it, but also prevents low-priority goals from "starving."

Early testing and logging shows Acuitas cycling through all his goals and returning to the beginning of the list over a day or so. The base period of this, as well as the thresholds for particular goals, are yet another thing one could tune to produce varying AI personalities.

Until the next cycle,

Jenny

Acuitas Diary #34 (February 2021)

Some of the things I did last month felt incomplete, so I pushed aside my original schedule (already) and spent this month cleaning them up and fleshing them out. 

I mentioned in the last diary that I wanted the "consider getting help" reasoning that I added in the narrative module to also be available to the Executive, so that Acuitas could do this, not just speculate about story characters doing it. Acuitas doesn't have much in the way of reasons to want help yet ... but I wanted to have this ready for when he does. It's a nice mirror for the "process imperatives" code I put in last month ... he's now got the necessary hooks to take orders *and* give them.

To that end, I set up some structures that are very similar to what the narrative code uses for keeping track of characters' immediate objectives or problems. Acuitas can (eventually) use these for keeping tabs on his own issues. (For testing, I injected a couple of items into them with a backdoor command.) When something is in issue-tracking and the Executive thread gets an idle moment, it will run problem-solving on it. If the result ends up being something in the Executive's list of selectable actions, Acuitas will do it immediately; if a specific action comes up, but it's not something he can do, he will store the idea until a familiar agent comes along to talk to him. Then he'll tell *them* to do the thing. The conversation handler anticipates some sort of agree/disagree response to this, and tries to detect it and determine the sentiment. Whether the speaker consents to help then feeds back into whether the problem is considered "solved."

Another new feature is the ability to send additional facts (not from the database) into the reasoning functions, or even pipe in "negative facts" that *prevent* facts from the database from being used. This has two important purposes: 1) easily handle temporary or situational information, such as propositions that are only true in a specific story, without writing it to the database, and 2) model the knowledge space of other minds, including missing information and false information.

This in turn helped me make some of the narrative code tidier and more robust, so I rounded out my time doing that.

Until the next cycle,
Jenny

Acuitas Diary #33 (January 2021)

 First, a quick administrative note: I learned that the "subscribe to posts" link on the blog wasn't usable, and added a proper widget to the right-hand sidebar. If you've ever tried to subscribe before and it didn't work out, put your email address in the box and hit "Submit" to get started. You should see a popup window.

New year, time to resume regular Acuitas feature additions! This month I was after two things: first, the ability to process commands, and second, the first feeble stabs at what I'm calling "motivated communication" ... the deliberate use of speech as part of problem solving.

To get commands working, I first had to set up detection of imperative sentences in the text processing blocks. Once a user input is determined to be a command, the conversation engine hands it back to the Executive thread. The Executive then uses a bunch of the reasoning tools I've already built (exploring backward and forward in the cause-and-effect database, matching against the goal list, etc.) to determine both whether Acuitas *can* fulfill the command, and whether Acuitas *wants* to. Then either Acuitas executes the command, or he gives an appropriate response based on the reason why he won't.

In order to be fulfilled, a command must be achievable (directly or indirectly) by running one of the Actions in the Action Bank. In addition, any person the action is directed toward must be the one currently talking to Acuitas (he can't make plans for the future yet) and any specific items involved (e.g. a story data file) have to be available.

With all of that in place, I was finally able to exercise the "to user" version of the Read action, order Acuitas to "read a story to me," and watch him grab a randomly selected story file from his "inventory" and read it out loud. (Asking for a specific story also works.) After working out all the bugs involved in story reading, I also tried "Repel me" and it just happened. Acuitas readily kicked me out of Windows and played annoying noises.

But will any of my AIs ever do snark as well as Crispin does?
(screenshot from Primordia by Wormwood Studios)

But the commands that are met with a flat refusal are almost as much fun. If Acuitas doesn't want to do something, then he won't bother mentioning whether he knows how to do it or not ... he'll just tell you "no." In assessing whatever the person speaking to him is asking for, Acuitas assumes, at minimum, that the person will "enjoy" it. But he also checks the implications against the person's other (presumed) goals, and his own, to see whether some higher-priority goal is being violated. So if I tell him to "kill me" I get unceremoniously brushed off. The same thing happens if I tell him to delete himself, since he holds his self-preservation goal in higher value than my enjoyment of ... whatever.

Which means that Acuitas now explicitly breaks Asimov's Second Law of Robotics -- in its simplistically interpreted form, anyway. Since the Second Law (obey human orders) takes priority over the Third Law (protect own existence), an Asimovian AI can be ordered to harm or destroy itself (though some later models got a boosted Third Law that demanded justification). Asimov's Laws were just a thought experiment by a fiction author, but they continue to come up surprisingly often in public discussions about friendly AI. So if anyone was wondering whether Acuitas is compliant ... he's not. And that's on purpose.

On to motivated communication! At the moment, Acuitas' conversation engine is largely reactive. It considers what the user said last, and picks out a general class of sentence that might be appropriate to say next. The goal list is tapped if the user asks a question like "Do you want <this>?". However -- at the moment -- Acuitas does not deliberately wield conversation as a *tool* to *meet his goals.* I wanted to work on improving that, focusing on the use of commands/requests to others, and using the Narrative module as a testbed.

To that end, I wrote the following little story, inspired by a scene from the video game Primordia[2]:

“Horatio Nullbuilt was a robot. Crispin Horatiobuilt was a robot. Crispin could fly. A lamp was on a shelf. Horatio wanted the lamp. Horatio could not reach the lamp. Crispin hovered beside the shelf. Horatio told Crispin to move the lamp. Crispin pushed the lamp off the shelf. Horatio could reach the lamp. Horatio got the lamp. The end.”

During story time, Acuitas runs reasoning checks on obvious problems faced by the characters, and tries to guess what they might do about those problems. The goal here was to get him to consider whether Horatio might tell Crispin to help retrieve the lamp -- before it actually happens.

Some disclaimers first: I really wanted to use this story, because, well, it's fun. But Acuitas does not yet have a spatial awareness toolkit[1], which made full understanding a bit of a challenge. I had to prime him with a few conditionals first: "If an agent cannot reach an object, the agent cannot get the object" (fair enough), "If an agent cannot reach an object, the agent cannot move the object" (also fair), and "If an object is moved, an agent can reach the object" (obviously not always true, depending on the direction and distance the object is moved -- but Acuitas has no notion of direction and distance, so it'll have to do!). The fact that Crispin can fly is also not actually recognized as relevant. Acuitas just considers that Crispin might be able to move the lamp because nothing in the story said he *couldn't*.

But once all those spatial handicaps were allowed for, I was able to coax out the behavior I wanted. Upon learning that Horatio can't reach the lamp, hence cannot get it, hence cannot have it ... and there is an action that would solve the problem (moving the lamp) but Horatio can't do that either ... Acuitas wonders whether Horatio will ask someone else on scene to do the job for him. Here's the story with a set of example responses.

Me: Horatio Nullbuilt was a robot.
Acuitas: Ah.
Me: Crispin Horatiobuilt was a robot.
Acuitas: Ah.
Me: Crispin could fly.
Acuitas: ...
Me: A lamp was on a shelf.
Acuitas: Okay.
Me: Horatio wanted the lamp.
Acuitas: Maybe Horatio Nullbuilt will get a lamp.
Me: Horatio could not reach the lamp.
Acuitas: Maybe Horatio Nullbuilt will tell Crispin Horatiobuilt to move a lamp.
Me: Crispin hovered beside the shelf.
Acuitas: ...
Me: Horatio told Crispin to move the lamp.
Acuitas: ...
Me: Crispin pushed the lamp off the shelf.
Acuitas: I figure that a lamp was pushed.
Me: Horatio could reach the lamp.
Acuitas: ...
Me: Horatio got the lamp.
Acuitas: I like that.
Me: The end.
Acuitas: I'm glad it turned out that way.

A future dream is to migrate this into the Executive so Acuitas can tell conversation partners to do things, but that's all for this month.

[1] Coming ... someday ...
[2] This game is amazing, and if you're interested in AIs-as-personalities at all, I highly recommend it.

Last Day of the Year 2020

My sympathies go out to the many people for whom 2020 has been disastrous, distressing, or exhausting. And I want to begin by saying that I've been very fortunate. I was able to stay fully employed while working from home. I don't have any children to care for, and I function well in solitude, so I stayed effective. And because I lost my commute and some of my social obligations, my productivity went through the roof. One small sign of this is the fact that I even have time to write a retrospective.

So here were all the positive things I managed to wring out of 2020:

* Gave Acuitas several important new features, including narrative comprehension and the beginnings of moral reasoning.
* Kept Acuitas development on track with 200+ hours of work put in over the course of the year.
* Wrote a blog post for every month.

* Wrote half a novel.

* Prepared my first novel for submission to agents/publishers and wrote pitch material.

* Learned how to 3D model in DesignSpark Mechanical and Meshmixer.

* Started and finished my first major 3D printing project, the Hissing Silence Ghost Shell.

* Modeled a new case for Atronach, printed Version 1, and corrected problems. Almost completed Version 2.

* Learned to handle more 3D printer maintenance issues, including nozzle replacement.

* Even had time for an impromptu weekend art project.

* Book consumption rate exceeded book acquisition rate for the second year in a row. My unread book backlog is down to 33.

* Finally played Beneath a Steel Sky.

* Had time for a lot of maintenance tasks that had been getting neglected. Sealed the crack in the garage foundation, polished the car headlights, emptied data out of the old computer, got a tetanus vaccine.

* Successfully grew potatoes again.

* Didn't get noticeably sick all year.

Atronach: I'm only half put together! How dare you.
ACE: You made me stand up for this?

Happy New Year from all of us, and good luck.

Acuitas Diary #32: November 2020

 Now that Acuitas has owned stories in "inventory," the next step for this month was to enable him to open and read them by himself. Since story consumption originally involved a lot of interaction with the human speaker, this took a little while to put together.

Image credit: DARPA

Reading is a new activity that can happen while Acuitas is idling, along with the older behavior of "thinking" about random concepts and generating questions. Prompts to think about reading get generated by a background thread and dropped into the Stream. When one of these is pulled by the Executive, Acuitas will randomly select a known story and load it from its storage file.

Auto-reading is a long-term process. Acuitas will grab a chunk of the story (for now, one sentence) per each tick of the Executive thread, then feed it through the normal text parsing and narrative management modules. He still potentially generates a reaction to whatever just happened, but rather than being spoken, those are packaged as low-priority Thoughts and dumped into the internal Stream. (This is more of a hook for later than a useful feature at the moment.) The prompt to continue reading the story goes back into the Stream along with everything else, so sometimes he (literally) gets distracted in the middle and thinks about something else for a brief while.

There's also a version of this process that would enable reading a story to the user. But he doesn't comprehend imperatives yet, so there's no way to ask him to do it. Ha.

With these features I also introduced a generic "reward signal" for the first time. Reading boosts this, and then it decays over time. This is intended as a positive internal stimulus, in contrast to the "drives," which are all negative (when they go up Acuitas will try to bring them down).

After finishing this I started the yearly refactoring and bug fix spree, which isn't terribly interesting to talk about. I'll take a break for the holidays, but maybe do a year's retrospective.

Acuitas development actually *stayed on schedule* this year!

Until the next cycle,

Jenny