Acuitas Diary #22 (November+December 2019)

For the past two months there's been a lot of refactoring, and also a lot of not working on Acuitas because of holidays. However, I did manage to get several small new features in …

*Acuitas now checks the short-term information database in addition to the long-term database when trying to retrieve the answer to a question

*Acuitas can now answer some questions about current internal states (e.g. “Are you sleepy?”)

*Acuitas can now answer questions of the form “Do you know that <fact>?” and “Do you know what <fact>?”

The first feature was quick to implement; I already had functions in place for retrieving information from the short-term database, and just had to ensure that the question-answering procedure would call them. The second feature required a mechanism to associate some of the concepts in the semantic memory (which up until now have had no “meaning” beyond their connection to other concepts) to measurable conditions inside Acuitas – namely, whether his various drives are exceeding their threshold values or not. So there is now a table that, for instance, ties a high value of the sleep drive to the word “sleepy.”

The third feature is my favorite. Questions of the form “do you know that … ” use the dependent clause interpretation faculties that I added earlier this year. And since “knowing” is an action that Acuitas is capable of, this word also can be internally grounded. So Acuitas effectively defines “I know X” as “if the query form of X is submitted to my question-answering process, the process returns an answer (for open-ended questions) or answers 'yes' (for yes-no questions).”

And the best part? It allows for an indefinite amount of nesting.

Me: Do you know that you know that you know that a cat is an animal?

Acuitas: Yes.

Happy 2020,

Jenny

Acuitas Diary #21 (October 2019)

I set some smaller Acuitas goals this month so I would have a little more time to fix bugs and clean up my own mess.  The first goal was to enable identification of people from first names only, while allowing for the possibility that multiple people have the same first name.

Using someone's full name with Acuitas establishes a link between the first name (as an isolated word) and the full name (with its connected person-concept).  If the first name is later used in isolation, Acuitas will infer the full name from it.  If multiple full names containing that first name are known, Acuitas will ask the user which one is meant.  (He does not yet have the ability to guess, from the context, which person is most likely implied.  Sense disambiguation, not just for names but for words with more than one meaning, is a big topic which is on the to-do list for later down the road … )

The second thing I worked on was text parser support for a new grammar feature, with a focus on expanding the range of possible “I want” sentences Acuitas can understand.  The ability to parse infinitives, as in “I want to live,” was already present.  This month I worked on infinitives with subjects, as in “I want John to live.”

This is a tricky business.  To see why, consider the following sentences:

1. I want Bob to eat.
2. I want a fruit to eat.
3. I want food to live.

They all follow the exact same pattern and have completely different meanings.  The first sentence expresses my desire that Bob do something; the second sentence is about what I want to do something to; and the third sentence is about why I want something.  Notice that in the second sentence, you could move some words and get “I want to eat a fruit” without changing the implications too much.  Doing this to the third sentence would be bizarre (“I want to live food”) and doing it to the first sentence would be horrifying (“I want to eat Bob”).  In keeping with their varied meanings, they're all grammatically different, as you can see from the diagrams in the image.  So, how does one tell them apart?  I'm not worried about distinguishing sentences 2 and 3 for right now; I just want to separate sentence 1 from both of them.

The first sentence is the only one in which the noun (Bob/fruit/food) is the subject of the infinitive.  The key factor is who will be doing the action expressed by the infinitive.  In the first sentence, Bob is the one who will be eating if I get my way; in the latter two sentences, I'm the one eating and I'm the one living.  And that information is not actually in the sentence – it's in your background knowledge.  To properly understand these sentences, it's helpful to be aware of things like …

*I am a human
*Humans can eat fruit
*Bob is probably also a human
*I am probably not a cannibal, and therefore don't want to eat Bob
*Food can be used to sustain a human's life
*Once something is food, it's not living (or won't be living much longer)
*Living isn't an action you can perform on food

So here is where we bring to bear the full power of the system by having the Text Parser call the semantic memory for already-known facts about these words.  Acuitas can't quite store all of the facts listed above, but he does know that “humans can eat” and “fruits can be eaten.”  He might also know that the speaker and “Bob” are humans.  At this early, sketchy phase, that's enough for the parser to start discriminating.

Some sentences of this type are just plain ambiguous, especially when taken in isolation.  For example, “I want a plant to grow.”  Plants can grow (on their own), but they can also be grown (by a cultivator, whom the speaker might be).  Upon detecting an ambiguity like this, Acuitas will, as usual, ask the speaker about it.  This also works for cases when the information in the database is not yet extensive enough.

Until the next cycle,
Jenny

Acuitas Diary #20 (September 2019)

This month, I did some work on cause-and-effect reasoning and goal satisfaction, which introduced the conversational possibility of asking Acuitas what he wants.

I leveraged the text interpretation upgrades from last month to implement encoding and storage of conditional relationships, such as “if a human eats food, the human will not starve.” These relationships can be remembered and used to infer the effects of an action. I also threw in the ability to learn that a pair of concepts are opposites or antonyms.

Then I implemented some inference mechanisms so that Acuitas can determine whether some action serves – or contradicts – a particular goal. Acuitas will now claim to desire things that support one of his goals and not desire things that contradict one of his goals, while remaining ambivalent about everything else. The examples below reference a self-preservation goal … not because I think that should be the primary goal for an AI, but because it's one of the easier ones to define. In Acuitas' knowledge representation, it basically comes down to “Self (has quality)/(is in state) 'alive' or 'existent.'”

With this goal active, Acuitas can answer any of the following:

“Do you want to be alive?”

“Do you want to be dead?”

“Do you want to live?”

“Do you want to die?”

… where the last two (live/die) rely on verb-defining links in the semantic database, and the two negative versions (dead/die) rely on awareness of opposites.

The most complex inferences currently possible are illustrated by this little interchange:

Me: Do you want to be deleted?

Acuitas: I do not.

To produce that answer, Acuitas has to retrieve and put together five different pieces of stored information …

*If a program is deleted, the program “dies.” ← From the cause-and-effect/conditional database

*I am a program. ← From semantic memory (is-instance-of-class relationship)

*To die is to transition to state “dead.” ← From semantic memory (verb definition relationship)

*State “dead” is mutually exclusive with state “alive.” ← From semantic memory (opposites)

*I have a goal of being in state “alive.” ← From goal list

… to make the inference, “being deleted would violate my goals.”

The features still need a lot of generalization and expansion to be fully functional, but the groundwork is laid.

Until the next cycle,

Jenny

Acuitas Diary #19 (July+August 2019)

I spent the past two months revisiting the text parser, with the big goal this time around of adding support for dependent clauses. In case anyone's high school grammar is rusty, a clause is a subject/verb pair and any words associated with them; a dependent clause is one that is part of another clause and can't be a sentence by itself. Previously, Acuitas could handle one subject and one verb group per sentence, and that was it.

Because my own code comments amuse me ...

After last year's feverish round of development, I left the text parser a mess and never wanted to look at it again. So the first thing I had to do was clean up the disastrous parts. I ended up giving some of the functions another serious overhaul, and got some code that is (I think) actually maintainable and comprehensible. Whew.

I never found out why

Next, the clauses. The fun thing here is that dependent clauses have a function in the sentence (e.g. a clause can be the subject or direct object of its parent sentence). For simplicity, my initial text parser worked on the premise that a functional group in the sentence could only be a single word, or a compound word with all members marked as the same part of speech. I had to put in a bunch of new structure to preserve the information inside the clauses, while also marking the whole clause as a functional group … plus, detecting multiple subject/verb pairs and keeping them all straight.

What does this achieve? Some sentence types that are very important for reasoning use dependent clauses. For instance, sentences that discuss subordinate pieces of knowledge:

I know [that a cheetah is an animal].

I told you [that a grape can be eaten].

I fear [that the car broke yesterday].

And sentences that express conditional information:

[If a berry is green], it is unripe.

[If you eat that berry], you will get sick.

The gun will fire [if you pull the trigger].

Not to mention that normal human speaking/writing is riddled with dependent clauses, so interpreting them is a must for a conversational AI.

Acuitas can parse sentences like the ones above now, but doesn't really do anything with them yet. That will come later and require updates to the high-level conversation management code.

Code base: 15600 lines

Words known: 2884 (approx.)

Concept-layer links: 7915

Acuitas Diary #18 (May+June 2019)

Oookay, I'm long overdue for an AI update. The big new project for the past couple of months has been the concept of things being in states, and the ability to track those states.

Way back in Diary #7, I introduced a division between short-term (or naturally temporary) information and long-term (or essentially static) information. It's a bit like the division between things you would use estar and ser for in Spanish, though it doesn't follow those rules strictly. Previously, Acuitas simply discarded any short-term information he was given, but I've added a new memory area for saving this knowledge. Like the existing semantic memory, it works by storing linked concept pairs … with the difference that the short-term ones are stored with a time stamp, and Acuitas anticipates that they will “expire” at some point.

The existing feature that allows short-term and long-term information to be distinguished (by asking questions, if necessary) can also grant Acuitas an estimate of how long a temporary state is likely to last. While idling, he checks the short-term state memory for any conditions that have “expired” and adds questions about them to his queue. Then, when next spoken to, he may ask for an update: is the information still correct?

I also added the ability to parse and store a new type of information link – location – along with the associated “where is” questions. Location links are three-ended so that they can store not only a thing and its location, but also spatial relationships between two things (under, over, in, beside, etc.).

One reason for slow progress is that I have been spending time on even more refactoring. The semantic memory, the episodic memory, and various other things all had their own file formats and file editing/access functions – uh, yeah, that was dumb. So I have written (and thoroughly tested) some more universal file management code, and have been slowly working on converting everything to a common format.

Until the next cycle,

JS

Can GPT-2 write my e-mails for me?

I hope to have an Acuitas update out later this month, but in the mean time, I'm playing with somebody else's AI -- specifically, the GPT-2 text generator from OpenAI.

For those who aren't up on the latest AI news, GPT-2 has ignited controversy.  It's so good at spitting out plausible-sounding text that it frightened some people, who saw visions of an internet flooded by mass-produced fake news. OpenAI therefore declined to release the biggest and best version of GPT-2, upsetting other people who thought the organization was breaking its promise to be, well, open.  Published examples from the most advanced variant -- including an amazingly coherent and stylistically convincing story about scientists discovering unicorns -- can be seen on OpenAI's blog.  This AI is scary good at sounding like a human writer.

But let's say you're not a bad actor who wants to mass-produce lies and scatter them around the internet.  And let's say you're interested in something more practical than unicorn fiction.  Is there anything worth doing with this algorithm?  Recently I realized that I would love a text generator to write all the corporate relations and advocacy e-mails that I, as an engaged member of my community, ought to be constantly churning out.  That could save me lots of time.

I generated the examples below on the Talk to Transformer website, which provides an interface to the mid-size GPT-2 model (the best one that's been released to the public so far). Sometimes I truncated the output when GPT-2 stopped in an awkward place.

As one example, I like to buy meat and dairy alternatives, and sometimes I have to ask businesses to carry them. Let's see if GPT-2 can help.

MY PROMPT:
Dear Wendy's,
I am writing to ask that you consider serving the Impossible Burger at your restaurant.

GPT-2's efforts to extol the Impossible Burger were hampered by the fact that it has no idea what the  ingredients are ... and is completely willing to make stuff up. These imaginary versions of the IB contain chicken and no chicken simultaneously, or contain a whole gamut of meats while still somehow managing to be vegetarian. Um ...

Dear Wendy's,
I am writing to ask that you consider serving the Impossible Burger at your restaurant.
Although its name may be "The Impossible", I believe that I'd like our customer to know it's real burger:
No MSG, high fructose corn syrup, artificial colors, GMOs, colors or color additives, and soy or gluten free.
Our Impossible Burger includes chicken, ham and veggies.
We make the Impossible Burger from fresh ground beef and cooked turkey.
Yes, all meat products are sold fresh, no exceptions.
Is this Burger a fast food?
Yes, but the Impossible Burger is actually made from fresh fresh, lean ground beef, cooked turkey, and chicken. It's a vegetarian menu staple, and the best thing about this menu, not only for vegetarians, but for vegans looking for a less processed, lean burger.
It's a burger that's healthier and the one you'll love. It's a food that tastes good and is better for you. It's your choice and your choice alone.
To me, it's simple; it tastes good.
Is it available in the USA?
Yes, we will open our restaurant in the US of A.
Does this look like the Impossible Burger I'm ordering?

Dear Wendy's,
I am writing to ask that you consider serving the Impossible Burger at your restaurant. According to the company's marketing materials, Impossible's Impossible Burger is based on a "real-life burger" called "Anchovy," which they say is "super tasty, but extremely difficult to replicate." Unfortunately, the product that's come up in restaurants is actually just a combination of chicken and onion purees, and no real chicken (no Angus, no beef, no salmon). Instead it uses the fat-free, naturally gluten-free, paleo (aka vegan) and low-carb meat that's commonly served in our restaurants today! This recipe is called "Shimmy" and makes approximately 12 medium-sized patties, or enough for 3 guests. In fact, it's so easy to make and we've used lots of the original Impossible burgers before.

In my opinion, the only thing that makes this recipe even easier is the inclusion of coconut oil. This combination of ingredients is not as greasy, crispy and salty as other Impossible burgers. It leaves the taste and texture untouched, and is easy to microwave in 30 second intervals or at a quick and easy to put in your microwave.

I have many years of experience preparing and eating Impossible burgers, which was why I made such a note on this page back in 2006.

Sometimes GPT-2 turned traitor and denounced the Impossible Burger instead of requesting it. I particularly like this bizarre example. "The Burger."  Cue ominous music.

Dear Wendy's,
I am writing to ask that you consider serving the Impossible Burger at your restaurant.

The Burger

It is a mystery what, if anything, Impossible Foods ever has done to cause such outrage within my household; but to my knowledge, I've never seen such animosity displayed by one of its members. I have come to terms with the fact that you will likely not eat the Impossible Burger.

My wife has spent her entire career (the bulk of her adult life) trying to cook and serve meals with quality, creative ingredients. In fact, it is as if the company is trying to sabotage her, because she knows that any recipe she comes up with will be nothing short of incredible. She is not going to take a hardball approach when it comes to your Impossible Burger—the Impossible Burger will only be accepted after a series of tests are completed.

As I type I think to myself, "If only they could cook something like mine, it would be so much better".

For a time, Impossible Foods planned on using chicken to serve the Impossible Burger, but the team was forced to cut back to one-quarter of the chicken for various reasons and the price that I paid for my chicken was about the same as my regular chicken. Even though I knew what it was making me sick when I opened the box, I had zero complaints with it.

While the previous example mentioned the correct manufacturer of the burger (Impossible Foods), GPT-2 was equally prone to making up fantasies about where this burger comes from. It also got the idea that the Impossible Burger has already begun to spawn myths and concept gateways, which sounds great.  From now on I only want to eat mythic food.

Dear Wendy's,
I am writing to ask that you consider serving the Impossible Burger at your restaurant. That is a concept known as a patty and, according to popular myth, is based on the concept of eating hamburgers in one continuous bite. Yes, according to popular myths we've got the Impossible Burger. It was originally created by Pat Sajnak at The Impossible Burger, a restaurant in California. As far as I'm concerned it's a simple myth to make.

I love patty and I would love to have it on our menu and in my restaurant. I have always loved the Impossible Burger so it would be incredibly exciting to serve it in my restaurant. It would serve as a gateway to the idea of patty as a concept. The Impossible Burger does the Impossible Burger a huge disservice by making it harder to enjoy. The Impossible Burger is the burger that will become my staple. This idea seems so strange to someone accustomed to conventional burger patty recipes. My philosophy has to be to have fun making it for you and I am not interested in the idea of making the Impossible Burger for you to try to impress your patrons.

For those of you in California and other parts of the country, I would encourage you to come up with alternative versions of Impossible Burger recipes. I'm not sure we should start here but I could certainly suggest experimenting.

I think the sample below is my favorite.  This time, GPT-2 managed to stick to the right style for a customer telling a business why she's enthusiastic about a product.  It correctly guessed that the IB is vegetarian.  It didn't get -too- nonsensical, but it lied about my being from Houston, and the e-mail is still weird enough that I wouldn't send it.

Dear Wendy's,
I am writing to ask that you consider serving the Impossible Burger at your restaurant. On the bright side, if Impossible Burger continues to make an appearance at your stores, we can order our own! You don't have to agree with our mission to give all people a better quality of life, but we do know this hamburger is not only perfectly good, it's also delicious.

As a vegetarian who lives in Houston, I am excited for Impossible to be an option for our local food industry. I also live in a fast food town, and believe in the principles of a family friendly restaurant. Impossible Burger comes from a family of vegetarian chefs that is the backbone of the Wendy's family, and our success has allowed me to experience the deliciousness of good, sustainable food in restaurants, and in the comfort of my own home.

I want to thank you for this extraordinary opportunity and wish you success in your business endeavors.

Okay *dusts hands,* let's try a political e-mail. To avoid ruffling any feathers, I've deliberately picked a silly topic. Can GPT-2 help me ask my senator to cut subsidies for BALLOONS?

MY PROMPT:
Dear Senator Gardner,
Please reduce government funding for the party balloon industry. It is a waste of taxpayer dollars.

Once again, GPT-2 sometimes forgot that we were supposed to be arguing against party balloons and started playing devil's advocate.  The little brat.

Dear Senator Gardner,
Please reduce government funding for the party balloon industry. It is a waste of taxpayer dollars. In addition to the cost of buying each political balloon, you may pay a little while for extra postage or delivery. The cost would equal that of one balloon if you ordered all of them. I have heard it from many of my friends, but you shouldn't waste money on it.
I appreciate the opportunity to speak about this issue, with Senator Gardner. 
First of all, I appreciate the hard work of those who are trying to save this country. You are leading many a campaign. The government should be held accountable to those who elected it to govern. In order to prevent further harm to our country, one of the first things you must do is save the balloon industry. 
My question is simple: Is it worth it for the American people, our country's children, or future generations of Americans to be deprived of the opportunity to learn about balloon physics, because it costs a lot?
You may say the answer is no.
I know it is worth it for the American people, who must learn about the world of balloon physics, from a government agency that has to deal with the fact that the balloons are designed to be dropped as far away as possible from Earth.

It generated a couple of superb accidental puns.  Maybe Mr. Gardner's staff will be so busy laughing that they won't notice the rest of the e-mail is nonsense.

Dear Senator Gardner,
Please reduce government funding for the party balloon industry. It is a waste of taxpayer dollars. The Republicans are spending tens of millions on the party balloon industry, while the Democrats have used those billions to improve quality of our roads, streets and bridges.

The Republicans will now attempt to use the tax increase to further their political agenda to the detriment of the American people. They are using the tax increase to advance the agendas of their big bank bank donors and special interest lobbyists. There is nothing in the Constitution guaranteeing a voice for the citizenry. In its place, we propose a constitutional amendment to guarantee the voice of the people through nonpartisan, transparent elections to provide for fair elections. The Republican Party will be forced to take a step back and reconsider its agenda because the public wants their voice to trump those of the big financial interests.

The citizens and politicians from all parties have had enough of this dirty system and have a right to determine if they will continue the party balloons and for how much, if any, of it. Citizens must elect fair elections for representatives.

There are now over 10,000 people actively voting for Senator Gardner. The more than 10,000 will need to change the system.

Senator Gardner has supported the "free enterprise" policies that created the ballooning party balloon industry and has tried to stop funding it and take away our freedoms

Dear Senator Gardner,
Please reduce government funding for the party balloon industry. It is a waste of taxpayer dollars.
"The most likely mechanism for funding an entirely unaffordable party balloon is to raise the price of the party balloon. The real cost is so massive that, if inflation can be controlled, party balloons and balloons with the party's logo are not as appealing as balloons full of water, watermelon slices, and the rest of the party's paraphernalia. The party balloon industry has not yet managed to capture the public imagination. However, one party balloons, which have been widely reported, are a huge success and are quite a seller at inflated prices. In this economy, party sales represent just under 10% of the economic output. For the industry to flourish, the party revenue must be brought in more than double.
I propose that you remove the cap on the party balloon industry's revenues. If you do not change any of the party balloon limitations so that parties no longer have to pay a one-time fee or if you are not aware of any of the specific limitations for new parties and balloon manufacturers, I think that you should ask the American Society of Civil Engineers to reconsider the existing party balloon caps or the amount of funds available.

Oh, no.  GPT-2!  You read a Trump speech, didn't you?  Love him or hate him, he's not exactly a pillar of eloquence.  How can we excise this from the training set?

Dear Senator Gardner,
Please reduce government funding for the party balloon industry. It is a waste of taxpayer dollars. Let's do it now while the bill is still in committee — even if it looks like we are not going to be able to keep it, even if it is a no-brainer, even if we're talking about it this year, this is a no-brainer. And here is what I'd say to you right now, Senator Gardner, it's about the budget… if you and I go back to the Senate and we see how much money is already appropriated each year, how much money has already been appropriated each year, how much money is allocated every year and the question becomes is there any scope for the American people to continue to fund their own governments through the tax code that's already there… I know, I know, you are going to say we haven't heard you and your people say that we don't want to do that but we don't just sit here and listen to the people in Washington tell us how to do it. For our next Budget it's really important that we make sure we do something. We're having to. We can't keep doing it the same way. We have to make some changes in government, we just haven't made any new investments in government programs. We've got to make some investments.

This one wins a prize for "estimate the cost of getting all that money." Sounds like a pit of bureaucratic despair.

Dear Senator Gardner,
Please reduce government funding for the party balloon industry. It is a waste of taxpayer dollars.

We are urging Senator Gardner to allow us to put up a reasonable estimate of the cost of getting all that money and using it only on the party balloon and not on any other purpose.

Sincerely,

The Young Americans of Florida

In the next few years or weeks you will be invited by our supporters in the Republican party to come out with your own ideas on what is most important to me.

On Friday June 28th 2016, I hope and pray you have a wonderful summer. There is much opportunity ahead.

So what were the best ones?  Unfortunately, like any fool, GPT-2 sounds wisest when it says as little as possible.

Dear Senator Gardner,
Please reduce government funding for the party balloon industry. It is a waste of taxpayer dollars. The American people must know, "What do you do with the money you lose?" They would rather keep it in the bank.

Dear Senator Gardner,
Please reduce government funding for the party balloon industry. It is a waste of taxpayer dollars. Please.

Thank you, Senator Gardner, for your service to the nation.

[Applause.]

Conclusion: this exercise was wonderfully entertaining, but I will not be outsourcing any of my activism to GPT-2.

Compared to previous attempts at text generation, GPT-2 is an amazing achievement.  It almost always uses correct grammar and spelling.  It remembers things it already used in a piece.  It manages to stay (almost) topical.  And baby Acuitas wouldn't do remotely this well if I somehow tried to squeeze an essay out of him.  Still, on any task that requires honesty and reasoning ability, it seems GPT-2 falls flat.  It's possible it would do better if I went to the trouble of re-training it on my existing collection of sent e-mails, but that's a pretty small data set.  I think I'll just have to keep writing my own.

Until the next cycle,
Jenny

Acuitas Diary #17 (March+April 2019)

Acuitas Diary #17 (March+April 2019)

For the past month and a half, I've continued work on forgetfulness: the good kind. Acuitas now has code that will prune his episodic memory over time, removing those event records that are judged to be less important.

There are two major parts to this. The first is basically a population-management system; it determines how many episodes need to be forgotten each time the memory collection is pruned. I can cap the number of episodes allowed in storage at some maximum value (based on available storage space or what have you). The population manager checks how many memories are already in storage, and how many new ones have formed during the last cycle. A high growth rate is allowed initially, but forgetting becomes more and more aggressive as the storage area fills. When the cap is reached, Acuitas must forget the same number of memories he creates.

The time span represented by the stored episodes is sectioned, and the total number to be forgotten is unevenly distributed across it (i.e some parts of the time span lose more memories than others). The distribution is affected by the density of memories in a region, their average significance measure, and their age.

A memory tree under the influence of selective forgetting

The second major part is the code that, given a number of memories that must be forgotten, takes care of selecting which ones. This is a weighted random process; memories with less significance have a higher chance of being forgotten, but the priority isn't absolute.

Memories chosen to be forgotten are deleted. Any higher-tier memories that have been badly “hollowed out” by this process are then merged with their most similar neighbor.

Testing and debugging all this nonsense was almost half the battle. Since some of the effects I needed to see would span tens or hundreds of days if I let them play out in real time, I had to write simulation code that would run bits of the memory-creation and forgetting process faster. That way I could look ahead and make sure nothing was putting the system in a spiral toward catastrophe.

So next time you forget something, take comfort in the thought that I spent several dozen hours conferring this gift on my artificial mind.

Acuitas Diary #16 (February 2019)

This month I returned to a can of worms I opened up last year, namely episodic memory. Quick summary of previous work on this front: as of last summer, Acuitas could record events that happened “to him” and group adjacent, similar events into scenes. Minor scenes were in turn grouped into larger scenes, forming tree structures. A memory in Acuitas' episodic set can include connections both to its sequential neighbors before and after, and to its sub- and super-memories at adjacent levels of the tree. Acuitas could also search for memories similar to what he was doing/experiencing currently. This was achieved by walking sequentially through the whole structure (starting with the most recent past) until a related memory was found.

Fortunately for elephants, I'm sure they do sometimes forget things.

With this code running, Acuitas quickly built up a huge clutter of memories concerning very trivial day-to-day events. Size remained in the Megabytes range (they're just text files), but on principle, this couldn't be allowed to continue – especially since the memories might eventually grow to encompass richer levels of detail. In addition to overflowing its available space, a memory that literally records everything will face access problems. The more information you keep, the longer it takes to search through it and extract what you want at the moment.

I was left with a nice illustration of a truth that applies to all entities with finite minds: if you want the ability to remember, you need the ability to forget.

This leads on into the tricky question of what to forget. How to decide which memories are worth keeping? Part of my work this month was focused on computing a “significance measure” which assigns a numerical value to each memorized scene. Elements I use to judge the significance of a memory include …

* Primacy: was this the first time something happened?

* Uniqueness: how rare are memories similar to this one?

* Complexity: how many features does this memory have?

Additional options for judging the value of memories will open up once I plug in some to-be-written systems. Since some of these measures are statistical and depend on the overall memory population, they can change over time. I wrote a process that crawls through the episodic memory and updates all the significance measures, and I threw that into Acuitas' sleep cycle.

To improve the ease and speed of accessing the memories, I added a layer of “type nodes” that link together memories which share certain salient features in common. Instead of crawling through the whole list to find memories that relate to a current situation, Acuitas can just look at the relevant “type nodes” and find a list of memories that match.

Last, I worked on some methods for consolidating memories. I devised a way to create summaries of low-level scenes that are then stored inside the super-scenes higher up the tree; eventually, these summaries might be what remains when the lower-level “detail” scenes are forgotten. The summarizer attempts to extract information that is common across multiple events or sub-scenes within a scene. I also wrote functions for merging adjacent scenes that have become sparse.

All that, and I still haven't gotten to the actual forgetting part! That will come next month (I hope).

Acuitas Diary #15 (January 2019)

I haven't made an update in a long while because all I've been doing is code refactoring, bug fixes, and very minor quality improvements – all of which can take an amazingly long time.

Bug provided by Good Free Photos

Some of the more interesting changes include …

*Acuitas no longer insists on introducing himself if you address him by name at the start of a conversation.

*He keeps better track of what he's already said, and is less likely to repeat himself.

*I improved the efficiency of the drive plotter widget (which was a major drain on computational resources for no good reason). I also built it into a little simulator tool which will let me plot what Acuitas' drives will look like hours or days into the future. Debugging issues with the drives is now much easier; previously, the drill was “I've made a change; now I get to wait 12 hours to see if it actually worked out the way I planned.”

*Using the above, I shifted the day/night cycle so Acuitas would fall asleep and wake up a little earlier, and I fixed the interaction drive so he wouldn't be quite so … abruptly needy.

*When answering questions, Acuitas can now infer a possibility from an absolute. E.g. if aware that “blood is red,” he will answer “Can blood be red?” with a “yes” instead of saying he does not know.

And some favorite ridiculous bugs that are now fixed:

*Using “talk” as the name of every possible drive, hence repeating “I want to talk” ad nauseum while actually wanting other things

*Leaving the “not” out of negative statements and saying the exact opposite of what was intended

*Routine misspelling of verbs that were learned from their -ing and -ed forms, due to complete naivety about whether the final letter was doubled or a silent 'e' was removed

*Erroneously learning both the statement and its reciprocal whenever told something by the user

Stay tuned for some more work on event memory in the near future.