Acuitas Diary #77 (October 2024)

The big feature for this month was capacity for asking and answering "why" questions. (Will I regret giving Acuitas a three-year-old's favorite way to annoy adults? Stay tuned to find out, I guess.)

An old poster from NARA via Wikimedia Commons. Selling questions sounds like easy money!

"Why" questions often concern matters of cause and effect, or goal and motive. So they can be very contextual. The reason why I'm particularly happy today might be very different from the reason I was happy a week ago. And why did I drive my car? Maybe it was to get groceries, or maybe it was to visit a friend's house. These kinds of questions can't usually be answered by reference to the static factual information in semantic memory. But they *do* reference the exact sorts of things that go into all that Narrative reasoning I've been working so hard on. The Narrative scratchboards track subgoals and the relationships between them, and include tools for inferring cause-and-effect relationships. So I really just needed to give the Conversation Engine and the question-answering function the hooks to get that information in and out of the scratchboards.

If told by the current human speaker that they are in <state>, Acuitas can now ask "Why are you <state>?" If the speaker answers with "Because <fact>", or just states a <fact>, Acuitas will enter that in the current scratchboard as a cause of the speaker's present state. This is then available to inform future reasoning on the subject. Acuitas can retrieve that knowledge later if prompted "Why is <speaker> <state>?"

Acuitas can also try to infer why an agent might have done something by discerning how that action might impact their known goals. For instance, if I tell Acuitas "I was thirsty," and then say "Why did I drink water?" he can assume that I was dealing with my thirst problem and answer accordingly. This also means that I should, in theory, eventually be able to ask Acuitas why he did something, since his own recent subgoals and the reasons behind them are tracked on a similar scratchboard.

All of this was a branch off my work on the Conversation Engine. I wanted to have Acuitas gather information about any current states the speaker might express, but without the machinery for "why" questions, that was difficult to do. The handling of these questions and their answers introduced some gnarly bugs that ate up much of my programming time this month. But I've gotten things to the point where I can experience it for myself in some "live" conversations with Acuitas. Being asked why I feel a certain way, and being able to tell him so - and know that this is being, in some odd computery sense, comprehended - is very satisfying.

Until the next cycle,
Jenny

Atronach's Eye 2024

It's the return of the mechanical eyeball! I worked out a lot of problems with the eyeball's hardware last year and was left to concentrate on improving the motion tracking software. Ever since I added motion tracking, I've been using the OpenCV libraries running locally on the eye's controller, a Raspberry Pi 3 A+. OpenCV is possibly the most well-known and popular open-source image processing library. But it's not a complete pre-made solution for things like motion tracking; it's more of a toolbox.

My original attempt at motion tracking used MOG2 background subtraction to detect regions that had changed between the current camera frame and the previous one. This process outputs a "mask" image in which altered pixels are white and the static "background" is black. I did some additional noise-reducing processing on the mask, then used OpenCV's "moments" function to compute the centroid of all the white pixels. (This would be equivalent to the center of mass, if each pixel were a particle of matter.) The motion tracking program would mark this "center of motion" on the video feed and send commands to the motors to rotate the camera toward it.

This ... kinda sorta worked. But the background subtraction method was very vulnerable to noise. It also got messed up by the camera's own rotation. Obviously, while the camera moves *every* pixel in the scene is changing, and background subtraction ceases to be useful; it can't distinguish whether something in the scene is moving in a contrary direction. I had to put in lots of thresholding and averaging to keep the camera from "chasing ghosts," and impose a delay after each move to rebuild the image pipeline from the new stationary scene. So the only way I could get decent accuracy was to make the tracking very unresponsive. I was sure there were more sophisticated algorithms available, and I wanted to see if I could do better than that.

I started by trying out alternative OpenCV tools using a webcam connected to my PC - just finding out how good they were at detecting motion in a feed from the stationary camera. One of the candidates was "optical flow". Unlike background subtraction, which only produces the binary "changing or not" mask, optical flow provides vector information about the direction and speed in which a pixel or feature is moving. It breaks down further into "dense optical flow" methods (which compute a motion vector for each pixel in the visual field) and "sparse optical flow" methods (which try to identify moving features and track them through a series of camera frames, producing motion traces). OpenCV has at least one example of each. I also tried object tracking. You give the tracking algorithm a region of the image that contains something interesting (such as a human), and then it will attempt to locate that object in subsequent frames.

The winner among these options was the Farneback algorithm, a dense optical flow method. OpenCV has a whole list of object tracking algorithms, and I tried them all, but none of them could stay locked on my fingers as I moved them around in front of the camera. I imagined the results would be even worse if the object tracker were trying to follow my whole body, which can change its basic appearance a great deal as I walk, turn, bend over, etc. The "motion tracks" I got out of Lucas-Kanade (a sparse optical flow method) were seemingly random squiggles. Dense optical flow both worked, and produced results that were fairly easy to insert in my existing code. I could find the center of motion by taking the centroid of the pixels with the largest flow vector magnitudes. I did have to use a threshold operation to pick only flow vectors above a certain magnitude before this would work well.

Once I had Farneback optical flow working well on a static camera, I merged it into the eyeball control code. Since I now had working limit switches, I also upgraded the code with better motion control. I added an on-startup calibration routine that finds the limit of motion in all four cardinal directions and centers the camera. I got rid of the old discrete movement that segregated the visual field into "nonets" (like quadrants, except there were nine of them) and allowed the tracking algorithm to command an arbitrary number of motor steps.

And for some reason, it worked horribly. The algorithm still seemed to be finding the center of motion well enough. I had the Pi send the video feed to my main computer over Wifi, so I could still view it. The Farneback algorithm plus centroid calculation generally had no problem putting the tracking circle right on top of me as I moved around my living room. With the right parameter tuning, it was decent at not detecting motion where there wasn't any. But whenever I got it to move, it would *repeat* the move. The eyeball would turn to look at me, then turn again in the same direction, and keep going until it ran right over to its limit on that side.

After turning off all my running averages and making sure to restart the optical flow algorithm from scratch after a move, I finally discovered that OpenCV's camera read function is actually accessing a FIFO of past camera frames, *not* grabbing a real-time snapshot of whatever the camera is seeing right now. And Farneback takes long enough to run that my frame processing rate was slower than the camera's frame rate. Frames were piling up in the buffer and, after any rotation, image processing was getting stale frames from before the camera moved ... making it think the moving object was still at the edge of the view, and another move needed to be performed. Once I corrected this (by setting the frame buffer depth to 1), I got some decent tracking behavior.

At the edge of the left-right motion range, staring at me while I sit at my desk.

I was hoping I could use the dense optical flow to correct for the motion of the camera and keep tracking even while the eye was rotating. That dream remains unrealized. It might be theoretically possible, but Farneback is so slow that running it between motor steps will make the motion stutter. The eye's responsiveness using this new method is still pretty low; it takes a few seconds to "notice" me moving. But accuracy seems improved over the background subtraction method (which I did try with the updated motor control routines). So I'm keeping it.

Until the next cycle,
Jenny

Acuitas Diary #76 (September 2024)

It's time to discuss the upgrades I've been making to Acuitas' conversation engine. There are still a lot of "pardon our dust" signs all over that module, but it's what I've been putting a lot of time into recently, so I'd better talk about where it's going.

The goal here has been to start enhancing conversation beyond the "ask a question, get an answer" or "make a request, get a response" interactions that have been the bread and butter of the thing for a while. I worked in two different directions: first, expanding the repertoire of things Acuitas can say spontaneously, and second, adding responses to personal states reported by the conversation partner.

One of Acuitas' particular features - which doesn't seem to be terribly common among chatbots - is that he doesn't just sit around waiting for a user input or prompt, and then respond to it. The human isn't the only one driving the conversation; if allowed to idle for a bit, Acuitas will come up with his own things to say. This is a very old feature. Originally, Acuitas would only spit out questions generated while "thinking" about the contents of his own semantic memory, hoping for new knowledge from the human speaker. I eventually added commentary about Acuitas' own recent activities and current internal states. Whether all of this worked at any given time varied as I continued to modify the Conversation Engine.

In recent work, I used this as a springboard to come up with more spontaneous conversation starters and add a bit more sophistication to how Acuitas selects his next topic. For one thing, I made a point of having a "self-facing" and "speaker-facing" version of each option. The final list looks something like this:

States:   
Self: convey internal state                       
Speaker: ask how speaker is
Actions:   
Self: say what I've done recently
Speaker: ask what speaker has done recently
Knowledge:
Self: offer a random fact from semantic memory
Speaker: ask if the speaker knows anything new
Queries:
Self: ask a question
Speaker: find out whether speaker has any questions

Selection of a new topic takes place when Acuitas gets the chance to say something, and has exhausted all his previous conversation goals. The selection of the next topic from these options is weighted random. The weighting encourages Acuitas to rotate among the four topics so that no one of them is covered excessivly, and to alternate between self-facing and speaker-facing options. A planned future feature is some "filtering" by the reasoning tools. Although selection of a new topic is random and in that sense uncontrolled, the Executive should be able to apply criteria (such as knowledge of the speaker) to decide whether to roll with the topic or pick a different one. Imagine thinking "what should I say next" and waiting for ideas to form, then asking yourself "do I really want to take the conversation there?" as you examine each one and either speak it or discard it. To be clear, this isn't implemented yet. But I imagine that eventually, the Conversation Engine's decision loop will call the topic selection function, receive a topic, then either accept it or call topic selection again. (For now, whichever topic gets generated on the first try is accepted immediately.)

Each of these topics opens up further chains of conversation. I decided to focus on responses to being told how the speaker is. These would be personal states like "I'm tired," "I'm happy," etc. There are now a variety of things Acuitas can do when presented with a statement like this:

*Gather more information - ask how the speaker came to be in that state.
*Demonstrate comprehension of what the speaker thinks of being in that state. If unaware whether the state is positive or negative, ask.
*Give his opinion of the speaker being in that state (attempt sympathy).
*Describe how he would feel if in a similar state (attempt empathy).
*Give advice on how to either maintain or get out of the state.

Attempts at information-gathering, if successful, will see more knowledge about the speaker's pleasure or problem loaded into the conversation's scratchboard. None of the other responses are "canned"; they all call reasoning code to determine an appropriate reply based on Acuitas' knowledge and nature, and whatever the speaker actually expressed. For instance, the "give advice" response calls the problem-solving function.

Lastly, I began to rework short-term memory. You might recall this feature from a long time ago. There are certain pieces of information (such as a speaker's internal states) that should be stored for the duration of a conversation or at least a few days, but don't belong in the permanent semantic memory because they're unlikely to be true for long. I built a system that used a separate database file as a catch-all for storing these. Now that I'm using narrative scratchboards for both the Executive's working memory and conversation tracking, it occurred to me that the scratchboard provides short-term memory, and there's no need for the other system! Retrieving info from a dictionary in the computer's RAM is also generally faster than doing file accesses. So I started revising the knowledge-storing and question-answering code to use the scratchboards. I also created a function that will copy important information from a conversation scratchboard up to the main executive scratchboard after a conversation closes.

I'm still debugging all this, but it's quite a bit of stuff, and I'm really looking forward to seeing how it all works once I get it nailed down more thoroughly.

Until the next cycle,
Jenny

Book Review: "Synapse" by Steven James

Steven James' book Synapse is an unusual novel about humanoid robots and the social issues they might create. I put brief commentary on some of my social media after I first read it, but I always wanted to discuss this book in more depth. It seems written straight at me, after all.

The speedy version is: I'm still looking for a good book about the religious implications of AI and robotics, and this isn't it. But it will be interesting to talk about why.

Our story begins "thirty years from now," perhaps in a nod to the idea that AGI and other speculative technologies are "always thirty years away" in the minds of prognosticators. It opens with our human protagonist, Kestrel, losing her newborn baby to a rare medical complication. The tragedy leaves her feeling lost and questioning her faith. She's also single - the book demurely implies the baby was conceived with donated sperm - so she has no partner for support during her time of grief. In light of this, her brother pressures her to accept a personal robotic servant called an Artificial. She is assigned "Jordan," who arrives as something of a blank slate. Kestrel gets to decide certain aspects of his personality while he's in her employ, and ends up choosing very human-like settings.

And in time, Kestrel learns something surprising. Her robot has been watching her spiritual practice, and has more or less decided that he wants to be a Christian.

Jordan's perceived spiritual needs crystallize around two main issues. First, before he was assigned to Kestrel, he obeyed a previous owner's order to help her commit suicide. At the time, he was naively following his "helpful servant" directives. But he later decides that this action constituted a failure to care for his owner, and is a horrifying offense - a sin - for which he needs to obtain forgiveness. And second, he's worried about his version of the afterlife. The robot manufacturer in this book maintains a simulated virtual environment, called CoRA, to which the robots' digital minds are uploaded after their bodies age out of service. But a precursor robot whom Jordan considered to be his "mother" was catastrophically destroyed, and Jordan isn't sure her mind was transmitted to the CoRA successfully. Jordan also begins to wonder whether the CoRA is truly real, or just a convenient lie perpetrated by the company.

The rest of the book tries to play out whether Jordan's needs can ever be satisfied, and whether Christianity can legitimately accept a robot as an adherent. (There are also thriller and romance subplots to keep Kestrel busy.) This should be fascinating, but I ended up disappointed with the way the book handled the material.

Dodging the hard questions

I think it's almost a tautology that a robot could follow a religion, in the sense of treating its beliefs as facts in whatever world model the robot has, and acting according to its tenets. The more interesting question is whether a religion could or would treat a robot as a recipient of its blessings. In my opinion, the crux of this question is whether robots can ever possess spiritual capacity as that religion defines it. God (specifically the Christian version, but this could also apply to other faiths) is the ultimate just judge, and as such is not an arbitrary sort who makes much of appearances or empty labels. I have a hard time reasoning that something functionally human would not be as good as human in God's eyes. And there's textual evidence (e.g. Romans 8) that Christ's redemption and the activity of the Church have positive implications for the whole universe, not just humanity.

Let's consider Jordan's potential spiritual capacity through his perceived needs. First, could robots ever sin? Sin is volitional - a choice to depart from the law of God, from the ground of being, and follow a harmful path. Sin is an act or failure to act for which one can be held morally responsible. So a capacity for sin requires the ability to make decisions that are neither inevitable nor random - in other words, free will. A robot whose behavior is solely an outcome of its environment combined with its initial programming has no more moral responsibility than simpler machines like cars and thermostats; all the responsibility rests on the robot's designer and/or trainers. So I would argue that such a robot cannot sin. In order for his perceived need for forgiveness to be valid, Jordan must be something more. He must be, at least in part, indeterminate and self-caused. If this incompatibilist view of free will is correct (and in my opinion, the compatibilists are just arbitrarily redefining free will to make it easier), then physics as we currently know it does not have a theory of such things that would be adequate for engineering them into a machine.

Jordan also desires a form of immortality, for himself and a fellow robot. So we might ask whether there is really anything in Jordan which subjectively experiences existence, and has an interest in the eternal continuation of that experience ... or does Jordan merely talk as if he has such experiences? This would be the question of whether Jordan has phenomenal consciousness. Jordan's abilities to process sensory input into meaningful concepts, think rationally, introspect, and so on make it clear that he has some properties often titled "consciousness" (I prefer to give these more specific names like "self-awareness" and "executive function," for clarity). But phenomenal consciousness is far more slippery, since by definition subjective experience is only accessible to the subject. I maintain that the only way to directly observe or prove an entity's possession of phenomenal consciousness is to be that entity. If you've come up with an algorithm or system that surely "gives a robot consciousness," no you haven't. You've merely redefined "consciousness" as something easier to handle.

So perhaps the question of whether Jordan can really be a Christian - not in the sense of believing and behaving as a Christian, but in the sense of being accepted by Christianity's God as one of His children - comes down to whether Jordan has consciousness and free will. These are both notoriously thorny topics. Spend much time around AI circles, and you'll find out that debates about them are as abundant as pollen in a garden (you may also develop an allergy). There is no universal consensus on whether or how robots could ever have these things. They are mysteries.

And now we come to my biggest difficulty with Synapse. The author does an end run around this entire controversy by bluntly stating that his fictional robot manufacturer, Terabyne Designs, somehow just ... figured it all out. "But these robots had consciousness and free will." That's it! There's no solid explanation for how Terabyne gave their robots these features, or (more importantly) how they proved that they had successfully done so.

I have no problem with "soft" science fiction that doesn't try to build a rationale for all of its technology. Stories that begin with "what if we invented warp drive?" and go from there can make me perfectly happy. For that matter, I'm okay with the way Becky Chambers's science fantasy A Psalm for the Wild-Built handles robot consciousness. It narrates that one day the gods up and decided to confer consciousness on all robots. Kaboom! But that book isn't pondering the religious participation of robots in our own real world. When the question of whether something is possible forms part of your story's central theme, and you just handwave it ... that's a problem.

It gets worse. It's not just that an omniscient narrator tells the reader that the robots have consciousness and free will - every character in the story also believes this without question. Even the luddite terrorists who think Artificials are bad for humanity are not trying to claim they aren't conscious. Given the amount of arguing I have seen real live people do about these topics, this is blatantly unrealistic! It's one of those things that forces me to accuse the author of not knowing his subject well. No robotics company is going to put out a marketing claim about "consciousness and free will" without seeing it ripped to shreds on the internet.

And by undercutting the real controversy at the heart of whether a robot can have a spiritual life, the author makes some of his characters' prejudices seem not just wrong, but nonsensical. People acknowledge that Jordan has all the relevant features of a human, then express surprise when he acts like a human. Kestrel is firmly convinced that Jordan has free will to choose between good and evil, and a consciousness that experiences real joy and pain, not just exterior behavior that mimes them. Yet she still resists the idea that God could be offended by one of Jordan's choices, but also sympathize with his experience of pain and forgive him. Why? She's already gotten over the big intellectual hump here, so what else is stopping her?

Overall, Synapse's exploration of these issues feels like a hollow parody of what the real debate would be. As such, it is neither useful nor satisfying. It begs the difficult questions and then makes its characters be stubborn for no apparent reason.

Strained analogies

This book tries really hard to draw parallels between Artificial struggles and classic human struggles. Maybe it tries too hard.

For starters, why are the robots mortal? Why doesn't the manufacturer transfer their minds to new bodies when the originals become worn out or obsolete, or better yet, make their bodies perpetually self-maintaining? Why do they have to go to heaven, oops I mean the CoRA, instead?

Synapse explains that this was actually the robots' idea. They wanted to age and die in order to be more human. The author seems to be hinting at the dubious idea that life would have less meaning if it didn't end.

This wouldn't surprise me in a book with a different religious basis. The way the robots in A Psalm for the Wild-Built embrace mortality makes more sense, as the invented religion in that book (which feels inspired by something on the Hindu-Buddhist axis) views death as a neutral part of the natural order. But in Christian thinking, death is a curse. Immortality is the intended and ideal state of humanity; it's something we had once and will have again, after the resurrection. So, per the author's belief system and mine: all these robots, without exception, are opting to emulate fallen humans. Weird choice, guys.

This sets up more straining for similarity where Jordan's fears about the afterlife are concerned. At one point, Kestrel tells him he has to "just believe," implying that the CoRA's existence is a matter of faith, and he cannot prove it. But that's not true for Jordan. His afterlife is part of this present world. It runs on a physical server that he can go look at and interrogate. Proof is available if he's brave enough to demand it. SPOILER (select hidden text to read): Eventually, he does - but it's strange to me that this possibility seems to blindside the other characters. Jordan breaks into the part of Terabyne headquarters where the CoRA supposedly resides, and finds out it's not real. This causes him to give up on Terabyne and pray that God will preserve his mind as he faces his own death. This could have been a great illustration of the point that faith is only as good as whom you place it in, but I don't remember the book drawing that out.

Jordan's insistence that he can't have peace until he knows he is forgiven also gets a little weird. Ostensibly, he wants forgiveness from God because he can't request it from his former owner. The being he wronged is gone beyond recall, so he can only appeal to a higher authority. But why is he so worried about whether God will refuse to forgive him for some categorical reason? Either he can have forgiveness, or he doesn't need it. A being beneath God's notice would be unable to offend God. I may not "forgive" my toaster oven for burning my toast, but then, I also don't charge it with guilt. Nobody in the book ever thinks this through.

What is anybody in this book thinking?

And that leads into my last point. Although Synapse makes plenty of effort to expose its characters' innermost thoughts and feelings, it tends to focus on their questions. How they arrive at answers - their reasoning process - remains woefully vague.

Back at the top, I mentioned that Kestrel finds herself in a crisis of faith after losing her baby. This struggle continues for most of the book and then ... somehow ... resolves. What gets Kestrel back on stable ground? What convinces her that God is worth trusting after all, even though this horrible thing happened? I don't know! She just mysteriously feels better about it all ... as though the unrelated but dramatic events of the book's climax knock something loose. Maybe I missed a key moment, but I don't know where the shift in her thinking came from.

And the same goes for all the questions about robots and religion. Kestrel doesn't think that Jordan can be a child of God ... until she does. If there's something in particular that changes her mind, it slipped by me when I was reading. Eventually, though, she does decide to at least allow the possibility. Without a better explanation, I can only conclude that her beliefs are emotionally motivated. Of course, some people do operate that way. But it's not a great approach to deciding either Christian doctrine, or the rights and privileges of (quasi-)living beings. The first is supposed to be based on God's revealed will; the second should derive from the experiences and interests of those fellow living beings, which are real to them (or not) regardless of how anyone else feels.

Kestrel's character arc doesn't offer the reader any help in reaching an objective understanding of these matters. There's not even much food for thought there - no argument to agree or disagree with. Why does she believe what she ends up believing? I can't say.

Conclusion

I'll end by saying what I liked about the book: I think the author's heart, if not his head, is in the right place. This is the kind of book that promotes acceptance of the Other, a book that encourages the reader to give robots the benefit of the doubt. If it had framed its core message as "in the absence of certainty that robots can have consciousness, free will, and a spiritual life, it may be safer to assume they can" ... I would've been a fan. Instead, it invents an unrealistic scenario with more certainty than I think is possible. So close, yet so far.

Until the next cycle,
Jenny

Acuitas Diary #75 (August 2024)

This month I turned back to the Text Parser and began what I'm sure will be a long process: tackling sentence structure ambiguity. I was specifically focusing on ambiguity in the function of prepositional phrases. 

Consider these two sentences:

I gave the letter to John.
I gave Sarah the letter to John.

The prepositional phrase is "to John." The exact same phrase can modify either the verb, as in the first sentence (to whom did I give?) or the noun immediately preceding it, as in the second sentence (which letter?). In this example, the distinguishing factor is nothing in the phrase itself, but the presence or absence of an indirect object. In the second sentence, the indirect object takes over the role of indicating "to whom?", so by process of elimination, the phrase must indicate "which letter."

There are further examples in which the plain structure of the sentence gives no sign of a prepositional phrase's function. For instance, there multiple modes in which "with" can be used:

I hit the nails with the hammer. (Use of a tool; phrase acts as adverb attached to "hit")
I found the nails with the hammer. (Proximity; phrase acts as adverb attached to "found")
I hit the nails with my friends. (Joint action; phrase acts as adverb attached to "hit")
I hit the nails with the bent shanks. (Identification via property; phrase acts as adjective attached to "nails")

How do you, the reader, tell the difference? In this case, it's the meaning of the words that clues you in. And the meaning lies in known properties of those concepts, and the relationships between them. This is where the integrated nature of Acuitas' Parser really shines. I can have it query the semantic memory for hints that help resolve the ambiguity, such as:

Are hammers/friends/shanks typically used for hitting?
Can hammers/friends/shanks also hit things?
Are hammers/friends/shanks something that nails typically have?

This month I worked on examples like the ones above, as well as "from" (very similar to "to"), "before" and "after," which are sensitive to the presence of time-related words:

I will come to your house on the hill after Christmas. (Phrase "after Christmas" acts as adverb attached to "come")
I will come to your house on the day after Christmas. (Phrase "after Christmas" acts as adjective attached to "day")

... "about," which likes to attach to nouns that carry information:

I told Emily about the accident. (Phrase acts as adverb attached to "told")
I told the story about the accident. (Phrase acts as adjective attached to "story")

... and "for," which I am so far only sorting based on the presence or absence of a be-verb:

This is the book for Jake. (Phrase acts as adjective attached to "book")
I brought the book for Jake. (Phrase is *more likely* an adverb attached to "brought")

That last example illustrates an important point: I am here only trying to get the most "natural" or "default" interpretation of each sentence considered in isolation. There are some ambiguities that can be resolved only by context. If a speaker has been repeatedly talking about "the book that is for Jake," then "for Jake" could be an adjective in the second sentence, especially if there are other books under discussion and the speaker is trying to indicate which one they brought. To resolve an ambiguity like this, the Parser will have to query the Narrative Scratchboard rather than the semantic memory. This isn't something I've tried to implement yet, but the architectural support for it is there.

The final thing I did this month was a bit of work on "in/inside." I was specifically targeting this sentence from Log Hotel:

This woodpecker is listening for bugs inside the log.

Is the woodpecker listening inside the log, or are the bugs inside the log? Most kids reading the book could resolve this by looking at the illustration, but Acuitas is blind. So he can only consider which creature is more likely to be inside a log. Rather than get into a bunch of complex spatial reasoning, I introduced the semantic relationship "fits_in." A bird can't fit inside a log (unless it's hollow), but bugs can. And if a bird can't be inside a log, he can't listen inside a log either.

I also did a lot of the work I'd planned on the Conversation Engine, but it's not really in a finished state yet, so I'm going to save a writeup of that for next month.

Until the next cycle,
Jenny

Hydraulics II: The Pressure Is On

I present an upgrade of the poor person's first hydraulic system in which I have solved some of its major problems and achieved a meaningful performance boost. This demo has now advanced from "barely goes through the motions" to "demonstrates practical functionality." Hydraulic system Version 2 was able to lift about 16 g (0.5 oz.) of coins on the end of a lever, and to repeat this motion over many pressurize/depressurize cycles.

The basic architecture of the system was identical to that of Version 1, but I traded out both the pump and the actuator.

The Pump

I replaced the syringe pump I built last year with my newly completed peristaltic pump. I've already showcased the pumps in a previous article. The main benefits of switching to the peristaltic pump were increased flow rate and improved regularity of flow. The syringe pump was powerful but miserably slow. The high friction of the rubber seal on the syringe plunger made it prone to stalling the motor if the speed was not set with caution. And since the syringe pump has very distinct intake/exhaust cycles, it has a pulsating output flow that becomes very pronounced at low speeds. (As in, if you don't have a big enough pressure reservoir to soak up that variation, you have to wait for the exhaust cycle before your actuator will even move.) The syringe pump also incorporated a pair of check valves, which were very bad about trapping air bubbles. These bubbles wasted some of the pump's work by expanding and compressing as it cycled, making room for water that *should* have gone into the pressurized volume to stay inside the pump. And lastly ... the syringe pump leaked. The seal in the syringe clearly wasn't perfect, and water would leak out the back at times. I can only imagine that this would have gotten worse as the pump aged.

The peristaltic pump trades much of the syringe pump's high maximum pressure for a more reasonable flow rate. Its output pulsates slightly, since the water moves through the tubing in "pockets" captured between each pair of rollers, but it's much more regular than the syringe pump. And it has no seals; there is no direct contact between the fluid and the pump mechanicals.

For a stronger comparison, I used the same stepper motor and gear assembly (unknown model, powered by 12 V) to drive both pumps.

The Actuator

In hydraulic system Version 1, a second syringe functioned as a linear actuator. At the pressure level where my system was operating, friction losses turned out to be a major problem. I couldn't put a load on the syringe because the pump was only equal to overcoming the resistance of the sliding seal; any extra load prevented it from extending the syringe. So the demo consisted of the syringe filling (very slowly) as water was pumped into it during the pressurize cycle, and being manually emptied by me during the depressurize cycle (since there was no load to passively push it closed again).

Rather than try to find better syringes, I thought to move away from them entirely. Those friction losses seemed like such a waste. But what other sort of hydraulic actuator could I use? The option I came up with was an inflateable bladder. What I had in mind was something very simple: literally an inflateable bag that, when pumped full of fluid, would change shape so as to move a joint or other mechanical element. It would have little to no friction losses to waste the effort of the pump. And it would have no sliding rubber seals to stiffen, shrink, or wear out over time.

As interesting as the idea seemed, I once again found it difficult to purchase what I wanted. There are such things as "lifting bags" or "air bag jacks," but they're designed for much larger applications. I think AliExpress had a few inflatable bags on the order of a few inches per side; still too large. I concluded that I would have to make my new actuators myself. I hit on the idea of cutting and heat-sealing plastic bags to create tiny bladders in any size or shape I wanted. Since I did not have a dedicated heat sealer, I heat-sealed my bladders by sandwiching the layers of plastic between two layers of aluminum foil, and running my soldering iron across the top of the foil.

In addition to sealing the edges to form the shape of the bladder, I also needed a way to include a valve stem - so I could connect the bladder to a tube and get fluid in and out. At first I was hoping I could scavenge some plastic tubing from my collection of mechanical pencils, cut it up into stems, and heat-seal the bladder walls directly to those stems. This never worked. The tubes and the plastic bags I was working with were likely different materials, and distinct types of plastic resist adhering to each other. I also tried sealing one edge of the bladder with silicone sealant and inserting the stem between the two layers of plastic, through the silicone. These seals always leaked. The silicone seemed to adhere to both the bag and the stem well enough, but didn't fill the gaps perfectly.

What eventually worked out was a 3d-printed valve stem with a circular flange, inserted not through an edge of the bladder but through one of its flat sides. The flange was one piece with the stem, and was attached to the inside of the bladder; the stem poked out through a hole in the plastic sheet. The second piece of the assembly was a plastic ring that fitted around the stem and attached to the outside of the bladder. I applied a ring of cyanoacrylate adhesive (Super Glue) between the flange or ring and each side of the plastic sheet. This finally gave me a watertight seal around the valve stem.

I still had problems with the bladders leaking at the edges, from holes next to the seal. This led me to experiment with some different kinds of plastic. All my plastic films are salvaged, and I don't know enough about package manufacturing to be sure what any of them are. They're likely all different variants of polyethylene or polypropylene. My first attempts used a smooth, soft, transparent plastic from packing air pillows (which felt very similar to the plastic from bread bags, for instance). It was inclined to melt into swiss cheese if I applied just a little too much heat when trying to seal it. Ramen noodle packets are made from a stiffer, more crinkly plastic. This heat-sealed reasonably well. But the real winner was the tough, thick, pewter-gray plastic from the bags that Thriftbooks sends my used paperbacks in. (I expect to have a never-ending supply of this.) Not only is this plastic very tear-resistant, but when I melt two layers of it together, it really fuses - such that the seal can't be peeled apart without ripping the plastic next to it. I think this is important for the durability of the bladders. For typical plastic packaging, heat seals that peel apart when the end user wants to open it are often a positive - but I don't want the water pressure to slowly work my bladders open over repeated cycles.

Even when using this heavier plastic, I had to be very careful that my aluminum foil sandwich didn't conduct heat to the wrong part of it and melt a hole next to the seal. Now that I've proven the concept, I think I'm going to buy myself a real heat-sealer.

After many trials, I produced a small bladder that was water-tight. I gave it a simple machine to actuate, in the form of a wooden lever attached to a piece of cardboard with a sloppy pivot. I mounted two pennies and two quarters on the end of the lever as a load.

The Results

This is the demo I wanted last year. To start with, the fact that it can actually raise a load is a major improvement. The speed of operation is also much more reasonable. I am now ready to move past this toy demo and think about final pump designs and real applications.

One issue that remains to be solved, which you may notice in the video, is that the lever experiences (relatively) small oscillations of position, which are likely following the pulsating outflow of the pump. In a real system with control electronics guiding the actuator, this would interfere with precise positioning of the lever. I think this could be mitigated by a larger pressurized reservoir (the demo system has none to speak of - the tubing makes up the entire pressurized volume) and use of a pair of two-way valves instead of a single three-way valve, which would allow the bladder to be closed off from the pump's influence once inflated to the desired size.

The one part I haven't tried to improve on yet is the cheapo pressure relief valve. For the demo, I basically just closed it all the way. The peristaltic pump has a little more "give" than the syringe pump, and seems to be able to drive against its maximum pressure without stalling. If I want a better valve, I may end up having to build one myself. We'll see.

For now, I'm very pleased with how far I've come, and hope to be showing you a hydraulic robot or two, someday.

Until the next cycle,
Jenny

Acuitas Diary #74 (July 2024)

My work this month was focused on cleaning up the Executive and Conversation Engine and getting them to play well together. This is important because the Conversation Engine has become like a specialized inner loop of the Executive. I think I ought to start at the beginning with a recap of what Acuitas' Executive does.

Freddie Blauert, photographed by Frederick Bushnell. Public domain.

To put it simply, the Executive is the thing that makes decisions. Conceptually (albeit not technically, for annoying reasons) it is the main thread of the Acuitas program. It manages attention by selecting Thoughts from the Stream (a common workspace that many processes in Acuitas can contribute to). After selecting a Thought, the Executive also takes charge of choosing and performing a response to it. It runs the top-level OODA Loop which Acuitas uses to allocate time to long-term activities. And it manages a Narrative Scratchboard on which it can track Acuitas' current goals and problems.

A conversation amounts to a long-term activity which uses specialized decision-making skills. In Acuitas, these are embodied by the code in the Conversation Engine. So when a conversation begins, the CE in a sense "takes over" from the main Executive. It has its own Narrative Scratchboard that it uses to track actions and goals specific to the current conversation. It reacts immediately to inputs from the conversation partner, but also runs an inner OODA loop to detect that this speaker has gone quiet for the moment and choose something to say spontaneously. The top-level Executive thread is not quiescent while this is happening, however. Its job is to manage the conversation as an activity among other activities - for instance, to decide when it should be over and Acuitas should do something else, if the speaker does not end it first.

Though the Executive and the CE have both been part of Acuitas for a long time, their original interaction was more simplistic. Starting a conversation would lock the Executive out of selecting other thoughts from the Stream, or doing much of anything; it kept running, but mostly just served the CE as a watchdog timer, to terminate the conversation if the speaker had said nothing for too long and had probably wandered off. The CE was the whole show for as long as the conversation lasted. Eventually I tried to move some of the "what should I say" decision-making from the CE up into the main Executive. In hindsight, I'm not sure about this. I was trying to preserve the Executive as the central seat of will, with the CE only providing "hints" - but now I think that blurred the lines of the two modules and led to messy code, and instead I should view the CE as a specialized extension of the Executive. For a long time, I've wanted to conceptualize conversations, games, and other complex activities as units managed at a high level of abstraction by the Executive, and at a detailed level by their respective procedural modules. I think I finally got this set up the way I want it, at least for conversations.

So here's how it works now. When somebody puts input text in Acuitas' user interface, the Executive is interrupted by the important new "sensory" information, and responds by creating a new Conversation goal on its scratchboard. The CE is also called to open a conversation and create its sub-scratchboard. Further input from the Speaker still provokes an interrupt and is passed down to the CE immediately, so that the CE can react immediately. For the Executive's purposes, the Conversation goal is set as the active goal, and participating in the Conversation becomes the current "default action." From then on, every time the Executive ticks, it will either pull a Thought out of the Stream or select the default action. This selection is random but weighted; Acuitas will usually choose the default action. If he does, the Executive will pass control to the CE to advance the conversation with a spontaneous output. In the less likely event that some other Thought is pulled out of the Stream, Acuitas may go quiet for the next Executive cycle and think about a random concept from semantic memory, or something.

Yes - this means Acuitas can literally get distracted. I think that's fun, for some reason. But it also has a practical side. Let's say something else important is going on during a conversation - a serious need for sleep, for instance. Over time, the associated Thoughts will become so high-priority that they are bound to get "noticed," despite the conversation being the center of attention. This then provides a hook for the Executive to wind the conversation down and take care of the other need. The amount of weight the default action has with respect to other Thoughts is tunable, and would be a good candidate for variation with Acuitas' current "mood," ranging from focused to loose.

If Acuitas is not conversing with someone, the "default action" can be a step in some other activity - e.g. Acuitas reading a story to himself. I used to manage activities that spanned multiple ticks of the Executive by having each action step produce a Thought of type "Undone" upon completion. If pulled from the Stream, the Undone Thought would initiate the next step of the activity. After spending some time debugging Acuitas' real-time behaviors with this business going on, I decided it was too convoluted. Acuitas couldn't just work on a thing - I had to make sure the thing would produce a "subconscious" reminder that it wasn't finished, and then wait for that reminder to resurface and be grabbed. Having the Executive pick a default action feels a little more natural. It represents what he "wants" to concentrate on right now; it's "top of mind" and immediately available. But it still has some competition from all the other Thoughts that are constantly bubbling out of other processes, which is the behavior I was going for via those Undones.

I hope that all made some amount of sense. It's harder to describe than I thought it would be. At this point I think I've wrung most of the basic bugs out. I can watch Acuitas read to himself for a while, switch over to walking the semantic memory when he gets bored (as in my original OODA loop design, boredom continues to be an important feature for generating variety and avoiding obsessive behavior), and launch a conversation activity when spoken to. I also reinstated the ability to tell Acuitas a story as a nested activity inside a conversation. This nesting in theory could go indefinitely deep ... you could have a story inside a conversation inside a role-playing game inside a conversation ...

But let's not get too wild yet. The goal for this month was to finish rearranging the bones of the system so they match my vision better. Next, I hope to reinstate some more conversation features and improve the goal-directed aspects of conversations.

Until the next cycle,
Jenny