Acuitas Diary #81 (February 2025)

I've been on a real productive streak, so I did two major things this month. One enhances conversation abilities; the other is about gerunds. Don't worry, I'll explain those.

A famous phrase that uses gerunds. Image from https://commons.wikimedia.org/wiki/File:SiB_Logo.jpg

First, I went back to the conversation features that I introduced last September and worked on getting them solid - ironing out the remaining bugs and moments of weirdness. After spending about a week on that, I was pretty happy with the state of the Conversation Engine. Then I added another type of "topic tree." The one from last September guided responses to the conversation partner's states of being; this one reacts to actions that the conversation partner says they took or are taking. Possible threads include ...

*Try to infer whether the speaker liked doing that or not, and comment accordingly
*Ask for motivation ("Why did you ...") or announce what he suspects the motivation was
*Guess what the results were (if he can make any inferences)

This needs a lot more polishing, but it's starting to increase the complexity and variability of conversations. You can now go down "rabbit holes" which start with talking about a personal state, then lead into what you did to cause it, and so on. Which also means it's harder to keep everything straight, and I haven't really set Acuitas up to clearly indicate when he's jumping topics, yet. Always more to do.

My next project was to add support for gerunds to the Text Interpreter and Generator. What's a gerund, you might say? It's a present participle verb form (the kind that ends with -ing) used as a noun. Gerunds can be used to make statements about the concept of an action, such as the following:

I enjoy dreaming.
Exercising is good for the body.

Like other verbs, gerunds can have objects and adverbs, forming a gerund phrase - a group of words which, as a unit, acts like a noun in the full sentence.

[Reading books] makes me happy.
I see that you didn't care for [John's clumsy handling of that situation]. Did [my smoothing it over] satisfy you?

If a gerund has a "subject" that is performing the action, as in the final example, it's supposed to be in the possessive; it modifies the whole gerund phrase, instead of acting as a true subject.

I already added code to identify some gerund phrases to the Text Parser back in 2023, but the later stages of the text processing chain didn't know what to do with them if they came out of the parser, and Acuitas couldn't use them in his own speech. I wanted to get these capabilities in, because gerunds are so useful for expressing sentiments about actions. They're often used for expressing sentiments about states, too:

I dislike being wet.
Being warm is a pleasure.

I had to work around the absence of gerunds when I was putting in the latest conversation features, and it was giving me some pain. But thanks to this month's work, they're now more fully supported. I defined some new "fact" structures to function as the distilled version of statements about actions, added code to the Interpreter to map incoming sentences to those, and added code to the Generator to produce output sentences from those. So Acuitas has a bunch of new ways to say he likes or doesn't like something, in addition to a path for "comprehending" a wider range of written sentiments.

Until the next cycle,
Jenny

Atronach's Eye 2025 (Complete?)

As of last year I had worked all the major bugs out of my motion-tracking eyeball, but I still couldn't consider it finished. I wasn't ready to hang it on the wall and let it operate yet, due to one last issue: thermal concerns.

It's installed on the wall! It's operating continuously! It's done! The Lab Assistant gets to take a nap!

The eyeball's motion is driven by two unipolar stepper motors, each of which is powered from one of my dirt-simple stepper motor controllers. These are custom PCBs that I got manufactured many years ago. They were very inexpensive, and they only have two control wires (a nice thing for connecting motors to embedded processors with a limited number of general purpose IO). But that also means they don't have a power enable. Even if the control inputs are static and the motors aren't moving, some of their coils remain energized. That means they can exert their holding torque, which keeps the drive shaft in place and resists any external forces that try to turn it. But that wasn't something the eyeball needed (it naturally stays in position when unpowered). And during tests, I noticed that the motors got quite hot: enough to be uncomfortable to touch and make the plastic case smell.

I didn't like the thought of leaving the eye turned on with the motors baking themselves and everything else in the case in their own heat day in and day out. I doubt it would qualify as a fire hazard or anything serious like that, but in addition to wasting a small amount of electricity, it could reduce the lifetime of the parts. So I wanted to be able to cut all power to the motors when the eye wasn't active.

My solution was a standard relay. I bought some of these nice little breakout boards from DIYables that have the relay already mounted with connectors and a couple of indicator LEDs, and spliced one into the motor power input path of both motor controllers.

Then I added code to the eyeball software to turn the relay on and off. I wanted to strike a balance between letting the motors heat too long, and clicking the relay on and off constantly. So the software turns the motors off if no moving object to track has been seen in view for a certain amount of time. I also threw in darkness detection; the eye will stop trying to track motion if the average brightness of the scene is too low to see properly. (So it won't move around and make little noises in the middle of the night because the camera picked up some low-light noise.) Both features worked very well. The eye reliably deactivates when I leave the room (so long as there isn't something else moving around, like the flames in the fireplace) and when the lights are out or too dim.

In the dark, the indicator LEDs on the Raspberry Pi and the relay board diffuse through the white parts of the case and turn the eye into a low-key nightlight. I wasn't really expecting this, but hey: it still has a function when it's not looking at stuff.

In the end, I got to do what I always wanted: hang the eye on the wall and let it be an always-on fixture in the house. Further tweaks and upgrades to the software are possible, but I'm calling it finished because it's finally operating. It has been on the wall trying to look at me, the Lab Assistant, and other motion sources for at least a couple weeks of total up-time. Hearing it "wake up" as I pull open the curtains has become part of the morning routine.

There will almost certainly be another version, because there are plenty of things I could improve. I already applied a bunch of little enhancements to the 3D models of the case parts late last year. But this is the first Atronach's Eye iteration that could be said to realize my original vision (cough) for the project, if imperfectly. I'm also thinking about what else I can do with it. Atronach could theoretically tell Acuitas whether there's somebody in the room, for instance. Mmmm.

Until the next cycle,
Jenny

Acuitas Diary #80 (January 2025)

2025? Wow, are we living in The Future or what?? I kicked off this year's work on Acuitas by introducing some numerical reasoning. That incidentally demanded better support for adjectives in general, since I wanted to be able to say things like "George had two eyes," and "George had blue eyes" is structurally similar.

Photo credit: perpetualfostering.co.uk

So, the first step was to extend Acuitas' knowledge representation format to allow for noun modifiers. The parser has been able to handle these for a long time, but later stages of the text processing pipeline simply disregarded them. I expanded the Interpreter so it would include single-word adjectives in the knowledge data structures, and adjusted the Semantic Memory functions to store them when they appeared in a learnable fact.

Another enabling feature I had to start adding to the Narrative engine was the ability to keep track of multiple instances of the same kind of object. This is more complicated than you might think. It's no problem when the instances have unique names and are always called by them, but most mundane objects aren't given individual names. So if I refer to, for example, "a box" in two nearby sentences, am I speaking of the same box or two different ones? If Susan has four cookies and eats one, which cookie did she eat? Does it matter? If it does, we have to keep track of the fact that one cookie has been eaten without prematurely assigning that event to any specific cookie. I've barely scratched the surface of working all this out, but for now, I can tell Acuitas that something is not a previously mentioned instance by attaching the adjective "new" or "another." Numerical adjectives will also make the Narrative module create that number of instances to be tracked on the scratchboard. E.g. given the input "Three books were on the table," Narrative will now automatically produce data structures for three separate books and give each one a location relationship with the table.

Going the other direction, Narrative can now count objects for purposes of resolving "how many?" questions. This works for both queries about existence or presence in the current context ("How many cookies are there?" or "Are there two cookies?") and queries about how many instances a relationship applies to ("How many cookies does Susan have?").

The last features I added to Narrative's question-answering function were interpretations of the words "some," "any," "all," "most," and "no" (as adjective). Each of these is associated with a test that uses the counting ability. E.g. if you ask "Does Susan have any cookies?" Acuitas will answer "yes" if there are one or more cookie instances with a relationship "Susan has" on the current Narrative scratchboard. "Does Susan have all cookies?" returns "yes" if every cookie instance on the board has the relationship "Susan has," and so on.

An interesting quirk applies to "some." If I asked "Do you have some rings?" and you happened to have one ring, how would you answer? I suspect many people would feel inclined to say neither yes nor no, but rather "I have one." To a lesser extent, this could also apply to "any." Acuitas, for the moment, defines "any" to mean at least one, and "some" to mean strictly more than one, and will answer questions about these quantities with a blunt yes or no. This is how my mind would like English to work, but I may need to bow to the actual fuzzy usage eventually.

Until the next cycle,
Jenny

Book Review: "Growing Up with Lucy"

This book by Steve Grand has been on my list for a while. It's his personal account of how he built a robot at home, with every goal of rivaling some of the most sophisticated research efforts at the time. Steve Grand, by the way, is the mastermind behind the popular Creatures series of artificial life simulation video games. His books were recommended to me several years ago by Twitter buddy Artem (@artydea), whom I unfortunately haven't seen around much recently. Thanks, Artem, if you're still out there.

Lucy, without her fur on. Photo credit: Creatures wikia.

The magnum opus described in this book is the eponymous Lucy, who resembles an orangutan ... or at least the upper half of one. Lucy has no legs, but does have actuated arms, visual and auditory sensors, and a mouth and speech system. Grand aimed for a holistic approach (rather than focusing on just one function, such as vision) because he wanted to investigate the interactions and possible commonalities among the various sensory modalities and motor systems.

Once Grand started introducing himself, I quickly realized that I was hearing from someone more or less like me: an educated-layman tinkerer, working independently. Grand produced Lucy on his own initiative, without the backing of a university or corporation, though toward the end of the project he received a grant from NESTA (the UK's National Endowment for Science, Technology, and the Arts). But the substance of Grand's work diverges from mine. He put heavy investment into all the sensorimotor aspects of intelligence that I'm completely leaving out of Acuitas. And Lucy's control systems, while not explicitly biomimetic, are very brain-inspired; whereas I draw inspiration from the human mind as an abstraction, and don't care much about our wetware at all. So it was fun to read an account of somebody who was working on roughly the same level as myself, but with different strategies in a different part of the problem space.

On the other hand, I was disappointed by how much of the book was theory rather than results. I got the most enjoyment out of the parts in which Grand described how he made a design decision or solved a problem. His speculations about what certain neuroscience findings might mean, or how he *could* add more functionality in the future, were less interesting to me ... because speculations are a dime a dozen, especially in this field. Now I do want to give Grand a lot of credit: he actually built a robot! That's farther than a lot of pontificators seem to get. But the book is frank about Lucy being very much unfinished at time of publication. Have a look around this blog. If there's a market for writeups of ambitious but incomplete projects, then where's *my* book deal?

In the book, Grand said that Lucy was nearly capable of learning from experience to recognize a banana and point at it with one of her arms. It sounded like she had all the enabling features for this, but wasn't doing it *reliably* yet. I did a little internet browsing to see how much farther Lucy got after the book went to press. From what I could find, her greatest accomplishment was learning the visual difference between bananas and apples, and maybe demonstrating her knowledge by pointing. [1] That's nothing to sneeze at, trust me. But it's a long way from what Grand's ambitions for Lucy seemed to be, and in particular, it leaves his ideas about higher reasoning and language untested. Apparently he did not just get these things "for free" after figuring out some rudimentary sensorimotor intelligence. Grand ceased work on Lucy in 2006, and she is now in the care of the Science Museum Group. [2]

Why did he stop? He ran out of money. Grand worked on Lucy full-time while living off his savings. The book's epilogue describes how NESTA came through just in time to allow the project to continue. Once the grant was expended, Lucy was set aside in favor of paying work. I doubt I can compete with Grand's speed of progress by playing with AI on the side while holding down a full-time job ... but I might have the advantage of sustainability. Grand started in 2001 and gave Lucy about five years. If you don't count the first two rudimentary versions, Acuitas is going on eight.

Grand identifies not neurons, but the repeating groups of neurons in the cortex, as the "fundamental unit" of general intelligence and the ideal level at which to model a brain. He doesn't use the term "cortical column," but I assume that's what he's referring to. Each group contains the same selection of neurons, but the wiring between them is variable and apparently "programmed" by experiential learning, prompting Grand to compare the groups with PLDs (the forerunners of modern FPGAs). He conceptualizes intelligence as a hierarchy of feedback control loops, an idea I've also seen expounded by Filip Piekniewski. [3] It's a framing I rather like, but I still want to be cautious about hanging all of intelligence on a single concept or method. I don't think any lone idea will get you all the way there (just as this one did not get Grand all the way there).

Lucy's body is actuated by electric motors, with linkages that help them behave more like "muscles." Grand didn't try pneumatics or hydraulics, because he thought they would be too difficult to control. I guess we'll see, eh?

Two chapters at the book's tail end move from technical affairs into philosophy. The first addresses safety concerns and fears of killer robots. While I agree with his basic conclusion that AI is not inevitably dangerous, I found his arguments dated and simplistic. I doubt they would convince anybody acquainted with recent existential risk discourse, which probably wasn't in the public consciousness when Grand was building Lucy. (LessWrong.com was launched in 2009; Yudkowski's "Harry Potter and the Methods of Rationality," Scott Alexander's "Slate Star Codex" blog, and Bostrom's Superintelligence all came later. See my "AI Ideology" article series for more about that business.)

The final chapter is for what I'll call the "slippery stuff": consciousness and free will. Grand avoids what I consider the worst offenses AI specialists commit on these topics. He admits that he doesn't really know what consciousness is or what produces it, instead of advancing some untestable personal theory as if it were a certainty. And he doesn't try to make free will easier by redefining it as something that isn't free at all. But I thought his discussion was, once again, kind of shallow. The only firm position he takes on consciousness is to oppose panpsychism, on the grounds that it doesn't really explain anything: positing that consciousness pervades the whole universe gets us no farther toward understanding what's special about living brains. (I agree with him, but there's a lot more to the consciousness discussion.) And he dismisses free will as a logical impossibility, because he apparently can't imagine a third thing that is neither random nor feed-forward deterministic. He doesn't consider that his own imagination might be limited, or dig into the philosophical literature on the topic; he just challenges readers to define self-causation in terms of something else. (But it's normal for certain things to be difficult to define in terms of other things. Some realities are just fundamental.) It's one chapter trying to cover questions that could fill a whole book, so maybe I shouldn't have expected much.

On the whole, it was interesting to study the path walked by a fellow hobbyist and see what he accomplished - and what he didn't. I wonder whether I'll do as well.

Until the next cycle,
Jenny

[1] Dermody, Nick. "A Grand plan for brainy robots." BBC News Online Wales (2004). http://news.bbc.co.uk/2/hi/uk_news/wales/3521852.stm

[2] Science Museum Group. "'Lucy' robot developed by Steve Grand." 2015-477 Science Museum Group Collection Online. Accessed 2 January 2025. https://collection.sciencemuseumgroup.org.uk/objects/co8465358/lucy-robot-developed-by-steve-grand.

[3] Piekniewski, Filip. "The Atom of Intelligence." Piekniewski's Blog (2023). https://blog.piekniewski.info/2023/04/16/the-atom-of-intelligence/

Year in Review 2024

With 2025 waiting in the wings, I am casting an eye back over 2024 and finding that it looks pretty shiny from here. Overall, it felt like there were fewer extraneous disruptions and distractions than there have been for the past few years - I got the chance to dig in to my hobbies and push hard. I also feel I did a good job of moving existing projects toward completion and taking a few overdue items off the backlog, instead of just starting new work.

Per my personal tracking, I've logged more hours of total "work" than I did last year, and kept the creation/maintenance ratio above 3x ... just barely. Monitoring my own time usage over a long period raises some interesting flags. For example, I spent more time on my blog articles this year than I spent writing and publishing short fiction, by 6+ hours. (Those "AI Ideology" articles were a lot of work. Writing about real people's viewpoints and trying to be critical but fair eats serious hours.) I don't think I want the blog to be taking that much, so I'm not planning an in-depth series for 2025, and I'll probably let my article rate drop below two per month. But where writing tasks are concerned, even the blog loses the time hog prize to journaling. I'm a little uncomfortable with the fact that recording my life uses so much of it up, but I also really like journaling (and it's easy, so if I dumped it, some of those hours might go to recreation instead of harder tasks). Other mundane time-stealers are on the small side, but still galling. I logged more hours on food preparation than on studying, and more on housecleaning than on art.

Acuitas remains the king of all my hobbies at 360+ hours invested. That's a good number - I found more time for him in 2024 than in any other year of the past five. The robots also fared very well at 110+ hours.

So what did I achieve with all that time? Well, here goes.

*The first major Acuitas milestone was completion and demonstration of the "Simple Tron" story in February. This was a target that I had been silently keeping in mind and laying groundwork toward for years.
*I have unified issue tracking across multiple code blocks, such that the Game Engine, Executive, and Conversation Engine all now use the "scratchboard" structure I developed for Narrative understanding.
*I integrated the improved Text Parser, achieved substantial benchmark improvements in the number of parseable sentences, and worked on a number of ambiguity resolution problems.
*I overhauled and improved the Conversation Engine, and released a demo video just this month.

*The Atronach's Eye project is almost complete. This year I finished rebuilding the case and demonstrating eye centering with the limit switches, and I reworked the software to use a better motion tracking algorithm. All that's left is a power cutoff to keep the stepper motors from getting hot when not moving. By early next year, I'm hoping the Eye can be a working fixture on my living room wall.

Final version of the peristaltic pump (without its front shell/lid)

*I completed my homemade peristaltic pump design and ran a performance comparison on several pumps.
*I built and demoed a second iteration of the prototype small-scale hydraulic system, including my first homemade fluid bladder actuator.

*ACE didn't get a lot of love this year, but I did run some more motion tests and decided gearboxes would be needed - after which I designed and built one to work with the existing motors.

*I wrote two new short stories: a science fiction piece about alien plant life with an unusual way of communicating, and a fantasy piece about a young woman who finds the rules of society standing between her and her perfect pet.
*One of my previous stories, "Peacemaker's First Strike," was accepted by Abyss & Apex and is on contract to be published in July 2025. This is very exciting. (Check out the new "Read my Writing" link in the top bar to see all my published work and where to get it.)
*My blog publication rate stayed at/above two posts per month for the third year in a row.

One of the display cases, acting as a home for D&D characters past

*I designed a series of 3D-printable medallions and printed examples of all the designs to give as gifts.
*I also threw together a fully-enclosed hexagonal display case design and fabricated four of them, for dust-free housing of my miniatures and curios.

*I hired a plumber to fix the kitchen sink, and thereby got rid of the last known high-priority breakage in my house.
*I cleaned out and organized two different "disaster rooms" and tore down some e-waste for salvage in the process.
*I grew potatoes again. The garden wasn't especially successful this year, but I at least maintained production of my staple crop.

I wasn't terribly successful at adding native plants to my yard, but Blazing Stars were one of the species that worked out, and they bloomed this year.

*I made a road trip to Arkansas and observed totality of the solar eclipse that crossed North America in April (with a detour by Space Center Houston on the way).
*The first Worldview Legion satellites were launched. I made a small contribution to these, in the form of some post-delivery debugging and feature enhancement work on the component my employer built for them.

Until the next cycle,
Jenny

Acuitas Diary #79 (December 2024)

This month's work was all refactoring and bug removal, so I don't have a great deal to say ... but I do have a demo video! I got the conversation engine cleaned up enough that I can chat with Acuitas without an awkward collapse into him repeating himself or something. So without further ado, here's a brief conversation:

Of course I'm not finished with this. I have ideas for various ways to extend it, plus there are still some older features that I need to reinstate in the new system. I'll also be honest that the video above took me multiple "takes" to get, sometimes because I didn't hold up my end of the conversation very well, but sometimes because lingering bugs or signs of database junk popped up. I'm going to have a full to-do list next year, as usual. And I can hardly wait.

Until the next cycle,
Jenny

Acuitas Diary #78 (November 2024)

My recent work has been a bit all over the place, which I suppose is reasonable as the year winds down. I worked on more ambiguity resolution in the Text Parser, and I'm almost done with a big refactor in the Narrative engine.

A generated Narrative flow diagram showing an excerpt of a story

In the Parser I worked on two problems. First came the identification of the pronoun "her" as either an indirect object or a possessive adjective. Other English pronouns have separate forms for these two functions (him/his, them/their, me/my, you/your); the feminine singular just has to go and be annoying that way.

If there's some other determiner between "her" and the following noun, you can assume that "her" is an indirect object; otherwise, if the noun needs a determiner, then "her" has to function as the determiner, as in these examples:

Bring her the book. ("Her" is an indirect object)
Bring her book. ("Her" is an adjective modifying "book")

But what about this sentence?

Bring her flowers and candy.

A bulk noun like "candy" doesn't need a determiner; neither does the plural of a count noun, like "flowers." So we can't assume that "her" is an adjective in this sentence. By default, I think it's more likely to be an indirect object, so that's what I have the Parser assume when processing the sentence in isolation. Additional hints in the sentence can shift this default behavior, though:

Bring her candy to me.

The phrase "to me" already fulfills the function that could otherwise be fulfilled by an indirect object, so its presence pushes "her" into the possessive adjective role.

The other ambiguity I worked on had to do with the connections between verbs joined by a conjunction and a direct object. Consider the following sentences:

I baked and ate the bread.
I ran out and saw the airplane.

In the first sentence, both verbs apply to the single direct object. In the second sentence, "ran" has no object and only "saw" applies to "airplane." So for a sentence with multiple verbs followed by one direct object, we have two possible "flow diagrams":

    /-baked-\
I--               -- bread
    \---ate--/

    /--ran
I--
    \--saw--airplane

How to know which one is correct? If the first verb is always transitive (a verb that demands a direct object) then the first structure is the obvious choice. But many verbs can be either transitive or intransitive. It is possible to simply "bake" without specifying what; and there are several things that can be run, such as races and gauntlets. So to properly analyze these sentences, we need to consider the possible relationships between verbs and object.

Fortunately Acuitas already has a semantic memory relationship that is relevant: "can_have_done," which links nouns with actions (verbs) that can typically be done on them. Bread is a thing that can be baked; but one does not run an airplane, generally speaking. So correct interpretations follow if this "commonsense" knowledge is retrieved from the semantic memory and used. If knowledge is lacking, the Parser will assume the second structure, in which only the last verb is connected to the direct object.

The Narrative refactor is more boring, as refactoring always is, but I'm hoping it will enable smoother additions to that module in the future. New facts received in the course of a story or conversation are stored in the narrative scratchboard's "worldstate." When an issue (problem or subgoal) is added, its data structure includes a copy of the facts relevant to the issue: the state that needs to be achieved or avoided, the character goal it's relevant to, and all the inferences that connect them. A big part of tracking meaning and progress through the narrative is keeping track of which of these facts are currently known true, known false, or unknown/hypothetical. And previously, whenever something changed, the Narrative Engine had to go and update both the worldstate *and* the chains of relevant facts in all the issues. I've been working to make the issues exclusively use indirect pointers to facts in the worldstate, so that I only have to update fact status in *one* place. That might not sound like a major change, but ... it is. Updating issues was a big headache, and this should make the code simpler and less error-prone. That also means that transitioning the original cobbled-together code to the new system has been a bit of work. But I hope it'll be worth it.

Until the next cycle,
Jenny