Acuitas Diary #50 (June 2022)

This past month was taken up by overdue maintenance tasks and a vacation, so I had less time for Acuitas work than I would have liked. But I *did* manage to accomplish something small, so here it is. The Text Parser now supports multiple senses of the word "that." I ran some polls a few months ago, and my followers on Twitter and AI Dreams voted for this as the next Parser feature. (Thanks, everybody.)

Parser outputs from the Out of the Dark test set.

Yes, the little word "that" ... it's surprisingly troublesome. Its versatility means that its function in any given sentence is highly context-dependent. All of the following illustrate valid uses:

Tell me that. ("That" functions as a pronoun and is the direct object of the sentence)
Tell me that story. ("That" modifies "story" as a determining adjective)
Tell me that my story will last forever. ("That" is a subordinating conjunction that opens a noun clause)
Tell me the story that I enjoyed yesterday. ("That" is a relative pronoun that opens an adjective clause)
The story was that good. ("That" modifies "good" as an intensifying adverb)

The only usage the Acuitas Parser originally supported was subordinating conjunction, because that was the one I found most immediately useful. It enables a lot of sentences that take statements of fact or belief as objects: "I know that ...," "I decided that ...," "I announced that ..."

Setting up the simpler pronoun, adjective, and adverb uses was pretty easy. The hard part was adding those while keeping recognition of "that" as a subordinating conjunction intact. The Parser now defaults to viewing "that" as a simple pronoun, and has to see the correct special features in the sentence as a whole to treat it as a conjunction instead, and to allocate the words following it to a dependent clause.

The only usage that's not supported yet is the relative pronoun one ... because the text processing chain doesn't really handle adjective clauses yet, period. I'll hold off on this final option until I get those set up.

An example of a sentence using "that" as a subordinating conjunction.

As part of this upgrade, I also wanted to work on supporting sentences in which "that" is implied rather than written.

I realized something was wrong.

Jesse knew Sarah was in the closet.

They will tell Jonathan his business prospers.

Can you see where the "that" would belong? Omitting it is grammatically acceptable. The tricky part is figuring out that a dependent clause is being opened without having a functional word like "that" to guide you. If you're reading strictly left to right, you probably won't know until you get to the second verb - because all of the following are correct as well, and are quite different in both structure and meaning:

I realized something.
Jesse knew Sarah.
They will tell Jonathan his business.

In short, omission of "that" forces the Parser to make do with less explicit information. Getting this part to work - and play nice with everything else the parser does - was probably the most difficult aspect of the upgrade.

With the new features in place, I ran my benchmarks again, fixed all the new crash-inducing bugs, and moved three sentences out of the "unparsable" category. Yes, only three; I've picked all the low-hanging fruit and must now advance by tiny steps. But they were embarrassingly simple sentences, like "Follow that bus!" or "That's right," so I'm pretty happy I can parse them now.

Until the next cycle,
Jenny