Acuitas Diary #68 (January 2024)

This month I cleaned up the code of the upgraded Text Parser and ran it on one of my benchmarks again. A quick review for those who might be new: I benchmark on text from children's books. I do some minimal preprocessing, such as separating sentences that contain quotes into the frame sentence and the quoted material. The material is fed to the Parser one sentence at a time. 

No helpful commas to offset that second participle phrase? Rude.

Results are binned into the following categories:

CRASHED: The Parser threw an exception while trying to process this sentence (this category is for my own debugging and should always be zero in final results)
UNPARSEABLE: The Parser does not yet support all grammatical constructs in this sentence, so no attempt was made to run the Parser on it
INCORRECT: This sentence was parsed incorrectly (Parser output did not match golden copy set by me)
CORRECT: This sentence was parsed correctly (Parser output matched golden copy set by me)

For parseable sentences, the benchmarking script also uses GraphViz to generate diagrams of both the golden data structure and the Parser output.

An example of the parser getting confused by two conjunctions in one sentence. This is on my to-do list: I think I at least have the tools to solve it now.

So far I have re-run just one of my three test sets, the easiest one: Log Hotel by Anne Schreiber, which I first added last July. Preparing for this included ...

*Reformatting the existing golden outputs to match some changes to the output format of the Parser
*Updating the diagramming code to handle new types of phrases and other features added to the Parser
*Preparing golden outputs for newly parseable sentences
*Fixing several bugs or insufficiencies that were causing incorrect parses

"One day" is a noun phrase, technically. It gets a red border as a sign that it is modifying the verb to indicate "when," an adverb function.

I also squeezed a couple new features into the Parser. I admit these were targeted at this benchmark: I added what was necessary to handle the last few sentences in the set. The Parser now supports noun phrases used as time adverbs (such as "one day" or "the next morning"), and some conjunction groups with more than two joined members (as in "I drool over cake and pie and cookies").

The end result? ALL sentences in this test set are now "parseable," and two thirds of the sentences are being parsed correctly. I'd like to work on the diagrams some more, and hopefully get my other two test sets upgraded, before I upload the results. Enjoy the samples for now.

Until the next cycle,
Jenny