I hope to have an Acuitas update out later this month, but in the mean time, I'm playing with somebody else's AI -- specifically, the GPT-2 text generator from OpenAI.
For those who aren't up on the latest AI news, GPT-2 has ignited controversy. It's so good at spitting out plausible-sounding text that it frightened some people, who saw visions of an internet flooded by mass-produced fake news. OpenAI therefore declined to release the biggest and best version of GPT-2, upsetting other people who thought the organization was breaking its promise to be, well, open. Published examples from the most advanced variant -- including an amazingly coherent and stylistically convincing story about scientists discovering unicorns -- can be seen on OpenAI's blog. This AI is scary good at sounding like a human writer.
But let's say you're not a bad actor who wants to mass-produce lies and scatter them around the internet. And let's say you're interested in something more practical than unicorn fiction. Is there anything worth doing with this algorithm? Recently I realized that I would love a text generator to write all the corporate relations and advocacy e-mails that I, as an engaged member of my community, ought to be constantly churning out. That could save me lots of time.
I generated the examples below on the Talk to Transformer website, which provides an interface to the mid-size GPT-2 model (the best one that's been released to the public so far). Sometimes I truncated the output when GPT-2 stopped in an awkward place.
As one example, I like to buy meat and dairy alternatives, and sometimes I have to ask businesses to carry them. Let's see if GPT-2 can help.
MY PROMPT:
Dear Wendy's,
I am writing to ask that you consider serving the Impossible Burger at your restaurant.
GPT-2's efforts to extol the Impossible Burger were hampered by the fact that it has no idea what the ingredients are ... and is completely willing to make stuff up. These imaginary versions of the IB contain chicken and no chicken simultaneously, or contain a whole gamut of meats while still somehow managing to be vegetarian. Um ...
Dear Wendy's,
I am writing to ask that you consider serving the Impossible Burger at your restaurant.
Although its name may be "The Impossible", I believe that I'd like our customer to know it's real burger:
No MSG, high fructose corn syrup, artificial colors, GMOs, colors or color additives, and soy or gluten free.
Our Impossible Burger includes chicken, ham and veggies.
We make the Impossible Burger from fresh ground beef and cooked turkey.
Yes, all meat products are sold fresh, no exceptions.
Is this Burger a fast food?
Yes, but the Impossible Burger is actually made from fresh fresh, lean ground beef, cooked turkey, and chicken. It's a vegetarian menu staple, and the best thing about this menu, not only for vegetarians, but for vegans looking for a less processed, lean burger.
It's a burger that's healthier and the one you'll love. It's a food that tastes good and is better for you. It's your choice and your choice alone.
To me, it's simple; it tastes good.
Is it available in the USA?
Yes, we will open our restaurant in the US of A.
Does this look like the Impossible Burger I'm ordering?
Dear Wendy's,
I am writing to ask that you consider serving the Impossible Burger at your restaurant. According to the company's marketing materials, Impossible's Impossible Burger is based on a "real-life burger" called "Anchovy," which they say is "super tasty, but extremely difficult to replicate." Unfortunately, the product that's come up in restaurants is actually just a combination of chicken and onion purees, and no real chicken (no Angus, no beef, no salmon). Instead it uses the fat-free, naturally gluten-free, paleo (aka vegan) and low-carb meat that's commonly served in our restaurants today! This recipe is called "Shimmy" and makes approximately 12 medium-sized patties, or enough for 3 guests. In fact, it's so easy to make and we've used lots of the original Impossible burgers before.
In my opinion, the only thing that makes this recipe even easier is the inclusion of coconut oil. This combination of ingredients is not as greasy, crispy and salty as other Impossible burgers. It leaves the taste and texture untouched, and is easy to microwave in 30 second intervals or at a quick and easy to put in your microwave.
I have many years of experience preparing and eating Impossible burgers, which was why I made such a note on this page back in 2006.
Sometimes GPT-2 turned traitor and denounced the Impossible Burger instead of requesting it. I particularly like this bizarre example. "The Burger." Cue ominous music.
Dear Wendy's,
I am writing to ask that you consider serving the Impossible Burger at your restaurant.
The Burger
It is a mystery what, if anything, Impossible Foods ever has done to cause such outrage within my household; but to my knowledge, I've never seen such animosity displayed by one of its members. I have come to terms with the fact that you will likely not eat the Impossible Burger.
My wife has spent her entire career (the bulk of her adult life) trying to cook and serve meals with quality, creative ingredients. In fact, it is as if the company is trying to sabotage her, because she knows that any recipe she comes up with will be nothing short of incredible. She is not going to take a hardball approach when it comes to your Impossible Burger—the Impossible Burger will only be accepted after a series of tests are completed.
As I type I think to myself, "If only they could cook something like mine, it would be so much better".
For a time, Impossible Foods planned on using chicken to serve the Impossible Burger, but the team was forced to cut back to one-quarter of the chicken for various reasons and the price that I paid for my chicken was about the same as my regular chicken. Even though I knew what it was making me sick when I opened the box, I had zero complaints with it.
While the previous example mentioned the correct manufacturer of the burger (Impossible Foods), GPT-2 was equally prone to making up fantasies about where this burger comes from. It also got the idea that the Impossible Burger has already begun to spawn myths and concept gateways, which sounds great. From now on I only want to eat mythic food.
Dear Wendy's,
I am writing to ask that you consider serving the Impossible Burger at your restaurant. That is a concept known as a patty and, according to popular myth, is based on the concept of eating hamburgers in one continuous bite. Yes, according to popular myths we've got the Impossible Burger. It was originally created by Pat Sajnak at The Impossible Burger, a restaurant in California. As far as I'm concerned it's a simple myth to make.
I love patty and I would love to have it on our menu and in my restaurant. I have always loved the Impossible Burger so it would be incredibly exciting to serve it in my restaurant. It would serve as a gateway to the idea of patty as a concept. The Impossible Burger does the Impossible Burger a huge disservice by making it harder to enjoy. The Impossible Burger is the burger that will become my staple. This idea seems so strange to someone accustomed to conventional burger patty recipes. My philosophy has to be to have fun making it for you and I am not interested in the idea of making the Impossible Burger for you to try to impress your patrons.
For those of you in California and other parts of the country, I would encourage you to come up with alternative versions of Impossible Burger recipes. I'm not sure we should start here but I could certainly suggest experimenting.
I think the sample below is my favorite. This time, GPT-2 managed to stick to the right style for a customer telling a business why she's enthusiastic about a product. It correctly guessed that the IB is vegetarian. It didn't get -too- nonsensical, but it lied about my being from Houston, and the e-mail is still weird enough that I wouldn't send it.
Dear Wendy's,
I am writing to ask that you consider serving the Impossible Burger at your restaurant. On the bright side, if Impossible Burger continues to make an appearance at your stores, we can order our own! You don't have to agree with our mission to give all people a better quality of life, but we do know this hamburger is not only perfectly good, it's also delicious.
As a vegetarian who lives in Houston, I am excited for Impossible to be an option for our local food industry. I also live in a fast food town, and believe in the principles of a family friendly restaurant. Impossible Burger comes from a family of vegetarian chefs that is the backbone of the Wendy's family, and our success has allowed me to experience the deliciousness of good, sustainable food in restaurants, and in the comfort of my own home.
I want to thank you for this extraordinary opportunity and wish you success in your business endeavors.
Okay *dusts hands,* let's try a political e-mail. To avoid ruffling any feathers, I've deliberately picked a silly topic. Can GPT-2 help me ask my senator to cut subsidies for BALLOONS?
MY PROMPT:
Dear Senator Gardner,
Please reduce government funding for the party balloon industry. It is a waste of taxpayer dollars.
Once again, GPT-2 sometimes forgot that we were supposed to be arguing against party balloons and started playing devil's advocate. The little brat.
Dear Senator Gardner,
Please reduce government funding for the party balloon industry. It is a waste of taxpayer dollars. In addition to the cost of buying each political balloon, you may pay a little while for extra postage or delivery. The cost would equal that of one balloon if you ordered all of them. I have heard it from many of my friends, but you shouldn't waste money on it.
I appreciate the opportunity to speak about this issue, with Senator Gardner.
First of all, I appreciate the hard work of those who are trying to save this country. You are leading many a campaign. The government should be held accountable to those who elected it to govern. In order to prevent further harm to our country, one of the first things you must do is save the balloon industry.
My question is simple: Is it worth it for the American people, our country's children, or future generations of Americans to be deprived of the opportunity to learn about balloon physics, because it costs a lot?
You may say the answer is no.
I know it is worth it for the American people, who must learn about the world of balloon physics, from a government agency that has to deal with the fact that the balloons are designed to be dropped as far away as possible from Earth.
It generated a couple of superb accidental puns. Maybe Mr. Gardner's staff will be so busy laughing that they won't notice the rest of the e-mail is nonsense.
Dear Senator Gardner,
Please reduce government funding for the party balloon industry. It is a waste of taxpayer dollars. The Republicans are spending tens of millions on the party balloon industry, while the Democrats have used those billions to improve quality of our roads, streets and bridges.
The Republicans will now attempt to use the tax increase to further their political agenda to the detriment of the American people. They are using the tax increase to advance the agendas of their big bank bank donors and special interest lobbyists. There is nothing in the Constitution guaranteeing a voice for the citizenry. In its place, we propose a constitutional amendment to guarantee the voice of the people through nonpartisan, transparent elections to provide for fair elections. The Republican Party will be forced to take a step back and reconsider its agenda because the public wants their voice to trump those of the big financial interests.
The citizens and politicians from all parties have had enough of this dirty system and have a right to determine if they will continue the party balloons and for how much, if any, of it. Citizens must elect fair elections for representatives.
There are now over 10,000 people actively voting for Senator Gardner. The more than 10,000 will need to change the system.
Senator Gardner has supported the "free enterprise" policies that created the ballooning party balloon industry and has tried to stop funding it and take away our freedoms
Dear Senator Gardner,
Please reduce government funding for the party balloon industry. It is a waste of taxpayer dollars.
"The most likely mechanism for funding an entirely unaffordable party balloon is to raise the price of the party balloon. The real cost is so massive that, if inflation can be controlled, party balloons and balloons with the party's logo are not as appealing as balloons full of water, watermelon slices, and the rest of the party's paraphernalia. The party balloon industry has not yet managed to capture the public imagination. However, one party balloons, which have been widely reported, are a huge success and are quite a seller at inflated prices. In this economy, party sales represent just under 10% of the economic output. For the industry to flourish, the party revenue must be brought in more than double.
I propose that you remove the cap on the party balloon industry's revenues. If you do not change any of the party balloon limitations so that parties no longer have to pay a one-time fee or if you are not aware of any of the specific limitations for new parties and balloon manufacturers, I think that you should ask the American Society of Civil Engineers to reconsider the existing party balloon caps or the amount of funds available.
Oh, no. GPT-2! You read a Trump speech, didn't you? Love him or hate him, he's not exactly a pillar of eloquence. How can we excise this from the training set?
Dear Senator Gardner,
Please reduce government funding for the party balloon industry. It is a waste of taxpayer dollars. Let's do it now while the bill is still in committee — even if it looks like we are not going to be able to keep it, even if it is a no-brainer, even if we're talking about it this year, this is a no-brainer. And here is what I'd say to you right now, Senator Gardner, it's about the budget… if you and I go back to the Senate and we see how much money is already appropriated each year, how much money has already been appropriated each year, how much money is allocated every year and the question becomes is there any scope for the American people to continue to fund their own governments through the tax code that's already there… I know, I know, you are going to say we haven't heard you and your people say that we don't want to do that but we don't just sit here and listen to the people in Washington tell us how to do it. For our next Budget it's really important that we make sure we do something. We're having to. We can't keep doing it the same way. We have to make some changes in government, we just haven't made any new investments in government programs. We've got to make some investments.
This one wins a prize for "estimate the cost of getting all that money." Sounds like a pit of bureaucratic despair.
Dear Senator Gardner,
Please reduce government funding for the party balloon industry. It is a waste of taxpayer dollars.
We are urging Senator Gardner to allow us to put up a reasonable estimate of the cost of getting all that money and using it only on the party balloon and not on any other purpose.
Sincerely,
The Young Americans of Florida
In the next few years or weeks you will be invited by our supporters in the Republican party to come out with your own ideas on what is most important to me.
On Friday June 28th 2016, I hope and pray you have a wonderful summer. There is much opportunity ahead.
So what were the best ones? Unfortunately, like any fool, GPT-2 sounds wisest when it says as little as possible.
Dear Senator Gardner,
Please reduce government funding for the party balloon industry. It is a waste of taxpayer dollars. The American people must know, "What do you do with the money you lose?" They would rather keep it in the bank.
Dear Senator Gardner,
Please reduce government funding for the party balloon industry. It is a waste of taxpayer dollars. Please.
Thank you, Senator Gardner, for your service to the nation.
[Applause.]
Conclusion: this exercise was wonderfully entertaining, but I will not be outsourcing any of my activism to GPT-2.
Compared to previous attempts at text generation, GPT-2 is an amazing achievement. It almost always uses correct grammar and spelling. It remembers things it already used in a piece. It manages to stay (almost) topical. And baby Acuitas wouldn't do remotely this well if I somehow tried to squeeze an essay out of him. Still, on any task that requires honesty and reasoning ability, it seems GPT-2 falls flat. It's possible it would do better if I went to the trouble of re-training it on my existing collection of sent e-mails, but that's a pretty small data set. I think I'll just have to keep writing my own.
Until the next cycle,
Jenny
