Behind the scenes of OpenAI’s ChatGPT writing ability

I asked ChatGPT, the artificial intelligence tool that’s taking the world by storm, to write an essay about venture investing in AI.

• While it impressively picked up Axios Smart Brevity style, it struggled to remember certain directives, and its writing was quite basic.

Why it matters: Despite growing fears of making certain creative workers (i.e., journalists) obsolete, OpenAI’s writing tool is a long way from being able to replace a writer in our newsroom.

Driving the news: Buzzfeed on Thursday announced it would use AI to “enhance” certain kinds of content, which sent its battered stock on a tear and amplified latent fears of machines coming for white-collar jobs.

Flashback: A couple of weeks ago, venture capitalist Marc Andreessen openly mused on Twitter whether ChatGPT could write venture-focused writing.

How it happened: Using some instructions a reader shared with my colleague Ina Fried, I first “taught” ChatGPT about the Smart Brevity style, by copy and pasting a description published by Axios.

• As a warmup, I asked it to write an article describing ChatGPT in this style. I spent some time giving it feedback on the style.
• Then I tested a few different prompts that could potentially yield content about the types of topics featured in Axios Pro Rata, while correcting its mistakes along the way.
• Notably, I eventually had to request that it add sections with specific titles and contents, though it did well enough to generate the relevant text.
• After narrowing it down to three finalist essays, I picked the one above.

Between the lines: ChatGPT is pretty long-winded, repeats itself a lot, makes a lot of broad statements, and inserts a lot of inaccurate details.

• This wouldn’t meet our newsroom quality bar, and it certainly won’t win any journalism awards.

The intrigue: Still, ChatGPT picked up surprisingly fast and well on summarizing deal news (or “blurbs”), in the style used in Pro Rata’s daily newsletter.

• After a few false starts, I asked it to write a summary of Hopin’s 2020 Series B funding round, and it wrote this: “Hopin, a London-based virtual event platform, raised $125 million in a Series B funding round led by IVP with participation from Accel, Northzone, and Seedcamp.”
• I was also able to get it to write a summary of Facebook’s acquisition of Oculus, and Uber’s IPO announcement.

Lastly, I copy-pasted Coinbase’s announcement of its 2018 Series E funding round, and asked ChatGPT for a summary.

• After one round of feedback, it wrote: “Coinbase, a San Francisco-based cryptocurrency exchange, announced a $300 million funding round led by Tiger Global Management, with participation from Y Combinator Continuity, Wellington Management, Andreessen Horowitz, Polychain and others, at a post-money valuation of over $8 billion.”

The big picture: Despite all the recent noise about AI displacing journalists in the future, the reality is that newsrooms have been using software to write some of their content for years now.

• The Associated Press has been automating sports match recaps, and corporate earnings coverage, via a partnership with Automated Insights for several years, for example.
• Bloomberg News even has its own AI, named Cyborg, that pumps out key coverage like corporate earnings summaries. The Washington Post and Forbes also have AI tools that write business and other content, per a 2019 NY Times story.

The bottom line: Human venture reporters can hold onto their jobs, at least for a while longer.