First impressions of GPT-3

This post and the next few posts are reflections based on my experience playing with GPT-3 for the last few days. It can be hard to tell how impressive a system really is (or isn't), especially one as hyped-up as GPT-3, without trying it first hand - so if you're reading this, you're probably someone who should apply for access and give it a go.

I think my number one takeaway is that GPT-3 can, more often than not, produce output that is just stunning in its originality and fluency - if you know how to talk to it.

GPT-3 essentially functions as the worlds most powerful autocomplete, it takes what you enter and then generates follow on text based on that.¹ What's cool is how well GPT-3 can tune its output based subtle indicators like indicating an age/profession/mood of the "author" in your prompt. This feels powerful in ways the Twitter highlights haven't conveyed. But since it is so open-ended, figuring out the right way to prompt the system to get a desired type of output feels more art than science.

Contrary to Twitter, my impression has been that GPT-3 is pretty brittle in many of the most hyped use-cases, like generating code or writing full articles. There's a minimal sense of "reasoning" (which makes sense given the model), and the system is prone to devolving into nonsense on more extended outputs. In that regard, GPT is nowhere close to what you would consider a general intelligence. But if you carefully craft your prompts, keep your output length constrained and tinker a bit, you can get excellent results.

Overall, GPT-3 feels to me like a herald for an "industrial revolution" in knowledge and creative work. I can imagine a wave of new human-in-the-loop systems built on GPT or like technology that will give knowledge workers a 10-100x more leverage. There's a lot left to figure out, especially from a UX perspective. Still, it does seem inevitable that many jobs (e.g., content writing, customer support, data entry) will be heavily augmented in short order (my guess is within 18 months). The path to full automation across a broader set of industries seems like it will likely be much longer (more like a 20-year time horizon). Still, given the trajectory of improvement, I wouldn't be surprised if my intuitions will be broken again by GPT-4.