How I built a Lenny chatbot?

  Lenny’s Newsletter is great, but it’s inherently one-sided. It talks to you, but you can’t talk back.   Wouldn’t it be awesome if you could ask Lenny’s Newsletter a question?

  Well, now that’s possible. 

  Over the course of last week I built an AI-powered chatbot for Lenny that uses his entire newsletter archive to answer any question you have about product, growth, and startups. It’s built with GPT-3 and it took a couple hours to do, end to end. In this post, I’ll break down exactly how the Lenny Bot works so you can learn to build one yourself.

   AI technologies like GPT-3 are still in their infancy, but they’re going to be everywhere soon. Staying on top of how they work is going to be crucial to your career in tech, and especially in building product. The best way to prepare for a fast-approaching future is to dive in and get your hands dirty. That’s what I’ve been doing over at Every—I’m writing weekly about my experiments with using GPT-3 to build chatbots, understand patterns in my thinking, and organize my notes. I’m trying to understand these technologies as deeply as possible, and I want to share with you what I’ve learned.

    It might seem intimidating to get started, especially if you don’t have a technical background. But I’m going to start at the very beginning. You’ll be able to understand what I’m talking about and begin using it yourself, no programming required. (And if you have any questions, you can always paste them into ChatGPT—it’ll give you good responses ;)

  Preamble: GPT-3 vs. ChatGPT

   You’ve probably heard of both GPT-3 and ChatGPT. Maybe you use those terms interchangeably, or maybe you’re not really sure what the difference is. It’s worth taking a minute to understand how they differ.

   GPT-3 and ChatGPT are both “large language models.” These are machine-learning models that are very good at generating natural-sounding text, code, and more. They’re trained using large data sets of text, and this helps them get very good at lots of natural-language tasks, like answering questions, writing marketing copy, and holding conversations. So what’s the difference between them? And why is it important?

   GPT-3 is a general-purpose language model—it can hold conversations, write code, complete a blog post, do translation tasks, and more. You can think of it like a flexible know-it-all that can expound on any topic you want. 

   ChatGPT is a version of GPT-3 that’s been turned into a friendly, inoffensive extrovert. Basically, it’s been specifically trained to be good at holding conversations. OpenAI does this by repeatedly holding conversations with the model and then rewarding it for good responses and punishing it for bad ones—a process called Reinforcement Learning from Human Feedback. 

   You’d think since we’re building a chatbot we’d use ChatGPT, right? Unfortunately not. OpenAI hasn’t created a way for us to interact with the ChatGPT model directly—you can only use it through the ChatGPT web app. So it’s not suitable for our purposes.

    We want to be able to interact with the model directly, not through an intervening app. So instead we’ll use GPT-3 for our explorations. It’ll give us all the power and flexibility we need to build a chatbot. 

    We’ll do it in two ways: using OpenAI’s Playground to start, and with a little bit of code after that. The Playground is a web app that lets you prompt GPT-3 directly and get responses back, making it a great place for us to experiment.

  Let’s start there and see how things go.

   The basics of GPT-3

   The really basic way to explain GPT-3 is that it likes to finish your sentences for you. You provide it with a starting set of words, and it tries to figure out the most likely set of words that follow from your input. You can provide any string of words. It’s very flexible and can talk about anything you want, from product management to astrophysics.

    The set of words you provide is called a prompt, and the answer you get back from GPT-3 is called a completion.

    Below is a simple example in the GPT-3 Playground. The non-green text is what I typed in as a prompt, and the green text is what GPT-3 returned as the completion.

   Turning GPT-3 into a chatbot

    Right now we have the bot answering questions, but how can we get it to actually chat with us?

Ideally we want it to get messages from the user and give responses back. And we want to give it a little bit of personality. It would be great if it sounded something like Lenny himself—warm, friendly, and smart. 

   What’s going on here? There are two intertwined problems: 

1.     GPT-3 tends to “hallucinate.” Hallucination is a technical term that refers to the model’s propensity to return nonsensical or false completions depending on what’s asked of it. Basically, the model is like a very smart and overeager 6-year-old. It will try its best to give you a good answer even if it doesn’t know what it’s talking about. OpenAI and other foundational-model companies are actively working on this problem, but it’s still pretty common. It’s compounded by the second problem.

2.      GPT-3 might not have the right data. GPT-3 has a knowledge cutoff—meaning all of the information it uses to produce its responses is frozen in 2021. Also, much of Lenny’s writing is behind a paywall. That means that even though GPT-3 has read the whole internet, it won’t have the material from his newsletter available to construct answers.

    So how could we construct a chatbot with GPT-3 that solves these problems? Ideally we want to feed GPT-3 the information it needs to answer questions on the fly. That way it will have the right information available and will be less likely to make things up. 

There’s a pretty easy way to do that.

   Stuffing context into the prompt

    When I was in high school, I had a physics teacher who allowed open-book tests. He would allow you to bring a single index card to the test with any of the formulas that you thought you needed to answer the questions.

    The idea was that memorizing the formulas didn’t matter so much. What mattered most was using your reasoning abilities to turn the formulas into the correct answer. 

People would come to the test with microscopic handwriting covering every inch of their notecard. It turns out that this was helpful! The formulas gave you the context you needed to think through the answers to the questions on the tests, so the tests became less about your memory and more about how well you understood the topic. (I got a B in that class, so my understanding was pretty average.)

   You can work with GPT-3 in a similar way. If, in your prompt, you include the equivalent of a notecard with context to help it answer the question, it will often get it right. (Its reasoning capabilities are better than mine.) 

     Let’s go back to an example GPT-3 failed on earlier and see if we can correct it with this technique. 

As I mentioned above, in “How to kickstart a consumer business,” Lenny notes that less than a third of the founders got their idea from trying to solve their own problem.