Why AI Shouldn’t Be Writing Your User Stories
When Open AI introduced ChatGPT in November 2022, there was an initial period of intense curiosity about what it could do.
A mad scramble followed to figure out how to use this new game-changing technology.
It wasn't long before product managers got into this game and could look at the challenge from two different angles:
- Where can I use AI in my product?
- How can I use AI to make my life easier as a product manager?
If you work on a product management platform, you might see those two angles intersect by building the automatic creation of user stories into your tool.
Brilliant!
But just because you can do something, doesn’t mean you should.
The case for using AI to write user stories
Seemingly overnight, a whole new class of AI for product managers has popped up—user story generators.
Some of them are even free!
Then, established product management platforms started adding AI capabilities into their tools, including push-button generation of user stories.
The benefits these tools, and the articles extolling them, promised were some variation on this list:
- Generating user stories with AI is faster and more efficient than manually creating them.
- Generating user stories with AI ensures a consistent format and hence brings clarity.
- Generating user stories with AI ensures more accurate user stories.
- Generating user stories with AI boosts creativity.
- Generating user stories with AI improves collaboration.
I’m going to address each of these assumptions, but first I think it’s important to reflect on the original intent of user stories.
What user stories were intended for
In his book, User Story Mapping, Jeff Patton succinctly described why user stories have that name:
Stories get their name from how they should be used, not what should be written.
-JEFF PATTON, USER STORY MAPPING
Jeff then expands on that thought with a quote from Kent Beck, who developed the user story concept.
"If we get together and talk about the problem we’re solving with software, who’ll use it, and why, then together, we can arrive at a solution and build shared understanding along the way."
User stories identify what someone wants to accomplish with your product and why.
There are three key things to remember about user stories that factor in deciding how, or even if, you should use AI help to create them.
1. User stories are placeholders for a conversation
They should act as a reminder that sparks a more in-depth discussion amongst your product team about what problem you’re helping your users solve.
You’ll probably want to note down what you talked about, but do it as a reminder and reference, not the only means of communicating requirements.
2. User stories are a planning tool
They allow you to slice the work to build your product based on things your users can accomplish with it, rather than listing the tasks you need to perform.
Slicing your work this way helps you get to details when you need to—not way ahead of time.
User stories also help you stay focused on what you will put in your product and what you won’t in a way that a list of tasks just doesn’t help you do.
Which is why brainstorming a bunch of user stories that you could do isn’t necessarily helpful either. More on that in a bit.
3. It’s not the writing that’s important
Because of those top two items, it turns out that how you write user stories really isn’t all that important.
As with many things in life, we can look to Seinfeld to illustrate the point.
In the episode The Alternate Side, Jerry reserves a rental car, only to find out they had his reservation, but they didn’t have his car. Here’s what happened.
Jerry Seinfeld about reserving rental cars:
"You see, you know how to take the reservation, you just don't know how to hold the reservation. And that's really the most important part of the reservation: the holding. Anybody can just take 'em!"
When you paraphrase that for user stories:
You see, you know how to write the user story, you just don’t know how to build shared understanding with the user story. And that’s really the most important part of the user story: the shared understanding. Anybody can just write 'em!
The user stories you write and don’t write are far more important than how you write them.
As long as your user story provides enough information for the product team to remember their discussions about what the user is trying to accomplish—that should be enough.
Why you shouldn’t use AI to write user stories
So with that context in mind about the intended use of user stories, let’s use the arguments for using AI to write user stories to explain why you shouldn’t.
Assumption #1: It's faster and more efficient
Counterpoint: Well, not necessarily.
Most of the articles that introduced using AI to write user stories imply or explicitly state that user stories are a necessary evil.
There’s griping about how long it takes to write all those user stories and get them just right. Wistful thoughts like, "If only we could make that task faster and more efficient!" Some even express a desire to do away with the brainstorming of different user stories.
When I see these gripes, I wonder if folks are missing the point. Noting down the initial user story shouldn’t be onerous. It’s a quick reminder to delve into more detail and note the relevant info from that discussion. It doesn’t need to be perfect to begin with.
Brainstorming a bunch of user stories and sticking them in the product backlog to deal with later is a bad habit that many product teams have picked up over the years.
A better approach is to start with a specific outcome and then identify the specific stories that help you get to that outcome.
Collaborative techniques like impact maps and opportunity solution trees help you identify those user stories.
Assumption #2. It's consistent in format and clarity
Counterpoint: Consistency in formatting isn't crucial early on.
On the surface, consistent and clear user stories are a great thing. The thing you have to ask is, is it important that user stories are in a consistent format when you first create them or after your product team has had a chance to discuss them?
I posit that editing for consistency right before the team is going to do development work is more important than staying consistent when you’re first writing them down.
You’re going to discuss the user story anyway, so write enough to spark the conversation and leave it there.
If you’re worried about not remembering what some small sentence fragment meant, that’s probably a sign that you’re creating the user story too far ahead of when you’re going to act on it.
Assumption #3: It boosts creativity
Counterpoint: It can also take you further from the point.
AI-powered tools promise to break you out of writer’s block through a range of pre-defined user story templates and prompts. That’s great if you want to generate a whole slew of off-the-wall ideas.
But as I mentioned earlier, you don’t want to stock your backlog up with a bunch of user story noise. Instead, you want to focus on those things that will help you make progress toward reaching outcomes.
The techniques I mentioned earlier provide a great way of identifying meaningful user stories focused on achieving outcomes. And remember, constraints are great for stimulating creativity.
Assumption #4: It's more accurate
Counterpoint: This is only true if your AI tool has a very comprehensive understanding of your product and users.
The argument for accuracy claims that generating stories based on large amounts of data and customer feedback leads to more accurate stories. The reasoning is those stories are more “accurate” because they align with customer needs.
This may be the best argument for using AI, but it relies on a couple of big “ifs. “
- You train your AI tool on your specific product data.
- You have a sufficient amount of product data to perform effective analysis.
The first “if” counts out all of those “free” tools build on GPT 3, 4, etc.
The second if isn’t very helpful if you’re building a new tool, or you’re just starting to capture customer feedback meaningfully.
Assumption #5: Improved collaboration
Counterpoint: I'm sorry—what?!
Yeah…I did a double-take when I saw this argument.
You’re telling me that having AI generate user stories—implying they’ll be ready to toss over the wall to developers the moment they're spat out of the tool—improves collaboration?
Then I looked at the explanations again. They mostly proclaim the collaborative benefits of user stories in general, not ones specifically created by AI.
Nice try, but I’m not convinced.
Are you suggesting I don't use AI at all?
I’m not a Luddite.
AI has a place, even in product management tools. Trying to automate the activities that require collaboration with your product team is not that place.
It makes sense to use AI to synthesize all that customer feedback you’re hopefully getting. It could also be useful for generating different test cases to make sure you have full coverage.
Doug Steele tackled this same topic on LinkedIn and suggested that you shouldn’t use AI to take over user story writing, but you can use its help to supplement your efforts.
Just because you can, doesn’t mean you should.
When it comes to writing user stories, or any potential use of AI, don’t look to use AI for the sake of using AI. Figure out where using AI will actually add value.
A great place to start is to change up the questions I asked above.
- Where might AI make sense in your product?
- Where does it make sense to use AI to help with your product management activities?
Asking the questions in that way will help you make sure you’re using AI responsibly—improving your product and making your PM life easier.
Don't forget to subscribe to our newsletter for more product management resources and guides, plus the latest podcasts, interviews, and other insights from industry leaders and experts.