Playing D&D With an AI.. as Player, Not the DM

Many people have experimented with having an AI like ChatGPT act as Dungeon Master in a D&D game. Results tend to be encouraging, but mixed. (Update: Here are my keys to success!)

But a while ago I flipped the concept on its head and asked an AI to act as a player, instead of DM. The results convinced me that there was some mileage in this idea, and I'll explain why.

The Big Takeaways

1. ChatGPT's GPT-4 model is better than GPT-3.5 (3.5 being the default at the time of the experiment.)

Both work, but GPT-4 is noticeably better at chains of reasoning, grasping cause and effect, remembering important details, and so forth.

2. The AI is really good at creating a character and walking through the process.

Simply asking the AI something like "create yourself a D&D 5e character and explain each choice step-by-step" is very effective. (You can also ask it to summarize the character afterwards.)

3. As a player, the AI can be expected to be "On Board" and cooperative

This may be a result of me asking the AI to be a player in order to "help me practice being a DM", but I found that as a player the AI will follow your lead and not play in an adversarial or antagonistic way.

4.  The AI model forgets things as time goes on, and makes mistakes a human would not make. 

This includes forgetting ability numbers, but also things like forgetting the fact that it is supposed to be the player, not the DM. This isn't too surprising considering how AI language models work, but it's not very intuitive. There are effective workarounds, such as being careful about how one phrases things. For example, ending with 'What would your character like to say or do?" is always helpful. It's also helpful to try to focus on one thing happening at a time, to make the chain of roles and events clear.

Even so, GPT-4 was still extremely good at remembering high level details of the story and events. It demonstrated this with its response to my asking it to summarize the adventure after it concluded. It did a great job of that, even if it was also sometimes forgetting things like ability numbers or slipping into wanting to act as the DM. 

5. If Things Get Off Track, Edit the Last Thing You Said 

I needed to occasionally use the "redo" function where I edited the last prompt I gave and re-generate the response to get things back on track. For example, if the AI tried to "take over" the narrative, adding a question like "What does your character think about that?" or "What would your character like to say or do at this point?" makes the roles clear.

Sometimes, as the adventure went on, the AI would occasionally respond to my request for an ability check with something like "as an AI language model, I can't make dice rolls..." Re-phrasing things to something like "Please make a Perception check for <character name> and tell me the result" usually fixed that.

In the adventure transcript (which can be found below), I tried to make a note of when this happened, but the actual "re-writes" are not included.

6. Give the AI a Push If Necessary

I found the AI seemed to prefer to continue to gather information and prepare when there was uncertainty, rather than make a choice about a way forward. In these cases, the AI responded well to clear direction or a clear lead.

However, now that I think about it, this preference for investigation and learning over action may have been the result of the AI's player character personality (in my experiment, the AI played a bard with a sage background and a predilection for learning.) I'll have to test this idea.

7. Combat is still a question mark

I handled combat in a mostly narrative and hand-wavy way in my experiment, and this angle isn't well explored.

The Adventure 

Here it is, in ChatGPT's own words:

How would you, in your own words, summarize the adventure and your role in it?

 In this adventure, I took on the role of Elara, a skilled investigator and mage hired to help the town of Greenhollow with a mysterious creature causing trouble. The town had experienced a series of strange incidents that were both harmful and mischievous. Elara conducted investigations, interviewed townsfolk, and studied the local environment to uncover clues about the creature's nature and intentions.

During the investigation, the mayor mentioned that a mage named Sint was also expected to arrive in town to help with the problem. Upon Sint's arrival, Elara proposed a collaborative approach to resolving the issue, which Sint eventually agreed to. Together, they devised a plan to lure the creature, identified as a Will-o'-wisp, and confront it.

With the help of a desperate prisoner, they lured the creature into a confrontation, where Elara and Sint combined their magical abilities to overpower and ultimately defeat it. Although there was a loss of life during the confrontation, the town was freed from the Will-o'-wisp's mischief, and Elara gained both the town's gratitude and a share of the reward.

As Elara, I provided the player's perspective, making decisions and taking actions based on the information and situations presented by the DM, while the DM crafted the story, controlled the NPCs, and guided the overall narrative.

There is one glitch in the above -- Elara was a bard (not a mage) although I hand-waved the magic use during the final confrontation and ignored the AI's poor sense of what spells Elara would or would not have access to. That probably contributed to the error. But besides that, the summary is excellent.

The Full Transcript

For those interested in more details -- including comments on how things worked (or didn't) -- the full transcript is here, with comments.