Unpacking AI Hype in Game Dev

Read Time: 8 Minutes

“How does AI relate to games and game dev?”

I want to write about AI in game dev, and I sit there, staring at that question thinking about how many different layers and angles can be taken.

I’m going to talk about what “AI” is, then talk about strengths and weaknesses, and how I see game dev adapting to it. I will NOT be talking much about the ethics of AI. So if that’s what you are looking for, this isn’t the newsletter for you. If you really wanted an “Ethics and AI” newsletter instead of this one, let me know!

TL;DR: “AI” as we think of it today is a tool that has been invented and will be used if individuals and companies perceive it to be worthwhile. I believe there are ways to use it that will help teams succeed while upholding good ethics, and I believe that a LOT of what people talk about are pipe dreams.

Image from https://markmcneilly.substack.com/p/the-best-memes-about-ai-part-deux

Defining “AI” - Not As Intuitive As You’d Think

In case you aren’t up to speed, let’s talk about what we mean when we say, “AI.” 

“AI” today refers almost entirely to large language models (LLMs). LLMs indirectly look at sets of THINGS like sentences and images and use what they gather to generate new things.

It is indirect because if you ask an image generating LLM to give you a picture of an elephant, it doesn’t go and look at a bunch of images of elephants and then draw something that seems similar, there’s a translation layer where words or parts of images are decomposed between the two. Some people call the process of “uploading the dataset in usable form” to an LLM as “training.” This is a bit of a misnomer, but it’s not the only misnomer nor the most egregious surrounding AI.

These large language models are - at their simplest - autocomplete on steroids. They have larger datasets to draw upon, so they are likely to generate things that seem to make sense at a much higher rate. The practical upshot of this is that they seem to be able to hold a conversation, answer questions, and do a ton of other things, like flying a drone, that are pretty impressive for a non-human.

The downside of this is that they “seem” to do some of these things, but aren’t actually “doing” them.

When people talk about “training” AIs, or AIs “thinking” or “creating” or “seeing” things, this is incorrect, because while our classic understanding of “AI” as “artificial intelligence” implies something like a robot or computer that can think like a human, the AIs of today (LLMs) don’t do any of that.

They “respond” to queries by giving an answer that would seem to be a reasonable next set of word bits or images based on what was input. The LLM doesn’t understand what it does, because there is nothing there to be “understanding.” It’s not a mind. It’s software that is good at giving responses that humans find believable.

While many companies are talking about how they are closing in on what they call “AGI” (artificial general intelligence) which would have those capabilities, the way LLMs work is not analogous to how humans (our only accessible example of a conscious general intelligence) think.

Achieving AGI would mean having a computer that could reason, discern, make judgment calls, think, learn, etc. in the same way that humans do. This is NOT what we have today. Further, despite the claims of companies - companies interested in massive valuations and investor dollars so they can continue bleeding cash everywhere (I’m a bit cynical here, I’ll admit it) - there is not a clear leap from “LLM” to “AGI.” In fact, I would say that were we to achieve AGI, it would need to be through a different path than the one we’re on. LLMs don’t have a mind, nor do we really know what a “mind” is. This is a serious issue for developing an AGI.

There has been a sleight of hand in all of this; the goalposts have moved. Our modern “AI” is not what we would have called “AI” 20 years ago. But we’ve redefined “AI” to “AGI” and now qualify LLMs as “AI” despite LLMs not being intelligent, just because those LLMs provide a facsimile of intelligence that we feel is compelling.

AIs (or really LLMs) can be used to generate artwork, stories, code, summaries, translations, and a lot of other stuff. 

AI in Game Dev

You can hopefully see how helpful LLMs could be to some people. If I don’t write well, or can’t draw, I can now ask an LLM of my choice to generate articulate sentences or an image that meets my specifications. It will produce things that match whatever input I provided.

So if I always had a great idea for something, but was short a few skillsets, now LLMs can step in and fill that gap, allowing me to generate the stuff I couldn’t do. 

For game dev, this has a ton of implications. Wanted to make a compelling fantasy world but don’t know how to art? That’s ok! Let LLMs generate the art for you. Don’t have time to write barks for NPCs? An LLM can handle it. Not sure how to set up some feature in unreal for your game? LLMs might just be able to code it for you.

Where do AIs fall short? Anything involving judgment. If you ask AIs to generate 15 images of orcs fighting elves, it can do it easily. If you ask it which image is the best based on your context, it will give you an answer, but it doesn’t know, because it can’t know, because it doesn’t think. 

I’ve tried to be careful above to use the word “generate” rather than “create” when talking about LLMs as well. Because it is the more accurate term. Humans can create novel things. We don’t know exactly how that works - we have very little understanding of human consciousness generally - but sometimes we are inspired to do something that has never been done. Some of this likely is the combination of pre-existing concepts, but some of it also seems to be the genuine invention of something new. 

LLMs can “kitbash” things together, they can amalgamate and combine and even seem to find “insights” within overlapping data sets, but all it can ever do is work with what was already there.

Where can these LLMs help with game dev?

I’ve talked with a ton of people about this, and the consensus seems to be that AI can be very helpful in prototyping things or creating reference material. An image-generating LLM can give me a thousand images in a short space of time. I can then select a few of them and say, “More like this, with <X different thing>.” Through repetition of this, a non-artist can zero in on concepts they like for what they are doing.

Using it to rapidly create game prototypes is also a valid use case. Don’t know how to get the UI right? Tell the LLM what you want. It might not get it just right the first time, but if you don’t know how to do it at all, spending a few minutes or even hours banging away at prompts to get what you want is still easier than learning to code on your own.

Then you also have the ability to let custom-built AI rig models and even do basic animations. Again, from the perspective of rapid prototyping this could be a huge win. 

I use these examples of concept art, animation, and engineering as just that: examples. 

The second broad place AI might bring a ton of value is in “polishing” something. Give AI a random image to “uprez” and it can do it. Similar to the above, you probably want it done a bunch of times to get some different options, then you additionally want an expert to go in and further touch up anything it creates, but it is helpful in getting you to “done” faster.

On the technical side I am far less convinced of the use of LLMs to “clean up” code. While some people claim that “AI agents” will be full stack operators doing QA to boot, the inability to discern is a huge achilles heel. I’m doubtful this will work in all but the simplest of contexts. The tech is still moving, but the way LLMs are built makes it difficult to see them being both effective AND trustworthy here. The complexity of game related tech stacks is no joke. The last thing I want is to let my AI “do some refinement” of the codebase, only to lose awareness of what my code even is and open myself up to bugs that were just not something an LLM has the capacity to deal with.

This also goes for QA. While some of the work is probably stuff AI can warn you about, a lot of game dev QA involves understanding things like what is supposed to happen vs what is not. There ARE probably ways that LLMs could help QA, like helping with ticketing systems and the like, but there is the problem of how much you can “trust” anything LLMs produce. Will this provide that much value past smoke testing? I don’t think so.

There’s some “around game” stuff too where current AI can be helpful. Think about localization. In the past, loc requires a lot of work and some lead time. But now, especially as AI is custom built for the purpose, you can get pretty dang good translation between languages in moments. Even with some slight imperfections, this could be “good enough” for many studio needs. It wasn’t as if the human-based system was perfect!

Image from https://legendsoflocalization.com/bad-translations/classic/

Where is AI not so helpful to game dev?

Any time you need something to be right, AI via LLMs are a bad option. The term “hallucinate” has been used to describe when these LLMs get something wrong, but it’s a bit of a misnomer. Outside of someone with serious mental issues or hitting psychedelics regularly, hallucinations are RARE in humans. But the mechanisms that lead to these AI “hallucinations” are the exact same that lead to correct answers. The “hallucination” is not an aberration, it’s just an output that happened to be wrong.

Example taken from: https://www.datacamp.com/blog/ai-hallucination

Take a look at the image above. If there is anything you would EXPECT “AI” to get right, it would be math questions. But here it doesn’t. It is CONFIDENTLY wrong, even providing FALSE EVIDENCE. Would you have checked?

This is troubling not because it comes up a ton in game dev, but because many people don’t realize that every LLM out there today is prone to error. The lack of education around how prone to being wrong LLMs are is a big deal. Treating LLMs as truthful is as if we took the first internet search we found and assumed it was correct no matter what it said. Some of this will get better with time as people realize AI is prone to error, but for right now the fact that LLMs will attempt to answer almost any question you ask authoritatively is a problem because people might just believe it.

I mentioned earlier that LLMs do not have the capacity to discern or judge, even if they appear to. The combination of an inability to discern combined with no awareness of correctness means that AI today is best used to assist an expert who understands and can fix any obvious mistakes.

If you don’t have experts who can make sure the LLMs aren’t making weird edge case errors that will blow up in your face later, you are going to be in trouble once you get past prototyping.

What about narrative disciplines? I mentioned “barks” above, simple short lines that NPCs might say. Here, similar to concepting, I would imagine you could have LLMs generate a large number and then have someone select from the list. I don’t know how much time this saves you, depends on the problem space, the quality of the prompt, and the narrative dev involved. 

For any important writing, I would not recommend any AI that exists unless you have no other option. The way I’ve described LLMs based on my own experience: it gets you to a passing high school (pre-university) grade. That’s not a high bar. Again, perhaps useful in prototyping, but don’t expect much.

When I look at the technical side of things, the biggest risks I’ve somewhat covered above. LLMs will create code. The code may work. It will definitely require someone who understands what codebases should look like for it to be effective for the long haul.

Perhaps you’ve seen a game team thinking they can use Unreal out of the box. In some ways, they can. But there’s a reason having engineers who have built things in Unreal is so prized by teams trying to build things in Unreal. Unreal out of the box does a generic job at a specific thing. Odds are, any particular team is trying to do something different. There are many teams that have gotten deep in development before realizing they need help getting Unreal to do what they want, only to bring in Unreal experts who have to redo a lot of the foundations because it just wasn’t set up to succeed. 

I believe AI will cause similar arcs if you attempt to use it instead of engineers, rather than as something to assist engineers. This applies to all disciplines. Don’t let AI do stuff without expert oversight. Make sure you’ve got an expert alongside it, whether that’s an artist, a writer, a QA, or whatever else. Otherwise you are setting yourself up for unpleasant and painful discoveries down the road. If you don’t have the right type of expert, recognize that you skate on thin ice.

Image from https://markmcneilly.substack.com/p/the-best-memes-about-ai-part-deux

Using AI To Create Gameplay/Dialogue In Games

“What if every NPC in Skyrim could have an extended conversation with you? Wouldn’t that be cool?”

I don’t know. There’s a novelty to that that’s interesting, and if that also led to quest generation and entirely unique experiences, that MIGHT be good? But part of me looks at that as one of those things that sounds cool but has far less attraction or engagement value. If I’m developing Skyrim and building quests and characters and things for the player to do, am I excited by players having a conversation with the first unimportant NPC they encounter and going off in a totally different direction? If I’m building that game, sure. But in many cases, I don’t think it’s better.

In this, I wonder if AI is akin to VR. VR sounds like it would be super cool, but still hasn’t broken into the mainstream. Will it get there? I don’t know. I just know all the attempts to date at making better headsets and better products aren’t leading to the breakthrough people have wanted. Could be a matter of time or innovation, or it could be something that sounds cool and isn’t that engaging in practice.

Image from https://markmcneilly.substack.com/p/the-best-memes-about-ai

I’m skeptical of the “AI integrated into gameplay” use case. But, there’s a ton of startups chasing stuff like this right now. I guess we’ll find out over time!

Where Does AI Take Us?

Alright, where does all of this lead us?

Image from https://www.boredpanda.com/chatgpt-ai-fails/

My thoughts to date:

1. I do not believe that AI in current LLM form or future forms will replace dev teams by leaving the future of game dev in the hands of solo devs surrounded by AI agents doing all the things they cannot.

2. I do not believe that AI will provide “10x” improvements in speed and productivity in game dev - there may be narrow parts of certain crafts where you see dramatic improvements, but overall the gains will be far less.

3. I do not believe that AI will get ENOUGH better than it is today to a point where it replaces most game dev disciplines or work more generally.

4. I do believe that AI is something worth taking seriously as a tool that can make experts more effective/productive.

5. I do not believe AI is getting close to “AGI” and gaining the ability to discern and judge effectively, or otherwise “create” rather than “generate.” Count this doubly for some sort of Terminator Skynet style super intelligence.

6. I do believe some regions of the world will use (and are using) AI without any ethical qualms. If this provides a meaningful competitive edge, every other region will need to take that seriously as they look at the ethical challenges of AI in a global marketplace.

7. Most importantly: I do believe I’m wrong about some or perhaps many of the things I’ve stated I do or do not believe above. This space is fast moving.

What I’m Being Told And Seeing In Game Dev

A proofreader of this newsletter said this:
I feel like this low-key your thesis. "AI exists, people are gonna use it. You need to care about whether it's good, about whether it makes your game better."

I love this summary. This last little bit will talk about how people have benefited from using AI in game dev to date.

From talking to devs and studios that have used AI in its current forms, I’m told that it can be useful - sometimes insanely so - where it works. And also that there are many places where it doesn’t work.

I anticipate LLMs being used heavily in the space of prototyping and in UGC platforms to make someone with little knowledge capable of getting farther faster than they ever could have before.

The studios that have engaged with it heavily are getting gains out of it. I’ve not heard anyone ON THE GROUND in game dev tell me they are 10x as effective, or only need 5 people to make AAA games like Destiny, Witcher, GTA, Total War, and other “big” products. But I have heard reports of double digit percentage improvements (20-50%). I’ve also heard that attempts to make games ENTIRELY with AI have not worked out - not that something doesn’t come out the other end, but nothing good does.

There’s more I could talk about here. Phases of development, indie vs AAA, mobile vs PC/Console, but I’m going to wrap here. If you want me to go into my thoughts on all of that in a future newsletter, let me know!

For now, hope this has provided additional perspective, and that some of you learned something!

How are YOU using AI? How much better do you think it will get?

Thanks to Susan, Alex, Gemma, Rob, Aaron, Daniel, Ryan, Chris and many others who had conversations with me over the last few months about AI. Cheers!

Final note: Reframing Agility still has available slots! If you want to get an industry-standard certification while taking a course that will actually help your team make games better, sign up here: https://www.buildingbettergames.gg/reframingagility

Reframing Agility will be April 14-15, from 8a-4p PT. Looking forward to it!

Whenever you’re ready, there are 3 ways we can help you…

—>Courses built by game devs for game devs - check out “Succeeding in Game Production” HERE.

—>Regular deep dives on critical game development topics on the BBG podcast

—>We’ve helped many high-profile game studios save a ton of money & time through building clear vision and leveling up leadership. If you’d like to work with us, please reach out at [email protected].