Head of Xbox Game Studios “Dreams” of Having AI as QA Testers

[Tsing]

Traditional QA testers might be out of a job in the coming future thanks to Xbox's Matt Booty in some part, who recently revealed that he had asked AI researchers at Microsoft to come up with an AI that could test games, effectively reducing the need for human QA testers.

Go to post

 

[Grimlakin]

Qa literally needs people that think outside of the box. Not ai with decision trees.

 

[MadMummy76]

That's not how AI works. Teaching an AI (or what is commonly called an AI today) to play a specific game possibly takes more effort and time than having humans test it. Besides any change to the game will make the AI data obsolete necessitating the purge of the learned data. The worst part is that in order to start teaching the AI how to play a game the game needs to be fully playable first. While a human can test subsystems independently even if some features are not even implemented as placeholders.

And as stated above, games often break when players do unintended things in them. If an ai is reinforced to play the game as intended it will never detect bugs that occur when you are not playing the game to complete the task intended by the designers.

 

[Uvilla]

I think qa testers will be substituted by ai . its a repetitive task that benefits from multiple instances in parallel. I don't see why not an ai can't try all manner of scenarios, and would do so relentlessly and can do so simultaneously many times over. Game and driver testing will surely be eliminated as human jobs, no question. AI brute force I think is perfect for this job.

 

[MadMummy76]

I think qa testers will be substituted by ai . its a repetitive task that benefits from multiple instances in parallel. I don't see why not an ai can't try all manner of scenarios, and would do so relentlessly and can do so simultaneously many times over. Game and driver testing will surely be eliminated as human jobs, no question. AI brute force I think is perfect for this job.

Because there is no clear parameters for testing. The AI would need to learn what is intended and what is unintended (aka bug), but in order to do that, it needs loads of samples. Bugs are freak occurences, there is no way to effectively teach an AI to recognise something like that.

Short of the kind of bugs that would crash the game I don't think current machine learning type AI is capable of replacing human QA testers.

 

[Brian_B]

I could see it doing some of the stupid testing that is more or less automated already -- character getting stuck in maps, errant clicks causing unintended results, model collision issues, etc.

But yeah, I can't really see it totally replacing human testing.