FaSE4Games '24: "Towards the Automatic Replication ..."
Towards the Automatic Replication of Gameplays to Support Game Debugging
Stefano Campanella, Emanuela Guglielmi, Rocco Oliveto, Gabriele Bavota, and Simone Scalabrino
(USI Lugano, Switzerland; University of Molise, Italy)
The video game industry has experienced a continuous growth in the last decades. In such a competitive market, it is fundamental to ensure a great gaming experience to the player avoiding, for example, bugs. However, video game testing is an extremely challenging activity, especially considering the extensive number of gaming scenarios that modern video games support (e.g., 3D worlds to explore). Thus, more often than not, numerous bugs are discovered only once the game is released and played by millions of users. For this reason, recent work in the literature suggested to exploit gameplay videos to support developers in identifying possible bugs missed during testing: given the large amount of gameplays posted every day on streaming platforms (> 2M hours), these gameplays are likely to document failures experienced by the player. Empirical evidence show the ability of these techniques to identify parts of the gameplay in which the failure was experienced. However, it could still be difficult for game developers to reproduce the bug. In this paper, we propose the idea of developing a technique able to automate this process, providing the game developer with all actions performed by the player to reach the faulty state shown in the gameplay. We present a simple approach which leverages the on-screen controls overlay available in some gameplay videos. We show that such an approach can replicate 47.2% of gameplays in our preliminary study run on a racing game. We discuss the strong limitations of this first attempt, listing directions for future work we plan to pursue in order to overcome them.
@InProceedings{FaSE4Games24p1,
author = {Stefano Campanella and Emanuela Guglielmi and Rocco Oliveto and Gabriele Bavota and Simone Scalabrino},
title = {Towards the Automatic Replication of Gameplays to Support Game Debugging},
booktitle = {Proc.\ FaSE4Games},
publisher = {ACM},
pages = {1-6},
doi = {10.1145/3663532.3664465},
year = {2024},
}
Publisher's Version
Article: fsews24fasegamesmain-id1-p doi:10.1145/3663532.3664465
FaSE4Games '24: "Towards the Automatic Replication ..."
Towards the Automatic Replication of Gameplays to Support Game Debugging
Stefano Campanella, Emanuela Guglielmi, Rocco Oliveto, Gabriele Bavota, and Simone Scalabrino
(USI Lugano, Switzerland; University of Molise, Italy)
The video game industry has experienced a continuous growth in the last decades. In such a competitive market, it is fundamental to ensure a great gaming experience to the player avoiding, for example, bugs. However, video game testing is an extremely challenging activity, especially considering the extensive number of gaming scenarios that modern video games support (e.g., 3D worlds to explore). Thus, more often than not, numerous bugs are discovered only once the game is released and played by millions of users. For this reason, recent work in the literature suggested to exploit gameplay videos to support developers in identifying possible bugs missed during testing: given the large amount of gameplays posted every day on streaming platforms (> 2M hours), these gameplays are likely to document failures experienced by the player. Empirical evidence show the ability of these techniques to identify parts of the gameplay in which the failure was experienced. However, it could still be difficult for game developers to reproduce the bug. In this paper, we propose the idea of developing a technique able to automate this process, providing the game developer with all actions performed by the player to reach the faulty state shown in the gameplay. We present a simple approach which leverages the on-screen controls overlay available in some gameplay videos. We show that such an approach can replicate 47.2% of gameplays in our preliminary study run on a racing game. We discuss the strong limitations of this first attempt, listing directions for future work we plan to pursue in order to overcome them.
@InProceedings{FaSE4Games24p1,
author = {Stefano Campanella and Emanuela Guglielmi and Rocco Oliveto and Gabriele Bavota and Simone Scalabrino},
title = {Towards the Automatic Replication of Gameplays to Support Game Debugging},
booktitle = {Proc.\ FaSE4Games},
publisher = {ACM},
pages = {1-6},
doi = {10.1145/3663532.3664465},
year = {2024},
}
Publisher's Version
Article: fsews24fasegamesmain-id1-p doi:10.1145/3663532.3664465
FaSE4Games '24: "Combining Neuroevolution with ..."
Combining Neuroevolution with the Search for Novelty to Improve the Generation of Test Inputs for Games
Patric Feldmeier and Gordon Fraser
(University of Passau, Germany)
As games challenge traditional automated white-box test generators, the Neatest approach generates test suites consisting of neural networks that exercise the source code by playing the games. Neatest generates these neural networks using an evolutionary algorithm that is guided by an objective function targeting individual source code statements. This approach works well if the objective function provides sufficient guidance, but deceiving or complex fitness landscapes may inhibit the search. In this paper, we investigate whether the issue of challenging fitness landscapes can be addressed by promoting novel behaviours during the search. Our case study on two Scratch games demonstrates that rewarding novel behaviours is a promising approach for overcoming challenging fitness landscapes, thus enabling future research on how to adapt the search algorithms to best use this information.
@InProceedings{FaSE4Games24p14,
author = {Patric Feldmeier and Gordon Fraser},
title = {Combining Neuroevolution with the Search for Novelty to Improve the Generation of Test Inputs for Games},
booktitle = {Proc.\ FaSE4Games},
publisher = {ACM},
pages = {14-19},
doi = {10.1145/3663532.3664467},
year = {2024},
}
Publisher's Version
Article: fsews24fasegamesmain-id3-p doi:10.1145/3663532.3664467
FaSE4Games '24: "Combining Neuroevolution with ..."
Combining Neuroevolution with the Search for Novelty to Improve the Generation of Test Inputs for Games
Patric Feldmeier and Gordon Fraser
(University of Passau, Germany)
As games challenge traditional automated white-box test generators, the Neatest approach generates test suites consisting of neural networks that exercise the source code by playing the games. Neatest generates these neural networks using an evolutionary algorithm that is guided by an objective function targeting individual source code statements. This approach works well if the objective function provides sufficient guidance, but deceiving or complex fitness landscapes may inhibit the search. In this paper, we investigate whether the issue of challenging fitness landscapes can be addressed by promoting novel behaviours during the search. Our case study on two Scratch games demonstrates that rewarding novel behaviours is a promising approach for overcoming challenging fitness landscapes, thus enabling future research on how to adapt the search algorithms to best use this information.
@InProceedings{FaSE4Games24p14,
author = {Patric Feldmeier and Gordon Fraser},
title = {Combining Neuroevolution with the Search for Novelty to Improve the Generation of Test Inputs for Games},
booktitle = {Proc.\ FaSE4Games},
publisher = {ACM},
pages = {14-19},
doi = {10.1145/3663532.3664467},
year = {2024},
}
Publisher's Version
Article: fsews24fasegamesmain-id3-p doi:10.1145/3663532.3664467
FaSE4Games '24: "Towards the Automatic Replication ..."
Towards the Automatic Replication of Gameplays to Support Game Debugging
Stefano Campanella, Emanuela Guglielmi, Rocco Oliveto, Gabriele Bavota, and Simone Scalabrino
(USI Lugano, Switzerland; University of Molise, Italy)
The video game industry has experienced a continuous growth in the last decades. In such a competitive market, it is fundamental to ensure a great gaming experience to the player avoiding, for example, bugs. However, video game testing is an extremely challenging activity, especially considering the extensive number of gaming scenarios that modern video games support (e.g., 3D worlds to explore). Thus, more often than not, numerous bugs are discovered only once the game is released and played by millions of users. For this reason, recent work in the literature suggested to exploit gameplay videos to support developers in identifying possible bugs missed during testing: given the large amount of gameplays posted every day on streaming platforms (> 2M hours), these gameplays are likely to document failures experienced by the player. Empirical evidence show the ability of these techniques to identify parts of the gameplay in which the failure was experienced. However, it could still be difficult for game developers to reproduce the bug. In this paper, we propose the idea of developing a technique able to automate this process, providing the game developer with all actions performed by the player to reach the faulty state shown in the gameplay. We present a simple approach which leverages the on-screen controls overlay available in some gameplay videos. We show that such an approach can replicate 47.2% of gameplays in our preliminary study run on a racing game. We discuss the strong limitations of this first attempt, listing directions for future work we plan to pursue in order to overcome them.
@InProceedings{FaSE4Games24p1,
author = {Stefano Campanella and Emanuela Guglielmi and Rocco Oliveto and Gabriele Bavota and Simone Scalabrino},
title = {Towards the Automatic Replication of Gameplays to Support Game Debugging},
booktitle = {Proc.\ FaSE4Games},
publisher = {ACM},
pages = {1-6},
doi = {10.1145/3663532.3664465},
year = {2024},
}
Publisher's Version
Article: fsews24fasegamesmain-id1-p doi:10.1145/3663532.3664465
FaSE4Games '24: "Unit Test Generation using ..."
Unit Test Generation using Large Language Models for Unity Game Development
Ciprian Paduraru, Alin Stefanescu, and Augustin Jianu
(University of Bucharest, Romania; certSIGN, Romania)
Challenges related to game quality, whether occurring during initial release or after updates, can result in player dissatisfaction, media scrutiny, and potential financial setbacks. These issues may stem from factors like software bugs, performance bottlenecks, or security vulnerabilities. Despite these challenges, game developers often rely on manual playtesting, highlighting the need for more robust and automated processes in game development. This research explores the application of Large Language Models (LLMs) for automating unit test creation in game development, with a specific focus on strongly typed programming languages like C++ and C#, widely used in the industry. The study centers around fine-tuning Code Llama, an advanced code generation model, to address common scenarios encountered in game development, including game engines and specific APIs or backends. Although the prototyping and evaluations primarily occurred within the Unity game engine, the proposed methods can be adapted to other internal or publicly available solutions. The evaluation outcomes demonstrate the effectiveness of these methods in enhancing existing unit test suites or automatically generating new tests based on natural language descriptions of class contexts and targeted methods.
@InProceedings{FaSE4Games24p7,
author = {Ciprian Paduraru and Alin Stefanescu and Augustin Jianu},
title = {Unit Test Generation using Large Language Models for Unity Game Development},
booktitle = {Proc.\ FaSE4Games},
publisher = {ACM},
pages = {7-13},
doi = {10.1145/3663532.3664466},
year = {2024},
}
Publisher's Version
Article: fsews24fasegamesmain-id2-p doi:10.1145/3663532.3664466
FaSE4Games '24: "A Data-Driven Analysis of ..."
A Data-Driven Analysis of Player Personalities for Different Game Genres
Xiaozhou Li, Valentina Lenarduzzi, and Davide Taibi
(University of Oulu, Finland)
Research on game genres and player types has been one of the pillars of game studies, providing a theoretical foundation for effective game design and development practices. Therein, various player types are proposed when the players are classified based on their preferences or play styles. However, players' social and behavioral traits, e.g., their personality, that can, to a certain extent, influence their ways of play and preferences of genres are limitedly studied. In this short paper, we investigate the different personalities of the players who play different game genres using the review data from the Steam online platform. For such a purpose, we collected 1.9 million player reviews for the 40 top-ranking games of the four most popular game genres. Using a pre-trained neural network classifier, we summarize the collective personality of the players for each selected game genre based on the Big Five personality model. The early results show that the collective player personality of different game genres differs and is worth considering in game design. This study aims to explore and initiate discussions on the latent connection between player personality and their preferences and behaviors and to draw attention to the effective adoption of data mining techniques for the domain of game studies. Furthermore, such studies shall contribute to considering enhanced user experience in game design and development.
@InProceedings{FaSE4Games24p20,
author = {Xiaozhou Li and Valentina Lenarduzzi and Davide Taibi},
title = {A Data-Driven Analysis of Player Personalities for Different Game Genres},
booktitle = {Proc.\ FaSE4Games},
publisher = {ACM},
pages = {20-25},
doi = {10.1145/3663532.3664468},
year = {2024},
}
Publisher's Version
Article: fsews24fasegamesmain-id4-p doi:10.1145/3663532.3664468
FaSE4Games '24: "A Data-Driven Analysis of ..."
A Data-Driven Analysis of Player Personalities for Different Game Genres
Xiaozhou Li, Valentina Lenarduzzi, and Davide Taibi
(University of Oulu, Finland)
Research on game genres and player types has been one of the pillars of game studies, providing a theoretical foundation for effective game design and development practices. Therein, various player types are proposed when the players are classified based on their preferences or play styles. However, players' social and behavioral traits, e.g., their personality, that can, to a certain extent, influence their ways of play and preferences of genres are limitedly studied. In this short paper, we investigate the different personalities of the players who play different game genres using the review data from the Steam online platform. For such a purpose, we collected 1.9 million player reviews for the 40 top-ranking games of the four most popular game genres. Using a pre-trained neural network classifier, we summarize the collective personality of the players for each selected game genre based on the Big Five personality model. The early results show that the collective player personality of different game genres differs and is worth considering in game design. This study aims to explore and initiate discussions on the latent connection between player personality and their preferences and behaviors and to draw attention to the effective adoption of data mining techniques for the domain of game studies. Furthermore, such studies shall contribute to considering enhanced user experience in game design and development.
@InProceedings{FaSE4Games24p20,
author = {Xiaozhou Li and Valentina Lenarduzzi and Davide Taibi},
title = {A Data-Driven Analysis of Player Personalities for Different Game Genres},
booktitle = {Proc.\ FaSE4Games},
publisher = {ACM},
pages = {20-25},
doi = {10.1145/3663532.3664468},
year = {2024},
}
Publisher's Version
Article: fsews24fasegamesmain-id4-p doi:10.1145/3663532.3664468
FaSE4Games '24: "Towards the Automatic Replication ..."
Towards the Automatic Replication of Gameplays to Support Game Debugging
Stefano Campanella, Emanuela Guglielmi, Rocco Oliveto, Gabriele Bavota, and Simone Scalabrino
(USI Lugano, Switzerland; University of Molise, Italy)
The video game industry has experienced a continuous growth in the last decades. In such a competitive market, it is fundamental to ensure a great gaming experience to the player avoiding, for example, bugs. However, video game testing is an extremely challenging activity, especially considering the extensive number of gaming scenarios that modern video games support (e.g., 3D worlds to explore). Thus, more often than not, numerous bugs are discovered only once the game is released and played by millions of users. For this reason, recent work in the literature suggested to exploit gameplay videos to support developers in identifying possible bugs missed during testing: given the large amount of gameplays posted every day on streaming platforms (> 2M hours), these gameplays are likely to document failures experienced by the player. Empirical evidence show the ability of these techniques to identify parts of the gameplay in which the failure was experienced. However, it could still be difficult for game developers to reproduce the bug. In this paper, we propose the idea of developing a technique able to automate this process, providing the game developer with all actions performed by the player to reach the faulty state shown in the gameplay. We present a simple approach which leverages the on-screen controls overlay available in some gameplay videos. We show that such an approach can replicate 47.2% of gameplays in our preliminary study run on a racing game. We discuss the strong limitations of this first attempt, listing directions for future work we plan to pursue in order to overcome them.
@InProceedings{FaSE4Games24p1,
author = {Stefano Campanella and Emanuela Guglielmi and Rocco Oliveto and Gabriele Bavota and Simone Scalabrino},
title = {Towards the Automatic Replication of Gameplays to Support Game Debugging},
booktitle = {Proc.\ FaSE4Games},
publisher = {ACM},
pages = {1-6},
doi = {10.1145/3663532.3664465},
year = {2024},
}
Publisher's Version
Article: fsews24fasegamesmain-id1-p doi:10.1145/3663532.3664465
FaSE4Games '24: "Unit Test Generation using ..."
Unit Test Generation using Large Language Models for Unity Game Development
Ciprian Paduraru, Alin Stefanescu, and Augustin Jianu
(University of Bucharest, Romania; certSIGN, Romania)
Challenges related to game quality, whether occurring during initial release or after updates, can result in player dissatisfaction, media scrutiny, and potential financial setbacks. These issues may stem from factors like software bugs, performance bottlenecks, or security vulnerabilities. Despite these challenges, game developers often rely on manual playtesting, highlighting the need for more robust and automated processes in game development. This research explores the application of Large Language Models (LLMs) for automating unit test creation in game development, with a specific focus on strongly typed programming languages like C++ and C#, widely used in the industry. The study centers around fine-tuning Code Llama, an advanced code generation model, to address common scenarios encountered in game development, including game engines and specific APIs or backends. Although the prototyping and evaluations primarily occurred within the Unity game engine, the proposed methods can be adapted to other internal or publicly available solutions. The evaluation outcomes demonstrate the effectiveness of these methods in enhancing existing unit test suites or automatically generating new tests based on natural language descriptions of class contexts and targeted methods.
@InProceedings{FaSE4Games24p7,
author = {Ciprian Paduraru and Alin Stefanescu and Augustin Jianu},
title = {Unit Test Generation using Large Language Models for Unity Game Development},
booktitle = {Proc.\ FaSE4Games},
publisher = {ACM},
pages = {7-13},
doi = {10.1145/3663532.3664466},
year = {2024},
}
Publisher's Version
Article: fsews24fasegamesmain-id2-p doi:10.1145/3663532.3664466
FaSE4Games '24: "Towards the Automatic Replication ..."
Towards the Automatic Replication of Gameplays to Support Game Debugging
Stefano Campanella, Emanuela Guglielmi, Rocco Oliveto, Gabriele Bavota, and Simone Scalabrino
(USI Lugano, Switzerland; University of Molise, Italy)
The video game industry has experienced a continuous growth in the last decades. In such a competitive market, it is fundamental to ensure a great gaming experience to the player avoiding, for example, bugs. However, video game testing is an extremely challenging activity, especially considering the extensive number of gaming scenarios that modern video games support (e.g., 3D worlds to explore). Thus, more often than not, numerous bugs are discovered only once the game is released and played by millions of users. For this reason, recent work in the literature suggested to exploit gameplay videos to support developers in identifying possible bugs missed during testing: given the large amount of gameplays posted every day on streaming platforms (> 2M hours), these gameplays are likely to document failures experienced by the player. Empirical evidence show the ability of these techniques to identify parts of the gameplay in which the failure was experienced. However, it could still be difficult for game developers to reproduce the bug. In this paper, we propose the idea of developing a technique able to automate this process, providing the game developer with all actions performed by the player to reach the faulty state shown in the gameplay. We present a simple approach which leverages the on-screen controls overlay available in some gameplay videos. We show that such an approach can replicate 47.2% of gameplays in our preliminary study run on a racing game. We discuss the strong limitations of this first attempt, listing directions for future work we plan to pursue in order to overcome them.
@InProceedings{FaSE4Games24p1,
author = {Stefano Campanella and Emanuela Guglielmi and Rocco Oliveto and Gabriele Bavota and Simone Scalabrino},
title = {Towards the Automatic Replication of Gameplays to Support Game Debugging},
booktitle = {Proc.\ FaSE4Games},
publisher = {ACM},
pages = {1-6},
doi = {10.1145/3663532.3664465},
year = {2024},
}
Publisher's Version
Article: fsews24fasegamesmain-id1-p doi:10.1145/3663532.3664465
FaSE4Games '24: "Unit Test Generation using ..."
Unit Test Generation using Large Language Models for Unity Game Development
Ciprian Paduraru, Alin Stefanescu, and Augustin Jianu
(University of Bucharest, Romania; certSIGN, Romania)
Challenges related to game quality, whether occurring during initial release or after updates, can result in player dissatisfaction, media scrutiny, and potential financial setbacks. These issues may stem from factors like software bugs, performance bottlenecks, or security vulnerabilities. Despite these challenges, game developers often rely on manual playtesting, highlighting the need for more robust and automated processes in game development. This research explores the application of Large Language Models (LLMs) for automating unit test creation in game development, with a specific focus on strongly typed programming languages like C++ and C#, widely used in the industry. The study centers around fine-tuning Code Llama, an advanced code generation model, to address common scenarios encountered in game development, including game engines and specific APIs or backends. Although the prototyping and evaluations primarily occurred within the Unity game engine, the proposed methods can be adapted to other internal or publicly available solutions. The evaluation outcomes demonstrate the effectiveness of these methods in enhancing existing unit test suites or automatically generating new tests based on natural language descriptions of class contexts and targeted methods.
@InProceedings{FaSE4Games24p7,
author = {Ciprian Paduraru and Alin Stefanescu and Augustin Jianu},
title = {Unit Test Generation using Large Language Models for Unity Game Development},
booktitle = {Proc.\ FaSE4Games},
publisher = {ACM},
pages = {7-13},
doi = {10.1145/3663532.3664466},
year = {2024},
}
Publisher's Version
Article: fsews24fasegamesmain-id2-p doi:10.1145/3663532.3664466
FaSE4Games '24: "A Data-Driven Analysis of ..."
A Data-Driven Analysis of Player Personalities for Different Game Genres
Xiaozhou Li, Valentina Lenarduzzi, and Davide Taibi
(University of Oulu, Finland)
Research on game genres and player types has been one of the pillars of game studies, providing a theoretical foundation for effective game design and development practices. Therein, various player types are proposed when the players are classified based on their preferences or play styles. However, players' social and behavioral traits, e.g., their personality, that can, to a certain extent, influence their ways of play and preferences of genres are limitedly studied. In this short paper, we investigate the different personalities of the players who play different game genres using the review data from the Steam online platform. For such a purpose, we collected 1.9 million player reviews for the 40 top-ranking games of the four most popular game genres. Using a pre-trained neural network classifier, we summarize the collective personality of the players for each selected game genre based on the Big Five personality model. The early results show that the collective player personality of different game genres differs and is worth considering in game design. This study aims to explore and initiate discussions on the latent connection between player personality and their preferences and behaviors and to draw attention to the effective adoption of data mining techniques for the domain of game studies. Furthermore, such studies shall contribute to considering enhanced user experience in game design and development.
@InProceedings{FaSE4Games24p20,
author = {Xiaozhou Li and Valentina Lenarduzzi and Davide Taibi},
title = {A Data-Driven Analysis of Player Personalities for Different Game Genres},
booktitle = {Proc.\ FaSE4Games},
publisher = {ACM},
pages = {20-25},
doi = {10.1145/3663532.3664468},
year = {2024},
}
Publisher's Version
Article: fsews24fasegamesmain-id4-p doi:10.1145/3663532.3664468