Introduction
Background
Psychiatry has historically relied on subjective observation for diagnosis, evaluation, and research. With technological advancement, the field has incorporated more biological measurements, including brain imaging and electroencephalography, as a part of a broader effort by researchers and psychiatrists to move towards data driven and evidence-based psychiatry1. Recent advancements have opened the possibility of using digital devices to collect measurements of biological responses that may in turn serve as digital biomarkers2,3,4,5.
In addition, the ongoing Covid-19 pandemic has made urgent the longstanding need for new methods of remotely monitoring cognitive performance and mental states due to extended periods of social isolation and uncertainty6,7,8,9,10,11. Remote monitoring via electronic devices can close the gap between the difficulty and costs of in-person evaluation and the restrictive realities of responding to the ongoing pandemic6,12.
Video games as cognitive measurement tools
Researchers have also attempted to employ custom video games for measurement as well as training. Similar to how cognitive tests measure cognitive abilities insofar as performance on tests is dependent on successful use of the target abilities26, games can measure cognitive abilities since performance in a video game depends on successful use of the cognitive abilities most relevant to the game’s design27,28,29,30,31,32,33,34. Prior research has suggested that a constellation of video games can be employed to obtain measures that approximate general problem-solving skills (e.g. fluid intelligence g)32,33, while other applications have demonstrated that gameplay can be designed around a single cognitive function to extract biomarker-like information about a player’s cognitive abilities35,36, in line with age-based35,37 and gender-based38 expectations of differences in cognition. In doing so, researchers need to account for prior gaming expertise since seasoned gamers can outperform novice gamers even if the game in question is novel for all participants38,39,40.
Design constraints and ecological validity
When devising built-for-purpose serious games, in both measurement and training, researchers rely on designing so that in-game success hinges on successful use of a focal cognitive function. In order to broaden the scope beyond single cognitive functions, some studies have employed a suite of mini-games and tasks, where individual games and tasks in the set are each designed to target a certain cognitive ability (e.g. memorization, fast reaction)18,41. However, such design constraints for interpretability come at a creative cost; simple and isolated game mechanics make for interpretable but often shallow gaming experiences, in which it’s difficult to maintain engagement for hours on end. Not only is such a creative constraint unappealing for the players, but it also makes the serious game endeavor much less appealing for game creators, hampering the development of the field.Additionally, restricting game design around a single cognitive domain limits the ability of the game to probe cognitive domains as relevant in ecologically valid situations. Traditional cognitive tests can overestimate42,43,44,45,46 or underestimate47,48,49 cognitive decline due to them not being valid indicators of cognitive abilities as relevant to day-to-day functioning and being subject to significant practice effects. Common features of tasks made to be more ecologically valid include the use of concrete over abstract entities43 and allowing for a relatively complex task structure that allows for the use of compensative strategies to offset reductions in processing speed42,46. By constructing complex tasks in video games that involve concrete entities and multiple cognitive domains, we can profile cognitive functions in a more ecologically valid setting.
A potential boon for the serious-game field, then, is to demonstrate that interpretability and complex gameplay need not be mutually exclusive. Showing that a purpose-built serious game can be deconstructed into distinct cognitive activities while retaining complex and non-test-like gameplay will provide one roadmap for researchers and game designers alike who are burdened by sacrificing interpretability or engagement.
Cognitive activity in a 3-D action game
Broadly, we considered two possible categories of cognitive functions that could reasonably be expected measured from an individual’s gameplay in a 3-D action game: (1) cognitive functions required for action video games in general, (2) cognitive functions specific to performing tasks in this specific game.In addition to the cognitive functions required for playing any generic action video game, there may also be cognitive demands placed by features not universal to all video games. Among action games, there is a subgenre called stealth games where the game’s incentives are structured to reward avoiding direct confrontation with enemies, and to penalize failure to circumvent enemy detection, much like in the real-life game “hide and seek”. To our knowledge there are no studies examining stealth-related behaviors as expressed in video games and baseline cognitive abilities, but studies of hide and seek and other deception-based games point to higher-order cognitive functions such as abstract thinking and theory of mind as being crucial to performance50,51.
You must be logged in to post a comment.