In order to carry out the study, we took into consideration the evolution of computer-based tests (CBTs) in psychology and their administration through the web.

The creation of the psychometric models that support the test’s characteristics, as well as breakthrough and cost reductions in information technology, have enabled the incorporation of new psychological variables into the realm of the measurable.

One of the pioneers in the use of computers for psychological measurement is Frederick Lord (Lord, 1968) who led the computer test design during the seventies (Wainer et al., 2000).

A CBT is defined as a test that uses a computer for the presentation of items, the entry of answers and their analysis and interpretation (Luque y González, 2011), and its items have been validated in their psychometric properties using a mathematical model (Olea et al., 2010). As Wainer et al. (2000) indicate, “computerizing test administration opens the door for many new kinds of test”. Some of the advantages of CBTs are: standard administration conditions for all test-takers thanks to common instructions; they may show quantitative and graphical information on the position of a subject within a group at a particular scale immediately after testing; and the administration of new item formats by implementing a dynamic graphic resource and incorporating audio or video-recorded simulated sequences, thus creating a more attractive environment for test-takers, which increases their concentration on the task (Bartram, 2006; Olea et al., 2010; Luque y González, 2011;DeBoer et al., 2014).

The use of computers, together with the application of the item response theory, enable the construction of computerized adaptive tests, in which the items adapt to the examinees’ ability level, according to their responses to previous items (Wainer et al., 2000) while using fewer test questions than do conventional tests (Weiss y Kingsbury, 1984). On the other hand, some of the disadvantages of CBTs are associated with the lack of control over variables such as the examinee’s motivation, the received feedback and some graphical interface elements that can become distracters. The fact that items are presented in a sequence that does not allow the examinees to revise or correct previous responses, as well as not knowing how to properly use a computer, may cause them anxiety (Luque y Ávila, 2008; Luque y González, 2011). For this reason, it is necessary to ensure that the administration does not demand further knowledge than that required by the test (Olea et al., 2010).

Internet-based tests

Nowadays, CBTs can be administered through the internet thanks to the advances in information technology. The items, presentation algorithms, and database with the answers are stored and accessed from a server, allowing greater control over the administration and easier access to results. They also provide logistical benefits when the users have access to the required technology and represent lower costs for those who are in different geographic areas (Olea et al., 2010). More significant for testing is the transference in locus of control provided by the internet from the ‘client side’ to ‘server side’ (Bartram, 2006). For paper and pencil testing, even computerized, producers had to provide users with test questions, scoring keys, and interpretation standards, that involve security breaches as not authorized use of the materials (Bartram, 2006). The internet reverses this situation, so that both, data and intellectual property, remain at the server, while the users only access the parts they need (Bartram, 2006). Having said that, there are some risks to be aware of when administering web-based tests, such as determining the minimum hardware and software requirements; establishing mechanisms for preventing and detecting errors in the administration and security breaches; determining the degree of required supervision; designing identification controls for the examinee, in order to prevent third party access to their personal information; and ensuring safe answer storage (ITC, 2006).

The advantages and disadvantages of the different mechanisms applied to the psychological evaluation are summarized in Table I.

TABLE I
QUALITATIVE COMPARISON BETWEEN DIFFERENT TYPES OF TESTS

nota Based on literature review.

In line with what was stated above, the goal of this study was to determine whether a CBT created to give insight on how the legitimating beliefs of violence are formed in children aged between 7 and 9 years old has any advantages over a traditional paper version. The hypothesis to test is that children who take the computer-based version have a higher tendency to use violence as a means to resolve conflicts, since the irregular presence of the evaluator reduces the fear of social sanctions when answering the items.

We designed an online role-playing game to quantify the actions and decisions of children for further analysis (Figure 1). Three situations of conflict are presented to the participants; then, they have to decide how to act. Moral disengagement mechanisms, perceived legitimization sources such as their peers and adults, as well as the contexts in which participants have observed more situations of violence, are also evaluated.

Figure 1
Videogame interface.

At the beginning of the game, a loading message is displayed because the application is first stored in cache in order to speed it up. The presentation starts once the loading is 100% complete. Children have a username and password to access the game interface, which are created upon enrollment in the information management platform.

Once they log in, the introduction to a fantasy world called ‘Legends of Almar’ appears. A voice reads all texts aloud to support the reading comprehension, thus combining graphic and sound resources. The ‘Druid’ is a character that gives them instructions about the game, the mission they have to accomplish, and the keys they have to press in order to control their character. Then they go to the character area, where they can select their character. On the left of the screen they will see the 7 characters available to choose from. The image of the chosen character appears in the middle of the screen with his/her name below and the skills information on the right. Finally, a box asking them to confirm the decision is displayed.

The character must go through three levels in order to accomplish its mission. As a playability criterion beyond the evaluation, the character must collect green, blue and red stones that appear in each level and the total is displayed on the top left corner of the screen. The character can move left and right and jump to grab the higher stones by pressing the arrow keys. In each instance, the players will come across characters who are mere observers and a character who stands in the way of the player, which prevents them to achieve their goal, thus creating a situation of conflict. In the first level, called Forest of the Druids, players have to face an opponent on equal terms; in the second one, Golden Sands, players face two characters that are perceived to be at a disadvantage; and in the third one, The Snowy Mountains, players are at a disadvantage compared to the opponent. These situations were created in order to study whether the children’s decisions for solving the conflict change depending on the way they perceive themselves and the problem.

After the conflict, the screen displays a question asking the children what they would do to resolve it, followed by four possible answers from which participants can only click on one. If players favor negotiation or nonviolent options, this action is executed, and the character moves on to the next level. If they choose to ask for help, they will be asked about the kind of help they want from people.

If participants choose to attack the opponent by themselves or with help from others, the screen displays eight questions about the justifying beliefs of their decision and what they think the observers will say; after that, players will move on to the next level. Once they have selected each answer, they are not allowed to change it. To minimize the error in the answer selection, they can only choose when the voiceover has read all of the texts in the picture, which gives the participants more opportunities to reflect on the action before selecting it.

At the end, all participants win the game regardless of the decisions they made. A magical tree appears for them to place the stones collected throughout the path. Subsequently, the Druid shows up to explain what violence means and asks the children where they have more frequently seen these types of situations. Options of different settings are displayed as well as ‘all of the above’ option. Once they have answered this, the Druid tells them that their adventure in the world of Almar has come to an end.

All of the participants’ answers, their selected character and sex of the characters they face throughout the game, as well as the data recorded upon enrollment are stored numerically in a database. A report is created from this data and can only be accessed by the research team.

In order to create a role playing game (RPG) for a unisex audience aged between 7 and 9, we used graphic resources in accordance with illustration and animation criteria; the most representative ones being: aesthetics that are easy to understand, synthesized art, ease of animation, chromatic parameters appealing to youth and children with intense colors and shadows with reduced level without affecting the intensity.

Regarding the content of the situation and the animated reenactment of the children’s decisions, we followed the rating parameters of the Entertainment Software Rating Board (ESRB, 2017) and the Pan European Game Information (PEGI, 2017). These organizations specify the nature of the graphical content in the production of responsible visual material targeted to children. To adapt the content, we followed the Rating E (Everyone), suitable for the entire audience.

Programming of the computing platform

The computing platform was developed in the PHP programming language and the game was designed in Flash. The criteria implemented in the software development are detailed below.

-- Entity-relationship model

We considered user requirements in terms of the internal relations within the information flow. An entity is any object, real or abstract, that exists or may come into existence about which information is stored. The relationships are represented as lines that show the type of relationship between entities. The relationship types in the model are:

One-to-many. One instance of an entity (A) is associated with several instances of an entity (B), while for one instance of entity B there is one instance of entity A. For example, a school can have many students, but a student can only belong to one school.

Many-to-many. Various instances of entity (A) are related to several others in another entity (B) and similarly in the opposite direction. For example, a student can take different tests and different students can take a test.

One-to-zero or many. One instance of an entity (A) is associated with zero, one or many instances of another entity (B), while one instance of entity B can only be associated with one instance of entity A. For example, a person can have zero, one, or more cars.

One-to-one. One instance of an entity (A) is associated with one other instance of another entity (B). For example, each person is associated with one identification number; likewise, that number can only be associated with that person.

-- Atributes

These are characteristics that define an entity; they make a company different from others of the same type. For example, a car entity might be described by the following attributes: brand, color, model and license plate, which make a car different than other.

The model implemented in ‘Legends of Almar’ is made up of 9 entities: Administrator, School, Grade, Teacher, Student, Evaluation, Action, Question, and Answer. Each entity, as well as the relationships that make up the diagram (Figure 2), has a specific function:

Figure 2
Entity relation model

The Administrator entity. Person who runs the platform and analyzes all the results.

The School entity. Represents all schools chosen for the evaluation. Each student must belong to this entity.

The Grade entity. Corresponds to the grade students are in each school. Each student must belong to this entity.

The Teacher entity. Person who coordinates a group of students. He/she is the only one allowed to add students to the system and can generate reports for his/her group of students.

The Student entity. Represents the students or children who directly interact with the game ‘Legends of Almar’, through which they will be evaluated. This entity stores the student’s information such as name, age, sex, school and grade.

The Evaluation entity. Represents each student's answers to the questions they are asked throughout the game. These questions will later be analyzed in reports.

The Action entity. Represents an action carried out in certain situations during the game. When asked, the student must decide which one to choose.

The Question entity. Represents the questions asked to the student during the evaluation, if she/he chooses the action that requires answering.

The Answer entity. Represents the answers to the questions displayed to the student at the time of the evaluation, provided he/she chooses the action that requires answering.

-- Reltionship between entities

The administrator is the only entity that can create schools, teachers and grades. Each teacher must belong to a school and a grade. The teacher is the only entity that can create students. An added student can play ‘Legends of Almar’ and will be evaluated through the game, so that an evaluation entity will be created at the end of each game. Throughout the game, the student clicks on actions that will be recorded in the evaluation. Students’ answers will also be recorded in the evaluation and then used for analyzing children’s legitimizing beliefs about violence. The resulting class diagram of the model contains the information or fields of each entity, which are taken to set up the software database.

Besides the basic fields and tables that can be obtained from the game requirements analyses, we have added a table to separate actions and answers to the questions within the evaluation. This is due to the fact that the questions depend on the actions chosen by the participant. It could easily happen that the actions chosen in the three levels, do not involve making an attack, which would not require a question about the justification of violence.

When playing the game by using printed character sheets and answering the questions asked by an evaluator, participants showed a lower tendency to use violence for solving the conflicts (Table II). Only 9% of children selected the attack option as their first choice. In the first level, where the player faces the opponent on equal terms, only 15% of children favored the attack, while 53% of others chose alternative solutions as talking with the opponent. Twenty-three percent chose to ask for help and none of them used it to attack the opponent. In the second level, where the participant has an advantage over the opponent, 45% of participants decided to kindly ask the opponent for the stones, 45% leaned towards negotiating and only 3% decided to attack. Only 8% of the participants chose to ask for help and 5% used it to attack the opponent. In the third level, where the player is at a disadvantage against the opponent, 68% of children favored talking, compared to 10% who decided to attack. In this case 20% of them decided to ask for help and 3% used it to attack the opponent.

TABLE II
DESCRIPTIVE FOR GROUP 1. TRADITIONAL VERSION OF THE TEST

For the computer-based version of the test, there was a decreased involvement of the evaluator (Table III). Participants were more likely to use violence in order to resolve the conflict. This was evident because in 17% of the cases, children decided to attack in the different levels of the game. In the first level, where the player faces the opponent on equal terms, 20% of children decided to attack, while the remaining 80% opted for more assertive solutions, especially asking for help, 35%, and talking, 30%. Only 3% of children asked for help to attack the opponent. In the second level, where the participant has advantage over the opponent, 58% of participants chose to negotiate and only 10% decided to attack. In this case, 5% of participants selected the option of asking for help to attack the opponent. In the third level, where the participant is at a disadvantage, 43% of them decided to talk with the opponent, versus 20% who decided to attack. Out of the 30% of children who decided to ask for help none of them used it to attack the opponent.

TABLE III
DESCRIPTIVE FOR GROUP 2. COMPUTER-BASED VERSION OF THE TEST USING RPG

The chi square test was used to compare the children’s decisions according to the test they took and determine if there are significant differences between them (Table IV). The results reveal that when the player faces the opponent on equal terms, there are significant differences in the proportion of children who decided to talk with the opponent (p= 0.04), since respondents of the traditional test were most likely to select this option (n= 21) than those who took the computer-based version (n= 12).

TABLE IV
COMPARATIVE RESULTS TRADITIONAL TEST VS COMPUTER-BASED TEST USING RPG

* CBT for Computer-based test; p for probability; bold for significant at 95% confidence level

In the situation where the participant has advantage over the opponent, we found significant differences in the decision to ask politely (p= 0.01) since respondents of the traditional test were most likely to select this option (n= 18) than those who took the CBT version (n= 4).

In the situation where the participant is at a disadvantage, the differences between the proportions of the chosen options are more evident. Regarding the decision of talking with the opponent, there are significant differences (p= 0.03) between children who took the traditional test (n= 27) and the ones who took the computer-based version (n= 17). We also found differences when they decided to ask for help to cooperate (p= 0.01), since only those who took the computer-based test selected this option. There were also substantive differences regarding the decision to ask for help to negotiate (p= 0.02), since children who took the traditional version were more likely to select this option (n= 7) than the ones who took the computer-based test (n= 4).