Abstract: Values-based behavior is a highly studied concept but difficult to assess within the Acceptance and Commitment Therapy’s framework. For this reason, the present study sought to psychometrically adapt the P-VQ in a sample of 368 adults. Validity evidence based on test content (analysis of representativeness of each item) and based on the relationship with other variables (concurrent, divergent, and incremental validity) were satisfactorily obtained. Exploratory Structural Equation Modeling analyses supported evidence in favor of a two-dimensional latent structure with a better fit compared to previous Confirmatory Factor Analysis proposals. Reliability analysis (Alpha and Omega) demonstrated robust internal consistency of the scale. Measurement invariance (configural, metric, and scalar) regarding gender was obtained as evidence of equity. In conclusion, the P-VQ demonstrated excellent psychometric properties with respect to validity, reliability, and fairness.
Keywords: Peruvian Version of the Valuing Questionnaire (P-VQ), psychometric adaptation, exploratory structural equation modeling, measurement invariance.
Resumen: La conducta basada en valores es un concepto altamente estudiado y difícil de evaluar dentro del marco de la Terapia de Aceptación y Compromiso. Por ello, este estudio pretendió adaptar psicométricamente el P-VQ en una muestra de 368 adultos. Se obtuvieron evidencias de validez vinculadas al contenido de la prueba (análisis de la representatividad de cada ítem) y en la relación con otras variables (validez concurrente, divergente e incremental). A través del modelamiento exploratorio de ecuaciones estructurales, se identificó evidencia a favor de una estructura latente bidimensional con un mejor ajuste en comparación a las propuestas previas basadas en análisis factoriales confirmatorios. El análisis de confiabilidad (Alfa y Omega) demostró una consistencia interna robusta de la escala. Como evidencia de equidad, se obtuvo la invarianza de la medida (configural, métrica y escalar) respecto al género. En conclusión, P-VQ demostró excelentes propiedades psicométricas con respecto a la validez, confiabilidad y equidad
Palabras clave: Versión Peruana del Cuestionario de Valoración (P-VQ), adaptación psicométrica, modelamiento exploratorio de ecuaciones estructurales, invarianza de medida.
Artículos originales
Psychometric Properties of a Peruvian Translation of the Valuing Questionnaire (P-VQ): An Exploratory Structural Equation Modelling Approach
Propiedades Psicométricas de la Traducción Peruana del Cuestionario de Valoración (P-VQ): Un Enfoque basado en el Modelamiento Exploratorio de Ecuaciones Estructurales.
Universidad Nacional de Córdoba
Recepción: 11 Enero 2022
Aprobación: 28 Julio 2022
Acceptance and Commitment Therapy (ACT) is considered as an effective contextual therapy for a series of mental health disorders and intrapersonal difficulties, such as anxiety (Arch et al., 2012), depression (Forman et al., 2007), impulsivity (Hasani et al., 2017; Morrison et al., 2020), and chronic pain (Wetherell et al., 2011). ACT has as a theoretical and epistemological background in functional contextualism, which is defined as "the action of the individual organism seen functionally, providing a strong emphasis on verbal behavior depending on the context" (Pérez-Acosta et al., 2002, p.108). Even though ACT’s theoretical roots are closely related to the principles of learning from radical behaviorism, verbal behavior has also been highly considered as an important core of this therapy, which was initially proposed by Skinner (1957) and later developed in greater detail by Hayes (1989). Verbal behavior is defined as a learning style based on the formation of relationships between verbal contents, called relational frameworks, and not necessarily due to environmental contingencies (Hayes, 1989).
When framing processes are generated, an individual will usually guide their conduct based on these and generate rigid patterns of behavior consequently. When this phenomenon is analyzed in clinical and psychopathological settings, psychological distress and human suffering are generally based on behaviors guided by verbal rules that, although they do not work in the current context of the subject, the person continues using them without being aware of it. In such a situation, ACT intends to promote adaptive and functional patterns according to the context in which the individual is immersed. This ability is called psychological flexibility (Polk et al., 2016).
The model of psychological flexibility is a unifying proposal that brings together a series of processes that aim to manage clinically relevant difficulties and problems related to the subject's functioning and adaptation (Hayes et al., 2012). Response styles related to these processes are also stipulated, being classified as an opening style, which includes the processes of acceptance and defusion; a focused style, including the processes of the present moment (mindfulness) and self-as-context; and a committed style, which considers action commitment processes and values (Hayes et al., 2012). One of the most exhaustive empirical and theoretical reviews developed on ACT interventions are those related to values-based behaviors because of their great relevance to generate valuable changes in clients’ lives (LeJeune & Luoma, 2019).
Values are defined as personal principles that allow behavioral regulation and are considered as a more relevant element compared to group rules or society expectations regarding subjects (Hayes et al., 2012). These are conceptualized as global consequences freely chosen and constructed at a verbal level, which come from dynamic and changing activity patterns that establish predominant reinforcements in relation to chosen life directions by the subject (Dahl et al., 2005; Wilson & DuFrene, 2009). They are also known as guiding principles, in the sense that they are general, abstract, with the function of providing directionality to the subject during life's difficulties, and although they present verbal components, they could be conceptualized as "wet abstractions because they represent desires that we always have” (Robb, 2007, p.121). Values present properties primarily of an appetitive nature (approach to a stimulus), rather than an avoidance function (moving away from a stimulus; LeJeune & Luoma, 2019). Finally, values are considered as fundamental since they direct behavioral patterns based on those objectives that are subjectively relevant to the client (Bach & Moran, 2008).
At a therapeutic level, values are fundamental because they (a) promote a constructive and consistent direction to what the client wants, (b) promote motivation and more flexible patterns depending on the context, (c) favor the manifestation of others processes of the model of psychological flexibility and (d) allow orientation towards more effective and pragmatic goals (Luoma et al., 2007).
Although values great relevance in the therapeutic process has been clearly identified, its assessment has currently presented several criticisms (Smout et al., 2014). Several assessment tools focused on this process have been proposed, such as exercises (e.g., Sweet Spot exercise [Wilson & DuFrene, 2009]), worksheets (e.g., Bull's eye [Dahl et al., 2009]), metaphors (Harris, 2009; Hayes et al., 2012), and psychometric measures (e.g., Valued Living Questionnaire [Wilson et al., 2010]).
Specifically, values psychometric measures have been criticized for their limitations at the domain level. Even though current measures asses’ different aspects of the client's life, they do not consider contextual features, which forces individuals to choose between non-representative answers (Smout et al., 2014). In addition, their content does not address obstacles that difficult achieving a valuable life nor make a clear differentiation between values and satisfaction with life (Carvalho et al., 2018). As an attempt to overcome these limitations, Smout et al. (2014) developed the Valuing Questionnaire (VQ), a new brief psychometric measure for values that has been clearly delimited at the conceptual level.
A series of adaptations of the VQ have been developed in specific populations, such as in patients with chronic pain in Brazil (Carvalho et al., 2018) and in Sweden (Rickardsson et al., 2019), adults with cardiovascular disease and risk in the United States (Kibbey et al., 2020) and adults in the process of weight loss in Persia (Nonahal et al., 2020), in addition to validations in Japanese (Doi et al., 2017), Iranian (Abdollah et al., 2018), Turkish (Aydin & Aydin, 2017) and Colombian (Ruiz et al., 2022) general samples. In each context, VQ has demonstrated satisfactory psychometric properties, a solid two-factor latent structure, theory-congruent relationships with other constructs, and adequate internal consistency reliability coefficients.
The current preliminary study aims to extend on previous adaptation processes to the Peruvian context, by assessing the psychometric properties of the VQ in a sample of Peruvian adults. Precisely, the study sought to obtain validity, reliability, and fairness evidence of the Peruvian version of the VQ, following the Standards for Educational and Psychological Testing (American Educational Research Association [AERA] et al., 2014) and the ITC Guidelines for Translating and Adapting Tests (International Test Commission [ITC], 2017).
The present study was performed on a total sample of 368 participants. Delimited inclusion criteria required having at least 18 years old, a Peruvian nationality, and to be residing in Lima, Peru during the data collection process. The sample could be described as a non-clinical group (89.95%) stated not being in current psychiatric and/or psychological treatment) of young adults (M = 24.85 years, SD = 8.49). Moreover, most participants were females (65.49%), undergraduate students (57.07%), and reported being single (84.71%).
Translation, data collection and data analysis procedures were presented to the Universidad de Lima Psychology faculty’s ethics board as part of a research project. Permission to translate and validate the VQ in Peru was granted by the original authors (Smout et al., 2014). Translation and adaptation were developed following certain international guidelines previously referred (AERA et al., 2014; ITC, 2017).
Translation was performed by an interdisciplinary team conformed by a native certified interpreter, an ACT specialist and a psychometrist (ITC, 2017). After several revisions, the resulting version was approved by the ACT specialist based on its representation of test content and was submitted to a panel of eight experts on ACT and/or Contextual Therapies who evaluated each item regarding test specifications and several criteria (representativeness, clarity in the items wording, and item’s utility). The expert’s suggestions were considered to develop the final version of the P-VQ (McGartland Rubio et al., 2003).
An online battery of measures was made available to participants. The first requirement for participating was to read the informed consent act which clarified the inclusion criteria and informed participants about the purpose of the study, expected duration, their right to decline to participate and to withdraw from the research at any stage, the confidentiality status and contact information from both authors. They were finally asked to complete some socio-demographic information items as well as five psychological measures, including the newly developed P-VQ.
Valuing Questionnaire (VQ)
VQ is an ACT-based self-report measure developed by Smout et al. (2014) to assess values-consistent living in clinical and non-clinical settings. The 10-item scale has a two-dimensional latent structure composed by Obstruction to valued living and Progress in valued living. Items present a response scale from Not at all true (0) to Completely true (6). Previous studies found support for the two-dimensional latent structure (Carvalho et al., 2018; Kibbey et al., 2020; Rickardsson et al., 2019;Smout et al., 2014), while others suggest allowing correlated errors between items 5 and 7 (Smout et al., 2014), or items 1 and 10, and 2 and 10 (Rickardsson et al., 2019) to improve fit. Nevertheless, most of these studies report excellent internal consistency indexes for both Obstruction (α = .76 - .89) and Progress (α = .81 - .88).
International Positive and Negative Schedule Short Form (I-SPANAS-SF)
Watson et al. (1988) developed the PANAS, a psychometric tool for measuring positive and negative trait affects. Thompson (2007) proposed the I-SPANAS-SF as a reduced version of the original measure, which had several cross-cultural psychometric studies in different contexts, including Peru. The Peruvian version of I-PANAS-SF (Gargurevich, 2010) has 10 items presented with a response scale from Never (1) to Always (5).
Cognitive Fusion Questionnaire (CFQ)
In the ACT framework, the tendency for behavior to be overly regulated and influenced by cognition is known as cognitive fusion. CFQ (Gillanders et al., 2014) aims to measure cognitive fusion through 7 Liker-type items, presented with a response scale ranging from Never true (1) to Always true (7).
Acceptance and Action Questionnaire - II (AAQ-II)
Experiential avoidance refers to the attempt to alter in any manner aversive private events such as thoughts, feelings, or physiological sensations, even if these attempts are incongruent with one’s values (Hayes et al., 1996). The AAQ-II is a measure of experiential avoidance developed by Bond et al. (2011) and composed by 7 Likert-type items, presented with a response scale from Never true (1) to Always true (7).
Satisfaction with Life Scale (SWLS)
SWLS aims to measure life satisfaction as a perceptive process on which an individual expresses a global assessment of its quality of life according to their own criteria. Diener et al. (1985) proposed SWLS as a unidimensional scale composed by 5 Likert-type items with a 7-point response scale from Strongly disagree (1) to Strongly agree (7).
An item analysis was conducted to examine descriptive statistics and assumptions for further procedures. As items are ordinal-level variables (Stevens, 1946), their treatment as interval-level variables requires at least five response categories and approximately normal distributions (Bandalos & Finney, 2018; Lloret-Segura et al., 2014; Watkins, 2018). Thereby, values of skewness (γ̂1) and kurtosis (γ̂2) greater than 2 were delimited to determine several deviations for normality (Muthén & Kaplan, 1985). A divergent stacked bar plot (Heiberger & Robbins, 2014) was employed to compare item response distributions, as well as examine relative frequencies on each category as indicator of floor/ceiling effects, and response styles (Van Vaerenbergh & Thomas, 2013) such as Aquiescence/Disaquiescence Response Style (ARS/DARS), Extreme Response Style (ERS), Mid Response Style (MRS).
As mentioned before, the Peruvian version of the VQ was submitted to eight experts (i.e., ACT and/or Contextual Therapists who have had at least 5 years of experience in contextual-behavioral interventions, and a profound knowledge regarding values’ conceptualization), who reviewed each item by several criteria, including representativeness, clarity in the wording of the items, and item’s utility. Among them, representativeness (i.e., the degree on which an item represents the construct intended to measure) was rated on a five-category response scale (McGartland Rubio et al., 2003). To provide validity evidence based on test content (Sireci & Faulkner, 2014), ratings were analyzed through Aiken’s V (Aiken, 1985), considering a .80 value as cutoff criteria suggested by Davis (1992).
The latent structure of the scale was studied through Confirmatory Factor Analysis (CFA), using a Pearson correlation matrix (Lloret-Segura et al., 2014; Rhemtulla et al., 2012; Watkins, 2018). MLR estimator was used to approach for deviations from normality (Brown, 2015; Wang & Wang, 2020). Three measurement models were initially tested. First, the original two-factor model proposed by Smout et al. (2014) and replicated by Carvalho et al. (2018) and Kibbey et al. (2020). Second, a modification of the original structure on which items 5 and 7 had correlated errors (Smout et al., 2014). Third, a model allowing correlated errors for items 1 and 10, and 2 and 10 (Rickardsson et al., 2019). As CFA restrictions on cross-loadings may result in biased inflated estimates of relationships between latent variables (Gomes et al., 2017) or compensatory ex post facto changes in model specification such as parceling or correlated errors (Asparouhov & Muthén, 2009), a fourth model was proposed based on Exploratory Structural Equation Modeling (ESEM). ESEM derives in a more realistic and less restrictive model while still holding a confirmatory approach (Marsh et al., 2009; 2014). MLR estimator was employed with a Target oblique rotation (Browne, 2001; Marsh et al., 2014). All four models were assessed through the Comparative Fit Index (CFI), Root Mean Squared Error of Approximation (RMSEA) and Standardized Root Mean Square Residual (SRMR). CFI ≥ .95, RMSEA ≤ .05 and SRMR ≤ .06 denoted an excellent fit, whereas CFI ≥ .90, RMSEA ≤ .08 and SRMR ≤ .08 pointed a reasonable fit (Keith, 2019). A simple structure (Thurstone, 1947) and salient factor loadings (λ > .4) were expected (Brown, 2015).
Relationships between Progress and Obstruction with other variables theoretically related to values were assessed through Factor Score Regression (FSR). FSR is an alternative to Structural Equation Modelling (SEM), characterized by a two-phase procedure in which factor scores are computed based on the measurement models; then, they are used as observed variables in a structural model (Devlieger & Rosseel, 2017; DiStefano et al., 2009; Hayes & Usami, 2020). When model complexity increases, FSR becomes a more viable alternative than SEM (Devlieger et al., 2019;Hayes & Usami, 2019); by considering the measurement models, factor scores represent a better indicator of each person location on the latent continuum than total scores (Mcneish & Gordon, 2020). The P-VQ factor scores were computed from the ESEM model; whilst the factor scores for the other variables were computed from independent measurement models tested through CFA. Model fit was assessed considering indexes CFI, RMSEA and SRMR for every measure; if necessary, re-specifications were done to improve fit (Schumacker & Lomax, 2016). Average Extracted Variance (AVE) was estimated for each latent construct, as well as internal consistency measures through the Alpha and Omega coefficients. Then, factor scores were correlated to provide validity evidence. Correlation magnitudes were established based on Cohen (1992) suggestions to determine small (r > .10), medium (r > .30) and large (r > .50) effect sizes. Additionally, factor scores were used in an incremental validity analysis with a similar approach as the one shown by Smout et al. (2014). A four-stage hierarchical regression was conducted with life satisfaction (SWLS) as a dependent variable. On the first stage, experiential avoidance (AAQ-II) was employed as the only predictor; the second stage added positive and negative affect (I-SPANAS-SF); the third stage included cognitive fusion (CFQ); finally, the fourth stage added both Obstruction and Progress (VQ). Independent variables in all stages were considered as non-orthogonal, since moderate to large relationships were found between them. The amount of explained variance attributed to VQ after controlling for other variables was assessed with ∆R2.
To assess reliability, three indexes of internal consistency were computed: Standardized Coefficient Alpha αstd, Coefficient Alpha α and Coefficient Omega ω (Cho, 2016; Falk & Savalei, 2011). Reliability values of .70 (Kline, 2020) were designated as expected thresholds.
Measurement invariance was assessed across gender groups to assure equanimity in the ESEM model (AERA et al., 2014; ITC, 2017). Nested models were compared to determine equivalence of model form (i.e., configural), factor loadings (i.e., metric) and items’ intercepts (i.e., scalar). Incremental indexes ∆CFI, ∆RMSEA and ∆SRMR were computed to determine presence of invariance (Putnick & Bornstein, 2016; Rutkowski & Svetina, 2014), in which values of ∆CFI > .010, ∆RMSEA > .015 and ∆SRMR > .030 indicated significant changes between models (Chen, 2007).
Most analyses were performed using Mplus (Muthén & Muthén, 2015, version 7.2) and R programming language, with the psych (Revelle, 2020, version 2.0.12), MVN (Korkmaz et al., 2019, version 5.8), and lavaan (Rosseel, 2020, version 0.6-7) packages.
Items’ descriptive statistics are presented in Table 1. Skewness and kurtosis indexes show non-several deviations from univariate normality. Therefore, treatment of items as interval-level variables is supported (Bandalos & Finney, 2018; Lloret-Segura et al., 2014; Watkins, 2018). CTT discrimination indexes indicate that most items contribute with large amounts of information about the participants latent trait rcit = .48 - .75. CTT endorsement indexes manifest that items provide information across the entire latent trait continuum M = 1.70 – 4.44. No missing values nor significant outliers regarding response patterns were detected.
Figure 1shows the item response distribution as relative frequencies. Obstruction’s items demonstrate a tendency of having more percentage of responses on the lower end of the scale, while Progress’ items show more responses on the upper end. The percentage of responses on both ends indicate potential ceiling effects for items 3, 7 and 9; and floor effects for item 8. Additionally, no clear pattern of response styles ARS, DARS, ERS nor MRS can be identified on the whole scale.
Items' Descriptive Statistics
Note M = Mean, SD = Standard deviation, Mdn = Median, γ̂1 = skewness index, γ̂2 = kurtosis index, rcit = corrected item-test correlation.
Figure 1
Items’ Response Distributions
Regarding content validity, all items achieved statistically significant Aiken’s V coefficients on the representativity criteria V = .94 – 1.00, p < .05. Furthermore, these values and their average (Mv = .98) are greater than the recommended threshold for a validity content coefficient V = .80 (Davis, 1992).
Table 2 shows the items’ Pearson correlation matrix. Moderate (r > .30) and large (r > .50) positive relationships were found for items that belong to the same dimension; on the contrary, negative relationships between items from different dimensions were small (r < .30) or irrelevant (r < .10). Kaiser-Meyer-Olkin test (Kaiser, 1974) suggested that the correlation matrix is appropriate for factor analysis KMO = .83. Additionally, Bartlett test of Sphericity indicated statistically significant relationships between items χ2 [9, N = 368] = 78.683, p < .001.
Items’ Correlation Matrix
Note Items 1, 2, 6, 8 and 10 measure Obstruction, while items 3, 4, 5, 7 and 9 measure Progress.
Mardia’s test for skewness γ̂1p = 675.57, p < .001 and kurtosis γ̂2p = 15.91, p < .001 indicated deviations from multivariate normality (Mardia, 1970). Moreover, Henze-Zirkler test (Henze & Zirkler,1990) showed a similar result HZ = 1.33, p < .001. Therefore, MLR estimator was used on CFA and ESEM models, to address deviations from normality.
Internal Structure of the Peruvian VQ
Note χ2 = Chi-squared, df = degrees of freedom, p = p value, CFI = Comparative Fit Index, RMSEA = Root Mean Square Error of Approximation, CI = Confidence Intervals, SRMR = Standardized Root Mean Square Residual.
Table 3 shows that, even though Models 1, 2 and 3 denoted a reasonable fit to data, Model 4 (ESEM) manifests the best fit. Comparisons between Models 1 and 4 are presented in Figure 2. Both CFA and ESEM models showed a simple structure with salient factor loadings λ > .4. As expected, CFA yields an inflated estimation of the relationship between Obstruction and Progress r = -.260, while ESEM yields a more accurate estimation r = -.220.
Figure 2
VQ’s CFA and ESEM Measurement Models
Note. VQ’s CFA model is shown on the left, while ESEM is shown on the right. Gray paths on the ESEM model indicate that items could load on both latent constructs.
Before testing relationships between variables, all measures’ internal structures were assessed through CFA, based on theoretical measurement models (see Table 4). High values of RMSEA for the AAQ-II and CFQ were found; thus, both models were re-specified. The AAQ-II model correlated errors between items 1 and 4 were allowed, based on previous studies (Edwards & Vowles, 2020); this modification improved fit significantly. Similarly, correlated errors for items 1 and 2, 2 and 3 were allowed in CFQ based on previous works (Lucena-Santos et al., 2017), yielding a better fit. High average extracted variance and internal consistency measures were estimated for the re-specified AAQ-II (AVE = .608, ω = .905); I-SPANAS-SF dimensions, Positive affect (AVE = .380, ω = .730), and Negative affect (AVE = .422, ω = .750); re-specified CFQ (AVE = .592, ω = .893); and SWLS (AVE = .572, ω = .870).
Once all measurement models were defined, correlations between latent variables were estimated through FSR. Factor scores were computed based on the original measurement models for the I-SPANAS-SF and SWLS, and based on the re-specified models for the AAQ-II and CFQ. Table 5 shows that Obstruction had moderate to strong positive relationships with negative affect, experiential avoidance, and cognitive fusion, and negative moderate relationships with positive affect and life satisfaction. The opposite tendency was found in Progress, as positive strong relationships were found with positive affect and life satisfaction, whereas negative moderate relationships were found with negative affect, experiential avoidance and cognitive fusion. Finally, Obstruction and Progress had a negative relationship.
Internal Structure of the AAQ-II, I-SPANAS-SF, CFQ and SWLS
Note χ2 = Chi-squared value, df = degrees of freedom, p = p value, CFI = Comparative Fit Index, RMSEA = Root Mean Square Error of Approximation, CI = Confidence Intervals, SRMR = Standardized Root Mean Square Residual.
aRe-specified models.
Relationships Between Values and Other Variables Using Factor Scores
Note All relationships were statistically significant (p < .001).
Results from Hierarchical Regression are shown in Table 6. Obstruction and Progress improved life satisfaction prediction ∆R2= 7.1%. In addition, Progress was a strong statistically significant predictor of life satisfaction (p < .001), after controlling for other variables.
Internal consistency estimates denoted acceptable values in all three estimated coefficients. Congeneric measures of reliability showed the highest value for Obstruction (ω = .800) and Progress (ω = .855), whereas Standardized Alpha and Coefficient Alpha demonstrated slight underestimated reliabilities for both Obstruction (αstd = .797; α = .796) and Progress (αstd = .850; α = .846). As Congeneric models tend to show a better fit, Omega indexes represent a more accurate estimation of reliability (Raykov & Marcoulides, 2017; Viladrich et al., 2017).
The ESEM model of the P-VQ showed an excellent fit for males (χ2 [26] = 31.587, p = .207; CFI = .982; RMSEA = .041, IC 90% [.000, .085]; SRMR = .037) and females (χ2 [26] = 43.061, p = .019; CFI = .977; RMSEA = .052, IC 90% [.021, .079]; SRMR = .029). As shown in Table 7, the model M1 denoted an acceptable fit, thus suggesting support for configural invariance. Furthermore, the incremental indexes ∆CFI, ∆RMSEAand ∆SRMR indicate evidence for metric and scalar invariance regarding gender.
Hierarchical Regression Analysis of Life Satisfaction
Note β = regression coefficient, p = p value, R2 = percentage of explained variance, ΔR2 = change in percentage explained variance.
Fit Measures of Nested Models: Configural, Metric and Scalar Invariance on the ESEM model
Note χ2 = Chi-squared value, df = degrees of freedom, CFI = Comparative Fit Index, RMSEA = Root Mean Square Error of Approximation, SRMR = Standardized Root Mean Square Residual. M1 = Configural invariance. M2 = Metric invariance. M3 = Scalar invariance
Note. χ2 = Chi-squared value, df = degrees of freedom, CFI = Comparative Fit Index, RMSEA = Root Mean Square Error of Approximation, SRMR = Standardized Root Mean Square Residual. M1 = Configural invariance. M2 = Metric invariance. M3 = Scalar invariance
The aim of the current study was to identify psychometric properties of the present measure in a mostly non-clinical sample of Peruvian adults. Data analysis methods from previous studies on the VQ (e.g., Carvalho et al., 2018; Kibbey et al., 2020; Smout et al., 2014) were implemented, along with new proposals (e.g., ESEM, FSR) to assess validity, reliability, and fairness.
Smout et al. (2014) developed the VQ with the intention to overcome deficiencies in previous measurement tools of values. Certainly, VQ is characterized by its simplicity and robust psychometric properties. Studies across different contexts have shown that VQ provides reliable measures and valid interpretations of values-consistent living (e.g., Rickardsson et al., 2019). To extend this framework, the VQ was adapted to Peru, starting with a Spanish translation done by an interdisciplinary team composed of a licensed native interpreter, an ACT specialist, and a psychometrist (ITC, 2017).
In order to obtain validity evidence based on test content, the translated version of the VQ was assessed through a panel of 8 experts on ACT and Contextual Therapies, who evaluated each item regarding several criteria, including representativeness (McGartland Rubio et al., 2003; Sireci & Faulkner, 2014). Based on their recommendations, item 2 (i.e., I was basically on “auto-pilot” most of the time) was modified since the concept was difficult to understand in the Peruvian context and was considered as an uncommon expression. It was adapted utilizing other terms (e.g., Básicamente, la mayor parte del tiempo he vivido dejándome llevar por la rutina del día a día [I have basically lived by letting myself be carried away by the day-to-day routine]). These changes allowed a better understanding of the original item’s meaning. The results of this review led to the final version, which was used in the data collection procedure.
Validity evidence based on the internal structure was confirmed since the original two-dimensional latent structure demonstrated a satisfactory fit to empirical data (AERA et al., 2014). Furthermore, the ESEM approach (Marsh et al., 2014) derived in a better fitting model than all previous CFA proposals in the literature (Carvalho et al., 2018; Kibbey et al., 2020; Rickardsson et al., 2019; Smout et al., 2014). This suggests that CFA restrictions on cross-loadings affect model fit, and ex post facto modifications in model specification may not be required (Asparouhov & Muthén, 2009; Gomes et al., 2017). In contrast, allowing cross-loadings through the ESEM method derives in a more realistic and less restrictive model (Marsh et al., 2009; 2014). Despite differences in model fit, both ESEM and CFA models yielded an expected simple structure (Thurstone, 1947) with salient loadings (Brown, 2015).
Validity evidence based on the relationship with other variables was established through convergent and discriminant studies. All correlations were consistent with theoretical premises. For instance, Obstruction is defined as a constant focus on undesired psychological experiences, the avoidance processes facing them, and the inattention of those values and aspects relevant to the subject’s life as a consequence (Smout et al., 2014); moderate to large positive relationships with experiential avoidance, cognitive fusion, and negative affect, and the moderate negative associations with positive affect and life satisfaction demonstrate a great degree of theoretical and empirical consistency (Carvalho et al., 2018; Pérez, 2014; Polk et al., 2016). Similarly, Progress is defined as the awareness of those personal elements considered relevant and important, which promote processes of perseverance and directionality (Hayes et al., 2012). Both the large positive associations with life satisfaction and positive affect, as well as the moderate negative associations with experiential avoidance and cognitive fusion have a conceptual sense highly congruent with the underlying theory, and previous research (Rickardsson et al., 2019; Hayes et al., 2012).
In further analyses on the relationship with other variables, an incremental validity assessment was proposed to follow the same methods implemented in previous studies. Obstruction and Progress accounted for statistically significant unique variance of life satisfaction after controlling for experiential avoidance, positive and negative affect, and cognitive fusion. As found by Smout et al. (2014), Progress proved to be a highly statistically significant predictor of life satisfaction, whereas Obstruction did not present the same condition. Similar results were found by Rickardsson et al. (2019) on quality of life, and by Kibbey et al. (2020) on general health status as dependent variables.
Reliability evidence based on internal consistency was addressed through three different coefficients with specific underlying measurement models (Dunn et al., 2013). In summary, all three coefficients demonstrated an excellent degree of internal consistency (Kline, 2020). On the other hand, fairness evidence based on differences between males and females was presented by a measurement invariance assessment. Configural, metric and scalar invariance were demonstrated, suggesting that the Peruvian version of the VQ denotes the same construct meaning for males and females (Putnick & Bornstein, 2016), allowing fair comparisons regarding gender based on the ESEM model.
The present study has some limitations to be acknowledged. The recruited sample was conformed by fewer participants compared to previous psychometric adaptations of the VQ, and could be described as a mainly a non-clinical group of young adults from Lima Peru; those characteristics could limit the generalizability of the results. The study was carried out during the COVID-19 pandemic, which may influence results since Peru is one the most affected countries regarding economical and socio-political aspects. Finally, the cross-sectional nature of this study, although common in psychometric studies, limits further assessments of validity, reliability and fairness which could be carried out in prospective or retrospective designs. It is important to note that even though a Colombian version of the VQ has been published previously (Ruiz et al., 2022), the current research intended to propose a particular version of the test that could be better understood by the Peruvian population. Based on that idea, relevant modifications were implemented in the wording of the items, specifically in item 2 (e.g., I have basically lived by letting myself be carried away by the day-to-day routine)and in item 10 (e.g., It seemed like I was just “doing things mechanically or by inertia,” rather than focusing on what was important to me).
Despite these limitations, the present study demonstrated that the P-VQ has strong psychometric properties regarding validity, reliability, and fairness, and that this measure is consistent with international modern standards on test construction, translation, and adaptation (AERA et al., 2014; ITC, 2017). Given that this is a preliminary exploratory study, the present findings are promising and lay the foundation for subsequent research. Future studies may assess the psychometric properties of the Peruvian version of the VQ in a clinical sample; specifically, a measurement invariance assessment between clinical and non-clinical participants may be useful to determine whether group comparisons between both are viable. Additionally, other sources of validity, reliability and fairness could be explored through alternative latent variable models. For instance, considering a bifactor model or Item Response Theory models which could bring more insight on item characteristics such as difficulty, discrimination and guessing.
The dataset supporting the results of this study was published on the OSF public repository and is available at https://doi.org/10.17605/OSF.IO/Z9BQK
The entire set of analytical methods supporting the results of this study was published on the OSF public repository and is available at https://doi.org/10.17605/OSF.IO/Z9BQK
Information related to this research and/or materials associated with the P-VQ are available upon request to the corresponding author. The P-VQ has been published as an appendix.
Peruvian Version of the Valuing Questionnaire (P-VQ) - Versión Peruana del Cuestionario de Valoración (P-VQ)
Lea atentamente cada enunciado. Luego, seleccione el número que mejor describa qué tan cierto ha sido cada enunciado para usted DURANTE LA SEMANA PASADA, INCLUYENDO EL DÍA DE HOY. Responda tomando en cuenta las alternativas de respuesta, que van desde el número 0, que significa para nada cierto, hasta el número 6, que significa completamente cierto.
1. Pasé mucho tiempo pensando en el pasado o el futuro, en lugar de involucrarme en actividades que eran importantes para mí.
2. Básicamente, la mayor parte del tiempo he vivido dejándome llevar por la rutina del día a día.
3. Trabajé para alcanzar mis metas incluso en los momentos en que no me sentía motivado(a) para hacerlo.
4. Estuve orgulloso(a) de cómo viví mi vida.
5. Progresé en las áreas de mi vida que más me importan.
6. Hubo pensamientos, sentimientos o recuerdos dolorosos que se interpusieron en lo que realmente quería hacer.
7. Seguí mejorando para lograr ser la clase de persona que quiero ser.
8. Cuando las cosas no sucedían de acuerdo al plan, me rendía fácilmente.
9. Sentí que tenía un propósito en la vida
10. Parecía que solo “hacía las cosas de forma mecánica o por inercia”, en lugar de enfocarme en lo que era importante para mí.
Citar este artículo como: Okumura-Clark, A., & Zegarra-López, A. (2023). Psychometric Properties of a Peruvian Translation of the Valuing Questionnaire (P-VQ): An Exploratory Structural Equation Modelling Approach. Revista Argentina de Ciencias del Comportamiento, 15(3), 56-71.
https://revistas.unc.edu.ar/index.php/racc/article/view/36469/44297 (pdf)
Enviar correspondencia a: Okumura-Clark, A. E-mail: aokumura@ulima.edu.pe
Items' Descriptive Statistics
Note M = Mean, SD = Standard deviation, Mdn = Median, γ̂1 = skewness index, γ̂2 = kurtosis index, rcit = corrected item-test correlation.
Figure 1
Items’ Response Distributions
Items’ Correlation Matrix
Note Items 1, 2, 6, 8 and 10 measure Obstruction, while items 3, 4, 5, 7 and 9 measure Progress.
Internal Structure of the Peruvian VQ
Note χ2 = Chi-squared, df = degrees of freedom, p = p value, CFI = Comparative Fit Index, RMSEA = Root Mean Square Error of Approximation, CI = Confidence Intervals, SRMR = Standardized Root Mean Square Residual.
Figure 2
VQ’s CFA and ESEM Measurement Models
Internal Structure of the AAQ-II, I-SPANAS-SF, CFQ and SWLS
Note χ2 = Chi-squared value, df = degrees of freedom, p = p value, CFI = Comparative Fit Index, RMSEA = Root Mean Square Error of Approximation, CI = Confidence Intervals, SRMR = Standardized Root Mean Square Residual.
Relationships Between Values and Other Variables Using Factor Scores
Note All relationships were statistically significant (p < .001).
Hierarchical Regression Analysis of Life Satisfaction
Note β = regression coefficient, p = p value, R2 = percentage of explained variance, ΔR2 = change in percentage explained variance.
Fit Measures of Nested Models: Configural, Metric and Scalar Invariance on the ESEM model
Note χ2 = Chi-squared value, df = degrees of freedom, CFI = Comparative Fit Index, RMSEA = Root Mean Square Error of Approximation, SRMR = Standardized Root Mean Square Residual. M1 = Configural invariance. M2 = Metric invariance. M3 = Scalar invariance
