Abstract: Despite the increasing progress in the use of instruments to assess emotional health in the perinatal and puerperal periods, it is emphasized that most of the tools used do not have psychometric evidence for the population of pregnant women. The Depression, Anxiety and Stress Scale (DASS-21) enables the measurement of anxiety, stress, and depression levels, simultaneously. The study aimed to assess the internal consistency of the DASS-21, the convergent validity of the scale in relation to the State-Trait Anxiety Inventory (STAI), the Beck Depression Inventory - second edition (BDI-II), and the Perceived Stress Scale (PSS) and the stability of the evaluations in a range of up to 12 weeks. Fifty-three (first evaluation) and 35 pregnant women (second evaluation) participated in the research. The scale showed adequate internal consistency, stability, and convergence rates for the investigated population. The DASS-21 can help practitioners and researchers of health, providing agility in the diagnosis of unfavorable emotional indicators in different gestational periods.
Keywords: Gestation, DASS-21, EDAE-21, translated version for Brazil, psychometric properties, maternal health.
Resumo: Apesar do crescente avanço no uso de instrumentos para a avaliação da saúde emocional nos períodos perinatal e puerperal, destaca-se que a maioria das ferramentas utilizadas não possui evidências psicométricas para a população de gestantes. A Escala de Depressão, Ansiedade e Estresse (EDAE-21) possibilita a mensuração simultânea de níveis de ansiedade, estresse e depressão. O estudo objetivou avaliar a consistência interna da EDAE-21, a validade convergente da escala em relação ao IDATE, à EEP e ao BDI-II, e a estabilidade das avaliações em um intervalo de até 12 semanas. Participaram da pesquisa 53 gestantes (primeira avaliação) e 35 gestantes (segunda avaliação). A escala apresentou índices de consistência interna, estabilidade e convergência adequados à população investigada. A EDAE-21 pode auxiliar profissionais e pesquisadores da saúde, proporcionando agilidade no diagnóstico de indicadores emocionais desfavoráveis em diferentes períodos gestacionais.
Palavras-chave: Gestação, DASS-21, EDAE-21, versão traduzida para o Brasil, propriedades psicométricas, saúde materna.
Resumen: A pesar del creciente progreso en el uso de instrumentos para evaluar la salud emocional en el período perinatal y puerperal, se destaca que la mayoría de las herramientas utilizadas no tienen evidencia psicométrica para mujeres embarazadas. La Escala de Depressión, Ansiedad y Estrés (EDAE-21) mide simultáneamente los niveles de ansiedad, estrés y depresión. El estudio evaluó la consistencia interna del EDAE-21, la validez convergente de la escala en relación con el STAI, el BDI-II y el EEP y la estabilidad de las evaluaciones en un intervalo de hasta 12 semanas. 53 mujeres embarazadas (primera evaluación) y 35 mujeres embarazadas (segunda evaluación) participaron en la investigación. La escala presentó valores de consistencia interna, estabilidad y convergencia adecuados para la población investigada. El EDAE-21 puede ayudar a los profesionales de la salud y a los investigadores, proporcionando agilidad en el diagnóstico de indicadores emocionales desfavorables en diferentes períodos de gestación.
Palabras clave: Gestación, DASS-21, EDAE-21, versión traducida para Brasil, propiedades psicométricas, salud maternal.
Psychological Assessment
Depression, Anxiety and Stress Scale (DASS-21): Psychometric properties in pregnant women
Escala de Depressão, Ansiedade e Estresse (EDAE-21): Propriedades psicométricas em gestantes brasileiras
Escala de depresión, ansiedad y estrés (edae-21): propiedades psicométricas en mujeres embarazadas brasileñas
Stefanny Aparecida Silva stefannysilva-@hotmail.com
Regina B. Zanon
Veronica Aparecida Pereira
Received: 16 June 2020
Accepted: 25 August 2021
Among the different forms of emotional health assessment, the use of scales, inventories, and other psychometric instruments contributes to the characterization of signs and symptoms, enabling prevention and treatment. The instruments are composed of specific items that allow the quantification of latent variables, inherent to unobservable constructs, such as anxiety, stress, and depression. Accordingly, they make it possible to characterize the intensity and frequency of these signs or symptoms, contributing to treatment, monitoring, and intervention.
However, unlike diagnostic interviews, assessment instruments in the area of emotional health cannot be used as a clinical diagnosis, but rather as a complement to it. Their use enables the standardization of language among healthcare providers, being able to guide decisions and behaviors based on the diagnosis. Healthcare providers, when choosing to use these instruments, should contextualize them to the reality of the person being evaluated (Gorenstein et al., 2015).
Recent studies have highlighted the need to measure levels of anxiety, stress, and depression in pregnant women as these symptoms can be misidentified as common physiological symptoms of pregnancy, such as difficulties with sleep or breathing, excessive tiredness, irritability, muscle weakness, and stomach problems. These symptoms commonly go unnoticed by health teams and by the pregnant women themselves, and, in many cases, this can lead to unfavorable outcomes, such as perinatal and/or postpartum depression (Ferrari, 2018).
Rating scales are often used in the treatment of patients with depressive and anxiety disorders, these being common debilitating psychiatric problems. Anxiety is inherent to human existence, however, at exaggerated levels, it can lead to pathologies. Several categories of anxiety have been postulated over the years, with one of the classic views being the one postulated by Biaggio et al. (1977). These authors subdivided anxiety into “trait” (emotional pattern of the individual’s emotional response) and “state” (subject’s anxiety at the time), which were presented in two inventories, with 20 items each, enabling the characterization of state anxiety - State-Trait Anxiety Inventory - state (STAI-S) (noticed in the last month) and trait anxiety - STAI-T (noticed most of the time). The questions are self-administered, with a Likert-type response scale ranging from one to four points. For the correction, inversion criteria are applied for the positive factors.
Another important emotional change during pregnancy is the stress, observed when individuals are faced with changes in their context especially if these changes impact them negatively. As pregnancy is a period of great changes in women’s lives, it is important to assess how they perceive and deal with stressful situations. Among possible instruments, the Perceived Stress Scale (PSS) proved to be relevant in the investigation of symptoms in pregnant women (Brito, 2018), with its application being quick and simple (14 self-administered questions), having scores from 1 to 4 points.
Depression rating scales were improved in the late 1950s as a result of advances in research with psychotropic drugs, more specifically antidepressants. Due to the multifactorial nature of depression, many instruments have emerged since then, and one of the most used scales is the Beck Depression Inventory (BDI-II), the only one that was translated and adapted to Portuguese and that presents instructions for its application, validity, and reliability in several studies (Gorenstein et al., 2015). In pregnant women, the BDI-II is considered an adequate instrument for the identification of somatic and non-somatic symptoms, helping to prevent pathologies, such as postpartum depression (Stramrood et al., 2013).
Developed by Lovibond and Lovibond (1995), the Depression, Anxiety and Stress Scale (DASS-21) allows the assessment of depression, anxiety, and stress indicators simultaneously. It has two best-known versions: one consisting of 42 items and a shorter one, with 21 items, which is the most used nowadays. The DASS-21 consists of three subscales with a total of 21 items (seven for anxiety, seven for depression, and seven for stress) evaluated using a Likert-type scale, ranging from zero to four points, related to the frequency of the observation of symptoms during the previous week. Its application is easy and rapid, and it is used in clinical and non-clinical environments with adolescents and adults. The translation of the DASS-21 into Portuguese was made by a specialist in mental health and psychiatric nursing and bilingual professor, and it was named the Escala de Depressão, Ansiedade e Estresse (EDAE-21) (Vignola & Tucci, 2014; Patias et al., 2016).
Even if the three constructs - anxiety, stress, and depression - are superimposed on each other (Vignola & Tucci, 2014), that is, correlated, there is still a great deficit of instruments that measure them simultaneously, such as the DASS-21. Scales that assess the three constructs, especially in the population of pregnant women, can be beneficial as they optimize the application time. Furthermore, despite the growing advancements in the use of instruments for the emotional assessment in the perinatal and puerperal periods, most of these do not have specific psychometric evidence for the population of pregnant women, highlighting the need for this (Trombetta et al., 2019).
In the international literature, the DASS-21 has been widely used in the population of pregnant women, associating aspects of anxiety, stress, and depression with biological factors of pregnancy (Miller, 2019), correlated with the high levels of emotional indicators that the instrument aims to measure. The populations of pregnant women in the studies that used the instrument were large and from different countries, such as Iran, the United States, and Peru, and all indicated that the DASS-21 would be adequate to measure and distinguish symptoms of anxiety, stress, and depression in perinatal populations (Miller, 2019).
National studies using the EDAE-21 with pregnant women are scarce. However, the study published by Castanheira et al. (2017), using the instrument with a population of 200 pregnant women, with an average of 31 weeks of gestation, highlighted that emotional indicators were correlated with sociodemographic variables and marital satisfaction, indicating that the instrument is able to distinguish emotional aspects, being subject to correlation with other variables.
Considering the specificities of pregnancy and the need to assess maternal emotional health at different periods, as well as the scarcity of studies investigating the psychometric properties of the EDAE-21 for use in Brazil, this study aimed to evaluate: 1. the internal consistency of the EDAE-21 for the population of pregnant women at two different times of application; 2. the convergent validity between the EDAE-21 in relation to the STAI (anxiety measure), the BDI-II (depression measure), and the PSS (stress measure); and 3. the stability of the evaluations over an interval of up to 12 weeks of reapplication. It should be emphasized that internal consistency, convergent validity, and stability are important to assess the reliability and validity of a measurement instrument, representing information about its psychometric properties (Souza et al., 2017). It should also be highlighted that validity is not a feature of the instrument, being determined from specific questions and populations (Souza et al., 2017), which makes research on the EDAE-21 with a sample of Brazilian pregnant women particularly important.
The descriptive data of the sample refer to two different periods of evaluation of the same population, with a sample loss in the second period of approximately 34% of the participants. In the first evaluation, there were 53 pregnant women with gestational age between 18 and 30 weeks. In the second one, only 35 of these women, with gestational age between 24 and 39 weeks, participated of the present study. In relation to age and education, in the first assessment, the pregnant women, on average, were 27.58 years old and had a mean of 11.49 years of schooling, with similar numbers in the second assessment (27.28 years old and 11.34 years of schooling). Most of these women were multiparous (71.7% in the first assessment and 65.7% in the second). It was also observed that the majority of the pregnant women lived with a partner (96.5% in the first assessment and 94.3% in the second), and most of them were employed (54.7% in the first assessment and 57.1% in the second). Regarding pregnancy, most were not planned (73.6% in the first assessment and 71.4% in the second), with a high percentage of the pregnant women presenting low gestational risk (62.3% in the first assessment and 71.4% in the second).
The pregnant women were being regularly monitored at the High-Risk Prenatal Outpatient Clinic of the University Hospital of the Federal University of Grande Dourados (Hospital Universitário da Universidade Federal da Grande Dourados [HU/UFGD]) or at the Primary Health Units (Unidade Básica de Saúde [UBS]) and Women’s Care Center (Centro de Assistência à Mulher [CAM]). As exclusion criteria, the following factors were considered: having chronic diseases or some disability that made it impossible to respond to the instruments proposed in the study, being indigenous, having a psychiatric history, and/or being pregnant with twins.
The following instruments were adopted in this research:
Sociodemographic characterization form: developed specifically for this study, covering age, education, marital status, number of previous pregnancies, number of living children, physical activity, pregnancy planned or not, main sources of financial and emotional support, among other aspects.
Escala de Depressão, Ansiedade e Estresse (EDAE-21) (Vignola & Tucci, 2014): it enables the assessment of depression, anxiety, and stress indicators simultaneously. It consists of three subscales with 21 items, evaluated using a Likert-type scale ranging from zero to four points, related to the frequency of observation of symptoms during the previous week (Patias et al., 2016). Patias et al. (2016) carried out a study with 426 adolescents of both sexes, aged between 12 and 18 years, and found alpha indexes of 0.86 for stress, 0.83 for anxiety, and 0.90 for depression. In this study, the EDAE-21 presented adequate psychometric properties for its tripartite model. The EDAE-21 is the focus instrument of this study, being administered in order to investigate convergent validity, internal consistency, and temporal stability in the population of pregnant women.
The Beck Depression Inventory (BDI-II): it is composed of 21 sets of statements rated from zero to three. The individual chooses within each set of statements the one that would best describe how they were feeling in the previous two weeks and on the present day. The measurement of depression is classified into four levels: minimal (scores less than 13), mild (scores from 14 to 19), moderate (scores between 20 and 28), and severe (scores 29 to 63). The total score is obtained through the sum of all individual items (Paranhos et al., 2010). In Brazil, Gorenstein and Andrade (1996) investigated the psychometric properties of the BDI-II, finding high internal consistency (0.81), indicating good construct validity of the Portuguese version. In the same study, the researchers investigated the discriminant validity of the BDI-II, finding significant differences in scores in different samples of patients clinically diagnosed as anxious and depressed, demonstrating a response pattern similar to that obtained with the original version of the instrument. In the present study, the BDI-II was used to verify the convergent validity of the EDAE-21 regarding symptoms of depression.
The State-Trait Anxiety Inventory (STAI) (Fioravanti, 2006): a self-administered instrument was applied to assess signs of state and trait anxiety. The inventory is composed of two scales, one for trait anxiety (STAI-T), more related to indicators of anxiety observed in the long term, and another for state anxiety (STAI-S), corresponding to anxiety based on observations of the previous month. Each scale is composed of 20 items, with responses given on a Likert-type scale, in which 1 represents “almost never” and 4, “almost always”. On both scales, total values greater than 45 points may indicate clinical anxiety (Biaggio et al., 1977). The instrument was translated and adapted for use in Brazil by Fioravanti (2006), presenting favorable indexes of internal consistency (Cronbach’s alpha of 0.89 for the STAI-S and 0.88 for the STAI-T) for a Brazilian sample of 655 university students. In the present study, the BDI-II was used to verify the convergent validity of the EDAE-21 in relation to symptoms of anxiety.
The Perceived Stress Scale (PSS) (Dias et al., 2015): it is a self-administered instrument composed of 14 questions with response options on a Likert-type scale ranging from zero to four. As there are questions with positive and negative denotations, the researcher performs inversions of the score, which can range from zero to 56 (Brito, 2018). Concerning the psychometric properties for Brazil, a study recently carried out with university students (n = 1,081) revealed adequate internal consistency values (Cronbach’s alpha = 0.83) (Dias et al., 2015). In the present study, the PSS was used to verify the convergent validity of the EDAE-21 with regard to stress symptoms.
The study is linked to a project carried out at a University Hospital, in a city of the state of Mato Grosso do Sul, approved by the Ethics Committee of the Federal University of Grande Dourados - under authorization No. 2.421.197. The pregnant women were approached and invited to participate in the study while waiting for outpatient consultations for their care or routine prenatal exams. The pregnant women agreed to participate in the study by signing the Free and Informed Consent Form. To characterize the sociodemographic data, the general data recording protocol was applied individually. Medical records of the pregnant women were also consulted for the collection of clinical and obstetric data.
To assess maternal emotional health, the pregnant women responded to the inventories and scales to characterize indicators of depression, anxiety, and stress. The instruments were self-administered, with their application time being approximately 25 minutes, in places previously prepared by the researcher for the greater convenience of the participants. All instruments were evaluated shortly after their application. Feedback was given verbally as soon as the pregnant woman finished the application of the instruments, and the researcher made it clear that it constituted possible emotional indicators, rather than a report. When the rates were above the expected, the researcher advised the participants to seek free psychological services in the city and provided relaxation guidelines to minimize the altered levels. For pregnant women with gestational diabetes mellitus (GDM), guidelines were given on nutrition.
After an interval between six and 12 weeks, the instruments were reapplied in order to verify the stability of the instruments for assessing emotional health at different gestational times, with a timeframe of up to 12 weeks.
All data were organized and exported to the Statistical Package for the Social Sciences (SPSS) software, version 24.0, for statistical analysis. To analyze the internal consistency, the reliability test was performed, based on Cronbach’s alpha, with an alpha requirement ≥ 0.70 (Souza et al., 2017). To analyze the convergent validity, Pearson’s correlation test was performed. Concerning the interpretation of the correlation values, the criteria proposed by Cohen (1988) were adopted, as this is one of the most used classifications in the literature, namely: “large” 0.50 to 1.00, “moderate” from 0.30 to 0.49, and “small” from 0.10 to 0.29. For the stability analysis, the analysis of the measurements observed at the two different times of pregnancy was also considered.
Among the psychometric properties of the instruments in the present study, the assessment of the reliability of the EDAE-21 was firstly performed by verifying the internal consistency with the other instruments (verification of Cronbach’s alpha) and stability analysis (intraclass correlation between the first and second evaluations). Initially, regarding the internal consistency, the results were organized according to the general consistency of the instruments. As shown in Table 2, it was found that all the instruments used in this study presented good levels of internal consistency, with alpha values above 0.70, both in the first and second assessments.
For the second reliability criterion, the stability of the EDAE-21 was verified in the first and second applications, based on the intraclass correlation analysis. The evaluations were carried out considering the gestational age, with a mean of 23.11 weeks (SD = 3.02) in the first evaluation and 31.6 (SD = 4.9) in the second. The mean time between assessments was 8.5 weeks. The intraclass correlation results for the EDAE-21 were 0.87 for EDAE-21 (anxiety), 0.86 for EDAE-21 (depression), and 0.75 for EDAE-21 (stress).
For the convergence analysis, correlations between the total scores of the instruments were verified in the first evaluation. The correlations between the specific items of the EDAE-21 for each of the constructs were positive, indicating a significant convergence for the STAI-S (r = 0.615, p = 0.000), and STAI-T (r = 0.600, p = 0.000), PSS (r = 0.731, p = 0.000), and BDI-II (r = 0.854, p = 0.000), as presented in Table 3.
The work of healthcare providers with pregnant women requires instruments and strategies that can optimize the time of diagnosis and intervention with this public, allowing more time to listen to their needs, doubts, and expectations. Therefore, the present study, by presenting evidence of the psychometric properties of a multimodal instrument, the EDAE-21, can help healthcare workers and researchers, providing agility in the screening for unfavorable emotional indicators in different gestational periods.
The present study investigated the internal consistency, stability, and convergent validity of the EDAE-21 for use with pregnant women, collaborating with previous studies (Trombetta et al., 2019) that highlight the need for reliable instruments to assess the emotional health of adult women in the gestational period. The data showed a strong consistency between the three subscales of the EDAE-21, considering that Cronbach’s alpha values were all above 0.70, which are considered ideal (Souza et al., 2017). The Cronbach’s alpha values found at the first and second evaluations were 0.84 and 0.91 for the anxiety subscale, 0.94 and 0.93 for the stress subscale, and 0.84 and 0.87 for the depression subscale. These values are close to the findings of Castanheira et al. (2017), who reported an alpha of 0.86 for depression, 0.83 for anxiety, and 0.88 for stress in a sample of pregnant women. It should be emphasized that the findings of psychometric studies on the EDAE-21 involving other populations also present high alpha levels in the three subscales (Patias et al., 2016), showing that the subparts of the scale seem to be sensitive to the measure of the investigated constructs in different populations.
The reliability of the EDAE-21 factor structure shows that the constructs were duly investigated from the different items that make up each of its scales, whose contents were properly understood by the pregnant women that participated in the study. Accordingly, the instrument was found to be effective and qualified to assess specific emotional states of the studied population. Regarding this aspect, it should be highlighted that pregnant women have particularities in their emotional states (Miller, 2019), with it being essential to investigate symptoms of different mental disorders, in order to avoid unfavorable outcomes, such as perinatal and/or postpartum depression (Ferrari, 2018).
Also concerning the reliability properties, the extent to which the EDAE-21 would be effective to investigate the same constructs over a period, that is, its temporal stability, was also verified. The literature is consistent in highlighting that intraclass correlation coefficient measurements should not be performed with a prolonged reapplication interval, as this favors the acquisition of new learning and causes an underestimation of stability, influencing the interpretation of the test-retest reliability. Likewise, short test-retest intervals can cause an overestimation of stability, as the short period can cause the individual to remember what they previously answered and merely reproduce their previous answers (Martins, 2006). The same author stated that, in case of doubt, it is preferable to opt for a longer time interval, even if at risk of underestimating stability, in which case the researcher may conclude that further studies need to be carried out for a given measure. In the case of overvaluation, the researcher will have a false security regarding these stability results.
In this study, the mean time of re-application between the test and retest was 8.5 weeks, which certainly ruled out the possibility of overestimation of stability. Statistical analysis, by calculating the intraclass correlation coefficient, showed indexes above 0.75 for the three constructs (anxiety, stress, and depression). Therefore, the instrument proved to be stable, which suggests a control of the sampling error of time for the EDAE-21. These results are in agreement with the literature, which found values above 0.80 for the three subscales with a sample of Brazilian adolescents (Silva et al., 2016).
The results also showed that the EDAE-21 has convergent validity in relation to the STAI-S (r = 0.615) and STAI-T (r = 0.600), BDI-II (r = 0.854), and PSS (r = 0.731) in a sample of Brazilian pregnant women, corroborating data from a DASS-21 validation study for the Brazilian adult population (Vignola & Tucci, 2014). Among the psychometric properties studied, the authors performed the verification of convergent validity between the DASS-21 and the Beck Anxiety Inventory (BAI), the Beck Depression Inventory (BDI), and the Inventory of Stress Symptom for adults (ISSL), finding the values 0.80, 0.86, and 0.74, respectively (Vignola & Tucci, 2014). Regarding the population of pregnant women, Castanheira et al. (2017) pointed out that the EDAE-21 shows that emotional indicators significantly correlate with sociodemographic variables and marital satisfaction.
Through the results, it can be noticed that the depression construct, assessed by the BDI-II, is the closest to the literature found regarding the convergent validity of the EDAE-21. The same cannot be affirmed for the other instruments since Vignola and Tucci (2014) performed the convergence analysis with different instruments for the assessment of anxiety and stress. While they assess the same constructs and have theoretically similar concepts, the instruments address the themes of anxiety and stress differently.
Investigating the convergent validity of an instrument is essential, as it demonstrates which instruments could be replaced by others that are psychometrically equivalent but have specific advantages. Regarding the population of pregnant women, using the EDAE-21, it is important to have respect for the patient at such a delicate moment in her follow-up in regard to the time spent in the evaluation. Accordingly, during the evaluation, there is an opportunity to listen and give advice about other needs indicated by the participants, since most pregnant women demonstrate a noticeable preference for being listened to and welcomed, rather than answering questionnaires. Some studies highlight that the EDAE-21 can measure the three constructs it proposes to without the need for complementary instruments. The DASS-21 itself was reduced from 42 to 21 items, considering the optimization of the time of its application (Vignola & Tucci, 2014).
Furthermore, the use of the EDAE-21 allows not only the optimization of the assessment time but also the time spent by providers to carry out the diagnosis of clinical states of anxiety, stress, and depression, favoring treatment and specialized referral, when necessary (Vignola & Tucci, 2014), and contributing to the standardization of the emotional health support, providing more reliable guidance to the professionals involved (Gorenstein et al., 2015). It should also be noted that the reassessment of emotional indicators of anxiety, stress, and depression at different times of pregnancy facilitates the identification of variables that could compromise the emotional health of women in the long term (Roman, 2019). Under these conditions and considering that the studied scale presented adequate psychometric evidence for the population of pregnant women, we considered that the EDAE-21 represents an important alternative for the brief assessment of emotional indicators in this population. Therefore, we suggested that, when used in future studies involving Brazilian pregnant women, the scale be named the EDAE-G - with the G standing for the word gestantes, which means pregnant in Portuguese.
After verifying some of the psychometric properties of the EDAE-21 instrument, we can state that the scale presents adequate internal consistency, stability, and convergence to be used with pregnant women. By verifying the psychometric qualities of the EDAE-21 for the population of pregnant women, healthcare providers and researchers can spend more time offering a space for listening and welcoming the women at this singular moment in their lives. Furthermore, the instrument provides support by speeding up the diagnosis of possible unfavorable indicators that can lead to postpartum depression. It should also be considered that the verification of emotional indicators in pregnant women, whether at low or high risk, can and should be performed at different times of the pregnancy, with the EDAE-21 instrument being ideal for this purpose due to its easy and rapid application.
Finally, some limitations and implications of the present study should be stated, starting with the size and characteristics of the sample. Despite the satisfactory results obtained, other studies can contribute by investigating evidence of validity and reliability of the EDAE-21 with larger populations of pregnant women, as well as involving other instruments and participants from other regions of the country. Studies performing more sophisticated analyses to investigate the internal structure of the scale - through Classical Test Theory (CTT) and Item Response Theory (IRT), for example - are also recommended. It is also necessary to carry out further studies that can verify other test-retest intervals in the population of pregnant women, considering the specific emotional changes of this period.
Correspondence concerning this article should be addressed to Stefanny Aparecida Silva, Rua Peri Carlos Pael Lopes, 1895, Vival dos Ipês, Dourados, MS, Brazil. CEP 79837-218. E-mail: stefannysilva-@hotmail.com
