INTRODUCTION

Antimicrobials are considered to be the second most consumed class of drugs in hospital settings, accounting for around 20% to 50% of hospital spending on drugs.¹ These drugs have revolutionized healthcare by enabling the treatment of serious, life-threatening infections. The COVID-19 pandemic has worsened the already existing global antimicrobial resistance crisis by increasing the use of antimicrobials to treat the disease, in addition to aggravating the lack of adequate management in infection control practices in health care facilities due to overcrowding of health care facilities and extensive use of these drugs to treat secondary bacterial infections. Furthermore, indiscriminate use has also been identified as one of the main causes of the development of antimicrobial resistance.^2-4

Antimicrobial resistance is a natural genetic phenomenon of microorganisms, in which genes encode proteins capable of protecting microorganisms from the action of antimicrobials, rendering them ineffective.⁵ Thus, available antimicrobials are no longer effective and threaten the lives of patients who are susceptible to bacterial infections, such as transplant patients, those who are undergoing chemotherapy, and those who are in ICUs. Antimicrobial resistance thus favors an imbalance in the economy by reducing the productivity of individuals and increasing health costs. In this scenario, a study indicates that antimicrobial resistance will cause 10 million deaths per year on a global scale by 2050 without effective actions to control this phenomenon.⁶

This has challenged the scientific community to seek strategies to combat antimicrobial resistance. The concept of the Antimicrobial Stewardship Program (ASP) was first adopted in the USA in 1997 and refers to a program aimed at preventing antimicrobial resistance in hospitals through coordinated interventions to improve and measure the use of antimicrobials, promoting the optimization of therapy, cost reduction, and patient safety.⁷

In this context, clinical guidelines from countries such as Brazil, the United States, Australia, and France advocate for the participation of pharmacists in the ASP due to the important role they play in the program's activities.^8-10 An American study found that the ASP contributed to better outcomes in the treatment of hospitalized patients, minimized antimicrobial resistance, and reduced healthcare costs.¹⁰ Another study, also conducted in the USA, showed a significant reduction in the consumption of fluoroquinolones, clindamycin, and ampicillin/sulbactam after the implementation of the ASP.^9,10 In both studies, pharmacists were actively involved in implementing and carrying out the actions inherent in the ASP. Despite this, barriers to pharmacists' work have been noted in different studies, limiting the benefits of the ASP in countries such as the USA, Australia, France, and Nigeria. In Nigeria, the barriers include a lack of training in ASP and insufficient support from hospital administrators. In France and Australia, lack of time and the significant volume of non-clinical activities are considered obstacles. In the United States, the lack of standardized treatment guidelines for infections is also a barrier.^9-11 In Brazil, there is a clear need to improve the elements of the ASP and define the responsibilities of the actions to optimize the management of the program. However, the factors that hinder the consolidation of the ASP have not yet been fully identified from the pharmacists' perspective, and there is a need to update the understanding of the facilitators and barriers affecting decision-making.¹²

Considering the unquestionable importance of the ASP, the regional differences of the program around the world, and the leading role of the pharmacist in the program's operational team, it is essential to better understand the reality of the ASP in Brazil and identify the reasons that interfere with its implementation in Brazilian hospitals from the perspective of these professionals. However, after a systematic search in the literature, no validated instruments were found to measure the perspective of pharmacists in relation to the ASP in hospitals and identify barriers to its consolidation. Therefore, the construction and validation of a data collection tool became necessary.

Questionnaires are self-administered tools in which respondents read the questions and provide written answers.¹³ They are considered an integral part of clinical practice, with growing interest from researchers due to their extensive applicability and robust scientific results. They also influence the formulation of health programs and institutional policies.¹⁴ However, it is known that there are significant differences in people's abilities to read, write, and comprehend, which can limit the application of questionnaires in research. It is worth noting that the ability to understand is not always associated only with specific groups such as children and the elderly, as functional illiteracy affects people from different social classes, age groups, and educational levels.^13,24 Thus, researchers emphasize that assessment tools only have the validity and reliability to produce accurate results if they are validated using appropriate methodologies that avoid language barriers in written communication and content flaws.^14,15

Therefore, this study aimed to develop and validate a tool to evaluate the Antimicrobial Stewardship Program and the barriers to its implementation in Brazilian hospitals from the perspective of pharmacists.

METHODS

Type of Survey

This is a methodological study. This study design is constantly applied in the development of new tools and consists of three stages: 1) Development, production and construction of tools; 2) Validation of tools; 3) Application of tools.¹³

Development, production, and construction of the data collection tool

At the beginning of the study, a literature search was conducted to determine if a validated tool for assessing pharmacists' perspectives on Antimicrobial Stewardship and the barriers to its implementation in Brazilian hospitals existed. The search was performed in June 2021 across the PubMed, LILACS, and SciELO databases using the Health Sciences Descriptors (DeCS) "Antimicrobial Stewardship," "Drug Resistance, Microbial," and "Pharmacists," combined with the Boolean operator AND. Grey literature was also searched using Google Scholar®. The searches included studies published within the five years preceding the start of this research (October 2016 to June 2021) to ensure that the information was up-to-date and aligned with the guidelines for combating antimicrobial resistance from the World Health Assembly in Geneva, which took place in 2015. As a result, the references found did not identify a tool to evaluate the ASP and the barriers to its implementation, leading to the decision to develop a tool for data collection.

The tool was developed based on the literature, the list of essential elements of the Antimicrobial Stewardship Program,¹⁶ and the researchers' clinical experience. In developing the tool, as recommended in the literature, clear and understandable language appropriate to the target audience and the research objectives was used, along with technical terms suited to the study population's knowledge level.¹⁷ Furthermore, during the development of the tool, the formatting style, title, filling instructions, domain measured, and scores were considered when defining the structure and sequence of the items, in order to make them less exhaustive and more interesting, applying a logical sequence, which increased the specificity of the tool.¹⁴ In order to facilitate availability and access, the tool was built using the Google Forms® survey manager.

The tool contained multiple-choice questions, short-answer questions, dichotomous questions, and questions with Likert-type measurement scales. The Likert-type scale is widely used to measure attitudes, skills, and qualities. It is simple, ordinal, and was expressed in the tool to indicate the degree of confidence in developing skills and qualities in performing activities, the degree of impact on barriers perceived to the participation in program management, and the degree of pharmacist’s participation in the program according to the statements in the header.¹³

It was decided to remove the central alternatives, i.e., neutral points such as "neither agree nor disagree", because they represent a possible lack of opinion and make it difficult for respondents to understand. This practice has also been supported by some authors due to ambiguous interpretation by researchers.¹⁸

Data collection tool validation

Once built, following the recommendations of the literature, the tool was subjected to a validation process encompassing content validity (to check the suitability of the items in relation to the domains of the construct) and semantic validation (to check comprehensibility by the population under study).^14,15,18,19

Content validation

The validation of content measures the extent to which the data collection tool achieves its intended purpose, and involves quantitative and qualitative aspects.¹⁸It can also be understood as an assessment.¹⁴ For the selection of evaluators, some selection criteria are suggested, including clinical experience in the area of the tool, research and publications on the subject, mastery of the concepts involved and methodological knowledge about the construction of tools. In addition, when selecting evaluators, it is recommended to analyze the characteristics of the tool in order to direct it to those who have the appropriate knowledge to evaluate it, as well as checking the training, professional qualifications, and availability of those who will take part in this stage of the validation.¹⁸ As for the number of evaluators, it is recommended that validation be carried out by a minimum of three and a maximum of 10 experts. As a result, seven expert evaluators with the following inclusion criteria were invited: areas of pharmaceutical assistance, hospital pharmacy, clinical pharmacy and Hospital Infection Control Committee (HICC) to form a committee to assess the tool's characteristics. An intentional non-probabilistic sample was used in accordance with the eligibility criteria described above.¹⁸

In the first phase, an e-mail was sent to the seven experts with the invitation letter in Portable Document Format (PDF), outlining the objective and justification for the study. The body of the e-mail contained a brief explanation of the study and a link to the evaluation tool in Google Forms® format. It is worth noting that only after accepting participation by signing an Informed Consent Form (ICF) the evaluators were given access to the evaluation itself. The response time was set at seven days.

The data collection tool was sent to the evaluators for evaluation, accompanied by a brief introduction to the study, an ICF, and instructions for evaluating the tool. A reminder was sent via WhatsApp® the following day. The criteria used to evaluate the attributes of each item in the tool were representativeness, clarity, objectivity, precision, and relevance. The scores were calculated using a Likert scale.

Additionally, a space was provided for the experts to suggest improvements. A general evaluation of the data collection tool was available at the end of the document to assess the comprehensiveness of the tool, i.e., to analyze the need to include or exclude any items.

The items were analyzed by calculating the Validity and Content Index (VCI), which measures the percentage of evaluators who agree with certain aspects and items in the tool.^17,18 This analysis consists of applying a four-point Likert scale. There are several options for the four-point Likert scale found in the literature and they depend on the objectives of each study. In this study, the following scale was adopted: 1 = strongly agree, 2 = agree, 3 = disagree and 4 = strongly disagree.¹⁸

At first, the IVC at item level (IVC-I) was used with a cut-off point equal to 1.0 or 100% for individual item evaluation (modifications and deletions of items) because it was a new tool. The choice of IVC-I was also justified by the number of evaluators (n=4), since the literature recommends IVC-I equal to 1 when there are five or fewer evaluators. The IVC was also calculated at scale level according to the mean (IVC/AVE) to assess the attributes representativeness, clarity, objectivity, precision and relevance, with a cut-off standard of 0.90.^13,14,18,19

Semantic validation

Semantic validation was conducted to verify the suitability of the data collection tool for the study population, with the inclusion criteria being Brazilian hospital pharmacists. In this process, recommendations for developing technological products were followed. According to the literature, 6 to 20 participants are sufficient. However, considering the possibility of losses or refusals, 22 pharmacists were invited using the snowball sampling technique. This technique is frequently used in virtual research, where the researcher sends an invitation to their contacts, who then recommend new potential participants, and so on, until the minimum number of participants required by the literature is reached.²⁰

The main purpose of the data collection tool, made available to the participants by email, was to assess the suitability of the material, analyze the degree of understanding and the need for modifications. The tool was evaluated only once after the semantic validation participant agreed to the ICF. Preliminary information was also provided on how to complete the document correctly, and the deadline for returning the completed form was limited to 15 days. Reminders were sent to participants every five days by the researchers.

The Suitability Assessment of Materials (SAM) checklist, already validated for use in Brazil, was used to measure the results of the pharmacists' analysis at this stage. This checklist is made up of six sections (content, text comprehension, illustration, presentation, motivation and cultural adaptation) totaling 22 factors with a scoring scale of 0 to 2, where 2 = excellent, 1 = adequate, 0 = not adequate and N/A if the factor cannot be assessed, totaling up to 44 points. When interpreting the results to check the suitability of the material for the target audience in percentages, the SAM is classified as superior when the result is 100%, adequate when it is 80% to 99.9% and inadequate or not acceptable when it is less than 80%_.^15,21

Statistical analysis

In addition to calculating IVC-I, IVC/AVE and SAM, the profile of the participants in the validation processes was analyzed using descriptive statistics. Categorical quantitative variables were analyzed using absolute frequency, relative frequency and median.

The research was conducted in accordance with the required ethical standards (MS Resolutions 466/2012 - 510/2016 - 580/2018) and the studywas approved on February 11, 2022 by the Ethics Committee of the Federal University of Alfenas (UNIFAL-MG) under protocol number 5.239.322 and CAAE 52933621.2.0000.5142.

RESULTS

The data collection tool in its final version contained 62 questions (Figure 1) organized into six sections: i) Demographic data; ii) Workplace; iii) Confidence in the performance of activities; iv) Barriers perceived to participation in program management; v) Essential elements of the program; and vi) Pharmacist participation in the ASP (Supplementary Material A).

Data Collection Tool Validation

Content Validation

After the deadline set for sending the content validation evaluations had passed, a response was received. As a result, a new contact was made by sending an email to seven experts, followed by a WhatsApp® message the following day to confirm the email had been received. A reminder email was also sent on the fifth day, and three more responses were received. As a result, a total of four evaluators took part in this validation stage.

After evaluation by the evaluators, the subjects were characterized, and the IVC-I and IVC-S/Ave were calculated. Modifications were made and the tool was revised so that it contained 67 items and was sent for a second evaluation. After this new evaluation, the IVC-I and IVC-S/Ave were calculated again according to tables 1 and 2. Based on the suggestions of the evaluators, further changes were made. The resulting version was sent for semantic validation. All the changes are described in Supplementary Material B.

The evaluators were mostly men (75%, N=3) with a mean age of 36 years (IQR=12.5; Max=46 years; Min=32 years). It was also found that 3 of the evaluators had master’s degrees in a variety of fields but related to antibiotic therapy. All had more than six years of training, and the majority (75%, N=3) worked in public hospitals.

Semantic validation

After the deadline set for sending the semantic validation evaluations, the tool was sent to 22 pharmacists, since the literature recommended six to 20 participants. Seven responses were obtained. As the number of respondents was in line with the literature, the SAM calculation was carried out (Table 3).

The participants in the semantic validation were mostly women (N=6; 85.7%) with a mean age of 39 years (IQR=11, Max=45 years; Min=30 years). It was also found that all of them worked in hospital pharmacies and the majority (N=4; 57.2%) in private hospitals.

DISCUSSION

When building and validating a data collection tool in the health sector, it contributes to both clinical and scientific practice, as these tools are relevant in the formulation of health programs and public policies.¹⁴ However, it is important to recognize that the development and validation of new tools are complex tasks that necessitate the consideration of cultural, economic, technological, and educational factors that are appropriate for the target audience and the country in which the tools are intended to be used.^14,22

Technological products, such as data collection tools, are only valid if they are capable of accurately assessing their intended objective rather than an unrelated construct. Additionally, the process of validating technological products is a form of psychometrics that has been adapted to meet the need to validate other types of products, with an emphasis on their content. However, changes in the study population and the mode of application can influence the psychometric properties, making it necessary to perform a specific validation of the tool that takes these aspects into account.^14,23

Moreover, it is important to note that even though the target audience for this study consists of individuals with university degrees, this does not exclude the possibility that some may experience functional illiteracy— the inability to interpret and understand texts, ideas, and perform simple mathematical calculations, despite being able to read. This issue affects individuals regardless of their education level or socioeconomic status. According to the Functional Literacy Indicator (INAF), approximately 29% of the Brazilian population is affected by this problem, which justifies the semantic validation of the tool developed by the researchers.²⁴

The IVC-I and IVC/Ave obtained attested to the validity of the content which proved to be relevant in measuring complex psychosocial traits, and the SAM attested to the semantic suitability in relation to the target audience.^13,15 In addition, it was observed that the use of WhatsApp® throughout the validation stages positively helped to clarify any doubts the participants had in real time and helped to bring them closer to the researchers. ²⁵ In addition, it was found that the number of participants was positively influenced by WhatsApp® messages and reminders sent on the second contact of the content validation, thus reaching the minimum number of judges recommended in the literature for this type of validation. The use of Google Forms® platform was also noteworthy as a facilitator in the validation process, which, as well as being free, allowed the data to be exported to Microsoft Excel®.

Thus, the tool, after validation, proved to be both innovative and pertinent, as there is no known validated tool in Portuguese that is suitable for the purpose of the study and the target population. This validation makes it possible to use the tool in robust studies that seek to make significant contributions to the social, economic, and environmental aspects related to pharmacists' perceptions of Antimicrobial Stewardship Programs (ASP).¹⁴ Its use as an online data collection strategy will also allow for economy, practicality, and logistical feasibility in future studies, especially when conducting surveys at times when social contact should be avoided.

Despite these benefits observed in the use of the technology during the development of the study, communication facilities in the digital universe favored an increase in the number of surveys carried out remotely during the Covid-19 pandemic and the overload of participation in surveys, which may have contributed to the absenteeism of professionals in the semantic validation stage. Other possible factors affecting absenteeism are also attributed to this period, such as the increase in electronic fraud, which may have led to a fear of clicking on the link to fill in the data collection tool, and the work overload caused by Covid-19. It should be noted that the target audience for the semantic validation was made up of professionals working on the so-called 'front line' during the pandemic. Despite these limitations, the tool has been validated and is capable of meeting the objectives for which it was designed.

In conclusion, the data collection tool developed has been validated and is compatible with the cultural, economic, technological, and educational characteristics of the target audience. It is anticipated that this tool will contribute to future studies aimed at assessing the perspective of pharmacists on ASP in Brazilian hospitals and identifying the barriers to its implementation.

Acknowledgments

This study was carried out with the support of the Coordination for the Improvement of Higher Education Personnel - Brazil (CAPES) - Funding Code 001.

REFERENCES

1. Almeida AS, Baptista PF, Lima TM. Atuação do farmacêutico no gerenciamento de antimicrobianos no Brasil: uma revisão da literatura. Brazilian Journal Health Pharmacy [Internet]. 2022 Outb 17; 4(2):01-20. https://doi.org/10.29327/226760.4.2-1.

2. Iregbu KC, Nwajiobi-Princewill PI, Medugu N, et al. Antimicrobial Stewardship Implementation in Nigerian Hospitals: Gaps and Challenges. African Journal of Clinical and Experimental Microbiology. 2021 Jan 16; 22(1):60-6. https://www.ajol.info/index.php/ajcem/article/view/203077.

3. Organização Pan-americana da Saúde. A resistência aos antimicrobianos, fomentada pela pandemia de COVID-19. Informe de Política, Novembro de 2021. 2021. https://iris.paho.org/bitstream/handle/10665.2/55936/OPASCDEAMRCOVID19220006_por.pdf?sequence=1&isAllowed=y.

4. World Health Organization. Antimicrobial stewardship programs in health-care facilities in low- and middle-income countries: a WHO practical toolkit. Vol. 1. Genebra; 2019. https://www.who.int/publications/i/item/9789241515481.

5. Lima MP, Batista RS, Moreira TR, et al. Evaluation of antimicrobial use in the basic health units of Ponte Nova / MG. Ciência E Natura. 2022 May 18; 43, (16). https://doi.org/10.5902/2179460X43575.

6. Oliveira M, Pereira KDS, Zamberlam RC, et al. Resistência bacteriana pelo uso indiscriminado de antibióticos :uma questão de saúde pública. Rease. 2020 Nov; 6 (11). https://periodicorease.pro.br/rease/article/view/279/141.

7. Dyar OJ, Huttner B, Schouten J, et al. What is antimicrobial stewardship? Clin Microbiol Infect. 2017; 23(11):793-8. https://www.sciencedirect.com/science/article/pii/S1198743X17304895.

8. Barlam TF, Cosgrove SE, Abbo LM, et al. Executive Summary: Implementing an Antibiotic Stewardship Program: Guidelines by the Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America. Clin Infect Dis. 2016 Mai 15; 62(10):1197-1202. https://academic.oup.com/cid/article/62/10/e51/2462846.

9. Abubakar U, Tangiisuran B. Nationwide survey of pharmacists’ involvement in antimicrobial stewardship programs in Nigerian tertiary hospitals. J Glob Antimicrob Resist. 2019 Out 8; 21:148-153. https://doi.org/10.1016/j.jgar.2019.10.007.

10. Weier N, Tebano G, Thilly N, et al. Pharmacist participation in antimicrobial stewardship in Australian and French hospitals: a cross-sectional nationwide survey. J Antimicrob Chemother. 2017 Out 30; 73(3):804-813. https://pubmed.ncbi.nlm.nih.gov/29237051/.

11. Magill SS, O'Leary E, Ray SM, et al. Assessment of the Appropriateness of Antimicrobial Use in US Hospitals. JAMA Netw Open. 2020 Mai 31; 4(3). https://pubmed.ncbi.nlm.nih.gov/33734417/.

12. Menezes RM, Carneiro M, Gonçalves MRS, et al. Desenvolvimento e validação de questionário para autoavaliação dos programas de gerenciamento de antimicrobianos em unidade de terapia intensiva adulto. Revista Científica Multidisciplinar Núcleo do Conhecimento. Ano. 07, Ed. 06, Vol. 05, pp. 175-215. Junho de 2022. ISSN: 2448-0959. https://www.nucleodoconhecimento.com.br/saude/questionario-para-autoavaliacao.

13. Polit DF, Beck CT. Fundamentos de Pesquisa em Enfermagem: Avaliação de evidências para a prática da enfermagem. 7º ed. Porto Alegre. Artmed. 2011. 669 p. https://pesquisa.bvsalud.org/portal/resource/pt/sms-10133.

14. Coluci MZO, Alexandre NMC, Milani D. Construção de instrumentos de medida na área da saúde. Cien Saude Colet. 2015 Marc; 20(3):925-36. https://www.scielo.br/j/csc/a/qTHcjt459YLYPM7Pt7Q7cSn/.

15. Souza CS, Turrini RNT, Poveda VB. Tradução e adaptação do instrumento “Avaliação de Adequação de Materiais” (SAM) para o português. Rev. enferm. UFPE online. 2015 Mar; 9(5):7854-61. https://repositorio.usp.br/directbitstream/3ab8a7a7-5e39-4ed5-8103-dfdfc428576f/TURRINI,%20R%20N%20T%20doc%20110e.p.

16. Agência Nacional de Vigilância Sanitária. Diretriz Nacional para Elaboração de Programa de Gerenciamento do Uso de Antimicrobianos em Serviços de Saúde. Agência Nacional de Vigilância Sanitária 2017 p. 90. https://antigo.anvisa.gov.br/documents/33852/271855/Diretriz+Nacional+para+Elabora%C3%A7%C3%A3o+de+Programa+de+Gerenciamento+do+Uso+de+Antimicrobianos+em+Servi%C3%A7os+de+Sa%C3%BAde/667979c2-7edc-411b-a7e0-49a6448880d4?version=1.0.

17. Keszei AP, Novaki M, Streiner LD. Introduction to health measurement scales. J Psychosom Res. 2010 Abr; 68(4):319-32. https://doi.org/10.1016/j.jpsychores.2010.01.006.

18. Aksah H, Ani AIC, Johar S, et al. Evaluating the Content Validity: Development of An Instrument for Measuring Functional Building Performance. International Journal of Global Optimization and Its Application. 2023 Mar.31; 2(1), 12-19. https://doi.org/10.56225/ijgoia.v2i1.161.

19. Souza GHS, Coelho JAPM, Esteves GGL, et al. Inventário de barreiras e facilitadores ao empreendedorismo: construção e validação de um instrumento. REAd Revista Eletrônica Administração (Porto Alegre). 2016 Dez ; 22(3):381-412. https://doi.org/10.1590/1413-2311.04315.57744.

20. Tanaka JRV, Sousa KHJF, Alves PJP, et al. Educational Technology on Urinary Incontinence during Pregnancy: Development and Validation of an Online course for the Brazilian Population. Aquichan. 2023 Mar; 23(1):2313. https://doi.org/10.5294/aqu i.2023.23.1.3.

21. Alvarez LD, Damiance PRM. O Suitability Assesmente of Materials (SAM) e avaliação de materiais educativos em saúde. Revista Intelecto Fema. 2020 Set; (3):13p. ISSN: 2596-0806. https://cepein.fema.edu.br/aplicacoes/extra-fema/consulta-acervo-biblioteca.

22. Meadows KA. So you want to do research? 5: Questionnaire design. Br J Community Nurs. 2013 Set 27; 8(12):562-570. https://pubmed.ncbi.nlm.nih.gov/14688664/.

23. Teixeira E. Validação e avaliação de produtos tecnológicos. 2021 Nov 13. http://retebrasil.com.br/arquivos/File/VALIDACAOAVALICAO.pdf.

24. Lourenço T. Escolas brasileiras ainda formam analfabetos funcionais. 2020 Nov 13. https://jornal.usp.br/atualidades/escolas-brasileiras-ainda-formam-analfabetos-funcionais/.

25. Pedroso GG, Ferreira ACVV, Silva CC, et al. Coleta de dados para pesquisa quantitativa online na pandemia da COVID-19: relato de experiência. Rev. Enferm. UFSM. 2022 abr. 7; 12(13):1-14. https://periodicos.ufsm.br/reufsm/article/view/67023.

Información adicional

redalyc-journal-id: 5704