INTRODUCTION

Antimicrobial resistance (AMR) is a global public health threat that is intrinsically linked to the use of antimicrobial agents (AMAs). Estimates from 2019 indicate that there were approximately 4.95 million deaths worldwide associated with bacterial AMR, with 1.27 million of these deaths directly attributable to this resistance.¹ A recent study reveals that approximately three-quarters of AMA consumption occurs in the community, where Primary Health Care (PHC) professionals play a key role in prescribing.²

Although Primary Health Care (PHC) is fundamental to reducing the use of AMA, allowing direct interaction with users and their families, there is still a gap in knowledge about the impact of Infection Prevention and Control (IPC) strategies and AMA Management Measures in the Brazilian context. This study seeks to fill this gap by proposing the construction of a tool capable of measuring AMA management actions and strategies in PHC. The aim of this study was to develop and validate a precise tool that evaluates the interactional triad between IPC, AMA and PHC.

METHODS

During the months of February to April 2022, this study was dedicated to designing and validating a questionnaire for PHC professionals. The questionnaire addressed IPC strategies and AMA management measures and was evaluated by experts who analyzed the representativeness of the content areas and the relevance of the objectives.

In this study, questionnaires with consolidated scales were used to collect data from health research, providing valuable insights into clinical and prescriptive AMA practice. This has contributed to improving strategies for managing and controlling the use of these drugs. The process of developing and validating the questionnaire consisted of five distinct Steps: Step 1 - Literature Review; Step 2 - Question Development; Step 3 - Expert Validation; Step 4 - Questionnaire Finalization; Step 5 - Questionnaire Dissemination. The steps for developing and validating the questionnaire are detailed in the flowchart below.

Content validation is fundamental in the validation of psychometric instruments, guaranteeing the accuracy of the items in measuring the proposed construct, as well as their relevance and representativeness for the target population and specific context; thus, Pasquali highlights methods and criteria for this validation, emphasizing the participation of experts in assessing the clarity, relevance and comprehensiveness of the items.³

Under this theoretical methodology, a convenience method was adopted to select the experts, taking into account criteria such as professional experience and involvement with the State Coordination Offices and the Ministry of Health. Initial communication took place via an e-mail invitation letter, detailing the aspects of the study. A total of 31 experts were contacted, and those who did not respond within 7 days were excluded from the evaluation process. The experts who responded were directed to take part in the validation via a virtual form. In this way, 15 experts actively contributed to the validation of the tool.

In order to make it easier for the experts to evaluate the questions, spaces were included adjacent to each item so that they could record their evaluations, as well as specific areas for comments and suggestions, guaranteeing comprehensive feedback. The experts analyzed the instruments in isolation and context, considering criteria such as vocabulary and the instructional sequence of the domains. They were responsible for determining whether each domain was adequate, inadequate or required changes.

The Content Validity Index (CVI) was calculated to assess the experts' agreement as to the representativeness of the items in relation to the content studied. According to Coluci et al,4 this index is calculated using the following formula:

In this study, only the items considered adequate by the experts were used to calculate the Content Validity Index (CVI). To determine the overall average CVI of the instrument, all the CVIs calculated individually were added together and divided by the total number of items. A minimum index of 0.75 was established as acceptable for both the evaluation of each item and the overall evaluation of the instrument.⁵

In accordance with the rules contained in Resolution No. 466/2012, this research project was submitted to and approved by the Research Ethics Committee on March 17, 2022, under Certificate of Presentation and Ethical Appreciation No. 5.413.514. The favorable opinion of the Research Ethics Committee was obtained under CAAE: 57866222.3.1001.5343.

RESULTS

This study was conducted in five main methodological Steps, each playing a crucial role in the construction and validation of the tool for assessing the interaction between IPC, AMA and PHC.^6-7

Step 1 - Literature Review: The review was conducted by accessing the LILACS (Latin American and Caribbean Health Sciences Literature) and SciELO (Scientific Electronic Library Online) databases. The health science descriptors in Portuguese or the corresponding terms in Spanish or English used in the search were: questionnaire; infection control; antimicrobial stewardship; primary health care. It was found that there were no previous studies that simultaneously addressed the topics of IPC and AMA management in Brazilian PHC. The questionnaire was segmented and organized into “domains” (D), i.e. sets of questions that addressed the same aspect. The multiple-choice variables were structured as dichotomous variables.

Step 2 - Developing the questions: given the difficulty in finding literature on the topics, we decided to use the texts of the National Program for the Prevention and Control of Healthcare-Related Infections (PNPCIRAS 2021-2025) of the National Health Surveillance Agency (ANVISA) and the National Action Plan for the Control of Antimicrobial Resistance in the Single Health Sector 2018 (PAN-BR) as guidelines.

Thus, the first two domains of the “D” questionnaire were designed to cover the following domains, respectively: D1) Health service profile and D2) Clinical and epidemiological profile of the health service.

For the preparation of the questions related to precautionary measures contained in domain D3): Actions related to the prevention and control of infections, we used as a reference the study “The role of Primary Care in the prevention of Healthcare-Related Infections” carried out by Maria Clara Padoveze and Rosely Moralez de Figueiredo.⁸ In the same domain, but in the subdomain entitled biosafety and waste management, the questions were based on RDC No. 222 of February 28, 2018, which provides for Technical Regulations for the management of health service waste.

With regard to domain D4) Actions for managing the use of AMA; D5) Health education on IPC measures; D6) Health education focusing on managing the use of AMA, the following were used as a basis, respectively: the National Guideline for Preparing a Program for Managing the Use of Antimicrobials in Health Services, published by Anvisa (2017)⁹, o (PAN-BR)⁶ and the Centers for Disease Control and Prevention's Checklist of Basic Elements for Antibiotic Use in Outpatient Settings (CDC, 2016).¹⁰

Step 3 - Validation of Experts: there was a predominance of females (86.6%), and the majority lived in the Midwest region (60%), followed by the Southeast region (20%). Around 53.3% had a degree in nursing, while 26.6% were pharmacists. All the experts were linked to public bodies related to Primary Health Care and Health Surveillance at state level. Representatives from the General Coordination for Guaranteeing Primary Care Attributes (CGGAP), the Family Health Department (DESF), the Primary Health Care Secretariat (SAPS) and the Ministry of Health (MS), as well as members of ANVISA's Health Services Surveillance and Monitoring Management (GVIMS), participated as experts at the federal level. We present the particularities recommended by experts for each domain addressed in the questionnaire (Table 2).

The experts proposed changes to the questionnaire's vocabulary, totaling a significant 91 suggestions. These suggestions reflect the importance of using precise and appropriate language to ensure the clarity and effectiveness of the questionnaire. By making changes to the vocabulary, the experts may have sought to make the questions more comprehensible to the target audience by eliminating excessively complex or ambiguous technical terms.

All the items in domains 1, 2, 5 and 6 had a CVI of over 0.75. However, in domains 3 and 4, the CVI was lower than the reference value (0.75), and the main changes made, based on the suggestions of the expert-evaluators, were terminological adjustments, spelling corrections and text detailing. The overall average CVI was 0.74 with standard deviation (SD ± 0.29) (Table 3).

Step 4 - Finalizing the Questionnaire: a comparison was made between the initial and final versions of the document. They observed a significant reduction in the number of questions, from a total of 135 in the first version to 102 in the final version.

The average time taken to complete a questionnaire is a crucial aspect to consider in its design and application. This metric not only affects the respondent's experience, but also influences the quality and integrity of the answers. Authors state that the questionnaire is a valuable tool for collecting data in research, due to its time savings, practicality, accuracy in obtaining answers, standardization, uniformity and greater openness on the part of the participants.¹¹ In this context, it is essential to understand the estimated time needed to complete the questionnaire, thus guaranteeing the active participation of the respondents and the accuracy of the data collected. In the current survey, the average time taken to complete the questionnaire was approximately 20 minutes.

Step 5 - Dissemination of the questionnaire: this took place by sending a link and a letter containing instructions on how to fill it out to the emails of the State Health Secretariats, State Health Surveillance Centers and Health Coordinating Offices. The questionnaire is now called: “National evaluation of strategies for infection control and management of antimicrobial use in Primary Health Care”, which can be accessed in full through supplementary data.

DISCUSSION

According to experts, improving the wording of questions is essential for the quality and effectiveness of questionnaires. Removing subjective questions and ensuring objectivity eliminates personal opinions, making the questionnaire more reliable and impartial. In addition, simplifying questions makes them easier to understand and increases the accuracy of responses. Changing terms avoids ambiguities and improves interpretation, while rearranging the order of questions positively influences participants’ perception and response, organizing the questionnaire in a logical and fluid manner.

This emphasis on adjustments is essential, as well-designed questionnaires are an effective research technique, consisting of a specific set of questions presented in writing to a group of individuals, with the aim of obtaining information about their opinions. This approach offers several advantages, including the ability to reach a large number of participants, even in geographically distant areas, the flexibility to answer questions at their convenience, and minimizing the influence of the interviewer on the respondents’ responses.

However, questionnaires can produce unexpected results due to the different interpretations that respondents may have of the items. Furthermore, excessive questionnaire length may result in a low response rate.¹² Given the vast territory of Brazil, the use of online questionnaires makes it possible to reach a wide variety of health professionals in different locations throughout the country. This ensures a more comprehensive and representative sample, while eliminating the need for printing, physical distribution and manual collection of questionnaires, resulting in a significant reduction in the costs associated with the survey.¹³

An online questionnaire offers the convenience of being accessed and answered at any time and place, providing greater flexibility and facilitating participation. For this reason, the development and validation of this online questionnaire was carefully planned and executed based on the latest evidence and demands on aspects related to infection prevention and control, with a view to meeting the need for actions aimed at the rational use of AMAs and the reduction of antimicrobial resistance in PHC. Health professionals working in Primary Health Care often deal with intense workloads. In this sense, it is essential that the tools available are resolutive, of high quality and capable of giving healthcare providers greater autonomy. This would make it easier for these professionals to access innovations.¹⁴

The items in a questionnaire should be designed to assess the desired results, and their psychometric properties should be evaluated for construct validity, internal consistency, reliability and other relevant aspects. The internal consistency of the instrument's items indicates how well they relate to each other and similarly represent the construct that the instrument aims to measure. On the other hand, reliability is measured by the squared correlation between the true score and the observed score, reflecting the stability and precision of the measures obtained.¹⁵

The antibiotic stewardship program assessment tool consists of the Core Elements of Hospital Antibiotic Stewardship Programs. It offers examples of the implementation of these elements, aimed at optimizing antibiotic prescribing, and can be applied according to the need or feasibility of each healthcare institution. Periodic use of the assessment tool provides data that is collected instantly and can be compiled and analyzed quickly and efficiently, making it possible to document the infrastructure and activities of the ongoing program and helping to identify areas for improvement. It is suggested to include specific details, such as points of contact or particular guidelines with dates, in the “comments” column, in order to provide useful references for the team responsible for antibiotic stewardship.¹⁶

Content validity is essential to ensure that the elements of the measuring instrument adequately represent the concept to be assessed. Therefore, when constructing a questionnaire, it is essential to address all relevant aspects of the phenomenon in question. The authors suggest guidelines for content validation, such as defining the scope of the questionnaire, involving experts in generating, evaluating and correcting the content, and using additional analyses to improve the instrument.¹⁷

It is commonly stated that ensuring the quality of health services on a global level requires a uniformly high approach, maintaining consistent standards. The first step to achieving this uniformity is to understand the contextual and cultural determinants specific to different countries and, from there, develop strategies to deal with these nuances. Decision-making in health is an intricate process, deeply influenced by context, which encompasses multiple participants and actions. This complexity is especially visible in decision-making related to antibiotic use, where different priorities and contextual factors influence clinical behavior.¹⁸

Content validity is the extent to which the content of an assessment instrument adequately reflects the construct being measured. As there is no specific statistical test to assess content validity, a qualitative approach is generally used, involving evaluation by a committee of experts. Subsequently, a quantitative approach can be carried out using the CVI.¹⁹ In this study, validation with a group of experts was essential, generating valuable comments that guided the revision of the questionnaires, this Step served to improve the questions, ensuring greater understanding.

In relation to the CVI, the score found ranged from 0.23 to 1.00, while the overall average was 0.74 (SD ± 0.29), in another study²⁰ the score given by the experts ranged from 0.777 to 1.00, with a mean of 0.902 (SD ± 0.076).

Thus, this study resulted in the creation of an accurate tool for assessing the interaction between IPC, AMA and PHC, with the aim of continuously improving the quality of health services, suggesting that the instruments assessed in this study showed a more consistent and stronger correspondence with the construct being measured, compared to the overall average found.

The changes implemented were a result of the feedback received, especially in relation to questions that proved difficult to interpret. Improving the wording of questions is essential for the quality and effectiveness of questionnaires. Excluding subjective questions and ensuring objectivity eliminates personal opinions, making the questionnaire more reliable and impartial. Simplifying questions makes them easier to understand, increasing the accuracy of responses. Changing terms avoids ambiguity and improves interpretation, while rearranging the order of questions positively influences the participants' perception and response, organizing the questionnaire in a logical and fluid manner.

These improvements suggested by the evaluators demonstrate a deep understanding of the importance of precise and objective questions. Ensuring clarity in the language used and a logical sequence in the questions promotes a better experience for respondents and increases the validity of the results obtained. In short, these strategies reflect a commitment to excellence in questionnaire design, aiming to maximize efficiency and validity in data collection.

The comparison between the questionnaire in the pre- and post-validation Steps of the experts shows that the reduction in questions can be attributed to factors such as the elimination of redundant or irrelevant questions for the research objectives. This analysis simplifies the questionnaire, making it easier to answer and reducing the time required to complete it. In addition, the experts may have identified questions that are not aligned with the research focus or that do not contribute significantly to the desired data collection.

Therefore, these questions were removed to ensure that the questionnaire remained concise and targeted. In summary, the reduction from 135 to 102 questions in the final version of the questionnaire indicates a process of refinement and optimization, aimed at improving the effectiveness and usability of the research instrument. Adapting the vocabulary was essential to ensure that the questionnaire was suitable for different demographic groups, taking into account linguistic and contextual variations. Overall, the revision of the questionnaire vocabulary reflects the experts' commitment to improving the quality of the research, which allowed for greater understanding by the participants.

Although the results of this study are promising, it is important to consider some limitations, such as: variable interpretation of the questions by participants may result in inconsistent responses, affecting the reliability of the data collected. The lack of direct comparison between the questionnaires used in these two key areas of public health may undermine the external validity and generalizability of the results. Without a reference point to compare the effectiveness and accuracy of the domain-specific questionnaires, it is difficult to determine whether the instruments are adequately capturing the nuances and complexities of infection prevention and control practices, as well as AMA management. Due to time constraints, it was not feasible to conduct a second round of evaluation with the experts, and it is suggested that future studies consider two-Step validation to ensure a more complete and robust evaluation. Conducting research on AMA management and infection prevention in primary health care in Brazil is crucial to promote the rational use of AMA, prevent adverse events, and improve the quality of health care provided to patients. These studies also contribute to optimizing health resources, strengthening epidemiological surveillance, and reducing the burden of infectious diseases, in addition to strengthening the Brazilian health system and contributing to global public health.²¹

A limitation of this study is the possible variability in the responses of PHC professionals due to regional and contextual differences in clinical practice and health policies. In addition, self-assessment may introduce biases, such as social desirability, where respondents may provide answers that they consider more acceptable than true.

This study contributes significantly to clinical practice by offering a validated tool that can be used to assess and improve IPC strategies and AMA management in PHC. The tool will allow more accurate monitoring of antimicrobial use, facilitating targeted interventions to reduce AMR. Furthermore, the application of the tool can promote greater awareness among health professionals about best practices in AMA use, encouraging a more rational and evidence-based approach.

The validation presented in this work showed that the developed questionnaires can be used as accurate and reliable tools to measure the implementation of AMA management programs at the national level, and can be replicated safely and reliably. The main objective was to build a robust and reliable instrument capable of providing relevant data to improve practices in this specific context, aiming at the continuous improvement of the quality of health services.

Acknowledgments

We sincerely thank the Brazilian Coordination for the Improvement of Higher Education Personnel (CAPES) for the financial support provided under Financing Code 001, which was essential for the development and execution of this research. We would also like to express our deep gratitude to all the experts who dedicated their time and knowledge to validate the tool used in this work. Their commitment and active collaboration were essential to ensure the quality and credibility of the research instruments.

REFERENCES

1. Murray CJ, Ikuta KS, Sharara F, et al. Global burden of bacterial antimicrobial resistance in 2019: a systematic analysis. The lancet. 2022 Feb 12;399(10325):629-55. https://doi.org/10.1016/S0140-6736(21)02724-0

2. Jeffs L, McIsaac W, Zahradnik M, et al. Barriers and facilitators to the uptake of an antimicrobial stewardship program in primary care: a qualitative study. PloS one. 2020 Mar 5;15(3):e0223822. https://doi.org/10.1371/journal.pone.0223822

3. Pasquali L. Psychometrics. Revista da Escola de Enfermagem da USP. 2009;43:992-9. https://doi.org/10.1590/S0080-62342009000500002

4. Alexandre NMC, Coluci MZO. Content validity in the development and adaptation processes of measurement instruments. Ciênc saúde coletiva . 2011 Jul;16(7):3061–8. https://doi.org/10.1590/S1413-81232011000800006

5. Freire IL, Santos FR, Nascimento AC, et al. Validação de questionário para a avaliação do conhecimento de docentes e discentes de enfermagem sobre o suporte básico de vida. Revista de Enfermagem UFPE on line. 2017 Dec;11(12):4953-60. https://doi.org/10.5205/1981-8963-v11i12a23556p4953-4960-2017

6. Brasil. Gerência de Vigilância e Monitoramento em Serviços de Saúde; Gerência Geral de Tecnologia em Serviços de Saúde; Agência Nacional de Vigilância Sanitária. Programa Nacional de Prevenção e Controle de Infecções Relacionadas à Assistência à Saúde 2021 a 2025 – PNPCIRAS, 2021.https://www.gov.br/anvisa/pt-br/centraisdeconteudo/publicacoes/servicosdesaude/publicacoes/pnpciras_2021_2025.pdf

7. Brasil. Ministério da Saúde. Plano de ação nacional de prevenção e controle da resistência aos antimicrobianos no âmbito da saúde única 2018-2022 (PAN-BR). Ministério da Saúde, Secretaria de Vigilância em Saúde, Departamento de Vigilância das Doenças Transmissíveis. – Brasília: Ministério da Saúde, 2019. https://www.gov.br/anvisa/pt-br/centraisdeconteudo/publicacoes/servicosdesaude/publicacoes/DiretrizGerenciamentoAntimicrobianosANVISA2023FINAL.pdf

8. Padoveze MC, Figueiredo RM. The role of primary care in the prevention and control of healthcare associated infections. Rev esc enferm USP. 2014 Dec;48(6):1137–44. https://doi.org/10.1590/S0080-623420140000700023

9. Brasil. Gerência de Vigilância e Monitoramento em Serviços de Saúde; Gerência Geral de Tecnologia em Serviços de Saúde; Agência Nacional de Vigilância Sanitária. Diretriz Nacional para Elaboração de Programa de Gerenciamento do Uso de Antimicrobianos em Serviços de Saúde. Brasília (DF): Anvisa; 2017. https://antigo.anvisa.gov.br/documents/33852/271855/Diretriz+Nacional+para+Elabora%C3%A7%C3%A3o+de+Programa+de+Gerenciamento+do+Uso+de+Antimicrobianos+em+Servi%C3%A7os+de+Sa%C3%BAde/667979c2-7edc-411b-a7e0-49a6448880d4?version=1.0

10. Sanchez GV. Core elements of outpatient antibiotic stewardship. MMWR. Recommendations and Reports. 2016;65. https://www.cdc.gov/mmwr/volumes/65/rr/pdfs/rr6506.pdf

11. Bastos JES, Sousa JMJ, Silva PMN, et al. O Uso do Questionário como Ferramenta Metodológica: potencialidades e desafios. Revista Brazilian Journal of Implantology and Health Sciences, V 5, N 3, P 623-636, 2023. https://doi.org/10.36557/2674-8169.2023v5n3p623-636

12. Ramos DK, Ribeiro FL, Anastácio BS, et al. Elaboration of questionnaires: some contributions. RSD. 2019 Jan 1 ;8(3):e4183828. https://doi.org/10.33448/rsd-v8i3.828

13. Cohen L, Manion L, Morrison K. Research methods in education. routledge; 2002 Sep 11.https://doi.org/10.4324/9780203224342

14. Oliveira IB, Peres AM, Martins MM, et al. Ações inovadoras desenvolvidas por enfermeiras na Atenção Primária à Saúde. Revista Brasileira de Enfermagem. 2021 Aug 20;75:e20200782. https://doi.org/10.1590/0034-7167-2020-0782

15. Heo M, Kim N, Faith MS. Statistical power as a function of Cronbach alpha of instrument questionnaire items. BMC medical research methodology. 2015 Dec;15:1-9. https://doi.org/10.1186/s12874-015-0070-6

16. Center of Disease Control. Core Elements of Hospital Antibiotic Stewardship Programs. Atlanta, GA: US Department of Health and Human Services, CDC; 2019. Available at https://www.cdc.gov/antibiotic-use/core-elements/hospital.html

17. Hoss M, Ten Caten CS. Processo de Validação Interna de um Questionário em uma Survey Research sobre ISO 9001: 2000. Produto & Produção. 2010 Jun 24;11(2). https://doi.org/10.22456/1983-8026.7240

18. Charani E, Smith I, Skodvin B, et al. Investigating the cultural and contextual determinants of antimicrobial stewardship programmes across low-, middle-and high-income countries—A qualitative study. PloS one. 2019 Jan 16;14(1):e0209847. https://doi.org/10.1371/journal.pone.0209847

19. Souza AC, Alexandre NMC, Guirardello EB. Propriedades psicométricas na avaliação de instrumentos: avaliação da confiabilidade e da validade. Epidemiol Serv Saúde. 2017 Jul;26(3):649–59. https://doi.org/10.5123/S1679-49742017000300022

20. Alvim ALS, Gazzinelli A, Couto BRGM. Construction and validation of instrument to assess the quality of infection control programs. Rev Gaúcha Enferm. 2021;42:e20200135. https://doi.org/10.1590/1983-1447.2021.20200135

21. Silva-Brandao RR, Oliveira SM, Correa JS, Zago LF, Fracolli LA, Padoveze MC, Currea GC. Coping with in-locus factors and systemic contradictions affecting antibiotic prescription and dispensing practices in primary care–A qualitative One Health study in Brazil. PLoS One. 2023 Jan 20;18(1):e0280575. https://doi.org/10.1371/journal.pone.0280575