Development of core competency evaluation index system for informatics nurses in China: a Delphi study

  1. http://orcid.org/0000-0002-2465-7117Xiajing Lou1,
  2. Mengxin Wang2,
  3. http://orcid.org/0000-0002-9391-2345Shihua Cao1,
  4. Qiong Zhang3,
  5. Jiani Yao1,
  6. Yankai Shi1,
  7. Lingling Cheng3,
  8. Xiaowei Xu4,
  9. Li Ning5,
  10. Chunling Yang6,
  11. Tingqi Shi7,
  12. Shuyuan Wang8
  1. 1Hangzhou Normal University, Hangzhou, Zhejiang, China
  2. 2The Second Affiliated Hospital of Wannan Medical College, Wuhu, Anhui, China
  3. 3Zhejiang Provincial People’s Hospital, Hangzhou, Zhejiang, China
  4. 4Zhejiang University School of Medicine First Affiliated Hospital, Hangzhou, Zhejiang, China
  5. 5Hangzhou First People’s Hospital, Hangzhou, Zhejiang, China
  6. 6Liaocheng People’s Hospital, Liaocheng, Shandong, China
  7. 7Nanjing Drum Tower Hospital, Nanjing, Jiangsu, China
  8. 8Liaoning University of Traditional Chinese Medicine Affiliated Hospital, Shenyang, Liaoning, China
  1. Correspondence to Professor Shihua Cao; csh{at}hznu.edu.cn

Abstract

Objective To develop a scientific and systematic core competency evaluation index system for informatics nurses in China. The goal is to support their training, assessment and performance evaluation.

Methods An initial set of evaluation indicators was created based on a review of existing literature and semi-structured interviews. From September to December 2023, the indicators were refined through two rounds of Delphi expert consultation. A total of 21 experts from 11 provinces in China completed the questionnaires.

Results The response rate for both rounds of questionnaire surveys was 100.0%. The expert authority coefficient was 0.934. The coordination coefficient among experts was statistically significant (p<0.05). After analysing expert feedback, the final index system included six first-, 17 second- and 69 third-level indicators. Each indicator was assigned a weight. The six first-level indicators and their weights were as follows: theoretical and practical skills (0.303), communication and coordination skills (0.152), professional development capabilities (0.210), critical thinking (0.050), nursing management skills (0.210) and occupational humanistic characteristics (0.075).

Conclusions This core competency evaluation index system provides a foundation for the training and assessment of informatics nurses in China. It can help develop skilled informatics nurses who improve care processes, enhance efficiency and support better patient outcomes.

  • Delphi Technique
  • EDUCATION & TRAINING (see Medical Education & Training)
  • Nurses
  • Nursing Care

Data availability statement

Data are available in a public, open access repository. All data relevant to the study are included in the article or uploaded as online supplemental information.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

STRENGTHS AND LIMITATIONS OF THIS STUDY

  • This study employed a rigorous Delphi method to establish the core competency evaluation index system for informatics nurses in China.

  • Only the nurse group is selected as the communications expert, which has certain limitations and will be continuously revised and adjusted in future trials.

  • Despite high expert authority coefficients, the generalisability of the results might be constrained by the specific Chinese medical context and practices.

Introduction

The Outline of China Nursing Career Development Plan (2021–2025) underscores the pivotal role of nursing informatisation in advancing the nursing industry.1 Nursing informatisation refers to the extensive application of modern information technology in the field of clinical nursing, effective development and utilisation of information resources, construction of advanced information infrastructure and informatisation development of traditional nursing work models to continuously improve the level of comprehensive nursing and accelerate modernisation.2 At present, nursing informatisation has become a key factor in the growth and depth of the nursing discipline.

China began integrating nursing and information technology in the 1980s.3 Since 2002, mobile systems for monitoring vital signs have spread rapidly. Other systems followed, including those for managing medical orders, drug delivery, nursing supplies and quality control.4 Although mobile nursing systems have brought many benefits, they also present challenges. Problems such as poor system integration, limited data extraction and repeated data entry can disrupt nursing workflows and threaten patient safety.5 In such cases, software engineers are expected to solve technical issues. However, engineers often lack clinical experience, whereas nurses may not understand how systems are developed. This gap makes communication difficult, and the resulting systems may not meet clinical needs. Informatics nurses (INs) serve as intermediaries between clinical staff and engineers. They play a key role in supporting nursing informatisation. Training INs has become a new model of workforce development and a growing focus among nursing experts in China.6 7

Since 1995, the American Nurses Association (ANA) has provided centralised management of INs. In the United States, training programmes and clinical practice systems for INs are now well-established.8 The American Medical Informatics Association estimates that up to 70 000 INs or specialists will be needed in the next 5 years.9 This points to strong career potential in the field. In China, many hospitals have created nursing informatics teams. These teams gather feedback from clinical staff and help address system challenges. They work to improve nursing informatisation and contribute to the design of better information systems.10–12 However, structured training is still lacking on the core competencies required for INs in China. Research on job-specific competencies in this role is also limited.13 14

This study aimed to develop a core competency evaluation index system for INs. The framework is based on the ‘theoretical of core competence’15 and ‘role theory’.16 It will serve as a practical tool for nursing managers to evaluate, train and assess INs in clinical settings.

Methods

Research group establishment

This research project’s team was led by a director of the nursing department of a class A tertiary hospital and a nursing informatics expert from a university. They have been engaged in nursing information and nursing management for more than 10 years and are well versed in the Delphi method. The members of the research team have scientific research experience, and >87.5% of the members are postgraduates, ensuring the scientific rigour of the research process.

Index system construction

A comprehensive literature search was conducted using Web of Science, PubMed, Medline, Springer Link, China National Knowledge Infrastructure, VIP (VIP Database for Chinese TechnicalPeriodicals) and Wanfang Data, covering all records up to January 2023. In addition, official websites such as the ANA,17 American Nursing Informatics Association,18 American Nurses Credentialing Centre19 and Healthcare Information and Management Systems Society20 were reviewed to gather information on current developments in nursing informatics both in China and internationally. The main retrieval terms included the following: ‘Nursing informatics’, ‘informatics nurse’, ‘Nursing informatics competencies’, ‘Informatics nursing certification’, ‘Core competence’, ‘Evaluation index’ and ‘Evaluation method’. Searches were limited to Chinese and English publications. In this study, the ‘snowballing approach’ was used to effectively search the data related to this study and read 289 retrieved literature.

Through purposive sampling, two members of the subject group conducted one-to-one semi-structured interviews with five INs and nine nursing informatics specialists for about 40 min. Both interviewers underwent training related to semi-structured interviews, had a thorough understanding of the survey content, survey instruments and interview techniques, clarified data collection and organisation methods and conducted simulations before the formal interviews. The sample size of the interview questions was based on the repeated data of the interviewees, and no new topics were discussed.

Based on the theory of core competence and role theory, a preliminary framework of competency indicators was developed. It included five primary indicators, 17 secondary indicators (theoretical and practical skills, interpersonal skills, specialty development capabilities, critical thinking and nursing management skills) and 81 tertiary indicators. These indicators were derived from the interview data and literature review.

Preparation of the expert correspondence questionnaire

The questionnaire consisted of three parts. The first part was a brief introduction and description of the topic. In the second part, experts were invited to evaluate the text content of the indicators of the questionnaire. In this part, a 5-level Likert scale was used to evaluate the indicators, and a content feedback column was set up to better revise the first draft. Here, 5, 4, 3, 2 and 1 point means very important, important, neutral, unimportant and very unimportant, respectively. Experts could put forward their views in the corresponding position of the questionnaire. The third part involved obtaining information from experts, including their age, workplace, professional title, research field, working years, judgement basis for indicator importance (Ca) and familiarity with indicator content (Cs).

Delphi expert consultation

The Delphi method is a qualitative research approach used to reach consensus through expert opinion on a real-world problem.21 The Delphi method employs an anonymous consultation process to ensure participants can express their views freely. When certain indicators generate disagreement among experts, the research team systematically analyses both scores and comments and makes a decision after joint discussion. In subsequent consultation rounds, experts review the disputed indicators along with summarised group feedback. The process repeats iteratively. Collect opinions and feedback several times until a consensus is reached.22 In this study, consensus was established based on two selection parameters: Kendall’s coordination coefficient W (Kendall’s W) and coefficient of variation (CV), with p<0.05 as the significant criterion.23

Selection of experts

The number of Delphi consultants should be controlled at 15–30 to avoid homogeneity of research objects.24 The inclusion criteria were as follows: (1) title of intermediate or above; (2) from class A tertiary hospital or university; (3) engaged in medical informatisation, nursing informatisation, nursing management, nursing education, clinical nursing and other related fields for more than 10 years, which means experts have accumulated a solid theoretical foundation and practical experience in the field and will avoid one-sided views caused by lack of experience25 ; and (4) interest in this topic and willingness to participate. The exclusion criteria were as follows: (1) failing to complete the questionnaire according to the content of the questionnaire and (2) failing to fill in and return the questionnaire within the specified time.

Implementation of expert consultation

The questionnaire was sent to experts by email, along with materials about the research background, aim of the study, demographic information collection form, instructions of scoring criteria and descriptions of the indicators, and reminded them to reply within 2 weeks. After the first round, the research team analysed the responses. Based on expert feedback, indicator items were revised through group discussion. A second-round questionnaire was then prepared and sent to the same group of experts. The questionnaire was sent to the same experts with the first-round survey by email, accompanied by a graph-based report detailing the results from the first round. According to the score of each indicator, the mean value, full score rate and CV in its importance assignment were calculated . The elements could be included when the CV was <0.25, and the mean value of importance assignment was >3.50.26 CV ≤25% was considered as the screening criteria, and at least 75% consistency was noted among experts.

Based on the second-round results, the team further revised the indicators by adding, merging or refining items. This process led to the final version of the core competency evaluation index system for INs.

Statistical methods

Data collection was conducted by one author (XL) between September and December 2023. IBM SPSS Statistics version 26.0 was used for descriptive statistical analysis of the data. Continuous data were expressed as means ± SD and categorical data as frequencies and percentages.27 The degree of experts’ activeness was expressed using the effective return rate of the questionnaire. Expert authority was assessed on the basis of judgement and experts’ familiarity with the questions, and Kendall’s coefficient of concordance was used to represent the degree of expert opinion coordination.28 The weight of each index was determined using the analytic hierarchy process (AHP). First, a hierarchical structure model was built based on the composition of the indicators. Then, expert scores were used to construct a judgement matrix using the Saaty scale. The final weight of each indicator was determined through hierarchical ranking and a consistency check. The level of significance for this test was set at α=0.05.29

Patient and public involvement

Patients and the public were not involved.

Results

General expert information

A total of 21 experts participating in the Delphi survey were from 21 class A tertiary hospitals spanning 11 provinces and municipalities in China. The Delphi expert panel demographics are listed in table 1.

Table 1

General information of experts (n=21)

Degree of expert activeness

In both rounds of the Delphi consultation, 21 questionnaires were distributed, and 21 valid responses were received in each round, resulting in an effective return rate of 100%. The consistent response rate and the number of suggestions submitted by experts reflect a high level of participation and engagement from the panel.30

Authority coefficient of experts

The authority coefficient of experts (Cr) was calculated on the basis of their judgment-making ability (Ca) and familiarity with the surveyed indicators (Cs); Cr was calculated by using the following formula: Cr=(Ca+Cs)/2.

In this study, the Ca, Cs and mean Cr were 0.962, 0.905 and 0.934, respectively. This indicates that the experts involved had a high degree of authority and credibility in the field.

Coordination degree of expert opinions

The coordination degree of expert opinions was expressed using the CV and coordination coefficient. The CV values ranged from 0.05 to 0.40 in the first round and 0.08 to 0.22 in the second round of consultation.

The coordination coefficients (W) ranged from 0.203 to 0.309 in the first round and 0.312 to 0.377 in the second round. For all first-, second- and third-level indicators, the p-values were <0.05, indicating statistically significant consensus among the experts. These results suggest a strong level of coordination in expert opinions. Detailed values are provided in table 2.

Table 2

Coordination coefficient of expert opinion

Results of expert consultation

Delphi round 1

In the first round of the Delphi consultation, unqualified indicators were removed based on the predefined screening criteria. Experts emphasised that communication and coordination abilities are more essential, leading to the revision of the first-level indicator from ‘interpersonal skills’ to ‘communication and coordination skills’. Additionally, several experts recommended including the professional attitude of INs as part of core competencies. As a result, a new first-level indicator, ‘occupational humanistic characteristics’, was added. Experts also noted that ‘professional knowledge and skills’ are fundamental competencies for INs and that nursing informatics specialisation should be more explicitly reflected. Consequently, the second-level indicators ‘specialised theoretical knowledge’ and ‘specialist practical skills’ were revised to ‘nursing informatics theoretical knowledge’ and ‘nursing informatics practical skills’ (online supplemental appendix 1). All revisions were made following a detailed review and discussion by the research team based on expert feedback. The updated indicator set was then used to initiate round 2 of the Delphi consultation.

Supplemental material

Delphi round 2

In the second round, the revised first- and second-level indicators were unanimously approved by the 21 experts and did not require further modifications. Experts suggested that the third-level indicator ‘master computer network technology related content’ should be revised to ‘familiar with computer network technology related content’. ‘Master the characteristics of different types of medical institutions’ should be revised to ‘master the application characteristics of nursing information technology in different departments’. ‘Master nursing professional knowledge such as medical nursing knowledge, surgical nursing knowledge and basic ‘nursing knowledge’ should be revised to ‘master nursing professional knowledge such as medical nursing knowledge, surgical nursing knowledge and basic nursing knowledge and be familiar with the medical and surgical workflow and its differences’. ‘Master the skills of clinical nursing operations’ should be revised to ‘familiar with all kinds of clinical nursing operation skills and procedures’. INs use nursing information systems to provide patients with better quality and safe nursing services. After the second round of expert consultation, we found that the CV values of the first-, second- and third-level indicators were all <0.2, the Kendall coefficient was significantly improved, and there were fewer new comments from the experts in the second round of the Delphi expert consultation. All of them indicate that there is a convergence of expert opinions. After comprehensive consideration, we decided to close the expert consultation. Six first-, 17 second- and 69 third-level indicators were finally established.

Weight analysis

Using the AHP, the weight of each indicator was determined. A hierarchical structure model was built, and a judgement matrix was established to calculate the weight. In the hierarchical structure model, the target level is the index system for core competency evaluation for INs. The criterion level is composed of six first- and 17 second-level indicators. The scheme level is composed of 69 third-level indicators. After that, a judgement matrix was built. When CR <0.1, the judgement matrix is satisfactory.31 After the judgement matrix met the requirements, the weights of the indicators were calculated (online supplemental appendix 2).

Discussion

The selection of experts is a critical aspect of the Delphi method, directly influencing the reliability of research outcomes.32 In this study, the panel consisted of 21 experts, all holding at least a bachelor’s degree, with 57% having over 20 years of work experience and 71.4% holding a deputy senior title or above. Their extensive experience spans medical informatics, nursing informatics, nursing management, nursing education and clinical nursing, enabling them to provide comprehensive perspectives on the evaluation indicators. The expert consultation showed a high level of engagement, with a 100% effective return rate for both rounds of questionnaires, indicating strong enthusiasm.33 The familiarity coefficient of experts, judgement coefficient, and authority coefficient were 0.905, 0.962 and 0.934, respectively, demonstrating high expert authority and lending credibility to the consultation results. The CV ranged from 0.05 to 0.40 in the first round and from 0.08 to 0.22 in the second round, indicating that experts’ opinions on the research content have minor fluctuations. After two rounds of consultation, Kendall’s coefficients of concordance for the first-, second- and third-level indicators were 0.332, 0.377 and 0.312, respectively, all with p<0.05, reflecting a significant degree of consensus and reliable research outcomes.34

Based on the theory of core competence and role theory and building on a systematic literature review and semi-structured interviews, this study constructed an indicator system comprising five dimensions: theoretical and practical skills, interpersonal skills, specialty development capabilities, critical thinking and nursing management skills. Following Delphi consultation, the system was finalised with six first-, 17 second- and 69 third-level indicators. Theoretical knowledge and practical skills are foundational for guiding the work of INs, receiving the highest overall weighted score of 0.303, underscoring experts’ recognition of their critical importance. This aligns with Zijuan Yu’s findings on the essential role of robust theoretical and practical competencies in nursing informatics master’s training programmes.35 Second-level indicators include ‘nursing information theoretical knowledge’ (weighted score, 0.163), ‘nursing information practical skills’ (0.090) and ‘relevant professional knowledge and skills’ (0.050). Given the scarcity of INs proficient in both nursing informatics and computer sciences in China, initial training efforts should prioritise the development of nurses with a strong theoretical foundation in nursing informatics. The specialised nature of IN roles demands diverse, interdisciplinary and systematic knowledge to effectively support hospital nursing informatisation.36 The first-level indicator communication and coordination skills ranked fourth with a weight of 0.152. Effective communication and coordination are pivotal in fostering teamwork, cultivating a positive work environment and enhancing operational efficiency.37 Secondary indicators—communication skills and coordination skills—were equally weighted at 0.076 each, emphasising their balanced importance. Effective communication underpins enhanced coordination, enabling INs to collaborate seamlessly with engineers and clinical staff.36 38 INs actively participate in developing nursing information projects that meet clinical needs, working closely with IT teams, medical staff and stakeholders to resolve clinical information challenges throughout project development and implementation.39 The first-level indicator for professional development capabilities received a weighted score of 0.210, ranking second. Nursing informatics is an interdisciplinary discipline, which has high requirements for the professionalism of INs, and they need to have sustainable development ability. Professional development capabilities encompass learning (ranking first at 0.114), teaching, innovation and scientific research abilities. INs face complex challenges requiring continuous learning to expand their roles and keep pace with rapid technological advancements, thereby preventing professional obsolescence.40 Although critical thinking ability had the lowest weight (0.050), it remains essential for clinical decision-making and problem-solving.41 INs must identify actual information issues, potential clinical risks and nursing staff needs, making informed judgments based on specific problem characteristics. Currently, the emphasis on critical thinking among Chinese nurses is moderate but is expected to grow as role perceptions evolve. Nursing management ability, with a weight of 0.210, is another key competency. This includes organisational leadership and project management, with project management scoring particularly high (0.158). In the United States, nurse informatics specialists often act as project managers, requiring expertise in nursing, clinical workflows and project planning.42 Occupational humanistic characteristics ranked fifth at a weighted score of 0.075, include occupational emotion, professional conduct and psychological adjustment ability. Many INs experience role ambiguity and difficulty gaining cooperation from clinical staff, leading to feelings of anxiety, loneliness or distress. Psychological adaptability is thus crucial for effective work performance.43

This study aimed to develop a core competency index system for INs based on the ‘theoretical of core competence’ and ‘role theory’. Comprehensive literature reviews and semi-structured interviews ensured the indicators were both comprehensive and representative, reflecting the current status of IN training in China.

This study had some limitations. First, with the rapid advancement of information technology, emerging technologies and tools continually reshape the skills required of INs. Consequently, the competency requirements identified in this study are not fixed but are expected to evolve alongside technological progress. Second, the Delphi consultation involved only nursing professionals as experts, which may have excluded valuable insights from other related disciplines, potentially leading to a limited perspective. Third, this study did not conduct a small-scale pilot test, which may result in undetected flaws in the clarity, comprehensiveness or structural validity of the Delphi questionnaire. Future research should consider implementation in real practice settings to address these limitations and improve the generalisability of the findings.

Conclusion

A core competency framework for INs was developed using the Delphi technique, encompassing six dimensions: theoretical and practical skills, communication and coordination skills, professional development capabilities, critical thinking, nursing management skills and occupational humanistic characteristics. This framework offers targeted guidance for INs to focus their training based on the weighted importance of each indicator, thereby enhancing their qualification, assessment and evaluation processes. For nursing practice, the improvement of the core competencies of INs can optimise the nursing process and efficiency. For example, the electronic medical record system currently used in China not only reduces errors in paper care records but also ensures that data is updated and shared in real time. Finally, for patient outcomes, the improvement of the core competency of INs can enhance patient safety, reduce medical errors and enable early risk identification, such as reducing medication errors through systematic drug reconciliation and electronic prescribing.

Data availability statement

Data are available in a public, open access repository. All data relevant to the study are included in the article or uploaded as online supplemental information.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

This study involves human participants and was approved by This study adhered to the tenets of the Declaration of Helsinki. This study was approved by the Ethic Committee of Hangzhou Normal University (approval number: 2022023). Informed consent was provided and obtained from all participants before the study commenced. Participants gave informed consent to participate in the study before taking part.

Acknowledgments

The authors wish to thank all participants, especially the 21 experts who helped with the analysis and interpretation of this study.

Read the full text or download the PDF:

Log in using your username and password