Introduction

On the last day of 2019, 44 pneumonia cases with unknown etiology were reported to the World Health Organization (WHO) Country Office of China. This was the first cluster of what would be defined later as coronavirus disease (COVID-19). By March 11th, 2020, the WHO declared COVID-19 as a pandemic aiming to accelerate the measures undertaken by the member states to avoid exponential growth of cases.1

Non-pharmacologic Interventions (NPIs) are deemed inevitable during the outbreaks of emerging infectious diseases due to a lack of effective drugs and vaccines. The NPIs, including public quarantine, social distancing, and case investigation and isolation, proved their efficacy in decreasing the production rate in countries that applied aggressive measures, e.g., Singapore, South Korea, and Taiwan.2

The outbreak of infectious diseases is widely perceived as a traumatic event leading to a significant increase in anxiety, depression, and fear levels.3 During the Severe Acute Respiratory Syndrome (SARS) outbreak, the quarantined individuals had more negative emotions like anxiety and anger, which are consistent with the findings from isolated mice experiments.4 Hu et al. found that the type of quarantine can affect anxiety during the COVID-19 outbreak in China.5 Mass quarantine restrictions on non-emergency health services, including psychiatric care, may adversely affect vulnerable populations’ access to mental health care, which cannot be provided by health professionals in isolation units and hospitals due to a lack of specialized training.6

The immediate increase of posttraumatic stress symptoms (PTSS) following the SARS outbreak had long-term consequences; SARS survivors with PTSS experienced persistent psychological distress and diminished social functioning in the four years after SARS treatment.7 Frontline healthcare workers of MERS in South Korea were experiencing PTSS until three years after the outbreak with numbness and sleeping disorders in the high-risk group.8 In Italy, 37% of national survey respondents had PTSS during the 3rd and 4th weeks of lockdown measures – suggesting that monitoring of the population’s mental health should be a critical priority during pandemics.9

It is worth noting that anxiety induced by the epidemic could lead to some problems. We have known for the long term that anxiety is linked to health risk behaviors. For instance, individuals’ dental anxiety leads to lower dental services and eventually causes dental problems. And, the negative dental health-related outcomes will reinforce the anxiety in turn. Interestingly, researchers found dental professionals’ regulation of patients’ anxiety-related responses may help break the cycle.10 Likewise, there is an association between anxiety and asthma control.11 Thus, it is necessary to monitor the COVID-19 induced anxiety and investigate if the anxiety has adverse effects on people’s health risk behaviors. If so, the intervention program to control people’s anxiety level is needed in public health management.

Psychobehavioral surveillance is critical for public health response during infectious disease outbreaks because it informs risk awareness strategies.12 Perceived risk during contagious disease outbreaks motivates people to adopt protective behaviors to reduce any potential hazards of an emerging epidemic.13 The relationship between fear and protective behaviors is not linear. Still, it can be explained by the inverted U-shaped Fear Drive Model of Janis, which demonstrates that a moderate level of fear motivates people to adopt protective behavior. Still, when this level is too high or too low, people are more likely to engage in risky behaviors.14

The level of knowledge significantly influenced protective behaviors and use of social media during the outbreak of COVID-19 and H1N1 influenza and positively impacted the epidemic week and viral serial interval.12,13,15 Public awareness of the mechanisms of infection transmission and common symptoms are usually found to be sufficient among the public. However, knowledge of prevention and care-seeking strategies may be distorted by misconceptions and inaccurate information.16 Infodemic following infectious disease outbreaks is a predictable phenomenon that challenges public health strategies; it prevents laymen from acting appropriately because of confusing, contradictory, and false information (i.e., fake news). Therefore it should be contained to escalate favorable behavioral changes.17

Notably, the definition of “high-risk” groups does not always correlate between epidemiology and psychology. The high-risk groups for infection, disease progression, and fatality are predicted to develop more anxious emotions and different behaviors based on their elevated perceived risk levels. Young adults also form a particularly high-risk group during health crises because they are more susceptible to be influenced by fake news from social media, have high levels of anxiety and depression, and engage in risk behaviors.3,5,18,19

During infectious disease outbreaks, public health and psychology researchers race against time to assess the early consequences of the emerging phenomena. Therefore they typically adopt generic instruments which might not be specific for use in crisis settings. To the best of our knowledge, there are no valid instruments to evaluate the induced anxiety, protective behaviors, and public knowledge following infectious disease outbreaks. Therefore, the primary objective of this work was to develop and validate the COVID-19 Induced Anxiety Scale (CIAS), Protective Behaviors towards COVID-19 Scale (PBCS), and COVID-19 Related Knowledge Scale (CRKS). The secondary objective was to investigate the impact of COVID-19 induced anxiety and related knowledge on protective behaviors of the adult population.

Methods

I. Participants

A self-administered questionnaire with multiple-choice items was created in Microsoft Forms (Microsoft Corp. Redmond, WA. 2020). The questionnaire’s functionality and user-friendliness were pre-tested before sending it to the participating volunteers by instant messaging applications, WhatsApp (WhatsApp Inc. Menlo Park, CA. 2020) and WeChat (Tencent Holdings Ltd. Shenzhen, China. 2020). University students filled out the questionnaire between March 25th-27th 2020, based on a personal invitation from the study investigators. The URL of the questionnaire was shortened using Bit.ly (Spectrum Equity. Boston, MA. 2020) to facilitate its sharing and to enable tracking of the visitors. The participants received gratitude after completing the investigators supplied with a factsheet of COVID-19 that includes the correct answers to the actual knowledge subscale questions. The study was conducted in accordance with the Declaration of Helsinki20 and reported in accordance with the Checklist for Reporting Results of Internet E-Surveys (CHERRIES).21 Ethical approval was waived by the university’s Research Ethics Committee because this study did not involve biomedical samples nor did impose greater than minimal risks of information or psychological harm. An electronic informed consent was obtained from each participant before filling the questionnaire. No identifying personal information was collected from the participants upon filling the questionnaire. All the study data were stored in Microsoft Drive in accordance with the General Data Protection Regulation (GDPR).22 Participants did not receive any incentives to participate in the study, and they could withdraw at any moment without having to justify.

II. Measures

1. COVID-19 Induced Anxiety Scale (CIAS)

A 5-point Likert scale with ten items, where “1” refers to “Totally disagree” and “5” refers to “Totally agree,” was developed to evaluate the anxiety induced by the COVID-19 outbreak highlighting the suggested sources of stress and anxious emotions, e.g., “When I or any family member go outside home during this COVID-19 outbreak I feel anxious”. There is only one factor estimated by all items based on the confirmatory factor analysis (CFA) model in the theoretical framework. The psychometric analysis for the scale was subsequently conducted. (Table 1)

Table 1.COVID-19 Induced Anxiety Scale (CIAS)
1 I am healthy, so I am not worried of being infected by a novel coronavirus. 1 2 3 4 5
2 Maintaining a positive mental state is helpful in preventing COVID-19 infection. 1 2 3 4 5
3 Anxiety and worries of others around me can increase my fear of COVID-19 outbreak. 1 2 3 4 5
4 When I or any family member go outside home during this COVID-19 outbreak I feel anxious. 1 2 3 4 5
5 It is a disgrace to get infected by COVID-19. 1 2 3 4 5
6 I am scared of individuals coming from the affected areas. 1 2 3 4 5
7 I will be restless and sleepless when I have fever, cough or other symptoms during COVID-19 outbreak. 1 2 3 4 5
8 Mastering more knowledge and information about COVID-19 can reduce my anxiety about it. 1 2 3 4 5
9 Updates of data about COVID-19 outbreak increase my anxiety and worries. 1 2 3 4 5
10 I am afraid to die if I get infected by COVID-19. 1 2 3 4 5

1: Totally Disagree, 2: Disagree, 3: Not Sure, 4: Agree, 5: Totally Agree

2. Protective Behaviors towards COVID-19 Scale (PBCS)

A 5-point Likert scale with 14 items, where “1” refers to “Not at all like me” and “5” refers to “Just like me,” was developed to evaluate people’s protective behaviors against coronavirus infection from 3 dimensions: Routine Protective Behaviors (RPB), Post-exposure Protective Behaviors (PPB), and Post-exposure Risky Behaviors (PRB). Items in RPB are aimed to measure individuals’ protective behaviors in daily life when facing the epidemic. For example, one item in RPB is “I cancel various parties in the event of COVID-19 outbreak immediately”. The PPB subscale mainly asks about people’s protective behaviors after exposure to possible infection. A sample item of PPB is “If I get in contact with someone from COVID-19 outbreak area, I should isolate myself”. Finally, the questions in exam people’s risky behaviors after the possible infective exposure. A sample item of PRB is “If my family member or my friend is in health condition after they come back from outbreak area, there is no need to take protective measures.” According to the previous theory in public survey research, such reversed questions can improve the accuracy of the survey.23 For the PRB dimension, the item responses were calculated reversely for further analysis. Consequently, the higher total scores for each subscale and the overall scale refer to the higher quality of protective behaviors. (Table 2)

Table 2.Protective Behaviors towards COVID-19 Scale (PBCS)
1 I keep my hands clean during the outbreak. 1 2 3 4 5
2 I cancel various parties in the event of COVID-19 outbreak immediately. 1 2 3 4 5
3 I cancel unnecessary travel plans in the event of COVID-19 outbreak immediately. 1 2 3 4 5
4 I do not visit any relatives or friends during the outbreak. 1 2 3 4 5
5 I cover my mouth and nose whenever I go out or in public. 1 2 3 4 5
6 If I get in contact with someone from COVID-19 outbreak area, I should isolate myself. 1 2 3 4 5
7 I shall report the health authorities if I have a family member who just came from COVID-19 outbreak area. 1 2 3 4 5
8 If my family member or my friend is in health condition after they come back from outbreak area, there is no need to take protective measures. 1 2 3 4 5
9 In my opinion, quarantine can be terminated in advance if there are no abnormal symptoms. 1 2 3 4 5
10 I should quarantine myself immediately if I have suspicious symptoms. 1 2 3 4 5
11 I can take medicine without doctor’s suggestion as long my suspicious are still mild. 1 2 3 4 5
12 If I have suspicious symptoms, I should try to get medical help immediately from professional doctors. 1 2 3 4 5
13 Concealing medical history is not good for myself and others. 1 2 3 4 5
14 If there is a suspicious infection or a confirmed case at home, I should notify the health authorities as soon as possible. 1 2 3 4 5

1: Not at all like me, 2: Not like me, 3: Not Sure, 4: Like me, 5: Just like me

A multiple-choice scale of 12 items was developed to assess public awareness of COVID-19 as an emerging infectious disease. Each item has one right option out of four available options. The items were stratified according to 6 major domains: 1) etiology; 2) epidemiological characteristics 3) signs and symptoms; 4) prevention strategies (self-protection); 5) prevention strategies (protection of others); and 6) management measures (while in-home quarantine). (Table 3)

Table 3.COVID-19 Related Knowledge Scale (CRKS)
Etiology
1. Coronaviruses family cause the following disease beside COVID-19.
2. The following sentences are fake news (myths) regarding the transmission of COVID-19, except.
Epidemiology
3. …….. are at high risk of developing severe illness due to COVID-19 infection.
4. The mortality (death) rate is the highest among which age group of the following.
Symptomology
5. The incubation period between the exposure to SARS-CoV-2 (infection) and showing symptoms (clinical manifestation) is.
6. COVID-19 symptoms include all the following except.
Prevention (Self-protection)
7. The following sentences are fake news (myths) regarding the prevention of COVID-19, except.
8. Alcohol-based hand sanitizers and surface disinfectants should be.
Prevention (Protection of others)
9. You should protect other people by taking the following measures, except.
10. Social distance aims to slow down the spread of the infection, it is necessary to keep …………. at least between yourself and others.
Management (Quarantine)
11. In the case of home isolation (home quarantine), all the following sentences are correct except.
12. In order to discontinue home isolation without having a test, people with COVID-19 can leave home after the three following conditions are fulfilled altogether, except.

III. Statistical Analysis

SPSS 25.0 (SPSS Inc. Chicago, IL. 2020) and the lavaan package in R was used for statistical analysis.24 Three major steps were taken: (1) the item analysis for CIAS and PBCS was conducted based on item-total correlation (54); (2) on the foundation of item analysis, we continued to refine the scales based on the CFA models, which aimed to investigate the structural validity of the CIAS and the PBCS; (3) we constructed the SEM model to explore the association of COVID-19 induced anxiety, COVID-19 related knowledge and the protective behaviors in more detail.

Results

I. Participants

A total of 215 university students from 17 countries filled out the questionnaire completely. The demographic characteristics of participants are presented in Table 4. The participation rate, defined as the ratio of users who completed the questionnaire / the users who viewed the first page of the survey, was 215/662 (32.5%).

Table 4.Demographic Characteristics of Participants
N (Number) % (Percentage)
Gender
Female 173 80.5
Male 42 19.5
Age
18-20 112 52.1
21-25 70 32.6
26-30 17 7.9
31-35 5 2.3
36-40 11 5.1
Country
China 160 74.4
Nigeria 26 12.1
Czechia 6 2.8
Poland 3 1.4
Other 20 9.3
Academic Level
Bachelor 152 70.7
Masters 49 22.8
Doctoral 14 6.5

II. Item Analysis

The item analysis was completed using the method of item-total correlation. According to the classical criteria, the item with item-total correlation coefficients below 0.3 should not be accepted.25 Thus, for the CIAS, except the first (r=0.17, p=0.01), the second (r=0.10, p=0.13) and the eighth (r=0.19, p<0.01) items, other items were accepted. For the PBCS, the item-total correlations of all questions in scale ranged from 0.34 to 0.73. Therefore, there is no item rejected based on item analysis in the scale.

III. Confirmatory Factor Analysis (CFA) for COVID-19 Induced Anxiety Scale (CIAS) and Protective Behaviors towards COVID-19 Scale (PBCS)

For the CIAS, we constructed the first CFA model. The latent level of anxiety was estimated by all remaining seven items in the original scale after item analysis rejected three items. However, this model noted the fifth item in CIAS contributed a low factor loading (0.17, less than the recommended 0.4 at the latent construct level).26 Thus, we continued to delete item 5 to test the new CFA model, where all factor loadings were above 0.4. Moreover, the model had a good fit (CFI=0.985, RMSEA=0.05, SRMR=0.04, chi-square/df=13.04/9) according to the joint criteria for good model fit (i.e., meeting 3 out of 4 criteria). The 4 specific criteria are (1) CFI>0.9; (2) RMSEA<0.09; (3) SRMR<0.09; (4) chi-square/df <5.26–28 The internal consistency was fair (Cronbach’s alpha=0.78).

For the PBCS, we assigned five items into the factor of routine protective behaviors (RPB), six items into post-exposure protective behaviors (PPB), and three items into post-exposure risky behaviors (PRB). All factor loadings were greater than 0.4 and the fit of the CFA model was acceptable (CFI=0.90, RMSEA=0.08, SRMR=0.06, chi-square/df= 179.15/74). The internal consistency of PBCS was good (Cronbach’s alpha=0.85).

Before the SEM, multiple regression was set up to probe if an individual’s demographic information (including gender and academic level) can predict protective behaviors. The results remarked gender accounts for a 5% variance of an individual’s protective behaviors, and that academic level does not explain the protective behaviors. For examining gender’s effect in greater detail, the T-test was done to compare the gender difference in protective behaviors. The outcome inferred females have significantly more protective behaviors than males (t= 3.3, p< 0.01). And the COVID-19 induced anxiety and COVID-19 related knowledge account for the variance of protective behaviors by an additional 22%.

Theoretically, people’s related knowledge and anxiety level should influence people’s behaviors towards COVID-19. Figure 1 depicts the impact of an individual’s COVID-19 related knowledge and COVID-19 induced anxiety on the protective behaviors tested using SEM.

Figure 1
Figure 1.The SEM for describing the impact of COVID-19 related knowledge and COVID-19 induced anxiety on three dimensions of protective behaviors towards COVID-19

The results suggest that COVID-19 related knowledge is positively associated with the three dimensions of protective behaviors towards COVID-19. However, interestingly, the COVID-19 induced anxiety was negatively linked to the dimension of risky postexposure behaviors (PRB) and positively to routine protective behaviors (RPB) and post-exposure protective behaviors (PPB).

Discussion

The primary objective of this work was to develop psychometrically sound scales to assess COVID-19 induced anxiety (CIAS), protective behaviors towards COVID-19 (PBCS), and COVID-19 related knowledge. The results indicated that COVID-19 induced the 6-item version of the CIAS can adequately measure anxiety level. Infectious disease outbreaks represent specific health-related crises that may impact people’s emotions in different patterns according to their emerging nature. Therefore, the CIAS was designed to cover the potential anxiety sources for the general population during the COVID-19 outbreak. The CIAS assesses anxiety from specific sources, including going outside of the house, disease-related stigma, contracting with people from outbreak areas, getting suspicious clinical symptoms, updating outbreak data, and resulting in death. Therefore, it represents a valuable tool in measuring the level of anxiety-related explicitly to COVID-19. Interpersonal anxiety transfer (transmission of anxiety from one person to another) may aggravate during infectious disease outbreaks through the object-directed social appraisal theory, which states that the person becomes significantly influenced by the information picked up from other person’s anxiety expression.29

The PBCS can also adequately estimate people’s protective behaviors towards COVID-19 from three aspects: routine protective behaviors (RPB), post-exposure protective behaviors (PPB), and post-exposure risk behaviors (PRB). During the outbreak, any recommended public health measures for personal protection suggested by official health authorities need to be widely perceived and adopted in a timely manner. Therefore regular hand-washing, social distancing practice, and face-masks wearing while in public were considered as the RPB of interest in this scale. In contrast to the PPB, which include self-isolation and informing local health authorities after coming from abroad, doing home quarantine and seeking medical advice as soon as suspicious symptoms arise, and reporting neighboring suspicious and confirmed cases, the PRB is about resisting health recommendations by concealing medical history and taking medications without medical advice.

Based on the hierarchical regression model, gender can explain the variation of the protective behaviors while the academic level cannot explain the behaviors. This finding is supported by the t-test, which indicated that compared to females, males have a lower level of protective behaviors towards COVID-19. However, this finding warrants further corroboration due to the unbalanced and selective nature of this study sample.

Our SEM model implied that an individual’s COVID-19 related knowledge predicts three dimensions of protective behaviors (RPB, PPB, and PRB) in a positive linear way. However, for the level of COVID-19 induced anxiety, it only augurs RPB and PPB by positive linear relationship while it is negatively associated with PRB. The gender differences in protective behaviors are consistent with the previous findings, which suggested that compared to females, males are more inclined towards risky behaviors for many specific events because females usually perceive more negative possible outcomes than males.30,31

The deduction that the level of knowledge towards COVID-19 increases the protective behaviors is consistent with behavioral predictions whereby specific knowledge helps an individual perform more healthy behaviors on the knowledge-related event. The Health Belief Model argues that an individual’s cognitive aspects of health belief, including risk perception and knowledge, can impact health-related behaviors.32 Generally, more comprehensive and accurate knowledge is linked to more health-promoting behaviors. For example, among the elderly population, there is a significant positive correlation between knowledge about Alzheimer’s symptoms and seeking behaviors for professional help.33 In college adults, knowledge about AIDS enhances HIV prevention behaviors.34 For schoolchildren, the awareness of the importance of physical exercise promotes their engagement in sports activities.35

The interesting phenomenon that the level of COVID-19 induced anxiety raises the possibility of routine protective behaviors and post-exposure behaviors and enhances the potential post-exposure risky behaviors underscores the complexity with which anxiety impacts behavior. On the one hand, some research indicated that anxiety causes risk-avoidant decisions and behaviors.36,37 The theory can explain why the decision style that anxiety implies the potential threat and helps people perceive lower vulnerability to the threat.38,39. However, on the other hand, anxiety can have a negative effect. It could accumulate and make one prone to risk-taking behaviors, especially under circumstances where an individual displays emotion regulation deficits.40,41

In conclusion, this study developed and provided initial validation for scales assessing induced anxiety CIAS, protective behavior PBCS, and related knowledge CRKS. These instruments can be rapidly adopted for other infectious diseases during the early phase of pandemic outbreaks. Knowledge is a facilitator for protective behaviors, while severe anxiety can indicate risk behaviors during the early phases of the epidemic. Therefore, public health strategies need to transmit timely evidence-based health information to the public and monitor community anxiety and posttraumatic stress symptoms.

Limitations

First, the participants in the sample were mainly from China, and most of them were female. The cultural differences in COVID-19 protective behaviors remain unclear, and future studies should evaluate whether the associations among protective behaviors, anxiety, and knowledge are stable across gender and cultural contexts.

Second, the knowledge scale was only designed based on a professional medical education framework but without explicit psychometric analysis testing (such as the IRT test). Establishing a scale validity is an ongoing process, and continuous work should ensue to cross-validate the scale in independent samples.

Third, it may be desirable, in addition to assessing actual knowledge, to assess perceived knowledge (e.g., as per Health Belief Model) to evaluate its impact on an individual’s protective behaviors towards COVID-19. Given the limited sample size, future research should continue to assess psychometric properties of the scales in more representative samples and further probe the associations among protective behaviors, anxiety, and knowledge (actual and perceived).

Practice Implications

  1. The 6-item version of CIAS and the 14-item version of PBCS are promising tools that can be rapidly adapted to evaluate communicable disease-induced anxiety and protective behaviors during the early phase of pandemic outbreaks.

  2. Knowledge is a crucial indicator for protective behaviors during the early phase of the outbreak. Therefore public health strategies need to transmit timely evidence-based health information to the public while also highlighting misconceptions circulated by unverified resources.

  3. Severe anxiety in response to infectious diseases outbreak should be monitored as it may lead to risk behaviors which can adversely affect individual’s health or disease outcomes. Therefore, public health strategies need to monitor community anxiety and posttraumatic stress symptoms.

Conflict of Interest

The authors have no conflicts of interest associated with the material presented in this paper.

Acknowledgements

The authors would like to thank Pascal Bassey MSc. (Masaryk University), Alkauthar Seun Enakele MSc. (Masaryk University), Islam Abdelgawwad BSc. (Damanhour University), Ave Põld DDS. (Charité – Universitätsmedizin Berlin), and each anonymously participating student who contributed to make this work possible.