Beta Fulltext view is in preview — article structure may vary. Browse all articles
Contents
Journal of Orthopedics & Bone Disorders Research Article 14 min read

Limited Effect of Generalized Systematic Lockdown on Urgent Orthopedic and Limb Trauma Surgery in a Geographical Area Mildly Affected by the First of SARS-CoV-2 Pandemic: A Multicentric Study (Limited Effect of Lockdown in an Area Mildly affected by the Pandemic)

Guillaume Villatte*, Anne-Sophie Ravery, Pierre-Sylvain Marcheix, Guillaume Sam-Wing, Benjamin Bouillet, Stéphane Descamps, Stéphane Boisgardand Roger Erivan
* Corresponding author
ISSN: 2577-297X  10.23880/jobd-16000206  Received: February 24, 2021  Published: April 07, 2021
  views
 19 references
 4 figures
 3 tables
PDF
Keywords
Epidemiology SARS-Cov-2 COVID-19 Pandemic Lockdown Trauma
Abstract

Purpose: The SARS-CoV-2 pandemic led to a systematic lockdown of populations in many countries along with the reorganization of care systems to respond to this unprecedented health crisis. This study determined (i) the number of urgent surgical operations carried out during the lockdown in a region of France mildly affected by the SARS-CoV-2 pandemic, and (ii) the types of surgery concerned (main characteristics). Methods: A retrospective, comparative, observational, multicentric study of urgent orthopedic and limb trauma surgery during lockdown was conducted. It included three reference care centers of which two were teaching hospitals. A comparison was made with data for the same period in 2019. For each patient, the following data were collected: date of surgery, age, sex, place of residence, and standardized descriptors used to classify the injury anatomically, context in which the injury occurred, and SARS-CoV-2 PCR status. Results: 1098 patients underwent surgery against 1266 over the same period in 2019, a non-significant difference of −13.3% (p = 0.11) [−39.2% in Week 13 (p = 0.02), +7.6% in Week 19 (p = 0.16)]. Except for bone injuries of the hand and wrist (+13.4%, p = 0.07), a decrease in activity was observed for all anatomic locations, and especially for the distal radius (−55.8%, p = 0.001), ankle (−40.5%, p = 0,01). The context in which the injury occurred changed, with in particular a significant decrease in the number of sport accidents (−80.0%, p < 0.001) and road traffic accidents (−41.8%, p = 0.04), and an increase in DIY and gardening accidents (+62.7%, p < 0.001). Conclusion: The decrease in emergency surgery was much less marked in this geographical area mildly affected by the pandemic. Health policy that sets ways to ensure continuity in the care needed by the population must be adapted locally when a global health crisis strikes. Level of proof: 4 Descriptive epidemiological study.

Introduction

The second quarter of 2020 saw the Covid-19 epidemic spread across Europe, causing an unprecedented healthcare, social, and economic crisis. At the beginning of autumn, the epidemic is still going on and rules limiting meetings and sometimes local lockdowns are maintained.

This new infectious disease, caused by the pathogen SARS-CoV-2, appeared at the end of 2019 in mainland China (Wuhan district) [1, 2]. The World Health Organization (WHO) declared a Covid-19 pandemic on 11 March 2020.

In response to this pandemic, health systems had to adapt, which included rescheduling all non-urgent medical and surgical care [3, 4] to reallocate resources (beds, care staff, etc.) to the emergency, infectiology and resuscitation services, which were in greatest need. In parallel, governments and their departments in each country took measures, of which the most emblematic one was the stay-at-home order or ‘lockdown’ [3]. In restricting the non- essential professional activities and also private activities of its population (ex. limitation of displacement for one hour and less than one-kilometer far from home) this measure sought to both curb the propagation of the virus within the country and relieve the national healthcare system. No data were then available to gage the true impact of a measure such as generalized systematic lockdown on urgent orthopedic and trauma surgery. First Chinese then European teams published figures soon after lockdown that showed marked reductions of such surgery, of at least 30% [5, 6, 7, 8, 9, 10, 11]. However, all these series were recorded in geographical areas with high population densities where the epidemic was especially active.

The present study set out to determine (i) the number of surgical operations in orthopedics and limb traumatology carried out during the first lockdown in a region of metropolitan France mildly affected by the SARS-CoV-2 epidemic, and (ii) the types of emergency surgery (main characteristics) performed during that period.

The hypothesis was that the decrease in surgical activity during lockdown would be less than 30% in this area.

Materials and Methods

A retrospective, comparative, observational, multi- centric study of emergency surgery in orthopedics and limb traumatology was conducted over the three months encompassing the health crisis due to the SARS-CoV-2 pandemic.

Three care centers took part in this study: two public teaching hospitals and one private hospital offering an ‘SOS hand injury’ service (French Federation of Emergency Hand Injury Services, FESUM). Together they serve two administrative districts with a total population of 1.03 million. They are reference centers for orthopedics and traumatology (‘trauma centers’) in their administrative regions (population about 2.1 million) [12].

These two administrative districts were mildly affected by the SARS-CoV-2 epidemic (Figure 1). The first cases found in this geographical area were detected on 5 March 2020. The total number of deaths recorded over the study period was 67 (3.5 per 100,000 inhabitants). By comparison, the Covid-19 death rate in the whole of France was 43 per 100,000 inhabitants (28,833 deaths) over the same period.

Figure 1: Comparison of distribution of anatomic locations treated during lockdown and the same period in 2019.
Click to enlarge
Figure 1: Comparison of distribution of anatomic locations treated during lockdown and the same period in 2019.

The medical records of all the patients undergoing surgery between 1 March and 1 June 2020 in these centers were analyzed anonymously. The following data were collected: date of surgery, patient’s age and sex, their main place of residence, the principal standardized descriptor categorizing the anatomic location of the condition (French CCAM code), namely proximal femur, pelvis, femoral and tibial diaphysis, knee, ankle, shoulder/humerus, distal radius, hand/wrist (soft tissue/bone), the context in which the injury occurred (sport accident, occupational accident, accident in everyday life, DIY/gardening accident, road traffic accident, infection, cancer), and lastly SARS-CoV-2 PCR status. The reference period for the data comparison was the same successive weeks in 2019. The data about the SARS-CoV-2 pandemic (rate of death) were obtained from French Public Health and www.coronavirus-statistiques.com (Eficiens) websites.

Statistics

The statistical calculations were carried out with ExcelTM software (Microsoft, Redmond, WA) and Addinsoft (2019) XLSTATTM statistical and data analysis solution (Long Island, NY). The results are given as means ± standard deviation with extremes for quantitative variables, or medians and quartiles for data with non-normal distributions. Normality of distribution was checked using the Shapiro-Wilk test. The significance of the mean differences was tested with the Student t-test for normally distributed data or else with the nonparametric Wilcoxon test. Frequency comparisons were made using the exact Fisher test or the Chi2 test as appropriate. There were no missing data. All the reports were analyzed. The significance threshold was set at p = 0.05, with confidence intervals at 95%.

Results

Volume of Activity

In the three care centers included in the study, 1098 patients underwent surgery during the lockdown against 1266 over the same period in 2019, a non-significant difference of −13.3% (p = 0.11). This trend in activity was superimposable in the three centers (p = 0.27).

The period of least activity was Week 13 (5 days after the start of lockdown) with a significant decrease in activity of 39.2% (103 patients versus 173 in 2019) (p = 0.02). Conversely, in the last week of lockdown (Week 19), activity increased non-significantly by 7.6% relative to 2019 (127 versus 118 patients) (p = 0.16). In all, three weeks out of the eight weeks that the lockdown lasted (Weeks 13, 14 and 15) showed an activity below 75% of that for the same period in 2019 (375 versus 505 patients) (Figure 2) (p = 0.04).

Figure 2: Evolution of the number of patients undergoing surgery between Weeks 12 and 19 in years 2019 and 2020 (* significant difference).
Click to enlarge
Figure 2: Evolution of the number of patients undergoing surgery between Weeks 12 and 19 in years 2019 and 2020 (* significant difference).

Types of Surgery

The demographic characteristics of the populations treated during the lockdown period and outside it showed no significant difference (Table 1).

Only two patients showed positive SARS-CoV-2 PCR tests in the study population (rate of contamination 0.0018%).

20192020p
Mean age (years)62 [15-105]64 [16-100]0.06
Sex (M/F) (%)49.3 / 50.750.2 / 49.80.78
Place of residence (urban/
rural) (%)
43.6 / 56.445.2 / 54.80.35

Table 1: Main demographic characteristics of the populations studied.

Figure 3: Comparison of the number of deaths due to the pandemic per administrative district in metropolitan France on 1 June 2020 (the solid outline shows the two districts where the study took place; the dotted outline shows the theoretical catchment area for the three reference centers).
Click to enlarge
Figure 3: Comparison of the number of deaths due to the pandemic per administrative district in metropolitan France on 1 June 2020 (the solid outline shows the two districts where the study took place; the dotted outline shows the theoretical catchment area for the three reference centers).

The anatomic locations of the conditions treated surgically during the lockdown differed relative to the comparison period (Figure 3). Only bone injuries of the hand and wrist (distal radius excepted) increased, though non-significantly (+13.4%, 187 versus 165 acts) (p = 0.07). All the other locations decreased significantly: distal radius (−55,8%, 43 versus 97 acts, p = 0.001), ankle (−40.5%, 39 versus 65 acts, p = 0.01), shoulder and humerus (−30.6%, 35 versus 50 acts p = 0.03) and femoral and tibial diaphysis (−29.6%, 21 versus 31 acts, p = 0.04). Injuries of the proximal femur and soft tissues of the hand and wrist decreased non- significantly (−5.9%, 130 versus 138 acts, and −10.6%, 508 versus 565 acts).

In the subcategory ‘fracture of the proximal femur in patients aged over 65 years’, activity increased non- significantly by 6.4% (p = 0.12) during lockdown. The place of residence of the patients was significantly different (p = 0.02) between the two time periods, with more patients from care homes for the elderly or care centers, than patients living in their own home (41/59% during lockdown versus 31/69% during the control period).

Lockdown modified the context in which the surgically treated injuries occurred (Figure 4). A significant decrease in the number of sport accidents (−80.0%, 26 versus 132 acts, p < 0.001), and road traffic accidents (−41.8%, 50 versus 87 acts, p = 0.04) was observed. There were non-significant decreases in surgery for occupational accidents (−36%, 67 versus 105 acts, p = 0.09), accidents in everyday life (−10.3%, 603 versus 672 acts, p = 0.23) and infections (−2.8%, 135 versus 138 acts, p = 0.99). Conversely, surgery due to DIY/ gardening accidents increased significantly by 62.7% (149 versus 93 acts, p < 0.001).

Figure 4: Evolution of injury rate by context of occurrence during the confinement and during the same period in 2019.
Click to enlarge
Figure 4: Evolution of injury rate by context of occurrence during the confinement and during the same period in 2019.

Discussion

In a geographical area mildly affected by the SARS-CoV-2 epidemic, this multi-centric comparative study showed that lockdown produced a limited decrease in emergency surgery in orthopedics and limb traumatology (−13.3%, p = 0.11).

In March 2020 numerous guidelines were published [13] to organize care provision for patients in orthopedics and traumatology during the pandemic so as to relieve pressure on the care system and free resources to deal with the SARS- CoV-2 epidemic. However, evidence that would help predict the volume of activity in this new context was lacking, with a consequent risk of mismatch between means and needs in patient care.

The results of our work are original in that they report a much lower decrease in surgical activity in an area mildly affected by the SARS-CoV-2 pandemic than in one where it was especially serious (Table 2). In the region studied, lockdown had a modest and temporary effect on the volume of emergency surgery. A decrease exceeding 15% was observed only for three weeks (Weeks 13–15), with a maximum decrease of 39.2% in Week 13. This period immediately followed the unprecedented political decision to order total national lockdown and when daily media coverage of the pandemic was intense [14]. However, in early April the epidemic affected our study region only mildly, the government issued some rules to relax lockdown for certain occupational groups and an end to lockdown was envisioned [15]. The combination of these different factors probably helped to reassure the local population, allowing a gradual resumption of activities. In parallel, we observed changes in the characteristics of the emergency surgery during this period. Restricted travel and gatherings resulted in a decrease in sport accidents (−80.0%) and road accident trauma (−41.8%), explaining a decrease in injuries to the femoral and tibial diaphysis (−29.6%) and ankle (−40.5%). Conversely, the locked down population was freer to indulge in DIY/gardening activities, with a correspondingly steady volume of hand and wrist injuries (−10.6% and +13.3%, p = 0.07). Among elderly persons (both in their own home and in care homes), lockdown induced social isolation and less stimulation, sometimes with loss of familiar bearings, conducting to a potential risk of fall which can explain a tendential rise in proximal femur factures (+6.4%, p = 0.12). These trends have been systematically reported in all the studies on the topic [5, 6, 8, 9, 10, 11, 13, 16].

City/Region (Country)Mean evolution of emergency
activity during lockdown
Death rate due to SARS-CoV-2 per
100,000 inhabitants
Paris (France) [9]−64.9%79
Madrid (Spain) [7]−54%195
Hubei (China) [8]−49.8%761
Lancashire (UK) [6]−43.2%90
Hornu (Belgium) [13]−32%91
Series−13.2%3.5

Table 2: Literature data on the evolution of orthopedic and traumatological emergency surgery during lockdown in various cities a

This study presents some biases and limits. It is a retrospective study with a risk of selection bias by data omission. To ensure good representativeness of the results, it was decided to include all the patients undergoing surgery identified by a national code specific to each procedure. This option can be criticized because the manually assigned descriptors may not have been exhaustive and not precisely fit the conditions treated, but it can be considered as sufficiently representative because the assignment of this code determines the remuneration of care centers and surgeons and underpins epidemiological surveys of surgical practice [17, 18, 19]. Lastly, despite the multicentric design and more than 1000 patients included (largest series on the topic concerning surgery patients), some of the results are non- significant, suggesting insufficient power.

Conclusion

In a region where the SARS-CoV-2 epidemic was mildly active, generalized systematic lockdown moderately and temporarily decreased the volume of emergency surgery in orthopedics and limb traumatology, and the anatomical distribution of emergency types was changed.

These findings are an example which argues that health policies to lockdown a population and to decide for the staffing, equipment, and operating facilities needed to ensure continuity of vital care during a health crisis should be adapted to local conditions rather than be applied nationally. They should allow for both the lockdown measures and the real intensity of the epidemic in the geographical area affected.

STROBE Guidelines for Authors of OTSR

To be used by authors of all observational clinical studies published in OTSR. For this purpose a cohort study (the term used by STROBE) is considered a longitudinal study typically reporting outcomes of treatment in one or more cohorts; a case-control study is one identifying factors in outcomes; a cross-sectional study is one to identify the prevalence of factors or characteristics in a population at a single point in time.

This checklist table is modified from The STROBE Initiative, www.strobe-statement.org and should be filled and submitted within the electronic submission.

Item NoRecommendationPlease insert check
where included
or N/A where not
applicable
Title and abstract1(a) Indicate the study’s design with a commonly used term in the title
or the abstract
ok
(b) Provide in the abstract an informative and balanced summary of
what was done and what was found
ok
Introduction
Background/
rationale
2Explain the scientific background and rationale for the investigation
being reported
ok
Objectives3State specific objectives, including any pre specified hypothesesok
Methods
Study design4Present key elements of study design early in the paperok
Setting5Describe the setting, locations, and relevant dates, including periods
of recruitment, treatment, follow-up, and data collection
ok
Participants6(a) Cohort study—Give the eligibility criteria, and the sources and
methods of selection of participants. Describe methods of follow-up
ok
Case-control study—Give the eligibility criteria, and the sources
and methods of case ascertainment and control selection. Give the
rationale for the choice of cases and controls
Cross-sectional study—Give the eligibility criteria, and the sources
and methods of selection of participants
(b) Cohort study—For matched studies, give matching criteria and
number of treated and untreated
N/A
Case-control study—For matched studies, give matching criteria and
the number of controls per case
Variables7Clearly define all outcomes, exposures, predictors, potential
confounders, and effect modifiers. Give diagnostic criteria, if
applicable
N/A
Data sources/
measurement
8*For each variable of interest, give sources of data and details of
methods of assessment (measurement). Describe comparability of
assessment methods if there is more than one group
ok
Bias9Describe any efforts to address potential sources of biasok
Study size10Explain how the study size was arrived atok
Quantitative
variables
11Explain how quantitative variables were handled in the analyses. If
applicable, describe which groupings were chosen and why
ok
Statistical methods12(a) Describe all statistical methods, including those used to control
for confounding
ok
(b) Describe any methods used to examine subgroups and
interactions
ok
(c) Explain how missing data were addressedok
(d) If applicable, explain how loss to follow-up was addressedok
(e) Describe any sensitivity analysesok
Results

Participants 13*

(a) Report numbers of individuals at each stage of study—eg, numbers potentially eligible, examined for eligibility, confirmed eligible, included in the study, completing follow-up, and analyzed ok (b) Give reasons for nonparticipation at each stage ok (a) Give characteristics of study participants (eg, demographic, clinical, social) and information on other treatments and potential confounders Ok (b) Indicate number of participants with missing data for each variable of interest Ok (c) Cohort study—Summarize follow-up time (eg, average and total amount) Ok Outcome data 15* Report numbers of outcome events or summary measures over time Ok (a) Give unadjusted estimates and, if applicable, confounder-adjusted estimates and their precision (eg, 95% confidence interval). Make clear which confounders were adjusted for and why they were included Main results 16 Ok (b) Report category boundaries when continuous variables were categorized Ok (c) If relevant, consider translating estimates of relative risk into absolute risk for a meaningful time period Ok Other analyses 17 Report other analyses done—eg analyses of subgroups and interactions, and sensitivity analyses Ok Discussion Key results 18 Summarise key results with reference to study objectives Ok Limitations 19 Discuss limitations of the study, taking into account sources of potential bias or imprecision. Discuss both direction and magnitude of any potential bias Ok Interpretation 20 Give a cautious overall interpretation of results considering objectives, limitations, multiplicity of analyses, results from similar studies, and other relevant evidence ok Generalisability 21 Discuss the generalisability (external validity) of the study results ok Other information Funding 22 Give the source of funding and the role of the funders for the present study and, if applicable, for the original study on which the present article is based ok *Give information separately for cases and controls. Note: An Explanation and Elaboration article discusses each checklist item and gives methodological background and published examples of transparent reporting. Information on the STROBE Initiative is available at http://www.strobe-statement.org.

References

  1. Zhu N, Zhang D, Wang W, Xingwang Li, Bo Yang, et al. (2020) A Novel Coronavirus from Patients with Pneumonia in China, 2019. N Engl J Med 382: 727-733.
  2. Lu H, Stratton CW, Tang YW (2020) Outbreak of pneumonia of unknown etiology in Wuhan, China: The mystery and the miracle. J Med Virol 92(4): 401-402.
  3. Sarac NJ, Sarac BA, Schoenbrunner AR, Jeffrey E Janis, Ryan K Harrison, et al. (2020) A Review of State Guidelines for Elective Orthopaedic Procedures during the COVID-19 Outbreak. J Bone Joint Surg Am 102(11): 942-945.
  4. Iacobucci G (2020) Covid-19: all non-urgent elective surgery is suspended for at least three months in England. BMJ 368: m1106.
  5. Christey G, Amey J, Campbell A, Smith A (2020) Variation in volumes and characteristics of trauma patients admitted to a level one trauma center during national level 4 lockdown for COVID-19 in New Zealand. N Z Med J 133(1513): 81-88
  6. Greenhalgh M, Dupley L, Unsworth R, Boden R (2020) Where did all the trauma go? A rapid review of the demands on orthopaedic services at a U.K. Major Trauma Centre during the COVID-19 pandemic. Int J Clin Pract 75(3): e13690.
  7. Luengo-Alonso G, Pérez-Tabernero FG-S, Tovar-Bazaga M, José Manuel Arguello-Cuenca, Emilio Calvo (2020) Critical adjustments in a department of orthopaedics through the COVID-19 pandemic. Int Orthop 44(8): 1557-1564.
  8. Lv H, Zhang Q, Yin Y, Zhiyong Hou, Yingze Zhang, et al. (2020) Epidemiologic characteristics of traumatic fractures during the outbreak of coronavirus disease 2019 (COVID-19) in China: A retrospective & comparative multi-center study. Injury 51(8): 1698-1704.
  9. Pichard R, Kopel L, Lejeune Q, Rafik Masmoudi, Emmanuel H Masmejean (2020) Impact of the COronaVIrus Disease 2019 lockdown on hand and upper limb emergencies: experience of a referred university trauma hand centre in Paris, France. Int Orthop 44: 1497-1501.
  10. Wong JSH, Cheung KMC (2020) Impact of COVID-19 on Orthopaedic and Trauma Service: An Epidemiological Study. J Bone Joint Surg Am 102(4): e80.
  11. Zagra L, Faraldi M, Pregliasco F, Anna Vinci, Giovanni Lombardi, et al. (2020) Changes of clinical activities in an orthopaedic institute in North Italy during the spread of COVID-19 pandemic: a seven-week observational analysis. Int Orthop 4: 1-8.
  12. Estimation de la population au 1er janvier 2020.
  13. Hernigou J, Morel X, Callewier A, Olivier Bath, Philippe Hernigou (2020) Staying home during “COVID-19” decreased fractures, but trauma did not quarantine in one hundred and twelve adults and twenty eight children and the “tsunami of recommendations” could not lockdown twelve elective operations. Int Orthop 44: 1473-1480.
  14. ÉTUDE. Information à la télé et coronavirus : l’INA a mesuré le temps d’antenne historique consacré au Covid-19. Rev Médias.
  15. (2020) Mesures de soutien.
  16. Nuñez JH, Sallent A, Lakhani K, Ernesto Guerra-Farfan, Nuria Vidal, et al. (2020) Impact of the COVID-19 Pandemic on an Emergency Traumatology Service: Experience at a Tertiary Trauma Centre in Spain. Injury 51(7): 1414-1418.
  17. Erivan R, Villatte G, Dartus J, Nicolas Reina, Stéphane Descamps, et al. (2019) Progression and projection for hip surgery in France, 2008-2070: Epidemiologic study with trend and projection analysis. Orthop Traumatol Surg Res OTSR 105(7): 1227-1235.
  18. Villatte G, Erivan R, Barth J, Nicolas Bonnevialle, Stéphane Boisgard, et al. (2020) Progression and projection for shoulder surgery in France, 2012-2070: Epidemiologic study with trend and projection analysis. Orthop Traumatol Surg Res OTSR 106(6): 1067-1077.
  19. Erivan R, Tardieu A, Villatte G, Matthieu Ollivier, Christophe Jacquet, et al. (2020) Knee surgery trends and projections in France from 2008 to 2070. Orthop Traumatol Surg Res OTSR 106(5): 893-902.
More from this journal

Cite this article

BibTeX
APA
RIS
@article{guillaume2021,
  title   = {Limited Effect of Generalized Systematic Lockdown on Urgent
Orthopedic and Limb Trauma Surgery in a Geographical
Area Mildly Affected by the First of SARS-CoV-2 Pandemic: A
Multicentric Study (Limited Effect of Lockdown in an Area Mildly
affected by the Pandemic)},
  author  = {Guillaume Villatte, Anne-Sophie Ravery, Pierre-Sylvain Marcheix, Guillaume Sam-Wing, Benjamin Bouillet, Stéphane Descamps, Stéphane Boisgardand Roger Erivan},
  journal = {Journal of Orthopedics & Bone Disorders},
  year    = {2021},
  volume  = {5},
  number  = {1},
  doi     = {10.23880/jobd-16000206}
}
Guillaume Villatte, Anne-Sophie Ravery, Pierre-Sylvain Marcheix, Guillaume Sam-Wing, Benjamin Bouillet, Stéphane Descamps, Stéphane Boisgardand Roger Erivan (2021). Limited Effect of Generalized Systematic Lockdown on Urgent
Orthopedic and Limb Trauma Surgery in a Geographical
Area Mildly Affected by the First of SARS-CoV-2 Pandemic: A
Multicentric Study (Limited Effect of Lockdown in an Area Mildly
affected by the Pandemic). Journal of Orthopedics & Bone Disorders, 5(1). https://doi.org/10.23880/jobd-16000206
TY  - JOUR
TI  - Limited Effect of Generalized Systematic Lockdown on Urgent
Orthopedic and Limb Trauma Surgery in a Geographical
Area Mildly Affected by the First of SARS-CoV-2 Pandemic: A
Multicentric Study (Limited Effect of Lockdown in an Area Mildly
affected by the Pandemic)
AU  - Guillaume Villatte, Anne-Sophie Ravery, Pierre-Sylvain Marcheix, Guillaume Sam-Wing, Benjamin Bouillet, Stéphane Descamps, Stéphane Boisgardand Roger Erivan
JO  - Journal of Orthopedics & Bone Disorders
PY  - 2021
VL  - 5
IS  - 1
DO  - 10.23880/jobd-16000206
ER  -