The effect of live demonstration and flipped classroom with continuous formative assessment on dental students’ orthodontic wire-bending performance

Background

Wire-bending skills is commonly taught through live demonstrations (LD) though flipped classroom (FC) method has gained popularity. Continuous formative assessment promotes personalised learning via closely monitored progress, with the identification of students’ strengths and weaknesses. This study aims to evaluate the effects of LD and FC teaching methods, supplemented with continuous formative assessment, on dental students’ learning of wire-bending skills for six types of removable orthodontic appliance components. A deeper understanding of the relative effectiveness between LD and FC teaching methods can help identify the most appropriate method to achieve student learning objectives, which is especially important given the current Covid-19 pandemic.

Methods

Forty third-year undergraduate dental students were randomly assigned into FC (n = 20) or LD (n = 20) cohort. Each student attended six teaching sessions, each to teach students’ competency in fabricating one type of wire component, for a total competency in fabricating six wire components over the course of six teaching sessions. Either LD or FC teaching methods were used. After each session, wire assignments had to be submitted. Wire assignments were then evaluated using a blinded wire-bending assessment protocol. As part of their formative assessment, the assessment results were distributed to students, lecturers, and technicians before the next session. After the first session (T0) and at the end of all six sessions (T1), students completed a self-reported questionnaire.

Results

The mean wire-bending scores for FC were significantly higher than LD for two of the six assignments, namely the Adams clasp (p < 0.01) and Z-spring (p = 0.03). Scores for both LD and FC increased significantly over time, which may be attributed to formative assessment. There was no statistically significant correlation between wire-bending scores and video usage. Students were satisfied with both teaching methods, according to T0 and T1 questionnaires.

Conclusions

在transferri LD和俱乐部都同样有效ng practical orthodontic wire-bending skills and well-received by students. Continuous formative assessment may have enhanced students’ learning of orthodontic wire-bending skills. Further studies with control group are recommended to investigate the effect of formative assessment on teaching practical dental skills.

In some dental schools, undergraduate students are taught wire-bending to train their manual dexterity. Traditionally, a live demonstration was used as the sole teaching method. More recently, either live video demonstration [1] or flipped classroom teaching [2] methods are now being employed.

The live demonstration (LD) teaching method can increase students’ confidence, improve communication skills and provide a better understanding as compared to didactic teaching [3]. However, it has also been associated with factors that decrease teaching effectiveness such as restricted view during demonstrations involving larger groups, limited repeatability due to time constraints, and the burden of manpower needed for every demonstration session [2]. To overcome these limitations, Alqahtani et al. (2015) suggested using procedural video demonstrations. They found that students performed equally well in their wire-bending skills whilst students who had video demonstration felt the steps presented were clear and easy to understand compared to those who had LD.

In flipped settings, students access teaching contents online before class, enabling interactive and collaborative activities during class to promote learning [4]. Flipped classroom (FC) provides a flexible platform for self-paced learning which helps to improve students’ learning interest [5]. Thus, it enhances personalised learning by allowing students to access resources to learn at their own preferred pace, way, and location. A meta-analysis comparing this approach to traditional teaching methods among various health education professions concluded a significant improvement in student learning with the employment of FC [6]. However, considering the variety of student learning styles and personality types, it was suggested that a more tailored teaching style be developed for increased effectiveness in medical and dental education. This necessitated the need for personalised monitoring of student learning in the FC method [7].

Continuous formative assessment can be used as a systematic approach for personalised learning. This allows monitoring of the students’ progress and enhances learning by enabling students to identify their strengths and weaknesses. It also enables teachers to identify students who are struggling early in the course and thus, address their learning needs without delay [8].

To the best of our knowledge, no study investigated the effects of LD and FC, supplemented with continuous formative assessment, during undergraduate orthodontic wire-bending teaching. Knowing the relative effectiveness of the LD and FC teaching methods allows the most appropriate method to be implemented to achieve desired student learning objectives, which is especially important given the current Covid-19 pandemic. Thus, this study aims to evaluate the effects of LD and FC teaching methods supplemented with continuous formative assessment on the performance of dental students in bending six different types of wire components for removable orthodontic appliances.

Objectives

This study embarks on the following objectives:

1. 1

   To compare students’ orthodontic wire-bending scores between two cohorts: FC and LD teaching methods.

2. 2

   To analyse the association of continuous formative assessment with orthodontic wire-bending scores of FC and LD cohorts over the six wire-bending sessions.

3. 3

   To investigate the frequency and correlation between usage of online video demonstration with students’ orthodontic wire-bending scores.

4. 4

   To compare student’s perceived satisfaction on the wire-bending demonstration for both cohorts: FC and LD teaching methods.

Null hypotheses

1. 1

   There is no difference in students’ orthodontic wire-bending scores between two cohorts: FC and LD teaching methods.

2. 2

   There is no association between continuous formative assessment with orthodontic wire-bending scores of FC and LD cohorts over the six wire-bending sessions.

3. 3

   There is no correlation between usage of online video demonstration with students’ orthodontic wire-bending scores.

4. 4

   There is no difference in student’s perceived satisfaction on the wire-bending demonstration for both cohorts: FC and LD teaching methods.

This was a prospective study conducted at the Faculty of Dentistry, University of Malaya from October 2019 to February 2020. Ethics approval was granted by the Medical Ethics Committee, Faculty of Dentistry, University of Malaya (DR CD1918/0106 (L).

Forty third-year undergraduate dental students in 2019 with no prior experience with orthodontic wire-bending were informed and consented to participate in this study. At the beginning of the academic year, students were randomly divided by the teaching institution into groups of ten students. None of the authors were involved in the group allocation. All students consented to participate in this study after understanding the study objectives and flow. The group leaders picked a sealed envelope containing the demonstration method (LD or FC) to randomly assign the groups either into a FC cohort (n = 20) or LD cohort (n = 20). For both cohorts, the skills of wire-bending were taught in a standardised manner with identical steps and standardised rubrics for each wire-bending. They were taught to bend six wire components for removable orthodontic appliances in the following order – Adams clasp, buccal canine retractor, palatal finger spring, Southend clasp, Z-spring, and Hawley labial bow. Students in each allocation cohorts were divided into smaller sub-groups to ensure the lecturer to student and technician to student ratio was 1:10. Figure1shows the flow chart of the study.

样本大小的计算

样本大小的计算was based on a known population. Since the population of interest to be used within this study were 40 undergraduate students from the third-year class of 2019, thus, it was decided to invite the whole class. Following voluntary informed consent, the class was divided into two equal-sized cohorts of students.

Study cohorts

Cohort 1 (live demonstration)

In the presence of a lecturer, trained technicians conducted the LD, with ten students encircling each technician. Students were free to ask questions during the LD and the technicians repeated the wire-bending steps as needed. An ideally prepared wire for each type of component was given to the students as a reference model during the sessions. LD sessions were scheduled a week before FC sessions to avoid exposure to flipped teaching videos. The LD students were given a week after each demonstration session to fabricate and submit their wire-bending product for blinded quality assessment. As part of continuous formative assessment, the scores and marking rubric of each type of wire component were returned to the students who scored below 60 %, as well as to the lecturers and technicians, before the next session. This enabled the students to ascertain their learning progress and use the opportunity to improve where necessary in the upcoming teaching session. A 60 % passing grade was set as an indicator of the minimum knowledge and skills required for the fabrication of orthodontic wire components [2]. Teachers (technicians and lecturers) closely monitored the students who scored below 60 % in subsequent LD sessions. This was repeated for all the lessons involving the six different types of wire components.

Cohort 2 (flipped classroom)

The same calibrated technicians who conducted the LD sessions pre-recorded demonstration videos for all the six wires exercises. These videos were then posted for the FC group on the university’s online learning platform once the LD group had submitted their task for each type of the wires. The FC group similarly had a week to view the video and practice wire-bending for the component assigned, before their teaching session. During the teaching session, students were free to ask the technician questions and to refer to the ideal wire. Students were required to apply the knowledge that they comprehended from watching the videos and submit their wire-bending product for blinded quality assessment at the end of the teaching session. During this session, a similar method of continuous formative assessment was employed for this cohort.

All submissions from both cohorts were blinded for grading using codes of numbers from 1 to 40, allocated by one of the researchers who was not involved in the grading. Each type of wire was graded by the same examiner.

Error in methods

Prior to the video recording, demonstration sessions and grading, the lecturers and technicians had training sessions on each item of the rubrics and on each wire-bending component to eliminate any variation in the teaching and assessment. Each wire component was assessed by one blinded assessor. The assessors were calibrated for intra-observer reliability. Rubrics for all the six wire components were based on specific criteria ensuring the best efficiency of the fabricated component, in relation to the purpose of the components and with the least complication when utilised within an orthodontic removable appliance as required in our curriculum (Table1). A score of one was given for each item of the rubrics if the wire-bending criteria were achieved and zero when not achieved. The total marks for each type of wire component were then scaled over ten regardless of the number of items in each rubric. To ensure reproducibility of the marks, every type of wire component from both cohorts was graded by one orthodontist and the intra-observer calibration for six assessors was undertaken by repeating the marking on ten wires two weeks apart prior to the final grading.

Questionnaire

A validated self-reported questionnaire adopted from Lau et al. (2019), was distributed to the students after the submission of the first wire component (T0) and the last wire component (T1) to assess their preference and satisfaction towards their learning experience [9]. The questionnaire consisted of demographic questions followed by twenty-nine items grouped under six domains: (1) Infrastructure and materials provided; (2) Demonstration method; (3) Teaching method; (4) Wire-bending task; (5) Teaching efficiency; (6) and Overall satisfaction. Responses were recorded using a 5-point Likert scale fromstrongly agree (score 1)tostrongly disagree (score 5), where lower scores would indicate stronger agreement. The T1 questionnaire had additional questions for the FC cohort to assess their usage and perception of the videos.

Statistical analysis

Statistical Package for Social Sciences (SPSS) Version 25.0 was used to analyse the quantitative data. Intra-examiner reliability was tested using Intraclass Correlation Coefficient (ICC). Normality was tested using Shapiro-Wilk test. The scores of wire-bending were normally distributed. Scores for the six types of wire-bending components between the two cohorts were assessed using independent t-test. A one-way repeated measure analysis of variance (RMANOVA) was conducted to analyse separately the orthodontic wire-bending scores for FC and LD cohorts with continuous formative assessment over the six wire-bending sessions. Descriptive statistics were used to report the usage of the video. Spearman’s correlation was used to test the association of video usage to students’ wire-bending skills. Independent t-test and Mann-Whitney U test were used to compare student’s perceived satisfaction of teaching methods on the wire-bending demonstration for normally and not normally distributed data, respectively. The alpha level of significance was set at 0.05.

The intra-examiner ICC for all the six examiners was greater than 0.90 indicating excellent agreement within the examiners for each of the six components.

Thirteen (32.5 %) male and 27 (67.5 %) female students participated in this study. The average age was 21.8 (± 0.2) years. Thirty-eight (95 %) were right-handed and only two (5 %) were left-handed.

Orthodontic wire-bending scores

A comparison of the mean of the wire-bending scores is presented in Table2. The mean wire-bending scores for FC were significantly higher than LD for two of the six components, namely the Adams clasp (p < 0.01) and Z-spring (p = 0.03). There was no statistically significant difference in the mean scores for other components between the two cohorts.

The association of continuous formative assessment with orthodontic wire-bending scores

Pairwise comparisons of wire-bending scores with the implementation of formative assessment for both cohorts are presented in Table3. Both LD and FC cohorts showed significant increase in wire-bending scores over the six wire-bending exercises [LD {F (3.589, 68.190) = 38.550, P < 0.0001}, FC {F (3.132, 59.509) = 19.917, P < 0.0001}]. Follow-up comparisons indicated that most pairwise differences were statistically significant (p < 0.05) for both cohorts. However, with Bonferroni adjustment, for both cohorts, no significant differences were found in their orthodontic wire-bending scores between certain wire combinations as shown in Table2.

The frequency and correlation between usage of online video demonstration with students’ orthodontic wire-bending scores

Table4shows the frequency of the students in the FC cohort in utilising the wire-bending videos. There was a mixed distribution of students who watched the videos in their full length, repeated watching or used the playing features of the video and practised the wire-bending procedures alongside the videos. In terms of their satisfaction with the quality of the videos, there were also reports of inadequate access, lack of satisfaction with the videos, and preferences for videos from other sites. Table4also shows the correlation between the students’ total wire-bending scores with the way they utilised the videos. Watching the entire length of the videos and students’ preference to watch other videos than what was uploaded on their online learning platform had a moderate correlation with the total wire-bending scores (r = 0.384; r = 0.396) but was not statistically significant (p > 0.05). Watching the videos before the wire-bending session had a negative moderate correlation (r = -0.361), which was not statistically significant (p > 0.05) too. Overall, no statistically significant correlation was found between the wire-bending scores and utilisation of videos.

Student’s perceived satisfaction on the wire-bending demonstrations

Student’s perceived satisfaction on the wire-bending demonstration for both cohorts is presented in Table5. T0 questionnaire showed that both cohorts were satisfied with their learning methods, with positive mean scores for all domains. However, there was a significant difference between LD and FC cohorts for two domains; demonstration method video/ live (p < 0.0001); and wire-bending task (p= 0.005)。两人都支持有限的T1,一ll positive satisfaction with LD and FC was maintained. The LD cohort expressed higher favour for the demonstration method received (p与teachi = 0.003)和整体满意度ng method (p = 0.011). However, the FC cohort was significantly more satisfied with the infrastructure and materials received (p < 0.0001).

Orthodontic wire-bending scores

The present study showed the effectiveness of personalised learning for teaching wire-bending skills either as LD or FC method. The FC method appeared to be the more effective way of teaching orthodontic wire-bending skills for two of the six components – Adams clasps and Z-spring. The former was the first wire tasked to learn by the inexperienced and unskilled students while the latter was one of the most complicated components to bend. Within the similar one-week submission deadline, the FC cohort had access to view the video demonstration and practise throughout the week before the classroom session and was able to attend the said session with prepared questions and some amount of acquired wire-bending skills. However, for the LD cohort, they could only ask questions during the two-hour classroom session where they watched the demonstration for the first time, and this was followed by a week of self-practice without any video guidance. Meanwhile, for the remaining four types of components, which may have required less wire-bending skills, our findings were in agreement with previous studies reporting that both LD and FC are equally effective in transferring the skills of orthodontic wire-bending [2]. The inter-group difference between the mean scores for Adams clasp and Z-spring were statistically significant but these values were not clinically significant.

The association of continuous formative assessment with orthodontic wire-bending scores

In the present study, the students’ performances improved over successive tasks regardless of the teaching method. This could be due to the use of formative assessment. However, the improved scores could be attributed to the students gaining skills, knowledge, confidence, and experience with increasing practise in wire-bending, allowing them to improve in performance over time. Formative assessment promoted personalised learning by providing feedback to the teachers and students to make any required improvement [8]. Students received their score and grade after each wire-bending session, which were also disseminated to the lecturers and laboratory technicians prior to the next demonstration session and enabled them to identify and assist students who required extra guidance. It might also have promoted the students to engage in a self-reflective process and work on improving their wire-bending skills [10].

Although all dental schools have some element of formative assessment in their curriculum, the effectiveness of formative assessment was not frequently investigated, let alone for an undergraduate orthodontic course [11]. Various approaches of formative assessment have been reported to improve dental students’ performance on other aspects of dental education, including the structured clinical operative test (SCOT), tutor assessment following problem-based learning, online formative assessment via online exam questions, mini-objective structured clinical examination (mini-OSCE), direct observation of procedural skills (DOPS), and competency-based formative progress assessment system [12,13,14,15,16,17]. Another new approach that has been attempted and well-received by schools includes a streamlined electronic formative feedback model (FFM) developed by Indiana University School of Dentistry (IUSD) [18]. Multisource feedback from colleagues and patients had been reported to be well-accepted by dental postgraduate students as an effective formative assessment tool to improve on professionalism [19]. The importance of formative assessment has slowly gained considerable attention by both dental schools and students. The result of our study suggested that continuous formative assessment may have an important and positive role in dental education especially in terms of teaching and learning clinical and laboratory skills.

It is observed that certain wire-bending tasks were more difficult and challenging e.g., Southend clasp and Z-spring. This explained the observation where certain pairwise differences on follow-up comparisons were not significantly different.

The frequency and correlation between usage of online video demonstration with students’ orthodontic wire-bending scores

The study demonstrated that instructional videos were not fully optimised by the FC cohort despite the perceived opportunities to learn from the videos at the students’ own time and pace. This may be attributed to their preference for learning styles or lack of complete satisfaction with the videos or access to them. Teachers intending to implement the FC method should bear in mind the challenges of implementing such instructional method. Even though not all FC students fully optimised the videos, the use of the videos had no association with their performance. It might be that the students picked up the concepts of the wire-bending through watching parts that they needed to learn and not necessarily the whole prepared video. These students also had opportunities to ask the teachers during the classroom sessions, which might have helped them understand concepts missing from the video. The finding of this study supports that learning is dynamic and can be taught in different ways, including the FC method. Past studies comparing traditional teaching methods with video-based teaching can achieve the same performance from students if the instructions were well-delivered [2,20,21].

Student’s perceived satisfaction on the wire-bending demonstrations

Both methods, LD and FC are suitable to teach wire-bending in orthodontics as both were received favourably by the students, although LD was rated significantly higher than FC. Students preferring LD may have been due to the limitation of the videos in presenting a three-dimensional procedure. The videos also did not allow direct engagement between the student and the technician before the classroom session. In terms of the tasks given, students initially were inclined towards LD for fabricating their wire components, but later FC was found to be equally accepted. Similarly, only at T1 did the FC cohort find the classroom arrangement to be conducive for the given task compared to LD at the end of the exercise. This may indicate that students require some time to familiarise with flipped learning before truly appreciating and embracing this teaching method [22].

Limitations of the study and recommendations

Ideally, a control group with students undergoing the orthodontic wire-bending sessions without formative assessment should be included to investigate the effectiveness of the formative assessment. However, it was deemed ethically inappropriate to withhold any student from the possible benefit of personalised learning in this cohort. Further quantitative or qualitative research into the effect of formative assessment on teaching orthodontic wire-bending is warranted.

A significant number of students were dissatisfied with the quality of videos provided on their online learning platform and some even preferred other similar videos available online. Future studies can use a mixed-method approach, incorporating interviews or focus group discussions with thematic analysis, to explore students’ perception, specifically why they preferred other similar videos online and how to improve the quality of the videos. Another suggestion is to involve students in the development of educational videos to incorporate elements that will appeal to end-users. Effective video development necessitates digital literacy, and additional educational technological support is crucial to assist in appropriate video integration in clinical teaching. Nonetheless, the students’ performances were unaffected by the video quality.

现场演示和翻转课堂equally effective methods of teaching orthodontic wire-bending practical skills. Continuous formative assessment and feedback as a form of personalised learning may have enhanced students’ learning of orthodontic wire-bending skills in both cohorts. Further studies with a control group are recommended to investigate the effect of formative assessment on teaching practical dental skills. FC should be viewed as a complement to the LD and a vehicle for achieving the goals of teaching and learning.

Yes (The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request).

LD:

Live demonstration

FC:

Flipped classroom

The authors would like to acknowledge the technicians who participated in this study. This study was funded by UM Learning Improvement and Teaching Enhancement Research (UMLiter Grant), University of Malaya (RU013U-2019).

Disclosure

The current study received funding from University of Malaya. The authors have no affiliations with or involvement in any organisation or entity with any financial, economic, or professional interests that may have influenced the design, execution or presentation of this scholarly work.

UM Learning Improvement and Teaching Enhancement Research (UMLiter Grant), University of Malaya (RU013U-2019) provided the fund for this research. The funder just provided the fund for the conduct of this research. The funder was not involved in the design of the study and collection, analysis, interpretation of data and writing of the manuscript.

Authors and Affiliations

1. Department of Paediatric Dentistry and Orthodontics, Faculty of Dentistry, University Malaya, Kuala Lumpur, Malaysia

   Saritha Sivarajan, Eunice Xinwei Soh, Nor Nadia Zakaria, Yasmin Kamarudin, May Nak Lau, Aufa D. Bahar & Norhidayah Mohd Tahir

2. Department of Paediatric Dentistry and Orthodontics, Faculty of Dentistry, University Malaya, Kuala Lumpur, Malaysia

   Wan Nurazreena Wan Hassan & Mang Chek Wey

3. Department of Paediatric Dentistry and Orthodontics, Faculty of Dentistry, University Malaya, Kuala Lumpur, Malaysia

   Siti Adibah Othman

4. Department of Paediatric Dentistry & Orthodontics, Faculty of Dentistry, University Malaya, Kuala Lumpur, Malaysia

   Roziana M Razi

5. Dean’s Office Faculty of Dentistry, University Malaya Deputy Head for University Malaya Dental Education Enhancement and Development Unit (UMDEED), Kuala Lumpur, Malaysia

   Zahra Naimie

Contributions

Each authors was involved in conception and design of the work, the acquisition, analysis and interpretation of data, and have drafted specific sections of the manuscript and substantively revised it. Authors also have approved the submitted version and have agreed both to be personally accountable for the author’s own contributions and to ensure that questions related to the accuracy or integrity of any part of the work. All authors contributed to the initial idea, conception of this study, development of the rubrics, supervision of the wire-bending demonstration sessions, data collection and analysis, drafted specific section of the manuscript based on task distributed by the corresponding author and critically reviewed the manuscript. SS lead this study, coordinated the wire-bending demonstration, data collection, analysis, interpretation and delegated the task of manuscript writing among all the members and critically reviewed this manuscript

Corresponding authors

Correspondence toSaritha SivarajanorMay Nak Lau.

Ethics approval and consent to participate

Written ethical approval was obtained from the Medical Ethics Committee, Faculty of Dentistry, University of Malaya. Reference number: DR CD1918/0106 (L). Written informed consent was obtained from all participants.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visithttp://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions