Integrating MOOCs
in Physics preliminary undergraduate education:
beyond large size lectures.
Juliana Raffaghelli, Patrizia Ghislandi, Susanna Sancassani, Luisa Canal,
Rocco Micciolo, Barbara Balossi, Matteo Bozzi, Laura di Sieno,
Immacolata Genco, Paolo Gondoni, Andrea Pini, Maurizio Zani
ICEM, Napoli 22/09/2017
Teaching lab. (ST2)
Maurizio Zani, Juliana Raffaghelli
2/24
Universities are changing!
Two problems inside this
institutional change are:

the increasing number of students in
higher education

increasing drop-outs (Brown et al.,
2012)
This entails:
Higher teachers' workload

the need of pedagogical innovation

the adoption of
technologies
to
amplify the teachers' activities
Yet working with large size lectures impose new challenges
Maurizio Zani, Juliana Raffaghelli
3/24
The challenge of working with LSL
Content delivery is frequently the
focus
Less often,
the teacher explores
strategies to introduce collaborative
and active pedagogies as in smaller
lectures (Allendoerfer et al., 2016)
These strategies could be supported
by
technology-enhanced
learning
approaches
(Mangram,
Haddix,
Ochanji & Masingila, 2015)
Maurizio Zani, Juliana Raffaghelli
4/24
Educational Technologies
Pedagogical Dimension
MOOC's and OER's integration (blended learning) Access to content with tracking systems
periodically analyzed in class by the teacher
Students' response System
Mass immediate feed-back
Visualizations based on Learning Analytics
(student and teacher side)
Self-regulation: tools for planning, monitoring and
assessing the own progress in learning
Automatic systems for Peer-evaluation
Reflective skills, critical thinking, content
knowledge
Etc.!
Etc.!
Some TEL strategies to promote effective learning in LSL
Content delivery is frequently the focus
Less often, the teacher explores strategies to introduce collaborative and active
pedagogies as in smaller lectures (Allendoerfer et al., 2016)
These
strategies could be supported by technology-enhanced
learning
approaches
These strategies are being studied separately, while there are less efforts to
understand how an integrated set of strategies could work.
Maurizio Zani, Juliana Raffaghelli
5/24
Our research questions
Which elements are particularly important in a setting for LSL?

Is there any pedagogical activity/resource within the learning design for
the LSL that makes a difference?
Which is the importance of MOOCs in this integrated approach?
Maurizio Zani, Juliana Raffaghelli
6/24
Outlook
MOOCs
Pre-courses
Quiz
Team
makes the difference!
Maurizio Zani, Juliana Raffaghelli
7/24
Quiz
Pre-courses
MOOCs of experimental physics
MOOCs
Maurizio Zani, Juliana Raffaghelli
8/24
MOOCs of experimental physics: ciak...
action

team
topics
structure
storyboard
shooting
post-editing
validation
quiz
450 h of work
result
170 video
150 quiz
1 director
team
8 teacher/tutor
7 staff
http://www.mauriziozani.it/wp/?p=2134
start to work!
Team
makes the difference!
Maurizio Zani, Juliana Raffaghelli
9/24
MOOCs of experimental physics: structure and certificate
quiz
quiz
quiz
quiz
week
week
week
week
lessonsexerciseinsights...lessonsexperimentssimulations...interviewcase studiesrole games...lessonssummaryforum...final
quiz
quiz > 60%
Maurizio Zani, Juliana Raffaghelli
10/24
Fisica 2
Fisica 1
MOOCs of experimental physics: topics
Mechanics (3 weeks)

kinematics of the material point and examples
of motions

dynamics of the material point and examples
of motions
work, energy, bumps and universal gravitation
Thermodynamics (2 weeks)

kinematics and dynamics of ideal liquids

temperature, ideal gas, heat, thermal
machines and entropy
Electromagnetism (2 weeks)

electric field, conductors, capacitors and
dielectric materials

electrical current, magnetic field and magnetic
materials
Optics (1 week)

electromagnetic waves, geometrical optics
and wave optics
Hints of modern physics (1 week)
2 courses
17 000 enrolled
http://www.pok.polimi.it
based on
( 45 000 enrolled)
Maurizio Zani, Juliana Raffaghelli
11/24
MOOCs
Pre-courses
Pre-courses in physics
Quiz
Maurizio Zani, Juliana Raffaghelli
12/24
Pre-courses in physics: students enrolled @ PoliMi
2016-2017: 5 teams in Milan + 1 team in Lecco
20%
freshmen (bachelor)
in engineering
Team
makes the difference!
Schools
Students
Freshmen
(bachelor)
In pre-
courses
Architecture
9 915
(23%)
1 201
(16%)
43
(4%)
Design
4 226
(10%)
755
(10%)
7
(1%)
Engineering
29 011
(67%)
5 568
(74%)
1 097
(95%)
total
43 152
7 524
1 147
Maurizio Zani, Juliana Raffaghelli
13/24
http://www.fisi.polimi.it/it/didattica/studenti/corsi_di_ripasso
Pre-courses in physics: structure
Fisica 2
Fisica 1
MOOCs
Lessons
Mechanics
Thermodynamics
Electromagnetism
12 h
8 h
12 h
12 quiz
8 quiz
12 quiz
16 quiz
16 quiz
Quiz
2+1 w
3+2 w
Maurizio Zani, Juliana Raffaghelli
14/24
Quiz
Pre-courses
Quiz on-line in real-time
MOOCs
Maurizio Zani, Juliana Raffaghelli
15/24
Quiz on-line in real-time
Goal

for teacher: check of the classroom status

for student: check of the comprehension (more than the knowledge)

"stimulus" to study the topics
Method
on-line (suitable for a large number of students)

real-time (using smartphone as a clicker)
Implementation

fast (max. 2-3 min), with no particular calculus
anonymous (but logging in with the personal code)
without credits (but with self-assessment & final score on the smartphone)
no discussion after the quiz (to separate "quiz effect" from "tutor effect")
Maurizio Zani, Juliana Raffaghelli
16/24
Summary
Pre-courses
Quiz
MOOCs
2 courses
170 video
150 quiz
17 000 enrolled
120 quiz
6 teams
1 100 enrolled
Maurizio Zani, Juliana Raffaghelli
17/24
LSL approach in Physics pre-courses: overall impact
Quinn Dombrowski (2011) Physics CC-BY 2.0
https://www.flickr.com/photos/25097263@N02/2376187960/
Applying the Student's t test, a
significant improvement in the mean
scores was found, ensuring that the
whole pre-courses were effective for
students' learning as assessed by
tests
Pre-test mean = 4.01
Post-test mean = 5.97

t = 9.78
df = 783
p 0.001
Maurizio Zani, Juliana Raffaghelli
18/24
Results comparing groups: Student's t test
Group 1 Group 2 Group 3 Group 4 Group 5 Group 6
Lecture Size N
211
104
90
112
111
13
Pre-test
Mean
4.03
3.72
3.57
4.33
4.30
3.63
SD
2.92
1.68
1.84
1.46
1.27
1.59
N
36
9
42
15
28
14
Post-test
Mean
6.66
5.03
4.72
5.31
7.21
6.72
SD
2.83
1.78
2.09
1.36
2.06
1.58
Effect size
d
0.90
0.78
0.60
0.68
1.99
1.95
Student's t
test
t
5.00
2.24
3.20
2.46
9.43
5.05
p-value
< 0.001
0.028
0.002
0.015
< 0.001 < 0.001
Maurizio Zani, Juliana Raffaghelli
19/24
Group 1 Group 2 Group 3 Group 4 Group 5 Group 6
Lecture Size N
211
104
90
112
111
13
Pre-test
Mean
4.03
3.72
3.57
4.33
4.30
3.63
SD
2.92
1.68
1.84
1.46
1.27
1.59
N
36
9
42
15
28
14
Post-test
Mean
6.66
5.03
4.72
5.31
7.21
6.72
SD
2.83
1.78
2.09
1.36
2.06
1.58
Effect size
d
0.90
0.78
0.60
0.68
1.99
1.95
Student's t
test
t
5.00
2.24
3.20
2.46
9.43
5.05
p-value
< 0.001
0.028
0.002
0.015
< 0.001 < 0.001
Results comparing groups: effect size
Effect size range values: d = 0.2 'small' 0.5 'medium' > 0.8 'large'
Maurizio Zani, Juliana Raffaghelli
20/24

large size lectures can reach similar or even better results than
smaller ones

(one large lecture -5- obtained the best effect size)
presumably due to the whole learning design (MOOCs + active
learning + self-monitoring and assessment) or some of its specific
elements.
A-ha... but what is
the impact of each
element?
According to these results...
Maurizio Zani, Juliana Raffaghelli
22/24
Survey on Students' satisfaction
Group 1 Group 2 Group 3 Group 4 Group 5 Group 6
Lecture Size N
211
104
90
112
111
13
Pre-test
Mean
4.03
3.72
3.57
4.33
4.30
3.63
SD
2.92
1.68
1.84
1.46
1.27
1.59
N
36
9
42
15
28
14
Post-test
Mean
6.66
5.03
4.72
5.31
7.21
6.72
SD
2.83
1.78
2.09
1.36
2.06
1.58
Effect size
d
0.90
0.78
0.60
0.68
1.99
1.95
Student's t
test
t
5.00
2.24
3.20
2.46
9.43
5.05
p-value
< 0.001
0.028
0.002
0.015
< 0.001 < 0.001
Statistic Test: One-Way Anova on mean scores per dimension
Dimension
Group
1
Group
2
Group
3
Group
4
Group
5
Group
6
F
p-
value
MOOCs
N
62
17
51
27
30
11
Mean
3.27
3.47
3.39
3.37
3.13
3.36
0.98
0.43
Active
Learning
N
84
27
69
35
43
14
Mean
3.52
2.89
3.23
2.86
2.81
3.64
9.41 <0.001
Self-
assessment
quizze
N
86
31
75
35
45
14
Mean
2.53
3.61
2.56
2.77
2.58
2.57
0.45
0.81
Tutor
N
86
31
75
35
45
14
Mean
3.69
2.65
3.32
2.77
2.82
3.79
15.26 <0.001
Maurizio Zani, Juliana Raffaghelli
22/24
pedagogical methods and teaching styles still make a difference!
technologies just help
but some technologies help more: the case of MOOCs
According to these results...
A-ha... well, now
tell us what works
of MOOCs!
Maurizio Zani, Juliana Raffaghelli
23/24
Discussion & Conclusions
The overall active learning approach was perceived in positive way, with
significant differences between groups.
The differences cannot be attributed to the class size.
The students' acceptance of the different technologies offered is diversified, but
the integration of MOOCs was considered positive by the students.
From our results we could cautiously infer that the teachers' abilities to handle
the number of participants in active learning as well as the whole quality of the
pedagogical approach are more important than the number of students' itself.
Teaching quality and pedagogical approach depends on institutional and team
efforts
In LSL, the influence of technology has to be carefully studied in relationship with
teaching as complex variable (the teacher, the teams, the institution).
Maurizio Zani, Juliana Raffaghelli
24/24
Thanks to teachers, tutors, staff:
Daniele Albricci, Barbara Balossi, Federico
Bottegoni,
Matteo
Bozzi,
Federica
Brambilla, Gianlorenzo Bussetti, Davide
Contini, Laura Di Sieno, Gianfranco Elia,
Immacolata Genco, Patrizia Ghislandi**,
Paolo Gondoni, Cristian Manzoni, Andrea
Parisi, Andrea Pini, Dario Polli, Juliana
Raffaghelli*, Marco Re, Rebecca Re,
Susanna Sancassani, Cristina Varisco,
Caterina Vozzi
Thanks for your attention!
Team makes the difference!
MOOCsPre-coursesPedagogyPolitecnico di Milano
Universit di Firenze*
Universit di Trento**