A defense of student evaluations: They’re biased, misleading, and extremely useful.
The answer requires us to think about power. If you look hard at the structure of academia, you will see a lot of teachers who, in one way or another, lack power: adjuncts and term hires (a large population, and growing); untenured faculty (especially in universities like mine); faculty, even tenured faculty, in schools where budget cuts loom; graduate students, always and everywhere. You might see evaluations as instruments by which students, or administrators, exercise power over those vulnerable employees. But if you are a student, and especially if you are a student who cares what grades you get or who needs recommendations, then teachers, for you—even adjuncts and graduate teaching assistants—hold power.
Chairmen and deans also need to know when classroom teaching fails: when a professor makes catastrophically wrong assumptions as to what students already know, for example, or when students find a professor incomprehensible thanks to her thick Scottish accent. My note: indeed, when chairmen and deans KNOW what they are doing and are NOT using evaluations for their own power.
Student Course Evaluations Get An ‘F’ : NPR Ed : NPR
Philip Stark is the chairman of the statistics department at the University of California, Berkeley. “I’ve been teaching at Berkeley since 1988, and the reliance on teaching evaluations has always bothered me,” he says.
Stark is the co-author of “An Evaluation of Course Evaluations,” a new paper that explains some of the reasons why.
Michele Pellizzari, an economics professor at the University of Geneva in Switzerland, has a more serious claim: that course evaluations may in fact measure, and thus motivate, the opposite of good teaching. Here’s what he found. The better the professors were, as measured by their students’ grades in later classes, the lower their ratings from students.
“Show me your stuff,” Stark says. “Syllabi, handouts, exams, video recordings of class, samples of students’ work. Let me know how your students do when they graduate. That seems like a much more holistic appraisal than simply asking students what they think.”
Giving Classroom Experiences (Like VR) More … Dimension
https://www.insidehighered.com/digital-learning/article/2018/11/02/virtual-reality-other-3-d-tools-enhance-classroom-experiences
at a session on the umbrella concept of “mixed reality” (abbreviated XR) here Thursday, attendees had some questions for the panel’s VR/AR/XR evangelists: Can these tools help students learn? Can institutions with limited budgets pull off ambitious projects? Can skeptical faculty members be convinced to experiment with unfamiliar technology?
All four — one each from Florida International University, Hamilton College, Syracuse University and Yale University — have just finished the first year of a joint research project commissioned by Educause and sponsored by Hewlett-Packard to investigate the potential for immersive technology to supplement and even transform classroom experiences.
Campus of the Future” report, written by Jeffrey Pomerantz
Yale has landed on a “hub model” for project development — instructors propose projects and partner with students with technological capabilities to tap into a centralized pool of equipment and funding. (My note: this is what I suggest in my Chapter 2 of Arnheim, Eliot & Rose (2012) Lib Guides)
Several panelists said they had already been getting started on mixed reality initiatives prior to the infusion of support from Educause and HP, which helped them settle on a direction
While 3-D printing might seem to lend itself more naturally to the hard sciences, Yale’s humanities departments have cottoned to the technology as a portal to answering tough philosophical questions.
institutions would be better served forgoing an early investment in hardware and instead gravitating toward free online products like Unity, Organon and You by Sharecare, all of which allow users to create 3-D experiences from their desktop computers.
+++++++++
Campus of the Future” report, written by Jeffrey Pomerantz
https://library.educause.edu/~/media/files/library/2018/8/ers1805.pdf?la=en
XR technologies encompassing 3D simulations, modeling, and production.
This project sought to identify
- current innovative uses of these 3D technologies,
- how these uses are currently impacting teaching and learning, and
- what this information can tell us about possible future uses for these technologies in higher education.
p. 5 Extended reality (XR) technologies, which encompass virtual reality (VR) and augmented reality (AR), are already having a dramatic impact on pedagogy in higher education. XR is a general term that covers a wide range of technologies along a continuum, with the real world at one end and fully immersive simulations at the other.
p. 6The Campus of the Future project was an exploratory evaluation of 3D technologies for instruction and research in higher education: VR, AR, 3D scanning, and 3D printing. The project sought to identify interesting and novel uses of 3D technology
p. 7 HP would provide the hardware, and EDUCAUSE would provide the methodological expertise to conduct an evaluation research project investigating the potential uses of 3D technologies in higher education learning and research.
The institutions that participated in the Campus of the Future project were selected because they were already on the cutting edge of integrating 3D technology into pedagogy. These institutions were therefore not representative, nor were they intended to be representative, of the state of higher education in the United States. These institutions were selected precisely because they already had a set of use cases for 3D technology available for study
p. 9 At some institutions, the group participating in the project was an academic unit (e.g., the Newhouse School of Communications at Syracuse University; the Graduate School of Education at Harvard University). At these institutions, the 3D technology provided by HP was deployed for use more or less exclusively by students and faculty affiliated with the particular academic unit.
p. 10 definitions
there is not universal agreement on the definitions of these
terms or on the scope of these technologies. Also, all of these technologies
currently exist in an active marketplace and, as in many rapidly changing markets, there is a tendency for companies to invent neologisms around 3D technology.
A 3D scanner is not a single device but rather a combination of hardware and
software. There are generally two pieces of hardware: a laser scanner and a digital
camera. The laser scanner bounces laser beams off the surface of an object to
determine its shape and contours.
p. 11 definitions
Virtual reality means that the wearer is completely immersed in a computer
simulation. Several types of VR headsets are currently available, but all involve
a lightweight helmet with a display in front of the eyes (see figure 2). In some
cases, this display may simply be a smartphone (e.g., Google Cardboard); in other
cases, two displays—one for each eye—are integrated into the headset (e.g., HTC
Vive). Most commercially available VR rigs also include handheld controllers
that enable the user to interact with the simulation by moving the controllers
in space and clicking on finger triggers or buttons.
p. 12 definitions
Augmented reality provides an “overlay” of some type over the real world through
the use of a headset or even a smartphone.
In an active technology marketplace, there is a tendency for new terms to be
invented rapidly and for existing terms to be used loosely. This is currently
happening in the VR and AR market space. The HP VR rig and the HTC Vive
unit are marketed as being immersive, meaning that the user is fully immersed in
a simulation—virtual reality. Many currently available AR headsets, however, are
marketed not as AR but rather as MR (mixed reality). These MR headsets have a
display in front of the eyes as well as a pair of front-mounted cameras; they are
therefore capable of supporting both VR and AR functionality.
p. 13 Implementation
Technical difficulties.
Technical issues can generally be divided into two broad categories: hardware
problems and software problems. There is, of course, a common third category:
human error.
p. 15 the technology learning curve
The well-known diffusion of innovations theoretical framework articulates five
adopter categories: innovators, early adopters, early majority, late majority, and
laggards. Everett M. Rogers, Diffusion of Innovations, 5th ed. (New York: Simon and Schuster, 2003).
It is also likely that staff in the campus IT unit or center for teaching and learning already know who (at least some of) these individuals are, since such faculty members are likely to already have had contact with these campus units.
Students may of course also be innovators and early adopters, and in fact
several participating institutions found that some of the most creative uses of 3D technology arose from student projects
p. 30 Zeynep Tufekci, in her book Twitter and Tear Gas
definition: There is no necessary distinction between AR and VR; indeed, much research
on the subject is based on a conception of a “virtuality continuum” from entirely
real to entirely virtual, where AR lies somewhere between those ends of the
spectrum. Paul Milgram and Fumio Kishino, “A Taxonomy of Mixed Reality Visual Displays,” IEICE Transactions on Information Systems, vol. E77-D, no. 12 (1994); Steve Mann, “Through the Glass, Lightly,” IEEE Technology and Society Magazine 31, no. 3 (2012): 10–14.
For the future of 3D technology in higher education to be realized, that
technology must become as much a part of higher education as any technology:
the learning management system (LMS), the projector, the classroom. New
technologies and practices generally enter institutions of higher education as
initiatives. Several active learning classroom initiatives are currently under
way,36 for example, as well as a multi-institution open educational resources
(OER) degree initiative.37
p. 32 Storytelling
Some scholars have argued that all human communication
is based on storytelling;41 certainly advertisers have long recognized that
storytelling makes for effective persuasion,42 and a growing body of research
shows that narrative is effective for teaching even topics that are not generally
thought of as having a natural story, for example, in the sciences.43
p. 33 accessibility
The experience of Gallaudet University highlights one of the most important
areas for development in 3D technology: accessibility for users with disabilities.
p. 34 instructional design
For that to be the case, 3D technologies must be incorporated into the
instructional design process for building and redesigning courses. And for that
to be the case, it is necessary for faculty and instructional designers to be familiar
with the capabilities of 3D technologies. And for that to be the case, it may
not be necessary but would certainly be helpful for instructional designers to
collaborate closely with the staff in campus IT units who support and maintain
this hardware.
Every institution of higher
education has a slightly different organizational structure, of course, but these
two campus units are often siloed. This siloing may lead to considerable friction
in conducting the most basic organizational tasks, such as setting up meetings
and apportioning responsibilities for shared tasks. Nevertheless, IT units and
centers for teaching and learning are almost compelled to collaborate in order
to support faculty who want to integrate 3D technology into their teaching. It
is necessary to bring the instructional design expertise of a center for teaching
and learning to bear on integrating 3D technology into an instructor’s teaching (My note: and where does this place SCSU?) Therefore,
one of the most critical areas in which IT units and centers for teaching and
learning can collaborate is in assisting instructors to develop this integration
and to develop learning objects that use 3D technology. p. 35 For 3D technology to really gain traction in higher education, it will need to be easier for instructors to deploy without such a large support team.
p. 35 Sites such as Thingiverse, Sketchfab, and Google Poly are libraries of freely
available, user-created 3D models.
ClassVR is a tool that enables the simultaneous delivery of a simulation to
multiple headsets, though the simulation itself may still be single-user.
p. 37 data management:
An institutional repository is a collection of an institution’s intellectual output, often consisting of preprint journal articles and conference papers and the data sets behind them.49 An
institutional repository is often maintained by either the library or a partnership
between the library and the campus IT unit. An institutional repository therefore has the advantage of the long-term curatorial approach of librarianship combined with the systematic backup management of the IT unit. (My note: leaves me wonder where does this put SCSU)
Sharing data sets is critical for collaboration and increasingly the default for
scholarship. Data is as much a product of scholarship as publications, and there
is a growing sentiment among scholars that it should therefore be made public.50
++++++++
more on VR in this IMS blog
https://blog.stcloudstate.edu/ims?s=virtual+reality+definition
Student Entitlement: Key Questions and Short Answers
By: Maryellen Weimer, PhD OCTOBER 4TH, 2017
https://www.facultyfocus.com/articles/teaching-professor-blog/student-entitlement-key-questions-short-answers
What is student entitlement?
“tendency to possess an expectation of academic success without taking personal responsibility for achieving that success.”
How widespread is it?
The research (and there’s not a lot) reports finding less student entitlement than faculty do.
Can a student be entitled without being rude and disruptive?
Yes. Students can have beliefs like those mentioned above and only discuss them with other students or not discuss them at all. Part of what makes entitlement challenging for teachers are those students who do verbally express the attitudes, often aggressively.
Are millennial students more entitled than previous generations? That’s another widely held assumption in the academic community, but support from research is indirect and inconsistent.
Is entitlement something that only happens in the academic environment? No, it has been studied, written about, and observed in other contexts (like work environments
What’s causing it?
A number think it’s the result of previous educational experiences and/or grade inflation. Some blame technology that gives students greater access to teachers and the expectation of immediate responses. Fairly regularly, student evaluations are blamed for the anonymous power and control they give students. And finally, there’s the rise in consumerism that’s now associated with education. Students (and their parents) pay (usually a lot) for college and the sense that those tuition dollars entitle them to certain things, is generally not what teachers think education entitles learners to receive.
How should teachers respond?
It helps if teachers clarify their expectations with constructive positive language and even more importantly with discussions of the rationales on which those expectations rest. Teacher authority gets most students to follow the rules, but force doesn’t generally change attitudes and those are what need to be fixed in this case.
October 18 for Student Entitlement: Truth, Fiction, or Some of Both and stay tuned for more in-depth information and resources that we’ll make available in Faculty Focus Premium in subsequent weeks.
References: Elias, R. Z. (2017). Academic entitlement and its relationship to cheating ethics. Journal of Education for Business, 92 (4), 194-199.
Greenberger, E., et. al. (2008). Self-entitled college students: Contributions of personality, parenting and motivational factors. Journal of Youth Adolescence, 37, 1193-1204.
++++++++++++++++++
Responding to Disruptive Students
Negative attention doesn’t help difficult students change their ways, but teachers can alter classroom dynamics through this exercise.
Mapping Behavior
Draw a map of your classroom, including doors, windows, desks, blackboards—all significant items and areas. I’m sure you’ve already got a clear idea of where the most challenging students usually sit. Now imagine teaching class on a regular day. Trace the paths you usually take across the room. Do you sometimes speed up for a particular reason?
Now put your breathing on the map. Are you conscious of the way you breathe during class? Use a new color and draw a wavy line on top of the lines and arrows you’ve already sketched. Does the wavy line look even, or have you drawn some chaotic or nervous zigzags? Could it be that you’ve sometimes forgotten to breathe?
Investing time in building physical and emotional familiarity with the learning environment, instead of nervously anticipating disruption, changes the educator’s perspective toward the whole class, their interaction with individual students, and their self-awareness. Negative attention stops being a solution—instead it is seen as a hindrance to the process of understanding students’ needs.
Tobin, T. J., Mandernach, B. J., & Taylor, A. H. (2015). Evaluating Online Teaching: Implementing Best Practices (1 edition). San Francisco, CA: Jossey-Bass.
- 5 measurable faculty competencies for on line teaching:
- attend to unique challenges of distance learning
- Be familiar with unique learning needs
- Achieve mastery of course content, structure , and organization
- Respond to student inquiries
- Provide detailed feedback
- Communicate effectively
- Promote a safe learning environment
- Monitor student progress
- Communicate course goals
- Provide evidence of teaching presence.
Best practices include:
- Making interactions challenging yet supportive for students
- Asking learners to be active participants in the learning process
- Acknowledging variety on the ways that students learn best
- Providing timely and constructive feedback
Evaluation principles
- Instructor knowledge
- Method of instruction
- Instructor-student rapport
- Teaching behaviors
- Enthusiastic teaching
- Concern for teaching
- Overall
8. The American Association for higher Education 9 principle4s of Good practice for assessing student learning from 1996 hold equally in the F2F and online environments:
the assessment of student learning beings with educational values
assessment is most effective when it reflects an understanding of learning as multidimensional, integrated and revealed in performance over time
assessment works best when the programs it seeks to improve have clear, explicitly stated purposes.
Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes.
Assessment works best when it is ongoing, not episodic
Assessment fosters wider improvement when representatives from across the educational community are involved
Assessment makes a difference when it begins with issues of use and illumines questions that people really care bout
Assessment is most likely to lead to improvements when it is part of the large set of conditions that promote change.
Through assessment, educators meet responsibilities to students and to the public.
9 most of the online teaching evaluation instruments in use today are created to evaluate content design rather than teaching practices.
29 stakeholders for the evaluation of online teaching
- faculty members with online teaching experience
- campus faculty members as a means of establishing equitable evaluation across modes of teaching
- contingent faculty members teaching online
- department or college administrators
- members of faculty unions or representative governing organizations
- administrative support specialists
- distance learning administrators
- technology specialists
- LMS administrators
- Faculty development and training specialists
- Institutional assessment and effectiveness specialists
- Students
Sample student rating q/s
University resources
Rate the effectiveness of the online library for locationg course materials
Based on your experience,
148. Checklist for Online Interactive Learning COIL
150. Quality Online Course Initiative QOCI
151 QM Rubric
154 The Online Insturctor Evaluation System OIES
163 Data Analytics: moving beyond student learning
- # of announcments posted per module
- # of contributions to the asynchronous discussion boards
- Quality of the contributions
- Timeliness of posting student grades
- Timelines of student feedback
- Quality of instructional supplements
- Quality of feedback on student work
- Frequency of logins
- 180 understanding big data
- reliability
- validity
- factor structure
187 a holistics valuation plan should include both formative evaluation, in which observations and rating are undertaken with the purposes of improving teaching and learning, and summative evaluation, in which observation and ratings are used in order to make personnel decisions, such as granting promotion and tenure, remediation, and asking contingent faculty to teach again.
195 separating teaching behaviors from content design
+++++++++++++++++
more on online teaching in this IMS blog
https://blog.stcloudstate.edu/ims?s=online+teaching
