Searching for "online evaluation"
company or group who is doing online program evaluation?
this information is extracted from the Blend-Online discussion list
Do you know any company or group who is doing online program evaluation? Our school is seeking a consulting group to come to review our online programs and identify areas relate to online learning that we need to improve.
Carrie Halpin, Ph.D. Professor/Instructional Designer & Technologist eLearning & Instructional Technology (eLIT) Virginia Western Community College 3095 Colonial Ave. SW, Roanoke, VA 24015 Office: Brown Library 102 Phone: 540-857-6636 Fax: 540-857-6138 Email: email@example.com
will do both of those things using a fee-for-service model versus a revenue-sharing model like Academic Partnerships or 2U. I have no personal experience with any OPM, but iDesign is the only one I know of that offers that ala carte type service.
more on evaluations regarding online teaching in this IMS blog
Tobin, T. J., Mandernach, B. J., & Taylor, A. H. (2015). Evaluating Online Teaching: Implementing Best Practices (1 edition). San Francisco, CA: Jossey-Bass.
- 5 measurable faculty competencies for on line teaching:
- attend to unique challenges of distance learning
- Be familiar with unique learning needs
- Achieve mastery of course content, structure , and organization
- Respond to student inquiries
- Provide detailed feedback
- Communicate effectively
- Promote a safe learning environment
- Monitor student progress
- Communicate course goals
- Provide evidence of teaching presence.
Best practices include:
- Making interactions challenging yet supportive for students
- Asking learners to be active participants in the learning process
- Acknowledging variety on the ways that students learn best
- Providing timely and constructive feedback
- Instructor knowledge
- Method of instruction
- Instructor-student rapport
- Teaching behaviors
- Enthusiastic teaching
- Concern for teaching
8. The American Association for higher Education 9 principle4s of Good practice for assessing student learning from 1996 hold equally in the F2F and online environments:
the assessment of student learning beings with educational values
assessment is most effective when it reflects an understanding of learning as multidimensional, integrated and revealed in performance over time
assessment works best when the programs it seeks to improve have clear, explicitly stated purposes.
Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes.
Assessment works best when it is ongoing, not episodic
Assessment fosters wider improvement when representatives from across the educational community are involved
Assessment makes a difference when it begins with issues of use and illumines questions that people really care bout
Assessment is most likely to lead to improvements when it is part of the large set of conditions that promote change.
Through assessment, educators meet responsibilities to students and to the public.
9 most of the online teaching evaluation instruments in use today are created to evaluate content design rather than teaching practices.
29 stakeholders for the evaluation of online teaching
- faculty members with online teaching experience
- campus faculty members as a means of establishing equitable evaluation across modes of teaching
- contingent faculty members teaching online
- department or college administrators
- members of faculty unions or representative governing organizations
- administrative support specialists
- distance learning administrators
- technology specialists
- LMS administrators
- Faculty development and training specialists
- Institutional assessment and effectiveness specialists
Sample student rating q/s
Rate the effectiveness of the online library for locationg course materials
Based on your experience,
148. Checklist for Online Interactive Learning COIL
150. Quality Online Course Initiative QOCI
151 QM Rubric
154 The Online Insturctor Evaluation System OIES
163 Data Analytics: moving beyond student learning
- # of announcments posted per module
- # of contributions to the asynchronous discussion boards
- Quality of the contributions
- Timeliness of posting student grades
- Timelines of student feedback
- Quality of instructional supplements
- Quality of feedback on student work
- Frequency of logins
- 180 understanding big data
- factor structure
187 a holistics valuation plan should include both formative evaluation, in which observations and rating are undertaken with the purposes of improving teaching and learning, and summative evaluation, in which observation and ratings are used in order to make personnel decisions, such as granting promotion and tenure, remediation, and asking contingent faculty to teach again.
195 separating teaching behaviors from content design
more on online teaching in this IMS blog
Gunawan, F. (2018). GAMIFICATION ANALYSIS AND IMPLEMENTATION IN ONLINE LEARNING. ICIC Express Letters, 12(12), 1195–1204.
Khan  has introduced the eight-dimensional elearning framework, a detailed self assessment instrument for institutions to evaluate the readiness and the opportunity of their e-learning classes to grow.
institutional, management, technological, pedagogical, ethical, interface design, resource support, and evaluation. Institutional refers to the administrative and academic part of the system. Management refers to the quality control, budget, and scheduling. Technological refers to the infrastructure, hardware, and software. Pedagogical refers to analysis, organization and learning strategies. Ethical refers to ethical, legal, and social and political influences. Interface design refers to the user interface, accessibility, and design content. Resource support refers to career services, journals, and online forums. Finally, the evaluation refers to the assessment of learners and educators.
gamification – definition
Modern gamification term was first introduced by
Nick Pelling in 2002 . Gamification is a concept that implements the game components
into the non-game contents such as education, marketing, administration, or even software
engineering . These components include points, badges, leaderboards, and quests.
Each of them serves the purpose to increase the level of user engagement in the learning
three components of engagement: cognitive, behavioral, and emotional .
Weakest students more likely to take online college classes but do worse in them
Protopsalt is is a professor at George Mason University, where he directs Center for Education Policy and Evaluation. He previously served as a senior official in the U.S. Department of Education.
The paper, “Does Online Education Live Up to Its Promise? A Look at the Evidence and Implications for Federal Policy,” was also written by Sandy Baum, an economist at the Urban Institute, a nonprofit research organization.
At four-year universities, students with high grades often did just as well in an online course, but those with low grades suffered more. Another 2017 study of students at a for-profit university which offers both in-person and online classes found that students who took an online class not only got lower grades in that class but also in future classes. Online students were more likely to drop out of college altogether than similar students who attended in-person classes.
The question is whether we should keep expanding online learning, with generous federal subsidies, to the most vulnerable students before colleges have tested and proven they can educate them adequately outside the classroom.
more on online learning in this IMS blog
Advancing Online Education in Minnesota State
Advancing Online Education – Full Report-1s94jfi
Defining Online Education
The term “online education” has been used as a blanket phrase for a number of fundamentally different educational models. Phrases like distance education, e-Learning, massively open online courses (MOOCs), hybrid/blended learning, immersive learning, personalized and/or adaptive learning, master courses, computer based instruction/tutorials, digital literacy and even competency based learning have all colored the definitions the public uses to define “online education.”
online education” as having the following characteristics:
- Students who enroll in online courses or programs may reside near or far from the campus(es) providing the course(s) or program.
- A student’s course load may include offering where attendance is required in person or where an instructor/students are not required to be in the same geographic location.
- Students may enroll in one or more individual online course offerings provided by one or more institutions to that may or may not satisfy degree/program requirements.
- Student may pursue a certificate, program, or degree where a substantial number of courses, perhaps all, are taken without being in the same geographic location as others.
Organizational Effectiveness Research Group (OERG),
As the workgroup considered strategies that could advance online education, they were asked to use the primary and secondary sources listed above to support the fifteen (15) strategies that were developed
define a goal as a broad aspirational outcome that we strive to attain. Four goal areas guide this document. These goal areas include access, quality, affordability and collaboration. Below is a description of each goal area and the assumptions made for Minnesota State.
Over twenty percent of existing Minnesota State students enroll in online courses as a way to satisfy course requirements. For some students, online education is a convenient option; for others, online is the only option available
The Higher Learning Commission (HLC) accreditation guidelines review the standards and processes institutions have in place to ensure quality in all of educational offerings, including online.
There are a number of ways in which institutions have demonstrated quality in individual courses and programs including the evaluation of course design, evaluation of instruction and assessment of student
a differential tuition rate to courses that are offered online. If we intend to have online education continue to be an affordable solution for students, Minnesota State and its institutions must be good stewards of these funds and ensure these funds support online education.
Online education requires different or additional services that need to be funded
transparency is important in tuition setting
Distance Minnesota is comprised of four institutions Alexandria Technical & Community College, Bemidji State University, Northland Community & Technical College, and Northwest Technical College) which collaborate to offer student support services, outreach, e-advising, faculty support, and administrative assistance for online education offerings.
strategies are defined as the overall plan used to identify how we can achieve each goal area.
Strategy 1: Ensure all student have online access to high quality support services
students enrolled in online education experiences should have access to “three areas of support including academic (such as tutoring, advising, and library); administrative (such as financial aid, and disability support); and technical (such as hardware reliability and uptime, and help desk).”
As a system, students have access to a handful of statewide services, include tutoring services through Smarthinking and test proctoring sites.
Strategy 2: Establish and maintain measures to assess and support student readiness for online education
A persistent issue for campuses has been to ensure that students who enroll in online course are aware of the expectations required to participate actively in an online course.
In addition to adhering to course expectations, students must have the technical competencies needed to perform the tasks required for online courses
Strategy 3: Ensure students have access to online and blended learning experiences in course and program offerings.
Strategy 4: These experiences should support and recognize diverse learning needs by applying a universal design for learning framework.
The OERG report included several references to efforts made by campuses related to the providing support and resources for universal design for learning, the workgroup did not offer any action steps.
Strategy 5: Expand access to professional development resources and services for faculty members
As online course are developed and while faculty members teach online courses, it is critical that faculty members have on-demand access to resources like technical support and course assistance.
5A. Statewide Faculty Support Services – Minnesota State provide its institutions and their faculty members with access to a centralized support center during extended hours with staff that can assist faculty members synchronously via phone, chat, text/SMS, or web conference
5C. Instructional Design and Technology Services – Establish a unit that will provide course design and instructional technology services to selected programs and courses from Minnesota State institutions.
Strategy 1: Establish and maintain a statewide approach for professional development for online education.
1B. Faculty Mentoring – Provide and sustain faculty mentoring programs that promote effective online pedagogy.
1C. Professional development for support staff – including instructional designers, D2L Brightspace site administrators and campus trainers, etc.)
more on online education in this IMS blog
The Online Discussion group for Blended and Online Learning leads an interesting discussion on course evaluations; here are the highlights:
When we first started in 1999, we included ~10 questions in addition to our standard questions that were different for online courses. This information was particularly useful as we grew our online offerings (i.e. Would you take another online course. 93-5% answered yes consistently. How would you rate the level of interactivity between you and the instructor? Between you and the other students?) These were administered via SurveyMonkey because there were no online evaluation services back then.
Now we have a single evaluation that is administered to all students regardless of the delivery format (online, hybrid, blended, F2F or intensive) The questions were designed to be relevant regardless of the delivery format. All of these evaluations are administered online…which has its downsides (e.g. response rate is less especially compared to what was captured in F2F classes in the past.) We continue to explore ways to increase the response rate.
Reta Chaffee Director of Educational Technology-Academic Affairs Granite State College 25 Hall Street Concord, NH 03301 (603) 513-1350
On Behalf Of Krajewski, Scott
Sent: Friday, February 17, 2017 1:00 PM
Subject: Re: [BLEND-ONLINE] Course Evaluations
Hi Hala, We have a standard online evaluation form for all courses. We do add 3 questions to the sports courses but otherwise we’re 100% standardized. We have a ton of info at
You might find this study (or the related literature) helpful — http://patricklowenthal.com/publications/Student-Perceptions-of-Online-Learning–Analysis-of-Online-Course-Evaluations.pdf
Patrick Patrick R. Lowenthal | Associate Professor Educational Technology, Boise State University http://www.patricklowenthal.com
On Behalf Of Rob Gibson
Sent: Friday, February 17, 2017 8:39 AM
Subject: Re: [BLEND-ONLINE] Course Evaluations
We use the IDEA evaluation framework combined with CampusLabs as the delivery engine.
- IDEA is a well-established evaluation process dating back to the 1970s.
- The CampusLabs delivery process (new as of about 2 years ago) provides students with a single URL to complete their evaluations – on-campus or on-line. Mobile friendly.
- It uses the same base evaluation criteria across the university. (That’s how IDEA is able to substantiate reliability and validity.) IDEA is matched against a national database using a CIP code. Hence, faculty can gather comparative data of their course against other similar courses in the university, or at the national level.
- While each department uses the same basic framework, there are modification that can be made. For example, custom questions can be added to the eval (these fall outside the scope of the comparative data) and the learning objectives can be modified by course, department, school, college. We have one School that has custom learning objectives for each course in their program. Objectives are set using a 3 point Likert scale.
Very easy to set up a survey administration. Data is retrievable within 48 hours after close.
more on online learning in this IMS blog
Discussion on the EDUCAUSE Blended and Online Learning Group’s listserv
develop anonymous mid-course student evaluations allowing students to reflect on course and progress and informing instructor about what is working or not in the course.
– what is working well for you in the course?
– what is not working well for you in the course?
- What is helping you learn?
- What is hindering your learning?
- What suggestions do you have to make the course better for you, your peers, or the instructor?
Katie Linder Research Director Extended Campus, Oregon State University 4943 The Valley Library Corvallis, Oregon 97331 Phone 541-737-4629 | Fax 541-737-2734 Email: firstname.lastname@example.org
At the University of Illinois, we have been using Informal Early Feedback as a way to gauge information from our students to help improve the courses before the end. Here are a couple of links to our site. The first is the main page on what IEF is and the second is the question bank we offer to faculty. This is a starting point for them, then we meet with those who want to work on tweaking them for their specific needs.
* About IEF: https://citl.illinois.edu/citl-101/measurement-evaluation/teaching-evaluation/ief
* Question Bank: https://citl.illinois.edu/citl-101/measurement-evaluation/teaching-evaluation/ief/ief-question-bank
If you have any questions at all, don’t hesitate to ask.
Sol Roberts-Lieb Associate Director, Center for Innovation in Teaching and Learning Pedagogy Strategy Team and Industry Liaison UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN
more on student evaluations in this IMS blog:
Moving beyond smile sheets: A case study on the evaluation and iterative improvement of an online faculty development program
The eCampus Quality Instruction Program (eQIP) is an online faculty development program developed to train faculty in designing and teaching fully online courses.
What is the best way to design and develop high- quality online courses and support faculty as they teach online?
Given faculty’s competing priorities and limited time, we contend that it is important for institutions, and specifically faculty developers, to analyze how much time faculty are spending in online faculty development activities as well as which parts are taking the most (or least) time. (p. 5)
A successful online faculty development program must include pedagogical support, technology support, and design and development support (Baran & Correia, 2014) that overcome obstacles about time, expertise, and motivation of faculty (Henning, 2012).(p. 17)
more on online learning in this IMS blog
Iran’s blogfather: Facebook, Instagram and Twitter are killing the web
is it possible that the Iranian government realized the evolution of social media and his respective obsolescence and this is why they freed him prematurely?
Blogs were gold and bloggers were rock stars back in 2008 when I was arrested.
The hyperlink was a way to abandon centralisation – all the links, lines and hierarchies – and replace them with something more distributed, a system of nodes and networks. Since I got out of jail, though, I’ve realised how much the hyperlink has been devalued, almost made obsolete.
Nearly every social network now treats a link as just the same as it treats any other object – the same as a photo, or a piece of text. You’re encouraged to post one single hyperlink and expose it to a quasi-democratic process of liking and plussing and hearting. But links are not objects, they are relations between objects. This objectivisation has stripped hyperlinks of their immense powers.
At the same time, these social networks tend to treat native text and pictures – things that are directly posted to them – with a lot more respect. One photographer friend explained to me how the images he uploads directly to Facebook receive many more likes than when he uploads them elsewhere and shares the link on Facebook.
Some networks, like Twitter, treat hyperlinks a little better. Others are far more paranoid. Instagram – owned by Facebook – doesn’t allow its audiences to leave whatsoever. You can put up a web address alongside your photos, but it won’t go anywhere. Lots of people start their daily online routine in these cul-de-sacs of social media, and their journeys end there. Many don’t even realise they are using the internet’s infrastructure when they like an Instagram photograph or leave a comment on a friend’s Facebook video. It’s just an app.
A most brilliant paragraph by some ordinary-looking person can be left outside the stream, while the silly ramblings of a celebrity gain instant internet presence. And not only do the algorithms behind the stream equate newness and popularity with importance, they also tend to show us more of what we have already liked. These services carefully scan our behaviour and delicately tailor our news feeds with posts, pictures and videos that they think we would most likely want to see.
Today the stream is digital media’s dominant form of organising information. It’s in every social network and mobile application.
The centralisation of information also worries me because it makes it easier for things to disappear.
But the scariest outcome of the centralisation of information in the age of social networks is something else: it is making us all much less powerful in relation to governments and corporations. Surveillance is increasingly imposed on civilised lives, and it gets worse as time goes by. The only way to stay outside of this vast apparatus of surveillance might be to go into a cave and sleep, even if you can’t make it 300 years.
Do student evaluations measure teaching effectiveness?Manager’s Choice
Mauricio Vasquez, Ph.D.Assistant Professor in MISTop Contributor
Higher Education institutions use course evaluations for a variety of purposes. They factor in retention analysis for adjuncts, tenure approval or rejection for full-time professors, even in salary bonuses and raises. But, are the results of course evaluations an objective measure of high quality scholarship in the classroom?
Dr. Pedro L.