Tag Archives: assessment

Badges: Learning Gain or Just a Game, and what’s wrong with that?

Badges are Digital image files with text metadata stating criteria for which the badge has been earned. Badges are (presently) self-certified by Learner or Earner and  Self-certified by Provider or Issuer. Below are resources for a short session I ran for the Technology Experimentation Group (TEG). Continue reading

A note on content, courses/curricula, and credentials

This note recounts a potted recent history of developments to do with online content and courses and speculates about the future of credentials in respect of the purpose of a university.

When learning management systems (LMS) or virtual learning environments (VLEs) were in their infancy around the turn of the century, faculty opposition to their introduction was sometimes expressed. Two reasons were often given (among others): if I put my lecture notes on the web no one will come to my class; and variations on a personal IPR theme: the students, the university or third party institutions will steal my content. Content was considered “king” and as long as universities and academics “owned” the content, their position was secure

MIT, with the Open Courseware initiative shattered the content-is-king myth. All the content from a leading university was made freely available: curricula, syllabus, reading lists, slide sets and exam papers.

The defensive focus shifted to courses. It wasn’t the content per se that was important. What the faculty and the institution did was select, organise, interpret, analyse and re-present content through curricula presented in courses (or modules, blocks, units, etc): sequenced events of limited duration (often a semester) presented in various modes (face to face and at distance)..

In 2007 groups of academics started to offer open online courses, hosted at universities but not requiring enrolment or a fee. This open online course movement became truly massive (MOOC) in 2011 when Stanford and MIT began to offer open online courses. From these beginnings spin out ventures (Coursera, Udacity, EdX, FutureLearn and others) started offering open online courses to the higher education “market”.

Now academic defensiveness has shifted. It is not the content and it is not the course or the curriculum that are offered uniquely by the university and faculty. It is the credential. Universities can award degrees and the degree, backed up by quality assurance processes is the guarantor of learning quality and the unique proposition which protects the value of the university.

Is it?

There is a movement towards micro-credentialling, or “badges” which I suggest is more important that many faculty and academics allow. I wrote briefly about badges here. I suggest this movement will continue the trend of opening up the university proposition and further challenge the role of the university in society.

A note on badges

On FSLT13 Badges were  awarded for completion of each of the four activities. Participants who wanted to collect the FSLT13 badgesl needed to register and enrol on the Moodle – AND needed to sign up for a Mozilla Backpack. Badges do not carry any academic credit but are a fun way to signal engagement with the course. Badges were be awarded using the WP Badger plug-in for WordPress, which implements the Mozilla open badges framework, Mozilla Backpack and Persona.

Why badges? We are doing this course to explore some of the developments on the cutting edge of contemporary learning and teaching practice. Badges for lifelong learning are on this rapidly approaching horizon: see Mozilla Open Badges Blog, HASTAC What’s on your badge list, and James Michie’s excellent and balanced presentation on badges on his Open Online Course #crit101.

First thoughts on the final (?) draft Strategy for Enhancing the Student Experience

Oxford Brookes Academic Enhancement and Standards Committee has made its final (?) modifications to the draft Strategy for Enhancing the Student Experience (SESE). The two objectives of this strategy are to:

  • [implement] approaches centred on critical reflection, impact evaluation and continuous enhancement of the student experience.
  • [maximise] student involvement in the development of policies and practices for teaching and learning and in extra-curricular, student led initiatives.

The strategy has principles and guidelines and (eventually) will have maps. Overall the tone is a bit directive: there is a lot of insistance and requirement, but the intentions are (I believe) worthy ones.

From a PCTHE perspective some of the key messages of this document are:

  • All staff who support learning will participate annually in high quality professional development (3.5.2)
  • All academic staff who support learning will engage with processes of evaluation, reflection and research into pedagogic practice (3.5.4)
  • The fundamental purpose of assessment will be to help students learn by providing formative feedback. (3.6.2) (so obvious but so often ignored in practice)
  • Students will be expected to take responsibility for their own learning, to actively engage with feedback and assessment (3.3.1) (this is perhaps one of the key messages of the Assessment Compact)
  • Students [will have the opportunity] to provide input and play a role from the outset in the development of new programmes. (4.3.4)

From a wider educational development perspective:

  • [With our] internationally recognised in-house expertise in educational development, we commit to routinely carrying out impact assessment, review and revision of all significant academic development initiatives and of measures taken forward in the SESE and the consequent strategy maps. (4.4.3.1)

Learning technology messages

  • [People will be] able to use technology to shape their own learning environment and interactions. (3.2.2)
  • [The] curriculum will be enriched by technologies that empower students’ development as self-regulating, digitally literate learners, able to shape their own learning interactions and author their own digital artefacts. (3.4.5)
  • The physical environment will be augmented by digital environments and technologies in ways which support a distinctive Brookes learning experience (3.7.2)
  • The functional access, skills and practices necessary to become a confident, agile adopter of a range of technologies for personal, academic and professional use. To be able to use appropriate technology to search for high-quality information; critically to evaluate and engage with the information obtained; reflect on and record learning, and professional and personal development; and engage productively in relevant online communities. (4.1. d)

Writing a Course Review

I recently met with a student who had been unsuccessful in achieving the criteria for one of the assignments on the PCTHE. Most of my comments are of a general nature regarding the writing of a course review, so I thought I would post them.

  1. Back up your assertions with evidence. As teachers and students in higher education we must be familiar with the many forms that evidence can take. Avoid unsubstantiated statements and in every case where an assertion is made, think of how it might be evidenced.
  2. In general we are interested in your experience, ideas and theories, not your ability to reproduce the theories of others. Theory is simply an answer to the question “why”? That is, one of the principal purposes of theory is explanatory. It does help to recognise the theories of others that came before and have also experienced similar questions and come up with their own explanations. One of the principal theories that underlies this course is Constructive Alignment. Frequently problems in course reviews arise at least in part due to a lack of such alignment. And, a solution might, in part, lie in taking an aligned approach to course and assessment design and implementation. Remember there should be alignment between objectives – or “intended learning outcomes” (what the learners should be able to DO), learning activities in the classroom (or wider teaching environment), which allow the students to practice the objectives, and assessments which allow the students to demonstrate that they can do what ever the objectives suggest. Assessments should be backed up with criteria by which that doing can be evaluated.
  3. Tell the story. Start at the begining:
  • In the first paragraph state the key point of the paper (write this after you have written the rest)
  • I first taught this course …
  • It attracts such and such kind of students. Be as precise as you can with the observable demographics: age range, gender mix, full-time/part-time, in employment or not, etc. Avoid simplistic psychometrics (learning styles, etc) unless you have a particular reason to use them.
  • The course as inherited had these characteristics:stage, CATS points, programme context; external professional bodies (referees); aims (what you and the referees want to achieve); learning objectives (what the students must DO); teaching activities; assessments
  • The course as inherited had these problems (whatever they were). Provide evidence for how you know these were problems: i.e. attendance, participation, engagement, outcome (exam) results, informal feedback, your own observation, discussion with students, discussion with colleagues, etc. Draw on the writings and theories of others to show that these problems are not unique to your situation. That is to say, use outside authorities to show that your problems may be generalisable to a wider context.
  • In light of the students and the problems observed, you have made the following changes: changes implemented during operation in the first year (i.e. through reflection IN practice); changes implemented in a revision (i.e. through reflection ON practice)
  • Evaluation: how will you know the changes have been effective?
  • Next steps

Evaluating blogs and reflection

The assessment or evaluation of critical reflective writing is problematic. Some take it as too personal and subjective and therefore do not presume to judge others’ reflection. On the other hand, in some disciplines reflection is formally incorporated as an assessed component.

I came across two sets of criteria for the assessment of students’ blog writing in the links in a post by Emma Duke Williams, via Stephen Downs’ On Line Daily. Emma questions the value of grading student blogging, but recognises that when used in an educational context it is practically inevitable. Stephen is, as would be expected, rather more robust in his rejection of the assessment of blogging, “Honestly, I think that the whole idea of grading student blogging is to miss the point of blogging. You may say it’s “inevitable” that staff will want to grade blogging – well, I say, don’t cater to that, don’t make it easier, don’t give them metrics – make them do their own work, so that they are completely culpable for ruining writing for children and youth.”

Well, here are the metrics. You decide if they are useful of not:

Both reminded me very closely of the criteria that the OCSLD team came up with for the overall assessment of participants on the Brookes Postgraduate certificate in teaching in Higher Education (PCTHE). I’ll post these shortly.