010304 Analyse: Evaluation and Assessment in Learning Design

Warning: The information on this page is indicative. The subject outline for a particular session, location and mode of offering is the authoritative source of all information about the subject for that offering. Required texts, recommended texts and references in particular are likely to change. Students will be provided with a subject outline once they enrol in the subject.

Subject handbook information prior to 2024 is available in the Archives.

UTS: Education: Professional Learning
Credit points: 3 cp
Result type: Grade, no marks

There are course requisites for this subject. See access conditions.

Description

Assessment of learning is a crucial part of effective learning design. Learning designers need to carefully consider how best to evaluate the learning taking place in their course. This subject introduces students to the different forms of assessment, different methods and tools used for assessment and how best to deploy assessment constructively. Students explore the importance of feedback, and how different technologies can assist in the delivery of feedback. Students are also introduced to different models of evaluation of learning designs, as well as considering how these models have developed and in which situations and contexts they might still be appropriate and given the opportunity to apply these models.

Subject learning objectives (SLOs)

a.	Compare and contrast different approaches to assessment, and analyse the appropriate uses for these approaches.
b.	Design effective assessments tasks for learning activities.
c.	Evaluate learning designs using different models to gauge their effectiveness.
d.	Reflect on and evaluate own and others’ learning and practice.
e.	Communicate assessment and/or evaluation information appropriate to the audience, context and purpose.

Course intended learning outcomes (CILOs)

This subject engages with the following Course Intended Learning Outcomes (CILOs), which are tailored to the Graduate Attributes set for all graduates of the Faculty of Arts and Social Sciences.

Research, appraise, devise and implement technology-enhanced learning strategies and tools. (1.1)
Develop and use various designs and tools in the formation of learning practices. (2.1)
Design technology-enhanced learning strategies and tools that are inclusive of racial, gender and class differences. (3.1)
Embed Indigenous Australian knowledges of technology and learning. (4.1)
Collaborate with learning designers, analysts and subject matter experts, using effective English communication skills, to design learning resources. (6.1)

Teaching and learning strategies

Learners engage in online or blended learning. There are a mix of group work activities and discussions, taking place both synchronously (in face to face sessions or online) and asynchronously. During these synchronous/ face to face sessions, there is also an ‘expression session’ hosted by industry or other Learning Design professionals, and the opportunity for students to undertake online labs with specific Learning Designs and technologies. The learning management system (LMS) incorporates a range of other interactive activities, such as quizzes, multimedia presentations, short video and audio recordings and peer critiques. There are regular formative assessments for students to track their progress.

Content (topics)

This subject introduces the differences between formative and summative assessment, and how to feedforward and feedback. In addition, there is discussion about the assessment design, practice and tools, and about evaluation practices and models (including CIPP, Five-Domain, Kirkpatrick’s, Success Case and Utilization-Focused Evaluation). Finally, there is an examination of the application of evaluation models in learning design contexts.

Assessment

Assessment task 1: Assessment Task Creation

Objective(s):

Weight:

60%

Length:

1000 words

Criteria linkages:

Criteria	Weight (%)	SLOs	CILOs
Applicability of assessment task to stated learning outcomes present in the context	35	b	2.1
Level of correlation between outcomes and assessment instruments	35	b	2.1
Suitability of the assessment task for participants	30	b	4.1

SLOs: subject learning objectives
CILOs: course intended learning outcomes

Assessment task 2: Practicing Evaluation for Online Courses

Intent:

Assessment 2 asks you to practice your skills in evaluation in relation to online skills, using a well known framework. You can choose to use EITHER:

An existing online course that is relevant to your personal or professional context OR
This subject (Analyse) as an example of an online course.

You will do the following:

Familiarise yourself with the Technology Enhanced Learning Accreditation Standards (TELAS). These are standard that have been developed to assist in the evaluation of online learning (with a specific focus in higher education).
Complete an evaluation of your chosen course by using the self evaluation tool.
Prepare a commentary describing your own development and reflection upon using the tool.

Objective(s):

a, c, d and e

Weight:

40%

Length:

700 words

Criteria linkages:

Criteria	Weight (%)	SLOs	CILOs
Consideration of the utility of the evaluation tool	40	a, c, d	3.1
�Summary of finding of the evaluation	30	c, d, e	1.1
Appropriateness of justification as professional communication	30	d, e	6.1

SLOs: subject learning objectives
CILOs: course intended learning outcomes

Minimum requirements

Students must complete and pass all assessment tasks in this subject to pass the subject.

Required texts

There are no required texts for this subject. Recommended readings will be available through the LMS. All readings will be open educational resources (so that there is no inequality between award students, micro-credential learners or short course learners).

References

Adrien, M. H., Anderson, G., Carden, F., & Lusthaus, C. (2002). Organizational Assessment: A Framework for Improving Performance. Retrieved from http://reflectlearn.org/recent-oa- publication/organizational-assessment-a-framework-for-improving-performance-0

Andia, L. (2008). Using the BEM for curriculum design: A case study. Performance Improvement, 47(6), 38-44.

Brinkerhoff, R. O. (2003). The success case method: Find out quickly what’s working and what’s not. Berrett-Koehler.

Chen, H.T. (1990). Theory-driven evaluation. Sage.

Chevalier, R. (2010). Gap analysis revisited. Performance Improvement, 49(7), 5-7.

Clark, D. R. (2010). Needs Assessment in Instructional Design. Retrieved 15th October, 2014 from http://nwlink.com/~donclark/hrd/isd/assessment.html

Kirkpatrick, D. L. (2006). Evaluating training programs: The four levels. Berrett-Koehler.

Leigh, D., Watkins, R., Platt, W. A., & Kaufman, R. (2000). Alternate models of needs assessment: Selecting the right one for your organization. Human Resource Development Quarterly, 11(1), 87-93.

Reiser, R. A., & Dempsey, J. V. (Eds.). (2012). Trends and issues in instructional design and technology. Pearson.

Richey, R. C., Klein, J. D., & Tracey, M. W. (2010). The instructional design knowledge base: Theory, research, and practice. Routledge.

Rossi, P.H., Lipsey, M.W., & Freeman, H. E. M. W. (2004). Evaluation: A systemic approach. Sage.

West, R. E. (2018). Foundations of Learning and Instructional Design Technology. EdTech Books.