99223 Monitoring, Evaluation and Learning

Warning: The information on this page is indicative. The subject outline for a particular session, location and mode of offering is the authoritative source of all information about the subject for that offering. Required texts, recommended texts and references in particular are likely to change. Students will be provided with a subject outline once they enrol in the subject.

Subject handbook information prior to 2024 is available in the Archives.

Description

Monitoring, Evaluation and Learning is a key part of project and policy design and implementation in government, civil society organisations and businesses. Students gain skills and knowledge to enable them to address questions such as the following: How do you know if a sustainability initiative has worked? Whether it worked or not, how do you and your organisation learn from the experience? In this subject, students draw on their understanding from studies in different disciplines, both humanistic and biophysical, to think about what kinds of indicators best enable evaluation of progress towards meeting sustainability objectives. Indicators used for sustainability can be as diverse as water quality, stakeholder satisfaction, social equity, fuel use intensity, and health statistics. Students analyse what kinds of data and methods are best suited to evaluating progress towards sustainability objectives, and how evaluation can best feed back into the policy/project cycle, to ensure effective organisational learning.

Subject learning objectives (SLOs)

Course intended learning outcomes (CILOs)

This subject engages with the following Course Intended Learning Outcomes (CILOs), which are tailored to the Graduate Attributes set for all graduates of the Faculty of Arts and Social Sciences (INT = International Studies CILOs):

• Employ teamwork and independent insight to understand and operationalise sustainability objectives (SE.1.1)
• Evaluate and assess the impact of sustainability initiatives (SE.1.2)
• Employ knowledge from and about local and international contexts to account for the differential impacts of sustainability needs and solutions (SE.3.1)
• Value indigenous knowledges as relevant precedents to understand the human-nature coexistence in Australian and international settings (SE.4.1)
• Communicate complex concepts clearly and effectively to a variety of audiences (SE.6.1)

Teaching and learning strategies

Teaching and learning activities are in two main forms:

1. Classes of varying lengths throughout the session, including lectures, seminar discussions and assignment workshops
2. Readings and other preparation for classes and assignments conducted outside class time

Studio WIL project

Studio teaching for the WIL project occurs mainly in classroom workshops, supported by set readings and other self-learning by students. The workshops include short presentations, guest lectures, structured individual and group learning activities, and small and large group discussions.

Prior to the beginning of the teaching session the subject coordinator arranges one or more real-world monitoring, evaluation and learning (MEL) problems from an organisation for students to tackle. Students work on the problem, and around a month later present works-in-progress solutions to the main contact person, who gives verbal feedback. Students incorporate the feedback and finalise reports on the MEL problem to present to the organisation contact person at the end of the teaching session.

Teaching staff workshop the studio problem with students in the face-to-face sessions. For example, a tutor may make a short presentation on a part of the WIL project, then ask students to build on that presentation by working individually or in groups to start researching and drafting relevant material for the work-integrated learning (WIL) project for a period of time, then have several students present their work to the group, have the group give feedback to those students, and so on.

Weekly readings and other self-learning activities

Understanding of MEL is fostered through active reading and watching activities supported by curated academic readings, media, websites, and videos. Student responses to the materials, will be discussed in classes with teaching staff, building on the self-learning conducted by students in preparation before class.

One low-stakes assessment task is included in the form of a quiz due in week 4. Student comprehension of the self-learning materials will be tested again with a short answer test at the end of the session.

Content (topics)

During the studio workshops students and tutors work on a real-world MEL problem brought to the class by a professional who works in MEL. At the end of the term students will present and submit a written report addressing the problem (Assessment task 3). The studio workshops will also include guest talks and Q&A sessions from other professionals working in the MEL area.

The weekly readings self-learning part of the subject builds towards the assessment tasks. The subject is made up of three main parts that build and are woven together during the session:

1. Overview – the main elements of MEL
2. Cases – MEL in practice
3. Complexities – the ethics of MEL, implications of MEL in terms of social inclusion, questions about what types of knowledge are most appropriate and not in MEL, and why

Overview

This practical part of the course includes what kinds of indicators and data can be used to track progress for different kinds of objectives. Content includes:

• The MEL systems embodied in the Sustainable Development Goals (SDGs) and reporting systems for those
• Differences between types of data and methods of analysis, including qualitative and quantitative, and which are most appropriate for what kinds of objectives, and why
• Participatory methods for MEL are foregrounded

Cases

Examples are used from around the world, giving students an international and intercultural appreciation of MEL.

Complexities

• Social diversity and MEL – the ways in which MEL practices can work for and against gender equality and diversity, cultural diversity, racism and colonialism, and ableism.
• Diverse knowledge systems and MEL – ways in which MEL practices can be expanded out from conventional scientific and statistical methods to include other knowledge systems, including local ecological knowledge and Indigenous knowledge systems.
• The ethics of MEL, including conflicts of interest, power dynamics, transparency, accountability, privacy and confidentiality.
• Considering MEL as a practice that renders people and issues legible to government and the implications of this. Certainly people and issues need to be legible to be addressed in policy, but there are also risks for marginalised groups. Two social theory concepts that illustrate these risks are Michel Foucault’s concept of ‘discipline’ as discussed in his later work Security, Population Territory, and James Scott’s Seeing like a State (see Indicative reference list for details).

Assessment

Assessment task 1: Quiz – Objectives, Indicators, Data

Assessment task 2: Studio MEL Project: Stakeholder Report

Assessment task 3: Presentation and Report – Studio MEL Project

Assessment task 4: Essay – MEL for Sustainability and Indigenous Peoples

Assessment task 5: Short Answer Test on Subject Content

Required texts

Required readings are made available via UTS Library and Canvas. Students are not required to purchase additional texts.

References

This is a list of indicative references that covers the kind of material we will cover in the subject. For required and suggested readings for weekly activities, workshops and assignments, see Canvas.

BetterEvaluation. (2021). BetterEvaluation. Website. betterevaluation.org/en

Chaplowe, S. G., & Cousins, J. B. (2015). Monitoring and Evaluation Training: A Systematic Approach. SAGE.

Foucault, M. (2008). Security, Territory, Population. Picador.

Gopichandran, V., & Krishna, A. K. I. (2013). Monitoring “monitoring” and evaluating “evaluation”: An ethical framework for monitoring and evaluation in public health. Journal of Medical Ethics, 39(1), 31–35. https://doi.org/10.1136/medethics-2012-100680

Kourantidou, M., Hoover, C., & Bailey, M. (2020). Conceptualizing indicators as boundary objects in integrating Inuit knowledge and western science for marine resource management. Arctic Science, 6(3), 279–306. https://doi.org/10.1139/as-2019-0013

Mangubhai, S., M., Donato-Hunt, C., & Kleiber, D. (2021). Monitoring evaluation and learning. In K. Barclay, S. Mangubhai, B. Leduc, C. Donato-Hunt, N. Makhoul, J. Kinch, & J. Kalsuak (Eds.), Pacific handbook for gender equity and social inclusion in coastal fisheries and aquaculture (2nd ed., p. 80). Pacific Community (SPC).

Markiewicz, A., & Patrick, I. (2015). Developing Monitoring and Evaluation Frameworks (1st ed.). SAGE.

Mueller-Hirth, N. (2012). If You Don’t Count, You Don’t Count: Monitoring and Evaluation in South African NGOs. Development and Change, 43(3), 649–670. https://doi.org/10.1111/j.1467- 7660.2012.01776.x

Prasad, A. (2018). Environmental performance auditing in the public sector: enabling sustainable development. Taylor & Francis.

Scott, J. C. (1998). Seeing Like a State: How certain schemes to improve the Human Condition Have Failed. Yale University Press.

Shaxson, L. (2016). Lessons for Building and Managing an Evidence Base for Policy (No. 10; Knowledge Sector Initiative (KSI)). http://www.ksi- indonesia.org/index.php/publications/2016/05/25/87/lessons-for-building-and-managing-an- evidence-base-for-policy.html

Singh, S., Holvoet, N., & Pandey, V. (2018). Bridging sustainability and Corporate Social Responsibility: Culture of Monitoring and Evaluation of CSR initiatives in India. Sustainability, 10(2353). https://doi.org/10.3390/su10072353

Starling, S. (2010). Monitoring and evaluating advocacy: Lessons from Oxfam GB’s climate change campaign. Development in Practice, 20(2), 277–286. https://doi.org/10.1080/09614520903564215

Sweetman, C., & Bowman, K. (Eds.). (2018). Gender, Monitoring, Evaluation and Learning. Practical Action Publishing.

Torrigiani, C. (2016). Evaluation and Social Capital: A Theory-Driven and Participatory Approach. Journal of the Knowledge Economy, 7(1), 248–258. https://doi.org/10.1007/s13132-014-0213-8