University of Technology Sydney

15643 Applied Evaluation

Warning: The information on this page is indicative. The subject outline for a particular session, location and mode of offering is the authoritative source of all information about the subject for that offering. Required texts, recommended texts and references in particular are likely to change. Students will be provided with a subject outline once they enrol in the subject.

Subject handbook information prior to 2022 is available in the Archives.

UTS: Design, Architecture and Building: Institute for Public Policy and Governance
Credit points: 6 cp
Result type: Grade and marks

There are course requisites for this subject. See access conditions.

Description

This subject introduces students to the foundation skills needed to undertake the evaluation of policies and programs, especially in a public sector context. The subject covers the skills needed to undertake an evaluation as well as the skills needed to manage and implement an evaluation.

This subject is required for students completing an Evidence and Evaluation sub-major within the Graduate Diploma/Master of Applied Policy.

Subject learning objectives (SLOs)

On successful completion of this subject, students should be able to:

1. Understand major evaluation designs, including process, outcome, and economic evaluations
2. Identify the evaluation need from a decision-maker and stakeholder perspective
3. Formulate the outcomes and impacts that determine program success
4. Use a variety of approaches to developing a logic model to guide evaluations
5. Articulate key evaluation questions to guide an evaluation
6. Understand data collection options and how they can support an evaluation
7. Know the options for presenting evaluation findings to maximise impact and learning
8. Critically evaluate evaluation reports and evaluation plans

Teaching and learning strategies

This subject will use a variety of learning strategies. These include formal presentations, group work, report presentations, case-studies, and in-class discussions. Students are expected to participate by reflecting on the course material and how it might apply in a real-world context.

We will make extensive use of real evaluation plans and reports to help apply knowledge. Students are encouraged to bring along any examples of evaluation plans and evaluation reports that they are familiar with.

Students will receive feedback on their contribution to the learning process. This will include informal feedback in class to support your learning and also formal feedback for your assessment’s tasks. Sometimes this feedback will be given to groups that you will work with.

Students will also be expected to participate in online discussions between the teaching blocks through the Canvas subject page.

Content (topics)

  • Planning and selecting an evaluation approach
  • Defining outcomes
  • Constructing a logic model
  • Evaluation-specific tools and methods;
  • Planning, conducting and managing evaluations;
  • Stakeholder engagement strategies; and
  • Evaluation reporting and implementation

Assessment

Assessment task 1: Assessing the quality of evaluation reports

Type: Report
Groupwork: Individual
Weight: 50%
Length:

1800–2000 words

Criteria:

1. Identifies 2 relevant evaluation reports - 10%

2. Uses the report assessment tool to assess the quality of each report - 20%

3. Identifies strengths of each report - 20%

4. Identifies weaknesses of each report - 20%

5. Suggests improvements to the evaluations - 20%

6. Discusses the validity and reliability of the tool - 10%

Assessment task 2: Evaluation planning

Type: Report
Groupwork: Individual
Weight: 50%
Length:

1800–2000 words

Criteria:

1. Articulate the program outcomes - 25%

2. Develop a logic model - 25%

3. Articulate a small number of Key Evaluation Questions - 20%

4. Select and describe an evaluation design that is appropriate - 30%

Minimum requirements

1. The Faculty of DAB expects that students attend 80% of classes for all enrolled subjects. Adequate achievement of subject criteria is difficult if classes are not attended.

2.Pursuant to UTS rule 2.5.1 students who do not satisfy attendance requirements may be refused permission to be considered for assessment by a Responsible Academic Officer.

Recommended texts

There is no required text for this subject. Instead students are to draw on a wide variety of resources that are available where appropriate. During the subject we will review any useful resources or readings that you have found. Some of these that you will find useful include:

Government guidelines

NSW Government Evaluation Toolkit and Guidelines
https://www.dpc.nsw.gov.au/tools-and-resources/evaluation-toolkit/

Queensland Government, Evaluation Framework, November 2021
www.treasury.qld.gov.au/resource/queensland-government-program-evaluation-guidelines/

Western Australian Government, Evaluation Guide 2015
https://www.wa.gov.au/government/document-collections/program-evaluation

ACT Government, Evaluation Policy Guidelines
www.cmd.act.gov.au/__data/assets/pdf_file/0004/175432/ACT-Evaluation-Policy-Guidelines.pdf

The Magenta Book: HM Treasury guidance on what to consider when designing an evaluation
https://www.gov.uk/government/publications/the-magenta-book

Many government agencies have their own evaluation guides, tools, and resources. You should find and use the ones that are relevant to your subject area of interest.

Websites

Betterevaluation.org