15643 Applied Evaluation
Warning: The information on this page is indicative. The subject outline for a
particular session, location and mode of offering is the authoritative source
of all information about the subject for that offering. Required texts, recommended texts and references in particular are likely to change. Students will be provided with a subject outline once they enrol in the subject.
Subject handbook information prior to 2024 is available in the Archives.
Credit points: 6 cp
Result type: Grade and marks
There are course requisites for this subject. See access conditions.
Description
This subject introduces students to the foundation skills needed to undertake the evaluation of policies and programs, especially in a public sector context. The subject covers the skills needed to undertake an evaluation as well as the skills needed to manage and implement an evaluation.
This subject is required for students completing an Evidence and Evaluation sub-major within the Graduate Diploma/Master of Applied Policy.
Subject learning objectives (SLOs)
On successful completion of this subject, students should be able to:
| 1. | Understand major evaluation designs, including process, outcome, and economic evaluations |
|---|---|
| 2. | Identify the evaluation need from a decision-maker and stakeholder perspective |
| 3. | Formulate the outcomes and impacts that determine program success |
| 4. | Use a variety of approaches to developing a logic model to guide evaluations |
| 5. | Articulate key evaluation questions to guide an evaluation |
| 6. | Understand data collection options and how they can support an evaluation |
| 7. | Know the options for presenting evaluation findings to maximise impact and learning |
| 8. | Critically evaluate evaluation reports and evaluation plans |
Contribution to the development of graduate attributes
The term CAPRI is used for the five Design, Architecture and Building faculty graduate attribute categories where:
C = communication and groupwork
A = attitudes and values
P = practical and professional
R = research and critique
I = innovation and creativity.
Course intended learning outcomes (CILOs) are linked to these categories using codes (e.g. C-1, A-3, P-4, etc.).
Teaching and learning strategies
This subject will use a variety of learning strategies. These include formal presentations, group work, report presentations, case-studies, and in-class discussions. Students are expected to participate by reflecting on the course material and how it might apply in a real-world context.
We will make extensive use of real evaluation plans and reports to help apply knowledge. Students are encouraged to bring along any examples of evaluation plans and evaluation reports that they are familiar with.
Students will receive feedback on their contribution to the learning process. This will include informal feedback in class to support your learning and also formal feedback for your assessment’s tasks. Sometimes this feedback will be given to groups that you will work with.
Students will also be expected to participate in online discussions between the teaching blocks through the Canvas subject page.
Content (topics)
- Planning and selecting an evaluation approach
- Defining outcomes
- Constructing a logic model
- Evaluation-specific tools and methods;
- Planning, conducting and managing evaluations;
- Stakeholder engagement strategies; and
- Evaluation reporting and implementation
Assessment
Assessment task 1: Assessing the quality of evaluation reports
| Type: | Report |
|---|---|
| Groupwork: | Individual |
| Weight: | 50% |
| Length: | 1800–2000 words |
| Criteria: | 1. Identifies 2 relevant evaluation reports - 10% 2. Uses the report assessment tool to assess the quality of each report - 20% 3. Identifies strengths of each report - 20% 4. Identifies weaknesses of each report - 20% 5. Suggests improvements to the evaluations - 20% 6. Discusses the validity and reliability of the tool - 10% |
Assessment task 2: Evaluation planning
| Type: | Report |
|---|---|
| Groupwork: | Individual |
| Weight: | 50% |
| Length: | 1800–2000 words |
| Criteria: | 1. Articulate the program outcomes - 25% 2. Develop a logic model - 25% 3. Articulate a small number of Key Evaluation Questions - 20% 4. Select and describe an evaluation design that is appropriate - 30% |
Minimum requirements
- The DAB attendance policy requires students to attend no less than 80% of formal teaching sessions (lectures and tutorials) for each class they are enrolled in to remain eligible for assessment.
- Pursuant to UTS rule 2.5.1 students who do not satisfy attendance requirements may be refused permission to be considered for assessment by a Responsible Academic Officer.
Recommended texts
There is no required text for this subject. Instead students are to draw on a wide variety of resources that are available where appropriate. During the subject we will review any useful resources or readings that you have found. Some of these that you will find useful include:
Government guidelinesNSW Government Evaluation Toolkit and Guidelines
https://www.dpc.nsw.gov.au/tools-and-resources/evaluation-toolkit/
Queensland Government, Evaluation Framework, November 2021
www.treasury.qld.gov.au/resource/queensland-government-program-evaluation-guidelines/
Western Australian Government, Evaluation Guide 2015
https://www.wa.gov.au/government/document-collections/program-evaluation
ACT Government, Evaluation Policy Guidelines
www.cmd.act.gov.au/__data/assets/pdf_file/0004/175432/ACT-Evaluation-Policy-Guidelines.pdf
The Magenta Book: HM Treasury guidance on what to consider when designing an evaluation
https://www.gov.uk/government/publications/the-magenta-book
Many government agencies have their own evaluation guides, tools, and resources. You should find and use the ones that are relevant to your subject area of interest.
Websites