37495 Statistical Design and Models for Evaluation Studies
Warning: The information on this page is indicative. The subject outline for a
particular session, location and mode of offering is the authoritative source
of all information about the subject for that offering. Required texts, recommended texts and references in particular are likely to change. Students will be provided with a subject outline once they enrol in the subject.
Subject handbook information prior to 2025 is available in the Archives.
Credit points: 6 cp
Result type: Grade and marks
Requisite(s): ((37252 Regression and Linear Models AND 37161 Probability and Random Variables) OR 36103 Statistical Thinking for Data Science )
Description
Traditional randomised control trials remains the ‘gold-standard’ for evaluation studies but there is increasing use of alternative model-based approaches. Building on the students’ knowledge of regression models, this subject explores the use of basic RCTs where appropriate, time series approaches useful for evaluating interventions such as the Sydney ‘Lock-Out Laws’, propensity score adjustment, and two-stage regression approaches. Where possible, the subject draws on real-world case-studies to illuminate the various designs and approaches. The subject also explores concepts of statistical design relating to internal verses external validity.
Subject learning objectives (SLOs)
Upon successful completion of this subject students should be able to:
| 1. | Present a coherent and clear statistical argument to both technical experts and informed lay people. |
|---|---|
| 2. | Identify the constraints that need to be considered when designing an experiment for an unfamiliar situation and to construct an appropriate design for that situation. |
| 3. | Explain how designed experiments can be used in industry and public policy and apply the concepts of designed experiments in these contexts. |
| 4. | Implement advanced techniques to solve authentic problems using industry standard software. |
| 5. | Suggest solutions to mitigate ethical issues when designing a study or presenting analysis. |
| 6. | Present a persuasive argument which includes a substantial methodological component to both expert and non-expert audiences in written form. |
| 7. | Produce a comprehensive report summarising a substantial research project. |
Course intended learning outcomes (CILOs)
This subject also contributes specifically to the development of following course intended learning outcomes:
- Demonstrate theoretical and technical knowledge of mathematical sciences including calculus, discrete mathematics, linear algebra, probability, statistics and quantitative management. (1.1)
- Evaluate mathematical and statistical approaches to problem solving, analysis, application, and critical thinking to make mathematical arguments, and conduct experiments based on analytical, numerical, statistical, algorithms to solve new problems. (2.1)
- Work autonomously or in teams to demonstrate professional and responsible analysis of real-life problems that require application of mathematics and statistics. (3.1)
- Design creative solutions to contemporary mathematical sciences-related issues by incorporating innovative methods, reflective practices and self-directed learning. (4.1)
- Use succinct and accurate presentation of reasoning and conclusions to communicate mathematical solutions, and their implications, to a variety of audiences, using a variety of approaches. (5.1)
Contribution to the development of graduate attributes
This subject contributes to the development of the following graduate attributes:
1. Disciplinary knowledge
The lectures, weekly laboratories and assignment impart skills necessary in a number of mathematical disciplines and demonstrate how to apply these skills to a variety of problems.
2. Research, inquiry and critical thinking
A major component of the weekly laboratories is the consideration of how best to design an experiment for a given situation. Statisticians who work in industry are often required to design experiments and evaluation studies and this subject imparts the necessary skills. This is reinforced by the assignment and evaluation design proposal.
3. Professional, ethical and social responsibility
The weekly laboratories help students learn to manage their own work and to accept their own work and to accept responsibility for their own learning and, together with the assignment, they give practice in computing skills, data handling and quantitative and graphical literacy skills. The subject also shows students how to find and assess information from other sources, both academic and computational, and the assignment and evaluation design proposal provide a chance for students to demonstrate their mastery of this aspect of the subject.
4. Reflection, innovation, creativity
The requirement to propose an evaluation design allows students to reflect on their learning and demonstrate initiative in creating an evaluation design that is appropriate.
5. Communication
The weekly laboratories, assignment and evaluation design proposal allow students to present written solutions to statistical problems using appropriate professional language.
Teaching and learning strategies
Weekly online and on-campus: 2hr workshop (online) and 2hr computer lab (on-campus).
Workshops will incorporate teaching and learning strategies including discussion of concepts and case studies. Face-to-face computer labs will include short presentations, discussion of readings and student work. These will be complemented by independent student reading to prepare for class where they will use the information to participate in discussions. These discussion sessions are inquiry-led, with students given a question or topic to dissect in more detail through critical questioning and investigation. Students are expected to support each hour of class time with an hour of private study.
Feedback in the format of worked solutions will be provided for each lab worksheet. There will also be opportunities to ask questions about these in labs. The solutions and reference material are also provided online for flexible access.
Content (topics)
This subject will cover research design principles, standard designs, assumption checking and variance-stabilising transformation, simple random sampling and stratification, cluster sampling, ratio and regression estimation, non-random designs for evaluation, selected approaches to evaluations.
Assessment
Assessment task 1: Weekly labs
| Intent: | This assessment item addresses the following graduate attributes: 1. Disciplinary knowledge 2. Research, inquiry and critical thinking 3. Professional, ethical and social responsibility 5. Communication |
|---|---|
| Objective(s): | This assessment task addresses subject learning objective(s): 1, 2, 3 and 5 This assessment task contributes to the development of course intended learning outcome(s): 1.1, 2.1, 3.1 and 5.1 |
| Type: | Exercises |
| Groupwork: | Individual |
| Weight: | 40% |
| Criteria: | Accuracy of analysis, correct choice of reasoning, clarity of communication using correct mathematical terminology. |
Assessment task 2: Assignment
| Intent: | This assessment item addresses the following graduate attributes: 1. Disciplinary knowledge 2. Research, inquiry and critical thinking 3. Professional, ethical and social responsibility 5. Communication |
|---|---|
| Objective(s): | This assessment task addresses subject learning objective(s): 1, 2, 4, 6 and 7 This assessment task contributes to the development of course intended learning outcome(s): 1.1, 2.1, 3.1 and 5.1 |
| Type: | Report |
| Groupwork: | Individual |
| Weight: | 30% |
| Criteria: | Evidence of correct use of R, accuracy of interpretation, accurate justification and implementation of the design approach chosen, quality of written expression, appropriate report structure followed. |
Assessment task 3: Evaluation design proposal
| Intent: | This assessment item addresses the following graduate attributes: 1. Disciplinary knowledge 2. Research, inquiry and critical thinking 3. Professional, ethical and social responsibility 4. Reflection, innovation, creativity 5. Communication |
|---|---|
| Objective(s): | This assessment task addresses subject learning objective(s): 1, 2, 3, 5, 6 and 7 This assessment task contributes to the development of course intended learning outcome(s): 1.1, 2.1, 3.1, 4.1 and 5.1 |
| Type: | Report |
| Groupwork: | Individual |
| Weight: | 30% |
| Criteria: | Correct application of knowledge and procedures of the design and analysis of experiments, correct choice of reasoning, accurate justification and implementation of the evaluation approach chosen, clarity of communication using correct mathematical/statistical terminology. |
Minimum requirements
In order to pass this subject, a student must achieve a final result of 50% or more.
Recommended texts
The recommended texts for this subject is:
- D. Montgomery, Design and Analysis of Experiments, Wiley, 8th Edition, 2013.
- A. Dean and D. Voss, Design and Analysis of Experiments, Springer, 1st Edition, 1999.
- Available electronically: http://find.lib.uts.edu.au/?R=OPAC_b2514171
Some useful references are:
- R.O. Kuehl, Statistical Principles of Research Design and Analysis, Second Edition, Duxbury Press, 2000.
- W.G. Cochran and G.M. Cox, Experimental Designs, Wiley, 1957.
- D. R. Cox, Planning of Experiments, Wiley, 1958.
Other resources
Please check Canvas for a collection of videos that are relevant to this subject.