41082 Introduction to Data Engineering
Warning: The information on this page is indicative. The subject outline for a
particular session, location and mode of offering is the authoritative source
of all information about the subject for that offering. Required texts, recommended texts and references in particular are likely to change. Students will be provided with a subject outline once they enrol in the subject.
Subject handbook information prior to 2024 is available in the Archives.
Credit points: 6 cp
Subject level:
Undergraduate
Result type: Grade and marksDescription
This subject introduces students to the foundations of data engineering and the data industry. Data is all pervasive in modern society. Data engineers both build the infrastructure that enables this, but also participate in the manipulation and analysis of the data.
The subject takes a practical, hands-on approach designed to inspire and motivate students in all facets of data engineering. These include the design of experiments to gather data, the processing and analysis of data, as well as efficient visualisation of data to allow effective decision-making and actuation in relation to data. Students collect their own data sets to use in class activities and a final project.
MATLAB is core to the subject and students become proficient in its use for modelling and analysis.
This is a field of practice subject undertaken by all students enrolled in a Data Science Engineering major. Students from other majors or faculties may also enrol.
Subject learning objectives (SLOs)
Upon successful completion of this subject students should be able to:
| 1. | Use practical skills to model and simulate complex and complicated systems. (D.1) |
|---|---|
| 2. | Understand, collect and store data. (D.1) |
| 3. | Validate, visualise and authenticate data. (D.1) |
| 4. | Characterise the role of data engineering in data science projects. (B.1) |
| 5. | Design, perform and evaluate a data gathering activity using the Matlab environment. (C.1) |
Course intended learning outcomes (CILOs)
This subject also contributes specifically to the development of the following Course Intended Learning Outcomes (CILOs):
- Socially Responsible: FEIT graduates identify, engage, interpret and analyse stakeholder needs and cultural perspectives, establish priorities and goals, and identify constraints, uncertainties and risks (social, ethical, cultural, legislative, environmental, economics etc.) to define the system requirements. (B.1)
- Design Oriented: FEIT graduates apply problem solving, design and decision-making methodologies to develop components, systems and processes to meet specified requirements. (C.1)
- Technically Proficient: FEIT graduates apply abstraction, mathematics and discipline fundamentals, software, tools and techniques to evaluate, implement and operate systems. (D.1)
Contribution to the development of graduate attributes
Engineers Australia Stage 1 Competencies
This subject contributes to the development of the following Engineers Australia Stage 1 Competencies:
- 1.1. Comprehensive, theory based understanding of the underpinning natural and physical sciences and the engineering fundamentals applicable to the engineering discipline.
- 1.2. Conceptual understanding of the mathematics, numerical analysis, statistics, and computer and information sciences which underpin the engineering discipline.
- 2.1. Application of established engineering methods to complex engineering problem solving.
- 2.2. Fluent application of engineering techniques, tools and resources.
- 3.1. Ethical conduct and professional accountability.
Teaching and learning strategies
The subject is carried out through a series of small and large data engineering tasks, supported by a combination of lectures and online resources. Students should prepare before every class by accessing the weekly preparation material in Canvas that will generally consist of short videos and guided questions on topics related to the subject matter presented in that class. In each session students will discuss the preparation material and then more in-depth content will be presented. Together with the lecture material, students will do exercises using MATLAB to practice their understanding of the concepts with verbal feedback from the tutors and group discussions and feedback across teams and students during class.
For the final 4 weeks of the subject, the students will be working on a group project. The students will group together in small teams and apply the knowledge they have gained in the course of the previous classes to design a data collection activity. Students are expected to apply learnt content, problem solving and research skills to write a specification document for the design of the experiment they wish to perform, collect, analyse and visualise the data from the experiment and present the solution to the class. To complete this task successfully, students will need to work effectively in a team as well as research the problem individually and in a group outside of the class and bring that knowledge to the sessions.
Students work in teams on the assignments involving problem definition, research, analysis and visualisation.
The faculty expects a commitment of nine hours per week for the subject, some of which occurs during class time. Students are expected to attend all timetabled sessions.
Ultimately, learning is the student's responsibility. It is an aim of this subject to help students develop strategies that will enable them to more effectively undertake the responsibility of learning. These strategies will help students throughout the rest of their course and later in practice. In this subject students are encouraged to recognise the resources around them, and to use them.
Specific strategies include:
- Establishment of study groups to encourage collaborative learning with group members.
- Plenary resource sessions during which information and guidance regarding the teaching staff’s expectations of students will be presented.
- Individual reflection.
Team projects help to develop skills such as:
- Teamwork (skills in working within team dynamics; leadership skills);
- Analysis and cognition (analysing task requirements; questioning; critically interpreting material; evaluating the work
- of others);
- Collaboration (conflict management and resolution; accepting intellectual criticism; flexibility; negotiation and compromise); and
- Organisational and time management skills.
Some of the concepts and ideas that students encounter while studying in this subject may be difficult to understand.
A range of supplementary learning materials such as MATLAB Academy online courses will be made available.
Content (topics)
- The fundamentals of data
- The basics of experimental design
- How to summarise, report on and draw conclusions from data
- Effective data visualisation
- Introduction to data fitting and modelling
- Introduction to advanced Matlab data analytics and machine learning
- Introduction to Complex Systems and their theory
Assessment
Assessment task 1: Learning Matlab for data engineering
| Intent: | Central to the subject is the ability to work with, summarise and visualise data and model complex systems. In particular, to use Matlab to do this. MATLAB is a tool used by Data Engineers in many different industries to collect, analyse, model, and visualise data. Hence, being familiar with MATLAB is beneficial to many careers in Data Engineering. In addition, Assessment Task 2 and Assessment Task 3 will involve the use of MATLAB to analyse and visualise data. |
|---|---|
| Objective(s): | This assessment task addresses the following subject learning objectives (SLOs): 1, 3 and 5 This assessment task contributes to the development of the following Course Intended Learning Outcomes (CILOs): C.1 and D.1 |
| Type: | Quiz/test |
| Groupwork: | Individual |
| Weight: | 25% |
Assessment task 2: Experimental Design Report
| Intent: | Data Engineers can not always rely on large volumes of data ready for analysis in advance. Data gathering is an important skill for any Data Engineer and often requires precise planning to ensure that the data collected is valid and actionable. This assessment task involves the planning and design of a data collection experiment with the specific goal to collect data to model a phenomena or answer a question typically set for data engineers. |
|---|---|
| Objective(s): | This assessment task addresses the following subject learning objectives (SLOs): 1, 2, 3 and 4 This assessment task contributes to the development of the following Course Intended Learning Outcomes (CILOs): B.1 and D.1 |
| Type: | Essay |
| Groupwork: | Group, individually assessed |
| Weight: | 25% |
| Length: | 4,500 words |
Assessment task 3: Report and Present on Data Engineering Project
| Intent: | Once data has been collected, the proper analysis and effective reporting on data and conclusions is a vital skill for any Data Engineer. This Assessment Task provides students with the opportunity to showcase the skills in critical analysis, reporting and visualisation that they have learnt during the subject. |
|---|---|
| Objective(s): | This assessment task addresses the following subject learning objectives (SLOs): 1, 2, 3, 4 and 5 This assessment task contributes to the development of the following Course Intended Learning Outcomes (CILOs): B.1, C.1 and D.1 |
| Type: | Project |
| Groupwork: | Group, individually assessed |
| Weight: | 50% |
Minimum requirements
In order to pass the subject, a student must achieve an overall mark of 50% or more.
Required texts
There is no single required textbook in this subject.
Recommended texts
Matlab. A Practical Introduction to Programming and Problem Solving. Stormy Attaway. 4E.
Elsevier: ISBN: 978-0-12-804525-1
This is an excellent book on Matlab. It is recommended that students purchase it both for learning Matlab, but also to keep as a reference for the later stages of their DE Data Engineering degree.
Introduction to MATLAB for Engineers, 3E. William J. Palm
McGraw-Hill: ISBN: 978-0-07-353487-9
This is another excellent book on Matlab. It would make a good alternative to the one described above.
Other resources
Matlab is an essential tool used in this subject, as well as the degree overall.
It is part of the process of developing a "mathematical mindset" amongst graduates.
- In order to gain the most from the subject and Matlab, all students are expected to create and use a Mathworks account.
- The University has a TAH licence with Mathworks.
- This means that all students can obtain a Mathworks account using their UTS student email accounts.
- Students should have created this account during preparation week before coming to the first formal session.
It is also recommended that students learn how to access the web version of Matlab using their Mathworks account