University of Technology Sydney

94691 Deep Learning

Warning: The information on this page is indicative. The subject outline for a particular session, location and mode of offering is the authoritative source of all information about the subject for that offering. Required texts, recommended texts and references in particular are likely to change. Students will be provided with a subject outline once they enrol in the subject.

Subject handbook information prior to 2025 is available in the Archives.

UTS: Transdisciplinary Innovation
Credit points: 8 cp
Result type: Grade, no marks

Requisite(s): 36106 Machine Learning Algorithms and Applications

Description

This subject gives students a holistic understanding of deep learning, based on readings of research papers, coding exercises and practical assignments. Students become familiar with the underlying principles of deep learning neural networks, as well as with commonly used architectures and their applications. On completion of the subject, students are able to build, test and deploy deep learning models using industry standard software.

Subject learning objectives (SLOs)

Upon successful completion of this subject students should be able to:

1. Form an intuitive and mathematical understanding of Vanilla Neural Networks
2. Articulate the strengths, weaknesses and use cases of a variety of Deep Learning neural network architectures.
3. Implement and optimise a variety of Deep Learning neural network architectures
4. Communicate the selection, training & analysis of deep learning models for business use cases in a context-appropriate manner
5. Synthesise and interpret relevant academic papers as well as other sources of knowledge relating to deep learning to build a model for current progress and potential future directions, commercial applications and ethical implications
6. Understand and utilise the latest work towards visualising and uncovering insights from traditionally ‘black-box’ models.

Course intended learning outcomes (CILOs)

This subject contributes specifically to the development of the following course intended learning outcomes:

  • Explore and test models and generalisations for describing the behaviour of sociotechnical systems and selecting data sources, taking into account the needs and values of different contexts and stakeholders (1.2)
  • Use transdisciplinary approaches to seeing and doing to uncover underrepresented, or misrepresented, elements of a system (1.4)
  • Critique contemporary trends and theoretical frameworks in data science for relevance to one's own practice (2.1)
  • Apply and assess data science concepts, theories, practices and tools for designing and managing data discovery investigations in professional environments that draw upon diverse data sources, including efforts to shed light on underrepresented components (2.4)
  • Develop, test, justify and deliver data project propositions, methodologies, analytics outcomes and recommendations for informing decision-making, both to specialist and non-specialist audiences (4.3)
  • Engage in active, reflective practice that supports flexible navigation of assumptions, alternatives and uncertainty in professional data science contexts (5.1)

Contribution to the development of graduate attributes

The subject gives students a practical introduction to Deep Learning with a focus on applications. Popular architectures are covered and students are given opportunities to apply and interpret them in realistic settings. Technical skills are developed through practical coding labs and assessment tasks. As this area is developing rapidly, students are also encouraged to develop self-learning skills via assessment tasks that require them to research and assimilate recent and historical papers on the topic.

The subject addresses the following graduate attributes (GA):

GA 1 Sociotechnical systems thinking

GA 2 Creative, analytical and rigorous sense making

GA 3 Create value in problem solving and inquiry

GA 4 Persuasive and robust communication

Teaching and learning strategies

This subject is conducted in 7 sessions with weekly activities & readings assigned between classes. The classes are divided into a lecture component and a ‘lab’ component. Each comprising 1-1.5 hours depending on the content for that particular week. The lab components themselves involve two types of activities:

1) 'Code together’ sessions in which the instructor and students build understanding through collaboratively coding solutions to problems or implementing the theoretical content of the week.

2) Practical coding tasks for students to complete themselves and in small groups.

Assignments are a mix of practical coding exercises, report writing (to a business stakeholder) and literature reviews. This means that students get exposed to historical and current academic research in the area while developing tangible skills to implement these technologies and communicate highly technical matters to nonspecialist business audiences.

Due to the rapidly advancing nature of this field it is critical for students to develop skills in quickly absorbing, dissecting and understanding academic research and their value to business problems.

Assessment

Assessment task 1: Assessment task 1: Building Neural Network Architectures

Intent:

Gain hands-on experience building neural network architectures for real-world business cases.

Objective(s):

This task addresses the following subject learning objectives:

1, 3 and 4

This assessment task contributes to the development of course intended learning outcome(s):

1.2, 2.4 and 4.3

Type: Report
Groupwork: Individual
Weight: 40%
Criteria:

Criteria:

  • Rigour in addressing technical brief in terms of completeness and appropriate coverage of test suite (20%)

  • Clear, efficient, concise code appropriately commented (25%)

  • Effort and results on model extension component (15%)

  • Quality of results including their assessment, interpretation and benchmarking (25%)

  • Well written report ensuring format and communication style is context appropriate (15%)

Assessment task 2: Assessment task 2: Deep learning research & its industrial applications

Intent:

Review Computer Vision academic research and build Deep Learning model for Image Classification

Objective(s):

This task addresses the following subject learning objectives:

2, 3, 4, 5 and 6

This assessment task contributes to the development of course intended learning outcome(s):

1.4, 2.1, 2.4, 4.3 and 5.1

Type: Report
Groupwork: Individual
Weight: 30%
Criteria:


Criteria for Part A:

  • Depth of understanding of how the architecture works and its key applications (10%)

  • Thorough historical analysis of key research & technologies (30%)

  • Clear articulation of how each advancement built upon and improved or solved a problem with the previous one (45%)

  • Insightful discussion of latest research and promising future directions (15%)

Criteria for Part B:

  • Persuasive justification & discussion of model, technical and architectural choices including assumptions, where relevant (30%)

  • Insightfulness and quality of results including their assessment, interpretation and recommended next steps (20%)

  • Clear, efficient, concise (working) model code appropriately commented (40%)

  • Appropriateness of format and communication style of written report (10%)

Assessment task 3: Assessment task 3: Deep Learning final project

Intent:

Gain familiarity with building sequence-based neural network architectures.

Objective(s):

This task addresses the following subject learning objectives:

1, 2, 3 and 4

This assessment task contributes to the development of course intended learning outcome(s):

1.2, 2.4, 4.3 and 5.1

Type: Report
Groupwork: Group, group assessed
Weight: 30%
Criteria:

(See assessment brief for further details)

  • Justification & discussion of model, technical & architectural choices including assumptions where relevant (30%)

  • Quality of results including their assessment, interpretation and recommended next steps (20%)

  • Clear, efficient, concise (working) model code appropriately commented (40%)

  • Well written report ensuring format and communication style is context appropriate (10%)

References

  • Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y., 2014. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. arXiv:1406.1078 [cs, stat].

  • Clevert, D.-A., Unterthiner, T., Hochreiter, S., 2015. Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs). arXiv:1511.07289 [cs].

  • Deng, J., Dong, W., Socher, R., Li, L., Li, K., Fei-Fei, L., 2009. ImageNet: A large-scale hierarchical image database, in: 2009 IEEE Conference on Computer Vision and Pattern Recognition. Presented at the 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. https://doi.org/10.1109/CVPR.2009.5206848

  • Glorot, X., Bengio, Y., n.d. Understanding the dif?culty of training deep feedforward neural networks 8.

  • Goodfellow, I.J., Warde-Farley, D., Mirza, M., Courville, A., Bengio, Y., 2013. Maxout Networks. arXiv:1302.4389 [cs, stat].

  • Greff, K., Srivastava, R.K., Koutník, J., Steunebrink, B.R., Schmidhuber, J., 2017. LSTM: A Search Space Odyssey. IEEE Transactions on Neural Networks and Learning Systems 28, 2222–2232. https://doi.org/10.1109/TNNLS.2016.2582924

  • He, K., Zhang, X., Ren, S., Sun, J., 2015a. Deep Residual Learning for Image Recognition. arXiv:1512.03385 [cs].

  • He, K., Zhang, X., Ren, S., Sun, J., 2015b. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. arXiv:1502.01852 [cs].

  • Hochreiter, S., Schmidhuber, J., 1997. Long Short-Term Memory. Neural Comput. 9, 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

  • Ioffe, S., Szegedy, C., 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. arXiv:1502.03167 [cs].

  • Johnson, J., Krishna, R., Stark, M., Li, L.-J., Shamma, D., Bernstein, M., Fei-Fei, L., 2015. Image Retrieval Using Scene Graphs. Presented at the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3668–3678.

  • Kessy, A., Lewin, A., Strimmer, K., 2018. Optimal whitening and decorrelation. The American Statistician 72, 309–314. https://doi.org/10.1080/00031305.2016.1277159

  • Krizhevsky, A., Sutskever, I., Hinton, G.E., 2012. ImageNet Classification with Deep Convolutional Neural Networks, in: Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1, NIPS’12. Curran Associates Inc., USA, pp. 1097–1105.

  • Lecun, Y., Bottou, L., Bengio, Y., Haffner, P., 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE 86, 2278–2324. https://doi.org/10.1109/5.726791

  • Lipton, Z.C., Berkowitz, J., Elkan, C., 2015. A Critical Review of Recurrent Neural Networks for Sequence Learning. arXiv:1506.00019 [cs].

  • Maas, A.L., Hannun, A.Y., Ng, A.Y., n.d. Recti?er Nonlinearities Improve Neural Network Acoustic Models 6.

  • Rumelhart, D.E., Hinton, G.E., Williams, R.J., 1986. Learning representations by back-propagating errors. Nature 323, 533–536. https://doi.org/10.1038/323533a0

  • Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., Bengio, Y., 2015. Show, Attend and Tell: Neural Image Caption Generation with Visual Attention. arXiv:1502.03044 [cs].