Neural Networks Assignment

Spring 2024, A3

Overview

Objective

In this assessment, you will be asked to build neural networks to conduct two main tasks, one involving a regression task, the other involving a classification task. For each task, there will be multiple subtasks, including i) short subtasks requiring you to answer short questions and explain a technical detail, or critically evaluate a method used in Neural Network modelling, and ii) longer subtasks for which you will have to write code and/or present some results. You will write all answers and code in a single Jupyter Notebook.

How to submit

Please submit your assignment as a zip file that includes the .ipynb file containing your work on the tasks, along with any additional image files that are used for figures in your report. I recommend you use university computers to complete this work, as they are well equipped with all the tools and hardware you need. You can use VScode to edit the ipynb file. Google colab is also an option, but free access can be very limited and unreliable. Please name the zip file XXXXXX.zip, where XXXXXX is your candidate number. Give the Notebook the filename XXXXXX.ipynb. Submit through Canvas Online (not Turnitin).

Grading

The Notebook specifies how many marks you will be awarded for each subtask. There are 163 marks available in total, and your final grade will be given as a percentage. In general, marks will be awarded for each distinct point that is made in response to a question, or for providing scientifically grounded reasons for your answers, clear presentation of your results in figures, or well structured and correct code.

Important notes

● You must use PyTorch to build, train, and test the NNs for this assessment. You are permitted to use any of the functions available in PyTorch, e.g. to import and process data, to define NN layers and optimisers, etc., but you are not permitted to load pre-trained NN models that have been built by others.

● Make sure that every figure has a corresponding caption that explains the details of what that figure is showing. Make sure that figures have labelled axes. If you are comparing multiple sets of results with each other, make sure that either i) they are plotted in the same figure, or ii) each figure uses the same scale for the axes.

Task summary

Task 1

In this task, you will be given code that maps N-dimensional input values, x, into 1-dimensional output values, y, where  and  is a normally distributed random variable with zero mean and unit variance, and the subscript  denotes individual samples. The code generates training data and test data, and the task will require you to build a multi-layer perceptron that performs non-linear regression on the data. You will be asked to demonstrate how different features of the dataset and different features of the model influence the performance of the regression task, providing reasons for your results. An illustration of the data, for 1-dimensional x, is shown in Figure 1.

Figure 1. Example of the quadratic function, where input x is mapped to output y with some noise. 16 data samples are shown as blue dots, and the noise-free quadratic function that has to be learned is shown in red.

Task 2

In this assignment, you will build a small convolutional neural network (CNN), based on the seminal paper of LeCunn et al. (1989) that is included in the assessment package. You will train the CNN on a classification task using the MNIST dataset, for which simple models can perform. very well. MNIST data (shown in the figure to the right) comprises greyscale images of handwritten digits, from 0 to 9, and the typical task is to correctly classify each image into one of the ten classes. You will be asked to investigate the invariance of the model to input translations, and discuss other properties of CNNs.

Figure 2. Example samples, ten from each of the ten classes, of the MNIST dataset. Samples are 32x32 greyscale images of handwritten digits, 0-9.

Further details

For each of the two tasks, you will be asked to complete a set of subtasks. These are all provided in a Jupyter Notebook, where you will be given space to write code in code cells and to write text answers in markdown cells. All responses to the subtasks must be provided in the same Jupyter Notebook, i.e. any additional Python files or text documents will not be marked. You must name the Notebook XXXXXX.ipynb, where XXXXXX is your candidate number.

All figures that show results from your models must be generated from code cells. Make sure those figures are still visible when you submit your assessment. I will not run any of your code cells to produce the output. If necessary for your responses, you may also include images as separate files that are loaded into the Notebook. Guidance for how to produce figures and how to use Latex markup within the Notebook is provided in the separate Notebook, useful_code_for_figures_equatons.ipynb.

All image files and the Notebook must be zipped together in a file called XXXXXX.zip, which is to be submitted electronically via Canvas. Make sure there are no subdirectories within the zip file.

When writing responses in markdown cells, you must use the cells with text in italics that says “Replace this text with your response”. Use these cells for writing your answers, unless you are asked to use a code cell.

Word count

Your responses in Markdown cells must not exceed a word limit of 2500 words in total, as determined by the Python script, count_jupyter_nb_words.py, that is provided with this assessment. An example of how to use count_jupyter_nb_words.py is provided as preamble comments within the file. When writing your responses, remember to delete the text “Replace this text with your response”, otherwise it will contribute to your word count. None of the other text, either in code cells, or in markdown cells that describe each subtask, will contribute to the word count. The script. outputs the number of words you have added to the document. If you exceed the word count, the script. will also output a short string, starting from where the word count is exceeded, so that you can locate where that happens. Any part of the Markdown cells that exceeds 2500 words will not be marked.

Code

All code to build, run, and analyse your models, as well as to plot results, must be provided in the code cells provided. If you need to create new code cells for a given subtask, you may do so immediately after the provided code cell. Important: the person marking your report will not run any of your code. Therefore, all figures must be generated and visible in the Notebook when you submit to Canvas, or you will miss available marks. In order for those marking your report to determine whether or not you have correctly completed the experiments, they may need to look at your code. To help us make sense of your code, and therefore to help us improve our feedback to you, please provide plenty of comprehensible comments, including general overviews for each section of code, and shorter comments every few lines of code (if not on every line). You will receive some marks for good commenting of your code.

Important points:

● You are not expected to produce the perfect NN models with competitive performance. What we are looking for is your scientific scrutiny of the model and the techniques you have employed.

● Your work will be assessed by humans who get tired/hungry/frustrated/intrigued/curious. While we make every effort to assess your work fairly, those humans are not perfect. Help them to assess your work, and to realise how talented you really are, by making your presentation clear and concise. Check that references to figures are consistent, check for spelling, grammar, and consistent terminology. Every little helps.