# Great progress has been made over the last 10 years in the success of artificial neural networks (ANN). A seminal paper describing some of these was co-authored by Hinton: “Deep Learning”, Yann LeCun, Yoshua Bengio and Geoffrey Hinton This paper has been presented in many forms; we will consider three of them

Computing and Information Systems/Creative Computing CO3311 Neural networks

Coursework assignments 1 and 2 2019–2020

Important

### Save your time - order a paper!

Get your paper written from scratch within the tight deadline. Our service is a reliable solution to all your troubles. Place an order on any task and we will take care of it. You won’t have to worry about the quality and deadlines

Order Paper NowYour coursework assignments should be submitted using the following file-naming conventions:

YourName_SRN_COxxxxcw#.pdf (e.g. GraceHopper_920000000_CO3311cw1.pdf)

• YourName is your full name as it appears on your student record (check your student portal).

• SRN is your Student Reference Number, for example 920000000.

• COXXXX is the course number, for example CO3311.

• cw# is either cw1 (coursework 1) or cw2 (coursework 2).

Any other parts of your submission (spreadsheets, code, etc.) should be submitted in a zip file that uses the same file-naming conventions: YourName_SRN_COxxxxcw#.zip

REMINDER: It is important that your submitted assignment is your own individual work and, for the most part, written in your own words. You must provide appropriate in-text citation for both paraphrase and quotation, with a detailed reference section at the end of your assignment (this should not be included in any word count). Copying, plagiarism and unaccredited and wholesale reproduction of material from books, online sources, etc. is unacceptable, and will be penalised (see our guide on how to avoid plagiarism on the VLE). You can also look at the end of any journal or conference paper to get an idea of how to cite your reference material appropriately.

Question 1

Great progress has been made over the last 10 years in the success of artificial neural networks (ANN). A seminal paper describing some of these was co-authored by Hinton:

“Deep Learning”, Yann LeCun, Yoshua Bengio and Geoffrey Hinton

This paper has been presented in many forms; we will consider three of them. For this question you are required to read the following three sources:

https://www.cs.toronto.edu/~hinton/absps/NatureDeepReview.pdf https://www.cs.tau.ac.il/~dcor/Graphics/pdf.slides/YY-Deep%20Learning.pdf http://www.personal.psu.edu/lxx6/DeepLearning.pdf

In an essay of about 1,000 words answer the following questions:

a) What concepts are introduced in these papers that extend those covered in the CO3311 subject guide and relevant readings?

b) Summarise each source.

c) Are these new concepts easy to understand given the material that you have already covered in CO3311?

d) What improvements would you have liked to see in these articles?

To answer these questions fully you are likely to require a search engine to find out more about the concepts that are mentioned in your answer to part (a). The material in Systematic evaluation of CNN advances on the ImageNet by Mishkin, Sergievskiy and Matas may be useful for this and for Question 2 below.

[20 marks]

In the articles given in Question 1 above, reference is made to a number of different activation functions. This question requires you to investigate what impact the use of these has on the accuracy and learning time of a single Backpropagation unit. In Question 2 of coursework assignment 2 you will be expected to take this further to investigate networks of three units, two in the hidden layer and one in the output layer.

a) Implement a 2-input single-unit Backpropagation network for each activation function that appears in the articles cited above.

b) Test your network with data for which you know the answers.

c) Train and evaluate your network on the data in Table 1 below. Collect data for the sum of square errors for each epoch after 0, 1, 10, 100, 1000 and 10000 epochs of training.

d) Carry out an analysis making a comparison of the results for each activation function in terms of how quickly the error reduces as a function of the number of epochs.

e) State any conclusions that can be made from your results.

Your answer should include:

• An introduction.

• A description of your program or spreadsheet, and how you developed and tested it, including an explanation of your test strategy.

• Sets of results as graphs of error vs epochs for each individual activation function as well as a graph with the results for all activations shown for comparison.

• An analysis of the results obtained including any surprising or interesting observations that you made.

• A conclusion.

For a good example of how to write up ANN experiments see:

Rectified Linear Units Improve Restricted Boltzmann Machines by Vinod Nair and Geoffrey Hinton (2010).

If you wish, you can modify the spreadsheet 2-sigmoid.xls, which is downloadable from the CO3311 course page on the VLE. If you prefer, you can use another piece of software or write your own.

In order to compare results from different functions, it is important to keep the scales on any graphs identical. Use log scales for both the x axis (number of epochs) and the y axis (sum of squared errors). Use the full width of a page (in portrait orientation) for each graph.

Table 1a Training data

Table 1b Validation data

Table 1c Evaluation data

[80 marks]

[Total 100 marks]

[END OF COURSEWORK ASSIGNMENT 1]

Question 1

Type ‘Deep learning’ into Google Scholar and find the following entry (this was the top result at the time of writing this coursework assignment):

Take note of the number of citations that the article has (which may differ when you do the search) as well as the number of versions.

By clicking on the number of citations, bring up a list of these and sort them by date with the newest first.

Many of the articles are freely available, however availability may depend on your location and whether you are at a teaching centre or not. The rightmost column of the results gives links that you can explore to see if you can obtain a free download. Browse these until you find two articles that you find readable and sufficiently detailed (giving technical details not just marketing material).

Write an account of an article showing:

i) Previous techniques used.

ii) Good progress using deep learning.

iii) Advantages of using deep learning.

iv) Any shortcomings of deep learning noted in the article.

Your essay (of about 1,000 words) should be structured as follows, and include the following sections with the appropriate headings:

• Introduction.

• Account of an article showing points i-iv (see above).

• Conclusions.

• References.

Remember to include citations and references for all sources used, and to do this in the Harvard format.

[25 marks]

This question requires you to extend the work done for Question 2 of coursework assignment 1 to the case of a 2-input 3-unit Backpropagation network (with two hidden units).

Table 2a contains a training set, Table 2b a validation set and Table 2c a test set. Your experiments must include the following:

a) Learning rates of 0.05, .1, .25, .5, .75 and 1.0.

b) Five replications for each set of learning rates, using different randomly selected initial values for all of the weights.

c) Plots of sum of square errors after 0, 1, 10, 100, 1000 and 10000 epochs. Use log scales for both x (number of epochs) and y (sum of errors) axes.

d) Discuss how the speed and accuracy achieved as a function of epochs varies between different activity functions.

e) Is one activity function clearly better than the others?

Table 2a Training set

Table 2b Validation set

Table 2c Test set

What to submit:

• A single PDF containing:

A description of the measure of errors that you used to monitor progress in your experiments.

A description of the way that you implemented the networks with enough detail to enable others to duplicate your work.

Summary table(s) and graphs of your results

A commentary on the results that you have obtained, describing any features that you feel are notable.

Any conclusions that you can make.

A list of references, using the Harvard system.

• A zip file including data files, program code or Excel spreadsheets for each of the experiments.

As for Question 2 of coursework assignment 1, you are advised to look at Rectified Linear Units Improve Restricted Boltzmann Machines by Nair and Hinton to see a good example of how ANN experiments can be written up.

[75 marks]

[Total 100 marks]

[END OF COURSEWORK ASSIGNMENT 2]

The post Great progress has been made over the last 10 years in the success of artificial neural networks (ANN). A seminal paper describing some of these was co-authored by Hinton: “Deep Learning”, Yann LeCun, Yoshua Bengio and Geoffrey Hinton This paper has been presented in many forms; we will consider three of them appeared first on Essaylink.