PRE2019 3 Group10

Group Members

Name	Student Number	Study	Email
Joris Goddijn	1244648	Computer Science	j.d.goddijn@student.tue.nl
Daniël Verloop	1263544	Computer Science	a.c.verloop@student.tue.nl
Ka Yip Fung	1245300	Computer Science	k.y.fung@student.tue.nl
Rik Maas	1244503	Computer Science	r.maas@student.tue.nl
Jeroen Struijk	1252070	Computer Science	j.j.struijk@student.tue.nl

Problem Statement

Neural Networks have been used extensively in a wide variety of fields. In health care, neural networks are used for detecting certain diseases, for example tumor detection. In businesses, neural networks are used to filter job applicants: this way the business only has to take the best applicants in for interviews. In self driving cars neural networks are used to parse the visual input of the sensors of the car.
A big issue of modern neural networks is that it is a black box approach. Experts do not understand what is going on inside the neural network. This is an issue when people use neural networks to make decisions with real impactful consequences.

Our aim is to provide a tool which our users can use to have a better understanding of how the neural network they are using is making decisions.

Objectives

We are going to make a web application in which users can see visualizations of the inner workings of a neural network. We will focus on showing visualizations of the individual layers of a neural network. The goal of our web application is to help users understand how their neural network is making decisions.

Users and what they need

The users are people who make use of Neural Networks, e.g. people in the health care or students.

These users have need of an environment in which the inner workings of a neural network is visualized in such a way that these inner workings become more understandable.
It also needs to be understandable for people who do not have much pre-knowledge about neural networks or machine learning. This explanation will be a combination of some kind of model and pictures together with text that explains what is happening. Furthermore, users need to be able to explore the model freely, in such a way that they can discover different aspects of the models with various inputs.

Approach

We will use a website where we can visualize the activation of the layers in the neural network by uploading a picture/file. Based on the neural net, the website runs a forward pass on a trained network. This yields a layer activation which is returned to the user to gain information on how the neural net detect these objects.
We will need both front-end and back-end development.

Front-end: Bootstrap with Less.

• Visualizations with explanation.
• Vue
  

Back-end: Node.js/Django wtih Vue.

• Network structures saved on the server-side.
• Our own datasets on the server available to the user.
• All computations will be done in the back-end.

Deliverables and Milestones

Week 3: Have a working website with at least 1 visualization.

• Fully functional front-end that has implemented all functionality needed for finalizing this project.
• Back-end must have a general framework for supporting several different networks/datasets.
• One visualization must be completed and fully functional to the user.

Week 7/8: Website ready for deployment, finish the project.

• Extending our visualizations with choices for several working datasets/networks.

If time allows, we want to make it possible to the user to construct his own networks for simple neural network tasks.

We want to deliver a website that is finished and user friendly. Using the site, users are able to get a better understanding of the inner workings of a convolutional neural network. This is done through visualizing the computations and transformations on the input data. Wherever necessary, we will add text to further explain the visualizations to the user.

State of the Art

Examples of existing playground where one can visualize Neural Networks:

• https://playground.tensorflow.org/
• https://convnetplayground.fastforwardlabs.com/#/models
  

The following pages explore ways to visualize Neural Networks:

• http://cs231n.github.io/understanding-cnn/
• http://yosinski.com/deepvis
• https://distill.pub/2017/feature-visualization

Task Division

^________________________________________________^

References

Joris:

AlexNet: Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems (pp. 1097-1105).

VGGNet: Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.

ResNet: Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in neural information processing systems (pp. 91-99).

Survey: Qin, Z., Yu, F., Liu, C., & Chen, X. (2018). How convolutional neural network see the world-A survey of convolutional neural network visualization methods. arXiv preprint arXiv:1804.11191.

TODO: reference slideset: http://cs231n.stanford.edu/slides/2019/cs231n_2019_lecture13.pdf

Ka Yip:
Qin, Zhuwei & Yu, Fuxun & Liu, Chenchen & Chen, Xiang. (2018). How convolutional neural networks see the world - A survey of convolutional neural network visualization methods. https://arxiv.org/abs/1804.11191
Features learned by convolutional NN are hard to identify and understand for humans. This paper shows how CNN interpretability can be improves using CNN visualizations. The paper talks about several visualization methods, e.g. Activation Maximization, Network Inversion, Deconvolutional NN and Network Dissection based visualization.

Yu, Wei & Yang, Kuiyuan & Bai, Yalong & Yao, Hongxun & Rui, Yong. (2014). Visualizing and comparing Convolutional Neural Networks. https://arxiv.org/abs/1412.6631
This paper attempts to understand the internal working of a CNN, among others it uses visualization to do so.

Zeiler, Matthew D & Fergus, Rob. (2014). Visualizing and Understanding Convolutional Networks. https://link.springer.com/chapter/10.1007/978-3-319-10590-1_53
This paper introduces a visualization technique that gives insight to the function of feature layers and how classifier operates. It is said that using the visualizations, the network can be improved to outperform other networks.

Zintgraf, Luisa M. & Cohen, Taco S. & Welling, Max. (2017). A New Method to Visualize Deep Neural Networks. https://arxiv.org/abs/1603.02518
This paper presents a visualization method for the response of a NN to a specific input. It will for example highlight areas that show why (or why not) a certain class was chosen.

Zhang, Quanshi & Wu, Ying Nian & Zhu, Song-Chun. (2018). Interpretable Convolutional Neural Networks. http://openaccess.thecvf.com/content_cvpr_2018/html/Zhang_Interpretable_Convolutional_Neural_CVPR_2018_paper.html
This paper uses a different approach to make CNN more understandable. Instead of focusing on visualization, it proposes a method to modify a traditional CNN to make it more understandable.

Daniël:

Rik:

Jeroen: