Neural Network - Rxharun

Neural Network

A neural network is a method in artificial intelligence that teaches computers to process data in a way that is inspired by the human brain. It is a type of machine learning process, called deep learning, that uses interconnected nodes or neurons in a layered structure that resembles the human brain. It creates an adaptive system that computers use to learn from their mistakes and improve continuously. Thus, artificial neural networks attempt to solve complicated problems, like summarizing documents or recognizing faces, with greater accuracy.

Types of neural networks models are listed below:

The nine types of neural networks are:

  • Perceptron
  • Feed Forward Neural Network
  • Multilayer Perceptron
  • Convolutional Neural Network
  • Radial Basis Functional Neural Network
  • Recurrent Neural Network
  • LSTM – Long Short-Term Memory
  • Sequence to Sequence Models
  • Modular Neural Network

An Introduction to Artificial Neural Network

Neural networks represent deep learning using artificial intelligence. Certain application scenarios are too heavy or out of scope for traditional machine learning algorithms to handle. As they are commonly known, Neural Network pitches in such scenarios and fills the gap. Also, enrol in the neural networks and deep learning course and enhance your skills today.

Artificial neural networks are inspired by the biological neurons within the human body which activate under certain circumstances resulting in a related action performed by the body in response. Artificial neural nets consist of various layers of interconnected artificial neurons powered by activation functions that help in switching them ON/OFF. Like traditional machine algorithms, here too, there are certain values that neural nets learn in the training phase.

Briefly, each neuron receives a multiplied version of inputs and random weights, which is then added with a static bias value (unique to each neuron layer); this is then passed to an appropriate activation function which decides the final value to be given out of the neuron. There are various activation functions available as per the nature of input values. Once the output is generated from the final neural net layer, the loss function (input vs output)is calculated, and backpropagation is performed where the weights are adjusted to make the loss minimum. Finding optimal values of weights is what the overall operation focuses around. Please refer to the following for a better understanding-

Weights are numeric values that are multiplied by inputs. In backpropagation, they are modified to reduce the loss. In simple words, weights are machine-learned values from Neural Networks. They self-adjust depending on the difference between predicted outputs vs training inputs.
The activation Function is a mathematical formula that helps the neuron to switch ON/OFF.

  • The input layer represents the dimensions of the input vector.
  • The hidden layer represents the intermediary nodes that divide the input space into regions with (soft) boundaries. It takes in a set of weighted inputs and produces output through an activation function.
  • The output layer represents the output of the neural network.

Types of Neural Networks

There are many types of neural networks available or that might be in the development stage. They can be classified depending on their: Structure, Data flow, Neurons used and their density, Layers and their depth activation filters etc. Also, learn about the Neural network in R to further your learning.

Types of Neural network. We are going to discuss the following neural networks:

A. Perceptron


The Perceptron model, proposed by Minsky-Papert is one of the simplest and oldest models of Neurons. It is the smallest unit of the neural network that does certain computations to detect features or business intelligence in the input data. It accepts weighted inputs and applies the activation function to obtain the output as the final result. Perceptron is also known as TLU(threshold logic unit)

Perceptron is a supervised learning algorithm that classifies the data into two categories, thus it is a binary classifier. A perceptron separates the input space into two categories by a hyperplane represented by the following equation:

Advantages of Perceptron
Perceptrons can implement Logic Gates like AND, OR, or NAND.

Disadvantages of Perceptron
Perceptrons can only learn linearly separable problems such as boolean AND problems. For non-linear problems such as the boolean XOR problem, it does not work.

B. Feed Forward Neural Networks

Applications on Feed Forward Neural Networks:

  • Simple classification (where traditional Machine-learning based classification algorithms have limitations)
  • Face recognition [Simple straight forward image processing]
  • Computer vision [Where target classes are difficult to classify]
  • Speech Recognition

The simplest form of neural networks where input data travels in one direction only, passing through artificial neural nodes and exiting through output nodes. Where hidden layers may or may not be present, input and output layers are present there. Based on this, they can be further classified as a single-layered or multi-layered feed-forward neural network.

The number of layers depend on the complexity of the function. It has uni-directional forward propagation but no backward propagation. Weights are static here. An activation function is fed by inputs which are multiplied by weights. To do so, the classifying activation function or step activation function is used. For example, The neuron is activated if it is above the threshold (usually 0) and the neuron produces 1 as an output. The neuron is not activated if it is below the threshold (usually 0) which is considered as -1. They are fairly simple to maintain and are equipped to deal with data that contains a lot of noise.

You Might Also Read  Online Dating Scams - Romance Scammers

Advantages of Feed Forward Neural Networks

  1. Less complex, easy to design & maintain
  2. Fast and speedy [One-way propagation]
  3. Highly responsive to noisy data

Disadvantages of Feed Forward Neural Networks:

  1. Cannot be used for deep learning [due to the absence of dense layers and back propagation]

C. Multilayer Perceptron

Applications on Multi-Layer Perceptron

  • Speech Recognition
  • Machine Translation
  • Complex Classification

An entry point towards complex neural nets where input data travels through various layers of artificial neurons. Every single node is connected to all neurons in the next layer which makes it a fully connected neural network. Input and output layers are present having multiple hidden Layers i.e. at least three or more layers in total. It has a bi-directional propagation i.e. forward propagation and backward propagation.

Inputs are multiplied with weights and fed to the activation function and in backpropagation, they are modified to reduce the loss. In simple words, weights are machine learned values from Neural Networks. They self-adjust depending on the difference between predicted outputs vs training inputs. Nonlinear activation functions are used followed by softmax as an output layer activation function.

Advantages of Multi-Layer Perceptron

  1. Used for deep learning [due to the presence of dense fully connected layers and back propagation]

Disadvantages of Multi-Layer Perceptron: 

  1. Comparatively complex to design and maintain

Comparatively slow (depends on the number of hidden layers)

D. Convolutional Neural Network

Applications on Convolution Neural Network

  • Image processing
  • Computer Vision
  • Speech Recognition
  • Machine translation

Convolution neural network contains a three-dimensional arrangement of neurons instead of the standard two-dimensional array. The first layer is called the convolutional layer. Each neuron in the convolutional layer only processes the information from a small part of the visual field. Input features are taken in batch-wise like a filter. The network understands the images in parts and can compute these operations multiple times to complete the full image processing. Processing involves the conversion of the image from RGB or HSI scale to grey scale. Furthering the changes in the pixel value will help to detect the edges and images can be classified into different categories.

Propagation is uni-directional where CNN contains one or more convolutional layers followed by pooling and bidirectional where the output of the convolution layer goes to a fully connected neural network for classifying the images as shown in the above diagram. Filters are used to extract certain parts of the image. In MLP the inputs are multiplied with weights and fed to the activation function. Convolution uses RELU and MLP uses a nonlinear activation function followed by softmax. Convolution neural networks show very effective results in image and video recognition, semantic parsing, and paraphrase detection.

Advantages of Convolution Neural Network:

  1. Used for deep learning with few parameters
  2. Fewer parameters to learn as compared to a fully connected layer

Disadvantages of Convolution Neural Network:

  • Comparatively complex to design and maintain
  • Comparatively slow [depends on the number of hidden layers]

E. Radial Basis Function Neural Networks

Radial Basis Function Network consists of an input vector followed by a layer of RBF neurons and an output layer with one node per category. Classification is performed by measuring the input’s similarity to data points from the training set where each neuron stores a prototype. This will be one of the examples from the training set.

When a new input vector [the n-dimensional vector that you are trying to classify] needs to be classified, each neuron calculates the Euclidean distance between the input and its prototype. For example, if we have two classes i.e. class A and Class B, then the new input to be classified is more close to class A prototypes than the class B prototypes. Hence, it could be tagged or classified as class A.

Each RBF neuron compares the input vector to its prototype and outputs a value ranging which is a measure of similarly from 0 to 1. As the input equals to the prototype, the output of that RBF neuron will be 1 and with the distance growing between the input and prototype the response falls off exponentially towards 0. The curve generated out of the neuron’s response tends toward a typical bell curve. The output layer consists of a set of neurons [one per category].

Application: Power Restoration
a. Powercut P1 needs to be restored first
b. Powercut P3 needs to be restored next, as it impacts more houses
c. Powercut P2 should be fixed last as it impacts only one house

F. Recurrent Neural Networks

Applications of Recurrent Neural Networks

  • Text processing like auto-suggest grammar checks, etc.
  • Text-to-speech processing
  • Image tagger
  • Sentiment Analysis
  • Translation

    Designed to save the output of a layer, Recurrent Neural Network is fed back to the input to help in predicting the outcome of the layer. The first layer is typically a feed-forward neural network followed by a recurrent neural network layer where some information it had in the previous time-step is remembered by a memory function. Forward propagation is implemented in this case. It stores information required for its future use. If the prediction is wrong, the learning rate is employed to make small changes. Hence, making it gradually increases toward making the right prediction during the backpropagation.
You Might Also Read  Welcome to University

Advantages of Recurrent Neural Networks

  1. Model sequential data where each sample can be assumed to be dependent on historical ones is one of the advantages.
  2. Used with convolution layers to extend the pixel effectiveness.

Disadvantages of Recurrent Neural Networks

  1. Gradient vanishing and exploding problems
  2. Training recurrent neural nets could be a difficult task
  3. Difficult to process long sequential data using ReLU as an activation function.

An improvement over RNN: LSTM (Long Short-Term Memory) Networks

LSTM networks are a type of RNN that uses special units in addition to standard units. LSTM units include a ‘memory cell’ that can maintain information in memory for long periods of time. A set of gates is used to control when information enters the memory when it’s output, and when it’s forgotten. There are three types of gates viz, Input gate, output gate and forget gate. The input gate decides how much information from the last sample will be kept in memory; the output gate regulates the amount of data passed to the next layer, and forget gates control the tearing rate of memory stored. This architecture lets them learn longer-term dependencies

G. Sequence-to-sequence models

A sequence-to-sequence model consists of two Recurrent Neural Networks. Here, there exists an encoder that processes the input and a decoder that processes the output. The encoder and decoder work simultaneously – either using the same parameter or different ones. This model, contrary to the actual RNN, is particularly applicable in those cases where the length of the input data is equal to the length of the output data. While they possess similar benefits and limitations to the RNN, these models are usually applied mainly in chatbots, machine translations, and question-answering systems.

H. Modular Neural Network 

Applications of Modular Neural Network

  1. Stock market prediction systems
  2. Adaptive MNN for character recognition 
  3. Compression of high-level input data

A modular neural network has a number of different networks that function independently and perform sub-tasks. The different networks do not really interact with or signal each other during the computation process. They work independently towards achieving the output.

As a result, a large and complex computational process are done significantly faster by breaking it down into independent components. The computation speed increases because the networks are not interacting with or even connected to each other.

Advantages of Modular Neural Network

  1. Efficient
  2. Independent training
  3. Robustness

Disadvantages of Modular Neural Network

  1. Moving target Problems

Why are neural networks important?

Neural networks can help computers make intelligent decisions with limited human assistance. This is because they can learn and model the relationships between input and output data that are nonlinear and complex. For instance, they can do the following tasks.

Make generalizations and inferences

Neural networks can comprehend unstructured data and make general observations without explicit training. For instance, they can recognize that two different input sentences have a similar meaning:

  • Can you tell me how to make the payment?
  • How do I transfer money?

A neural network would know that both sentences mean the same thing. Or it would be able to broadly recognize that Baxter Road is a place, but Baxter Smith is a person’s name.

What are neural networks used for?

Neural networks have several use cases across many industries, such as the following:

  • Medical diagnosis by medical image classification
  • Targeted marketing by social network filtering and behavioral data analysis
  • Financial predictions by processing historical data of financial instruments
  • Electrical load and energy demand forecasting
  • Process and quality control
  • Chemical compound identification

We give four of the important applications of neural networks below.

Computer vision

Computer vision is the ability of computers to extract information and insights from images and videos. With neural networks, computers can distinguish and recognize images similar to humans. Computer vision has several applications, such as the following:

  • Visual recognition in self-driving cars so they can recognize road signs and other road users
  • Content moderation to automatically remove unsafe or inappropriate content from image and video archives
  • Facial recognition to identify faces and recognize attributes like open eyes, glasses, and facial hair
  • Image labeling to identify brand logos, clothing, safety gear, and other image details

Speech recognition

Neural networks can analyze human speech despite varying speech patterns, pitch, tone, language, and accent. Virtual assistants like Amazon Alexa and automatic transcription software use speech recognition to do tasks like these:

  • Assist call center agents and automatically classify calls
  • Convert clinical conversations into documentation in real time
  • Accurately subtitle videos and meeting recordings for wider content reach
You Might Also Read  What Is Proxy Server? - All About You Need To Know

Natural language processing

Natural language processing (NLP) is the ability to process natural, human-created text. Neural networks help computers gather insights and meaning from text data and documents. NLP has several use cases, including in these functions:

  • Automated virtual agents and chatbots
  • Automatic organization and classification of written data
  • Business intelligence analysis of long-form documents like emails and forms
  • Indexing of key phrases that indicate sentiment, like positive and negative comments on social media
  • Document summarization and article generation for a given topic

Recommendation engines

Neural networks can track user activity to develop personalized recommendations. They can also analyze all user behavior and discover new products or services that interest a specific user. For example, Curalate, a Philadelphia-based startup, helps brands convert social media posts into sales. Brands use Curalate’s intelligent product tagging (IPT) service to automate the collection and curation of user-generated social content. IPT uses neural networks to automatically find and recommend products relevant to the user’s social media activity. Consumers don’t have to hunt through online catalogs to find a specific product from a social media image. Instead, they can use Curalate’s auto product tagging to purchase the product with ease.

How do neural networks work?

The human brain is the inspiration behind neural network architecture. Human brain cells, called neurons, form a complex, highly interconnected network and send electrical signals to each other to help humans process information. Similarly, an artificial neural network is made of artificial neurons that work together to solve a problem. Artificial neurons are software modules, called nodes, and artificial neural networks are software programs or algorithms that, at their core, use computing systems to solve mathematical calculations.

Simple neural network architecture

A basic neural network has interconnected artificial neurons in three layers:

Input Layer

Information from the outside world enters the artificial neural network from the input layer. Input nodes process the data, analyze or categorize it, and pass it on to the next layer.

Hidden Layer

Hidden layers take their input from the input layer or other hidden layers. Artificial neural networks can have a large number of hidden layers. Each hidden layer analyzes the output from the previous layer, processes it further, and passes it on to the next layer.

Output Layer

The output layer gives the final result of all the data processing by the artificial neural network. It can have single or multiple nodes. For instance, if we have a binary (yes/no) classification problem, the output layer will have one output node, which will give the result as 1 or 0. However, if we have a multi-class classification problem, the output layer might consist of more than one output node.

Deep neural network architecture

Deep neural networks, or deep learning networks, have several hidden layers with millions of artificial neurons linked together. A number, called weight, represents the connections between one node and another. The weight is a positive number if one node excites another, or negative if one node suppresses the other. Nodes with higher weight values have more influence on the other nodes.
Theoretically, deep neural networks can map any input type to any output type. However, they also need much more training as compared to other machine learning methods. They need millions of examples of training data rather than perhaps the hundreds or thousands that a simpler network might need.


1. What are the 9 major types of neural networks?

The nine types of neural networks are:

  • Perceptron
  • Feed Forward Neural Network
  • Multilayer Perceptron
  • Convolutional Neural Network
  • Radial Basis Functional Neural Network
  • Recurrent Neural Network
  • LSTM – Long Short-Term Memory
  • Sequence to Sequence Models
  • Modular Neural Network

2. What is a neural network and its types?

Neural Networks are artificial networks used in Machine Learning that work in a similar fashion to the human nervous system. Many things are connected in various ways for a neural network to mimic and work like the human brain. Neural networks are basically used in computational models.

3. What is CNN and DNN?

A deep neural network (DNN) is an artificial neural network (ANN) with multiple layers between the input and output layers. They can model complex non-linear relationships. Convolutional Neural Networks (CNN) are an alternative type of DNN that allow the modeling of both time and space correlations in multivariate signals.

4. How does CNN differ from Ann?

CNN is a specific kind of ANN that has one or more layers of convolutional units. The class of ANN covers several architectures including Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN) eg LSTM and GRU, Autoencoders, and Deep Belief Networks.

5. Why is CNN better than MLP?

Multilayer Perceptron (MLP) is great for MNIST as it is a simpler and more straightforward dataset, but it lags when it comes to real-world application in computer vision, specifically image classification as compared to CNN which is great.

Consumer Information – TrustArc The Leader in Privacy Management SoftwareLooking online for info on your child's health? Here are some tipsJanja Kristan - Chief Administrative Officer - AACI | LinkedIn
Translate »
Shop From Rx Harun