Natural Language Processing in practice from a to z - Simple demonstration

Introduction 

Human language analysis is one of the most significant uses of computer science in both business and higher education. Through the use of natural language processing, computers are now able to read text, hear speech, and comprehend it. In this article, we'll explain and demonstrate NLP using a well-known machine learning framework.

What is NLP?

Natural language processing or NLP  is the computer program's capacity to understand natural language or human language as it is spoken and written.

It is a branch of computer science and artificial intelligence (AI) that deals with how computers interpret human language.

What is natural language processing used for?

Uses for NLP include:

• Examining content in emails, conversations, or social media,
• Text summarization reduces duplication among data gathered from many sources by summarizing the meaning of documents and information,
• Sentiment analysis identifies the overarching emotions or personal opinions. appropriate for opinion mining,
• Conversion of text to speech and speech to text.

Natural language processing categories

There are two main categories of NLP:
Natural language understanding (NLU): It is the NLP task of extracting insights from natural language inputs.
Natural language generation (NLG): it is the NLP task of building coherent sentences in Natural Language from structured data.

The best way to develop NLP skills and turn academic information into useful practical experience is to build real-world NLP projects.  Okay, we’ve done enough introductions. Let’s move on to the implementation.

Use case

In this tutorial, we'll develop a model that can produce lyrics based on Adele's lyrics.

In this use case we will use:  

• Dataset : Song Lyrics
• Model: Recurrent Neural Network (RNN)
• Python Framework: TensorFlow

The main steps to follow :

• Import the appropriate libraries
• Building the Word Vocabulary
• Preprocessing the Dataset
• Build, compile and train the model 
• Generating lyrics

Import the appropriate libraries

import tensorflow as tf
import numpy as np 
from tensorflow.keras.preprocessing.sequence import pad_sequences
from tensorflow.keras.layers import Embedding, LSTM, Dense, Bidirectional
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.models import Sequential
from tensorflow.keras.optimizers import Adam
import matplotlib.pyplot as plt

Building the Word Vocabulary

Load the dataset

text = open("Put file path here!!", 'rb').read().decode(encoding = 'utf-8')

In this step, we will first convert the data to lowercase and split it. 

Since a computer can only comprehend numerical values, we must create a word index dictionary using the fit on texts function. 

corpus = text.lower().split("\n")
tokenizer = Tokenizer()
tokenizer.fit_on_texts(corpus)
total_words=len(tokenizer.word_index)+1

print('word index dictionary:', tokenizer.word_index)
print('total words:', total_words)

Preprocessing the Dataset

In the pre-processing stage, we need to prepare the data to feed it to the neural network. Our data must be represented by sequences of the same size. For better understanding, please look at the example below.

input_seq=[]

for line in corpus: 
    token_list=tokenizer.texts_to_sequences([line])[0]
    for i in range(1, len(token_list)):
        sequence_gram = token_list[:i+1]
        input_seq.append(sequence_gram)
max_se_len = max([len(x) for x in input_seq])
input_seq= np.array(pad_sequences(input_seq, maxlen=max_se_len, padding='pre'))

How to generate lyrics?

• Enter the input lyrics. Feed it to the reccurent neural network to generate one word after the seed lyrics.
  Input:  'love' --> Output: 'to'
  Input:  'difference between' --> Output: 'us'
• Add a new word to the seed lyrics
  

  New input: 'love to'

  New input: 'difference between us'
• Repeat the technique described above as many times as you like while feeding the neural network new seed lyrics.
  

  'love'--> 'love to' --> 'love to me' -->..... -->'..off the lights'

  New input: 'difference between' --> 'difference between us' --> ... --> '... i hear my words'
  
  

xs, labels = input_seq[:,:-1],input_seq[:,-1]
ys = tf.keras.utils.to_categorical(labels, num_classes=total_words)

Build, compile and train the model 

embed_dim=100
lstm_units = 150

learning_rate = 0.01
model= Sequential([
    Embedding(total_words, embed_dim, input_length=max_se_len-1),
    Bidirectional(LSTM(lstm_units)),
    Dense(total_words, activation='softmax')
])
#compile model 
model.compile(loss="categorical_crossentropy",
             optimizer=tf.keras.optimizers.Adam(learning_rate=learning_rate),
             metrics=['accuracy'])

model.summary()

Accuracy:

history= model.fit(xs, ys, epochs=20)

def plot(history, string):
    plt.plot(history.history['accuracy'])
    plt.xlabel("Epochs")
    plt.ylabel(string)
    plt.show()
plot(history, 'accuracy')

Generating lyrics

text_input=input('Fist words')

nb_words=int(input('Number of words'))
for _ in range(nb_words):
    sequence=tokenizer.texts_to_sequences([text_input])[0]
    padded=np.array(pad_sequences([sequence], maxlen=max_se_len-1, padding='pre'))
    pred=model.predict(padded)
    prediction=(np.argmax(pred, axis=-1))[0]
    if prediction !=0:
        output=tokenizer.index_word[prediction]
        text_input+=" "+output


print('Result :   ', text_input)

Medical datasets for ML and DL (with projects ideas)

Artificial intelligence (AI) has made significant advances in practically every field, and the healthcare industry is no exception. 

A huge amount of data is needed to develop a high-performance machine learning model. The data are crucial in the creation of a flawless model. The machine learning model will interact with fresh data based on past data.

Selecting the most appropriate data set for your search topic is a difficult task. In this post, we'll provide medical datasets for use in a variety of projects. 

Note that the models (in the sub-heading "Models") are just examples; the model chosen should be based on the task and the type of data available.

Machine Learning Datasets for  Brain Tumor diseases

1) Multimodal Brain Tumor Segmentation (BraTS) 2020

It contains 3D multimodal MRI images of patients with glioblastoma (GBM/HGG) and lower-grade glioma (LGG). T2 Fluid Attenuated Inversion Recovery, T2-weighted, and T1-weighted imaging modalities are included. Each one was obtained by the use of clinical protocols and various scanners.  Neuroradiologist reviews and approves the image annotation with is comprised GD-enhancing tumor, the peritumoral edema, and the necrotic and non-enhancing tumor core. 

1.1) Data link

You can request this dataset from BraTS 2020.

1.2) Machine learning project idea where this dataset can be used

Its main application is the segmentation of brain tumors.  So, for example, the objective is to develop an algorithm that uses an artificial neural network to predict the segmentation of each 3D image. You can also use it to create a classification model (there are two classes in the dataset).

1.3) Models

This dataset is mostly used with 3D CNN architectures.

2) Brain Tumor Detection 2020 - Br35H 

It contains 3060 images obtained using MRI technologies for patients with and without brain tumors. There are three folders in all (No, Yes, and Pred). The folders "no" and "yes" contain 1500 images of people without brain tumors and 1500 images of people with brain tumors, respectively. All of the images are available in jpg format.

2.1) Data link

This dataset is available on Brain Tumor Detection 2020 - Br35H .

2.2) Machine learning project idea where this dataset can be used

It's employed in different applications, including classification models (it has two classes) and tumor location detection (segmentation).

2.3) Models

 It's most commonly utilized with convolutional neural networks.

Machine Learning Datasets for  Brest Tumor diseases

1) Breast Cancer Wisconsin (Diagnostic) Data Set

William H. Wolberg, a physician, generated this data set. There are one file and 32 columns in it (in CSV format). It's obtained from women who have a lot of breast tissue. The outcome of the diagnosis is either benign cancer (B) or malignant cancer (M). There are 57 benign and 212 malignant cells in this collection. None is used to represent missing attributes.

1.1) Data link

This dataset is available on Breast Cancer Wisconsin (Diagnostic) Data Set.

1.2) Machine learning project idea where this dataset can be used

It can be utilized to create a classification model (with two classes: M for malignant and B for benign).

1.3) Models

We can employ a variety of machine learning algorithms (Support Vector Machines, Decision Tree Algorithm, ...). This dataset, for example, is utilized in a classification challenge with the K-Nearest Neighbors Algorithm.

Machine Learning Datasets for  Heart Diseases

1) Heart Failure Prediction Dataset

This dataset contains one file (in CSV format)  and 12 columns ( age, sex, ChestPainType, ...) and is utilized in the context of cardiovascular diseases (CVDs). It's a massive heart dataset. This dataset was created by integrating five different heart datasets (Cleveland, Hungarian, Swiss, Long Beach VA, and Stalog (Heart) Data Set).

1.1) Data link

This dataset is available on Heart Failure Prediction Dataset.

1.2) Machine learning project idea where this dataset can be used  

The majority of the time, this dataset is used in classification tasks.

1.3) Models

We can use a variety of machine learning models with this dataset, including Random Forest, K-Nearest NeighborsClassifier, LogisticRegression, and so on.

Machine Learning Datasets for Corona Virus

1) COVID-19 Detection X-Ray Dataset

The COVID-19 Detection X-Ray Dataset contains x-ray images of COVID-19 patients, bacterial and viral pneumonia patients, and healthy images. There are three folders and 5073 images in it (in jpeg format).

1.1) Data link

This dataset may be found on COVID-19 Detection X-Ray Dataset.

1.2) Machine learning project idea where this dataset can be used

This dataset can be used in classification projects.

1.3) Models

 It's most commonly utilized with convolutional neural networks.

2) COVID-19 Open Research Dataset Challenge (CORD-19)

This dataset is created by Allen Institute for AI. It represents a source of scholarly articles related to coronaviruses like COVID-19 and SARS.  it includes 596k files and 3439 columns (in CSV format). 

2.1) Data link

This dataset is available on CORD-19.

2.2) Machine learning project idea where this dataset can be used

 This dataset can be used to cluster papers depending on their impact, or to generate COVID-19 Ontology, etc.

2.3) Models

Different models are used with this dataset like Kmeans.

3) Novel Corona Virus 2019 Dataset

Coronavirus is a novel virus that first appeared in Wuhan, China. It didn't seem to be related to any other virus. The causes for the virus's emergence aren't entirely apparent. This dataset offers details regarding this. It is available from January 22, 2020. There are 6 files (in CSV format) and 2513 columns in this dataset.

3.1) Data link

To download this dataset click here.

3.2) Project idea where this dataset can be used

You can use this dataset to provide an overview of how COVID 19 has spread around the world. 

Machine Learning Datasets for  Alzheimer's Diseases 

1) OASIS-3: Longitudinal Neuroimaging, Clinical, and Cognitive Dataset for Normal Aging and Alzheimer’s Disease

This dataset contains over 2000 MRI sessions which include T1w, T2w, FLAIR, ASL, SWI, time of flight, resting-state BOLD, and DTI sequences. It includes 3D images for normal aging and Alzheimer's Disease

1.1) Data link

You can obtain the dataset from OASIS

1.2) Machine learning project idea where this dataset can be used

It can be used for classification tasks. 

1.3) Models

 It's most commonly utilized with convolutional neural networks.

Machine Learning Datasets for  Chronic Disease

1) Chronic Disease Indicators

This dataset includes one file and 34 columns. Over the last 15 years, a range of health-related questions has been evaluated at various times and locations across the United States. Confidence intervals and demographic stratification are included in the data.

1.1) Data link

This dataset is available on Chronic Disease Indicators.

1.2) Machine learning project idea where this dataset can be used

This dataset can be used for classification tasks; however, combining it with another dataset, for example, to develop a more performant model, is highly recommended.

Reinforcement Learning (RL) applications

Artificial intelligence (AI) is a huge branch of computer science that encompasses a wide range of topics. This domain has made remarkable advancements, progressing from basic programs in the 1950s to sophisticated programs that can mimic human behavior. The branch of machine learning is one of the most AI challenging disciplines.

What is Reinforcement Learning (RL)?

 The machine learning field includes multiple subfields.  After looking at deep neural networks a few days ago, today we'll look at one of the most interesting subfields of machine learning (Ml) which is known as Reinforcement Learning (RL).  This area of machine learning enables the creation of powerful models for making anonymous decisions.  

in RL, an agent learns from his or her mistakes and experiences while interacting with his environment (everything with which the agent has a direct or indirect interaction). As a result of his behaviors, an agent will receive rewards and punishments from his environment.  The setting of his behavior will realize according to the reward function result (rewards/punishments). He always tries to make the optimal decision following the current situation (state). generally, the goal of a model is to maximize rewards.

The agent does not need predefined data throughout the training process (like in supervised and unsupervised learning).

The RL algorithms enable the development of extremely intelligent systems that will have a significant impact on the world. Recently, with the  RL methods, researchers have found solutions to various complex problems, for example, self-driving cars.

Reinforcement Learning (RL) applications

RL is used in a range of disciplines, including robotics, health care, industrial domains, games,  business and production management problems.

Self-driving cars

This application is a well difficult task due to the agent may encounter new barriers at every time(a situation not seen before).

It learned how to solve the problem based on trial and error. Making agents learn from real-world trials is the key to developing a practical system that can ensure people's safety.

Healthcare

Also, reinforcement learning is utilized to create a system capable of performing a certain task, such as drug development or medical diagnostics.

Reinforcement learning can be used to predict disease and treatment, allowing for rapid intervention and thus increasing the rate of recovery.

Robotic applications 

Today's research shows that robots can perform many of the same jobs as human and may even be faster. Even the use of robots can help in the optimization of time and money (does not require a high salary).

Now, instead of using low level language to discuss with machines, we can use natural language. We can also use robots to work, interact, ask questions, and give the order to robots and receive a response. we have seen many interactive robots like the robot called Sophia.

Games

An agent may play games, and in a short amount of time, he can become an expert. He can learn to play a variety of games, including atari and video games such as Super Mario.

Conclusion 

Rl has a wide range of applications, including health care, games, robots, and so on. We may also point to the usage of reinforcement learning in recommendation systems, which suggest results for users based on their previous actions.

A simple explanation with examples of Generative Adversarial Networks (GANs)

 

Introduction 

Machine learning algorithms have had a lot of success in recent years, owing to their incredible intelligence in performing tasks. Through the input of previous data, the algorithm might appear to get smarter and more accurate at predicting results without being explicitly programmed. In the 1950s, the concepts "machine learning" and "artificial intelligence" first appeared.

 In 1950, Alan Mathison Turing, a British logician, and computer science pioneer, published his seminal paper "Computing Machinery and Intelligence", in which he developed the “Turing Test”  for assessing whether a machine is intelligent.  The question "Can machines think?" is addressed in this paper.  In this study, the subject "Can machines think?" is addressed. The idea is to make an observer converses with two counterparts, one a human and the other a machine, behind a locked door. Turing claims that if an observer cannot tell the difference between a human and a machine, the computer has passed the test and is intelligent. In 1951, Christopher Strachey built the first Intelligence artificial program (Strachey’s checkers (draughts)). This program was capable of playing a complete game of checkers at a good speed in 1952. 

Artificial intelligence has progressed further. He develops the ability to recognize and respond to events in domains such as commerce recommendation systems, and medical, among others. Several algorithms have also emerged such as decision tree, Naive Bayes, support vector machine (SVM), Random Forest, k-nearest neighbor (KNN), neural network algorithms, etc.  They are utilized in a variety of applications, including classification and regression. 

The most significant machine learning method is the neural network, which is based on the biological brain of animals. It is made up of three layers: an input layer, an output layer, and a hidden layer in the center. This technique is then improved to solve more difficult problems. This enhancement entails the addition of hidden layers. We now have several hidden layers instead of a single hidden layer, which is called deep learning. Deep learning is a type of artificial neural network that has several hidden layers (2 or more). 

In latest years, deep learning has encouraged researchers in making significant advances in the field of artificial intelligence.  Supervised machine learning algorithms ( Supervised learning teaches models to produce the desired output using a training set which comprises inputs and correct outputs.), unsupervised machine learning algorithms (Unsupervised learning uses a training set of inputs with no correct outputs to educate models to create the desired output.), semi-supervised machine learning algorithms (semi-supervised learning teaches models to generate the intended result.), and reinforcement machine learning algorithms (reinforcement learning is a training strategy that rewards desirable actions while penalizing undesirable ones. A reinforcement learning agent can sense and comprehend its environment, act, and learn via trial and error in general.) are the four types of deep learning algorithms.

There are several Deep learning architectures such as Convolution Neural Network (CNN), Autoencoders, and Recurrent Neural Network (RNN) are used for classification (such as medical image classification, face detection, spam classification, etc.), segmentation (segmentation of satellite images, segmentation of medical images: of breast cancer, etc.), text classification (example: understanding the meaning of the text), etc.

One day in 2014, Ian Goodfellow and his colleagues decided to launch a computer vision challenge with the goal of creating a system that could create photos by itself.  As a result, they come up with the idea of a Generative Adversarial Network, which is now the most impactful architecture.

In this article, we will try to answer these: what are Generative adversarial networks and how do they work? what are the applications of generative adversarial networks? what is the difference between  GAN and convolution neuron network (CNN)? 

What are Generative adversarial networks and how do they work? 

GAN is a neural network architecture, it is the most popular one in the last years,  according to Yann LeCun, are the most fascinating notion in Machine Learning in the last ten years. 

It's made up of two neural networks. The generator neural network produces new data instances, while the discriminator neural network examines them for authenticity.  

To further comprehend, imagine an apprentice chef taking an exam in which the subject is to create a new cuisine using the components specified by the master. Following the dinner preparation, the master will determine whether or not this meal should be included in the list of meals to be served in restaurants.

The generator model entails employing distribution to produce synthetic samples from random noise, while the discriminator acts as a controller, classifying the data generated as fake or real. 

The generator's goal is to create artificial data that the discriminator identifies as real.

GAN architecture 

GAN architecture below includes: 

• Random Input / Random noise: the input of the generative models. It is going to be utilized to build the synthetic data. This noise is converted into a useful result. We can get the GAN to create a wide range of data by introducing noise;
• Generator: is a neural network that will generate data (face image in our case) using the random noise;
• Training dataset: is made up of a huge number of samples. These samples will be used by the discriminator to compare the synthetic data with real data;
• Discriminator: is a type of classifier, which consists in distinguishing the real data from the data produced by the generator.

Generative adversarial network architecture

Let's take a closer look at the generator and discriminator

These models are the crux of the GAN model. At the same time, both GAN networks are being trained. The Generator (G) seeks to raise the amount of fake data classified as real data by the discriminator, while the Discriminator (D) aims to decrease the quantity of fake data classified as real data by the discriminator.

The generator, discriminator, and loss functions formula are shown in the figure below.

Where: 

θ^(G): generator parameter

θ^(D): discriminant parameter

Generative adversarial network architecture with functions

The generator training process:  

It can any deep neural network architecture that is suitable for the application domain and capable to do the generating of synthetic data from the distribution of the training set. 

The generator's training process begins with random noise. Random noise is used to generate the output. This data will be fed into the discriminator, which will classify it (as fake or real). It can measure the loss based on this classification, and its goal is to update the weight through backpropagation.

Generator training process

The discriminator training process: 

The discriminator classifies both real and fake data throughout the training phase. A loss function discriminator can be used to regulate the classification. The loss function output will increase if the discriminator misclassifies a true sample as false or a false sample as true. It ignores the generator loss function and just utilizes its loss function (shown below). Finally, it adjusts the weights through backpropagation.

Discriminator training process

What are the applications of generative adversarial networks?

The most effective neural network in the field of machine learning after 2014 is the Generative Adversarial Network (GAN). This architecture is employed in a variety of sectors like,

Face Aging 

Several researchers have used the GAN to simulate the Aging face, like, Grigory Antipov, et al.  2017 (Face aging with conditional generative adversarial networks).

There are a lot of similar apps available on the Play Store right now.

Generate Image Datasets

Ian Goodfellow's objective for 2014 is to generate images. He expressed this purpose in their original paper, which was the first time he introduced this model.

He put the suggested model to the test with a variety of image datasets, including the MNIST, CIFAR-10, and Toronto Face Database. GAN produces good results, as seen in the figure below. 

This image is taken from the original paper of  Ian Goodfellow, et al., 2014 [source]

Several researchers have used the generative adversarial neural network to produce images in diverse tasks, such as the development of face images, images of bedrooms, and so on, utilizing datasets such as Large-scale Scene Understanding (LSUN), Imagenet-1k, and so on.

Healthcare

We've all heard about the Coronavirus and how people scrambled to get the correct vaccination in a short amount of time. Let's begin with this concept! We can suggest the concept of drug and vaccine development. Using the most relevant dataset, GAN can create the proper drugs in a reasonable length of time.

Translating images

When we discuss GAN's applications, we fully understand the significance of this framework. GAN may be used to convert a nighttime image into a daytime image.

What is a Conditional GAN (cGAN), and how does it work?

More data, like labels, may be added to GANs to help them create better images.

How Conditional GAN (cGAN) works

The purpose of conditional GAN is to improve the GAN by adding new data. Because of the richness of the input, the generator could be guided and the convergence process accelerated. 

The noise and label from the previous input are combined in the generator so, it gains the ability to manage its output. 

What is the difference between  GAN and convolution neuron network (CNN)?

Convolutional neural networks are a sort of deep neural network design that is used to process spatial data such as images. It's made up of a lot of convolution and pooling layers which make it more capable to analyse images. Convolution neural network is used in classification, segmentation, object detection, recognizing a face. 

=> CNN has a different use than GNN; it cannot replace GNN, but it may act as a discriminator in GNN, for example (in this instance CNN is included in GNN).

Summary

• Goodfellow created the generative adversarial neural network in 2014, and it has since become the most significant neural network today;
  
  
• The GAN consists of two neural networks: the generator and the discriminator;
  
  
• The generator generates fake data from random noise, and the discriminator classifies the generated data as fake or real;
  
  
• Conditional GAN (cGAN) is a GAN that uses labels as input data for the discriminator and generator in order to speed up and enhance convergence.

References

Paper : 

• Goodfellow I, et al. Generative adversarial nets. 2014.
  
  
• Antipov G,et al.. Face aging with conditional generative adversarial networks. 2017.
  
  
• Yu F,  et al. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop.2015

Articles: 

• Computing Machinery and Intelligence
  
  
• 18 Impressive Applications of Generative Adversarial Networks (GANs)

• The Discriminator

4 Keys to build a good models in Machine Learning

Introduction 

Machine learning (ML), a branch of artificial intelligence (AI) that is widely employed, is one of the most rapidly evolving topics in technology. Machine Learning has emerged as a vital thing of cutting-edge commercial enterprise and studies. Based on Donald Hebb’s 1949 version of mind interaction, withinside the beyond 70 years the era has long gone from a laptop beating a human at a recreation of checkers to something this is a part of our everyday lives. ML allows social media websites run, as an instance via way of means of robotically tagging pix on Facebook, and has even helped to make self-using cars a reality.  Machine learning is also improving health equity by being utilized for diseases prediction, such as the identification of malignant tumors, difficult-to-detect malignancies such as Barrett's esophagus, brain cancer, or COVID-19 detection.

While developing machine learning models to answer a specific problem, some developers run across issues. To build such a model, the fundamental points must be considered.

In this tutorial, we will focus on the most important aspects of having the appropriate model. 

1. Choose the appropriate dataset

The selection of a dataset is a crucial stage, and the developer must pay close attention to it.

Data is today's most valuable commodity, and it drives critical choices. An algorithm in machine learning learns how to react with previously unseen occurrences from previous experiences. As a result, the data serve as a model's guide and reference.

In order to get better predictions, data scientists spend a lot of time preprocessing data before integrating it into a model. They used different techniques like data normalization and data augmentation (which aims to increase the size of limited datasets by generating multiple samples different from existing samples).  Accurate data is the key that makes the model reach the goal quickly. The quality and quantity of data directly influence the performance of the model.  

the process of dataset preparation can help to avoid the overfitting problem. 

Data may be found in a variety of formats, including structured data (in the form of rows and columns) and unstructured data (for example image, text, ...). There are also labeled datasets for supervised learning like classification (in essence, divides a piece of data into classes), and unlabeled datasets for unsupervised learning.

2. Hyperparameter tuning in machine learning algorithms 

Generally, the building of a good model goes through three essential stages: training, validation, and test. in the training step, getting the best parameters is the key to reach a good result. As a result, data scientists employ the gridsearch function (for more information click here). This function is responsible for selecting the best hyperparameters based on the data and model. 

The following are some of the parameters to adjust:

Batch size

which is the number of samples that the model will utilize during an iteration. For example, if a dataset comprises 100 images, we may use four iterations with a batch size of 25 to train the model on all of them (at each iteration we will use 25 images). 
There are three different cases of patch sizes: 

• batch mode: when the data set trail equals the batch size (i.e. iteration number = epoch number), batch mode is used.
• Mini-batch mode: is used when the batch size is smaller than the data set size (but greater than 1).
• stochastic mode: in this case, the batch size is equal to 1, therefore the parameters and gradient are modified after each image.

Epochs 

An epoch represents the number of passes the machine learning algorithm has made on the training data set.

In the video below, a basic example will help you understand the differences between epoch, batch size, and iteration.

Learning rate 

The learning rate is a programmable hyperparameter that determines how much the model changes in response to the predicted error each time it is run.

3. Choose the model

KNN, neural network (which includes Deep learning), SVM, and other algorithms are all models of machine learning.  The modeling of such a model is tied to the problem's needs. For example, to classify animal images, a model must be capable of analyzing the images, extracting key features, and then classifying the image. As a result, a convolution neural network is the best model in this case.

Let's take a look at some of the most well-known examples and the models that may be used.

• Convolution neural network (CNN) : Image processing / face detection.
• Recurrent Neural Networks: Text classification
• Generative Models: image creation 
• Logistic regression: Text editing
• fully connected networks: animal image classification
• K-means:  image segmentation/ document clustering
• Auto-encoders: removing noise from an image
• k-Nearest Neighbors (KNN): predicting a person's job
• Decision tree: determine whether soccer players can play according to weather information
• Naive Bayes: Sentiment Analysis/ Spam filtering
• Random Forest: Product Recommendation
• Support vector machine (SVM): Handwriting recognition

4. Train and evaluate the model  

During the training phase, the models use the existing data to build themselves. They learn from datasets and try to determine the link between inputs and outputs, ultimately generating the function that will be used to predict the result of fresh data. During this step, the model will gradually enhance the initialized weight until the ideal weight is reached.

The error curve will decrease as accuracy improves with a good model. When the accuracy curve drops after reaching its top (and the error curve grows after it reaches the peak), there is an overfitting problem (overwork is a risk with the model since it learns so much). As a result, the epoch number must be chosen carefully.

We get at the evaluation step after developing a model. Depending on the model applications, we may utilize a variety of evaluation metrics at this stage. The evaluation matrix, for example, can be utilized in the classification process.

There are also other metrics to use Accuracy, Recall, and Precision

With :

TP: True positive

TN: True negatives

FP: False positive

FN: False negatives

you can find the explanation here

Recapitulation

In order to create a good machine learning model, you need to consider key factors, such as model selection. Model selection is based on the application; for problem x, model y is the ideal suitable. To get the desired goal, it is also required to carefully choose hyperparameters.

The best deep learning architecture - Model performance

We all learn throughout our lives as people. To react with such an action, we rely on the recurrence of events, memory, experiences, and intellect. Deep neural network algorithms imitate the resonance of the human brain.  They entail creating a mathematical model that learns from previous data (experiences) in order to react to new data.

The beginners in deep learning asked always what is The best deep learning architecture? in this article, we try to answer this question by providing a detailed explanation.

The Most popular architectures

During these years, deep learning has improved its significant performance in different complex tasks such as speech recognition, object detection, etc. 

Autoencoder 

It is made of an encoder that compresses the input into a latent-space representation and a decoder that reconstructs the output from this representation. It is used in an unsupervised environment, It is commonly used to detect anomalies by learning the normal data flawlessly and identifying outliers (data that does not belong to the normal data). For instance, it can identify salt and pepper noise in a picture and then eliminate it by reconstructing the image using the decoder.

Recurrent neural network 

Recurrent neural networks (RNN) are a type of deep learning that is based on previous occurrences. It dealt with sequential data problems by using memory. This type of neural network is used to solve difficult problems involving the concepts of time and sequence. For instance, in videos that show a series of images over time.
It might also be used to solve similar problems like speech recognition, where the meaning of a word is linked to the meaning of words that come before it.

Convolution neural network 

The Convolutional Neural Network (CNN) is a class of neural networks. It can extract useful information from an image using convolutional layers. As a result, it is commonly utilized when the input is an image.

CNN is mostly made up of convolutional layers and pooling layers. The convolutional layer is used to extract the most relevant features from the image and the Pooling layer is used to keep that relevant data.

It is frequently used in classification, object recognition, segmentation, and other problems. Many deep learning models have been proposed in this area since the appearance of coronavirus, for example. From an x-ray image of the chest, these models can predict if the patient has this disease or not.

Example of a convolution product

What is the best architecture? 

We can't assert that one architecture is more efficient than the other. Each architecture has its own set of specs that may be compared to the others. According to previous studies, recurrent neural networks are more efficient in speech recognition, whereas convolution neural networks are more efficient in image classification and segmentation. However, this is not always the case, the autoencoder can also be used for segmentation.

Other parameters influence the performance of such a deep learning architecture, like:

• Learning rate,  
• Epochs,  
• Number of layers ( especially hidden layers),
• Batch size,
• etc,

Developers are always looking for methods to improve model accuracy. They can use a variety of strategies to enhance it. We may begin by avoiding the most well-known issue, which is Overfitting.

Overfitting Problem

This problem arises when the model learns the specifics in the training data by heart. The model learns insignificant detail in the training phase. As the model's accuracy improves, it reaches its peak (100 percent ). As a result of the overfitting, the model is unable to retain the same performance (in the training phase) when a new dataset is used in the validation stage. 

Figures (a) and (b) present the problem of overfitting.

Hyperparameters Tuning

The values of the hyperparameters, which may be adjusted using a Gridsearch function, are directly connected to the performance of the neural network architecture. With this function, you may test the various values that are manually given with a specified model. The outcome of this function is that it selects the best values from a set of options.

Example: an excerpt from gridsearch function

params = {
    'batch_size': [10,20],
    'lr': [0.01, 0.001, 0.0001,0.00001, 0.000001,0.0000001],
    'max_epochs': [10, 20, 50],
}
	

Output: best score: 0.95, best params: {'batch_size': 10, 'lr': 0.01, 'max_epochs': 10}

Learning rate: 

Is a positive value between 0 and 1 that specifies the model's learning speed.  A model with a high learning rate may converge too rapidly to a suboptimal solution, whereas a model with a low learning rate may stall the process. So, this value must be changed according to your model in order to be reliable.

Batch size: 

The batch size refers to the number of samples that will be used in a single training iteration. When the number of batch sizes is equal to the number of dataset sizes, and the number of iterations is equal to the number of epochs, Batch Mode is used. Stochastic Mode is utilized when the number of batch sizes equals one.  Mini-batch mode is used when the number of batch sizes is greater than one but less than the dataset size.

Epochs : 

Is a hyperparameter that indicates how many times the model learned the simples from the training set. Because the datasets are limited, we must repeat the model learning process several times in order to enhance the parameters of the neural network ( weight, etc.).

As the number of epochs grows, the parameters improve. The model shifts from underfitting to optimum as the number of epochs increases. When the number of epochs is increased in an uncontrolled manner, it might shift from an optimum state to an overfit state.

Optimization algorithm 

The optimization algorithm must also be selected based on the model. therefore the most appropriate way is to test several algorithms and then compare the results. The best algorithm is the one that produces the lowest loss values.

Data

Yes! A deep learning architecture's performance can be influenced by data.

To create and test a model, you'll typically need three datasets.

• The training dataset, which is the biggest, is utilized in the training phase to fit the parameters. Different optimization parameters are employed as gradient descendent.

• The validation dataset contains data that the model did not observe during the training phase. For example, in the classification of fruits and vegetables, the training dataset and validation dataset both contain images of fruits and vegetables, but each dataset contains images that differ from the other in terms of position, degree of color,  background, etc.

• The test dataset is used in the test stage to reflect model performance.

Note: Although it is not a mandate that datasets are divided 60 percent, 20%, and 20%, most developers do so.

Data is critical to the development of a successful model. A number of preprocessing techniques, such as data augmentation, data normalization, and data shuffling, are used to make data more useful and relevant.