[PYTHON] I implemented ResNet!

Read this article about the readership

Those who have basic knowledge of Deep Learning convolutional neural networks (CNN) and understand the meanings of the following words Example) --Convolution

What is ResNet

It is one of the CNN methods and can add more layers than other CNNs. As a feature, at the end of the module, the input data of the module is added to the data processed in the module (shortcut connection). For more information, please visit here.

Operating environment

GoogleColaboratory

Sample program

#Installation of required libraries
import tensorflow.keras as keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D
from tensorflow.keras.layers import MaxPooling2D
from tensorflow.keras.layers import Flatten
from tensorflow.keras.layers import Dense
from tensorflow.keras.datasets import cifar10

#A class that takes CIFAR10 data and converts it to a vector
class CIFAR10Dataset():
  def __init__(self):
    self.image_shape = (32, 32, 3)
    self.num_classes = 10

  #Acquire training data and test data.
  def get_batch(self):
    (x_train, y_train), (x_test, y_test) = cifar10.load_data()
    x_train, x_test = [self.change_vec(img_data) for img_data in [x_train, x_test]]
    y_train, y_test = [self.change_vec(img_data, label_data=True) for img_data in [y_train, y_test]]
    return x_train, y_train, x_test, y_test

  #If it is an objective variable, change it to a class vector. Explanatory variables are standardized.
  def change_vec(self, img_data, label=False):
    if label:
      data = keras.utils.to_categorical(img_data, self.num_classes)
    else:
      img_data = img_data.astype("float32")
      img_data /= 255
      shape = (img_data.shape[0],) + self.image_shape
      img_data = img_data.reshape(shape)
    return img_data

#A function that sets and returns a deep learning model
def network(input_shape, num_classes,  count):
  filter_count = 32
  inputs = Input(shape=input_shape)
  x = Conv2D(32, kernel_size=3, padding="same", activation="relu")(inputs)
  x = BatchNormalization()(x)
  for i in range(count):
    shutcut = x #Get module input data for shortcut connection.
    x = Conv2D(filter_count, kernel_size=3, padding="same")(x)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = Dropout(rate=0.3)(x)
    x = Conv2D(filter_count, kernel_size=3, padding="same")(x)
    x = BatchNormalization()(x)
    x = Concatenate()([x, shutcut]) #Shortcut connection
    if i != count - 1:
      x = MaxPooling2D(pool_size=2)(x)
      filter_count = filter_count * 2
  x = Flatten()(x)
  x = BatchNormalization()(x)
  x = Dense(1024, activation="relu")(x)
  x = Dropout(rate=0.3)(x)
  x = BatchNormalization()(x)
  x = Dense(1024, activation="relu")(x)
  x = Dropout(rate=0.3)(x)
  x = BatchNormalization()(x)
  x = Dense(num_classes, activation="softmax")(x)
  model = Model(inputs=inputs, outputs=x)
  print(model.summary())
  return model

#Class to train the model
class Trainer():
  #Compile the model and set the settings for training in private properties.
  def __init__(self, model, loss, optimizer):
    self._model = model
    self._model.compile(
        loss=loss,
        optimizer=optimizer,
        metrics=["accuracy"]
    )
    self._verbose = 1
    self._batch_size = 128
    self._epochs = 30

  #Actual learning
  def fit(self, x_train, y_train, x_test, y_test):
    self._model.fit(
        x_train,
        y_train,
        batch_size=self._batch_size,
        epochs=self._epochs,
        verbose=self._verbose,
        validation_data=(x_test, y_test)
    )
    return self._model

dataset = CIFAR10Dataset() #Instantiation of CIFAR10 Dataset to retrieve data
model = network(dataset.image_shape, dataset.num_classes, 4) #Get the model

x_train, y_train, x_test, y_test = dataset.get_batch()  #Acquisition of training data and test data
trainer = Trainer(model, loss="categorical_crossentropy", optimizer="adam") #Instantiation of Trainer with model, loss function, and optimization algorithm as arguments
model = trainer.fit(x_train, y_train, x_test, y_test) #Model learning

#Model evaluation
score = model.evaluate(x_test, y_test, verbose=0)
print('Test loss: ', score[0])
print('Test accuracy: ', score[1])

References

Intuition Deep Learning Why ResNet shows good performance

Recommended Posts

I implemented ResNet!
I implemented CycleGAN (1)
Qiskit: I implemented VQE
I implemented Python Logging
I implemented VQE with Blueqat
I implemented Extreme learning machine
[Python] I implemented peripheral Gibbs sampling
I implemented Attention Seq2Seq with PyTorch
I implemented a two-layer neural network
I implemented ChangeFinder (change point detection)
I implemented the K-means method (clustering method)
I implemented Shake-Shake Regularization (ShakeNet) with PyTorch
[Reinforcement learning] I implemented / explained R2D3 (Keras-RL)
I tried CNN fine tuning with Resnet
I implemented NSGA-II, a multi-objective optimization problem.
CheckIO (Python)> Non-unique Elements> I implemented it
Implemented DQN in TensorFlow (I wanted to ...)
I implemented DCGAN and tried to generate apples
I implemented the inverse gamma function in python
I implemented Google's Speech to text in Django
I read and implemented the Variants of UKR
I implemented Human In The Loop ― Part ① Dashboard ―