[PYTHON] I made a VGG16 model using TensorFlow (on the way)

Introduction

――I will touch Python after a long time. I'm addicted to it now, so I'd like to post only the content and execution results and update it as it progresses. --Finally, previously, I tried to move the SSD model with copy and paste, but I want to be able to make things within myself. .. ――This time, the VGG16 model will be applied in the deep learning model, but there are not many that are carefully explained about what was implemented using TensorFlow, and the SSD model also has the concept of this VGG16 model. Since it is included, I tried it as a stepping stone. --Initially, it was implemented using TF-Slim, but it was redundant at this time because an error occurred when creating the model. I tried to implement it as it is.

Premise

-Learning data and Learning program are used in this area.

Implementation

reference

http://tensorflow.classcat.com/2016/10/18/tensorflow-vgg-model/

program

network.py


#!/usr/local/bin/python
# -*- coding: utf-8 -*-

import numpy as np
import tensorflow as tf

def vgg16(image, keep_prob):

    def weight_variable(shape):
        initial = tf.truncated_normal(shape, stddev=0.1)
        return tf.Variable(initial)

    def bias_variable(shape):
        initial = tf.constant(0.1, shape=shape)
        return tf.Variable(initial)

    def conv2d(conv, weight):
        return tf.nn.conv2d(conv, weight, strides=[1, 1, 1, 1], padding='SAME')

    def max_pool_2x2(conv):
        return tf.nn.max_pool(conv, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')

    with tf.name_scope('conv1_1') as scope:
        weight = weight_variable([3, 3, 3, 64])
        bias = bias_variable([64])
        conv = conv2d(image, weight)
        out = tf.nn.bias_add(conv, bias)
        conv1_1 = tf.nn.relu(out, name=scope)

    with tf.name_scope('conv1_2') as scope:
        weight = weight_variable([3, 3, 64, 64])
        bias = bias_variable([64])
        conv = conv2d(conv1_1, weight)
        out = tf.nn.bias_add(conv, bias)
        conv1_2 = tf.nn.relu(out, name=scope)

    with tf.name_scope('pool1') as scope:
        pool1 = max_pool_2x2(conv1_2)

    with tf.name_scope('conv2_1') as scope:
        weight = weight_variable([3, 3, 64, 128])
        bias = bias_variable([128])
        conv = conv2d(pool1, weight)
        out = tf.nn.bias_add(conv, bias)
        conv2_1 = tf.nn.relu(out, name=scope)

    with tf.name_scope('conv2_2') as scope:
        weight = weight_variable([3, 3, 128, 128])
        bias = bias_variable([128])
        conv = conv2d(conv2_1, weight)
        out = tf.nn.bias_add(conv, bias)
        conv2_2 = tf.nn.relu(out, name=scope)

    with tf.name_scope('pool2') as scope:
        pool2 = max_pool_2x2(conv2_2)

    with tf.name_scope('conv3_1') as scope:
        weight = weight_variable([3, 3, 128, 256])
        bias = bias_variable([256])
        conv = conv2d(pool2, weight)
        out = tf.nn.bias_add(conv, bias)
        conv3_1 = tf.nn.relu(out, name=scope)

    with tf.name_scope('conv3_2') as scope:
        weight = weight_variable([3, 3, 256, 256])
        bias = bias_variable([256])
        conv = conv2d(conv3_1, weight)
        out = tf.nn.bias_add(conv, bias)
        conv3_2 = tf.nn.relu(out, name=scope)

    with tf.name_scope('conv3_3') as scope:
        weight = weight_variable([3, 3, 256, 256])
        bias = bias_variable([256])
        conv = conv2d(conv3_2, weight)
        out = tf.nn.bias_add(conv, bias)
        conv3_3 = tf.nn.relu(out, name=scope)

    with tf.name_scope('pool3') as scope:
        pool3 = max_pool_2x2(conv3_3)

    with tf.name_scope('conv4_1') as scope:
        weight = weight_variable([3, 3, 256, 512])
        bias = bias_variable([512])
        conv = conv2d(pool3, weight)
        out = tf.nn.bias_add(conv, bias)
        conv4_1 = tf.nn.relu(out, name=scope)

    with tf.name_scope('conv4_2') as scope:
        weight = weight_variable([3, 3, 512, 512])
        bias = bias_variable([512])
        conv = conv2d(conv4_1, weight)
        out = tf.nn.bias_add(conv, bias)
        conv4_2 = tf.nn.relu(out, name=scope)

    with tf.name_scope('conv4_3') as scope:
        weight = weight_variable([3, 3, 512, 512])
        bias = bias_variable([512])
        conv = conv2d(conv4_2, weight)
        out = tf.nn.bias_add(conv, bias)
        conv4_3 = tf.nn.relu(out, name=scope)

    with tf.name_scope('pool4') as scope:
        pool4 = max_pool_2x2(conv4_3)

    with tf.name_scope('conv5_1') as scope:
        weight = weight_variable([3, 3, 512, 512])
        bias = bias_variable([512])
        conv = conv2d(pool4, weight)
        out = tf.nn.bias_add(conv, bias)
        conv5_1 = tf.nn.relu(out, name=scope)

    with tf.name_scope('conv5_2') as scope:
        weight = weight_variable([3, 3, 512, 512])
        bias = bias_variable([512])
        conv = conv2d(conv5_1, weight)
        out = tf.nn.bias_add(conv, bias)
        conv5_2 = tf.nn.relu(out, name=scope)

    with tf.name_scope('conv5_3') as scope:
        weight = weight_variable([3, 3, 512, 512])
        bias = bias_variable([512])
        conv = conv2d(conv5_2, weight)
        out = tf.nn.bias_add(conv, bias)
        conv5_3 = tf.nn.relu(out, name=scope)

    with tf.name_scope('pool5') as scope:
        pool5 = max_pool_2x2(conv5_3)

    with tf.name_scope('fc6') as scope:
        shape = int(np.prod(pool5.get_shape()[1:]))
        weight = weight_variable([shape, 4096])
        bias = bias_variable([4096])
        pool5_flat = tf.reshape(pool5, [-1, shape])
        fc6 = tf.nn.relu(tf.nn.bias_add(tf.matmul(pool5_flat, weight), bias))
        # fc6_drop = tf.nn.dropout(fc6, keep_prob)

    with tf.name_scope('fc7') as scope:
        weight = weight_variable([4096, 4096])
        bias = bias_variable([4096])
        fc7 = tf.nn.relu(tf.nn.bias_add(tf.matmul(fc6, weight), bias))
        # fc7_drop = tf.nn.dropout(fc7, keep_prob)

    with tf.name_scope('fc8') as scope:
        weight = weight_variable([4096, 3])
        bias = bias_variable([3])
        fc8 = tf.nn.bias_add(tf.matmul(fc7, weight), bias)

    with tf.name_scope('softmax') as scope:
        probs = tf.nn.softmax(fc8)

    return probs

Execution result

Learning doesn't go on at all!

2017-08-01 17:39:33.518478: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-08-01 17:39:33.518513: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-08-01 17:39:33.518523: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-08-01 17:39:33.518532: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
step 0, training accuracy 0.588235
step 1, training accuracy 0.588235
step 2, training accuracy 0.588235
step 3, training accuracy 0.588235
step 4, training accuracy 0.588235
step 5, training accuracy 0.588235
step 6, training accuracy 0.588235
step 7, training accuracy 0.588235

Visualization

It seems to be correct as a composition, but ...

FireShot Capture 041 - TensorBoard - http___localhost_6006_#graphs.jpg

What I tried

--The image size of the VGG model is 224x224 by default, which seems to be at least 48x48, so I set it to 48 or 224. → No change.

――I tried to reduce the number of layers. → No change.

――I changed the learning data. → No change.

――The notation is the same as it is now, and I changed it to the normal CNN layer. → Learning was done normally.

(゚ д ゚)

Recommended Posts

I made a VGG16 model using TensorFlow (on the way)
I made a Dir en gray face classifier using TensorFlow --- ⑦ Learning model
I made a Line-bot using Python!
I tried refactoring the CNN model of TensorFlow using TF-Slim
I tried hosting a TensorFlow deep learning model using TensorFlow Serving
I made a function to check the model of DCGAN
I made a demo that lets the model learned in the Tensorflow mnist tutorial distinguish the handwritten numbers written on the canvas.
I made a Dir en gray face classifier using TensorFlow --(1) Introduction
I made a Dir en gray face classifier using TensorFlow-④ Face extraction
[Kaggle] I made a collection of questions using the Titanic tutorial
I tried playing a ○ ✕ game using TensorFlow
Beginner: I made a launcher using dictionary
I built a TensorFlow environment on windows10
I made a Dir en gray face classifier using TensorFlow --⑩ Face classification test
I made a kitchen timer to be displayed on the status bar!
I made a Dir en gray face classifier using TensorFlow --⑥ Learning program
I made a Dir en gray face classifier using TensorFlow --⑬ Playing (final)
I made a Dir en gray face classifier using TensorFlow --- ⑧ Learning execution
I made a Dir en gray face classifier using TensorFlow --⑫ Web release
I tried using the trained model VGG16 of the deep learning library Keras
I made a Dir en gray face classifier using TensorFlow --② Environment construction
I made an image discrimination (cifar10) model using a convolutional neural network.
A addictive story when using tensorflow on Android
I tried to make a ○ ✕ game using TensorFlow
Create a GUI on the terminal using curses
I did a little research on the class
I made a LINE BOT that returns a terrorist image using the Flickr API
Create a REST API using the model learned in Lobe and TensorFlow Serving.
I made a Dir en gray face classifier using TensorFlow --⑪ Web release preparation
I made a program to look up words on the window (previous development)
I made a script to record the active window using win32gui of Python
〇✕ I made a game
I made a login / logout process using Python Bottle.
I tried hosting a Pytorch sample model using TorchServe
I made a code to convert illustration2vec to keras model
I made a Python3 environment on Ubuntu with direnv.
Make the model a string on a Django HTML template
[MNIST] I tried Fine Tuning using the ImageNet model.
I made a command to markdown the table clipboard
I made a school festival introduction game using Ren’py
PyTorch Learning Note 2 (I tried using a pre-trained model)
I want to take a screenshot of the site on Docker using any font
Implementation of VGG16 using Keras created without using a trained model
PyTorch learning memo (I made the same model as Karas)
I made a quick feed reader using feedparser in Python
I made a dot picture of the image of Irasutoya. (part1)
I made an anomaly detection model that works on iOS
I made a python text
I tried using magenta / TensorFlow
I made a discord bot
Try to model a multimodal distribution using the EM algorithm
I made a dot picture of the image of Irasutoya. (part2)
I made a muscle training estimation app using Qore SDK
I made an original program guide using the NHK program guide API.
I made a Chatbot using LINE Messaging API and Python
I made a neural network generator that runs on FPGA
[For beginners] I tried using the Tensorflow Object Detection API
I tried using "Asciichart Py" which can draw a beautiful graph on the console with Python.
[Python] I tried to make a simple program that works on the command line using argparse.
I made a program to solve (hint) Saizeriya's spot the difference
Control the motor with a motor driver using python on Raspberry Pi 3!