[PYTHON] Difference and compatibility verification between keras and tf.keras # 1

End of development of Keras

It's been a while since I posted it. As you can see on github, the multi-backend Keras seems to have ended development in April 2020. It will be replaced by Keras in Tensorflow in the future. keras-team (For convenience, multi-backend Keras is referred to as mb-keras, and Tensorflow Keras is referred to as tf-keras) I would like to compare and verify mb-keras and tf-keras in this article.

About mb-keras Tensorflow support

As mentioned on github, it seems that it supports up to Tensorflow 2.0. As of May 4, 2020, at the time of writing this article, Tensorflow's latest stabilizer is 2.1.0.

environment

In writing this article, I re-installed Tensorflow from CUDA etc., but I created a poor environment. mb-keras supports up to Python 3.6 ... (tensorflow supports up to Python 3.7) mb-keras 2.3.1 supports up to tensorflow 2.0, so this is good.

python -V
3.7.6
>>> import keras
>>> keras.__version__
'2.3.1'
>>> import tensorflow
>>> tensorflow.__version__
'2.0.0'

alias

It's basically the same, just with tensorflow in the head.

mb-keras


from keras.layers import Dense, Conv2D

tf-keras


from tensorflow.keras.layers import Dense, Conv2D

Object compatibility

Probably ** not **. However, there is no need for compatibility ... Check with the code below.

About Model and Layer

def mb_layers():
    from keras.layers import Input, Dense, Activation
    input_l = Input(shape=(64,))
    hidden = Dense(10)(input_l)
    output_l = Activation('softmax')(hidden)
    return input_l, output_l
def tf_layers():
    from tensorflow.keras.layers import Input, Dense, Activation
    input_l = Input(shape=(64,))
    hidden = Dense(10)(input_l)
    output_l = Activation('softmax')(hidden)
    return input_l, output_l

if __name__ == '__main__':
    mb_i, mb_o = mb_layers()
    tf_i, tf_o = tf_layers()
    
    from keras.models import Model as mbModel
    from tensorflow.keras.models import Model as tfModel
    
    # mb_in_tf = mbModel(tf_i, tf_o) #---- ※1
    # tf_in_mb = tfModel(mb_i, mb_o) #---- ※2
    mb_in_mb = mbModel(mb_i, mb_o)
    mb_in_mb.summary()               #---- ※3
    tf_in_tf = tfModel(tf_i, tf_o)
    tf_in_tf.summary()               #---- ※4

stdout


Model: "model_7"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_23 (InputLayer)        (None, 64)                0         
_________________________________________________________________
dense_14 (Dense)             (None, 10)                650       
_________________________________________________________________
activation_12 (Activation)   (None, 10)                0         
=================================================================
Total params: 650
Trainable params: 650
Non-trainable params: 0
_________________________________________________________________

stdout


Model: "model_2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_24 (InputLayer)        [(None, 64)]              0         
_________________________________________________________________
dense_14 (Dense)             (None, 10)                650       
_________________________________________________________________
activation_11 (Activation)   (None, 10)                0         
=================================================================
Total params: 650
Trainable params: 650
Non-trainable params: 0
_________________________________________________________________

The output shape of the InputLayer is different, but can it be implemented as the same architecture? I want to check it from the next time onwards. For the time being, it seems that the making of Layer is different between mb-keras and tf-keras, so it is certain that it can not be thrown into Model.

About Optimizer

Optimizer can use tf-keras Optimizer for Model made with mb-keras.

model.compile


    from keras.optimizers import Adam as mbAdam
    from tensorflow.keras.optimizers import Adam as tfAdam
    mb_in_mb.compile(tfAdam(), 'mse')    #----※1
    tf_in_tf.compile(mbAdam(), 'mse')    #----※2

About usable layers

This seems to be quite different. ○ is implemented, × is not implemented. Note that not everything is a layer, as we just arranged the ones taken out with dir () almost as they are. Looking at the difference in Attention, is tf-keras more advanced in revision and abolition? (Override method and all lowercase letters are function-like, so I removed them from the list)

Layer mb-keras tf-keras
AbstractRNNCell ×
Activation
ActivityRegularization
Add
AdditiveAttention ×
AlphaDropout
AtrousConvolution1D ×
AtrousConvolution2D ×
Attention ×
Average
AveragePooling1D
AveragePooling2D
AveragePooling3D
AvgPool1D
AvgPool2D
AvgPool3D
BatchNormalization
Bidirectional
Concatenate
Conv1D
Conv2D
Conv2DTranspose
Conv3D
Conv3DTranspose
ConvLSTM2D
ConvLSTM2DCell ×
ConvRecurrent2D ×
Convolution1D
Convolution2D ×
Convolution2D ×
Convolution3D
Convolution3DTranspose ×
Cropping1D
Cropping2D
Cropping3D
CuDNNGRU ×
CuDNNLSTM ×
Deconvolution2D ×
Deconvolution3D ×
Dense
DenseFeatures ×
DepthwiseConv2D
Dot
Dropout
ELU
Embedding
Flatten
GRU
GRUCell
GaussianDropout
GaussianNoise
GlobalAveragePooling1D
GlobalAveragePooling2D
GlobalAveragePooling3D
GlobalAvgPool1D
GlobalAvgPool2D
GlobalAvgPool3D
GlobalMaxPool1D
GlobalMaxPool2D
GlobalMaxPool3D
GlobalMaxPooling1D
GlobalMaxPooling2D
GlobalMaxPooling3D
Highway ×
Input
InputLayer
InputSpec
LSTM
LSTMCell
Lambda
Layer
LayerNormalization ×
LeakyReLU
LocallyConnected1D
LocallyConnected2D
Masking
MaxPool1D
MaxPool2D
MaxPool3D
MaxPooling1D
MaxPooling2D
MaxPooling3D
Maximum
MaxoutDense ×
Minimum
Multiply
PReLU
Permute
RNN
ReLU
Recurrent ×
RepeatVector
Reshape
SeparableConv1D
SeparableConv2D
SeparableConvolution1D ×
SeparableConvolution2D ×
SimpleRNN
SimpleRNNCell
Softmax
SpatialDropout1D
SpatialDropout2D
SpatialDropout3D
StackedRNNCells
Subtract
ThresholdedReLU
TimeDistributed
UpSampling1D
UpSampling2D
UpSampling3D
Wrapper ※ ×
ZeroPadding1D
ZeroPadding2D
ZeroPadding3D

This summary

It turns out that mb-keras and tf-keras are almost incompatible. However, I don't feel the need.

As a case that seems to be a problem, mb-keras supports up to Python 3.6, so it will not be supported even if Python is updated in the future. On the other hand, the development of tf-keras will continue in the future. If you have created an AI system that uses mb-keras, you may be overwhelmed by the times, so you should immediately consider replacing it with tf-keras. As a concrete example that comes to mind first, ** Can a trained model created with mb-keras be loaded with tf-keras and inferred? **Toka Toka··· (I want to record after # 2 when I don't know when.)

Actually, I just built an in-house AI system with mb-keras before GW ... If you think that keras hasn't been updated recently, this is the place to go. I got it ...

Recommended Posts

Difference and compatibility verification between keras and tf.keras # 1
Difference between process and job
Difference between "categorical_crossentropy" and "sparse_categorical_crossentropy"
Difference between regression and classification
Difference between np.array and np.arange
Difference between MicroPython and CPython
Difference between ps a and ps -a
Difference between return and print-Python
Difference between Ruby and Python split
Difference between java and python (memo)
Difference between list () and [] in Python
Difference between SQLAlchemy filter () and filter_by ()
Memorandum (difference between csv.reader and csv.dictreader)
(Note) Difference between gateway and default gateway
Difference between Numpy randint and Random randint
Difference between sort and sorted (memorial)
Difference between python2 series and python3 series dict.keys ()
Python --Difference between exec and eval
[Python] Difference between randrange () and randint ()
[Python] Difference between sorted and sorted (Colaboratory)
[Xg boost] Difference between softmax and softprob
difference between statements (statements) and expressions (expressions) in Python
[Django ORM] Difference between values () and only ()
Difference between @classmethod and @staticmethod in Python
Difference between append and + = in Python list
Difference between nonlocal and global in Python
Difference between linear regression, Ridge regression and Lasso regression
Difference between docker-compose env_file and .env file
[Python Iroha] Difference between List and Tuple
[python] Difference between rand and randn output
speed difference between wsgi, Bottle and Flask
Difference between ls -l and cat command
What is the difference between `pip` and` conda`?
Difference between using and import on shield language
[python] Difference between variables and self. Variables in class
About the difference between "==" and "is" in python
About the difference between PostgreSQL su and sudo
What is the difference between Unix and Linux?
Consideration of the difference between ROC curve and PR curve
The rough difference between Unicode and UTF-8 (and their friends)
Investigate the relationship between TensorFlow and Keras in transition
Can BERT tell the difference between "candy (candy)" and "candy (rain)"?
Difference between Ruby and Python in terms of variables
What is the difference between usleep, nanosleep and clock_nanosleep?
Difference between Numpy (n,) and (n, 1) notation [Difference between horizontal vector and vertical vector]
Difference between return, return None, and no return description in Python
How to use argparse and the difference between optparse
optuna, keras and titanic
Between parametric and nonparametric
What is the difference between a symbolic link and a hard link?
Python module num2words Difference in behavior between English and Russian
Python> Difference between inpbt and print (inpbt) output> [1. 2. 3.] / array ([1., 2., 3.], dtype = float32)
Understand the difference between cumulative assignment to variables and cumulative assignment to objects
List concatenation method in python, difference between list.extend () and “+” operator
Difference between SQLAlchemy back_populates and backref and when neither is used
The difference between foreground and background processes understood by the principle