Note
Go to the end to download the full example code.
Global Akida workflow
Using the MNIST dataset, this example shows the definition and training of a keras floating point model, its quantization to 8-bit with the help of calibration, its quantization to 4-bit using QAT and its conversion to Akida. Notice that the performance of the original keras floating point model is maintained throughout the Akida flow. Please refer to the Akida user guide for further information.
Note
Please refer to the TensorFlow tf.keras.models module for model creation/import details and the TensorFlow Guide for TensorFlow usage.
The MNIST example below is light enough so that a GPU is not needed for training.
Global Akida workflow
1. Create and train
1.1. Load and reshape MNIST dataset
import numpy as np
import matplotlib.cm as cm
import matplotlib.pyplot as plt
from keras.datasets import mnist
# Load MNIST dataset
(x_train, y_train), (x_test, y_test) = mnist.load_data()
# Add a channels dimension to the image sets as Akida expects 4-D inputs (corresponding to
# (num_samples, width, height, channels). Note: MNIST is a grayscale dataset and is unusual
# in this respect - most image data already includes a channel dimension, and this step will
# not be necessary.
x_train = np.expand_dims(x_train, -1)
x_test = np.expand_dims(x_test, -1)
# Display a few images from the test set
f, axarr = plt.subplots(1, 4)
for i in range(0, 4):
axarr[i].imshow(x_test[i].reshape((28, 28)), cmap=cm.Greys_r)
axarr[i].set_title('Class %d' % y_test[i])
plt.show()
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
8192/11490434 [..............................] - ETA: 0s
16384/11490434 [..............................] - ETA: 45s
49152/11490434 [..............................] - ETA: 36s
81920/11490434 [..............................] - ETA: 34s
147456/11490434 [..............................] - ETA: 23s
212992/11490434 [..............................] - ETA: 19s
327680/11490434 [..............................] - ETA: 14s
475136/11490434 [>.............................] - ETA: 10s
753664/11490434 [>.............................] - ETA: 7s
1130496/11490434 [=>............................] - ETA: 5s
1753088/11490434 [===>..........................] - ETA: 3s
2736128/11490434 [======>.......................] - ETA: 2s
4136960/11490434 [=========>....................] - ETA: 1s
5750784/11490434 [==============>...............] - ETA: 0s
7536640/11490434 [==================>...........] - ETA: 0s
9191424/11490434 [======================>.......] - ETA: 0s
10797056/11490434 [===========================>..] - ETA: 0s
11490434/11490434 [==============================] - 1s 0us/step
1.2. Model definition
Note that at this stage, there is nothing specific to the Akida IP. The model constructed below, as inspired by this example, is a completely standard Keras CNN model.
import keras
model_keras = keras.models.Sequential([
keras.layers.Rescaling(1. / 255, input_shape=(28, 28, 1)),
keras.layers.Conv2D(filters=32, kernel_size=3, strides=2),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
# Separable layer
keras.layers.DepthwiseConv2D(kernel_size=3, padding='same', strides=2),
keras.layers.Conv2D(filters=64, kernel_size=1, padding='same'),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
keras.layers.Flatten(),
keras.layers.Dense(10)
], 'mnistnet')
model_keras.summary()
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling (Rescaling) (None, 28, 28, 1) 0
conv2d (Conv2D) (None, 13, 13, 32) 320
batch_normalization (Batch (None, 13, 13, 32) 128
Normalization)
re_lu (ReLU) (None, 13, 13, 32) 0
depthwise_conv2d (Depthwis (None, 7, 7, 32) 320
eConv2D)
conv2d_1 (Conv2D) (None, 7, 7, 64) 2112
batch_normalization_1 (Bat (None, 7, 7, 64) 256
chNormalization)
re_lu_1 (ReLU) (None, 7, 7, 64) 0
flatten (Flatten) (None, 3136) 0
dense (Dense) (None, 10) 31370
=================================================================
Total params: 34506 (134.79 KB)
Trainable params: 34314 (134.04 KB)
Non-trainable params: 192 (768.00 Byte)
_________________________________________________________________
1.3. Model training
Given the model created above, train the model and check its accuracy. The model should achieve a test accuracy over 98% after 10 epochs.
from keras.optimizers import Adam
model_keras.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-3),
metrics=['accuracy'])
_ = model_keras.fit(x_train, y_train, epochs=10, validation_split=0.1)
Epoch 1/10
1/1688 [..............................] - ETA: 1:03:39 - loss: 2.8431 - accuracy: 0.0625
21/1688 [..............................] - ETA: 4s - loss: 1.5248 - accuracy: 0.5104
42/1688 [..............................] - ETA: 4s - loss: 1.1159 - accuracy: 0.6458
63/1688 [>.............................] - ETA: 4s - loss: 0.9047 - accuracy: 0.7168
84/1688 [>.............................] - ETA: 3s - loss: 0.7900 - accuracy: 0.7530
105/1688 [>.............................] - ETA: 3s - loss: 0.7111 - accuracy: 0.7798
126/1688 [=>............................] - ETA: 3s - loss: 0.6467 - accuracy: 0.8003
146/1688 [=>............................] - ETA: 3s - loss: 0.6039 - accuracy: 0.8151
167/1688 [=>............................] - ETA: 3s - loss: 0.5623 - accuracy: 0.8292
189/1688 [==>...........................] - ETA: 3s - loss: 0.5265 - accuracy: 0.8391
210/1688 [==>...........................] - ETA: 3s - loss: 0.5003 - accuracy: 0.8470
231/1688 [===>..........................] - ETA: 3s - loss: 0.4802 - accuracy: 0.8544
252/1688 [===>..........................] - ETA: 3s - loss: 0.4601 - accuracy: 0.8609
273/1688 [===>..........................] - ETA: 3s - loss: 0.4458 - accuracy: 0.8647
293/1688 [====>.........................] - ETA: 3s - loss: 0.4320 - accuracy: 0.8686
314/1688 [====>.........................] - ETA: 3s - loss: 0.4148 - accuracy: 0.8739
335/1688 [====>.........................] - ETA: 3s - loss: 0.4057 - accuracy: 0.8773
356/1688 [=====>........................] - ETA: 3s - loss: 0.3946 - accuracy: 0.8809
377/1688 [=====>........................] - ETA: 3s - loss: 0.3833 - accuracy: 0.8842
398/1688 [======>.......................] - ETA: 3s - loss: 0.3706 - accuracy: 0.8883
419/1688 [======>.......................] - ETA: 3s - loss: 0.3611 - accuracy: 0.8913
441/1688 [======>.......................] - ETA: 3s - loss: 0.3519 - accuracy: 0.8941
462/1688 [=======>......................] - ETA: 3s - loss: 0.3423 - accuracy: 0.8971
483/1688 [=======>......................] - ETA: 2s - loss: 0.3342 - accuracy: 0.8995
504/1688 [=======>......................] - ETA: 2s - loss: 0.3269 - accuracy: 0.9015
526/1688 [========>.....................] - ETA: 2s - loss: 0.3212 - accuracy: 0.9032
547/1688 [========>.....................] - ETA: 2s - loss: 0.3156 - accuracy: 0.9051
568/1688 [=========>....................] - ETA: 2s - loss: 0.3088 - accuracy: 0.9074
590/1688 [=========>....................] - ETA: 2s - loss: 0.3027 - accuracy: 0.9090
611/1688 [=========>....................] - ETA: 2s - loss: 0.2983 - accuracy: 0.9105
632/1688 [==========>...................] - ETA: 2s - loss: 0.2941 - accuracy: 0.9116
653/1688 [==========>...................] - ETA: 2s - loss: 0.2886 - accuracy: 0.9135
674/1688 [==========>...................] - ETA: 2s - loss: 0.2823 - accuracy: 0.9153
694/1688 [===========>..................] - ETA: 2s - loss: 0.2771 - accuracy: 0.9170
715/1688 [===========>..................] - ETA: 2s - loss: 0.2721 - accuracy: 0.9187
735/1688 [============>.................] - ETA: 2s - loss: 0.2685 - accuracy: 0.9195
756/1688 [============>.................] - ETA: 2s - loss: 0.2644 - accuracy: 0.9205
777/1688 [============>.................] - ETA: 2s - loss: 0.2607 - accuracy: 0.9218
799/1688 [=============>................] - ETA: 2s - loss: 0.2572 - accuracy: 0.9229
820/1688 [=============>................] - ETA: 2s - loss: 0.2539 - accuracy: 0.9240
841/1688 [=============>................] - ETA: 2s - loss: 0.2501 - accuracy: 0.9252
863/1688 [==============>...............] - ETA: 2s - loss: 0.2465 - accuracy: 0.9262
884/1688 [==============>...............] - ETA: 1s - loss: 0.2432 - accuracy: 0.9271
905/1688 [===============>..............] - ETA: 1s - loss: 0.2406 - accuracy: 0.9281
926/1688 [===============>..............] - ETA: 1s - loss: 0.2373 - accuracy: 0.9291
947/1688 [===============>..............] - ETA: 1s - loss: 0.2340 - accuracy: 0.9299
968/1688 [================>.............] - ETA: 1s - loss: 0.2311 - accuracy: 0.9309
989/1688 [================>.............] - ETA: 1s - loss: 0.2285 - accuracy: 0.9316
1010/1688 [================>.............] - ETA: 1s - loss: 0.2263 - accuracy: 0.9323
1031/1688 [=================>............] - ETA: 1s - loss: 0.2242 - accuracy: 0.9330
1052/1688 [=================>............] - ETA: 1s - loss: 0.2220 - accuracy: 0.9336
1073/1688 [==================>...........] - ETA: 1s - loss: 0.2201 - accuracy: 0.9342
1094/1688 [==================>...........] - ETA: 1s - loss: 0.2179 - accuracy: 0.9349
1115/1688 [==================>...........] - ETA: 1s - loss: 0.2161 - accuracy: 0.9355
1136/1688 [===================>..........] - ETA: 1s - loss: 0.2143 - accuracy: 0.9360
1157/1688 [===================>..........] - ETA: 1s - loss: 0.2117 - accuracy: 0.9367
1179/1688 [===================>..........] - ETA: 1s - loss: 0.2096 - accuracy: 0.9373
1200/1688 [====================>.........] - ETA: 1s - loss: 0.2076 - accuracy: 0.9379
1222/1688 [====================>.........] - ETA: 1s - loss: 0.2058 - accuracy: 0.9385
1243/1688 [=====================>........] - ETA: 1s - loss: 0.2043 - accuracy: 0.9388
1264/1688 [=====================>........] - ETA: 1s - loss: 0.2019 - accuracy: 0.9395
1286/1688 [=====================>........] - ETA: 0s - loss: 0.1998 - accuracy: 0.9400
1307/1688 [======================>.......] - ETA: 0s - loss: 0.1979 - accuracy: 0.9404
1328/1688 [======================>.......] - ETA: 0s - loss: 0.1958 - accuracy: 0.9410
1349/1688 [======================>.......] - ETA: 0s - loss: 0.1943 - accuracy: 0.9414
1370/1688 [=======================>......] - ETA: 0s - loss: 0.1922 - accuracy: 0.9420
1390/1688 [=======================>......] - ETA: 0s - loss: 0.1907 - accuracy: 0.9424
1410/1688 [========================>.....] - ETA: 0s - loss: 0.1894 - accuracy: 0.9429
1431/1688 [========================>.....] - ETA: 0s - loss: 0.1883 - accuracy: 0.9432
1453/1688 [========================>.....] - ETA: 0s - loss: 0.1871 - accuracy: 0.9437
1474/1688 [=========================>....] - ETA: 0s - loss: 0.1859 - accuracy: 0.9441
1494/1688 [=========================>....] - ETA: 0s - loss: 0.1844 - accuracy: 0.9444
1515/1688 [=========================>....] - ETA: 0s - loss: 0.1833 - accuracy: 0.9448
1536/1688 [==========================>...] - ETA: 0s - loss: 0.1821 - accuracy: 0.9452
1557/1688 [==========================>...] - ETA: 0s - loss: 0.1809 - accuracy: 0.9455
1577/1688 [===========================>..] - ETA: 0s - loss: 0.1799 - accuracy: 0.9456
1598/1688 [===========================>..] - ETA: 0s - loss: 0.1789 - accuracy: 0.9460
1619/1688 [===========================>..] - ETA: 0s - loss: 0.1777 - accuracy: 0.9463
1640/1688 [============================>.] - ETA: 0s - loss: 0.1767 - accuracy: 0.9466
1662/1688 [============================>.] - ETA: 0s - loss: 0.1763 - accuracy: 0.9467
1683/1688 [============================>.] - ETA: 0s - loss: 0.1750 - accuracy: 0.9471
1688/1688 [==============================] - ETA: 0s - loss: 0.1747 - accuracy: 0.9472
1688/1688 [==============================] - 7s 3ms/step - loss: 0.1747 - accuracy: 0.9472 - val_loss: 0.0755 - val_accuracy: 0.9780
Epoch 2/10
1/1688 [..............................] - ETA: 4s - loss: 0.0206 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0653 - accuracy: 0.9769
44/1688 [..............................] - ETA: 3s - loss: 0.0639 - accuracy: 0.9808
66/1688 [>.............................] - ETA: 3s - loss: 0.0651 - accuracy: 0.9811
87/1688 [>.............................] - ETA: 3s - loss: 0.0619 - accuracy: 0.9817
109/1688 [>.............................] - ETA: 3s - loss: 0.0635 - accuracy: 0.9802
130/1688 [=>............................] - ETA: 3s - loss: 0.0647 - accuracy: 0.9803
151/1688 [=>............................] - ETA: 3s - loss: 0.0615 - accuracy: 0.9812
171/1688 [==>...........................] - ETA: 3s - loss: 0.0677 - accuracy: 0.9793
192/1688 [==>...........................] - ETA: 3s - loss: 0.0670 - accuracy: 0.9790
212/1688 [==>...........................] - ETA: 3s - loss: 0.0654 - accuracy: 0.9797
233/1688 [===>..........................] - ETA: 3s - loss: 0.0658 - accuracy: 0.9792
254/1688 [===>..........................] - ETA: 3s - loss: 0.0677 - accuracy: 0.9783
275/1688 [===>..........................] - ETA: 3s - loss: 0.0663 - accuracy: 0.9790
297/1688 [====>.........................] - ETA: 3s - loss: 0.0683 - accuracy: 0.9780
318/1688 [====>.........................] - ETA: 3s - loss: 0.0705 - accuracy: 0.9779
339/1688 [=====>........................] - ETA: 3s - loss: 0.0702 - accuracy: 0.9782
360/1688 [=====>........................] - ETA: 3s - loss: 0.0707 - accuracy: 0.9778
381/1688 [=====>........................] - ETA: 3s - loss: 0.0718 - accuracy: 0.9774
402/1688 [======>.......................] - ETA: 3s - loss: 0.0726 - accuracy: 0.9771
424/1688 [======>.......................] - ETA: 3s - loss: 0.0730 - accuracy: 0.9769
445/1688 [======>.......................] - ETA: 3s - loss: 0.0716 - accuracy: 0.9772
466/1688 [=======>......................] - ETA: 2s - loss: 0.0713 - accuracy: 0.9775
487/1688 [=======>......................] - ETA: 2s - loss: 0.0705 - accuracy: 0.9777
507/1688 [========>.....................] - ETA: 2s - loss: 0.0707 - accuracy: 0.9773
529/1688 [========>.....................] - ETA: 2s - loss: 0.0723 - accuracy: 0.9770
550/1688 [========>.....................] - ETA: 2s - loss: 0.0730 - accuracy: 0.9767
571/1688 [=========>....................] - ETA: 2s - loss: 0.0734 - accuracy: 0.9767
593/1688 [=========>....................] - ETA: 2s - loss: 0.0725 - accuracy: 0.9769
614/1688 [=========>....................] - ETA: 2s - loss: 0.0731 - accuracy: 0.9767
635/1688 [==========>...................] - ETA: 2s - loss: 0.0723 - accuracy: 0.9771
656/1688 [==========>...................] - ETA: 2s - loss: 0.0725 - accuracy: 0.9772
677/1688 [===========>..................] - ETA: 2s - loss: 0.0726 - accuracy: 0.9774
698/1688 [===========>..................] - ETA: 2s - loss: 0.0723 - accuracy: 0.9773
719/1688 [===========>..................] - ETA: 2s - loss: 0.0719 - accuracy: 0.9774
741/1688 [============>.................] - ETA: 2s - loss: 0.0723 - accuracy: 0.9774
762/1688 [============>.................] - ETA: 2s - loss: 0.0730 - accuracy: 0.9773
783/1688 [============>.................] - ETA: 2s - loss: 0.0735 - accuracy: 0.9773
804/1688 [=============>................] - ETA: 2s - loss: 0.0734 - accuracy: 0.9773
825/1688 [=============>................] - ETA: 2s - loss: 0.0727 - accuracy: 0.9774
846/1688 [==============>...............] - ETA: 2s - loss: 0.0724 - accuracy: 0.9776
867/1688 [==============>...............] - ETA: 1s - loss: 0.0719 - accuracy: 0.9777
888/1688 [==============>...............] - ETA: 1s - loss: 0.0722 - accuracy: 0.9776
909/1688 [===============>..............] - ETA: 1s - loss: 0.0725 - accuracy: 0.9776
930/1688 [===============>..............] - ETA: 1s - loss: 0.0730 - accuracy: 0.9773
951/1688 [===============>..............] - ETA: 1s - loss: 0.0728 - accuracy: 0.9773
972/1688 [================>.............] - ETA: 1s - loss: 0.0722 - accuracy: 0.9775
992/1688 [================>.............] - ETA: 1s - loss: 0.0724 - accuracy: 0.9776
1012/1688 [================>.............] - ETA: 1s - loss: 0.0727 - accuracy: 0.9776
1033/1688 [=================>............] - ETA: 1s - loss: 0.0724 - accuracy: 0.9777
1054/1688 [=================>............] - ETA: 1s - loss: 0.0726 - accuracy: 0.9778
1075/1688 [==================>...........] - ETA: 1s - loss: 0.0721 - accuracy: 0.9778
1096/1688 [==================>...........] - ETA: 1s - loss: 0.0716 - accuracy: 0.9779
1117/1688 [==================>...........] - ETA: 1s - loss: 0.0714 - accuracy: 0.9779
1138/1688 [===================>..........] - ETA: 1s - loss: 0.0713 - accuracy: 0.9779
1159/1688 [===================>..........] - ETA: 1s - loss: 0.0710 - accuracy: 0.9780
1180/1688 [===================>..........] - ETA: 1s - loss: 0.0711 - accuracy: 0.9778
1202/1688 [====================>.........] - ETA: 1s - loss: 0.0707 - accuracy: 0.9778
1223/1688 [====================>.........] - ETA: 1s - loss: 0.0705 - accuracy: 0.9779
1244/1688 [=====================>........] - ETA: 1s - loss: 0.0709 - accuracy: 0.9779
1265/1688 [=====================>........] - ETA: 1s - loss: 0.0707 - accuracy: 0.9779
1286/1688 [=====================>........] - ETA: 0s - loss: 0.0709 - accuracy: 0.9780
1308/1688 [======================>.......] - ETA: 0s - loss: 0.0706 - accuracy: 0.9781
1329/1688 [======================>.......] - ETA: 0s - loss: 0.0705 - accuracy: 0.9781
1350/1688 [======================>.......] - ETA: 0s - loss: 0.0700 - accuracy: 0.9783
1371/1688 [=======================>......] - ETA: 0s - loss: 0.0697 - accuracy: 0.9783
1392/1688 [=======================>......] - ETA: 0s - loss: 0.0700 - accuracy: 0.9782
1413/1688 [========================>.....] - ETA: 0s - loss: 0.0699 - accuracy: 0.9783
1434/1688 [========================>.....] - ETA: 0s - loss: 0.0700 - accuracy: 0.9782
1455/1688 [========================>.....] - ETA: 0s - loss: 0.0697 - accuracy: 0.9783
1475/1688 [=========================>....] - ETA: 0s - loss: 0.0696 - accuracy: 0.9783
1496/1688 [=========================>....] - ETA: 0s - loss: 0.0698 - accuracy: 0.9783
1517/1688 [=========================>....] - ETA: 0s - loss: 0.0696 - accuracy: 0.9784
1537/1688 [==========================>...] - ETA: 0s - loss: 0.0694 - accuracy: 0.9784
1558/1688 [==========================>...] - ETA: 0s - loss: 0.0693 - accuracy: 0.9785
1579/1688 [===========================>..] - ETA: 0s - loss: 0.0690 - accuracy: 0.9785
1600/1688 [===========================>..] - ETA: 0s - loss: 0.0689 - accuracy: 0.9785
1622/1688 [===========================>..] - ETA: 0s - loss: 0.0689 - accuracy: 0.9785
1643/1688 [============================>.] - ETA: 0s - loss: 0.0689 - accuracy: 0.9785
1664/1688 [============================>.] - ETA: 0s - loss: 0.0693 - accuracy: 0.9784
1684/1688 [============================>.] - ETA: 0s - loss: 0.0691 - accuracy: 0.9784
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0690 - accuracy: 0.9784 - val_loss: 0.0672 - val_accuracy: 0.9833
Epoch 3/10
1/1688 [..............................] - ETA: 4s - loss: 0.0054 - accuracy: 1.0000
22/1688 [..............................] - ETA: 4s - loss: 0.0487 - accuracy: 0.9830
43/1688 [..............................] - ETA: 3s - loss: 0.0388 - accuracy: 0.9884
64/1688 [>.............................] - ETA: 3s - loss: 0.0399 - accuracy: 0.9868
85/1688 [>.............................] - ETA: 3s - loss: 0.0439 - accuracy: 0.9846
106/1688 [>.............................] - ETA: 3s - loss: 0.0442 - accuracy: 0.9847
127/1688 [=>............................] - ETA: 3s - loss: 0.0407 - accuracy: 0.9860
148/1688 [=>............................] - ETA: 3s - loss: 0.0401 - accuracy: 0.9861
169/1688 [==>...........................] - ETA: 3s - loss: 0.0414 - accuracy: 0.9854
190/1688 [==>...........................] - ETA: 3s - loss: 0.0432 - accuracy: 0.9857
211/1688 [==>...........................] - ETA: 3s - loss: 0.0444 - accuracy: 0.9858
232/1688 [===>..........................] - ETA: 3s - loss: 0.0450 - accuracy: 0.9856
253/1688 [===>..........................] - ETA: 3s - loss: 0.0440 - accuracy: 0.9860
274/1688 [===>..........................] - ETA: 3s - loss: 0.0444 - accuracy: 0.9857
295/1688 [====>.........................] - ETA: 3s - loss: 0.0456 - accuracy: 0.9853
317/1688 [====>.........................] - ETA: 3s - loss: 0.0464 - accuracy: 0.9849
339/1688 [=====>........................] - ETA: 3s - loss: 0.0451 - accuracy: 0.9853
360/1688 [=====>........................] - ETA: 3s - loss: 0.0450 - accuracy: 0.9855
381/1688 [=====>........................] - ETA: 3s - loss: 0.0449 - accuracy: 0.9856
402/1688 [======>.......................] - ETA: 3s - loss: 0.0449 - accuracy: 0.9854
423/1688 [======>.......................] - ETA: 3s - loss: 0.0464 - accuracy: 0.9849
443/1688 [======>.......................] - ETA: 3s - loss: 0.0473 - accuracy: 0.9845
463/1688 [=======>......................] - ETA: 2s - loss: 0.0476 - accuracy: 0.9843
484/1688 [=======>......................] - ETA: 2s - loss: 0.0473 - accuracy: 0.9843
506/1688 [=======>......................] - ETA: 2s - loss: 0.0486 - accuracy: 0.9838
526/1688 [========>.....................] - ETA: 2s - loss: 0.0479 - accuracy: 0.9840
547/1688 [========>.....................] - ETA: 2s - loss: 0.0488 - accuracy: 0.9838
568/1688 [=========>....................] - ETA: 2s - loss: 0.0493 - accuracy: 0.9834
589/1688 [=========>....................] - ETA: 2s - loss: 0.0490 - accuracy: 0.9836
610/1688 [=========>....................] - ETA: 2s - loss: 0.0492 - accuracy: 0.9834
631/1688 [==========>...................] - ETA: 2s - loss: 0.0485 - accuracy: 0.9837
652/1688 [==========>...................] - ETA: 2s - loss: 0.0480 - accuracy: 0.9839
673/1688 [==========>...................] - ETA: 2s - loss: 0.0478 - accuracy: 0.9839
694/1688 [===========>..................] - ETA: 2s - loss: 0.0483 - accuracy: 0.9839
715/1688 [===========>..................] - ETA: 2s - loss: 0.0481 - accuracy: 0.9840
736/1688 [============>.................] - ETA: 2s - loss: 0.0491 - accuracy: 0.9837
758/1688 [============>.................] - ETA: 2s - loss: 0.0492 - accuracy: 0.9837
779/1688 [============>.................] - ETA: 2s - loss: 0.0490 - accuracy: 0.9838
799/1688 [=============>................] - ETA: 2s - loss: 0.0490 - accuracy: 0.9838
820/1688 [=============>................] - ETA: 2s - loss: 0.0497 - accuracy: 0.9835
841/1688 [=============>................] - ETA: 2s - loss: 0.0499 - accuracy: 0.9835
862/1688 [==============>...............] - ETA: 2s - loss: 0.0493 - accuracy: 0.9836
883/1688 [==============>...............] - ETA: 1s - loss: 0.0493 - accuracy: 0.9835
904/1688 [===============>..............] - ETA: 1s - loss: 0.0498 - accuracy: 0.9834
925/1688 [===============>..............] - ETA: 1s - loss: 0.0498 - accuracy: 0.9834
946/1688 [===============>..............] - ETA: 1s - loss: 0.0501 - accuracy: 0.9833
967/1688 [================>.............] - ETA: 1s - loss: 0.0500 - accuracy: 0.9834
988/1688 [================>.............] - ETA: 1s - loss: 0.0506 - accuracy: 0.9833
1009/1688 [================>.............] - ETA: 1s - loss: 0.0507 - accuracy: 0.9833
1030/1688 [=================>............] - ETA: 1s - loss: 0.0506 - accuracy: 0.9833
1051/1688 [=================>............] - ETA: 1s - loss: 0.0509 - accuracy: 0.9832
1071/1688 [==================>...........] - ETA: 1s - loss: 0.0515 - accuracy: 0.9832
1092/1688 [==================>...........] - ETA: 1s - loss: 0.0515 - accuracy: 0.9832
1113/1688 [==================>...........] - ETA: 1s - loss: 0.0514 - accuracy: 0.9832
1133/1688 [===================>..........] - ETA: 1s - loss: 0.0515 - accuracy: 0.9832
1153/1688 [===================>..........] - ETA: 1s - loss: 0.0513 - accuracy: 0.9832
1174/1688 [===================>..........] - ETA: 1s - loss: 0.0519 - accuracy: 0.9830
1195/1688 [====================>.........] - ETA: 1s - loss: 0.0524 - accuracy: 0.9828
1216/1688 [====================>.........] - ETA: 1s - loss: 0.0528 - accuracy: 0.9828
1237/1688 [====================>.........] - ETA: 1s - loss: 0.0527 - accuracy: 0.9829
1258/1688 [=====================>........] - ETA: 1s - loss: 0.0527 - accuracy: 0.9828
1279/1688 [=====================>........] - ETA: 1s - loss: 0.0526 - accuracy: 0.9828
1300/1688 [======================>.......] - ETA: 0s - loss: 0.0526 - accuracy: 0.9828
1321/1688 [======================>.......] - ETA: 0s - loss: 0.0522 - accuracy: 0.9829
1342/1688 [======================>.......] - ETA: 0s - loss: 0.0522 - accuracy: 0.9829
1363/1688 [=======================>......] - ETA: 0s - loss: 0.0521 - accuracy: 0.9829
1384/1688 [=======================>......] - ETA: 0s - loss: 0.0519 - accuracy: 0.9830
1404/1688 [=======================>......] - ETA: 0s - loss: 0.0523 - accuracy: 0.9830
1425/1688 [========================>.....] - ETA: 0s - loss: 0.0519 - accuracy: 0.9830
1446/1688 [========================>.....] - ETA: 0s - loss: 0.0520 - accuracy: 0.9830
1467/1688 [=========================>....] - ETA: 0s - loss: 0.0518 - accuracy: 0.9831
1488/1688 [=========================>....] - ETA: 0s - loss: 0.0517 - accuracy: 0.9830
1508/1688 [=========================>....] - ETA: 0s - loss: 0.0519 - accuracy: 0.9830
1528/1688 [==========================>...] - ETA: 0s - loss: 0.0518 - accuracy: 0.9830
1549/1688 [==========================>...] - ETA: 0s - loss: 0.0516 - accuracy: 0.9831
1570/1688 [==========================>...] - ETA: 0s - loss: 0.0518 - accuracy: 0.9831
1591/1688 [===========================>..] - ETA: 0s - loss: 0.0516 - accuracy: 0.9831
1613/1688 [===========================>..] - ETA: 0s - loss: 0.0514 - accuracy: 0.9832
1634/1688 [============================>.] - ETA: 0s - loss: 0.0514 - accuracy: 0.9832
1655/1688 [============================>.] - ETA: 0s - loss: 0.0512 - accuracy: 0.9832
1676/1688 [============================>.] - ETA: 0s - loss: 0.0514 - accuracy: 0.9832
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0514 - accuracy: 0.9831 - val_loss: 0.0674 - val_accuracy: 0.9812
Epoch 4/10
1/1688 [..............................] - ETA: 4s - loss: 0.0093 - accuracy: 1.0000
22/1688 [..............................] - ETA: 4s - loss: 0.0248 - accuracy: 0.9886
43/1688 [..............................] - ETA: 4s - loss: 0.0291 - accuracy: 0.9891
64/1688 [>.............................] - ETA: 4s - loss: 0.0305 - accuracy: 0.9883
85/1688 [>.............................] - ETA: 3s - loss: 0.0293 - accuracy: 0.9893
106/1688 [>.............................] - ETA: 3s - loss: 0.0336 - accuracy: 0.9888
127/1688 [=>............................] - ETA: 3s - loss: 0.0335 - accuracy: 0.9887
148/1688 [=>............................] - ETA: 3s - loss: 0.0352 - accuracy: 0.9888
169/1688 [==>...........................] - ETA: 3s - loss: 0.0371 - accuracy: 0.9884
191/1688 [==>...........................] - ETA: 3s - loss: 0.0383 - accuracy: 0.9876
212/1688 [==>...........................] - ETA: 3s - loss: 0.0377 - accuracy: 0.9878
233/1688 [===>..........................] - ETA: 3s - loss: 0.0372 - accuracy: 0.9877
254/1688 [===>..........................] - ETA: 3s - loss: 0.0383 - accuracy: 0.9875
275/1688 [===>..........................] - ETA: 3s - loss: 0.0379 - accuracy: 0.9873
296/1688 [====>.........................] - ETA: 3s - loss: 0.0388 - accuracy: 0.9870
317/1688 [====>.........................] - ETA: 3s - loss: 0.0387 - accuracy: 0.9870
338/1688 [=====>........................] - ETA: 3s - loss: 0.0390 - accuracy: 0.9868
359/1688 [=====>........................] - ETA: 3s - loss: 0.0384 - accuracy: 0.9871
380/1688 [=====>........................] - ETA: 3s - loss: 0.0385 - accuracy: 0.9872
400/1688 [======>.......................] - ETA: 3s - loss: 0.0393 - accuracy: 0.9872
421/1688 [======>.......................] - ETA: 3s - loss: 0.0399 - accuracy: 0.9872
442/1688 [======>.......................] - ETA: 3s - loss: 0.0404 - accuracy: 0.9868
463/1688 [=======>......................] - ETA: 3s - loss: 0.0402 - accuracy: 0.9871
484/1688 [=======>......................] - ETA: 2s - loss: 0.0405 - accuracy: 0.9868
505/1688 [=======>......................] - ETA: 2s - loss: 0.0401 - accuracy: 0.9869
527/1688 [========>.....................] - ETA: 2s - loss: 0.0399 - accuracy: 0.9870
548/1688 [========>.....................] - ETA: 2s - loss: 0.0401 - accuracy: 0.9869
569/1688 [=========>....................] - ETA: 2s - loss: 0.0404 - accuracy: 0.9867
589/1688 [=========>....................] - ETA: 2s - loss: 0.0408 - accuracy: 0.9866
609/1688 [=========>....................] - ETA: 2s - loss: 0.0410 - accuracy: 0.9868
630/1688 [==========>...................] - ETA: 2s - loss: 0.0413 - accuracy: 0.9868
650/1688 [==========>...................] - ETA: 2s - loss: 0.0413 - accuracy: 0.9868
672/1688 [==========>...................] - ETA: 2s - loss: 0.0408 - accuracy: 0.9870
693/1688 [===========>..................] - ETA: 2s - loss: 0.0404 - accuracy: 0.9871
714/1688 [===========>..................] - ETA: 2s - loss: 0.0407 - accuracy: 0.9871
735/1688 [============>.................] - ETA: 2s - loss: 0.0407 - accuracy: 0.9869
755/1688 [============>.................] - ETA: 2s - loss: 0.0405 - accuracy: 0.9870
776/1688 [============>.................] - ETA: 2s - loss: 0.0408 - accuracy: 0.9869
798/1688 [=============>................] - ETA: 2s - loss: 0.0411 - accuracy: 0.9869
819/1688 [=============>................] - ETA: 2s - loss: 0.0407 - accuracy: 0.9870
840/1688 [=============>................] - ETA: 2s - loss: 0.0405 - accuracy: 0.9871
862/1688 [==============>...............] - ETA: 2s - loss: 0.0405 - accuracy: 0.9872
884/1688 [==============>...............] - ETA: 1s - loss: 0.0404 - accuracy: 0.9872
905/1688 [===============>..............] - ETA: 1s - loss: 0.0405 - accuracy: 0.9872
925/1688 [===============>..............] - ETA: 1s - loss: 0.0412 - accuracy: 0.9870
946/1688 [===============>..............] - ETA: 1s - loss: 0.0410 - accuracy: 0.9871
967/1688 [================>.............] - ETA: 1s - loss: 0.0409 - accuracy: 0.9870
988/1688 [================>.............] - ETA: 1s - loss: 0.0412 - accuracy: 0.9870
1009/1688 [================>.............] - ETA: 1s - loss: 0.0410 - accuracy: 0.9871
1030/1688 [=================>............] - ETA: 1s - loss: 0.0406 - accuracy: 0.9871
1051/1688 [=================>............] - ETA: 1s - loss: 0.0404 - accuracy: 0.9871
1072/1688 [==================>...........] - ETA: 1s - loss: 0.0403 - accuracy: 0.9872
1093/1688 [==================>...........] - ETA: 1s - loss: 0.0400 - accuracy: 0.9873
1114/1688 [==================>...........] - ETA: 1s - loss: 0.0400 - accuracy: 0.9872
1135/1688 [===================>..........] - ETA: 1s - loss: 0.0401 - accuracy: 0.9872
1156/1688 [===================>..........] - ETA: 1s - loss: 0.0402 - accuracy: 0.9872
1178/1688 [===================>..........] - ETA: 1s - loss: 0.0400 - accuracy: 0.9873
1199/1688 [====================>.........] - ETA: 1s - loss: 0.0400 - accuracy: 0.9872
1220/1688 [====================>.........] - ETA: 1s - loss: 0.0399 - accuracy: 0.9873
1241/1688 [=====================>........] - ETA: 1s - loss: 0.0397 - accuracy: 0.9874
1262/1688 [=====================>........] - ETA: 1s - loss: 0.0398 - accuracy: 0.9873
1283/1688 [=====================>........] - ETA: 0s - loss: 0.0400 - accuracy: 0.9872
1304/1688 [======================>.......] - ETA: 0s - loss: 0.0399 - accuracy: 0.9873
1323/1688 [======================>.......] - ETA: 0s - loss: 0.0401 - accuracy: 0.9873
1341/1688 [======================>.......] - ETA: 0s - loss: 0.0404 - accuracy: 0.9873
1360/1688 [=======================>......] - ETA: 0s - loss: 0.0402 - accuracy: 0.9873
1381/1688 [=======================>......] - ETA: 0s - loss: 0.0401 - accuracy: 0.9874
1403/1688 [=======================>......] - ETA: 0s - loss: 0.0402 - accuracy: 0.9872
1424/1688 [========================>.....] - ETA: 0s - loss: 0.0401 - accuracy: 0.9872
1446/1688 [========================>.....] - ETA: 0s - loss: 0.0401 - accuracy: 0.9872
1466/1688 [=========================>....] - ETA: 0s - loss: 0.0405 - accuracy: 0.9871
1487/1688 [=========================>....] - ETA: 0s - loss: 0.0406 - accuracy: 0.9871
1508/1688 [=========================>....] - ETA: 0s - loss: 0.0409 - accuracy: 0.9870
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0411 - accuracy: 0.9869
1550/1688 [==========================>...] - ETA: 0s - loss: 0.0411 - accuracy: 0.9868
1571/1688 [==========================>...] - ETA: 0s - loss: 0.0409 - accuracy: 0.9868
1593/1688 [===========================>..] - ETA: 0s - loss: 0.0409 - accuracy: 0.9868
1615/1688 [===========================>..] - ETA: 0s - loss: 0.0407 - accuracy: 0.9868
1636/1688 [============================>.] - ETA: 0s - loss: 0.0405 - accuracy: 0.9869
1657/1688 [============================>.] - ETA: 0s - loss: 0.0407 - accuracy: 0.9868
1678/1688 [============================>.] - ETA: 0s - loss: 0.0406 - accuracy: 0.9868
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0407 - accuracy: 0.9868 - val_loss: 0.0612 - val_accuracy: 0.9850
Epoch 5/10
1/1688 [..............................] - ETA: 4s - loss: 0.0132 - accuracy: 1.0000
22/1688 [..............................] - ETA: 4s - loss: 0.0253 - accuracy: 0.9943
42/1688 [..............................] - ETA: 4s - loss: 0.0274 - accuracy: 0.9926
62/1688 [>.............................] - ETA: 4s - loss: 0.0266 - accuracy: 0.9914
83/1688 [>.............................] - ETA: 3s - loss: 0.0279 - accuracy: 0.9913
104/1688 [>.............................] - ETA: 3s - loss: 0.0264 - accuracy: 0.9910
125/1688 [=>............................] - ETA: 3s - loss: 0.0245 - accuracy: 0.9918
146/1688 [=>............................] - ETA: 3s - loss: 0.0247 - accuracy: 0.9919
168/1688 [=>............................] - ETA: 3s - loss: 0.0252 - accuracy: 0.9916
189/1688 [==>...........................] - ETA: 3s - loss: 0.0252 - accuracy: 0.9914
210/1688 [==>...........................] - ETA: 3s - loss: 0.0251 - accuracy: 0.9911
230/1688 [===>..........................] - ETA: 3s - loss: 0.0269 - accuracy: 0.9906
251/1688 [===>..........................] - ETA: 3s - loss: 0.0264 - accuracy: 0.9907
273/1688 [===>..........................] - ETA: 3s - loss: 0.0261 - accuracy: 0.9907
294/1688 [====>.........................] - ETA: 3s - loss: 0.0259 - accuracy: 0.9908
315/1688 [====>.........................] - ETA: 3s - loss: 0.0266 - accuracy: 0.9904
336/1688 [====>.........................] - ETA: 3s - loss: 0.0261 - accuracy: 0.9904
356/1688 [=====>........................] - ETA: 3s - loss: 0.0270 - accuracy: 0.9903
377/1688 [=====>........................] - ETA: 3s - loss: 0.0276 - accuracy: 0.9901
397/1688 [======>.......................] - ETA: 3s - loss: 0.0271 - accuracy: 0.9904
418/1688 [======>.......................] - ETA: 3s - loss: 0.0264 - accuracy: 0.9907
439/1688 [======>.......................] - ETA: 3s - loss: 0.0263 - accuracy: 0.9909
460/1688 [=======>......................] - ETA: 3s - loss: 0.0262 - accuracy: 0.9909
481/1688 [=======>......................] - ETA: 2s - loss: 0.0257 - accuracy: 0.9911
501/1688 [=======>......................] - ETA: 2s - loss: 0.0269 - accuracy: 0.9909
522/1688 [========>.....................] - ETA: 2s - loss: 0.0271 - accuracy: 0.9908
543/1688 [========>.....................] - ETA: 2s - loss: 0.0274 - accuracy: 0.9907
564/1688 [=========>....................] - ETA: 2s - loss: 0.0273 - accuracy: 0.9906
585/1688 [=========>....................] - ETA: 2s - loss: 0.0270 - accuracy: 0.9908
606/1688 [=========>....................] - ETA: 2s - loss: 0.0268 - accuracy: 0.9909
627/1688 [==========>...................] - ETA: 2s - loss: 0.0266 - accuracy: 0.9909
648/1688 [==========>...................] - ETA: 2s - loss: 0.0268 - accuracy: 0.9908
668/1688 [==========>...................] - ETA: 2s - loss: 0.0267 - accuracy: 0.9909
688/1688 [===========>..................] - ETA: 2s - loss: 0.0266 - accuracy: 0.9909
708/1688 [===========>..................] - ETA: 2s - loss: 0.0268 - accuracy: 0.9907
729/1688 [===========>..................] - ETA: 2s - loss: 0.0268 - accuracy: 0.9907
750/1688 [============>.................] - ETA: 2s - loss: 0.0276 - accuracy: 0.9908
771/1688 [============>.................] - ETA: 2s - loss: 0.0279 - accuracy: 0.9907
791/1688 [=============>................] - ETA: 2s - loss: 0.0278 - accuracy: 0.9908
812/1688 [=============>................] - ETA: 2s - loss: 0.0279 - accuracy: 0.9907
833/1688 [=============>................] - ETA: 2s - loss: 0.0283 - accuracy: 0.9907
854/1688 [==============>...............] - ETA: 2s - loss: 0.0292 - accuracy: 0.9905
875/1688 [==============>...............] - ETA: 1s - loss: 0.0297 - accuracy: 0.9902
896/1688 [==============>...............] - ETA: 1s - loss: 0.0298 - accuracy: 0.9903
917/1688 [===============>..............] - ETA: 1s - loss: 0.0301 - accuracy: 0.9903
938/1688 [===============>..............] - ETA: 1s - loss: 0.0303 - accuracy: 0.9902
960/1688 [================>.............] - ETA: 1s - loss: 0.0303 - accuracy: 0.9902
981/1688 [================>.............] - ETA: 1s - loss: 0.0305 - accuracy: 0.9901
1001/1688 [================>.............] - ETA: 1s - loss: 0.0306 - accuracy: 0.9902
1022/1688 [=================>............] - ETA: 1s - loss: 0.0315 - accuracy: 0.9899
1043/1688 [=================>............] - ETA: 1s - loss: 0.0316 - accuracy: 0.9899
1063/1688 [=================>............] - ETA: 1s - loss: 0.0316 - accuracy: 0.9899
1084/1688 [==================>...........] - ETA: 1s - loss: 0.0318 - accuracy: 0.9899
1106/1688 [==================>...........] - ETA: 1s - loss: 0.0320 - accuracy: 0.9898
1128/1688 [===================>..........] - ETA: 1s - loss: 0.0319 - accuracy: 0.9899
1148/1688 [===================>..........] - ETA: 1s - loss: 0.0317 - accuracy: 0.9900
1168/1688 [===================>..........] - ETA: 1s - loss: 0.0318 - accuracy: 0.9900
1188/1688 [====================>.........] - ETA: 1s - loss: 0.0317 - accuracy: 0.9900
1209/1688 [====================>.........] - ETA: 1s - loss: 0.0316 - accuracy: 0.9900
1230/1688 [====================>.........] - ETA: 1s - loss: 0.0316 - accuracy: 0.9900
1251/1688 [=====================>........] - ETA: 1s - loss: 0.0314 - accuracy: 0.9901
1272/1688 [=====================>........] - ETA: 1s - loss: 0.0311 - accuracy: 0.9902
1293/1688 [=====================>........] - ETA: 0s - loss: 0.0312 - accuracy: 0.9900
1314/1688 [======================>.......] - ETA: 0s - loss: 0.0319 - accuracy: 0.9898
1335/1688 [======================>.......] - ETA: 0s - loss: 0.0319 - accuracy: 0.9898
1356/1688 [=======================>......] - ETA: 0s - loss: 0.0321 - accuracy: 0.9897
1376/1688 [=======================>......] - ETA: 0s - loss: 0.0321 - accuracy: 0.9897
1397/1688 [=======================>......] - ETA: 0s - loss: 0.0325 - accuracy: 0.9896
1417/1688 [========================>.....] - ETA: 0s - loss: 0.0325 - accuracy: 0.9896
1438/1688 [========================>.....] - ETA: 0s - loss: 0.0326 - accuracy: 0.9896
1459/1688 [========================>.....] - ETA: 0s - loss: 0.0327 - accuracy: 0.9895
1480/1688 [=========================>....] - ETA: 0s - loss: 0.0328 - accuracy: 0.9895
1501/1688 [=========================>....] - ETA: 0s - loss: 0.0329 - accuracy: 0.9894
1521/1688 [==========================>...] - ETA: 0s - loss: 0.0330 - accuracy: 0.9894
1542/1688 [==========================>...] - ETA: 0s - loss: 0.0331 - accuracy: 0.9893
1563/1688 [==========================>...] - ETA: 0s - loss: 0.0333 - accuracy: 0.9892
1585/1688 [===========================>..] - ETA: 0s - loss: 0.0334 - accuracy: 0.9892
1606/1688 [===========================>..] - ETA: 0s - loss: 0.0335 - accuracy: 0.9892
1627/1688 [===========================>..] - ETA: 0s - loss: 0.0334 - accuracy: 0.9891
1646/1688 [============================>.] - ETA: 0s - loss: 0.0334 - accuracy: 0.9892
1665/1688 [============================>.] - ETA: 0s - loss: 0.0335 - accuracy: 0.9892
1684/1688 [============================>.] - ETA: 0s - loss: 0.0333 - accuracy: 0.9892
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0333 - accuracy: 0.9892 - val_loss: 0.0619 - val_accuracy: 0.9850
Epoch 6/10
1/1688 [..............................] - ETA: 4s - loss: 0.0263 - accuracy: 1.0000
21/1688 [..............................] - ETA: 4s - loss: 0.0285 - accuracy: 0.9911
42/1688 [..............................] - ETA: 4s - loss: 0.0289 - accuracy: 0.9896
63/1688 [>.............................] - ETA: 4s - loss: 0.0280 - accuracy: 0.9891
84/1688 [>.............................] - ETA: 3s - loss: 0.0295 - accuracy: 0.9896
105/1688 [>.............................] - ETA: 3s - loss: 0.0265 - accuracy: 0.9911
126/1688 [=>............................] - ETA: 3s - loss: 0.0258 - accuracy: 0.9916
148/1688 [=>............................] - ETA: 3s - loss: 0.0258 - accuracy: 0.9913
169/1688 [==>...........................] - ETA: 3s - loss: 0.0259 - accuracy: 0.9906
190/1688 [==>...........................] - ETA: 3s - loss: 0.0270 - accuracy: 0.9905
211/1688 [==>...........................] - ETA: 3s - loss: 0.0253 - accuracy: 0.9911
233/1688 [===>..........................] - ETA: 3s - loss: 0.0253 - accuracy: 0.9910
253/1688 [===>..........................] - ETA: 3s - loss: 0.0252 - accuracy: 0.9910
274/1688 [===>..........................] - ETA: 3s - loss: 0.0247 - accuracy: 0.9911
295/1688 [====>.........................] - ETA: 3s - loss: 0.0237 - accuracy: 0.9914
316/1688 [====>.........................] - ETA: 3s - loss: 0.0245 - accuracy: 0.9914
337/1688 [====>.........................] - ETA: 3s - loss: 0.0244 - accuracy: 0.9916
358/1688 [=====>........................] - ETA: 3s - loss: 0.0241 - accuracy: 0.9915
379/1688 [=====>........................] - ETA: 3s - loss: 0.0244 - accuracy: 0.9913
400/1688 [======>.......................] - ETA: 3s - loss: 0.0241 - accuracy: 0.9916
421/1688 [======>.......................] - ETA: 3s - loss: 0.0241 - accuracy: 0.9917
441/1688 [======>.......................] - ETA: 3s - loss: 0.0235 - accuracy: 0.9919
462/1688 [=======>......................] - ETA: 2s - loss: 0.0233 - accuracy: 0.9921
483/1688 [=======>......................] - ETA: 2s - loss: 0.0228 - accuracy: 0.9924
504/1688 [=======>......................] - ETA: 2s - loss: 0.0233 - accuracy: 0.9923
525/1688 [========>.....................] - ETA: 2s - loss: 0.0229 - accuracy: 0.9924
546/1688 [========>.....................] - ETA: 2s - loss: 0.0229 - accuracy: 0.9925
567/1688 [=========>....................] - ETA: 2s - loss: 0.0231 - accuracy: 0.9923
588/1688 [=========>....................] - ETA: 2s - loss: 0.0234 - accuracy: 0.9922
609/1688 [=========>....................] - ETA: 2s - loss: 0.0237 - accuracy: 0.9920
629/1688 [==========>...................] - ETA: 2s - loss: 0.0240 - accuracy: 0.9921
649/1688 [==========>...................] - ETA: 2s - loss: 0.0240 - accuracy: 0.9922
669/1688 [==========>...................] - ETA: 2s - loss: 0.0238 - accuracy: 0.9923
691/1688 [===========>..................] - ETA: 2s - loss: 0.0237 - accuracy: 0.9923
712/1688 [===========>..................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9921
733/1688 [============>.................] - ETA: 2s - loss: 0.0257 - accuracy: 0.9916
754/1688 [============>.................] - ETA: 2s - loss: 0.0257 - accuracy: 0.9917
775/1688 [============>.................] - ETA: 2s - loss: 0.0256 - accuracy: 0.9917
795/1688 [=============>................] - ETA: 2s - loss: 0.0256 - accuracy: 0.9916
817/1688 [=============>................] - ETA: 2s - loss: 0.0257 - accuracy: 0.9916
838/1688 [=============>................] - ETA: 2s - loss: 0.0256 - accuracy: 0.9916
860/1688 [==============>...............] - ETA: 2s - loss: 0.0252 - accuracy: 0.9918
881/1688 [==============>...............] - ETA: 1s - loss: 0.0252 - accuracy: 0.9918
903/1688 [===============>..............] - ETA: 1s - loss: 0.0253 - accuracy: 0.9918
925/1688 [===============>..............] - ETA: 1s - loss: 0.0252 - accuracy: 0.9918
946/1688 [===============>..............] - ETA: 1s - loss: 0.0249 - accuracy: 0.9919
966/1688 [================>.............] - ETA: 1s - loss: 0.0246 - accuracy: 0.9919
987/1688 [================>.............] - ETA: 1s - loss: 0.0245 - accuracy: 0.9920
1008/1688 [================>.............] - ETA: 1s - loss: 0.0246 - accuracy: 0.9919
1029/1688 [=================>............] - ETA: 1s - loss: 0.0245 - accuracy: 0.9920
1050/1688 [=================>............] - ETA: 1s - loss: 0.0247 - accuracy: 0.9920
1070/1688 [==================>...........] - ETA: 1s - loss: 0.0246 - accuracy: 0.9919
1091/1688 [==================>...........] - ETA: 1s - loss: 0.0251 - accuracy: 0.9917
1112/1688 [==================>...........] - ETA: 1s - loss: 0.0251 - accuracy: 0.9918
1131/1688 [===================>..........] - ETA: 1s - loss: 0.0252 - accuracy: 0.9918
1151/1688 [===================>..........] - ETA: 1s - loss: 0.0255 - accuracy: 0.9918
1171/1688 [===================>..........] - ETA: 1s - loss: 0.0253 - accuracy: 0.9919
1191/1688 [====================>.........] - ETA: 1s - loss: 0.0256 - accuracy: 0.9917
1212/1688 [====================>.........] - ETA: 1s - loss: 0.0255 - accuracy: 0.9917
1233/1688 [====================>.........] - ETA: 1s - loss: 0.0259 - accuracy: 0.9916
1254/1688 [=====================>........] - ETA: 1s - loss: 0.0264 - accuracy: 0.9914
1275/1688 [=====================>........] - ETA: 1s - loss: 0.0268 - accuracy: 0.9913
1296/1688 [======================>.......] - ETA: 0s - loss: 0.0269 - accuracy: 0.9913
1318/1688 [======================>.......] - ETA: 0s - loss: 0.0273 - accuracy: 0.9912
1339/1688 [======================>.......] - ETA: 0s - loss: 0.0274 - accuracy: 0.9912
1360/1688 [=======================>......] - ETA: 0s - loss: 0.0273 - accuracy: 0.9912
1381/1688 [=======================>......] - ETA: 0s - loss: 0.0274 - accuracy: 0.9911
1402/1688 [=======================>......] - ETA: 0s - loss: 0.0274 - accuracy: 0.9911
1423/1688 [========================>.....] - ETA: 0s - loss: 0.0272 - accuracy: 0.9911
1444/1688 [========================>.....] - ETA: 0s - loss: 0.0272 - accuracy: 0.9911
1465/1688 [=========================>....] - ETA: 0s - loss: 0.0275 - accuracy: 0.9911
1486/1688 [=========================>....] - ETA: 0s - loss: 0.0277 - accuracy: 0.9911
1507/1688 [=========================>....] - ETA: 0s - loss: 0.0280 - accuracy: 0.9910
1529/1688 [==========================>...] - ETA: 0s - loss: 0.0280 - accuracy: 0.9910
1550/1688 [==========================>...] - ETA: 0s - loss: 0.0282 - accuracy: 0.9909
1571/1688 [==========================>...] - ETA: 0s - loss: 0.0281 - accuracy: 0.9908
1592/1688 [===========================>..] - ETA: 0s - loss: 0.0284 - accuracy: 0.9908
1613/1688 [===========================>..] - ETA: 0s - loss: 0.0286 - accuracy: 0.9907
1634/1688 [============================>.] - ETA: 0s - loss: 0.0288 - accuracy: 0.9906
1655/1688 [============================>.] - ETA: 0s - loss: 0.0288 - accuracy: 0.9906
1676/1688 [============================>.] - ETA: 0s - loss: 0.0287 - accuracy: 0.9906
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0286 - accuracy: 0.9906 - val_loss: 0.0504 - val_accuracy: 0.9873
Epoch 7/10
1/1688 [..............................] - ETA: 4s - loss: 0.0042 - accuracy: 1.0000
22/1688 [..............................] - ETA: 3s - loss: 0.0355 - accuracy: 0.9915
43/1688 [..............................] - ETA: 3s - loss: 0.0296 - accuracy: 0.9913
63/1688 [>.............................] - ETA: 3s - loss: 0.0238 - accuracy: 0.9921
84/1688 [>.............................] - ETA: 3s - loss: 0.0254 - accuracy: 0.9918
105/1688 [>.............................] - ETA: 3s - loss: 0.0226 - accuracy: 0.9920
127/1688 [=>............................] - ETA: 3s - loss: 0.0206 - accuracy: 0.9929
148/1688 [=>............................] - ETA: 3s - loss: 0.0192 - accuracy: 0.9935
169/1688 [==>...........................] - ETA: 3s - loss: 0.0182 - accuracy: 0.9941
190/1688 [==>...........................] - ETA: 3s - loss: 0.0184 - accuracy: 0.9941
211/1688 [==>...........................] - ETA: 3s - loss: 0.0191 - accuracy: 0.9936
232/1688 [===>..........................] - ETA: 3s - loss: 0.0184 - accuracy: 0.9939
253/1688 [===>..........................] - ETA: 3s - loss: 0.0182 - accuracy: 0.9941
274/1688 [===>..........................] - ETA: 3s - loss: 0.0185 - accuracy: 0.9940
295/1688 [====>.........................] - ETA: 3s - loss: 0.0183 - accuracy: 0.9941
315/1688 [====>.........................] - ETA: 3s - loss: 0.0180 - accuracy: 0.9942
336/1688 [====>.........................] - ETA: 3s - loss: 0.0179 - accuracy: 0.9940
357/1688 [=====>........................] - ETA: 3s - loss: 0.0182 - accuracy: 0.9938
377/1688 [=====>........................] - ETA: 3s - loss: 0.0178 - accuracy: 0.9939
398/1688 [======>.......................] - ETA: 3s - loss: 0.0183 - accuracy: 0.9938
420/1688 [======>.......................] - ETA: 3s - loss: 0.0188 - accuracy: 0.9935
441/1688 [======>.......................] - ETA: 3s - loss: 0.0190 - accuracy: 0.9934
462/1688 [=======>......................] - ETA: 2s - loss: 0.0189 - accuracy: 0.9934
484/1688 [=======>......................] - ETA: 2s - loss: 0.0188 - accuracy: 0.9933
505/1688 [=======>......................] - ETA: 2s - loss: 0.0185 - accuracy: 0.9935
526/1688 [========>.....................] - ETA: 2s - loss: 0.0183 - accuracy: 0.9936
547/1688 [========>.....................] - ETA: 2s - loss: 0.0185 - accuracy: 0.9935
568/1688 [=========>....................] - ETA: 2s - loss: 0.0187 - accuracy: 0.9935
588/1688 [=========>....................] - ETA: 2s - loss: 0.0186 - accuracy: 0.9936
608/1688 [=========>....................] - ETA: 2s - loss: 0.0184 - accuracy: 0.9937
629/1688 [==========>...................] - ETA: 2s - loss: 0.0184 - accuracy: 0.9936
650/1688 [==========>...................] - ETA: 2s - loss: 0.0184 - accuracy: 0.9936
671/1688 [==========>...................] - ETA: 2s - loss: 0.0182 - accuracy: 0.9938
692/1688 [===========>..................] - ETA: 2s - loss: 0.0180 - accuracy: 0.9938
712/1688 [===========>..................] - ETA: 2s - loss: 0.0181 - accuracy: 0.9938
733/1688 [============>.................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9938
754/1688 [============>.................] - ETA: 2s - loss: 0.0180 - accuracy: 0.9938
775/1688 [============>.................] - ETA: 2s - loss: 0.0184 - accuracy: 0.9937
795/1688 [=============>................] - ETA: 2s - loss: 0.0187 - accuracy: 0.9936
816/1688 [=============>................] - ETA: 2s - loss: 0.0187 - accuracy: 0.9936
837/1688 [=============>................] - ETA: 2s - loss: 0.0187 - accuracy: 0.9936
858/1688 [==============>...............] - ETA: 2s - loss: 0.0189 - accuracy: 0.9935
878/1688 [==============>...............] - ETA: 1s - loss: 0.0188 - accuracy: 0.9935
900/1688 [==============>...............] - ETA: 1s - loss: 0.0191 - accuracy: 0.9934
921/1688 [===============>..............] - ETA: 1s - loss: 0.0191 - accuracy: 0.9935
942/1688 [===============>..............] - ETA: 1s - loss: 0.0193 - accuracy: 0.9933
963/1688 [================>.............] - ETA: 1s - loss: 0.0198 - accuracy: 0.9932
984/1688 [================>.............] - ETA: 1s - loss: 0.0199 - accuracy: 0.9930
1005/1688 [================>.............] - ETA: 1s - loss: 0.0201 - accuracy: 0.9930
1026/1688 [=================>............] - ETA: 1s - loss: 0.0200 - accuracy: 0.9930
1046/1688 [=================>............] - ETA: 1s - loss: 0.0200 - accuracy: 0.9930
1067/1688 [=================>............] - ETA: 1s - loss: 0.0200 - accuracy: 0.9931
1087/1688 [==================>...........] - ETA: 1s - loss: 0.0202 - accuracy: 0.9930
1108/1688 [==================>...........] - ETA: 1s - loss: 0.0202 - accuracy: 0.9930
1129/1688 [===================>..........] - ETA: 1s - loss: 0.0201 - accuracy: 0.9930
1150/1688 [===================>..........] - ETA: 1s - loss: 0.0204 - accuracy: 0.9929
1171/1688 [===================>..........] - ETA: 1s - loss: 0.0211 - accuracy: 0.9927
1192/1688 [====================>.........] - ETA: 1s - loss: 0.0213 - accuracy: 0.9927
1213/1688 [====================>.........] - ETA: 1s - loss: 0.0212 - accuracy: 0.9927
1234/1688 [====================>.........] - ETA: 1s - loss: 0.0213 - accuracy: 0.9927
1255/1688 [=====================>........] - ETA: 1s - loss: 0.0212 - accuracy: 0.9927
1276/1688 [=====================>........] - ETA: 1s - loss: 0.0212 - accuracy: 0.9928
1297/1688 [======================>.......] - ETA: 0s - loss: 0.0213 - accuracy: 0.9927
1317/1688 [======================>.......] - ETA: 0s - loss: 0.0214 - accuracy: 0.9927
1337/1688 [======================>.......] - ETA: 0s - loss: 0.0214 - accuracy: 0.9927
1358/1688 [=======================>......] - ETA: 0s - loss: 0.0217 - accuracy: 0.9926
1379/1688 [=======================>......] - ETA: 0s - loss: 0.0217 - accuracy: 0.9927
1399/1688 [=======================>......] - ETA: 0s - loss: 0.0218 - accuracy: 0.9926
1418/1688 [========================>.....] - ETA: 0s - loss: 0.0220 - accuracy: 0.9926
1439/1688 [========================>.....] - ETA: 0s - loss: 0.0223 - accuracy: 0.9925
1460/1688 [========================>.....] - ETA: 0s - loss: 0.0226 - accuracy: 0.9924
1481/1688 [=========================>....] - ETA: 0s - loss: 0.0227 - accuracy: 0.9924
1501/1688 [=========================>....] - ETA: 0s - loss: 0.0230 - accuracy: 0.9924
1522/1688 [==========================>...] - ETA: 0s - loss: 0.0230 - accuracy: 0.9924
1542/1688 [==========================>...] - ETA: 0s - loss: 0.0231 - accuracy: 0.9923
1563/1688 [==========================>...] - ETA: 0s - loss: 0.0230 - accuracy: 0.9923
1584/1688 [===========================>..] - ETA: 0s - loss: 0.0230 - accuracy: 0.9923
1605/1688 [===========================>..] - ETA: 0s - loss: 0.0232 - accuracy: 0.9923
1626/1688 [===========================>..] - ETA: 0s - loss: 0.0234 - accuracy: 0.9922
1648/1688 [============================>.] - ETA: 0s - loss: 0.0233 - accuracy: 0.9922
1669/1688 [============================>.] - ETA: 0s - loss: 0.0233 - accuracy: 0.9922
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0234 - accuracy: 0.9922 - val_loss: 0.0658 - val_accuracy: 0.9857
Epoch 8/10
1/1688 [..............................] - ETA: 4s - loss: 0.0029 - accuracy: 1.0000
22/1688 [..............................] - ETA: 4s - loss: 0.0060 - accuracy: 0.9986
43/1688 [..............................] - ETA: 4s - loss: 0.0114 - accuracy: 0.9964
64/1688 [>.............................] - ETA: 3s - loss: 0.0150 - accuracy: 0.9951
85/1688 [>.............................] - ETA: 3s - loss: 0.0134 - accuracy: 0.9956
107/1688 [>.............................] - ETA: 3s - loss: 0.0126 - accuracy: 0.9959
128/1688 [=>............................] - ETA: 3s - loss: 0.0123 - accuracy: 0.9958
149/1688 [=>............................] - ETA: 3s - loss: 0.0132 - accuracy: 0.9958
170/1688 [==>...........................] - ETA: 3s - loss: 0.0126 - accuracy: 0.9960
191/1688 [==>...........................] - ETA: 3s - loss: 0.0128 - accuracy: 0.9961
212/1688 [==>...........................] - ETA: 3s - loss: 0.0131 - accuracy: 0.9959
233/1688 [===>..........................] - ETA: 3s - loss: 0.0133 - accuracy: 0.9958
254/1688 [===>..........................] - ETA: 3s - loss: 0.0132 - accuracy: 0.9957
275/1688 [===>..........................] - ETA: 3s - loss: 0.0134 - accuracy: 0.9958
295/1688 [====>.........................] - ETA: 3s - loss: 0.0134 - accuracy: 0.9958
315/1688 [====>.........................] - ETA: 3s - loss: 0.0135 - accuracy: 0.9957
336/1688 [====>.........................] - ETA: 3s - loss: 0.0140 - accuracy: 0.9956
356/1688 [=====>........................] - ETA: 3s - loss: 0.0137 - accuracy: 0.9957
377/1688 [=====>........................] - ETA: 3s - loss: 0.0141 - accuracy: 0.9952
398/1688 [======>.......................] - ETA: 3s - loss: 0.0141 - accuracy: 0.9954
419/1688 [======>.......................] - ETA: 3s - loss: 0.0141 - accuracy: 0.9953
440/1688 [======>.......................] - ETA: 3s - loss: 0.0140 - accuracy: 0.9952
461/1688 [=======>......................] - ETA: 3s - loss: 0.0140 - accuracy: 0.9952
482/1688 [=======>......................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9950
503/1688 [=======>......................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9948
524/1688 [========>.....................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9948
546/1688 [========>.....................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9950
567/1688 [=========>....................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9949
588/1688 [=========>....................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9949
609/1688 [=========>....................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9949
631/1688 [==========>...................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9947
652/1688 [==========>...................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9947
673/1688 [==========>...................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9948
693/1688 [===========>..................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9948
714/1688 [===========>..................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9947
735/1688 [============>.................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9947
756/1688 [============>.................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9948
777/1688 [============>.................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9948
798/1688 [=============>................] - ETA: 2s - loss: 0.0158 - accuracy: 0.9946
819/1688 [=============>................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9947
840/1688 [=============>................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9946
861/1688 [==============>...............] - ETA: 2s - loss: 0.0161 - accuracy: 0.9945
882/1688 [==============>...............] - ETA: 1s - loss: 0.0163 - accuracy: 0.9945
903/1688 [===============>..............] - ETA: 1s - loss: 0.0162 - accuracy: 0.9945
924/1688 [===============>..............] - ETA: 1s - loss: 0.0160 - accuracy: 0.9946
945/1688 [===============>..............] - ETA: 1s - loss: 0.0160 - accuracy: 0.9946
966/1688 [================>.............] - ETA: 1s - loss: 0.0160 - accuracy: 0.9946
987/1688 [================>.............] - ETA: 1s - loss: 0.0160 - accuracy: 0.9946
1008/1688 [================>.............] - ETA: 1s - loss: 0.0160 - accuracy: 0.9946
1029/1688 [=================>............] - ETA: 1s - loss: 0.0158 - accuracy: 0.9947
1050/1688 [=================>............] - ETA: 1s - loss: 0.0159 - accuracy: 0.9947
1071/1688 [==================>...........] - ETA: 1s - loss: 0.0160 - accuracy: 0.9946
1092/1688 [==================>...........] - ETA: 1s - loss: 0.0160 - accuracy: 0.9946
1114/1688 [==================>...........] - ETA: 1s - loss: 0.0163 - accuracy: 0.9945
1135/1688 [===================>..........] - ETA: 1s - loss: 0.0162 - accuracy: 0.9945
1156/1688 [===================>..........] - ETA: 1s - loss: 0.0168 - accuracy: 0.9944
1177/1688 [===================>..........] - ETA: 1s - loss: 0.0173 - accuracy: 0.9943
1198/1688 [====================>.........] - ETA: 1s - loss: 0.0173 - accuracy: 0.9943
1219/1688 [====================>.........] - ETA: 1s - loss: 0.0173 - accuracy: 0.9943
1240/1688 [=====================>........] - ETA: 1s - loss: 0.0172 - accuracy: 0.9943
1262/1688 [=====================>........] - ETA: 1s - loss: 0.0170 - accuracy: 0.9944
1283/1688 [=====================>........] - ETA: 0s - loss: 0.0171 - accuracy: 0.9943
1304/1688 [======================>.......] - ETA: 0s - loss: 0.0172 - accuracy: 0.9943
1325/1688 [======================>.......] - ETA: 0s - loss: 0.0172 - accuracy: 0.9943
1346/1688 [======================>.......] - ETA: 0s - loss: 0.0171 - accuracy: 0.9943
1367/1688 [=======================>......] - ETA: 0s - loss: 0.0173 - accuracy: 0.9943
1388/1688 [=======================>......] - ETA: 0s - loss: 0.0174 - accuracy: 0.9942
1409/1688 [========================>.....] - ETA: 0s - loss: 0.0176 - accuracy: 0.9941
1430/1688 [========================>.....] - ETA: 0s - loss: 0.0176 - accuracy: 0.9941
1451/1688 [========================>.....] - ETA: 0s - loss: 0.0179 - accuracy: 0.9940
1472/1688 [=========================>....] - ETA: 0s - loss: 0.0180 - accuracy: 0.9939
1492/1688 [=========================>....] - ETA: 0s - loss: 0.0179 - accuracy: 0.9939
1513/1688 [=========================>....] - ETA: 0s - loss: 0.0179 - accuracy: 0.9939
1534/1688 [==========================>...] - ETA: 0s - loss: 0.0180 - accuracy: 0.9939
1555/1688 [==========================>...] - ETA: 0s - loss: 0.0183 - accuracy: 0.9938
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0185 - accuracy: 0.9937
1596/1688 [===========================>..] - ETA: 0s - loss: 0.0186 - accuracy: 0.9937
1616/1688 [===========================>..] - ETA: 0s - loss: 0.0186 - accuracy: 0.9936
1638/1688 [============================>.] - ETA: 0s - loss: 0.0189 - accuracy: 0.9936
1659/1688 [============================>.] - ETA: 0s - loss: 0.0193 - accuracy: 0.9935
1680/1688 [============================>.] - ETA: 0s - loss: 0.0194 - accuracy: 0.9934
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0196 - accuracy: 0.9934 - val_loss: 0.0692 - val_accuracy: 0.9847
Epoch 9/10
1/1688 [..............................] - ETA: 4s - loss: 0.0914 - accuracy: 0.9688
21/1688 [..............................] - ETA: 4s - loss: 0.0230 - accuracy: 0.9926
41/1688 [..............................] - ETA: 4s - loss: 0.0151 - accuracy: 0.9954
62/1688 [>.............................] - ETA: 4s - loss: 0.0151 - accuracy: 0.9940
83/1688 [>.............................] - ETA: 4s - loss: 0.0154 - accuracy: 0.9944
103/1688 [>.............................] - ETA: 3s - loss: 0.0141 - accuracy: 0.9948
123/1688 [=>............................] - ETA: 3s - loss: 0.0134 - accuracy: 0.9949
144/1688 [=>............................] - ETA: 3s - loss: 0.0129 - accuracy: 0.9950
165/1688 [=>............................] - ETA: 3s - loss: 0.0125 - accuracy: 0.9955
186/1688 [==>...........................] - ETA: 3s - loss: 0.0124 - accuracy: 0.9956
208/1688 [==>...........................] - ETA: 3s - loss: 0.0130 - accuracy: 0.9952
230/1688 [===>..........................] - ETA: 3s - loss: 0.0133 - accuracy: 0.9950
250/1688 [===>..........................] - ETA: 3s - loss: 0.0144 - accuracy: 0.9942
271/1688 [===>..........................] - ETA: 3s - loss: 0.0141 - accuracy: 0.9943
293/1688 [====>.........................] - ETA: 3s - loss: 0.0140 - accuracy: 0.9943
314/1688 [====>.........................] - ETA: 3s - loss: 0.0143 - accuracy: 0.9942
335/1688 [====>.........................] - ETA: 3s - loss: 0.0147 - accuracy: 0.9941
355/1688 [=====>........................] - ETA: 3s - loss: 0.0161 - accuracy: 0.9937
376/1688 [=====>........................] - ETA: 3s - loss: 0.0157 - accuracy: 0.9939
397/1688 [======>.......................] - ETA: 3s - loss: 0.0158 - accuracy: 0.9940
418/1688 [======>.......................] - ETA: 3s - loss: 0.0158 - accuracy: 0.9940
439/1688 [======>.......................] - ETA: 3s - loss: 0.0156 - accuracy: 0.9941
460/1688 [=======>......................] - ETA: 3s - loss: 0.0157 - accuracy: 0.9940
481/1688 [=======>......................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9941
502/1688 [=======>......................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9941
523/1688 [========>.....................] - ETA: 2s - loss: 0.0161 - accuracy: 0.9940
543/1688 [========>.....................] - ETA: 2s - loss: 0.0162 - accuracy: 0.9940
564/1688 [=========>....................] - ETA: 2s - loss: 0.0166 - accuracy: 0.9941
585/1688 [=========>....................] - ETA: 2s - loss: 0.0168 - accuracy: 0.9939
606/1688 [=========>....................] - ETA: 2s - loss: 0.0167 - accuracy: 0.9939
627/1688 [==========>...................] - ETA: 2s - loss: 0.0169 - accuracy: 0.9939
648/1688 [==========>...................] - ETA: 2s - loss: 0.0168 - accuracy: 0.9939
669/1688 [==========>...................] - ETA: 2s - loss: 0.0166 - accuracy: 0.9940
690/1688 [===========>..................] - ETA: 2s - loss: 0.0165 - accuracy: 0.9941
711/1688 [===========>..................] - ETA: 2s - loss: 0.0164 - accuracy: 0.9942
732/1688 [============>.................] - ETA: 2s - loss: 0.0163 - accuracy: 0.9942
753/1688 [============>.................] - ETA: 2s - loss: 0.0162 - accuracy: 0.9942
773/1688 [============>.................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9943
794/1688 [=============>................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9943
814/1688 [=============>................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9944
835/1688 [=============>................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9944
856/1688 [==============>...............] - ETA: 2s - loss: 0.0160 - accuracy: 0.9943
877/1688 [==============>...............] - ETA: 1s - loss: 0.0161 - accuracy: 0.9943
898/1688 [==============>...............] - ETA: 1s - loss: 0.0161 - accuracy: 0.9943
919/1688 [===============>..............] - ETA: 1s - loss: 0.0159 - accuracy: 0.9944
939/1688 [===============>..............] - ETA: 1s - loss: 0.0157 - accuracy: 0.9945
960/1688 [================>.............] - ETA: 1s - loss: 0.0156 - accuracy: 0.9945
981/1688 [================>.............] - ETA: 1s - loss: 0.0157 - accuracy: 0.9945
1002/1688 [================>.............] - ETA: 1s - loss: 0.0157 - accuracy: 0.9945
1023/1688 [=================>............] - ETA: 1s - loss: 0.0156 - accuracy: 0.9945
1044/1688 [=================>............] - ETA: 1s - loss: 0.0155 - accuracy: 0.9946
1066/1688 [=================>............] - ETA: 1s - loss: 0.0154 - accuracy: 0.9946
1088/1688 [==================>...........] - ETA: 1s - loss: 0.0153 - accuracy: 0.9946
1110/1688 [==================>...........] - ETA: 1s - loss: 0.0153 - accuracy: 0.9947
1131/1688 [===================>..........] - ETA: 1s - loss: 0.0156 - accuracy: 0.9947
1152/1688 [===================>..........] - ETA: 1s - loss: 0.0156 - accuracy: 0.9947
1173/1688 [===================>..........] - ETA: 1s - loss: 0.0154 - accuracy: 0.9948
1194/1688 [====================>.........] - ETA: 1s - loss: 0.0157 - accuracy: 0.9948
1214/1688 [====================>.........] - ETA: 1s - loss: 0.0156 - accuracy: 0.9949
1235/1688 [====================>.........] - ETA: 1s - loss: 0.0158 - accuracy: 0.9948
1257/1688 [=====================>........] - ETA: 1s - loss: 0.0157 - accuracy: 0.9948
1277/1688 [=====================>........] - ETA: 1s - loss: 0.0159 - accuracy: 0.9947
1297/1688 [======================>.......] - ETA: 0s - loss: 0.0161 - accuracy: 0.9946
1317/1688 [======================>.......] - ETA: 0s - loss: 0.0161 - accuracy: 0.9946
1338/1688 [======================>.......] - ETA: 0s - loss: 0.0162 - accuracy: 0.9945
1359/1688 [=======================>......] - ETA: 0s - loss: 0.0161 - accuracy: 0.9946
1379/1688 [=======================>......] - ETA: 0s - loss: 0.0162 - accuracy: 0.9945
1400/1688 [=======================>......] - ETA: 0s - loss: 0.0167 - accuracy: 0.9945
1421/1688 [========================>.....] - ETA: 0s - loss: 0.0169 - accuracy: 0.9945
1442/1688 [========================>.....] - ETA: 0s - loss: 0.0171 - accuracy: 0.9944
1462/1688 [========================>.....] - ETA: 0s - loss: 0.0171 - accuracy: 0.9945
1483/1688 [=========================>....] - ETA: 0s - loss: 0.0171 - accuracy: 0.9945
1503/1688 [=========================>....] - ETA: 0s - loss: 0.0170 - accuracy: 0.9946
1523/1688 [==========================>...] - ETA: 0s - loss: 0.0173 - accuracy: 0.9944
1544/1688 [==========================>...] - ETA: 0s - loss: 0.0173 - accuracy: 0.9944
1565/1688 [==========================>...] - ETA: 0s - loss: 0.0173 - accuracy: 0.9943
1585/1688 [===========================>..] - ETA: 0s - loss: 0.0172 - accuracy: 0.9944
1605/1688 [===========================>..] - ETA: 0s - loss: 0.0171 - accuracy: 0.9944
1626/1688 [===========================>..] - ETA: 0s - loss: 0.0171 - accuracy: 0.9944
1647/1688 [============================>.] - ETA: 0s - loss: 0.0170 - accuracy: 0.9944
1668/1688 [============================>.] - ETA: 0s - loss: 0.0172 - accuracy: 0.9944
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0172 - accuracy: 0.9944 - val_loss: 0.0540 - val_accuracy: 0.9872
Epoch 10/10
1/1688 [..............................] - ETA: 4s - loss: 0.0048 - accuracy: 1.0000
21/1688 [..............................] - ETA: 4s - loss: 0.0051 - accuracy: 1.0000
42/1688 [..............................] - ETA: 4s - loss: 0.0075 - accuracy: 0.9985
63/1688 [>.............................] - ETA: 3s - loss: 0.0099 - accuracy: 0.9980
84/1688 [>.............................] - ETA: 3s - loss: 0.0130 - accuracy: 0.9963
105/1688 [>.............................] - ETA: 3s - loss: 0.0114 - accuracy: 0.9970
127/1688 [=>............................] - ETA: 3s - loss: 0.0110 - accuracy: 0.9973
147/1688 [=>............................] - ETA: 3s - loss: 0.0109 - accuracy: 0.9974
167/1688 [=>............................] - ETA: 3s - loss: 0.0104 - accuracy: 0.9974
188/1688 [==>...........................] - ETA: 3s - loss: 0.0101 - accuracy: 0.9975
209/1688 [==>...........................] - ETA: 3s - loss: 0.0104 - accuracy: 0.9973
231/1688 [===>..........................] - ETA: 3s - loss: 0.0121 - accuracy: 0.9968
252/1688 [===>..........................] - ETA: 3s - loss: 0.0120 - accuracy: 0.9967
273/1688 [===>..........................] - ETA: 3s - loss: 0.0121 - accuracy: 0.9966
294/1688 [====>.........................] - ETA: 3s - loss: 0.0129 - accuracy: 0.9964
315/1688 [====>.........................] - ETA: 3s - loss: 0.0131 - accuracy: 0.9961
336/1688 [====>.........................] - ETA: 3s - loss: 0.0128 - accuracy: 0.9963
357/1688 [=====>........................] - ETA: 3s - loss: 0.0124 - accuracy: 0.9963
378/1688 [=====>........................] - ETA: 3s - loss: 0.0125 - accuracy: 0.9963
399/1688 [======>.......................] - ETA: 3s - loss: 0.0123 - accuracy: 0.9962
420/1688 [======>.......................] - ETA: 3s - loss: 0.0123 - accuracy: 0.9962
440/1688 [======>.......................] - ETA: 3s - loss: 0.0124 - accuracy: 0.9960
460/1688 [=======>......................] - ETA: 2s - loss: 0.0124 - accuracy: 0.9960
480/1688 [=======>......................] - ETA: 2s - loss: 0.0121 - accuracy: 0.9962
501/1688 [=======>......................] - ETA: 2s - loss: 0.0119 - accuracy: 0.9962
522/1688 [========>.....................] - ETA: 2s - loss: 0.0120 - accuracy: 0.9962
543/1688 [========>.....................] - ETA: 2s - loss: 0.0118 - accuracy: 0.9963
564/1688 [=========>....................] - ETA: 2s - loss: 0.0116 - accuracy: 0.9964
584/1688 [=========>....................] - ETA: 2s - loss: 0.0114 - accuracy: 0.9965
606/1688 [=========>....................] - ETA: 2s - loss: 0.0116 - accuracy: 0.9964
627/1688 [==========>...................] - ETA: 2s - loss: 0.0116 - accuracy: 0.9964
648/1688 [==========>...................] - ETA: 2s - loss: 0.0113 - accuracy: 0.9965
669/1688 [==========>...................] - ETA: 2s - loss: 0.0114 - accuracy: 0.9964
690/1688 [===========>..................] - ETA: 2s - loss: 0.0113 - accuracy: 0.9965
711/1688 [===========>..................] - ETA: 2s - loss: 0.0114 - accuracy: 0.9965
732/1688 [============>.................] - ETA: 2s - loss: 0.0113 - accuracy: 0.9966
753/1688 [============>.................] - ETA: 2s - loss: 0.0116 - accuracy: 0.9964
774/1688 [============>.................] - ETA: 2s - loss: 0.0116 - accuracy: 0.9964
795/1688 [=============>................] - ETA: 2s - loss: 0.0118 - accuracy: 0.9963
816/1688 [=============>................] - ETA: 2s - loss: 0.0117 - accuracy: 0.9964
837/1688 [=============>................] - ETA: 2s - loss: 0.0118 - accuracy: 0.9963
858/1688 [==============>...............] - ETA: 2s - loss: 0.0116 - accuracy: 0.9964
879/1688 [==============>...............] - ETA: 1s - loss: 0.0114 - accuracy: 0.9965
900/1688 [==============>...............] - ETA: 1s - loss: 0.0114 - accuracy: 0.9965
921/1688 [===============>..............] - ETA: 1s - loss: 0.0115 - accuracy: 0.9964
942/1688 [===============>..............] - ETA: 1s - loss: 0.0116 - accuracy: 0.9964
963/1688 [================>.............] - ETA: 1s - loss: 0.0117 - accuracy: 0.9964
983/1688 [================>.............] - ETA: 1s - loss: 0.0117 - accuracy: 0.9962
1004/1688 [================>.............] - ETA: 1s - loss: 0.0118 - accuracy: 0.9961
1025/1688 [=================>............] - ETA: 1s - loss: 0.0118 - accuracy: 0.9961
1046/1688 [=================>............] - ETA: 1s - loss: 0.0120 - accuracy: 0.9960
1066/1688 [=================>............] - ETA: 1s - loss: 0.0121 - accuracy: 0.9959
1087/1688 [==================>...........] - ETA: 1s - loss: 0.0125 - accuracy: 0.9958
1108/1688 [==================>...........] - ETA: 1s - loss: 0.0126 - accuracy: 0.9957
1128/1688 [===================>..........] - ETA: 1s - loss: 0.0129 - accuracy: 0.9956
1149/1688 [===================>..........] - ETA: 1s - loss: 0.0129 - accuracy: 0.9956
1170/1688 [===================>..........] - ETA: 1s - loss: 0.0131 - accuracy: 0.9955
1191/1688 [====================>.........] - ETA: 1s - loss: 0.0132 - accuracy: 0.9955
1212/1688 [====================>.........] - ETA: 1s - loss: 0.0136 - accuracy: 0.9955
1233/1688 [====================>.........] - ETA: 1s - loss: 0.0136 - accuracy: 0.9954
1254/1688 [=====================>........] - ETA: 1s - loss: 0.0139 - accuracy: 0.9954
1275/1688 [=====================>........] - ETA: 1s - loss: 0.0141 - accuracy: 0.9953
1296/1688 [======================>.......] - ETA: 0s - loss: 0.0140 - accuracy: 0.9954
1317/1688 [======================>.......] - ETA: 0s - loss: 0.0141 - accuracy: 0.9954
1338/1688 [======================>.......] - ETA: 0s - loss: 0.0142 - accuracy: 0.9954
1359/1688 [=======================>......] - ETA: 0s - loss: 0.0142 - accuracy: 0.9954
1380/1688 [=======================>......] - ETA: 0s - loss: 0.0141 - accuracy: 0.9953
1401/1688 [=======================>......] - ETA: 0s - loss: 0.0141 - accuracy: 0.9954
1421/1688 [========================>.....] - ETA: 0s - loss: 0.0139 - accuracy: 0.9954
1442/1688 [========================>.....] - ETA: 0s - loss: 0.0139 - accuracy: 0.9954
1463/1688 [=========================>....] - ETA: 0s - loss: 0.0140 - accuracy: 0.9954
1483/1688 [=========================>....] - ETA: 0s - loss: 0.0142 - accuracy: 0.9953
1504/1688 [=========================>....] - ETA: 0s - loss: 0.0142 - accuracy: 0.9953
1525/1688 [==========================>...] - ETA: 0s - loss: 0.0141 - accuracy: 0.9953
1547/1688 [==========================>...] - ETA: 0s - loss: 0.0141 - accuracy: 0.9953
1568/1688 [==========================>...] - ETA: 0s - loss: 0.0142 - accuracy: 0.9953
1589/1688 [===========================>..] - ETA: 0s - loss: 0.0143 - accuracy: 0.9952
1610/1688 [===========================>..] - ETA: 0s - loss: 0.0144 - accuracy: 0.9951
1631/1688 [===========================>..] - ETA: 0s - loss: 0.0143 - accuracy: 0.9952
1653/1688 [============================>.] - ETA: 0s - loss: 0.0144 - accuracy: 0.9952
1673/1688 [============================>.] - ETA: 0s - loss: 0.0147 - accuracy: 0.9951
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0147 - accuracy: 0.9951 - val_loss: 0.0633 - val_accuracy: 0.9862
score = model_keras.evaluate(x_test, y_test, verbose=0)
print('Test accuracy:', score[1])
Test accuracy: 0.9835000038146973
2. Quantize
2.1. 8-bit quantization
An Akida accelerator processes 8 or 4-bits integer activations and weights. Therefore, the floating point Keras model must be quantized in preparation to run on an Akida accelerator.
The QuantizeML quantize function can be used to quantize a Keras model for Akida. For this step in this example, an “8/8/8” quantization scheme will be applied to the floating point Keras model to produce 8-bit weights in the first layer, 8-bit weights in all other layers, and 8-bit activations.
The quantization process results in a Keras model with custom QuantizeML quantized layers substituted for the original Keras layers.
All Keras API functions can be applied on this new model: summary(), compile(), fit(). etc.
Note
The quantize function applies several transformations to
the original model. For example, it folds the batch normalization layers into the
corresponding neural layers. The new weights are computed according to this folding
operation.
from quantizeml.models import quantize, QuantizationParams
qparams = QuantizationParams(input_weight_bits=8, weight_bits=8, activation_bits=8)
model_quantized = quantize(model_keras, qparams=qparams)
/usr/local/lib/python3.11/dist-packages/quantizeml/models/quantize.py:488: UserWarning: Quantizing per-axis with random calibration samples is not accurate. Set QuantizationParams.per_tensor_activations=True when calibrating with random samples.
warnings.warn("Quantizing per-axis with random calibration samples is not accurate. "
1/1024 [..............................] - ETA: 3:44
51/1024 [>.............................] - ETA: 0s
102/1024 [=>............................] - ETA: 0s
154/1024 [===>..........................] - ETA: 0s
206/1024 [=====>........................] - ETA: 0s
257/1024 [======>.......................] - ETA: 0s
307/1024 [=======>......................] - ETA: 0s
357/1024 [=========>....................] - ETA: 0s
408/1024 [==========>...................] - ETA: 0s
459/1024 [============>.................] - ETA: 0s
511/1024 [=============>................] - ETA: 0s
562/1024 [===============>..............] - ETA: 0s
614/1024 [================>.............] - ETA: 0s
665/1024 [==================>...........] - ETA: 0s
717/1024 [====================>.........] - ETA: 0s
769/1024 [=====================>........] - ETA: 0s
820/1024 [=======================>......] - ETA: 0s
871/1024 [========================>.....] - ETA: 0s
923/1024 [==========================>...] - ETA: 0s
975/1024 [===========================>..] - ETA: 0s
1024/1024 [==============================] - 1s 986us/step
model_quantized.summary()
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling_input (InputLaye [(None, 28, 28, 1)] 0
r)
rescaling (QuantizedRescal (None, 28, 28, 1) 0
ing)
conv2d (QuantizedConv2D) (None, 13, 13, 32) 320
re_lu (QuantizedReLU) (None, 13, 13, 32) 64
depthwise_conv2d (Quantize (None, 7, 7, 32) 384
dDepthwiseConv2D)
conv2d_1 (QuantizedConv2D) (None, 7, 7, 64) 2112
re_lu_1 (QuantizedReLU) (None, 7, 7, 64) 128
flatten (QuantizedFlatten) (None, 3136) 0
dense (QuantizedDense) (None, 10) 31370
dequantizer (Dequantizer) (None, 10) 0
=================================================================
Total params: 34378 (134.29 KB)
Trainable params: 34122 (133.29 KB)
Non-trainable params: 256 (1.00 KB)
_________________________________________________________________
Note
Note that the number of parameters for the floating and quantized models differs, a consequence of the BatchNormalization folding and the additional parameters added for quantization. For further details, please refer to their respective summary.
Check the quantized model accuracy.
def compile_evaluate(model):
""" Compiles and evaluates the model, then return accuracy score. """
model.compile(metrics=['accuracy'])
return model.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after 8-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 8-bit quantization: 0.9787999987602234
2.2. Effect of calibration
The previous call to quantize was made with random samples for calibration
(default parameters). While the observed drop in accuracy is minimal, that is
around 1%, it can be worse on more complex models. Therefore, it is advised to
use a set of real samples from the training set for calibration during a call
to quantize.
Note that this remains a calibration step rather than a training step in that
no output labels are required. Furthermore, any relevant data could be used for
calibration. The recommended settings for calibration that are widely used to
obtain the zoo performance are:
1024 samples
a batch size of 100
2 epochs
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 2s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 1ms/step
Check the accuracy for the quantized and calibrated model.
print('Test accuracy after calibration:', compile_evaluate(model_quantized))
Test accuracy after calibration: 0.9830999970436096
Calibrating with real samples on this model recovers the initial float accuracy.
2.3. 4-bit quantization
The accuracy of the 8/8/8 quantized model is equal to that of the Keras floating point model. In some cases, a smaller memory size for the model is required. This can be accomplished through quantization of the model to smaller bitwidths.
The model will now be quantized to 8/4/4, that is 8-bit weights in the first layer with 4-bit weights and activations in all other layers. Such a quantization scheme will usually introduce a performance drop.
qparams = QuantizationParams(input_weight_bits=8, weight_bits=4, activation_bits=4)
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 999us/step
Check the 4-bit quantized accuracy.
print('Test accuracy after 4-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 4-bit quantization: 0.9800000190734863
2.4. Model fine tuning (Quantization Aware Training)
When a model suffers from an accuracy drop after quantization, fine tuning or Quantization Aware Training (QAT) may recover some or all of the original performance.
Note that since this is a fine tuning step, both the number of epochs and learning rate are expected to be lower than during the initial float training.
model_quantized.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-4),
metrics=['accuracy'])
model_quantized.fit(x_train, y_train, epochs=5, validation_split=0.1)
Epoch 1/5
1/1688 [..............................] - ETA: 2:15:05 - loss: 0.0876 - accuracy: 0.9688
9/1688 [..............................] - ETA: 10s - loss: 0.0303 - accuracy: 0.9931
17/1688 [..............................] - ETA: 10s - loss: 0.0252 - accuracy: 0.9908
25/1688 [..............................] - ETA: 10s - loss: 0.0189 - accuracy: 0.9937
33/1688 [..............................] - ETA: 10s - loss: 0.0176 - accuracy: 0.9934
42/1688 [..............................] - ETA: 10s - loss: 0.0186 - accuracy: 0.9926
51/1688 [..............................] - ETA: 10s - loss: 0.0166 - accuracy: 0.9939
60/1688 [>.............................] - ETA: 10s - loss: 0.0192 - accuracy: 0.9927
68/1688 [>.............................] - ETA: 10s - loss: 0.0185 - accuracy: 0.9926
77/1688 [>.............................] - ETA: 10s - loss: 0.0185 - accuracy: 0.9931
85/1688 [>.............................] - ETA: 10s - loss: 0.0174 - accuracy: 0.9937
94/1688 [>.............................] - ETA: 10s - loss: 0.0166 - accuracy: 0.9940
103/1688 [>.............................] - ETA: 10s - loss: 0.0156 - accuracy: 0.9945
112/1688 [>.............................] - ETA: 9s - loss: 0.0163 - accuracy: 0.9941
121/1688 [=>............................] - ETA: 9s - loss: 0.0163 - accuracy: 0.9943
129/1688 [=>............................] - ETA: 9s - loss: 0.0154 - accuracy: 0.9947
138/1688 [=>............................] - ETA: 9s - loss: 0.0149 - accuracy: 0.9950
146/1688 [=>............................] - ETA: 9s - loss: 0.0155 - accuracy: 0.9944
154/1688 [=>............................] - ETA: 9s - loss: 0.0152 - accuracy: 0.9945
162/1688 [=>............................] - ETA: 9s - loss: 0.0149 - accuracy: 0.9946
171/1688 [==>...........................] - ETA: 9s - loss: 0.0150 - accuracy: 0.9943
179/1688 [==>...........................] - ETA: 9s - loss: 0.0144 - accuracy: 0.9946
187/1688 [==>...........................] - ETA: 9s - loss: 0.0146 - accuracy: 0.9943
195/1688 [==>...........................] - ETA: 9s - loss: 0.0146 - accuracy: 0.9944
203/1688 [==>...........................] - ETA: 9s - loss: 0.0158 - accuracy: 0.9943
212/1688 [==>...........................] - ETA: 9s - loss: 0.0158 - accuracy: 0.9943
221/1688 [==>...........................] - ETA: 9s - loss: 0.0154 - accuracy: 0.9945
229/1688 [===>..........................] - ETA: 9s - loss: 0.0151 - accuracy: 0.9945
237/1688 [===>..........................] - ETA: 9s - loss: 0.0147 - accuracy: 0.9947
245/1688 [===>..........................] - ETA: 9s - loss: 0.0149 - accuracy: 0.9944
253/1688 [===>..........................] - ETA: 9s - loss: 0.0148 - accuracy: 0.9944
262/1688 [===>..........................] - ETA: 9s - loss: 0.0146 - accuracy: 0.9944
270/1688 [===>..........................] - ETA: 8s - loss: 0.0148 - accuracy: 0.9944
278/1688 [===>..........................] - ETA: 8s - loss: 0.0148 - accuracy: 0.9944
287/1688 [====>.........................] - ETA: 8s - loss: 0.0145 - accuracy: 0.9944
296/1688 [====>.........................] - ETA: 8s - loss: 0.0144 - accuracy: 0.9946
305/1688 [====>.........................] - ETA: 8s - loss: 0.0145 - accuracy: 0.9944
313/1688 [====>.........................] - ETA: 8s - loss: 0.0145 - accuracy: 0.9944
322/1688 [====>.........................] - ETA: 8s - loss: 0.0142 - accuracy: 0.9946
330/1688 [====>.........................] - ETA: 8s - loss: 0.0139 - accuracy: 0.9947
338/1688 [=====>........................] - ETA: 8s - loss: 0.0138 - accuracy: 0.9948
346/1688 [=====>........................] - ETA: 8s - loss: 0.0137 - accuracy: 0.9949
354/1688 [=====>........................] - ETA: 8s - loss: 0.0136 - accuracy: 0.9949
363/1688 [=====>........................] - ETA: 8s - loss: 0.0142 - accuracy: 0.9947
372/1688 [=====>........................] - ETA: 8s - loss: 0.0143 - accuracy: 0.9947
380/1688 [=====>........................] - ETA: 8s - loss: 0.0143 - accuracy: 0.9947
389/1688 [=====>........................] - ETA: 8s - loss: 0.0140 - accuracy: 0.9949
398/1688 [======>.......................] - ETA: 8s - loss: 0.0138 - accuracy: 0.9950
406/1688 [======>.......................] - ETA: 8s - loss: 0.0136 - accuracy: 0.9951
414/1688 [======>.......................] - ETA: 8s - loss: 0.0138 - accuracy: 0.9951
422/1688 [======>.......................] - ETA: 7s - loss: 0.0138 - accuracy: 0.9950
430/1688 [======>.......................] - ETA: 7s - loss: 0.0136 - accuracy: 0.9951
438/1688 [======>.......................] - ETA: 7s - loss: 0.0135 - accuracy: 0.9952
446/1688 [======>.......................] - ETA: 7s - loss: 0.0136 - accuracy: 0.9950
455/1688 [=======>......................] - ETA: 7s - loss: 0.0135 - accuracy: 0.9951
464/1688 [=======>......................] - ETA: 7s - loss: 0.0137 - accuracy: 0.9950
472/1688 [=======>......................] - ETA: 7s - loss: 0.0138 - accuracy: 0.9948
481/1688 [=======>......................] - ETA: 7s - loss: 0.0137 - accuracy: 0.9949
490/1688 [=======>......................] - ETA: 7s - loss: 0.0138 - accuracy: 0.9948
499/1688 [=======>......................] - ETA: 7s - loss: 0.0137 - accuracy: 0.9949
508/1688 [========>.....................] - ETA: 7s - loss: 0.0136 - accuracy: 0.9950
516/1688 [========>.....................] - ETA: 7s - loss: 0.0135 - accuracy: 0.9950
524/1688 [========>.....................] - ETA: 7s - loss: 0.0134 - accuracy: 0.9950
533/1688 [========>.....................] - ETA: 7s - loss: 0.0132 - accuracy: 0.9951
542/1688 [========>.....................] - ETA: 7s - loss: 0.0131 - accuracy: 0.9951
551/1688 [========>.....................] - ETA: 7s - loss: 0.0129 - accuracy: 0.9952
559/1688 [========>.....................] - ETA: 7s - loss: 0.0129 - accuracy: 0.9951
568/1688 [=========>....................] - ETA: 7s - loss: 0.0132 - accuracy: 0.9950
577/1688 [=========>....................] - ETA: 6s - loss: 0.0133 - accuracy: 0.9949
585/1688 [=========>....................] - ETA: 6s - loss: 0.0132 - accuracy: 0.9950
593/1688 [=========>....................] - ETA: 6s - loss: 0.0131 - accuracy: 0.9950
601/1688 [=========>....................] - ETA: 6s - loss: 0.0133 - accuracy: 0.9950
610/1688 [=========>....................] - ETA: 6s - loss: 0.0132 - accuracy: 0.9950
618/1688 [=========>....................] - ETA: 6s - loss: 0.0133 - accuracy: 0.9950
627/1688 [==========>...................] - ETA: 6s - loss: 0.0132 - accuracy: 0.9951
636/1688 [==========>...................] - ETA: 6s - loss: 0.0132 - accuracy: 0.9950
644/1688 [==========>...................] - ETA: 6s - loss: 0.0132 - accuracy: 0.9950
653/1688 [==========>...................] - ETA: 6s - loss: 0.0131 - accuracy: 0.9950
661/1688 [==========>...................] - ETA: 6s - loss: 0.0132 - accuracy: 0.9950
669/1688 [==========>...................] - ETA: 6s - loss: 0.0132 - accuracy: 0.9950
677/1688 [===========>..................] - ETA: 6s - loss: 0.0134 - accuracy: 0.9950
686/1688 [===========>..................] - ETA: 6s - loss: 0.0134 - accuracy: 0.9950
694/1688 [===========>..................] - ETA: 6s - loss: 0.0134 - accuracy: 0.9950
703/1688 [===========>..................] - ETA: 6s - loss: 0.0133 - accuracy: 0.9950
711/1688 [===========>..................] - ETA: 6s - loss: 0.0134 - accuracy: 0.9949
720/1688 [===========>..................] - ETA: 6s - loss: 0.0134 - accuracy: 0.9950
728/1688 [===========>..................] - ETA: 6s - loss: 0.0134 - accuracy: 0.9950
736/1688 [============>.................] - ETA: 5s - loss: 0.0133 - accuracy: 0.9950
744/1688 [============>.................] - ETA: 5s - loss: 0.0134 - accuracy: 0.9950
753/1688 [============>.................] - ETA: 5s - loss: 0.0134 - accuracy: 0.9949
761/1688 [============>.................] - ETA: 5s - loss: 0.0135 - accuracy: 0.9949
769/1688 [============>.................] - ETA: 5s - loss: 0.0134 - accuracy: 0.9949
777/1688 [============>.................] - ETA: 5s - loss: 0.0136 - accuracy: 0.9948
785/1688 [============>.................] - ETA: 5s - loss: 0.0135 - accuracy: 0.9949
794/1688 [=============>................] - ETA: 5s - loss: 0.0135 - accuracy: 0.9948
803/1688 [=============>................] - ETA: 5s - loss: 0.0136 - accuracy: 0.9947
812/1688 [=============>................] - ETA: 5s - loss: 0.0135 - accuracy: 0.9948
820/1688 [=============>................] - ETA: 5s - loss: 0.0134 - accuracy: 0.9948
828/1688 [=============>................] - ETA: 5s - loss: 0.0134 - accuracy: 0.9948
837/1688 [=============>................] - ETA: 5s - loss: 0.0135 - accuracy: 0.9948
845/1688 [==============>...............] - ETA: 5s - loss: 0.0134 - accuracy: 0.9948
854/1688 [==============>...............] - ETA: 5s - loss: 0.0134 - accuracy: 0.9948
862/1688 [==============>...............] - ETA: 5s - loss: 0.0133 - accuracy: 0.9949
870/1688 [==============>...............] - ETA: 5s - loss: 0.0135 - accuracy: 0.9949
878/1688 [==============>...............] - ETA: 5s - loss: 0.0134 - accuracy: 0.9949
886/1688 [==============>...............] - ETA: 5s - loss: 0.0133 - accuracy: 0.9950
895/1688 [==============>...............] - ETA: 4s - loss: 0.0134 - accuracy: 0.9949
904/1688 [===============>..............] - ETA: 4s - loss: 0.0133 - accuracy: 0.9950
912/1688 [===============>..............] - ETA: 4s - loss: 0.0134 - accuracy: 0.9950
921/1688 [===============>..............] - ETA: 4s - loss: 0.0134 - accuracy: 0.9950
929/1688 [===============>..............] - ETA: 4s - loss: 0.0133 - accuracy: 0.9950
938/1688 [===============>..............] - ETA: 4s - loss: 0.0133 - accuracy: 0.9950
947/1688 [===============>..............] - ETA: 4s - loss: 0.0133 - accuracy: 0.9950
956/1688 [===============>..............] - ETA: 4s - loss: 0.0133 - accuracy: 0.9950
965/1688 [================>.............] - ETA: 4s - loss: 0.0132 - accuracy: 0.9950
973/1688 [================>.............] - ETA: 4s - loss: 0.0132 - accuracy: 0.9950
981/1688 [================>.............] - ETA: 4s - loss: 0.0131 - accuracy: 0.9950
989/1688 [================>.............] - ETA: 4s - loss: 0.0131 - accuracy: 0.9950
998/1688 [================>.............] - ETA: 4s - loss: 0.0131 - accuracy: 0.9950
1006/1688 [================>.............] - ETA: 4s - loss: 0.0131 - accuracy: 0.9950
1014/1688 [=================>............] - ETA: 4s - loss: 0.0130 - accuracy: 0.9950
1022/1688 [=================>............] - ETA: 4s - loss: 0.0130 - accuracy: 0.9950
1030/1688 [=================>............] - ETA: 4s - loss: 0.0130 - accuracy: 0.9950
1038/1688 [=================>............] - ETA: 4s - loss: 0.0130 - accuracy: 0.9951
1046/1688 [=================>............] - ETA: 4s - loss: 0.0129 - accuracy: 0.9951
1055/1688 [=================>............] - ETA: 3s - loss: 0.0130 - accuracy: 0.9951
1063/1688 [=================>............] - ETA: 3s - loss: 0.0129 - accuracy: 0.9951
1072/1688 [==================>...........] - ETA: 3s - loss: 0.0129 - accuracy: 0.9951
1080/1688 [==================>...........] - ETA: 3s - loss: 0.0128 - accuracy: 0.9952
1088/1688 [==================>...........] - ETA: 3s - loss: 0.0128 - accuracy: 0.9952
1097/1688 [==================>...........] - ETA: 3s - loss: 0.0127 - accuracy: 0.9952
1106/1688 [==================>...........] - ETA: 3s - loss: 0.0127 - accuracy: 0.9953
1114/1688 [==================>...........] - ETA: 3s - loss: 0.0126 - accuracy: 0.9953
1123/1688 [==================>...........] - ETA: 3s - loss: 0.0127 - accuracy: 0.9953
1131/1688 [===================>..........] - ETA: 3s - loss: 0.0126 - accuracy: 0.9953
1140/1688 [===================>..........] - ETA: 3s - loss: 0.0126 - accuracy: 0.9953
1148/1688 [===================>..........] - ETA: 3s - loss: 0.0125 - accuracy: 0.9953
1156/1688 [===================>..........] - ETA: 3s - loss: 0.0125 - accuracy: 0.9954
1164/1688 [===================>..........] - ETA: 3s - loss: 0.0126 - accuracy: 0.9953
1172/1688 [===================>..........] - ETA: 3s - loss: 0.0126 - accuracy: 0.9953
1181/1688 [===================>..........] - ETA: 3s - loss: 0.0126 - accuracy: 0.9953
1189/1688 [====================>.........] - ETA: 3s - loss: 0.0125 - accuracy: 0.9953
1197/1688 [====================>.........] - ETA: 3s - loss: 0.0125 - accuracy: 0.9953
1205/1688 [====================>.........] - ETA: 3s - loss: 0.0125 - accuracy: 0.9954
1214/1688 [====================>.........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9953
1222/1688 [====================>.........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9953
1230/1688 [====================>.........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9954
1238/1688 [=====================>........] - ETA: 2s - loss: 0.0126 - accuracy: 0.9953
1246/1688 [=====================>........] - ETA: 2s - loss: 0.0126 - accuracy: 0.9953
1255/1688 [=====================>........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9953
1264/1688 [=====================>........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9954
1273/1688 [=====================>........] - ETA: 2s - loss: 0.0126 - accuracy: 0.9953
1281/1688 [=====================>........] - ETA: 2s - loss: 0.0126 - accuracy: 0.9953
1290/1688 [=====================>........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9954
1298/1688 [======================>.......] - ETA: 2s - loss: 0.0126 - accuracy: 0.9953
1307/1688 [======================>.......] - ETA: 2s - loss: 0.0125 - accuracy: 0.9954
1315/1688 [======================>.......] - ETA: 2s - loss: 0.0125 - accuracy: 0.9953
1323/1688 [======================>.......] - ETA: 2s - loss: 0.0125 - accuracy: 0.9953
1332/1688 [======================>.......] - ETA: 2s - loss: 0.0125 - accuracy: 0.9953
1340/1688 [======================>.......] - ETA: 2s - loss: 0.0126 - accuracy: 0.9953
1348/1688 [======================>.......] - ETA: 2s - loss: 0.0127 - accuracy: 0.9953
1357/1688 [=======================>......] - ETA: 2s - loss: 0.0126 - accuracy: 0.9953
1366/1688 [=======================>......] - ETA: 2s - loss: 0.0126 - accuracy: 0.9953
1375/1688 [=======================>......] - ETA: 1s - loss: 0.0126 - accuracy: 0.9953
1383/1688 [=======================>......] - ETA: 1s - loss: 0.0125 - accuracy: 0.9953
1391/1688 [=======================>......] - ETA: 1s - loss: 0.0125 - accuracy: 0.9954
1399/1688 [=======================>......] - ETA: 1s - loss: 0.0124 - accuracy: 0.9954
1407/1688 [========================>.....] - ETA: 1s - loss: 0.0124 - accuracy: 0.9954
1416/1688 [========================>.....] - ETA: 1s - loss: 0.0124 - accuracy: 0.9954
1424/1688 [========================>.....] - ETA: 1s - loss: 0.0123 - accuracy: 0.9954
1432/1688 [========================>.....] - ETA: 1s - loss: 0.0123 - accuracy: 0.9954
1440/1688 [========================>.....] - ETA: 1s - loss: 0.0122 - accuracy: 0.9955
1448/1688 [========================>.....] - ETA: 1s - loss: 0.0122 - accuracy: 0.9955
1456/1688 [========================>.....] - ETA: 1s - loss: 0.0122 - accuracy: 0.9955
1465/1688 [=========================>....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9955
1473/1688 [=========================>....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9955
1482/1688 [=========================>....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9955
1491/1688 [=========================>....] - ETA: 1s - loss: 0.0122 - accuracy: 0.9955
1499/1688 [=========================>....] - ETA: 1s - loss: 0.0122 - accuracy: 0.9955
1507/1688 [=========================>....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9955
1516/1688 [=========================>....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9955
1525/1688 [==========================>...] - ETA: 1s - loss: 0.0121 - accuracy: 0.9955
1534/1688 [==========================>...] - ETA: 0s - loss: 0.0121 - accuracy: 0.9955
1542/1688 [==========================>...] - ETA: 0s - loss: 0.0121 - accuracy: 0.9955
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0120 - accuracy: 0.9955
1559/1688 [==========================>...] - ETA: 0s - loss: 0.0120 - accuracy: 0.9955
1568/1688 [==========================>...] - ETA: 0s - loss: 0.0120 - accuracy: 0.9955
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0120 - accuracy: 0.9955
1585/1688 [===========================>..] - ETA: 0s - loss: 0.0120 - accuracy: 0.9955
1594/1688 [===========================>..] - ETA: 0s - loss: 0.0120 - accuracy: 0.9955
1602/1688 [===========================>..] - ETA: 0s - loss: 0.0120 - accuracy: 0.9955
1610/1688 [===========================>..] - ETA: 0s - loss: 0.0120 - accuracy: 0.9955
1618/1688 [===========================>..] - ETA: 0s - loss: 0.0119 - accuracy: 0.9955
1627/1688 [===========================>..] - ETA: 0s - loss: 0.0119 - accuracy: 0.9955
1635/1688 [============================>.] - ETA: 0s - loss: 0.0119 - accuracy: 0.9955
1644/1688 [============================>.] - ETA: 0s - loss: 0.0119 - accuracy: 0.9955
1652/1688 [============================>.] - ETA: 0s - loss: 0.0119 - accuracy: 0.9955
1660/1688 [============================>.] - ETA: 0s - loss: 0.0120 - accuracy: 0.9955
1668/1688 [============================>.] - ETA: 0s - loss: 0.0120 - accuracy: 0.9955
1677/1688 [============================>.] - ETA: 0s - loss: 0.0119 - accuracy: 0.9955
1685/1688 [============================>.] - ETA: 0s - loss: 0.0119 - accuracy: 0.9955
1688/1688 [==============================] - 17s 7ms/step - loss: 0.0119 - accuracy: 0.9955 - val_loss: 0.0527 - val_accuracy: 0.9897
Epoch 2/5
1/1688 [..............................] - ETA: 11s - loss: 0.0021 - accuracy: 1.0000
9/1688 [..............................] - ETA: 10s - loss: 0.0079 - accuracy: 0.9965
18/1688 [..............................] - ETA: 10s - loss: 0.0083 - accuracy: 0.9948
26/1688 [..............................] - ETA: 10s - loss: 0.0089 - accuracy: 0.9952
34/1688 [..............................] - ETA: 10s - loss: 0.0097 - accuracy: 0.9936
42/1688 [..............................] - ETA: 10s - loss: 0.0100 - accuracy: 0.9933
50/1688 [..............................] - ETA: 10s - loss: 0.0098 - accuracy: 0.9944
58/1688 [>.............................] - ETA: 10s - loss: 0.0106 - accuracy: 0.9946
66/1688 [>.............................] - ETA: 10s - loss: 0.0100 - accuracy: 0.9953
75/1688 [>.............................] - ETA: 10s - loss: 0.0101 - accuracy: 0.9950
83/1688 [>.............................] - ETA: 10s - loss: 0.0103 - accuracy: 0.9951
92/1688 [>.............................] - ETA: 10s - loss: 0.0098 - accuracy: 0.9956
100/1688 [>.............................] - ETA: 10s - loss: 0.0097 - accuracy: 0.9956
109/1688 [>.............................] - ETA: 10s - loss: 0.0094 - accuracy: 0.9960
117/1688 [=>............................] - ETA: 10s - loss: 0.0089 - accuracy: 0.9963
125/1688 [=>............................] - ETA: 9s - loss: 0.0086 - accuracy: 0.9965
133/1688 [=>............................] - ETA: 9s - loss: 0.0082 - accuracy: 0.9967
141/1688 [=>............................] - ETA: 9s - loss: 0.0081 - accuracy: 0.9969
149/1688 [=>............................] - ETA: 9s - loss: 0.0082 - accuracy: 0.9971
157/1688 [=>............................] - ETA: 9s - loss: 0.0085 - accuracy: 0.9970
166/1688 [=>............................] - ETA: 9s - loss: 0.0084 - accuracy: 0.9970
175/1688 [==>...........................] - ETA: 9s - loss: 0.0087 - accuracy: 0.9968
184/1688 [==>...........................] - ETA: 9s - loss: 0.0084 - accuracy: 0.9969
192/1688 [==>...........................] - ETA: 9s - loss: 0.0086 - accuracy: 0.9967
201/1688 [==>...........................] - ETA: 9s - loss: 0.0086 - accuracy: 0.9967
210/1688 [==>...........................] - ETA: 9s - loss: 0.0085 - accuracy: 0.9967
219/1688 [==>...........................] - ETA: 9s - loss: 0.0083 - accuracy: 0.9969
228/1688 [===>..........................] - ETA: 9s - loss: 0.0083 - accuracy: 0.9968
237/1688 [===>..........................] - ETA: 9s - loss: 0.0082 - accuracy: 0.9970
245/1688 [===>..........................] - ETA: 9s - loss: 0.0083 - accuracy: 0.9968
253/1688 [===>..........................] - ETA: 9s - loss: 0.0082 - accuracy: 0.9969
261/1688 [===>..........................] - ETA: 9s - loss: 0.0083 - accuracy: 0.9968
269/1688 [===>..........................] - ETA: 8s - loss: 0.0081 - accuracy: 0.9969
277/1688 [===>..........................] - ETA: 8s - loss: 0.0079 - accuracy: 0.9970
285/1688 [====>.........................] - ETA: 8s - loss: 0.0079 - accuracy: 0.9969
293/1688 [====>.........................] - ETA: 8s - loss: 0.0078 - accuracy: 0.9970
301/1688 [====>.........................] - ETA: 8s - loss: 0.0078 - accuracy: 0.9970
310/1688 [====>.........................] - ETA: 8s - loss: 0.0077 - accuracy: 0.9971
319/1688 [====>.........................] - ETA: 8s - loss: 0.0077 - accuracy: 0.9972
328/1688 [====>.........................] - ETA: 8s - loss: 0.0076 - accuracy: 0.9971
336/1688 [====>.........................] - ETA: 8s - loss: 0.0075 - accuracy: 0.9972
345/1688 [=====>........................] - ETA: 8s - loss: 0.0076 - accuracy: 0.9971
354/1688 [=====>........................] - ETA: 8s - loss: 0.0075 - accuracy: 0.9972
362/1688 [=====>........................] - ETA: 8s - loss: 0.0074 - accuracy: 0.9972
370/1688 [=====>........................] - ETA: 8s - loss: 0.0075 - accuracy: 0.9972
379/1688 [=====>........................] - ETA: 8s - loss: 0.0074 - accuracy: 0.9973
387/1688 [=====>........................] - ETA: 8s - loss: 0.0075 - accuracy: 0.9973
395/1688 [======>.......................] - ETA: 8s - loss: 0.0076 - accuracy: 0.9972
403/1688 [======>.......................] - ETA: 8s - loss: 0.0075 - accuracy: 0.9973
412/1688 [======>.......................] - ETA: 8s - loss: 0.0074 - accuracy: 0.9973
420/1688 [======>.......................] - ETA: 7s - loss: 0.0073 - accuracy: 0.9974
428/1688 [======>.......................] - ETA: 7s - loss: 0.0074 - accuracy: 0.9973
436/1688 [======>.......................] - ETA: 7s - loss: 0.0074 - accuracy: 0.9973
444/1688 [======>.......................] - ETA: 7s - loss: 0.0076 - accuracy: 0.9973
452/1688 [=======>......................] - ETA: 7s - loss: 0.0075 - accuracy: 0.9973
461/1688 [=======>......................] - ETA: 7s - loss: 0.0076 - accuracy: 0.9973
470/1688 [=======>......................] - ETA: 7s - loss: 0.0075 - accuracy: 0.9973
478/1688 [=======>......................] - ETA: 7s - loss: 0.0077 - accuracy: 0.9973
486/1688 [=======>......................] - ETA: 7s - loss: 0.0076 - accuracy: 0.9973
495/1688 [=======>......................] - ETA: 7s - loss: 0.0075 - accuracy: 0.9973
504/1688 [=======>......................] - ETA: 7s - loss: 0.0075 - accuracy: 0.9973
512/1688 [========>.....................] - ETA: 7s - loss: 0.0075 - accuracy: 0.9973
520/1688 [========>.....................] - ETA: 7s - loss: 0.0075 - accuracy: 0.9973
529/1688 [========>.....................] - ETA: 7s - loss: 0.0075 - accuracy: 0.9973
538/1688 [========>.....................] - ETA: 7s - loss: 0.0074 - accuracy: 0.9974
546/1688 [========>.....................] - ETA: 7s - loss: 0.0074 - accuracy: 0.9974
554/1688 [========>.....................] - ETA: 7s - loss: 0.0074 - accuracy: 0.9974
563/1688 [=========>....................] - ETA: 7s - loss: 0.0073 - accuracy: 0.9974
571/1688 [=========>....................] - ETA: 7s - loss: 0.0073 - accuracy: 0.9975
579/1688 [=========>....................] - ETA: 6s - loss: 0.0073 - accuracy: 0.9975
587/1688 [=========>....................] - ETA: 6s - loss: 0.0072 - accuracy: 0.9975
596/1688 [=========>....................] - ETA: 6s - loss: 0.0072 - accuracy: 0.9975
604/1688 [=========>....................] - ETA: 6s - loss: 0.0075 - accuracy: 0.9975
612/1688 [=========>....................] - ETA: 6s - loss: 0.0074 - accuracy: 0.9975
621/1688 [==========>...................] - ETA: 6s - loss: 0.0074 - accuracy: 0.9975
629/1688 [==========>...................] - ETA: 6s - loss: 0.0074 - accuracy: 0.9975
638/1688 [==========>...................] - ETA: 6s - loss: 0.0073 - accuracy: 0.9976
647/1688 [==========>...................] - ETA: 6s - loss: 0.0073 - accuracy: 0.9975
656/1688 [==========>...................] - ETA: 6s - loss: 0.0073 - accuracy: 0.9976
664/1688 [==========>...................] - ETA: 6s - loss: 0.0072 - accuracy: 0.9976
673/1688 [==========>...................] - ETA: 6s - loss: 0.0072 - accuracy: 0.9976
681/1688 [===========>..................] - ETA: 6s - loss: 0.0071 - accuracy: 0.9976
689/1688 [===========>..................] - ETA: 6s - loss: 0.0072 - accuracy: 0.9976
697/1688 [===========>..................] - ETA: 6s - loss: 0.0071 - accuracy: 0.9976
706/1688 [===========>..................] - ETA: 6s - loss: 0.0074 - accuracy: 0.9975
714/1688 [===========>..................] - ETA: 6s - loss: 0.0074 - accuracy: 0.9975
723/1688 [===========>..................] - ETA: 6s - loss: 0.0075 - accuracy: 0.9974
732/1688 [============>.................] - ETA: 6s - loss: 0.0076 - accuracy: 0.9974
740/1688 [============>.................] - ETA: 5s - loss: 0.0075 - accuracy: 0.9975
748/1688 [============>.................] - ETA: 5s - loss: 0.0075 - accuracy: 0.9975
756/1688 [============>.................] - ETA: 5s - loss: 0.0080 - accuracy: 0.9973
764/1688 [============>.................] - ETA: 5s - loss: 0.0080 - accuracy: 0.9973
772/1688 [============>.................] - ETA: 5s - loss: 0.0080 - accuracy: 0.9972
780/1688 [============>.................] - ETA: 5s - loss: 0.0081 - accuracy: 0.9972
789/1688 [=============>................] - ETA: 5s - loss: 0.0081 - accuracy: 0.9972
797/1688 [=============>................] - ETA: 5s - loss: 0.0081 - accuracy: 0.9973
805/1688 [=============>................] - ETA: 5s - loss: 0.0080 - accuracy: 0.9973
813/1688 [=============>................] - ETA: 5s - loss: 0.0080 - accuracy: 0.9973
821/1688 [=============>................] - ETA: 5s - loss: 0.0080 - accuracy: 0.9973
829/1688 [=============>................] - ETA: 5s - loss: 0.0080 - accuracy: 0.9972
837/1688 [=============>................] - ETA: 5s - loss: 0.0081 - accuracy: 0.9972
846/1688 [==============>...............] - ETA: 5s - loss: 0.0080 - accuracy: 0.9973
855/1688 [==============>...............] - ETA: 5s - loss: 0.0080 - accuracy: 0.9973
863/1688 [==============>...............] - ETA: 5s - loss: 0.0080 - accuracy: 0.9973
871/1688 [==============>...............] - ETA: 5s - loss: 0.0080 - accuracy: 0.9973
879/1688 [==============>...............] - ETA: 5s - loss: 0.0079 - accuracy: 0.9973
887/1688 [==============>...............] - ETA: 5s - loss: 0.0079 - accuracy: 0.9973
896/1688 [==============>...............] - ETA: 4s - loss: 0.0078 - accuracy: 0.9973
904/1688 [===============>..............] - ETA: 4s - loss: 0.0078 - accuracy: 0.9974
913/1688 [===============>..............] - ETA: 4s - loss: 0.0078 - accuracy: 0.9974
921/1688 [===============>..............] - ETA: 4s - loss: 0.0078 - accuracy: 0.9974
929/1688 [===============>..............] - ETA: 4s - loss: 0.0078 - accuracy: 0.9973
937/1688 [===============>..............] - ETA: 4s - loss: 0.0078 - accuracy: 0.9974
945/1688 [===============>..............] - ETA: 4s - loss: 0.0077 - accuracy: 0.9974
954/1688 [===============>..............] - ETA: 4s - loss: 0.0077 - accuracy: 0.9974
962/1688 [================>.............] - ETA: 4s - loss: 0.0077 - accuracy: 0.9974
971/1688 [================>.............] - ETA: 4s - loss: 0.0077 - accuracy: 0.9974
979/1688 [================>.............] - ETA: 4s - loss: 0.0077 - accuracy: 0.9974
988/1688 [================>.............] - ETA: 4s - loss: 0.0077 - accuracy: 0.9975
996/1688 [================>.............] - ETA: 4s - loss: 0.0077 - accuracy: 0.9975
1005/1688 [================>.............] - ETA: 4s - loss: 0.0078 - accuracy: 0.9974
1013/1688 [=================>............] - ETA: 4s - loss: 0.0077 - accuracy: 0.9974
1022/1688 [=================>............] - ETA: 4s - loss: 0.0077 - accuracy: 0.9975
1031/1688 [=================>............] - ETA: 4s - loss: 0.0077 - accuracy: 0.9975
1040/1688 [=================>............] - ETA: 4s - loss: 0.0076 - accuracy: 0.9975
1049/1688 [=================>............] - ETA: 4s - loss: 0.0076 - accuracy: 0.9975
1057/1688 [=================>............] - ETA: 3s - loss: 0.0076 - accuracy: 0.9975
1066/1688 [=================>............] - ETA: 3s - loss: 0.0075 - accuracy: 0.9975
1074/1688 [==================>...........] - ETA: 3s - loss: 0.0075 - accuracy: 0.9976
1083/1688 [==================>...........] - ETA: 3s - loss: 0.0075 - accuracy: 0.9976
1091/1688 [==================>...........] - ETA: 3s - loss: 0.0075 - accuracy: 0.9976
1100/1688 [==================>...........] - ETA: 3s - loss: 0.0075 - accuracy: 0.9976
1108/1688 [==================>...........] - ETA: 3s - loss: 0.0074 - accuracy: 0.9976
1116/1688 [==================>...........] - ETA: 3s - loss: 0.0074 - accuracy: 0.9976
1124/1688 [==================>...........] - ETA: 3s - loss: 0.0074 - accuracy: 0.9976
1133/1688 [===================>..........] - ETA: 3s - loss: 0.0075 - accuracy: 0.9976
1142/1688 [===================>..........] - ETA: 3s - loss: 0.0074 - accuracy: 0.9976
1150/1688 [===================>..........] - ETA: 3s - loss: 0.0074 - accuracy: 0.9977
1158/1688 [===================>..........] - ETA: 3s - loss: 0.0074 - accuracy: 0.9977
1167/1688 [===================>..........] - ETA: 3s - loss: 0.0075 - accuracy: 0.9976
1175/1688 [===================>..........] - ETA: 3s - loss: 0.0074 - accuracy: 0.9977
1183/1688 [====================>.........] - ETA: 3s - loss: 0.0074 - accuracy: 0.9976
1191/1688 [====================>.........] - ETA: 3s - loss: 0.0075 - accuracy: 0.9976
1199/1688 [====================>.........] - ETA: 3s - loss: 0.0075 - accuracy: 0.9977
1208/1688 [====================>.........] - ETA: 3s - loss: 0.0074 - accuracy: 0.9977
1216/1688 [====================>.........] - ETA: 2s - loss: 0.0074 - accuracy: 0.9977
1225/1688 [====================>.........] - ETA: 2s - loss: 0.0074 - accuracy: 0.9977
1234/1688 [====================>.........] - ETA: 2s - loss: 0.0073 - accuracy: 0.9977
1243/1688 [=====================>........] - ETA: 2s - loss: 0.0073 - accuracy: 0.9977
1251/1688 [=====================>........] - ETA: 2s - loss: 0.0073 - accuracy: 0.9977
1259/1688 [=====================>........] - ETA: 2s - loss: 0.0074 - accuracy: 0.9977
1268/1688 [=====================>........] - ETA: 2s - loss: 0.0074 - accuracy: 0.9977
1276/1688 [=====================>........] - ETA: 2s - loss: 0.0074 - accuracy: 0.9977
1284/1688 [=====================>........] - ETA: 2s - loss: 0.0073 - accuracy: 0.9977
1292/1688 [=====================>........] - ETA: 2s - loss: 0.0073 - accuracy: 0.9977
1301/1688 [======================>.......] - ETA: 2s - loss: 0.0074 - accuracy: 0.9976
1309/1688 [======================>.......] - ETA: 2s - loss: 0.0074 - accuracy: 0.9977
1318/1688 [======================>.......] - ETA: 2s - loss: 0.0074 - accuracy: 0.9977
1327/1688 [======================>.......] - ETA: 2s - loss: 0.0074 - accuracy: 0.9977
1335/1688 [======================>.......] - ETA: 2s - loss: 0.0074 - accuracy: 0.9977
1344/1688 [======================>.......] - ETA: 2s - loss: 0.0074 - accuracy: 0.9977
1353/1688 [=======================>......] - ETA: 2s - loss: 0.0073 - accuracy: 0.9977
1361/1688 [=======================>......] - ETA: 2s - loss: 0.0073 - accuracy: 0.9977
1370/1688 [=======================>......] - ETA: 1s - loss: 0.0074 - accuracy: 0.9977
1378/1688 [=======================>......] - ETA: 1s - loss: 0.0073 - accuracy: 0.9977
1387/1688 [=======================>......] - ETA: 1s - loss: 0.0073 - accuracy: 0.9977
1396/1688 [=======================>......] - ETA: 1s - loss: 0.0073 - accuracy: 0.9978
1404/1688 [=======================>......] - ETA: 1s - loss: 0.0073 - accuracy: 0.9978
1413/1688 [========================>.....] - ETA: 1s - loss: 0.0073 - accuracy: 0.9978
1421/1688 [========================>.....] - ETA: 1s - loss: 0.0073 - accuracy: 0.9978
1430/1688 [========================>.....] - ETA: 1s - loss: 0.0072 - accuracy: 0.9978
1438/1688 [========================>.....] - ETA: 1s - loss: 0.0072 - accuracy: 0.9978
1446/1688 [========================>.....] - ETA: 1s - loss: 0.0072 - accuracy: 0.9978
1455/1688 [========================>.....] - ETA: 1s - loss: 0.0073 - accuracy: 0.9978
1463/1688 [=========================>....] - ETA: 1s - loss: 0.0073 - accuracy: 0.9978
1472/1688 [=========================>....] - ETA: 1s - loss: 0.0073 - accuracy: 0.9978
1480/1688 [=========================>....] - ETA: 1s - loss: 0.0073 - accuracy: 0.9978
1489/1688 [=========================>....] - ETA: 1s - loss: 0.0072 - accuracy: 0.9978
1498/1688 [=========================>....] - ETA: 1s - loss: 0.0072 - accuracy: 0.9978
1507/1688 [=========================>....] - ETA: 1s - loss: 0.0072 - accuracy: 0.9978
1515/1688 [=========================>....] - ETA: 1s - loss: 0.0072 - accuracy: 0.9978
1524/1688 [==========================>...] - ETA: 1s - loss: 0.0072 - accuracy: 0.9978
1532/1688 [==========================>...] - ETA: 0s - loss: 0.0072 - accuracy: 0.9978
1540/1688 [==========================>...] - ETA: 0s - loss: 0.0071 - accuracy: 0.9978
1548/1688 [==========================>...] - ETA: 0s - loss: 0.0071 - accuracy: 0.9978
1557/1688 [==========================>...] - ETA: 0s - loss: 0.0071 - accuracy: 0.9979
1565/1688 [==========================>...] - ETA: 0s - loss: 0.0071 - accuracy: 0.9979
1573/1688 [==========================>...] - ETA: 0s - loss: 0.0071 - accuracy: 0.9979
1581/1688 [===========================>..] - ETA: 0s - loss: 0.0071 - accuracy: 0.9979
1589/1688 [===========================>..] - ETA: 0s - loss: 0.0070 - accuracy: 0.9979
1597/1688 [===========================>..] - ETA: 0s - loss: 0.0070 - accuracy: 0.9979
1606/1688 [===========================>..] - ETA: 0s - loss: 0.0070 - accuracy: 0.9979
1614/1688 [===========================>..] - ETA: 0s - loss: 0.0070 - accuracy: 0.9979
1622/1688 [===========================>..] - ETA: 0s - loss: 0.0070 - accuracy: 0.9979
1630/1688 [===========================>..] - ETA: 0s - loss: 0.0070 - accuracy: 0.9979
1639/1688 [============================>.] - ETA: 0s - loss: 0.0070 - accuracy: 0.9979
1647/1688 [============================>.] - ETA: 0s - loss: 0.0070 - accuracy: 0.9979
1655/1688 [============================>.] - ETA: 0s - loss: 0.0070 - accuracy: 0.9979
1664/1688 [============================>.] - ETA: 0s - loss: 0.0070 - accuracy: 0.9979
1672/1688 [============================>.] - ETA: 0s - loss: 0.0070 - accuracy: 0.9979
1681/1688 [============================>.] - ETA: 0s - loss: 0.0070 - accuracy: 0.9979
1688/1688 [==============================] - 11s 7ms/step - loss: 0.0070 - accuracy: 0.9979 - val_loss: 0.0518 - val_accuracy: 0.9893
Epoch 3/5
1/1688 [..............................] - ETA: 10s - loss: 0.0020 - accuracy: 1.0000
9/1688 [..............................] - ETA: 10s - loss: 0.0147 - accuracy: 0.9931
17/1688 [..............................] - ETA: 10s - loss: 0.0103 - accuracy: 0.9963
25/1688 [..............................] - ETA: 10s - loss: 0.0094 - accuracy: 0.9962
33/1688 [..............................] - ETA: 10s - loss: 0.0079 - accuracy: 0.9972
41/1688 [..............................] - ETA: 10s - loss: 0.0069 - accuracy: 0.9977
49/1688 [..............................] - ETA: 10s - loss: 0.0063 - accuracy: 0.9981
58/1688 [>.............................] - ETA: 10s - loss: 0.0065 - accuracy: 0.9978
66/1688 [>.............................] - ETA: 10s - loss: 0.0073 - accuracy: 0.9976
74/1688 [>.............................] - ETA: 10s - loss: 0.0078 - accuracy: 0.9975
82/1688 [>.............................] - ETA: 10s - loss: 0.0073 - accuracy: 0.9977
90/1688 [>.............................] - ETA: 10s - loss: 0.0073 - accuracy: 0.9979
99/1688 [>.............................] - ETA: 10s - loss: 0.0074 - accuracy: 0.9978
107/1688 [>.............................] - ETA: 10s - loss: 0.0071 - accuracy: 0.9980
115/1688 [=>............................] - ETA: 9s - loss: 0.0070 - accuracy: 0.9981
124/1688 [=>............................] - ETA: 9s - loss: 0.0066 - accuracy: 0.9982
132/1688 [=>............................] - ETA: 9s - loss: 0.0063 - accuracy: 0.9983
140/1688 [=>............................] - ETA: 9s - loss: 0.0060 - accuracy: 0.9984
148/1688 [=>............................] - ETA: 9s - loss: 0.0064 - accuracy: 0.9981
157/1688 [=>............................] - ETA: 9s - loss: 0.0062 - accuracy: 0.9982
166/1688 [=>............................] - ETA: 9s - loss: 0.0065 - accuracy: 0.9979
175/1688 [==>...........................] - ETA: 9s - loss: 0.0064 - accuracy: 0.9980
184/1688 [==>...........................] - ETA: 9s - loss: 0.0068 - accuracy: 0.9978
192/1688 [==>...........................] - ETA: 9s - loss: 0.0066 - accuracy: 0.9979
200/1688 [==>...........................] - ETA: 9s - loss: 0.0068 - accuracy: 0.9978
209/1688 [==>...........................] - ETA: 9s - loss: 0.0069 - accuracy: 0.9978
217/1688 [==>...........................] - ETA: 9s - loss: 0.0067 - accuracy: 0.9978
225/1688 [==>...........................] - ETA: 9s - loss: 0.0066 - accuracy: 0.9979
234/1688 [===>..........................] - ETA: 9s - loss: 0.0064 - accuracy: 0.9980
242/1688 [===>..........................] - ETA: 9s - loss: 0.0063 - accuracy: 0.9981
250/1688 [===>..........................] - ETA: 9s - loss: 0.0063 - accuracy: 0.9981
258/1688 [===>..........................] - ETA: 9s - loss: 0.0063 - accuracy: 0.9982
266/1688 [===>..........................] - ETA: 9s - loss: 0.0062 - accuracy: 0.9982
274/1688 [===>..........................] - ETA: 8s - loss: 0.0061 - accuracy: 0.9983
282/1688 [====>.........................] - ETA: 8s - loss: 0.0062 - accuracy: 0.9982
290/1688 [====>.........................] - ETA: 8s - loss: 0.0061 - accuracy: 0.9983
299/1688 [====>.........................] - ETA: 8s - loss: 0.0062 - accuracy: 0.9982
308/1688 [====>.........................] - ETA: 8s - loss: 0.0061 - accuracy: 0.9983
316/1688 [====>.........................] - ETA: 8s - loss: 0.0061 - accuracy: 0.9982
325/1688 [====>.........................] - ETA: 8s - loss: 0.0060 - accuracy: 0.9983
333/1688 [====>.........................] - ETA: 8s - loss: 0.0059 - accuracy: 0.9983
342/1688 [=====>........................] - ETA: 8s - loss: 0.0060 - accuracy: 0.9983
350/1688 [=====>........................] - ETA: 8s - loss: 0.0060 - accuracy: 0.9983
358/1688 [=====>........................] - ETA: 8s - loss: 0.0060 - accuracy: 0.9983
366/1688 [=====>........................] - ETA: 8s - loss: 0.0062 - accuracy: 0.9981
375/1688 [=====>........................] - ETA: 8s - loss: 0.0062 - accuracy: 0.9981
384/1688 [=====>........................] - ETA: 8s - loss: 0.0063 - accuracy: 0.9981
392/1688 [=====>........................] - ETA: 8s - loss: 0.0062 - accuracy: 0.9982
401/1688 [======>.......................] - ETA: 8s - loss: 0.0061 - accuracy: 0.9982
409/1688 [======>.......................] - ETA: 8s - loss: 0.0061 - accuracy: 0.9982
417/1688 [======>.......................] - ETA: 8s - loss: 0.0060 - accuracy: 0.9983
426/1688 [======>.......................] - ETA: 7s - loss: 0.0060 - accuracy: 0.9983
435/1688 [======>.......................] - ETA: 7s - loss: 0.0062 - accuracy: 0.9983
443/1688 [======>.......................] - ETA: 7s - loss: 0.0061 - accuracy: 0.9983
451/1688 [=======>......................] - ETA: 7s - loss: 0.0061 - accuracy: 0.9983
459/1688 [=======>......................] - ETA: 7s - loss: 0.0062 - accuracy: 0.9982
468/1688 [=======>......................] - ETA: 7s - loss: 0.0062 - accuracy: 0.9983
476/1688 [=======>......................] - ETA: 7s - loss: 0.0061 - accuracy: 0.9983
485/1688 [=======>......................] - ETA: 7s - loss: 0.0060 - accuracy: 0.9983
493/1688 [=======>......................] - ETA: 7s - loss: 0.0059 - accuracy: 0.9984
501/1688 [=======>......................] - ETA: 7s - loss: 0.0059 - accuracy: 0.9984
510/1688 [========>.....................] - ETA: 7s - loss: 0.0059 - accuracy: 0.9984
519/1688 [========>.....................] - ETA: 7s - loss: 0.0060 - accuracy: 0.9983
527/1688 [========>.....................] - ETA: 7s - loss: 0.0059 - accuracy: 0.9983
535/1688 [========>.....................] - ETA: 7s - loss: 0.0059 - accuracy: 0.9984
543/1688 [========>.....................] - ETA: 7s - loss: 0.0058 - accuracy: 0.9984
551/1688 [========>.....................] - ETA: 7s - loss: 0.0060 - accuracy: 0.9984
560/1688 [========>.....................] - ETA: 7s - loss: 0.0059 - accuracy: 0.9984
569/1688 [=========>....................] - ETA: 7s - loss: 0.0059 - accuracy: 0.9984
577/1688 [=========>....................] - ETA: 7s - loss: 0.0059 - accuracy: 0.9984
585/1688 [=========>....................] - ETA: 6s - loss: 0.0060 - accuracy: 0.9983
594/1688 [=========>....................] - ETA: 6s - loss: 0.0060 - accuracy: 0.9983
602/1688 [=========>....................] - ETA: 6s - loss: 0.0059 - accuracy: 0.9983
611/1688 [=========>....................] - ETA: 6s - loss: 0.0059 - accuracy: 0.9983
619/1688 [==========>...................] - ETA: 6s - loss: 0.0059 - accuracy: 0.9983
627/1688 [==========>...................] - ETA: 6s - loss: 0.0059 - accuracy: 0.9983
635/1688 [==========>...................] - ETA: 6s - loss: 0.0059 - accuracy: 0.9983
643/1688 [==========>...................] - ETA: 6s - loss: 0.0059 - accuracy: 0.9983
652/1688 [==========>...................] - ETA: 6s - loss: 0.0058 - accuracy: 0.9983
660/1688 [==========>...................] - ETA: 6s - loss: 0.0058 - accuracy: 0.9983
669/1688 [==========>...................] - ETA: 6s - loss: 0.0058 - accuracy: 0.9983
677/1688 [===========>..................] - ETA: 6s - loss: 0.0059 - accuracy: 0.9982
686/1688 [===========>..................] - ETA: 6s - loss: 0.0059 - accuracy: 0.9983
694/1688 [===========>..................] - ETA: 6s - loss: 0.0059 - accuracy: 0.9983
702/1688 [===========>..................] - ETA: 6s - loss: 0.0059 - accuracy: 0.9983
711/1688 [===========>..................] - ETA: 6s - loss: 0.0058 - accuracy: 0.9983
720/1688 [===========>..................] - ETA: 6s - loss: 0.0058 - accuracy: 0.9983
729/1688 [===========>..................] - ETA: 6s - loss: 0.0058 - accuracy: 0.9983
738/1688 [============>.................] - ETA: 6s - loss: 0.0058 - accuracy: 0.9983
746/1688 [============>.................] - ETA: 5s - loss: 0.0058 - accuracy: 0.9983
755/1688 [============>.................] - ETA: 5s - loss: 0.0058 - accuracy: 0.9983
764/1688 [============>.................] - ETA: 5s - loss: 0.0058 - accuracy: 0.9984
772/1688 [============>.................] - ETA: 5s - loss: 0.0058 - accuracy: 0.9984
780/1688 [============>.................] - ETA: 5s - loss: 0.0058 - accuracy: 0.9984
789/1688 [=============>................] - ETA: 5s - loss: 0.0057 - accuracy: 0.9984
797/1688 [=============>................] - ETA: 5s - loss: 0.0057 - accuracy: 0.9984
805/1688 [=============>................] - ETA: 5s - loss: 0.0057 - accuracy: 0.9984
813/1688 [=============>................] - ETA: 5s - loss: 0.0057 - accuracy: 0.9984
822/1688 [=============>................] - ETA: 5s - loss: 0.0057 - accuracy: 0.9984
830/1688 [=============>................] - ETA: 5s - loss: 0.0056 - accuracy: 0.9984
839/1688 [=============>................] - ETA: 5s - loss: 0.0056 - accuracy: 0.9984
848/1688 [==============>...............] - ETA: 5s - loss: 0.0056 - accuracy: 0.9985
856/1688 [==============>...............] - ETA: 5s - loss: 0.0055 - accuracy: 0.9985
865/1688 [==============>...............] - ETA: 5s - loss: 0.0055 - accuracy: 0.9985
874/1688 [==============>...............] - ETA: 5s - loss: 0.0056 - accuracy: 0.9985
882/1688 [==============>...............] - ETA: 5s - loss: 0.0056 - accuracy: 0.9985
890/1688 [==============>...............] - ETA: 5s - loss: 0.0056 - accuracy: 0.9985
898/1688 [==============>...............] - ETA: 4s - loss: 0.0056 - accuracy: 0.9985
906/1688 [===============>..............] - ETA: 4s - loss: 0.0055 - accuracy: 0.9985
914/1688 [===============>..............] - ETA: 4s - loss: 0.0056 - accuracy: 0.9985
922/1688 [===============>..............] - ETA: 4s - loss: 0.0055 - accuracy: 0.9985
931/1688 [===============>..............] - ETA: 4s - loss: 0.0055 - accuracy: 0.9985
939/1688 [===============>..............] - ETA: 4s - loss: 0.0055 - accuracy: 0.9985
947/1688 [===============>..............] - ETA: 4s - loss: 0.0055 - accuracy: 0.9985
955/1688 [===============>..............] - ETA: 4s - loss: 0.0055 - accuracy: 0.9985
963/1688 [================>.............] - ETA: 4s - loss: 0.0055 - accuracy: 0.9985
972/1688 [================>.............] - ETA: 4s - loss: 0.0054 - accuracy: 0.9985
980/1688 [================>.............] - ETA: 4s - loss: 0.0054 - accuracy: 0.9985
988/1688 [================>.............] - ETA: 4s - loss: 0.0054 - accuracy: 0.9985
997/1688 [================>.............] - ETA: 4s - loss: 0.0055 - accuracy: 0.9985
1005/1688 [================>.............] - ETA: 4s - loss: 0.0055 - accuracy: 0.9985
1013/1688 [=================>............] - ETA: 4s - loss: 0.0055 - accuracy: 0.9985
1021/1688 [=================>............] - ETA: 4s - loss: 0.0054 - accuracy: 0.9985
1029/1688 [=================>............] - ETA: 4s - loss: 0.0054 - accuracy: 0.9985
1037/1688 [=================>............] - ETA: 4s - loss: 0.0054 - accuracy: 0.9986
1045/1688 [=================>............] - ETA: 4s - loss: 0.0054 - accuracy: 0.9985
1054/1688 [=================>............] - ETA: 4s - loss: 0.0054 - accuracy: 0.9985
1062/1688 [=================>............] - ETA: 3s - loss: 0.0054 - accuracy: 0.9986
1070/1688 [==================>...........] - ETA: 3s - loss: 0.0054 - accuracy: 0.9986
1078/1688 [==================>...........] - ETA: 3s - loss: 0.0053 - accuracy: 0.9986
1086/1688 [==================>...........] - ETA: 3s - loss: 0.0053 - accuracy: 0.9986
1095/1688 [==================>...........] - ETA: 3s - loss: 0.0053 - accuracy: 0.9986
1103/1688 [==================>...........] - ETA: 3s - loss: 0.0053 - accuracy: 0.9986
1111/1688 [==================>...........] - ETA: 3s - loss: 0.0053 - accuracy: 0.9986
1119/1688 [==================>...........] - ETA: 3s - loss: 0.0053 - accuracy: 0.9986
1127/1688 [===================>..........] - ETA: 3s - loss: 0.0053 - accuracy: 0.9986
1135/1688 [===================>..........] - ETA: 3s - loss: 0.0052 - accuracy: 0.9987
1144/1688 [===================>..........] - ETA: 3s - loss: 0.0052 - accuracy: 0.9987
1152/1688 [===================>..........] - ETA: 3s - loss: 0.0052 - accuracy: 0.9987
1161/1688 [===================>..........] - ETA: 3s - loss: 0.0052 - accuracy: 0.9987
1169/1688 [===================>..........] - ETA: 3s - loss: 0.0052 - accuracy: 0.9987
1177/1688 [===================>..........] - ETA: 3s - loss: 0.0052 - accuracy: 0.9986
1185/1688 [====================>.........] - ETA: 3s - loss: 0.0053 - accuracy: 0.9987
1193/1688 [====================>.........] - ETA: 3s - loss: 0.0053 - accuracy: 0.9987
1202/1688 [====================>.........] - ETA: 3s - loss: 0.0053 - accuracy: 0.9987
1211/1688 [====================>.........] - ETA: 3s - loss: 0.0052 - accuracy: 0.9987
1219/1688 [====================>.........] - ETA: 2s - loss: 0.0052 - accuracy: 0.9987
1227/1688 [====================>.........] - ETA: 2s - loss: 0.0052 - accuracy: 0.9987
1236/1688 [====================>.........] - ETA: 2s - loss: 0.0052 - accuracy: 0.9987
1245/1688 [=====================>........] - ETA: 2s - loss: 0.0053 - accuracy: 0.9987
1253/1688 [=====================>........] - ETA: 2s - loss: 0.0053 - accuracy: 0.9987
1261/1688 [=====================>........] - ETA: 2s - loss: 0.0053 - accuracy: 0.9987
1270/1688 [=====================>........] - ETA: 2s - loss: 0.0053 - accuracy: 0.9987
1278/1688 [=====================>........] - ETA: 2s - loss: 0.0053 - accuracy: 0.9987
1286/1688 [=====================>........] - ETA: 2s - loss: 0.0052 - accuracy: 0.9987
1295/1688 [======================>.......] - ETA: 2s - loss: 0.0052 - accuracy: 0.9987
1304/1688 [======================>.......] - ETA: 2s - loss: 0.0052 - accuracy: 0.9987
1312/1688 [======================>.......] - ETA: 2s - loss: 0.0052 - accuracy: 0.9987
1320/1688 [======================>.......] - ETA: 2s - loss: 0.0052 - accuracy: 0.9987
1328/1688 [======================>.......] - ETA: 2s - loss: 0.0052 - accuracy: 0.9987
1337/1688 [======================>.......] - ETA: 2s - loss: 0.0054 - accuracy: 0.9987
1345/1688 [======================>.......] - ETA: 2s - loss: 0.0054 - accuracy: 0.9987
1354/1688 [=======================>......] - ETA: 2s - loss: 0.0053 - accuracy: 0.9987
1362/1688 [=======================>......] - ETA: 2s - loss: 0.0053 - accuracy: 0.9986
1370/1688 [=======================>......] - ETA: 2s - loss: 0.0053 - accuracy: 0.9987
1378/1688 [=======================>......] - ETA: 1s - loss: 0.0053 - accuracy: 0.9986
1387/1688 [=======================>......] - ETA: 1s - loss: 0.0054 - accuracy: 0.9986
1396/1688 [=======================>......] - ETA: 1s - loss: 0.0054 - accuracy: 0.9986
1404/1688 [=======================>......] - ETA: 1s - loss: 0.0054 - accuracy: 0.9986
1413/1688 [========================>.....] - ETA: 1s - loss: 0.0054 - accuracy: 0.9986
1421/1688 [========================>.....] - ETA: 1s - loss: 0.0055 - accuracy: 0.9985
1429/1688 [========================>.....] - ETA: 1s - loss: 0.0055 - accuracy: 0.9986
1438/1688 [========================>.....] - ETA: 1s - loss: 0.0055 - accuracy: 0.9986
1446/1688 [========================>.....] - ETA: 1s - loss: 0.0055 - accuracy: 0.9986
1455/1688 [========================>.....] - ETA: 1s - loss: 0.0055 - accuracy: 0.9986
1464/1688 [=========================>....] - ETA: 1s - loss: 0.0055 - accuracy: 0.9985
1473/1688 [=========================>....] - ETA: 1s - loss: 0.0055 - accuracy: 0.9985
1481/1688 [=========================>....] - ETA: 1s - loss: 0.0055 - accuracy: 0.9985
1490/1688 [=========================>....] - ETA: 1s - loss: 0.0055 - accuracy: 0.9986
1498/1688 [=========================>....] - ETA: 1s - loss: 0.0054 - accuracy: 0.9986
1507/1688 [=========================>....] - ETA: 1s - loss: 0.0054 - accuracy: 0.9986
1515/1688 [=========================>....] - ETA: 1s - loss: 0.0055 - accuracy: 0.9986
1523/1688 [==========================>...] - ETA: 1s - loss: 0.0055 - accuracy: 0.9986
1531/1688 [==========================>...] - ETA: 0s - loss: 0.0055 - accuracy: 0.9986
1540/1688 [==========================>...] - ETA: 0s - loss: 0.0055 - accuracy: 0.9985
1549/1688 [==========================>...] - ETA: 0s - loss: 0.0055 - accuracy: 0.9985
1557/1688 [==========================>...] - ETA: 0s - loss: 0.0055 - accuracy: 0.9985
1565/1688 [==========================>...] - ETA: 0s - loss: 0.0055 - accuracy: 0.9985
1574/1688 [==========================>...] - ETA: 0s - loss: 0.0054 - accuracy: 0.9985
1582/1688 [===========================>..] - ETA: 0s - loss: 0.0054 - accuracy: 0.9985
1591/1688 [===========================>..] - ETA: 0s - loss: 0.0054 - accuracy: 0.9985
1600/1688 [===========================>..] - ETA: 0s - loss: 0.0054 - accuracy: 0.9986
1609/1688 [===========================>..] - ETA: 0s - loss: 0.0054 - accuracy: 0.9986
1617/1688 [===========================>..] - ETA: 0s - loss: 0.0054 - accuracy: 0.9986
1626/1688 [===========================>..] - ETA: 0s - loss: 0.0054 - accuracy: 0.9986
1635/1688 [============================>.] - ETA: 0s - loss: 0.0054 - accuracy: 0.9986
1643/1688 [============================>.] - ETA: 0s - loss: 0.0055 - accuracy: 0.9985
1651/1688 [============================>.] - ETA: 0s - loss: 0.0054 - accuracy: 0.9985
1659/1688 [============================>.] - ETA: 0s - loss: 0.0054 - accuracy: 0.9985
1667/1688 [============================>.] - ETA: 0s - loss: 0.0054 - accuracy: 0.9985
1676/1688 [============================>.] - ETA: 0s - loss: 0.0054 - accuracy: 0.9985
1684/1688 [============================>.] - ETA: 0s - loss: 0.0054 - accuracy: 0.9985
1688/1688 [==============================] - 11s 7ms/step - loss: 0.0054 - accuracy: 0.9985 - val_loss: 0.0528 - val_accuracy: 0.9900
Epoch 4/5
1/1688 [..............................] - ETA: 10s - loss: 0.0130 - accuracy: 1.0000
9/1688 [..............................] - ETA: 10s - loss: 0.0083 - accuracy: 1.0000
18/1688 [..............................] - ETA: 10s - loss: 0.0055 - accuracy: 1.0000
27/1688 [..............................] - ETA: 10s - loss: 0.0057 - accuracy: 0.9988
35/1688 [..............................] - ETA: 10s - loss: 0.0048 - accuracy: 0.9991
43/1688 [..............................] - ETA: 10s - loss: 0.0043 - accuracy: 0.9993
52/1688 [..............................] - ETA: 10s - loss: 0.0039 - accuracy: 0.9994
60/1688 [>.............................] - ETA: 10s - loss: 0.0043 - accuracy: 0.9990
68/1688 [>.............................] - ETA: 10s - loss: 0.0040 - accuracy: 0.9991
77/1688 [>.............................] - ETA: 10s - loss: 0.0038 - accuracy: 0.9992
86/1688 [>.............................] - ETA: 10s - loss: 0.0039 - accuracy: 0.9993
94/1688 [>.............................] - ETA: 10s - loss: 0.0038 - accuracy: 0.9993
102/1688 [>.............................] - ETA: 9s - loss: 0.0039 - accuracy: 0.9994
110/1688 [>.............................] - ETA: 9s - loss: 0.0040 - accuracy: 0.9991
118/1688 [=>............................] - ETA: 9s - loss: 0.0040 - accuracy: 0.9989
126/1688 [=>............................] - ETA: 9s - loss: 0.0039 - accuracy: 0.9990
135/1688 [=>............................] - ETA: 9s - loss: 0.0041 - accuracy: 0.9991
144/1688 [=>............................] - ETA: 9s - loss: 0.0040 - accuracy: 0.9991
152/1688 [=>............................] - ETA: 9s - loss: 0.0040 - accuracy: 0.9992
160/1688 [=>............................] - ETA: 9s - loss: 0.0039 - accuracy: 0.9992
168/1688 [=>............................] - ETA: 9s - loss: 0.0039 - accuracy: 0.9993
176/1688 [==>...........................] - ETA: 9s - loss: 0.0038 - accuracy: 0.9993
184/1688 [==>...........................] - ETA: 9s - loss: 0.0039 - accuracy: 0.9992
193/1688 [==>...........................] - ETA: 9s - loss: 0.0039 - accuracy: 0.9992
202/1688 [==>...........................] - ETA: 9s - loss: 0.0038 - accuracy: 0.9992
210/1688 [==>...........................] - ETA: 9s - loss: 0.0042 - accuracy: 0.9991
218/1688 [==>...........................] - ETA: 9s - loss: 0.0042 - accuracy: 0.9991
227/1688 [===>..........................] - ETA: 9s - loss: 0.0041 - accuracy: 0.9992
235/1688 [===>..........................] - ETA: 9s - loss: 0.0044 - accuracy: 0.9991
244/1688 [===>..........................] - ETA: 9s - loss: 0.0043 - accuracy: 0.9991
252/1688 [===>..........................] - ETA: 9s - loss: 0.0042 - accuracy: 0.9991
260/1688 [===>..........................] - ETA: 8s - loss: 0.0042 - accuracy: 0.9992
269/1688 [===>..........................] - ETA: 8s - loss: 0.0042 - accuracy: 0.9992
277/1688 [===>..........................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9991
285/1688 [====>.........................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9991
294/1688 [====>.........................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9991
302/1688 [====>.........................] - ETA: 8s - loss: 0.0042 - accuracy: 0.9992
311/1688 [====>.........................] - ETA: 8s - loss: 0.0041 - accuracy: 0.9992
320/1688 [====>.........................] - ETA: 8s - loss: 0.0041 - accuracy: 0.9992
329/1688 [====>.........................] - ETA: 8s - loss: 0.0041 - accuracy: 0.9992
338/1688 [=====>........................] - ETA: 8s - loss: 0.0041 - accuracy: 0.9993
346/1688 [=====>........................] - ETA: 8s - loss: 0.0040 - accuracy: 0.9993
355/1688 [=====>........................] - ETA: 8s - loss: 0.0041 - accuracy: 0.9992
363/1688 [=====>........................] - ETA: 8s - loss: 0.0041 - accuracy: 0.9992
371/1688 [=====>........................] - ETA: 8s - loss: 0.0040 - accuracy: 0.9992
380/1688 [=====>........................] - ETA: 8s - loss: 0.0040 - accuracy: 0.9993
388/1688 [=====>........................] - ETA: 8s - loss: 0.0040 - accuracy: 0.9993
396/1688 [======>.......................] - ETA: 8s - loss: 0.0041 - accuracy: 0.9992
404/1688 [======>.......................] - ETA: 8s - loss: 0.0041 - accuracy: 0.9991
413/1688 [======>.......................] - ETA: 8s - loss: 0.0042 - accuracy: 0.9991
421/1688 [======>.......................] - ETA: 7s - loss: 0.0042 - accuracy: 0.9991
429/1688 [======>.......................] - ETA: 7s - loss: 0.0042 - accuracy: 0.9991
437/1688 [======>.......................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9991
445/1688 [======>.......................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9991
453/1688 [=======>......................] - ETA: 7s - loss: 0.0042 - accuracy: 0.9991
462/1688 [=======>......................] - ETA: 7s - loss: 0.0042 - accuracy: 0.9991
471/1688 [=======>......................] - ETA: 7s - loss: 0.0042 - accuracy: 0.9991
480/1688 [=======>......................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9991
488/1688 [=======>......................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9991
496/1688 [=======>......................] - ETA: 7s - loss: 0.0042 - accuracy: 0.9991
505/1688 [=======>......................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9991
513/1688 [========>.....................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9990
522/1688 [========>.....................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9990
531/1688 [========>.....................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9991
539/1688 [========>.....................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9991
547/1688 [========>.....................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9991
556/1688 [========>.....................] - ETA: 7s - loss: 0.0045 - accuracy: 0.9990
564/1688 [=========>....................] - ETA: 7s - loss: 0.0046 - accuracy: 0.9990
572/1688 [=========>....................] - ETA: 7s - loss: 0.0046 - accuracy: 0.9990
580/1688 [=========>....................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9990
588/1688 [=========>....................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9990
596/1688 [=========>....................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9990
604/1688 [=========>....................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9990
613/1688 [=========>....................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9990
621/1688 [==========>...................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9990
629/1688 [==========>...................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9990
637/1688 [==========>...................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9990
645/1688 [==========>...................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9990
653/1688 [==========>...................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9990
661/1688 [==========>...................] - ETA: 6s - loss: 0.0047 - accuracy: 0.9990
669/1688 [==========>...................] - ETA: 6s - loss: 0.0047 - accuracy: 0.9990
678/1688 [===========>..................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9990
686/1688 [===========>..................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9990
695/1688 [===========>..................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9991
703/1688 [===========>..................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9991
711/1688 [===========>..................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9990
720/1688 [===========>..................] - ETA: 6s - loss: 0.0047 - accuracy: 0.9990
729/1688 [===========>..................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9991
737/1688 [============>.................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9991
745/1688 [============>.................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9991
753/1688 [============>.................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9991
761/1688 [============>.................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9991
770/1688 [============>.................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9991
778/1688 [============>.................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9991
786/1688 [============>.................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9991
794/1688 [=============>................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9991
802/1688 [=============>................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9991
811/1688 [=============>................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9991
819/1688 [=============>................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9991
828/1688 [=============>................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9991
837/1688 [=============>................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9991
846/1688 [==============>...............] - ETA: 5s - loss: 0.0046 - accuracy: 0.9991
855/1688 [==============>...............] - ETA: 5s - loss: 0.0046 - accuracy: 0.9990
863/1688 [==============>...............] - ETA: 5s - loss: 0.0046 - accuracy: 0.9991
871/1688 [==============>...............] - ETA: 5s - loss: 0.0046 - accuracy: 0.9991
879/1688 [==============>...............] - ETA: 5s - loss: 0.0046 - accuracy: 0.9990
887/1688 [==============>...............] - ETA: 5s - loss: 0.0046 - accuracy: 0.9990
895/1688 [==============>...............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9990
904/1688 [===============>..............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9990
913/1688 [===============>..............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9990
922/1688 [===============>..............] - ETA: 4s - loss: 0.0047 - accuracy: 0.9990
931/1688 [===============>..............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9990
939/1688 [===============>..............] - ETA: 4s - loss: 0.0047 - accuracy: 0.9990
947/1688 [===============>..............] - ETA: 4s - loss: 0.0047 - accuracy: 0.9990
955/1688 [===============>..............] - ETA: 4s - loss: 0.0047 - accuracy: 0.9990
963/1688 [================>.............] - ETA: 4s - loss: 0.0047 - accuracy: 0.9990
972/1688 [================>.............] - ETA: 4s - loss: 0.0047 - accuracy: 0.9990
980/1688 [================>.............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9989
989/1688 [================>.............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9989
997/1688 [================>.............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9989
1006/1688 [================>.............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9989
1014/1688 [=================>............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9989
1022/1688 [=================>............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9988
1031/1688 [=================>............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9988
1039/1688 [=================>............] - ETA: 4s - loss: 0.0049 - accuracy: 0.9988
1047/1688 [=================>............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9988
1056/1688 [=================>............] - ETA: 3s - loss: 0.0048 - accuracy: 0.9988
1064/1688 [=================>............] - ETA: 3s - loss: 0.0048 - accuracy: 0.9988
1073/1688 [==================>...........] - ETA: 3s - loss: 0.0048 - accuracy: 0.9988
1081/1688 [==================>...........] - ETA: 3s - loss: 0.0048 - accuracy: 0.9988
1090/1688 [==================>...........] - ETA: 3s - loss: 0.0048 - accuracy: 0.9988
1098/1688 [==================>...........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1106/1688 [==================>...........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1114/1688 [==================>...........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1122/1688 [==================>...........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1130/1688 [===================>..........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1139/1688 [===================>..........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1147/1688 [===================>..........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1156/1688 [===================>..........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1164/1688 [===================>..........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1173/1688 [===================>..........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1182/1688 [====================>.........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1191/1688 [====================>.........] - ETA: 3s - loss: 0.0048 - accuracy: 0.9988
1199/1688 [====================>.........] - ETA: 3s - loss: 0.0048 - accuracy: 0.9988
1207/1688 [====================>.........] - ETA: 3s - loss: 0.0048 - accuracy: 0.9988
1216/1688 [====================>.........] - ETA: 2s - loss: 0.0048 - accuracy: 0.9988
1224/1688 [====================>.........] - ETA: 2s - loss: 0.0048 - accuracy: 0.9988
1232/1688 [====================>.........] - ETA: 2s - loss: 0.0048 - accuracy: 0.9988
1241/1688 [=====================>........] - ETA: 2s - loss: 0.0048 - accuracy: 0.9988
1250/1688 [=====================>........] - ETA: 2s - loss: 0.0048 - accuracy: 0.9988
1258/1688 [=====================>........] - ETA: 2s - loss: 0.0048 - accuracy: 0.9988
1266/1688 [=====================>........] - ETA: 2s - loss: 0.0048 - accuracy: 0.9988
1274/1688 [=====================>........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9988
1282/1688 [=====================>........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9989
1291/1688 [=====================>........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9989
1299/1688 [======================>.......] - ETA: 2s - loss: 0.0047 - accuracy: 0.9988
1308/1688 [======================>.......] - ETA: 2s - loss: 0.0047 - accuracy: 0.9989
1317/1688 [======================>.......] - ETA: 2s - loss: 0.0047 - accuracy: 0.9989
1325/1688 [======================>.......] - ETA: 2s - loss: 0.0047 - accuracy: 0.9989
1333/1688 [======================>.......] - ETA: 2s - loss: 0.0047 - accuracy: 0.9989
1342/1688 [======================>.......] - ETA: 2s - loss: 0.0047 - accuracy: 0.9989
1350/1688 [======================>.......] - ETA: 2s - loss: 0.0046 - accuracy: 0.9989
1359/1688 [=======================>......] - ETA: 2s - loss: 0.0046 - accuracy: 0.9989
1368/1688 [=======================>......] - ETA: 2s - loss: 0.0046 - accuracy: 0.9989
1376/1688 [=======================>......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9989
1385/1688 [=======================>......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9989
1393/1688 [=======================>......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9989
1401/1688 [=======================>......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9989
1409/1688 [========================>.....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9989
1417/1688 [========================>.....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9989
1425/1688 [========================>.....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9989
1433/1688 [========================>.....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9989
1442/1688 [========================>.....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9989
1450/1688 [========================>.....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9989
1458/1688 [========================>.....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9989
1466/1688 [=========================>....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9989
1474/1688 [=========================>....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9989
1482/1688 [=========================>....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9989
1490/1688 [=========================>....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9989
1498/1688 [=========================>....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9989
1507/1688 [=========================>....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9989
1515/1688 [=========================>....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9989
1523/1688 [==========================>...] - ETA: 1s - loss: 0.0045 - accuracy: 0.9989
1532/1688 [==========================>...] - ETA: 0s - loss: 0.0045 - accuracy: 0.9989
1541/1688 [==========================>...] - ETA: 0s - loss: 0.0045 - accuracy: 0.9989
1550/1688 [==========================>...] - ETA: 0s - loss: 0.0045 - accuracy: 0.9989
1558/1688 [==========================>...] - ETA: 0s - loss: 0.0045 - accuracy: 0.9989
1567/1688 [==========================>...] - ETA: 0s - loss: 0.0045 - accuracy: 0.9989
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0045 - accuracy: 0.9989
1585/1688 [===========================>..] - ETA: 0s - loss: 0.0045 - accuracy: 0.9990
1594/1688 [===========================>..] - ETA: 0s - loss: 0.0045 - accuracy: 0.9989
1602/1688 [===========================>..] - ETA: 0s - loss: 0.0045 - accuracy: 0.9989
1611/1688 [===========================>..] - ETA: 0s - loss: 0.0045 - accuracy: 0.9989
1620/1688 [===========================>..] - ETA: 0s - loss: 0.0045 - accuracy: 0.9989
1628/1688 [===========================>..] - ETA: 0s - loss: 0.0045 - accuracy: 0.9989
1637/1688 [============================>.] - ETA: 0s - loss: 0.0045 - accuracy: 0.9990
1646/1688 [============================>.] - ETA: 0s - loss: 0.0045 - accuracy: 0.9990
1655/1688 [============================>.] - ETA: 0s - loss: 0.0046 - accuracy: 0.9989
1663/1688 [============================>.] - ETA: 0s - loss: 0.0046 - accuracy: 0.9989
1671/1688 [============================>.] - ETA: 0s - loss: 0.0046 - accuracy: 0.9989
1679/1688 [============================>.] - ETA: 0s - loss: 0.0046 - accuracy: 0.9989
1688/1688 [==============================] - ETA: 0s - loss: 0.0045 - accuracy: 0.9989
1688/1688 [==============================] - 11s 7ms/step - loss: 0.0045 - accuracy: 0.9989 - val_loss: 0.0515 - val_accuracy: 0.9900
Epoch 5/5
1/1688 [..............................] - ETA: 10s - loss: 5.4295e-04 - accuracy: 1.0000
10/1688 [..............................] - ETA: 10s - loss: 0.0018 - accuracy: 1.0000
18/1688 [..............................] - ETA: 10s - loss: 0.0024 - accuracy: 1.0000
26/1688 [..............................] - ETA: 10s - loss: 0.0049 - accuracy: 0.9988
35/1688 [..............................] - ETA: 10s - loss: 0.0041 - accuracy: 0.9991
44/1688 [..............................] - ETA: 10s - loss: 0.0035 - accuracy: 0.9993
52/1688 [..............................] - ETA: 10s - loss: 0.0041 - accuracy: 0.9994
60/1688 [>.............................] - ETA: 10s - loss: 0.0042 - accuracy: 0.9995
69/1688 [>.............................] - ETA: 10s - loss: 0.0039 - accuracy: 0.9995
77/1688 [>.............................] - ETA: 10s - loss: 0.0037 - accuracy: 0.9996
85/1688 [>.............................] - ETA: 10s - loss: 0.0042 - accuracy: 0.9996
93/1688 [>.............................] - ETA: 10s - loss: 0.0041 - accuracy: 0.9997
102/1688 [>.............................] - ETA: 10s - loss: 0.0042 - accuracy: 0.9994
111/1688 [>.............................] - ETA: 9s - loss: 0.0040 - accuracy: 0.9994
119/1688 [=>............................] - ETA: 9s - loss: 0.0041 - accuracy: 0.9992
128/1688 [=>............................] - ETA: 9s - loss: 0.0041 - accuracy: 0.9993
136/1688 [=>............................] - ETA: 9s - loss: 0.0045 - accuracy: 0.9991
144/1688 [=>............................] - ETA: 9s - loss: 0.0044 - accuracy: 0.9991
152/1688 [=>............................] - ETA: 9s - loss: 0.0045 - accuracy: 0.9992
160/1688 [=>............................] - ETA: 9s - loss: 0.0044 - accuracy: 0.9992
169/1688 [==>...........................] - ETA: 9s - loss: 0.0043 - accuracy: 0.9993
177/1688 [==>...........................] - ETA: 9s - loss: 0.0043 - accuracy: 0.9993
186/1688 [==>...........................] - ETA: 9s - loss: 0.0042 - accuracy: 0.9993
194/1688 [==>...........................] - ETA: 9s - loss: 0.0041 - accuracy: 0.9994
202/1688 [==>...........................] - ETA: 9s - loss: 0.0041 - accuracy: 0.9994
210/1688 [==>...........................] - ETA: 9s - loss: 0.0040 - accuracy: 0.9994
218/1688 [==>...........................] - ETA: 9s - loss: 0.0039 - accuracy: 0.9994
226/1688 [===>..........................] - ETA: 9s - loss: 0.0039 - accuracy: 0.9994
234/1688 [===>..........................] - ETA: 9s - loss: 0.0040 - accuracy: 0.9993
242/1688 [===>..........................] - ETA: 9s - loss: 0.0040 - accuracy: 0.9994
251/1688 [===>..........................] - ETA: 9s - loss: 0.0040 - accuracy: 0.9994
259/1688 [===>..........................] - ETA: 9s - loss: 0.0040 - accuracy: 0.9994
268/1688 [===>..........................] - ETA: 8s - loss: 0.0039 - accuracy: 0.9994
276/1688 [===>..........................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9993
285/1688 [====>.........................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9993
293/1688 [====>.........................] - ETA: 8s - loss: 0.0044 - accuracy: 0.9993
301/1688 [====>.........................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9993
309/1688 [====>.........................] - ETA: 8s - loss: 0.0042 - accuracy: 0.9993
317/1688 [====>.........................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9992
325/1688 [====>.........................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9992
333/1688 [====>.........................] - ETA: 8s - loss: 0.0044 - accuracy: 0.9992
341/1688 [=====>........................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9992
349/1688 [=====>........................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9992
358/1688 [=====>........................] - ETA: 8s - loss: 0.0042 - accuracy: 0.9992
367/1688 [=====>........................] - ETA: 8s - loss: 0.0042 - accuracy: 0.9991
375/1688 [=====>........................] - ETA: 8s - loss: 0.0042 - accuracy: 0.9992
384/1688 [=====>........................] - ETA: 8s - loss: 0.0042 - accuracy: 0.9992
392/1688 [=====>........................] - ETA: 8s - loss: 0.0041 - accuracy: 0.9992
401/1688 [======>.......................] - ETA: 8s - loss: 0.0041 - accuracy: 0.9992
410/1688 [======>.......................] - ETA: 8s - loss: 0.0041 - accuracy: 0.9992
419/1688 [======>.......................] - ETA: 7s - loss: 0.0041 - accuracy: 0.9993
427/1688 [======>.......................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9993
436/1688 [======>.......................] - ETA: 7s - loss: 0.0041 - accuracy: 0.9992
444/1688 [======>.......................] - ETA: 7s - loss: 0.0041 - accuracy: 0.9992
452/1688 [=======>......................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9992
461/1688 [=======>......................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9993
469/1688 [=======>......................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9992
477/1688 [=======>......................] - ETA: 7s - loss: 0.0042 - accuracy: 0.9991
486/1688 [=======>......................] - ETA: 7s - loss: 0.0042 - accuracy: 0.9991
494/1688 [=======>......................] - ETA: 7s - loss: 0.0041 - accuracy: 0.9991
503/1688 [=======>......................] - ETA: 7s - loss: 0.0042 - accuracy: 0.9991
512/1688 [========>.....................] - ETA: 7s - loss: 0.0041 - accuracy: 0.9991
520/1688 [========>.....................] - ETA: 7s - loss: 0.0041 - accuracy: 0.9991
528/1688 [========>.....................] - ETA: 7s - loss: 0.0041 - accuracy: 0.9991
537/1688 [========>.....................] - ETA: 7s - loss: 0.0041 - accuracy: 0.9991
545/1688 [========>.....................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9991
554/1688 [========>.....................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9992
563/1688 [=========>....................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9992
572/1688 [=========>....................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9992
580/1688 [=========>....................] - ETA: 6s - loss: 0.0039 - accuracy: 0.9992
588/1688 [=========>....................] - ETA: 6s - loss: 0.0039 - accuracy: 0.9992
597/1688 [=========>....................] - ETA: 6s - loss: 0.0039 - accuracy: 0.9992
605/1688 [=========>....................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9992
613/1688 [=========>....................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9992
621/1688 [==========>...................] - ETA: 6s - loss: 0.0039 - accuracy: 0.9992
630/1688 [==========>...................] - ETA: 6s - loss: 0.0039 - accuracy: 0.9992
638/1688 [==========>...................] - ETA: 6s - loss: 0.0039 - accuracy: 0.9992
647/1688 [==========>...................] - ETA: 6s - loss: 0.0039 - accuracy: 0.9992
655/1688 [==========>...................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9992
664/1688 [==========>...................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9992
673/1688 [==========>...................] - ETA: 6s - loss: 0.0039 - accuracy: 0.9992
681/1688 [===========>..................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9992
689/1688 [===========>..................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9992
697/1688 [===========>..................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9992
706/1688 [===========>..................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9992
714/1688 [===========>..................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9992
722/1688 [===========>..................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9992
731/1688 [===========>..................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9992
739/1688 [============>.................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9992
747/1688 [============>.................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9992
755/1688 [============>.................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9992
764/1688 [============>.................] - ETA: 5s - loss: 0.0038 - accuracy: 0.9992
772/1688 [============>.................] - ETA: 5s - loss: 0.0038 - accuracy: 0.9992
780/1688 [============>.................] - ETA: 5s - loss: 0.0038 - accuracy: 0.9992
789/1688 [=============>................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9992
797/1688 [=============>................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9992
805/1688 [=============>................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9991
813/1688 [=============>................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9991
822/1688 [=============>................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9991
831/1688 [=============>................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9991
840/1688 [=============>................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9991
849/1688 [==============>...............] - ETA: 5s - loss: 0.0039 - accuracy: 0.9992
858/1688 [==============>...............] - ETA: 5s - loss: 0.0039 - accuracy: 0.9992
866/1688 [==============>...............] - ETA: 5s - loss: 0.0039 - accuracy: 0.9992
874/1688 [==============>...............] - ETA: 5s - loss: 0.0039 - accuracy: 0.9992
882/1688 [==============>...............] - ETA: 5s - loss: 0.0039 - accuracy: 0.9991
891/1688 [==============>...............] - ETA: 5s - loss: 0.0039 - accuracy: 0.9992
899/1688 [==============>...............] - ETA: 4s - loss: 0.0039 - accuracy: 0.9992
908/1688 [===============>..............] - ETA: 4s - loss: 0.0039 - accuracy: 0.9992
917/1688 [===============>..............] - ETA: 4s - loss: 0.0039 - accuracy: 0.9992
925/1688 [===============>..............] - ETA: 4s - loss: 0.0038 - accuracy: 0.9992
933/1688 [===============>..............] - ETA: 4s - loss: 0.0038 - accuracy: 0.9992
942/1688 [===============>..............] - ETA: 4s - loss: 0.0038 - accuracy: 0.9992
950/1688 [===============>..............] - ETA: 4s - loss: 0.0038 - accuracy: 0.9992
959/1688 [================>.............] - ETA: 4s - loss: 0.0038 - accuracy: 0.9992
967/1688 [================>.............] - ETA: 4s - loss: 0.0038 - accuracy: 0.9992
975/1688 [================>.............] - ETA: 4s - loss: 0.0038 - accuracy: 0.9992
984/1688 [================>.............] - ETA: 4s - loss: 0.0038 - accuracy: 0.9992
992/1688 [================>.............] - ETA: 4s - loss: 0.0038 - accuracy: 0.9992
1000/1688 [================>.............] - ETA: 4s - loss: 0.0038 - accuracy: 0.9992
1008/1688 [================>.............] - ETA: 4s - loss: 0.0038 - accuracy: 0.9992
1016/1688 [=================>............] - ETA: 4s - loss: 0.0038 - accuracy: 0.9992
1024/1688 [=================>............] - ETA: 4s - loss: 0.0038 - accuracy: 0.9992
1032/1688 [=================>............] - ETA: 4s - loss: 0.0038 - accuracy: 0.9992
1040/1688 [=================>............] - ETA: 4s - loss: 0.0038 - accuracy: 0.9992
1048/1688 [=================>............] - ETA: 4s - loss: 0.0039 - accuracy: 0.9992
1057/1688 [=================>............] - ETA: 3s - loss: 0.0038 - accuracy: 0.9992
1066/1688 [=================>............] - ETA: 3s - loss: 0.0038 - accuracy: 0.9992
1074/1688 [==================>...........] - ETA: 3s - loss: 0.0038 - accuracy: 0.9992
1082/1688 [==================>...........] - ETA: 3s - loss: 0.0038 - accuracy: 0.9992
1090/1688 [==================>...........] - ETA: 3s - loss: 0.0038 - accuracy: 0.9992
1098/1688 [==================>...........] - ETA: 3s - loss: 0.0039 - accuracy: 0.9992
1107/1688 [==================>...........] - ETA: 3s - loss: 0.0038 - accuracy: 0.9992
1115/1688 [==================>...........] - ETA: 3s - loss: 0.0038 - accuracy: 0.9992
1123/1688 [==================>...........] - ETA: 3s - loss: 0.0038 - accuracy: 0.9992
1132/1688 [===================>..........] - ETA: 3s - loss: 0.0039 - accuracy: 0.9992
1141/1688 [===================>..........] - ETA: 3s - loss: 0.0038 - accuracy: 0.9992
1150/1688 [===================>..........] - ETA: 3s - loss: 0.0038 - accuracy: 0.9992
1158/1688 [===================>..........] - ETA: 3s - loss: 0.0038 - accuracy: 0.9992
1166/1688 [===================>..........] - ETA: 3s - loss: 0.0038 - accuracy: 0.9992
1174/1688 [===================>..........] - ETA: 3s - loss: 0.0038 - accuracy: 0.9992
1182/1688 [====================>.........] - ETA: 3s - loss: 0.0038 - accuracy: 0.9992
1191/1688 [====================>.........] - ETA: 3s - loss: 0.0038 - accuracy: 0.9992
1199/1688 [====================>.........] - ETA: 3s - loss: 0.0038 - accuracy: 0.9992
1207/1688 [====================>.........] - ETA: 3s - loss: 0.0038 - accuracy: 0.9992
1216/1688 [====================>.........] - ETA: 2s - loss: 0.0038 - accuracy: 0.9992
1224/1688 [====================>.........] - ETA: 2s - loss: 0.0038 - accuracy: 0.9992
1233/1688 [====================>.........] - ETA: 2s - loss: 0.0038 - accuracy: 0.9992
1241/1688 [=====================>........] - ETA: 2s - loss: 0.0038 - accuracy: 0.9992
1250/1688 [=====================>........] - ETA: 2s - loss: 0.0038 - accuracy: 0.9992
1259/1688 [=====================>........] - ETA: 2s - loss: 0.0038 - accuracy: 0.9992
1268/1688 [=====================>........] - ETA: 2s - loss: 0.0038 - accuracy: 0.9992
1277/1688 [=====================>........] - ETA: 2s - loss: 0.0038 - accuracy: 0.9992
1286/1688 [=====================>........] - ETA: 2s - loss: 0.0038 - accuracy: 0.9992
1295/1688 [======================>.......] - ETA: 2s - loss: 0.0038 - accuracy: 0.9992
1304/1688 [======================>.......] - ETA: 2s - loss: 0.0038 - accuracy: 0.9992
1312/1688 [======================>.......] - ETA: 2s - loss: 0.0038 - accuracy: 0.9992
1320/1688 [======================>.......] - ETA: 2s - loss: 0.0038 - accuracy: 0.9992
1328/1688 [======================>.......] - ETA: 2s - loss: 0.0038 - accuracy: 0.9992
1336/1688 [======================>.......] - ETA: 2s - loss: 0.0038 - accuracy: 0.9992
1344/1688 [======================>.......] - ETA: 2s - loss: 0.0039 - accuracy: 0.9992
1353/1688 [=======================>......] - ETA: 2s - loss: 0.0039 - accuracy: 0.9992
1362/1688 [=======================>......] - ETA: 2s - loss: 0.0039 - accuracy: 0.9992
1370/1688 [=======================>......] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1378/1688 [=======================>......] - ETA: 1s - loss: 0.0038 - accuracy: 0.9992
1387/1688 [=======================>......] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1396/1688 [=======================>......] - ETA: 1s - loss: 0.0038 - accuracy: 0.9992
1405/1688 [=======================>......] - ETA: 1s - loss: 0.0038 - accuracy: 0.9992
1414/1688 [========================>.....] - ETA: 1s - loss: 0.0038 - accuracy: 0.9992
1423/1688 [========================>.....] - ETA: 1s - loss: 0.0038 - accuracy: 0.9992
1432/1688 [========================>.....] - ETA: 1s - loss: 0.0038 - accuracy: 0.9992
1440/1688 [========================>.....] - ETA: 1s - loss: 0.0038 - accuracy: 0.9992
1449/1688 [========================>.....] - ETA: 1s - loss: 0.0038 - accuracy: 0.9992
1457/1688 [========================>.....] - ETA: 1s - loss: 0.0038 - accuracy: 0.9992
1466/1688 [=========================>....] - ETA: 1s - loss: 0.0038 - accuracy: 0.9992
1475/1688 [=========================>....] - ETA: 1s - loss: 0.0038 - accuracy: 0.9992
1484/1688 [=========================>....] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1493/1688 [=========================>....] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1502/1688 [=========================>....] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1511/1688 [=========================>....] - ETA: 1s - loss: 0.0038 - accuracy: 0.9992
1520/1688 [==========================>...] - ETA: 1s - loss: 0.0039 - accuracy: 0.9992
1529/1688 [==========================>...] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1538/1688 [==========================>...] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1547/1688 [==========================>...] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1555/1688 [==========================>...] - ETA: 0s - loss: 0.0038 - accuracy: 0.9992
1564/1688 [==========================>...] - ETA: 0s - loss: 0.0038 - accuracy: 0.9992
1573/1688 [==========================>...] - ETA: 0s - loss: 0.0038 - accuracy: 0.9992
1582/1688 [===========================>..] - ETA: 0s - loss: 0.0038 - accuracy: 0.9992
1590/1688 [===========================>..] - ETA: 0s - loss: 0.0038 - accuracy: 0.9992
1598/1688 [===========================>..] - ETA: 0s - loss: 0.0038 - accuracy: 0.9992
1606/1688 [===========================>..] - ETA: 0s - loss: 0.0038 - accuracy: 0.9992
1614/1688 [===========================>..] - ETA: 0s - loss: 0.0038 - accuracy: 0.9992
1622/1688 [===========================>..] - ETA: 0s - loss: 0.0038 - accuracy: 0.9992
1630/1688 [===========================>..] - ETA: 0s - loss: 0.0038 - accuracy: 0.9992
1638/1688 [============================>.] - ETA: 0s - loss: 0.0038 - accuracy: 0.9992
1647/1688 [============================>.] - ETA: 0s - loss: 0.0038 - accuracy: 0.9992
1655/1688 [============================>.] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1664/1688 [============================>.] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1672/1688 [============================>.] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1681/1688 [============================>.] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1688/1688 [==============================] - 11s 7ms/step - loss: 0.0039 - accuracy: 0.9992 - val_loss: 0.0531 - val_accuracy: 0.9902
<keras.src.callbacks.History object at 0x717df7f6e6d0>
score = model_quantized.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after fine tuning:', score)
Test accuracy after fine tuning: 0.9871000051498413
3. Convert
3.1 Convert to Akida model
When the quantized model produces satisfactory performance, it can be converted to the native Akida format. The convert function returns a model in Akida format ready for inference.
As with Keras, the summary() method provides a textual representation of the Akida model.
from cnn2snn import convert
model_akida = convert(model_quantized)
model_akida.summary()
Model Summary
______________________________________________
Input shape Output shape Sequences Layers
==============================================
[28, 28, 1] [1, 1, 10] 1 5
______________________________________________
__________________________________________________________________
Layer (type) Output shape Kernel shape
=============== SW/conv2d-dequantizer_2 (Software) ===============
conv2d (InputConv2D) [13, 13, 32] (3, 3, 1, 32)
__________________________________________________________________
depthwise_conv2d (DepthwiseConv2D) [7, 7, 32] (3, 3, 32, 1)
__________________________________________________________________
conv2d_1 (Conv2D) [7, 7, 64] (1, 1, 32, 64)
__________________________________________________________________
dense (Dense1D) [1, 1, 10] (3136, 10)
__________________________________________________________________
dequantizer_2 (Dequantizer) [1, 1, 10] N/A
__________________________________________________________________
3.2. Check performance
accuracy = model_akida.evaluate(x_test, y_test)
print('Test accuracy after conversion:', accuracy)
# For non-regression purposes
assert accuracy > 0.96
Test accuracy after conversion: 0.9811000227928162
3.3 Show predictions for a single image
Display one of the test images, such as the first image in the dataset from above, to visualize the output of the model.
# Test a single example
sample_image = 0
image = x_test[sample_image]
outputs = model_akida.predict(image.reshape(1, 28, 28, 1))
print('Input Label: %i' % y_test[sample_image])
f, axarr = plt.subplots(1, 2)
axarr[0].imshow(x_test[sample_image].reshape((28, 28)), cmap=cm.Greys_r)
axarr[0].set_title('Class %d' % y_test[sample_image])
axarr[1].bar(range(10), outputs.squeeze())
axarr[1].set_xticks(range(10))
plt.show()
print(outputs.squeeze())
Input Label: 7
[-16.761532 -10.645415 -3.1135828 1.6053915 -17.277428 -4.339548
-30.717518 9.208757 -6.4874353 -1.8374195]
Consider the output from the model above. As is typical in backprop-trained models, the final layer is a Dense layer with one neuron for each of the 10 classes in the dataset. The goal of training is to maximize the response of the neuron corresponding to the label of each training sample while minimizing the responses of the other neurons.
In the bar chart above, you can see the outputs from all 10 neurons. It is easy to see that neuron 7 responds much more strongly than the others. The first sample is indeed a number 7.
Total running time of the script: (2 minutes 23.886 seconds)