Note
Go to the end to download the full example code.
Global Akida workflow
Using the MNIST dataset, this example shows the definition and training of a keras floating point model, its quantization to 8-bit with the help of calibration, its quantization to 4-bit using QAT and its conversion to Akida. Notice that the performance of the original keras floating point model is maintained throughout the Akida flow. Please refer to the Akida user guide for further information.
Note
Please refer to the TensorFlow tf.keras.models module for model creation/import details and the TensorFlow Guide for TensorFlow usage.
The MNIST example below is light enough so that a GPU is not needed for training.
Global Akida workflow
1. Create and train
1.1. Load and reshape MNIST dataset
import numpy as np
import matplotlib.cm as cm
import matplotlib.pyplot as plt
from keras.datasets import mnist
# Load MNIST dataset
(x_train, y_train), (x_test, y_test) = mnist.load_data()
# Add a channels dimension to the image sets as Akida expects 4-D inputs (corresponding to
# (num_samples, width, height, channels). Note: MNIST is a grayscale dataset and is unusual
# in this respect - most image data already includes a channel dimension, and this step will
# not be necessary.
x_train = np.expand_dims(x_train, -1)
x_test = np.expand_dims(x_test, -1)
# Display a few images from the test set
f, axarr = plt.subplots(1, 4)
for i in range(0, 4):
axarr[i].imshow(x_test[i].reshape((28, 28)), cmap=cm.Greys_r)
axarr[i].set_title('Class %d' % y_test[i])
plt.show()
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
8192/11490434 [..............................] - ETA: 0s
16384/11490434 [..............................] - ETA: 36s
49152/11490434 [..............................] - ETA: 32s
81920/11490434 [..............................] - ETA: 31s
147456/11490434 [..............................] - ETA: 22s
221184/11490434 [..............................] - ETA: 17s
335872/11490434 [..............................] - ETA: 13s
475136/11490434 [>.............................] - ETA: 10s
745472/11490434 [>.............................] - ETA: 7s
1114112/11490434 [=>............................] - ETA: 5s
1720320/11490434 [===>..........................] - ETA: 3s
2629632/11490434 [=====>........................] - ETA: 2s
3874816/11490434 [=========>....................] - ETA: 1s
5275648/11490434 [============>.................] - ETA: 0s
6586368/11490434 [================>.............] - ETA: 0s
7831552/11490434 [===================>..........] - ETA: 0s
9060352/11490434 [======================>.......] - ETA: 0s
10305536/11490434 [=========================>....] - ETA: 0s
11042816/11490434 [===========================>..] - ETA: 0s
11490434/11490434 [==============================] - 1s 0us/step
1.2. Model definition
Note that at this stage, there is nothing specific to the Akida IP. The model constructed below, as inspired by this example, is a completely standard Keras CNN model.
import keras
model_keras = keras.models.Sequential([
keras.layers.Rescaling(1. / 255, input_shape=(28, 28, 1)),
keras.layers.Conv2D(filters=32, kernel_size=3, strides=2),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
# Separable layer
keras.layers.DepthwiseConv2D(kernel_size=3, padding='same', strides=2),
keras.layers.Conv2D(filters=64, kernel_size=1, padding='same'),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
keras.layers.Flatten(),
keras.layers.Dense(10)
], 'mnistnet')
model_keras.summary()
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling (Rescaling) (None, 28, 28, 1) 0
conv2d (Conv2D) (None, 13, 13, 32) 320
batch_normalization (Batch (None, 13, 13, 32) 128
Normalization)
re_lu (ReLU) (None, 13, 13, 32) 0
depthwise_conv2d (Depthwis (None, 7, 7, 32) 320
eConv2D)
conv2d_1 (Conv2D) (None, 7, 7, 64) 2112
batch_normalization_1 (Bat (None, 7, 7, 64) 256
chNormalization)
re_lu_1 (ReLU) (None, 7, 7, 64) 0
flatten (Flatten) (None, 3136) 0
dense (Dense) (None, 10) 31370
=================================================================
Total params: 34506 (134.79 KB)
Trainable params: 34314 (134.04 KB)
Non-trainable params: 192 (768.00 Byte)
_________________________________________________________________
1.3. Model training
Given the model created above, train the model and check its accuracy. The model should achieve a test accuracy over 98% after 10 epochs.
from keras.optimizers import Adam
model_keras.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-3),
metrics=['accuracy'])
_ = model_keras.fit(x_train, y_train, epochs=10, validation_split=0.1)
Epoch 1/10
1/1688 [..............................] - ETA: 1:03:08 - loss: 3.0761 - accuracy: 0.0938
24/1688 [..............................] - ETA: 3s - loss: 1.4955 - accuracy: 0.5234
47/1688 [..............................] - ETA: 3s - loss: 1.0861 - accuracy: 0.6556
70/1688 [>.............................] - ETA: 3s - loss: 0.8868 - accuracy: 0.7219
93/1688 [>.............................] - ETA: 3s - loss: 0.7584 - accuracy: 0.7641
115/1688 [=>............................] - ETA: 3s - loss: 0.6801 - accuracy: 0.7878
136/1688 [=>............................] - ETA: 3s - loss: 0.6298 - accuracy: 0.8038
158/1688 [=>............................] - ETA: 3s - loss: 0.5827 - accuracy: 0.8192
180/1688 [==>...........................] - ETA: 3s - loss: 0.5451 - accuracy: 0.8326
203/1688 [==>...........................] - ETA: 3s - loss: 0.5108 - accuracy: 0.8431
226/1688 [===>..........................] - ETA: 3s - loss: 0.4851 - accuracy: 0.8508
249/1688 [===>..........................] - ETA: 3s - loss: 0.4581 - accuracy: 0.8592
272/1688 [===>..........................] - ETA: 3s - loss: 0.4379 - accuracy: 0.8649
295/1688 [====>.........................] - ETA: 3s - loss: 0.4211 - accuracy: 0.8703
318/1688 [====>.........................] - ETA: 3s - loss: 0.4105 - accuracy: 0.8740
341/1688 [=====>........................] - ETA: 3s - loss: 0.3972 - accuracy: 0.8785
364/1688 [=====>........................] - ETA: 2s - loss: 0.3828 - accuracy: 0.8824
387/1688 [=====>........................] - ETA: 2s - loss: 0.3703 - accuracy: 0.8864
410/1688 [======>.......................] - ETA: 2s - loss: 0.3579 - accuracy: 0.8909
433/1688 [======>.......................] - ETA: 2s - loss: 0.3486 - accuracy: 0.8943
456/1688 [=======>......................] - ETA: 2s - loss: 0.3401 - accuracy: 0.8969
479/1688 [=======>......................] - ETA: 2s - loss: 0.3319 - accuracy: 0.8991
502/1688 [=======>......................] - ETA: 2s - loss: 0.3232 - accuracy: 0.9018
525/1688 [========>.....................] - ETA: 2s - loss: 0.3171 - accuracy: 0.9034
548/1688 [========>.....................] - ETA: 2s - loss: 0.3111 - accuracy: 0.9051
571/1688 [=========>....................] - ETA: 2s - loss: 0.3037 - accuracy: 0.9071
594/1688 [=========>....................] - ETA: 2s - loss: 0.2969 - accuracy: 0.9095
617/1688 [=========>....................] - ETA: 2s - loss: 0.2901 - accuracy: 0.9116
640/1688 [==========>...................] - ETA: 2s - loss: 0.2854 - accuracy: 0.9133
661/1688 [==========>...................] - ETA: 2s - loss: 0.2803 - accuracy: 0.9149
682/1688 [===========>..................] - ETA: 2s - loss: 0.2753 - accuracy: 0.9165
704/1688 [===========>..................] - ETA: 2s - loss: 0.2706 - accuracy: 0.9178
726/1688 [===========>..................] - ETA: 2s - loss: 0.2668 - accuracy: 0.9190
747/1688 [============>.................] - ETA: 2s - loss: 0.2626 - accuracy: 0.9203
769/1688 [============>.................] - ETA: 2s - loss: 0.2584 - accuracy: 0.9215
791/1688 [=============>................] - ETA: 2s - loss: 0.2543 - accuracy: 0.9230
813/1688 [=============>................] - ETA: 1s - loss: 0.2508 - accuracy: 0.9242
835/1688 [=============>................] - ETA: 1s - loss: 0.2475 - accuracy: 0.9251
857/1688 [==============>...............] - ETA: 1s - loss: 0.2450 - accuracy: 0.9259
879/1688 [==============>...............] - ETA: 1s - loss: 0.2423 - accuracy: 0.9266
900/1688 [==============>...............] - ETA: 1s - loss: 0.2397 - accuracy: 0.9272
921/1688 [===============>..............] - ETA: 1s - loss: 0.2372 - accuracy: 0.9280
942/1688 [===============>..............] - ETA: 1s - loss: 0.2346 - accuracy: 0.9286
964/1688 [================>.............] - ETA: 1s - loss: 0.2319 - accuracy: 0.9294
985/1688 [================>.............] - ETA: 1s - loss: 0.2293 - accuracy: 0.9301
1006/1688 [================>.............] - ETA: 1s - loss: 0.2271 - accuracy: 0.9308
1027/1688 [=================>............] - ETA: 1s - loss: 0.2256 - accuracy: 0.9315
1048/1688 [=================>............] - ETA: 1s - loss: 0.2236 - accuracy: 0.9319
1069/1688 [=================>............] - ETA: 1s - loss: 0.2219 - accuracy: 0.9325
1090/1688 [==================>...........] - ETA: 1s - loss: 0.2193 - accuracy: 0.9333
1111/1688 [==================>...........] - ETA: 1s - loss: 0.2174 - accuracy: 0.9339
1132/1688 [===================>..........] - ETA: 1s - loss: 0.2154 - accuracy: 0.9343
1154/1688 [===================>..........] - ETA: 1s - loss: 0.2137 - accuracy: 0.9348
1176/1688 [===================>..........] - ETA: 1s - loss: 0.2114 - accuracy: 0.9355
1197/1688 [====================>.........] - ETA: 1s - loss: 0.2095 - accuracy: 0.9361
1218/1688 [====================>.........] - ETA: 1s - loss: 0.2079 - accuracy: 0.9365
1239/1688 [=====================>........] - ETA: 1s - loss: 0.2062 - accuracy: 0.9370
1259/1688 [=====================>........] - ETA: 0s - loss: 0.2041 - accuracy: 0.9377
1279/1688 [=====================>........] - ETA: 0s - loss: 0.2023 - accuracy: 0.9382
1299/1688 [======================>.......] - ETA: 0s - loss: 0.2009 - accuracy: 0.9387
1318/1688 [======================>.......] - ETA: 0s - loss: 0.1993 - accuracy: 0.9391
1338/1688 [======================>.......] - ETA: 0s - loss: 0.1983 - accuracy: 0.9395
1358/1688 [=======================>......] - ETA: 0s - loss: 0.1966 - accuracy: 0.9401
1378/1688 [=======================>......] - ETA: 0s - loss: 0.1952 - accuracy: 0.9404
1398/1688 [=======================>......] - ETA: 0s - loss: 0.1943 - accuracy: 0.9407
1418/1688 [========================>.....] - ETA: 0s - loss: 0.1928 - accuracy: 0.9411
1438/1688 [========================>.....] - ETA: 0s - loss: 0.1915 - accuracy: 0.9416
1458/1688 [========================>.....] - ETA: 0s - loss: 0.1905 - accuracy: 0.9420
1478/1688 [=========================>....] - ETA: 0s - loss: 0.1893 - accuracy: 0.9424
1498/1688 [=========================>....] - ETA: 0s - loss: 0.1877 - accuracy: 0.9429
1518/1688 [=========================>....] - ETA: 0s - loss: 0.1863 - accuracy: 0.9432
1538/1688 [==========================>...] - ETA: 0s - loss: 0.1856 - accuracy: 0.9435
1558/1688 [==========================>...] - ETA: 0s - loss: 0.1849 - accuracy: 0.9437
1578/1688 [===========================>..] - ETA: 0s - loss: 0.1836 - accuracy: 0.9440
1598/1688 [===========================>..] - ETA: 0s - loss: 0.1826 - accuracy: 0.9444
1618/1688 [===========================>..] - ETA: 0s - loss: 0.1813 - accuracy: 0.9447
1638/1688 [============================>.] - ETA: 0s - loss: 0.1806 - accuracy: 0.9451
1658/1688 [============================>.] - ETA: 0s - loss: 0.1793 - accuracy: 0.9455
1678/1688 [============================>.] - ETA: 0s - loss: 0.1784 - accuracy: 0.9458
1688/1688 [==============================] - ETA: 0s - loss: 0.1778 - accuracy: 0.9460
1688/1688 [==============================] - 7s 3ms/step - loss: 0.1778 - accuracy: 0.9460 - val_loss: 0.0858 - val_accuracy: 0.9763
Epoch 2/10
1/1688 [..............................] - ETA: 3s - loss: 0.0485 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0752 - accuracy: 0.9779
46/1688 [..............................] - ETA: 3s - loss: 0.0887 - accuracy: 0.9749
69/1688 [>.............................] - ETA: 3s - loss: 0.0808 - accuracy: 0.9755
92/1688 [>.............................] - ETA: 3s - loss: 0.0752 - accuracy: 0.9789
115/1688 [=>............................] - ETA: 3s - loss: 0.0770 - accuracy: 0.9780
137/1688 [=>............................] - ETA: 3s - loss: 0.0774 - accuracy: 0.9779
160/1688 [=>............................] - ETA: 3s - loss: 0.0781 - accuracy: 0.9779
183/1688 [==>...........................] - ETA: 3s - loss: 0.0777 - accuracy: 0.9775
206/1688 [==>...........................] - ETA: 3s - loss: 0.0776 - accuracy: 0.9774
229/1688 [===>..........................] - ETA: 3s - loss: 0.0793 - accuracy: 0.9763
252/1688 [===>..........................] - ETA: 3s - loss: 0.0758 - accuracy: 0.9774
275/1688 [===>..........................] - ETA: 3s - loss: 0.0769 - accuracy: 0.9766
298/1688 [====>.........................] - ETA: 3s - loss: 0.0785 - accuracy: 0.9758
321/1688 [====>.........................] - ETA: 3s - loss: 0.0792 - accuracy: 0.9755
343/1688 [=====>........................] - ETA: 3s - loss: 0.0809 - accuracy: 0.9751
366/1688 [=====>........................] - ETA: 2s - loss: 0.0795 - accuracy: 0.9752
389/1688 [=====>........................] - ETA: 2s - loss: 0.0803 - accuracy: 0.9750
412/1688 [======>.......................] - ETA: 2s - loss: 0.0801 - accuracy: 0.9752
435/1688 [======>.......................] - ETA: 2s - loss: 0.0803 - accuracy: 0.9753
457/1688 [=======>......................] - ETA: 2s - loss: 0.0809 - accuracy: 0.9751
478/1688 [=======>......................] - ETA: 2s - loss: 0.0812 - accuracy: 0.9748
499/1688 [=======>......................] - ETA: 2s - loss: 0.0803 - accuracy: 0.9749
520/1688 [========>.....................] - ETA: 2s - loss: 0.0794 - accuracy: 0.9751
541/1688 [========>.....................] - ETA: 2s - loss: 0.0777 - accuracy: 0.9755
562/1688 [========>.....................] - ETA: 2s - loss: 0.0766 - accuracy: 0.9760
584/1688 [=========>....................] - ETA: 2s - loss: 0.0759 - accuracy: 0.9761
606/1688 [=========>....................] - ETA: 2s - loss: 0.0773 - accuracy: 0.9762
627/1688 [==========>...................] - ETA: 2s - loss: 0.0763 - accuracy: 0.9763
649/1688 [==========>...................] - ETA: 2s - loss: 0.0771 - accuracy: 0.9760
670/1688 [==========>...................] - ETA: 2s - loss: 0.0763 - accuracy: 0.9763
691/1688 [===========>..................] - ETA: 2s - loss: 0.0757 - accuracy: 0.9766
712/1688 [===========>..................] - ETA: 2s - loss: 0.0757 - accuracy: 0.9767
734/1688 [============>.................] - ETA: 2s - loss: 0.0752 - accuracy: 0.9768
755/1688 [============>.................] - ETA: 2s - loss: 0.0750 - accuracy: 0.9768
776/1688 [============>.................] - ETA: 2s - loss: 0.0749 - accuracy: 0.9767
797/1688 [=============>................] - ETA: 2s - loss: 0.0741 - accuracy: 0.9768
818/1688 [=============>................] - ETA: 2s - loss: 0.0741 - accuracy: 0.9767
840/1688 [=============>................] - ETA: 1s - loss: 0.0734 - accuracy: 0.9768
861/1688 [==============>...............] - ETA: 1s - loss: 0.0738 - accuracy: 0.9767
883/1688 [==============>...............] - ETA: 1s - loss: 0.0736 - accuracy: 0.9767
903/1688 [===============>..............] - ETA: 1s - loss: 0.0732 - accuracy: 0.9768
923/1688 [===============>..............] - ETA: 1s - loss: 0.0734 - accuracy: 0.9768
943/1688 [===============>..............] - ETA: 1s - loss: 0.0736 - accuracy: 0.9767
963/1688 [================>.............] - ETA: 1s - loss: 0.0739 - accuracy: 0.9766
983/1688 [================>.............] - ETA: 1s - loss: 0.0740 - accuracy: 0.9766
1003/1688 [================>.............] - ETA: 1s - loss: 0.0738 - accuracy: 0.9766
1024/1688 [=================>............] - ETA: 1s - loss: 0.0739 - accuracy: 0.9766
1044/1688 [=================>............] - ETA: 1s - loss: 0.0738 - accuracy: 0.9766
1064/1688 [=================>............] - ETA: 1s - loss: 0.0736 - accuracy: 0.9766
1084/1688 [==================>...........] - ETA: 1s - loss: 0.0735 - accuracy: 0.9766
1104/1688 [==================>...........] - ETA: 1s - loss: 0.0739 - accuracy: 0.9763
1124/1688 [==================>...........] - ETA: 1s - loss: 0.0740 - accuracy: 0.9764
1144/1688 [===================>..........] - ETA: 1s - loss: 0.0736 - accuracy: 0.9765
1164/1688 [===================>..........] - ETA: 1s - loss: 0.0731 - accuracy: 0.9766
1184/1688 [====================>.........] - ETA: 1s - loss: 0.0729 - accuracy: 0.9767
1203/1688 [====================>.........] - ETA: 1s - loss: 0.0729 - accuracy: 0.9767
1223/1688 [====================>.........] - ETA: 1s - loss: 0.0729 - accuracy: 0.9767
1243/1688 [=====================>........] - ETA: 1s - loss: 0.0731 - accuracy: 0.9768
1262/1688 [=====================>........] - ETA: 1s - loss: 0.0729 - accuracy: 0.9767
1282/1688 [=====================>........] - ETA: 0s - loss: 0.0730 - accuracy: 0.9768
1302/1688 [======================>.......] - ETA: 0s - loss: 0.0728 - accuracy: 0.9768
1322/1688 [======================>.......] - ETA: 0s - loss: 0.0726 - accuracy: 0.9769
1342/1688 [======================>.......] - ETA: 0s - loss: 0.0726 - accuracy: 0.9769
1362/1688 [=======================>......] - ETA: 0s - loss: 0.0722 - accuracy: 0.9770
1382/1688 [=======================>......] - ETA: 0s - loss: 0.0723 - accuracy: 0.9770
1401/1688 [=======================>......] - ETA: 0s - loss: 0.0725 - accuracy: 0.9770
1421/1688 [========================>.....] - ETA: 0s - loss: 0.0726 - accuracy: 0.9770
1441/1688 [========================>.....] - ETA: 0s - loss: 0.0724 - accuracy: 0.9770
1460/1688 [========================>.....] - ETA: 0s - loss: 0.0722 - accuracy: 0.9771
1480/1688 [=========================>....] - ETA: 0s - loss: 0.0721 - accuracy: 0.9771
1500/1688 [=========================>....] - ETA: 0s - loss: 0.0721 - accuracy: 0.9771
1520/1688 [==========================>...] - ETA: 0s - loss: 0.0724 - accuracy: 0.9771
1540/1688 [==========================>...] - ETA: 0s - loss: 0.0725 - accuracy: 0.9770
1559/1688 [==========================>...] - ETA: 0s - loss: 0.0724 - accuracy: 0.9770
1579/1688 [===========================>..] - ETA: 0s - loss: 0.0726 - accuracy: 0.9770
1598/1688 [===========================>..] - ETA: 0s - loss: 0.0723 - accuracy: 0.9772
1617/1688 [===========================>..] - ETA: 0s - loss: 0.0721 - accuracy: 0.9773
1637/1688 [============================>.] - ETA: 0s - loss: 0.0723 - accuracy: 0.9773
1657/1688 [============================>.] - ETA: 0s - loss: 0.0720 - accuracy: 0.9773
1676/1688 [============================>.] - ETA: 0s - loss: 0.0721 - accuracy: 0.9772
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0720 - accuracy: 0.9772 - val_loss: 0.0892 - val_accuracy: 0.9742
Epoch 3/10
1/1688 [..............................] - ETA: 4s - loss: 0.0168 - accuracy: 1.0000
21/1688 [..............................] - ETA: 4s - loss: 0.0582 - accuracy: 0.9836
41/1688 [..............................] - ETA: 4s - loss: 0.0591 - accuracy: 0.9832
61/1688 [>.............................] - ETA: 4s - loss: 0.0526 - accuracy: 0.9836
81/1688 [>.............................] - ETA: 4s - loss: 0.0495 - accuracy: 0.9861
101/1688 [>.............................] - ETA: 4s - loss: 0.0484 - accuracy: 0.9858
121/1688 [=>............................] - ETA: 4s - loss: 0.0506 - accuracy: 0.9848
141/1688 [=>............................] - ETA: 3s - loss: 0.0495 - accuracy: 0.9849
161/1688 [=>............................] - ETA: 3s - loss: 0.0493 - accuracy: 0.9851
181/1688 [==>...........................] - ETA: 3s - loss: 0.0499 - accuracy: 0.9845
200/1688 [==>...........................] - ETA: 3s - loss: 0.0499 - accuracy: 0.9845
220/1688 [==>...........................] - ETA: 3s - loss: 0.0470 - accuracy: 0.9857
240/1688 [===>..........................] - ETA: 3s - loss: 0.0493 - accuracy: 0.9849
260/1688 [===>..........................] - ETA: 3s - loss: 0.0496 - accuracy: 0.9849
280/1688 [===>..........................] - ETA: 3s - loss: 0.0491 - accuracy: 0.9852
300/1688 [====>.........................] - ETA: 3s - loss: 0.0499 - accuracy: 0.9849
320/1688 [====>.........................] - ETA: 3s - loss: 0.0494 - accuracy: 0.9848
340/1688 [=====>........................] - ETA: 3s - loss: 0.0492 - accuracy: 0.9850
360/1688 [=====>........................] - ETA: 3s - loss: 0.0503 - accuracy: 0.9847
380/1688 [=====>........................] - ETA: 3s - loss: 0.0507 - accuracy: 0.9843
400/1688 [======>.......................] - ETA: 3s - loss: 0.0509 - accuracy: 0.9841
420/1688 [======>.......................] - ETA: 3s - loss: 0.0495 - accuracy: 0.9846
440/1688 [======>.......................] - ETA: 3s - loss: 0.0485 - accuracy: 0.9848
460/1688 [=======>......................] - ETA: 3s - loss: 0.0481 - accuracy: 0.9848
479/1688 [=======>......................] - ETA: 3s - loss: 0.0489 - accuracy: 0.9845
499/1688 [=======>......................] - ETA: 3s - loss: 0.0494 - accuracy: 0.9843
519/1688 [========>.....................] - ETA: 3s - loss: 0.0499 - accuracy: 0.9843
538/1688 [========>.....................] - ETA: 2s - loss: 0.0504 - accuracy: 0.9843
558/1688 [========>.....................] - ETA: 2s - loss: 0.0504 - accuracy: 0.9842
578/1688 [=========>....................] - ETA: 2s - loss: 0.0499 - accuracy: 0.9844
598/1688 [=========>....................] - ETA: 2s - loss: 0.0499 - accuracy: 0.9844
618/1688 [=========>....................] - ETA: 2s - loss: 0.0498 - accuracy: 0.9844
638/1688 [==========>...................] - ETA: 2s - loss: 0.0494 - accuracy: 0.9846
658/1688 [==========>...................] - ETA: 2s - loss: 0.0490 - accuracy: 0.9847
677/1688 [===========>..................] - ETA: 2s - loss: 0.0487 - accuracy: 0.9848
697/1688 [===========>..................] - ETA: 2s - loss: 0.0484 - accuracy: 0.9848
717/1688 [===========>..................] - ETA: 2s - loss: 0.0478 - accuracy: 0.9849
736/1688 [============>.................] - ETA: 2s - loss: 0.0484 - accuracy: 0.9847
756/1688 [============>.................] - ETA: 2s - loss: 0.0484 - accuracy: 0.9845
776/1688 [============>.................] - ETA: 2s - loss: 0.0483 - accuracy: 0.9845
796/1688 [=============>................] - ETA: 2s - loss: 0.0494 - accuracy: 0.9842
816/1688 [=============>................] - ETA: 2s - loss: 0.0497 - accuracy: 0.9840
836/1688 [=============>................] - ETA: 2s - loss: 0.0502 - accuracy: 0.9839
855/1688 [==============>...............] - ETA: 2s - loss: 0.0501 - accuracy: 0.9839
875/1688 [==============>...............] - ETA: 2s - loss: 0.0496 - accuracy: 0.9841
895/1688 [==============>...............] - ETA: 2s - loss: 0.0497 - accuracy: 0.9841
915/1688 [===============>..............] - ETA: 1s - loss: 0.0500 - accuracy: 0.9839
935/1688 [===============>..............] - ETA: 1s - loss: 0.0503 - accuracy: 0.9838
955/1688 [===============>..............] - ETA: 1s - loss: 0.0502 - accuracy: 0.9837
975/1688 [================>.............] - ETA: 1s - loss: 0.0504 - accuracy: 0.9837
995/1688 [================>.............] - ETA: 1s - loss: 0.0508 - accuracy: 0.9836
1015/1688 [=================>............] - ETA: 1s - loss: 0.0507 - accuracy: 0.9837
1034/1688 [=================>............] - ETA: 1s - loss: 0.0506 - accuracy: 0.9838
1054/1688 [=================>............] - ETA: 1s - loss: 0.0509 - accuracy: 0.9837
1074/1688 [==================>...........] - ETA: 1s - loss: 0.0508 - accuracy: 0.9838
1094/1688 [==================>...........] - ETA: 1s - loss: 0.0512 - accuracy: 0.9838
1114/1688 [==================>...........] - ETA: 1s - loss: 0.0509 - accuracy: 0.9839
1134/1688 [===================>..........] - ETA: 1s - loss: 0.0511 - accuracy: 0.9839
1153/1688 [===================>..........] - ETA: 1s - loss: 0.0510 - accuracy: 0.9838
1173/1688 [===================>..........] - ETA: 1s - loss: 0.0509 - accuracy: 0.9838
1193/1688 [====================>.........] - ETA: 1s - loss: 0.0513 - accuracy: 0.9838
1213/1688 [====================>.........] - ETA: 1s - loss: 0.0512 - accuracy: 0.9837
1233/1688 [====================>.........] - ETA: 1s - loss: 0.0512 - accuracy: 0.9838
1253/1688 [=====================>........] - ETA: 1s - loss: 0.0513 - accuracy: 0.9837
1273/1688 [=====================>........] - ETA: 1s - loss: 0.0513 - accuracy: 0.9837
1293/1688 [=====================>........] - ETA: 1s - loss: 0.0519 - accuracy: 0.9836
1313/1688 [======================>.......] - ETA: 0s - loss: 0.0518 - accuracy: 0.9837
1333/1688 [======================>.......] - ETA: 0s - loss: 0.0518 - accuracy: 0.9837
1353/1688 [=======================>......] - ETA: 0s - loss: 0.0518 - accuracy: 0.9837
1372/1688 [=======================>......] - ETA: 0s - loss: 0.0518 - accuracy: 0.9837
1392/1688 [=======================>......] - ETA: 0s - loss: 0.0516 - accuracy: 0.9837
1411/1688 [========================>.....] - ETA: 0s - loss: 0.0520 - accuracy: 0.9836
1431/1688 [========================>.....] - ETA: 0s - loss: 0.0519 - accuracy: 0.9837
1451/1688 [========================>.....] - ETA: 0s - loss: 0.0519 - accuracy: 0.9837
1470/1688 [=========================>....] - ETA: 0s - loss: 0.0520 - accuracy: 0.9836
1489/1688 [=========================>....] - ETA: 0s - loss: 0.0518 - accuracy: 0.9837
1508/1688 [=========================>....] - ETA: 0s - loss: 0.0518 - accuracy: 0.9837
1526/1688 [==========================>...] - ETA: 0s - loss: 0.0519 - accuracy: 0.9838
1544/1688 [==========================>...] - ETA: 0s - loss: 0.0518 - accuracy: 0.9838
1562/1688 [==========================>...] - ETA: 0s - loss: 0.0519 - accuracy: 0.9838
1580/1688 [===========================>..] - ETA: 0s - loss: 0.0522 - accuracy: 0.9837
1598/1688 [===========================>..] - ETA: 0s - loss: 0.0527 - accuracy: 0.9836
1616/1688 [===========================>..] - ETA: 0s - loss: 0.0526 - accuracy: 0.9836
1633/1688 [============================>.] - ETA: 0s - loss: 0.0526 - accuracy: 0.9836
1650/1688 [============================>.] - ETA: 0s - loss: 0.0528 - accuracy: 0.9835
1667/1688 [============================>.] - ETA: 0s - loss: 0.0527 - accuracy: 0.9836
1684/1688 [============================>.] - ETA: 0s - loss: 0.0526 - accuracy: 0.9837
1688/1688 [==============================] - 5s 3ms/step - loss: 0.0525 - accuracy: 0.9837 - val_loss: 0.0564 - val_accuracy: 0.9843
Epoch 4/10
1/1688 [..............................] - ETA: 4s - loss: 0.0018 - accuracy: 1.0000
20/1688 [..............................] - ETA: 4s - loss: 0.0278 - accuracy: 0.9922
39/1688 [..............................] - ETA: 4s - loss: 0.0392 - accuracy: 0.9904
57/1688 [>.............................] - ETA: 4s - loss: 0.0388 - accuracy: 0.9885
76/1688 [>.............................] - ETA: 4s - loss: 0.0401 - accuracy: 0.9873
95/1688 [>.............................] - ETA: 4s - loss: 0.0419 - accuracy: 0.9862
114/1688 [=>............................] - ETA: 4s - loss: 0.0405 - accuracy: 0.9860
133/1688 [=>............................] - ETA: 4s - loss: 0.0418 - accuracy: 0.9854
152/1688 [=>............................] - ETA: 4s - loss: 0.0424 - accuracy: 0.9852
171/1688 [==>...........................] - ETA: 4s - loss: 0.0424 - accuracy: 0.9852
190/1688 [==>...........................] - ETA: 4s - loss: 0.0424 - accuracy: 0.9850
209/1688 [==>...........................] - ETA: 4s - loss: 0.0462 - accuracy: 0.9846
228/1688 [===>..........................] - ETA: 3s - loss: 0.0464 - accuracy: 0.9848
247/1688 [===>..........................] - ETA: 3s - loss: 0.0454 - accuracy: 0.9851
266/1688 [===>..........................] - ETA: 3s - loss: 0.0440 - accuracy: 0.9857
285/1688 [====>.........................] - ETA: 3s - loss: 0.0434 - accuracy: 0.9859
304/1688 [====>.........................] - ETA: 3s - loss: 0.0421 - accuracy: 0.9861
322/1688 [====>.........................] - ETA: 3s - loss: 0.0416 - accuracy: 0.9863
341/1688 [=====>........................] - ETA: 3s - loss: 0.0403 - accuracy: 0.9866
360/1688 [=====>........................] - ETA: 3s - loss: 0.0402 - accuracy: 0.9869
379/1688 [=====>........................] - ETA: 3s - loss: 0.0400 - accuracy: 0.9868
398/1688 [======>.......................] - ETA: 3s - loss: 0.0405 - accuracy: 0.9867
417/1688 [======>.......................] - ETA: 3s - loss: 0.0411 - accuracy: 0.9866
436/1688 [======>.......................] - ETA: 3s - loss: 0.0414 - accuracy: 0.9867
455/1688 [=======>......................] - ETA: 3s - loss: 0.0404 - accuracy: 0.9871
474/1688 [=======>......................] - ETA: 3s - loss: 0.0407 - accuracy: 0.9868
493/1688 [=======>......................] - ETA: 3s - loss: 0.0400 - accuracy: 0.9871
512/1688 [========>.....................] - ETA: 3s - loss: 0.0403 - accuracy: 0.9871
531/1688 [========>.....................] - ETA: 3s - loss: 0.0405 - accuracy: 0.9868
550/1688 [========>.....................] - ETA: 3s - loss: 0.0402 - accuracy: 0.9869
569/1688 [=========>....................] - ETA: 3s - loss: 0.0401 - accuracy: 0.9871
588/1688 [=========>....................] - ETA: 2s - loss: 0.0398 - accuracy: 0.9870
607/1688 [=========>....................] - ETA: 2s - loss: 0.0398 - accuracy: 0.9871
626/1688 [==========>...................] - ETA: 2s - loss: 0.0397 - accuracy: 0.9872
645/1688 [==========>...................] - ETA: 2s - loss: 0.0403 - accuracy: 0.9871
664/1688 [==========>...................] - ETA: 2s - loss: 0.0405 - accuracy: 0.9870
683/1688 [===========>..................] - ETA: 2s - loss: 0.0401 - accuracy: 0.9871
702/1688 [===========>..................] - ETA: 2s - loss: 0.0402 - accuracy: 0.9871
721/1688 [===========>..................] - ETA: 2s - loss: 0.0400 - accuracy: 0.9872
740/1688 [============>.................] - ETA: 2s - loss: 0.0400 - accuracy: 0.9872
759/1688 [============>.................] - ETA: 2s - loss: 0.0398 - accuracy: 0.9873
778/1688 [============>.................] - ETA: 2s - loss: 0.0397 - accuracy: 0.9874
797/1688 [=============>................] - ETA: 2s - loss: 0.0402 - accuracy: 0.9873
816/1688 [=============>................] - ETA: 2s - loss: 0.0400 - accuracy: 0.9873
835/1688 [=============>................] - ETA: 2s - loss: 0.0399 - accuracy: 0.9872
854/1688 [==============>...............] - ETA: 2s - loss: 0.0405 - accuracy: 0.9871
873/1688 [==============>...............] - ETA: 2s - loss: 0.0405 - accuracy: 0.9871
892/1688 [==============>...............] - ETA: 2s - loss: 0.0405 - accuracy: 0.9871
911/1688 [===============>..............] - ETA: 2s - loss: 0.0401 - accuracy: 0.9873
930/1688 [===============>..............] - ETA: 2s - loss: 0.0407 - accuracy: 0.9872
949/1688 [===============>..............] - ETA: 2s - loss: 0.0411 - accuracy: 0.9870
968/1688 [================>.............] - ETA: 1s - loss: 0.0414 - accuracy: 0.9870
986/1688 [================>.............] - ETA: 1s - loss: 0.0415 - accuracy: 0.9868
1005/1688 [================>.............] - ETA: 1s - loss: 0.0417 - accuracy: 0.9867
1023/1688 [=================>............] - ETA: 1s - loss: 0.0415 - accuracy: 0.9866
1041/1688 [=================>............] - ETA: 1s - loss: 0.0417 - accuracy: 0.9864
1059/1688 [=================>............] - ETA: 1s - loss: 0.0417 - accuracy: 0.9865
1078/1688 [==================>...........] - ETA: 1s - loss: 0.0415 - accuracy: 0.9865
1097/1688 [==================>...........] - ETA: 1s - loss: 0.0414 - accuracy: 0.9866
1116/1688 [==================>...........] - ETA: 1s - loss: 0.0416 - accuracy: 0.9864
1135/1688 [===================>..........] - ETA: 1s - loss: 0.0420 - accuracy: 0.9863
1154/1688 [===================>..........] - ETA: 1s - loss: 0.0419 - accuracy: 0.9864
1173/1688 [===================>..........] - ETA: 1s - loss: 0.0422 - accuracy: 0.9862
1192/1688 [====================>.........] - ETA: 1s - loss: 0.0423 - accuracy: 0.9862
1211/1688 [====================>.........] - ETA: 1s - loss: 0.0428 - accuracy: 0.9861
1230/1688 [====================>.........] - ETA: 1s - loss: 0.0426 - accuracy: 0.9862
1249/1688 [=====================>........] - ETA: 1s - loss: 0.0428 - accuracy: 0.9861
1268/1688 [=====================>........] - ETA: 1s - loss: 0.0428 - accuracy: 0.9861
1286/1688 [=====================>........] - ETA: 1s - loss: 0.0427 - accuracy: 0.9862
1305/1688 [======================>.......] - ETA: 1s - loss: 0.0426 - accuracy: 0.9862
1324/1688 [======================>.......] - ETA: 0s - loss: 0.0422 - accuracy: 0.9863
1343/1688 [======================>.......] - ETA: 0s - loss: 0.0422 - accuracy: 0.9864
1362/1688 [=======================>......] - ETA: 0s - loss: 0.0425 - accuracy: 0.9863
1381/1688 [=======================>......] - ETA: 0s - loss: 0.0424 - accuracy: 0.9863
1400/1688 [=======================>......] - ETA: 0s - loss: 0.0427 - accuracy: 0.9861
1419/1688 [========================>.....] - ETA: 0s - loss: 0.0427 - accuracy: 0.9861
1438/1688 [========================>.....] - ETA: 0s - loss: 0.0429 - accuracy: 0.9861
1457/1688 [========================>.....] - ETA: 0s - loss: 0.0434 - accuracy: 0.9861
1476/1688 [=========================>....] - ETA: 0s - loss: 0.0433 - accuracy: 0.9861
1495/1688 [=========================>....] - ETA: 0s - loss: 0.0431 - accuracy: 0.9862
1514/1688 [=========================>....] - ETA: 0s - loss: 0.0430 - accuracy: 0.9863
1533/1688 [==========================>...] - ETA: 0s - loss: 0.0430 - accuracy: 0.9863
1552/1688 [==========================>...] - ETA: 0s - loss: 0.0429 - accuracy: 0.9863
1571/1688 [==========================>...] - ETA: 0s - loss: 0.0429 - accuracy: 0.9864
1590/1688 [===========================>..] - ETA: 0s - loss: 0.0428 - accuracy: 0.9864
1609/1688 [===========================>..] - ETA: 0s - loss: 0.0428 - accuracy: 0.9864
1627/1688 [===========================>..] - ETA: 0s - loss: 0.0429 - accuracy: 0.9863
1646/1688 [============================>.] - ETA: 0s - loss: 0.0428 - accuracy: 0.9863
1665/1688 [============================>.] - ETA: 0s - loss: 0.0429 - accuracy: 0.9863
1687/1688 [============================>.] - ETA: 0s - loss: 0.0428 - accuracy: 0.9863
1688/1688 [==============================] - 5s 3ms/step - loss: 0.0427 - accuracy: 0.9863 - val_loss: 0.0540 - val_accuracy: 0.9868
Epoch 5/10
1/1688 [..............................] - ETA: 3s - loss: 0.0028 - accuracy: 1.0000
25/1688 [..............................] - ETA: 3s - loss: 0.0171 - accuracy: 0.9937
48/1688 [..............................] - ETA: 3s - loss: 0.0252 - accuracy: 0.9909
70/1688 [>.............................] - ETA: 3s - loss: 0.0294 - accuracy: 0.9902
93/1688 [>.............................] - ETA: 3s - loss: 0.0280 - accuracy: 0.9899
115/1688 [=>............................] - ETA: 3s - loss: 0.0286 - accuracy: 0.9908
138/1688 [=>............................] - ETA: 3s - loss: 0.0269 - accuracy: 0.9916
161/1688 [=>............................] - ETA: 3s - loss: 0.0257 - accuracy: 0.9922
184/1688 [==>...........................] - ETA: 3s - loss: 0.0249 - accuracy: 0.9925
207/1688 [==>...........................] - ETA: 3s - loss: 0.0252 - accuracy: 0.9925
230/1688 [===>..........................] - ETA: 3s - loss: 0.0241 - accuracy: 0.9929
253/1688 [===>..........................] - ETA: 3s - loss: 0.0239 - accuracy: 0.9930
276/1688 [===>..........................] - ETA: 3s - loss: 0.0245 - accuracy: 0.9925
299/1688 [====>.........................] - ETA: 3s - loss: 0.0255 - accuracy: 0.9924
322/1688 [====>.........................] - ETA: 3s - loss: 0.0265 - accuracy: 0.9922
346/1688 [=====>........................] - ETA: 2s - loss: 0.0263 - accuracy: 0.9922
369/1688 [=====>........................] - ETA: 2s - loss: 0.0270 - accuracy: 0.9918
392/1688 [=====>........................] - ETA: 2s - loss: 0.0270 - accuracy: 0.9919
415/1688 [======>.......................] - ETA: 2s - loss: 0.0278 - accuracy: 0.9918
437/1688 [======>.......................] - ETA: 2s - loss: 0.0281 - accuracy: 0.9916
459/1688 [=======>......................] - ETA: 2s - loss: 0.0282 - accuracy: 0.9916
482/1688 [=======>......................] - ETA: 2s - loss: 0.0291 - accuracy: 0.9912
504/1688 [=======>......................] - ETA: 2s - loss: 0.0292 - accuracy: 0.9912
527/1688 [========>.....................] - ETA: 2s - loss: 0.0293 - accuracy: 0.9912
549/1688 [========>.....................] - ETA: 2s - loss: 0.0294 - accuracy: 0.9910
572/1688 [=========>....................] - ETA: 2s - loss: 0.0293 - accuracy: 0.9910
595/1688 [=========>....................] - ETA: 2s - loss: 0.0303 - accuracy: 0.9907
616/1688 [=========>....................] - ETA: 2s - loss: 0.0307 - accuracy: 0.9905
638/1688 [==========>...................] - ETA: 2s - loss: 0.0305 - accuracy: 0.9905
659/1688 [==========>...................] - ETA: 2s - loss: 0.0304 - accuracy: 0.9905
680/1688 [===========>..................] - ETA: 2s - loss: 0.0301 - accuracy: 0.9904
701/1688 [===========>..................] - ETA: 2s - loss: 0.0310 - accuracy: 0.9901
723/1688 [===========>..................] - ETA: 2s - loss: 0.0320 - accuracy: 0.9900
744/1688 [============>.................] - ETA: 2s - loss: 0.0317 - accuracy: 0.9901
765/1688 [============>.................] - ETA: 2s - loss: 0.0317 - accuracy: 0.9901
786/1688 [============>.................] - ETA: 2s - loss: 0.0319 - accuracy: 0.9900
806/1688 [=============>................] - ETA: 2s - loss: 0.0319 - accuracy: 0.9900
826/1688 [=============>................] - ETA: 1s - loss: 0.0318 - accuracy: 0.9900
845/1688 [==============>...............] - ETA: 1s - loss: 0.0317 - accuracy: 0.9899
865/1688 [==============>...............] - ETA: 1s - loss: 0.0316 - accuracy: 0.9900
885/1688 [==============>...............] - ETA: 1s - loss: 0.0313 - accuracy: 0.9901
905/1688 [===============>..............] - ETA: 1s - loss: 0.0310 - accuracy: 0.9902
925/1688 [===============>..............] - ETA: 1s - loss: 0.0311 - accuracy: 0.9902
945/1688 [===============>..............] - ETA: 1s - loss: 0.0317 - accuracy: 0.9900
965/1688 [================>.............] - ETA: 1s - loss: 0.0315 - accuracy: 0.9901
985/1688 [================>.............] - ETA: 1s - loss: 0.0316 - accuracy: 0.9900
1005/1688 [================>.............] - ETA: 1s - loss: 0.0316 - accuracy: 0.9899
1025/1688 [=================>............] - ETA: 1s - loss: 0.0320 - accuracy: 0.9898
1044/1688 [=================>............] - ETA: 1s - loss: 0.0318 - accuracy: 0.9898
1064/1688 [=================>............] - ETA: 1s - loss: 0.0322 - accuracy: 0.9898
1084/1688 [==================>...........] - ETA: 1s - loss: 0.0320 - accuracy: 0.9899
1104/1688 [==================>...........] - ETA: 1s - loss: 0.0326 - accuracy: 0.9898
1124/1688 [==================>...........] - ETA: 1s - loss: 0.0326 - accuracy: 0.9897
1144/1688 [===================>..........] - ETA: 1s - loss: 0.0326 - accuracy: 0.9898
1164/1688 [===================>..........] - ETA: 1s - loss: 0.0327 - accuracy: 0.9898
1184/1688 [====================>.........] - ETA: 1s - loss: 0.0329 - accuracy: 0.9898
1204/1688 [====================>.........] - ETA: 1s - loss: 0.0327 - accuracy: 0.9899
1224/1688 [====================>.........] - ETA: 1s - loss: 0.0324 - accuracy: 0.9899
1244/1688 [=====================>........] - ETA: 1s - loss: 0.0323 - accuracy: 0.9899
1264/1688 [=====================>........] - ETA: 1s - loss: 0.0322 - accuracy: 0.9899
1283/1688 [=====================>........] - ETA: 0s - loss: 0.0324 - accuracy: 0.9898
1303/1688 [======================>.......] - ETA: 0s - loss: 0.0325 - accuracy: 0.9898
1323/1688 [======================>.......] - ETA: 0s - loss: 0.0326 - accuracy: 0.9897
1343/1688 [======================>.......] - ETA: 0s - loss: 0.0325 - accuracy: 0.9898
1363/1688 [=======================>......] - ETA: 0s - loss: 0.0325 - accuracy: 0.9898
1383/1688 [=======================>......] - ETA: 0s - loss: 0.0327 - accuracy: 0.9897
1403/1688 [=======================>......] - ETA: 0s - loss: 0.0329 - accuracy: 0.9897
1423/1688 [========================>.....] - ETA: 0s - loss: 0.0332 - accuracy: 0.9896
1443/1688 [========================>.....] - ETA: 0s - loss: 0.0332 - accuracy: 0.9896
1463/1688 [=========================>....] - ETA: 0s - loss: 0.0331 - accuracy: 0.9896
1483/1688 [=========================>....] - ETA: 0s - loss: 0.0332 - accuracy: 0.9896
1503/1688 [=========================>....] - ETA: 0s - loss: 0.0334 - accuracy: 0.9895
1523/1688 [==========================>...] - ETA: 0s - loss: 0.0332 - accuracy: 0.9896
1543/1688 [==========================>...] - ETA: 0s - loss: 0.0332 - accuracy: 0.9895
1563/1688 [==========================>...] - ETA: 0s - loss: 0.0331 - accuracy: 0.9895
1582/1688 [===========================>..] - ETA: 0s - loss: 0.0331 - accuracy: 0.9895
1602/1688 [===========================>..] - ETA: 0s - loss: 0.0331 - accuracy: 0.9894
1621/1688 [===========================>..] - ETA: 0s - loss: 0.0335 - accuracy: 0.9894
1641/1688 [============================>.] - ETA: 0s - loss: 0.0335 - accuracy: 0.9893
1661/1688 [============================>.] - ETA: 0s - loss: 0.0337 - accuracy: 0.9892
1680/1688 [============================>.] - ETA: 0s - loss: 0.0337 - accuracy: 0.9892
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0338 - accuracy: 0.9891 - val_loss: 0.0636 - val_accuracy: 0.9815
Epoch 6/10
1/1688 [..............................] - ETA: 4s - loss: 0.0300 - accuracy: 0.9688
21/1688 [..............................] - ETA: 4s - loss: 0.0167 - accuracy: 0.9940
41/1688 [..............................] - ETA: 4s - loss: 0.0173 - accuracy: 0.9931
61/1688 [>.............................] - ETA: 4s - loss: 0.0214 - accuracy: 0.9928
81/1688 [>.............................] - ETA: 4s - loss: 0.0198 - accuracy: 0.9938
101/1688 [>.............................] - ETA: 4s - loss: 0.0193 - accuracy: 0.9935
121/1688 [=>............................] - ETA: 4s - loss: 0.0206 - accuracy: 0.9930
141/1688 [=>............................] - ETA: 4s - loss: 0.0217 - accuracy: 0.9927
161/1688 [=>............................] - ETA: 3s - loss: 0.0206 - accuracy: 0.9932
181/1688 [==>...........................] - ETA: 3s - loss: 0.0219 - accuracy: 0.9931
201/1688 [==>...........................] - ETA: 3s - loss: 0.0211 - accuracy: 0.9933
221/1688 [==>...........................] - ETA: 3s - loss: 0.0213 - accuracy: 0.9929
241/1688 [===>..........................] - ETA: 3s - loss: 0.0216 - accuracy: 0.9929
261/1688 [===>..........................] - ETA: 3s - loss: 0.0212 - accuracy: 0.9929
281/1688 [===>..........................] - ETA: 3s - loss: 0.0215 - accuracy: 0.9924
301/1688 [====>.........................] - ETA: 3s - loss: 0.0217 - accuracy: 0.9925
321/1688 [====>.........................] - ETA: 3s - loss: 0.0219 - accuracy: 0.9923
340/1688 [=====>........................] - ETA: 3s - loss: 0.0217 - accuracy: 0.9925
358/1688 [=====>........................] - ETA: 3s - loss: 0.0215 - accuracy: 0.9926
379/1688 [=====>........................] - ETA: 3s - loss: 0.0219 - accuracy: 0.9927
401/1688 [======>.......................] - ETA: 3s - loss: 0.0225 - accuracy: 0.9924
424/1688 [======>.......................] - ETA: 3s - loss: 0.0224 - accuracy: 0.9926
447/1688 [======>.......................] - ETA: 3s - loss: 0.0224 - accuracy: 0.9925
470/1688 [=======>......................] - ETA: 3s - loss: 0.0238 - accuracy: 0.9926
493/1688 [=======>......................] - ETA: 3s - loss: 0.0241 - accuracy: 0.9924
516/1688 [========>.....................] - ETA: 2s - loss: 0.0242 - accuracy: 0.9923
539/1688 [========>.....................] - ETA: 2s - loss: 0.0246 - accuracy: 0.9922
562/1688 [========>.....................] - ETA: 2s - loss: 0.0249 - accuracy: 0.9923
585/1688 [=========>....................] - ETA: 2s - loss: 0.0256 - accuracy: 0.9920
608/1688 [=========>....................] - ETA: 2s - loss: 0.0260 - accuracy: 0.9919
631/1688 [==========>...................] - ETA: 2s - loss: 0.0261 - accuracy: 0.9918
654/1688 [==========>...................] - ETA: 2s - loss: 0.0265 - accuracy: 0.9915
677/1688 [===========>..................] - ETA: 2s - loss: 0.0266 - accuracy: 0.9914
699/1688 [===========>..................] - ETA: 2s - loss: 0.0261 - accuracy: 0.9916
721/1688 [===========>..................] - ETA: 2s - loss: 0.0263 - accuracy: 0.9915
744/1688 [============>.................] - ETA: 2s - loss: 0.0261 - accuracy: 0.9916
767/1688 [============>.................] - ETA: 2s - loss: 0.0264 - accuracy: 0.9916
790/1688 [=============>................] - ETA: 2s - loss: 0.0263 - accuracy: 0.9917
812/1688 [=============>................] - ETA: 2s - loss: 0.0264 - accuracy: 0.9916
835/1688 [=============>................] - ETA: 2s - loss: 0.0261 - accuracy: 0.9918
857/1688 [==============>...............] - ETA: 1s - loss: 0.0258 - accuracy: 0.9919
880/1688 [==============>...............] - ETA: 1s - loss: 0.0256 - accuracy: 0.9920
902/1688 [===============>..............] - ETA: 1s - loss: 0.0255 - accuracy: 0.9920
924/1688 [===============>..............] - ETA: 1s - loss: 0.0255 - accuracy: 0.9920
947/1688 [===============>..............] - ETA: 1s - loss: 0.0255 - accuracy: 0.9919
970/1688 [================>.............] - ETA: 1s - loss: 0.0259 - accuracy: 0.9919
993/1688 [================>.............] - ETA: 1s - loss: 0.0261 - accuracy: 0.9918
1016/1688 [=================>............] - ETA: 1s - loss: 0.0262 - accuracy: 0.9918
1039/1688 [=================>............] - ETA: 1s - loss: 0.0260 - accuracy: 0.9919
1062/1688 [=================>............] - ETA: 1s - loss: 0.0260 - accuracy: 0.9918
1084/1688 [==================>...........] - ETA: 1s - loss: 0.0262 - accuracy: 0.9918
1106/1688 [==================>...........] - ETA: 1s - loss: 0.0263 - accuracy: 0.9917
1127/1688 [===================>..........] - ETA: 1s - loss: 0.0262 - accuracy: 0.9918
1146/1688 [===================>..........] - ETA: 1s - loss: 0.0261 - accuracy: 0.9918
1165/1688 [===================>..........] - ETA: 1s - loss: 0.0269 - accuracy: 0.9916
1185/1688 [====================>.........] - ETA: 1s - loss: 0.0269 - accuracy: 0.9916
1205/1688 [====================>.........] - ETA: 1s - loss: 0.0270 - accuracy: 0.9915
1225/1688 [====================>.........] - ETA: 1s - loss: 0.0269 - accuracy: 0.9915
1245/1688 [=====================>........] - ETA: 1s - loss: 0.0270 - accuracy: 0.9914
1265/1688 [=====================>........] - ETA: 1s - loss: 0.0270 - accuracy: 0.9914
1286/1688 [=====================>........] - ETA: 0s - loss: 0.0270 - accuracy: 0.9914
1306/1688 [======================>.......] - ETA: 0s - loss: 0.0269 - accuracy: 0.9914
1326/1688 [======================>.......] - ETA: 0s - loss: 0.0271 - accuracy: 0.9913
1346/1688 [======================>.......] - ETA: 0s - loss: 0.0273 - accuracy: 0.9913
1366/1688 [=======================>......] - ETA: 0s - loss: 0.0274 - accuracy: 0.9912
1386/1688 [=======================>......] - ETA: 0s - loss: 0.0277 - accuracy: 0.9910
1406/1688 [=======================>......] - ETA: 0s - loss: 0.0280 - accuracy: 0.9910
1425/1688 [========================>.....] - ETA: 0s - loss: 0.0283 - accuracy: 0.9909
1445/1688 [========================>.....] - ETA: 0s - loss: 0.0283 - accuracy: 0.9909
1465/1688 [=========================>....] - ETA: 0s - loss: 0.0284 - accuracy: 0.9909
1485/1688 [=========================>....] - ETA: 0s - loss: 0.0283 - accuracy: 0.9908
1505/1688 [=========================>....] - ETA: 0s - loss: 0.0282 - accuracy: 0.9908
1525/1688 [==========================>...] - ETA: 0s - loss: 0.0281 - accuracy: 0.9908
1545/1688 [==========================>...] - ETA: 0s - loss: 0.0281 - accuracy: 0.9909
1565/1688 [==========================>...] - ETA: 0s - loss: 0.0279 - accuracy: 0.9909
1585/1688 [===========================>..] - ETA: 0s - loss: 0.0280 - accuracy: 0.9909
1605/1688 [===========================>..] - ETA: 0s - loss: 0.0281 - accuracy: 0.9909
1626/1688 [===========================>..] - ETA: 0s - loss: 0.0281 - accuracy: 0.9909
1646/1688 [============================>.] - ETA: 0s - loss: 0.0281 - accuracy: 0.9908
1665/1688 [============================>.] - ETA: 0s - loss: 0.0281 - accuracy: 0.9908
1685/1688 [============================>.] - ETA: 0s - loss: 0.0282 - accuracy: 0.9908
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0282 - accuracy: 0.9908 - val_loss: 0.0586 - val_accuracy: 0.9828
Epoch 7/10
1/1688 [..............................] - ETA: 3s - loss: 0.0854 - accuracy: 0.9688
24/1688 [..............................] - ETA: 3s - loss: 0.0230 - accuracy: 0.9909
46/1688 [..............................] - ETA: 3s - loss: 0.0230 - accuracy: 0.9905
69/1688 [>.............................] - ETA: 3s - loss: 0.0263 - accuracy: 0.9905
91/1688 [>.............................] - ETA: 3s - loss: 0.0222 - accuracy: 0.9921
114/1688 [=>............................] - ETA: 3s - loss: 0.0207 - accuracy: 0.9923
137/1688 [=>............................] - ETA: 3s - loss: 0.0199 - accuracy: 0.9927
160/1688 [=>............................] - ETA: 3s - loss: 0.0179 - accuracy: 0.9937
183/1688 [==>...........................] - ETA: 3s - loss: 0.0187 - accuracy: 0.9937
205/1688 [==>...........................] - ETA: 3s - loss: 0.0183 - accuracy: 0.9937
227/1688 [===>..........................] - ETA: 3s - loss: 0.0191 - accuracy: 0.9935
250/1688 [===>..........................] - ETA: 3s - loss: 0.0191 - accuracy: 0.9940
272/1688 [===>..........................] - ETA: 3s - loss: 0.0183 - accuracy: 0.9940
295/1688 [====>.........................] - ETA: 3s - loss: 0.0176 - accuracy: 0.9943
318/1688 [====>.........................] - ETA: 3s - loss: 0.0174 - accuracy: 0.9943
341/1688 [=====>........................] - ETA: 3s - loss: 0.0176 - accuracy: 0.9939
364/1688 [=====>........................] - ETA: 2s - loss: 0.0170 - accuracy: 0.9942
387/1688 [=====>........................] - ETA: 2s - loss: 0.0178 - accuracy: 0.9939
410/1688 [======>.......................] - ETA: 2s - loss: 0.0181 - accuracy: 0.9939
433/1688 [======>.......................] - ETA: 2s - loss: 0.0181 - accuracy: 0.9938
456/1688 [=======>......................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9938
478/1688 [=======>......................] - ETA: 2s - loss: 0.0186 - accuracy: 0.9935
501/1688 [=======>......................] - ETA: 2s - loss: 0.0185 - accuracy: 0.9935
524/1688 [========>.....................] - ETA: 2s - loss: 0.0189 - accuracy: 0.9933
547/1688 [========>.....................] - ETA: 2s - loss: 0.0192 - accuracy: 0.9932
570/1688 [=========>....................] - ETA: 2s - loss: 0.0199 - accuracy: 0.9929
592/1688 [=========>....................] - ETA: 2s - loss: 0.0199 - accuracy: 0.9929
615/1688 [=========>....................] - ETA: 2s - loss: 0.0198 - accuracy: 0.9930
638/1688 [==========>...................] - ETA: 2s - loss: 0.0202 - accuracy: 0.9929
660/1688 [==========>...................] - ETA: 2s - loss: 0.0204 - accuracy: 0.9929
683/1688 [===========>..................] - ETA: 2s - loss: 0.0205 - accuracy: 0.9928
705/1688 [===========>..................] - ETA: 2s - loss: 0.0204 - accuracy: 0.9928
728/1688 [===========>..................] - ETA: 2s - loss: 0.0202 - accuracy: 0.9929
751/1688 [============>.................] - ETA: 2s - loss: 0.0200 - accuracy: 0.9930
774/1688 [============>.................] - ETA: 2s - loss: 0.0203 - accuracy: 0.9929
797/1688 [=============>................] - ETA: 2s - loss: 0.0202 - accuracy: 0.9929
820/1688 [=============>................] - ETA: 1s - loss: 0.0201 - accuracy: 0.9929
842/1688 [=============>................] - ETA: 1s - loss: 0.0201 - accuracy: 0.9930
865/1688 [==============>...............] - ETA: 1s - loss: 0.0201 - accuracy: 0.9930
888/1688 [==============>...............] - ETA: 1s - loss: 0.0200 - accuracy: 0.9930
911/1688 [===============>..............] - ETA: 1s - loss: 0.0202 - accuracy: 0.9929
934/1688 [===============>..............] - ETA: 1s - loss: 0.0202 - accuracy: 0.9929
957/1688 [================>.............] - ETA: 1s - loss: 0.0201 - accuracy: 0.9930
980/1688 [================>.............] - ETA: 1s - loss: 0.0204 - accuracy: 0.9929
1003/1688 [================>.............] - ETA: 1s - loss: 0.0206 - accuracy: 0.9929
1026/1688 [=================>............] - ETA: 1s - loss: 0.0206 - accuracy: 0.9929
1049/1688 [=================>............] - ETA: 1s - loss: 0.0206 - accuracy: 0.9929
1072/1688 [==================>...........] - ETA: 1s - loss: 0.0207 - accuracy: 0.9928
1094/1688 [==================>...........] - ETA: 1s - loss: 0.0207 - accuracy: 0.9927
1117/1688 [==================>...........] - ETA: 1s - loss: 0.0211 - accuracy: 0.9926
1140/1688 [===================>..........] - ETA: 1s - loss: 0.0210 - accuracy: 0.9926
1163/1688 [===================>..........] - ETA: 1s - loss: 0.0212 - accuracy: 0.9925
1186/1688 [====================>.........] - ETA: 1s - loss: 0.0214 - accuracy: 0.9926
1209/1688 [====================>.........] - ETA: 1s - loss: 0.0214 - accuracy: 0.9926
1232/1688 [====================>.........] - ETA: 1s - loss: 0.0219 - accuracy: 0.9925
1255/1688 [=====================>........] - ETA: 0s - loss: 0.0218 - accuracy: 0.9925
1277/1688 [=====================>........] - ETA: 0s - loss: 0.0221 - accuracy: 0.9925
1299/1688 [======================>.......] - ETA: 0s - loss: 0.0225 - accuracy: 0.9925
1319/1688 [======================>.......] - ETA: 0s - loss: 0.0228 - accuracy: 0.9923
1339/1688 [======================>.......] - ETA: 0s - loss: 0.0225 - accuracy: 0.9924
1359/1688 [=======================>......] - ETA: 0s - loss: 0.0226 - accuracy: 0.9924
1378/1688 [=======================>......] - ETA: 0s - loss: 0.0228 - accuracy: 0.9923
1398/1688 [=======================>......] - ETA: 0s - loss: 0.0230 - accuracy: 0.9923
1418/1688 [========================>.....] - ETA: 0s - loss: 0.0231 - accuracy: 0.9922
1437/1688 [========================>.....] - ETA: 0s - loss: 0.0232 - accuracy: 0.9922
1457/1688 [========================>.....] - ETA: 0s - loss: 0.0231 - accuracy: 0.9922
1477/1688 [=========================>....] - ETA: 0s - loss: 0.0231 - accuracy: 0.9922
1497/1688 [=========================>....] - ETA: 0s - loss: 0.0233 - accuracy: 0.9922
1517/1688 [=========================>....] - ETA: 0s - loss: 0.0232 - accuracy: 0.9922
1537/1688 [==========================>...] - ETA: 0s - loss: 0.0231 - accuracy: 0.9922
1557/1688 [==========================>...] - ETA: 0s - loss: 0.0231 - accuracy: 0.9922
1577/1688 [===========================>..] - ETA: 0s - loss: 0.0231 - accuracy: 0.9922
1597/1688 [===========================>..] - ETA: 0s - loss: 0.0232 - accuracy: 0.9921
1617/1688 [===========================>..] - ETA: 0s - loss: 0.0231 - accuracy: 0.9922
1637/1688 [============================>.] - ETA: 0s - loss: 0.0231 - accuracy: 0.9922
1657/1688 [============================>.] - ETA: 0s - loss: 0.0231 - accuracy: 0.9922
1676/1688 [============================>.] - ETA: 0s - loss: 0.0232 - accuracy: 0.9922
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0232 - accuracy: 0.9922 - val_loss: 0.0464 - val_accuracy: 0.9890
Epoch 8/10
1/1688 [..............................] - ETA: 3s - loss: 0.0138 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0062 - accuracy: 1.0000
46/1688 [..............................] - ETA: 3s - loss: 0.0091 - accuracy: 0.9986
69/1688 [>.............................] - ETA: 3s - loss: 0.0112 - accuracy: 0.9977
91/1688 [>.............................] - ETA: 3s - loss: 0.0121 - accuracy: 0.9969
114/1688 [=>............................] - ETA: 3s - loss: 0.0118 - accuracy: 0.9970
137/1688 [=>............................] - ETA: 3s - loss: 0.0118 - accuracy: 0.9968
159/1688 [=>............................] - ETA: 3s - loss: 0.0122 - accuracy: 0.9965
181/1688 [==>...........................] - ETA: 3s - loss: 0.0114 - accuracy: 0.9967
203/1688 [==>...........................] - ETA: 3s - loss: 0.0121 - accuracy: 0.9963
225/1688 [==>...........................] - ETA: 3s - loss: 0.0121 - accuracy: 0.9961
248/1688 [===>..........................] - ETA: 3s - loss: 0.0132 - accuracy: 0.9956
271/1688 [===>..........................] - ETA: 3s - loss: 0.0124 - accuracy: 0.9960
294/1688 [====>.........................] - ETA: 3s - loss: 0.0122 - accuracy: 0.9960
317/1688 [====>.........................] - ETA: 3s - loss: 0.0121 - accuracy: 0.9960
340/1688 [=====>........................] - ETA: 3s - loss: 0.0125 - accuracy: 0.9957
363/1688 [=====>........................] - ETA: 2s - loss: 0.0128 - accuracy: 0.9955
386/1688 [=====>........................] - ETA: 2s - loss: 0.0131 - accuracy: 0.9953
409/1688 [======>.......................] - ETA: 2s - loss: 0.0130 - accuracy: 0.9953
431/1688 [======>.......................] - ETA: 2s - loss: 0.0127 - accuracy: 0.9954
454/1688 [=======>......................] - ETA: 2s - loss: 0.0128 - accuracy: 0.9954
477/1688 [=======>......................] - ETA: 2s - loss: 0.0129 - accuracy: 0.9954
500/1688 [=======>......................] - ETA: 2s - loss: 0.0134 - accuracy: 0.9951
523/1688 [========>.....................] - ETA: 2s - loss: 0.0140 - accuracy: 0.9949
545/1688 [========>.....................] - ETA: 2s - loss: 0.0141 - accuracy: 0.9949
568/1688 [=========>....................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9949
591/1688 [=========>....................] - ETA: 2s - loss: 0.0143 - accuracy: 0.9951
614/1688 [=========>....................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9951
637/1688 [==========>...................] - ETA: 2s - loss: 0.0141 - accuracy: 0.9952
660/1688 [==========>...................] - ETA: 2s - loss: 0.0139 - accuracy: 0.9953
683/1688 [===========>..................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9952
706/1688 [===========>..................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9951
729/1688 [===========>..................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9949
752/1688 [============>.................] - ETA: 2s - loss: 0.0148 - accuracy: 0.9951
775/1688 [============>.................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9950
798/1688 [=============>................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9950
821/1688 [=============>................] - ETA: 1s - loss: 0.0153 - accuracy: 0.9950
844/1688 [==============>...............] - ETA: 1s - loss: 0.0154 - accuracy: 0.9949
866/1688 [==============>...............] - ETA: 1s - loss: 0.0158 - accuracy: 0.9949
889/1688 [==============>...............] - ETA: 1s - loss: 0.0160 - accuracy: 0.9948
912/1688 [===============>..............] - ETA: 1s - loss: 0.0162 - accuracy: 0.9948
936/1688 [===============>..............] - ETA: 1s - loss: 0.0161 - accuracy: 0.9948
958/1688 [================>.............] - ETA: 1s - loss: 0.0163 - accuracy: 0.9948
981/1688 [================>.............] - ETA: 1s - loss: 0.0165 - accuracy: 0.9948
1003/1688 [================>.............] - ETA: 1s - loss: 0.0166 - accuracy: 0.9947
1026/1688 [=================>............] - ETA: 1s - loss: 0.0165 - accuracy: 0.9948
1049/1688 [=================>............] - ETA: 1s - loss: 0.0165 - accuracy: 0.9947
1072/1688 [==================>...........] - ETA: 1s - loss: 0.0164 - accuracy: 0.9947
1095/1688 [==================>...........] - ETA: 1s - loss: 0.0163 - accuracy: 0.9947
1118/1688 [==================>...........] - ETA: 1s - loss: 0.0163 - accuracy: 0.9948
1141/1688 [===================>..........] - ETA: 1s - loss: 0.0164 - accuracy: 0.9947
1164/1688 [===================>..........] - ETA: 1s - loss: 0.0168 - accuracy: 0.9946
1186/1688 [====================>.........] - ETA: 1s - loss: 0.0170 - accuracy: 0.9946
1209/1688 [====================>.........] - ETA: 1s - loss: 0.0170 - accuracy: 0.9946
1232/1688 [====================>.........] - ETA: 1s - loss: 0.0169 - accuracy: 0.9946
1255/1688 [=====================>........] - ETA: 0s - loss: 0.0173 - accuracy: 0.9945
1278/1688 [=====================>........] - ETA: 0s - loss: 0.0174 - accuracy: 0.9945
1301/1688 [======================>.......] - ETA: 0s - loss: 0.0179 - accuracy: 0.9943
1324/1688 [======================>.......] - ETA: 0s - loss: 0.0181 - accuracy: 0.9942
1346/1688 [======================>.......] - ETA: 0s - loss: 0.0185 - accuracy: 0.9941
1369/1688 [=======================>......] - ETA: 0s - loss: 0.0189 - accuracy: 0.9940
1392/1688 [=======================>......] - ETA: 0s - loss: 0.0190 - accuracy: 0.9940
1415/1688 [========================>.....] - ETA: 0s - loss: 0.0190 - accuracy: 0.9940
1438/1688 [========================>.....] - ETA: 0s - loss: 0.0191 - accuracy: 0.9939
1461/1688 [========================>.....] - ETA: 0s - loss: 0.0190 - accuracy: 0.9939
1484/1688 [=========================>....] - ETA: 0s - loss: 0.0189 - accuracy: 0.9939
1507/1688 [=========================>....] - ETA: 0s - loss: 0.0192 - accuracy: 0.9938
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0195 - accuracy: 0.9938
1553/1688 [==========================>...] - ETA: 0s - loss: 0.0196 - accuracy: 0.9938
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0202 - accuracy: 0.9936
1599/1688 [===========================>..] - ETA: 0s - loss: 0.0204 - accuracy: 0.9936
1622/1688 [===========================>..] - ETA: 0s - loss: 0.0206 - accuracy: 0.9935
1644/1688 [============================>.] - ETA: 0s - loss: 0.0205 - accuracy: 0.9935
1667/1688 [============================>.] - ETA: 0s - loss: 0.0207 - accuracy: 0.9935
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0207 - accuracy: 0.9935 - val_loss: 0.0588 - val_accuracy: 0.9853
Epoch 9/10
1/1688 [..............................] - ETA: 3s - loss: 3.4678e-04 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0131 - accuracy: 0.9974
47/1688 [..............................] - ETA: 3s - loss: 0.0216 - accuracy: 0.9960
70/1688 [>.............................] - ETA: 3s - loss: 0.0190 - accuracy: 0.9955
93/1688 [>.............................] - ETA: 3s - loss: 0.0174 - accuracy: 0.9953
116/1688 [=>............................] - ETA: 3s - loss: 0.0171 - accuracy: 0.9954
139/1688 [=>............................] - ETA: 3s - loss: 0.0159 - accuracy: 0.9957
162/1688 [=>............................] - ETA: 3s - loss: 0.0164 - accuracy: 0.9956
184/1688 [==>...........................] - ETA: 3s - loss: 0.0168 - accuracy: 0.9952
206/1688 [==>...........................] - ETA: 3s - loss: 0.0154 - accuracy: 0.9958
229/1688 [===>..........................] - ETA: 3s - loss: 0.0158 - accuracy: 0.9955
252/1688 [===>..........................] - ETA: 3s - loss: 0.0151 - accuracy: 0.9958
275/1688 [===>..........................] - ETA: 3s - loss: 0.0152 - accuracy: 0.9957
298/1688 [====>.........................] - ETA: 3s - loss: 0.0146 - accuracy: 0.9958
321/1688 [====>.........................] - ETA: 3s - loss: 0.0142 - accuracy: 0.9958
344/1688 [=====>........................] - ETA: 3s - loss: 0.0139 - accuracy: 0.9959
367/1688 [=====>........................] - ETA: 2s - loss: 0.0140 - accuracy: 0.9957
390/1688 [=====>........................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9957
413/1688 [======>.......................] - ETA: 2s - loss: 0.0141 - accuracy: 0.9957
436/1688 [======>.......................] - ETA: 2s - loss: 0.0137 - accuracy: 0.9959
459/1688 [=======>......................] - ETA: 2s - loss: 0.0137 - accuracy: 0.9960
482/1688 [=======>......................] - ETA: 2s - loss: 0.0136 - accuracy: 0.9960
504/1688 [=======>......................] - ETA: 2s - loss: 0.0135 - accuracy: 0.9960
527/1688 [========>.....................] - ETA: 2s - loss: 0.0137 - accuracy: 0.9958
550/1688 [========>.....................] - ETA: 2s - loss: 0.0137 - accuracy: 0.9959
573/1688 [=========>....................] - ETA: 2s - loss: 0.0135 - accuracy: 0.9959
596/1688 [=========>....................] - ETA: 2s - loss: 0.0139 - accuracy: 0.9958
619/1688 [==========>...................] - ETA: 2s - loss: 0.0141 - accuracy: 0.9957
641/1688 [==========>...................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9958
663/1688 [==========>...................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9957
686/1688 [===========>..................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9957
709/1688 [===========>..................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9958
732/1688 [============>.................] - ETA: 2s - loss: 0.0143 - accuracy: 0.9957
755/1688 [============>.................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9957
778/1688 [============>.................] - ETA: 2s - loss: 0.0143 - accuracy: 0.9957
802/1688 [=============>................] - ETA: 1s - loss: 0.0143 - accuracy: 0.9957
825/1688 [=============>................] - ETA: 1s - loss: 0.0144 - accuracy: 0.9957
848/1688 [==============>...............] - ETA: 1s - loss: 0.0143 - accuracy: 0.9957
871/1688 [==============>...............] - ETA: 1s - loss: 0.0146 - accuracy: 0.9955
894/1688 [==============>...............] - ETA: 1s - loss: 0.0144 - accuracy: 0.9956
917/1688 [===============>..............] - ETA: 1s - loss: 0.0145 - accuracy: 0.9955
940/1688 [===============>..............] - ETA: 1s - loss: 0.0147 - accuracy: 0.9954
963/1688 [================>.............] - ETA: 1s - loss: 0.0150 - accuracy: 0.9954
985/1688 [================>.............] - ETA: 1s - loss: 0.0149 - accuracy: 0.9954
1008/1688 [================>.............] - ETA: 1s - loss: 0.0150 - accuracy: 0.9953
1031/1688 [=================>............] - ETA: 1s - loss: 0.0153 - accuracy: 0.9952
1054/1688 [=================>............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9953
1076/1688 [==================>...........] - ETA: 1s - loss: 0.0152 - accuracy: 0.9953
1099/1688 [==================>...........] - ETA: 1s - loss: 0.0152 - accuracy: 0.9953
1122/1688 [==================>...........] - ETA: 1s - loss: 0.0155 - accuracy: 0.9952
1145/1688 [===================>..........] - ETA: 1s - loss: 0.0154 - accuracy: 0.9952
1167/1688 [===================>..........] - ETA: 1s - loss: 0.0158 - accuracy: 0.9951
1190/1688 [====================>.........] - ETA: 1s - loss: 0.0159 - accuracy: 0.9950
1212/1688 [====================>.........] - ETA: 1s - loss: 0.0162 - accuracy: 0.9949
1234/1688 [====================>.........] - ETA: 1s - loss: 0.0161 - accuracy: 0.9950
1257/1688 [=====================>........] - ETA: 0s - loss: 0.0160 - accuracy: 0.9950
1280/1688 [=====================>........] - ETA: 0s - loss: 0.0160 - accuracy: 0.9950
1303/1688 [======================>.......] - ETA: 0s - loss: 0.0160 - accuracy: 0.9950
1326/1688 [======================>.......] - ETA: 0s - loss: 0.0160 - accuracy: 0.9949
1349/1688 [======================>.......] - ETA: 0s - loss: 0.0161 - accuracy: 0.9949
1372/1688 [=======================>......] - ETA: 0s - loss: 0.0160 - accuracy: 0.9949
1394/1688 [=======================>......] - ETA: 0s - loss: 0.0160 - accuracy: 0.9948
1417/1688 [========================>.....] - ETA: 0s - loss: 0.0163 - accuracy: 0.9948
1440/1688 [========================>.....] - ETA: 0s - loss: 0.0163 - accuracy: 0.9948
1462/1688 [========================>.....] - ETA: 0s - loss: 0.0164 - accuracy: 0.9947
1484/1688 [=========================>....] - ETA: 0s - loss: 0.0164 - accuracy: 0.9947
1506/1688 [=========================>....] - ETA: 0s - loss: 0.0163 - accuracy: 0.9948
1529/1688 [==========================>...] - ETA: 0s - loss: 0.0163 - accuracy: 0.9948
1552/1688 [==========================>...] - ETA: 0s - loss: 0.0163 - accuracy: 0.9948
1575/1688 [==========================>...] - ETA: 0s - loss: 0.0164 - accuracy: 0.9947
1598/1688 [===========================>..] - ETA: 0s - loss: 0.0163 - accuracy: 0.9947
1620/1688 [===========================>..] - ETA: 0s - loss: 0.0163 - accuracy: 0.9948
1643/1688 [============================>.] - ETA: 0s - loss: 0.0161 - accuracy: 0.9948
1666/1688 [============================>.] - ETA: 0s - loss: 0.0160 - accuracy: 0.9948
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0162 - accuracy: 0.9948 - val_loss: 0.0596 - val_accuracy: 0.9868
Epoch 10/10
1/1688 [..............................] - ETA: 3s - loss: 0.0272 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0163 - accuracy: 0.9922
46/1688 [..............................] - ETA: 3s - loss: 0.0141 - accuracy: 0.9939
69/1688 [>.............................] - ETA: 3s - loss: 0.0105 - accuracy: 0.9959
92/1688 [>.............................] - ETA: 3s - loss: 0.0109 - accuracy: 0.9963
114/1688 [=>............................] - ETA: 3s - loss: 0.0105 - accuracy: 0.9964
134/1688 [=>............................] - ETA: 3s - loss: 0.0112 - accuracy: 0.9967
153/1688 [=>............................] - ETA: 3s - loss: 0.0109 - accuracy: 0.9969
173/1688 [==>...........................] - ETA: 3s - loss: 0.0113 - accuracy: 0.9966
193/1688 [==>...........................] - ETA: 3s - loss: 0.0113 - accuracy: 0.9964
213/1688 [==>...........................] - ETA: 3s - loss: 0.0122 - accuracy: 0.9962
233/1688 [===>..........................] - ETA: 3s - loss: 0.0121 - accuracy: 0.9961
252/1688 [===>..........................] - ETA: 3s - loss: 0.0119 - accuracy: 0.9959
272/1688 [===>..........................] - ETA: 3s - loss: 0.0118 - accuracy: 0.9959
292/1688 [====>.........................] - ETA: 3s - loss: 0.0118 - accuracy: 0.9959
312/1688 [====>.........................] - ETA: 3s - loss: 0.0114 - accuracy: 0.9961
331/1688 [====>.........................] - ETA: 3s - loss: 0.0112 - accuracy: 0.9961
351/1688 [=====>........................] - ETA: 3s - loss: 0.0109 - accuracy: 0.9963
371/1688 [=====>........................] - ETA: 3s - loss: 0.0113 - accuracy: 0.9960
391/1688 [=====>........................] - ETA: 3s - loss: 0.0111 - accuracy: 0.9962
411/1688 [======>.......................] - ETA: 3s - loss: 0.0109 - accuracy: 0.9964
431/1688 [======>.......................] - ETA: 3s - loss: 0.0112 - accuracy: 0.9960
451/1688 [=======>......................] - ETA: 3s - loss: 0.0109 - accuracy: 0.9961
471/1688 [=======>......................] - ETA: 3s - loss: 0.0113 - accuracy: 0.9961
491/1688 [=======>......................] - ETA: 3s - loss: 0.0118 - accuracy: 0.9959
511/1688 [========>.....................] - ETA: 2s - loss: 0.0124 - accuracy: 0.9958
531/1688 [========>.....................] - ETA: 2s - loss: 0.0125 - accuracy: 0.9958
551/1688 [========>.....................] - ETA: 2s - loss: 0.0127 - accuracy: 0.9956
571/1688 [=========>....................] - ETA: 2s - loss: 0.0126 - accuracy: 0.9956
591/1688 [=========>....................] - ETA: 2s - loss: 0.0124 - accuracy: 0.9957
611/1688 [=========>....................] - ETA: 2s - loss: 0.0124 - accuracy: 0.9957
631/1688 [==========>...................] - ETA: 2s - loss: 0.0125 - accuracy: 0.9956
651/1688 [==========>...................] - ETA: 2s - loss: 0.0125 - accuracy: 0.9956
671/1688 [==========>...................] - ETA: 2s - loss: 0.0123 - accuracy: 0.9956
691/1688 [===========>..................] - ETA: 2s - loss: 0.0123 - accuracy: 0.9956
711/1688 [===========>..................] - ETA: 2s - loss: 0.0123 - accuracy: 0.9956
731/1688 [===========>..................] - ETA: 2s - loss: 0.0124 - accuracy: 0.9956
751/1688 [============>.................] - ETA: 2s - loss: 0.0128 - accuracy: 0.9956
770/1688 [============>.................] - ETA: 2s - loss: 0.0129 - accuracy: 0.9956
789/1688 [=============>................] - ETA: 2s - loss: 0.0130 - accuracy: 0.9955
809/1688 [=============>................] - ETA: 2s - loss: 0.0132 - accuracy: 0.9954
829/1688 [=============>................] - ETA: 2s - loss: 0.0133 - accuracy: 0.9954
849/1688 [==============>...............] - ETA: 2s - loss: 0.0134 - accuracy: 0.9954
869/1688 [==============>...............] - ETA: 2s - loss: 0.0134 - accuracy: 0.9954
889/1688 [==============>...............] - ETA: 2s - loss: 0.0136 - accuracy: 0.9953
909/1688 [===============>..............] - ETA: 1s - loss: 0.0137 - accuracy: 0.9952
929/1688 [===============>..............] - ETA: 1s - loss: 0.0136 - accuracy: 0.9953
949/1688 [===============>..............] - ETA: 1s - loss: 0.0135 - accuracy: 0.9953
969/1688 [================>.............] - ETA: 1s - loss: 0.0137 - accuracy: 0.9952
989/1688 [================>.............] - ETA: 1s - loss: 0.0137 - accuracy: 0.9952
1009/1688 [================>.............] - ETA: 1s - loss: 0.0136 - accuracy: 0.9953
1030/1688 [=================>............] - ETA: 1s - loss: 0.0135 - accuracy: 0.9953
1050/1688 [=================>............] - ETA: 1s - loss: 0.0136 - accuracy: 0.9953
1070/1688 [==================>...........] - ETA: 1s - loss: 0.0135 - accuracy: 0.9953
1090/1688 [==================>...........] - ETA: 1s - loss: 0.0139 - accuracy: 0.9952
1110/1688 [==================>...........] - ETA: 1s - loss: 0.0138 - accuracy: 0.9953
1130/1688 [===================>..........] - ETA: 1s - loss: 0.0145 - accuracy: 0.9952
1150/1688 [===================>..........] - ETA: 1s - loss: 0.0146 - accuracy: 0.9951
1170/1688 [===================>..........] - ETA: 1s - loss: 0.0148 - accuracy: 0.9951
1190/1688 [====================>.........] - ETA: 1s - loss: 0.0149 - accuracy: 0.9950
1210/1688 [====================>.........] - ETA: 1s - loss: 0.0151 - accuracy: 0.9949
1230/1688 [====================>.........] - ETA: 1s - loss: 0.0152 - accuracy: 0.9949
1250/1688 [=====================>........] - ETA: 1s - loss: 0.0152 - accuracy: 0.9948
1270/1688 [=====================>........] - ETA: 1s - loss: 0.0152 - accuracy: 0.9948
1290/1688 [=====================>........] - ETA: 1s - loss: 0.0152 - accuracy: 0.9948
1310/1688 [======================>.......] - ETA: 0s - loss: 0.0154 - accuracy: 0.9947
1330/1688 [======================>.......] - ETA: 0s - loss: 0.0156 - accuracy: 0.9946
1349/1688 [======================>.......] - ETA: 0s - loss: 0.0156 - accuracy: 0.9946
1369/1688 [=======================>......] - ETA: 0s - loss: 0.0157 - accuracy: 0.9945
1389/1688 [=======================>......] - ETA: 0s - loss: 0.0156 - accuracy: 0.9945
1409/1688 [========================>.....] - ETA: 0s - loss: 0.0157 - accuracy: 0.9945
1429/1688 [========================>.....] - ETA: 0s - loss: 0.0158 - accuracy: 0.9944
1449/1688 [========================>.....] - ETA: 0s - loss: 0.0158 - accuracy: 0.9944
1469/1688 [=========================>....] - ETA: 0s - loss: 0.0158 - accuracy: 0.9944
1488/1688 [=========================>....] - ETA: 0s - loss: 0.0158 - accuracy: 0.9944
1508/1688 [=========================>....] - ETA: 0s - loss: 0.0157 - accuracy: 0.9944
1528/1688 [==========================>...] - ETA: 0s - loss: 0.0157 - accuracy: 0.9945
1548/1688 [==========================>...] - ETA: 0s - loss: 0.0157 - accuracy: 0.9945
1568/1688 [==========================>...] - ETA: 0s - loss: 0.0156 - accuracy: 0.9945
1588/1688 [===========================>..] - ETA: 0s - loss: 0.0156 - accuracy: 0.9945
1608/1688 [===========================>..] - ETA: 0s - loss: 0.0157 - accuracy: 0.9944
1628/1688 [===========================>..] - ETA: 0s - loss: 0.0157 - accuracy: 0.9944
1648/1688 [============================>.] - ETA: 0s - loss: 0.0158 - accuracy: 0.9944
1668/1688 [============================>.] - ETA: 0s - loss: 0.0158 - accuracy: 0.9944
1688/1688 [==============================] - ETA: 0s - loss: 0.0159 - accuracy: 0.9944
1688/1688 [==============================] - 5s 3ms/step - loss: 0.0159 - accuracy: 0.9944 - val_loss: 0.0615 - val_accuracy: 0.9857
score = model_keras.evaluate(x_test, y_test, verbose=0)
print('Test accuracy:', score[1])
Test accuracy: 0.9836999773979187
2. Quantize
2.1. 8-bit quantization
An Akida accelerator processes 8 or 4-bits integer activations and weights. Therefore, the floating point Keras model must be quantized in preparation to run on an Akida accelerator.
The QuantizeML quantize function can be used to quantize a Keras model for Akida. For this step in this example, an “8/8/8” quantization scheme will be applied to the floating point Keras model to produce 8-bit weights in the first layer, 8-bit weights in all other layers, and 8-bit activations.
The quantization process results in a Keras model with custom QuantizeML quantized layers substituted for the original Keras layers.
All Keras API functions can be applied on this new model: summary(), compile(), fit(). etc.
Note
The quantize function applies several transformations to
the original model. For example, it folds the batch normalization layers into the
corresponding neural layers. The new weights are computed according to this folding
operation.
from quantizeml.models import quantize, QuantizationParams
qparams = QuantizationParams(input_weight_bits=8, weight_bits=8, activation_bits=8)
model_quantized = quantize(model_keras, qparams=qparams)
/usr/local/lib/python3.11/dist-packages/quantizeml/models/quantize.py:494: UserWarning: Quantizing per-axis with random calibration samples is not accurate. Set QuantizationParams.per_tensor_activations=True when calibrating with random samples.
warnings.warn("Quantizing per-axis with random calibration samples is not accurate. "
1/1024 [..............................] - ETA: 3:21
55/1024 [>.............................] - ETA: 0s
114/1024 [==>...........................] - ETA: 0s
144/1024 [===>..........................] - ETA: 1s
201/1024 [====>.........................] - ETA: 1s
259/1024 [======>.......................] - ETA: 1s
316/1024 [========>.....................] - ETA: 1s
374/1024 [=========>....................] - ETA: 0s
433/1024 [===========>..................] - ETA: 0s
491/1024 [=============>................] - ETA: 0s
547/1024 [===============>..............] - ETA: 0s
603/1024 [================>.............] - ETA: 0s
660/1024 [==================>...........] - ETA: 0s
715/1024 [===================>..........] - ETA: 0s
773/1024 [=====================>........] - ETA: 0s
830/1024 [=======================>......] - ETA: 0s
887/1024 [========================>.....] - ETA: 0s
944/1024 [==========================>...] - ETA: 0s
1000/1024 [============================>.] - ETA: 0s
1024/1024 [==============================] - 1s 1ms/step
model_quantized.summary()
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling_input (InputLaye [(None, 28, 28, 1)] 0
r)
rescaling (QuantizedRescal (None, 28, 28, 1) 0
ing)
conv2d (QuantizedConv2D) (None, 13, 13, 32) 320
re_lu (QuantizedReLU) (None, 13, 13, 32) 64
depthwise_conv2d (Quantize (None, 7, 7, 32) 384
dDepthwiseConv2D)
conv2d_1 (QuantizedConv2D) (None, 7, 7, 64) 2112
re_lu_1 (QuantizedReLU) (None, 7, 7, 64) 128
flatten (QuantizedFlatten) (None, 3136) 0
dense (QuantizedDense) (None, 10) 31370
dequantizer (Dequantizer) (None, 10) 0
=================================================================
Total params: 34378 (134.29 KB)
Trainable params: 34122 (133.29 KB)
Non-trainable params: 256 (1.00 KB)
_________________________________________________________________
Note
Note that the number of parameters for the floating and quantized models differs, a consequence of the BatchNormalization folding and the additional parameters added for quantization. For further details, please refer to their respective summary.
Check the quantized model accuracy.
def compile_evaluate(model):
""" Compiles and evaluates the model, then return accuracy score. """
model.compile(metrics=['accuracy'])
return model.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after 8-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 8-bit quantization: 0.9779000282287598
2.2. Effect of calibration
The previous call to quantize was made with random samples for calibration
(default parameters). While the observed drop in accuracy is minimal, that is
around 1%, it can be worse on more complex models. Therefore, it is advised to
use a set of real samples from the training set for calibration during a call
to quantize.
Note that this remains a calibration step rather than a training step in that
no output labels are required. Furthermore, any relevant data could be used for
calibration. The recommended settings for calibration that are widely used to
obtain the zoo performance are:
1024 samples
a batch size of 100
2 epochs
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 987us/step
Check the accuracy for the quantized and calibrated model.
print('Test accuracy after calibration:', compile_evaluate(model_quantized))
Test accuracy after calibration: 0.9833999872207642
Calibrating with real samples on this model recovers the initial float accuracy.
2.3. 4-bit quantization
The accuracy of the 8/8/8 quantized model is equal to that of the Keras floating point model. In some cases, a smaller memory size for the model is required. This can be accomplished through quantization of the model to smaller bitwidths.
The model will now be quantized to 8/4/4, that is 8-bit weights in the first layer with 4-bit weights and activations in all other layers. Such a quantization scheme will usually introduce a performance drop.
qparams = QuantizationParams(input_weight_bits=8, weight_bits=4, activation_bits=4)
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 974us/step
Check the 4-bit quantized accuracy.
print('Test accuracy after 4-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 4-bit quantization: 0.9818000197410583
2.4. Model fine tuning (Quantization Aware Training)
When a model suffers from an accuracy drop after quantization, fine tuning or Quantization Aware Training (QAT) may recover some or all of the original performance.
Note that since this is a fine tuning step, both the number of epochs and learning rate are expected to be lower than during the initial float training.
model_quantized.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-4),
metrics=['accuracy'])
model_quantized.fit(x_train, y_train, epochs=5, validation_split=0.1)
Epoch 1/5
1/1688 [..............................] - ETA: 2:07:33 - loss: 0.0017 - accuracy: 1.0000
11/1688 [..............................] - ETA: 8s - loss: 0.0125 - accuracy: 0.9915
22/1688 [..............................] - ETA: 8s - loss: 0.0194 - accuracy: 0.9915
33/1688 [..............................] - ETA: 8s - loss: 0.0213 - accuracy: 0.9924
43/1688 [..............................] - ETA: 8s - loss: 0.0203 - accuracy: 0.9927
54/1688 [..............................] - ETA: 8s - loss: 0.0180 - accuracy: 0.9931
65/1688 [>.............................] - ETA: 8s - loss: 0.0182 - accuracy: 0.9928
76/1688 [>.............................] - ETA: 8s - loss: 0.0193 - accuracy: 0.9926
86/1688 [>.............................] - ETA: 7s - loss: 0.0188 - accuracy: 0.9924
96/1688 [>.............................] - ETA: 7s - loss: 0.0174 - accuracy: 0.9932
107/1688 [>.............................] - ETA: 7s - loss: 0.0173 - accuracy: 0.9933
118/1688 [=>............................] - ETA: 7s - loss: 0.0178 - accuracy: 0.9931
128/1688 [=>............................] - ETA: 7s - loss: 0.0168 - accuracy: 0.9937
139/1688 [=>............................] - ETA: 7s - loss: 0.0170 - accuracy: 0.9937
150/1688 [=>............................] - ETA: 7s - loss: 0.0170 - accuracy: 0.9935
160/1688 [=>............................] - ETA: 7s - loss: 0.0162 - accuracy: 0.9939
170/1688 [==>...........................] - ETA: 7s - loss: 0.0159 - accuracy: 0.9939
181/1688 [==>...........................] - ETA: 7s - loss: 0.0151 - accuracy: 0.9943
191/1688 [==>...........................] - ETA: 7s - loss: 0.0144 - accuracy: 0.9946
201/1688 [==>...........................] - ETA: 7s - loss: 0.0140 - accuracy: 0.9949
211/1688 [==>...........................] - ETA: 7s - loss: 0.0138 - accuracy: 0.9950
222/1688 [==>...........................] - ETA: 7s - loss: 0.0135 - accuracy: 0.9951
232/1688 [===>..........................] - ETA: 7s - loss: 0.0135 - accuracy: 0.9953
243/1688 [===>..........................] - ETA: 7s - loss: 0.0138 - accuracy: 0.9951
254/1688 [===>..........................] - ETA: 7s - loss: 0.0137 - accuracy: 0.9953
264/1688 [===>..........................] - ETA: 7s - loss: 0.0138 - accuracy: 0.9954
275/1688 [===>..........................] - ETA: 7s - loss: 0.0134 - accuracy: 0.9956
286/1688 [====>.........................] - ETA: 6s - loss: 0.0130 - accuracy: 0.9957
296/1688 [====>.........................] - ETA: 6s - loss: 0.0128 - accuracy: 0.9958
307/1688 [====>.........................] - ETA: 6s - loss: 0.0125 - accuracy: 0.9958
318/1688 [====>.........................] - ETA: 6s - loss: 0.0125 - accuracy: 0.9958
328/1688 [====>.........................] - ETA: 6s - loss: 0.0126 - accuracy: 0.9958
339/1688 [=====>........................] - ETA: 6s - loss: 0.0125 - accuracy: 0.9959
350/1688 [=====>........................] - ETA: 6s - loss: 0.0126 - accuracy: 0.9958
360/1688 [=====>........................] - ETA: 6s - loss: 0.0129 - accuracy: 0.9956
370/1688 [=====>........................] - ETA: 6s - loss: 0.0129 - accuracy: 0.9957
381/1688 [=====>........................] - ETA: 6s - loss: 0.0130 - accuracy: 0.9957
392/1688 [=====>........................] - ETA: 6s - loss: 0.0129 - accuracy: 0.9957
402/1688 [======>.......................] - ETA: 6s - loss: 0.0131 - accuracy: 0.9956
413/1688 [======>.......................] - ETA: 6s - loss: 0.0130 - accuracy: 0.9957
424/1688 [======>.......................] - ETA: 6s - loss: 0.0127 - accuracy: 0.9958
435/1688 [======>.......................] - ETA: 6s - loss: 0.0127 - accuracy: 0.9958
445/1688 [======>.......................] - ETA: 6s - loss: 0.0127 - accuracy: 0.9958
455/1688 [=======>......................] - ETA: 6s - loss: 0.0125 - accuracy: 0.9959
466/1688 [=======>......................] - ETA: 6s - loss: 0.0123 - accuracy: 0.9960
477/1688 [=======>......................] - ETA: 6s - loss: 0.0122 - accuracy: 0.9960
487/1688 [=======>......................] - ETA: 5s - loss: 0.0122 - accuracy: 0.9960
497/1688 [=======>......................] - ETA: 5s - loss: 0.0120 - accuracy: 0.9960
508/1688 [========>.....................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9961
519/1688 [========>.....................] - ETA: 5s - loss: 0.0122 - accuracy: 0.9961
530/1688 [========>.....................] - ETA: 5s - loss: 0.0121 - accuracy: 0.9962
540/1688 [========>.....................] - ETA: 5s - loss: 0.0122 - accuracy: 0.9961
551/1688 [========>.....................] - ETA: 5s - loss: 0.0120 - accuracy: 0.9962
562/1688 [========>.....................] - ETA: 5s - loss: 0.0120 - accuracy: 0.9962
573/1688 [=========>....................] - ETA: 5s - loss: 0.0121 - accuracy: 0.9962
584/1688 [=========>....................] - ETA: 5s - loss: 0.0121 - accuracy: 0.9961
594/1688 [=========>....................] - ETA: 5s - loss: 0.0120 - accuracy: 0.9962
604/1688 [=========>....................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9962
614/1688 [=========>....................] - ETA: 5s - loss: 0.0118 - accuracy: 0.9963
625/1688 [==========>...................] - ETA: 5s - loss: 0.0116 - accuracy: 0.9963
636/1688 [==========>...................] - ETA: 5s - loss: 0.0116 - accuracy: 0.9963
646/1688 [==========>...................] - ETA: 5s - loss: 0.0116 - accuracy: 0.9963
657/1688 [==========>...................] - ETA: 5s - loss: 0.0117 - accuracy: 0.9963
668/1688 [==========>...................] - ETA: 5s - loss: 0.0117 - accuracy: 0.9963
679/1688 [===========>..................] - ETA: 5s - loss: 0.0122 - accuracy: 0.9962
690/1688 [===========>..................] - ETA: 4s - loss: 0.0121 - accuracy: 0.9962
700/1688 [===========>..................] - ETA: 4s - loss: 0.0121 - accuracy: 0.9962
710/1688 [===========>..................] - ETA: 4s - loss: 0.0121 - accuracy: 0.9962
721/1688 [===========>..................] - ETA: 4s - loss: 0.0120 - accuracy: 0.9962
732/1688 [============>.................] - ETA: 4s - loss: 0.0120 - accuracy: 0.9962
743/1688 [============>.................] - ETA: 4s - loss: 0.0119 - accuracy: 0.9963
754/1688 [============>.................] - ETA: 4s - loss: 0.0118 - accuracy: 0.9963
764/1688 [============>.................] - ETA: 4s - loss: 0.0118 - accuracy: 0.9963
775/1688 [============>.................] - ETA: 4s - loss: 0.0117 - accuracy: 0.9963
785/1688 [============>.................] - ETA: 4s - loss: 0.0117 - accuracy: 0.9963
796/1688 [=============>................] - ETA: 4s - loss: 0.0118 - accuracy: 0.9963
806/1688 [=============>................] - ETA: 4s - loss: 0.0117 - accuracy: 0.9963
817/1688 [=============>................] - ETA: 4s - loss: 0.0117 - accuracy: 0.9963
828/1688 [=============>................] - ETA: 4s - loss: 0.0117 - accuracy: 0.9963
838/1688 [=============>................] - ETA: 4s - loss: 0.0117 - accuracy: 0.9963
848/1688 [==============>...............] - ETA: 4s - loss: 0.0117 - accuracy: 0.9963
859/1688 [==============>...............] - ETA: 4s - loss: 0.0116 - accuracy: 0.9964
869/1688 [==============>...............] - ETA: 4s - loss: 0.0115 - accuracy: 0.9964
879/1688 [==============>...............] - ETA: 4s - loss: 0.0117 - accuracy: 0.9963
890/1688 [==============>...............] - ETA: 3s - loss: 0.0116 - accuracy: 0.9963
901/1688 [===============>..............] - ETA: 3s - loss: 0.0116 - accuracy: 0.9963
912/1688 [===============>..............] - ETA: 3s - loss: 0.0116 - accuracy: 0.9963
923/1688 [===============>..............] - ETA: 3s - loss: 0.0115 - accuracy: 0.9963
934/1688 [===============>..............] - ETA: 3s - loss: 0.0114 - accuracy: 0.9964
945/1688 [===============>..............] - ETA: 3s - loss: 0.0114 - accuracy: 0.9964
955/1688 [===============>..............] - ETA: 3s - loss: 0.0114 - accuracy: 0.9964
966/1688 [================>.............] - ETA: 3s - loss: 0.0113 - accuracy: 0.9964
977/1688 [================>.............] - ETA: 3s - loss: 0.0113 - accuracy: 0.9964
988/1688 [================>.............] - ETA: 3s - loss: 0.0112 - accuracy: 0.9965
998/1688 [================>.............] - ETA: 3s - loss: 0.0112 - accuracy: 0.9965
1008/1688 [================>.............] - ETA: 3s - loss: 0.0111 - accuracy: 0.9965
1018/1688 [=================>............] - ETA: 3s - loss: 0.0112 - accuracy: 0.9965
1028/1688 [=================>............] - ETA: 3s - loss: 0.0111 - accuracy: 0.9965
1039/1688 [=================>............] - ETA: 3s - loss: 0.0111 - accuracy: 0.9965
1050/1688 [=================>............] - ETA: 3s - loss: 0.0111 - accuracy: 0.9965
1060/1688 [=================>............] - ETA: 3s - loss: 0.0111 - accuracy: 0.9965
1071/1688 [==================>...........] - ETA: 3s - loss: 0.0111 - accuracy: 0.9965
1082/1688 [==================>...........] - ETA: 3s - loss: 0.0111 - accuracy: 0.9965
1093/1688 [==================>...........] - ETA: 2s - loss: 0.0110 - accuracy: 0.9965
1103/1688 [==================>...........] - ETA: 2s - loss: 0.0111 - accuracy: 0.9965
1114/1688 [==================>...........] - ETA: 2s - loss: 0.0111 - accuracy: 0.9964
1125/1688 [==================>...........] - ETA: 2s - loss: 0.0111 - accuracy: 0.9964
1136/1688 [===================>..........] - ETA: 2s - loss: 0.0110 - accuracy: 0.9965
1146/1688 [===================>..........] - ETA: 2s - loss: 0.0110 - accuracy: 0.9965
1157/1688 [===================>..........] - ETA: 2s - loss: 0.0110 - accuracy: 0.9965
1168/1688 [===================>..........] - ETA: 2s - loss: 0.0109 - accuracy: 0.9965
1179/1688 [===================>..........] - ETA: 2s - loss: 0.0109 - accuracy: 0.9965
1189/1688 [====================>.........] - ETA: 2s - loss: 0.0109 - accuracy: 0.9965
1200/1688 [====================>.........] - ETA: 2s - loss: 0.0108 - accuracy: 0.9965
1210/1688 [====================>.........] - ETA: 2s - loss: 0.0108 - accuracy: 0.9965
1220/1688 [====================>.........] - ETA: 2s - loss: 0.0107 - accuracy: 0.9965
1231/1688 [====================>.........] - ETA: 2s - loss: 0.0107 - accuracy: 0.9966
1242/1688 [=====================>........] - ETA: 2s - loss: 0.0107 - accuracy: 0.9966
1253/1688 [=====================>........] - ETA: 2s - loss: 0.0107 - accuracy: 0.9966
1263/1688 [=====================>........] - ETA: 2s - loss: 0.0107 - accuracy: 0.9965
1274/1688 [=====================>........] - ETA: 2s - loss: 0.0107 - accuracy: 0.9965
1284/1688 [=====================>........] - ETA: 2s - loss: 0.0107 - accuracy: 0.9966
1295/1688 [======================>.......] - ETA: 1s - loss: 0.0107 - accuracy: 0.9966
1306/1688 [======================>.......] - ETA: 1s - loss: 0.0106 - accuracy: 0.9966
1316/1688 [======================>.......] - ETA: 1s - loss: 0.0106 - accuracy: 0.9966
1326/1688 [======================>.......] - ETA: 1s - loss: 0.0105 - accuracy: 0.9967
1337/1688 [======================>.......] - ETA: 1s - loss: 0.0105 - accuracy: 0.9967
1347/1688 [======================>.......] - ETA: 1s - loss: 0.0106 - accuracy: 0.9966
1357/1688 [=======================>......] - ETA: 1s - loss: 0.0105 - accuracy: 0.9966
1367/1688 [=======================>......] - ETA: 1s - loss: 0.0105 - accuracy: 0.9966
1377/1688 [=======================>......] - ETA: 1s - loss: 0.0105 - accuracy: 0.9966
1388/1688 [=======================>......] - ETA: 1s - loss: 0.0106 - accuracy: 0.9966
1399/1688 [=======================>......] - ETA: 1s - loss: 0.0106 - accuracy: 0.9965
1410/1688 [========================>.....] - ETA: 1s - loss: 0.0106 - accuracy: 0.9965
1421/1688 [========================>.....] - ETA: 1s - loss: 0.0106 - accuracy: 0.9965
1431/1688 [========================>.....] - ETA: 1s - loss: 0.0105 - accuracy: 0.9965
1442/1688 [========================>.....] - ETA: 1s - loss: 0.0106 - accuracy: 0.9965
1453/1688 [========================>.....] - ETA: 1s - loss: 0.0105 - accuracy: 0.9965
1464/1688 [=========================>....] - ETA: 1s - loss: 0.0105 - accuracy: 0.9965
1475/1688 [=========================>....] - ETA: 1s - loss: 0.0105 - accuracy: 0.9965
1486/1688 [=========================>....] - ETA: 1s - loss: 0.0105 - accuracy: 0.9965
1497/1688 [=========================>....] - ETA: 0s - loss: 0.0105 - accuracy: 0.9965
1508/1688 [=========================>....] - ETA: 0s - loss: 0.0105 - accuracy: 0.9965
1519/1688 [=========================>....] - ETA: 0s - loss: 0.0105 - accuracy: 0.9965
1529/1688 [==========================>...] - ETA: 0s - loss: 0.0105 - accuracy: 0.9965
1539/1688 [==========================>...] - ETA: 0s - loss: 0.0105 - accuracy: 0.9965
1550/1688 [==========================>...] - ETA: 0s - loss: 0.0104 - accuracy: 0.9965
1561/1688 [==========================>...] - ETA: 0s - loss: 0.0104 - accuracy: 0.9965
1571/1688 [==========================>...] - ETA: 0s - loss: 0.0104 - accuracy: 0.9965
1582/1688 [===========================>..] - ETA: 0s - loss: 0.0103 - accuracy: 0.9965
1592/1688 [===========================>..] - ETA: 0s - loss: 0.0103 - accuracy: 0.9966
1603/1688 [===========================>..] - ETA: 0s - loss: 0.0102 - accuracy: 0.9966
1613/1688 [===========================>..] - ETA: 0s - loss: 0.0103 - accuracy: 0.9966
1623/1688 [===========================>..] - ETA: 0s - loss: 0.0102 - accuracy: 0.9966
1633/1688 [============================>.] - ETA: 0s - loss: 0.0102 - accuracy: 0.9966
1644/1688 [============================>.] - ETA: 0s - loss: 0.0102 - accuracy: 0.9966
1654/1688 [============================>.] - ETA: 0s - loss: 0.0101 - accuracy: 0.9966
1665/1688 [============================>.] - ETA: 0s - loss: 0.0101 - accuracy: 0.9967
1676/1688 [============================>.] - ETA: 0s - loss: 0.0101 - accuracy: 0.9967
1686/1688 [============================>.] - ETA: 0s - loss: 0.0101 - accuracy: 0.9967
1688/1688 [==============================] - 15s 6ms/step - loss: 0.0101 - accuracy: 0.9967 - val_loss: 0.0523 - val_accuracy: 0.9882
Epoch 2/5
1/1688 [..............................] - ETA: 8s - loss: 0.0019 - accuracy: 1.0000
12/1688 [..............................] - ETA: 8s - loss: 0.0029 - accuracy: 1.0000
23/1688 [..............................] - ETA: 8s - loss: 0.0036 - accuracy: 1.0000
33/1688 [..............................] - ETA: 8s - loss: 0.0091 - accuracy: 0.9981
44/1688 [..............................] - ETA: 8s - loss: 0.0073 - accuracy: 0.9986
54/1688 [..............................] - ETA: 8s - loss: 0.0065 - accuracy: 0.9988
65/1688 [>.............................] - ETA: 8s - loss: 0.0073 - accuracy: 0.9976
76/1688 [>.............................] - ETA: 8s - loss: 0.0066 - accuracy: 0.9979
87/1688 [>.............................] - ETA: 7s - loss: 0.0064 - accuracy: 0.9978
98/1688 [>.............................] - ETA: 7s - loss: 0.0070 - accuracy: 0.9974
108/1688 [>.............................] - ETA: 7s - loss: 0.0067 - accuracy: 0.9977
118/1688 [=>............................] - ETA: 7s - loss: 0.0065 - accuracy: 0.9979
129/1688 [=>............................] - ETA: 7s - loss: 0.0062 - accuracy: 0.9981
139/1688 [=>............................] - ETA: 7s - loss: 0.0061 - accuracy: 0.9982
150/1688 [=>............................] - ETA: 7s - loss: 0.0060 - accuracy: 0.9983
161/1688 [=>............................] - ETA: 7s - loss: 0.0064 - accuracy: 0.9981
172/1688 [==>...........................] - ETA: 7s - loss: 0.0062 - accuracy: 0.9982
182/1688 [==>...........................] - ETA: 7s - loss: 0.0060 - accuracy: 0.9983
193/1688 [==>...........................] - ETA: 7s - loss: 0.0059 - accuracy: 0.9984
204/1688 [==>...........................] - ETA: 7s - loss: 0.0058 - accuracy: 0.9985
215/1688 [==>...........................] - ETA: 7s - loss: 0.0059 - accuracy: 0.9983
226/1688 [===>..........................] - ETA: 7s - loss: 0.0062 - accuracy: 0.9982
237/1688 [===>..........................] - ETA: 7s - loss: 0.0061 - accuracy: 0.9983
248/1688 [===>..........................] - ETA: 7s - loss: 0.0062 - accuracy: 0.9982
258/1688 [===>..........................] - ETA: 7s - loss: 0.0061 - accuracy: 0.9983
269/1688 [===>..........................] - ETA: 7s - loss: 0.0063 - accuracy: 0.9983
280/1688 [===>..........................] - ETA: 7s - loss: 0.0062 - accuracy: 0.9983
290/1688 [====>.........................] - ETA: 6s - loss: 0.0061 - accuracy: 0.9984
300/1688 [====>.........................] - ETA: 6s - loss: 0.0063 - accuracy: 0.9983
310/1688 [====>.........................] - ETA: 6s - loss: 0.0065 - accuracy: 0.9982
320/1688 [====>.........................] - ETA: 6s - loss: 0.0065 - accuracy: 0.9981
330/1688 [====>.........................] - ETA: 6s - loss: 0.0065 - accuracy: 0.9982
341/1688 [=====>........................] - ETA: 6s - loss: 0.0065 - accuracy: 0.9983
351/1688 [=====>........................] - ETA: 6s - loss: 0.0064 - accuracy: 0.9983
361/1688 [=====>........................] - ETA: 6s - loss: 0.0065 - accuracy: 0.9983
372/1688 [=====>........................] - ETA: 6s - loss: 0.0065 - accuracy: 0.9983
383/1688 [=====>........................] - ETA: 6s - loss: 0.0064 - accuracy: 0.9984
393/1688 [=====>........................] - ETA: 6s - loss: 0.0064 - accuracy: 0.9983
403/1688 [======>.......................] - ETA: 6s - loss: 0.0065 - accuracy: 0.9982
413/1688 [======>.......................] - ETA: 6s - loss: 0.0069 - accuracy: 0.9981
423/1688 [======>.......................] - ETA: 6s - loss: 0.0069 - accuracy: 0.9981
433/1688 [======>.......................] - ETA: 6s - loss: 0.0067 - accuracy: 0.9981
444/1688 [======>.......................] - ETA: 6s - loss: 0.0067 - accuracy: 0.9981
455/1688 [=======>......................] - ETA: 6s - loss: 0.0067 - accuracy: 0.9981
466/1688 [=======>......................] - ETA: 6s - loss: 0.0068 - accuracy: 0.9981
476/1688 [=======>......................] - ETA: 6s - loss: 0.0068 - accuracy: 0.9980
487/1688 [=======>......................] - ETA: 6s - loss: 0.0068 - accuracy: 0.9981
497/1688 [=======>......................] - ETA: 5s - loss: 0.0068 - accuracy: 0.9981
508/1688 [========>.....................] - ETA: 5s - loss: 0.0067 - accuracy: 0.9981
518/1688 [========>.....................] - ETA: 5s - loss: 0.0067 - accuracy: 0.9981
529/1688 [========>.....................] - ETA: 5s - loss: 0.0066 - accuracy: 0.9981
540/1688 [========>.....................] - ETA: 5s - loss: 0.0066 - accuracy: 0.9981
550/1688 [========>.....................] - ETA: 5s - loss: 0.0065 - accuracy: 0.9981
561/1688 [========>.....................] - ETA: 5s - loss: 0.0065 - accuracy: 0.9981
571/1688 [=========>....................] - ETA: 5s - loss: 0.0065 - accuracy: 0.9981
582/1688 [=========>....................] - ETA: 5s - loss: 0.0065 - accuracy: 0.9981
592/1688 [=========>....................] - ETA: 5s - loss: 0.0065 - accuracy: 0.9982
603/1688 [=========>....................] - ETA: 5s - loss: 0.0064 - accuracy: 0.9982
614/1688 [=========>....................] - ETA: 5s - loss: 0.0064 - accuracy: 0.9982
625/1688 [==========>...................] - ETA: 5s - loss: 0.0064 - accuracy: 0.9983
636/1688 [==========>...................] - ETA: 5s - loss: 0.0064 - accuracy: 0.9983
646/1688 [==========>...................] - ETA: 5s - loss: 0.0064 - accuracy: 0.9983
656/1688 [==========>...................] - ETA: 5s - loss: 0.0064 - accuracy: 0.9983
667/1688 [==========>...................] - ETA: 5s - loss: 0.0064 - accuracy: 0.9983
678/1688 [===========>..................] - ETA: 5s - loss: 0.0063 - accuracy: 0.9983
688/1688 [===========>..................] - ETA: 4s - loss: 0.0063 - accuracy: 0.9983
698/1688 [===========>..................] - ETA: 4s - loss: 0.0062 - accuracy: 0.9983
708/1688 [===========>..................] - ETA: 4s - loss: 0.0062 - accuracy: 0.9983
718/1688 [===========>..................] - ETA: 4s - loss: 0.0061 - accuracy: 0.9983
728/1688 [===========>..................] - ETA: 4s - loss: 0.0061 - accuracy: 0.9984
738/1688 [============>.................] - ETA: 4s - loss: 0.0061 - accuracy: 0.9983
749/1688 [============>.................] - ETA: 4s - loss: 0.0063 - accuracy: 0.9983
759/1688 [============>.................] - ETA: 4s - loss: 0.0062 - accuracy: 0.9983
769/1688 [============>.................] - ETA: 4s - loss: 0.0062 - accuracy: 0.9983
780/1688 [============>.................] - ETA: 4s - loss: 0.0063 - accuracy: 0.9983
790/1688 [=============>................] - ETA: 4s - loss: 0.0063 - accuracy: 0.9983
801/1688 [=============>................] - ETA: 4s - loss: 0.0064 - accuracy: 0.9983
811/1688 [=============>................] - ETA: 4s - loss: 0.0064 - accuracy: 0.9983
821/1688 [=============>................] - ETA: 4s - loss: 0.0064 - accuracy: 0.9983
832/1688 [=============>................] - ETA: 4s - loss: 0.0063 - accuracy: 0.9983
843/1688 [=============>................] - ETA: 4s - loss: 0.0063 - accuracy: 0.9983
854/1688 [==============>...............] - ETA: 4s - loss: 0.0063 - accuracy: 0.9984
864/1688 [==============>...............] - ETA: 4s - loss: 0.0062 - accuracy: 0.9984
874/1688 [==============>...............] - ETA: 4s - loss: 0.0063 - accuracy: 0.9983
885/1688 [==============>...............] - ETA: 4s - loss: 0.0063 - accuracy: 0.9983
896/1688 [==============>...............] - ETA: 3s - loss: 0.0063 - accuracy: 0.9983
907/1688 [===============>..............] - ETA: 3s - loss: 0.0063 - accuracy: 0.9983
917/1688 [===============>..............] - ETA: 3s - loss: 0.0062 - accuracy: 0.9984
928/1688 [===============>..............] - ETA: 3s - loss: 0.0062 - accuracy: 0.9984
939/1688 [===============>..............] - ETA: 3s - loss: 0.0061 - accuracy: 0.9984
950/1688 [===============>..............] - ETA: 3s - loss: 0.0062 - accuracy: 0.9984
960/1688 [================>.............] - ETA: 3s - loss: 0.0062 - accuracy: 0.9984
970/1688 [================>.............] - ETA: 3s - loss: 0.0062 - accuracy: 0.9984
980/1688 [================>.............] - ETA: 3s - loss: 0.0062 - accuracy: 0.9984
990/1688 [================>.............] - ETA: 3s - loss: 0.0063 - accuracy: 0.9984
1000/1688 [================>.............] - ETA: 3s - loss: 0.0063 - accuracy: 0.9984
1010/1688 [================>.............] - ETA: 3s - loss: 0.0062 - accuracy: 0.9984
1020/1688 [=================>............] - ETA: 3s - loss: 0.0062 - accuracy: 0.9984
1031/1688 [=================>............] - ETA: 3s - loss: 0.0062 - accuracy: 0.9984
1041/1688 [=================>............] - ETA: 3s - loss: 0.0061 - accuracy: 0.9984
1052/1688 [=================>............] - ETA: 3s - loss: 0.0061 - accuracy: 0.9985
1063/1688 [=================>............] - ETA: 3s - loss: 0.0061 - accuracy: 0.9984
1073/1688 [==================>...........] - ETA: 3s - loss: 0.0061 - accuracy: 0.9985
1084/1688 [==================>...........] - ETA: 3s - loss: 0.0062 - accuracy: 0.9984
1095/1688 [==================>...........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9985
1106/1688 [==================>...........] - ETA: 2s - loss: 0.0062 - accuracy: 0.9984
1117/1688 [==================>...........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9985
1127/1688 [===================>..........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9984
1137/1688 [===================>..........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9985
1148/1688 [===================>..........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9985
1159/1688 [===================>..........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9985
1169/1688 [===================>..........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9985
1180/1688 [===================>..........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9985
1191/1688 [====================>.........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9985
1201/1688 [====================>.........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9985
1211/1688 [====================>.........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9985
1221/1688 [====================>.........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9985
1231/1688 [====================>.........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9985
1242/1688 [=====================>........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9985
1252/1688 [=====================>........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9985
1262/1688 [=====================>........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9985
1272/1688 [=====================>........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9985
1283/1688 [=====================>........] - ETA: 2s - loss: 0.0062 - accuracy: 0.9985
1294/1688 [=====================>........] - ETA: 1s - loss: 0.0062 - accuracy: 0.9985
1305/1688 [======================>.......] - ETA: 1s - loss: 0.0061 - accuracy: 0.9985
1316/1688 [======================>.......] - ETA: 1s - loss: 0.0061 - accuracy: 0.9985
1326/1688 [======================>.......] - ETA: 1s - loss: 0.0061 - accuracy: 0.9985
1336/1688 [======================>.......] - ETA: 1s - loss: 0.0061 - accuracy: 0.9985
1347/1688 [======================>.......] - ETA: 1s - loss: 0.0061 - accuracy: 0.9985
1357/1688 [=======================>......] - ETA: 1s - loss: 0.0061 - accuracy: 0.9985
1367/1688 [=======================>......] - ETA: 1s - loss: 0.0061 - accuracy: 0.9985
1378/1688 [=======================>......] - ETA: 1s - loss: 0.0061 - accuracy: 0.9985
1388/1688 [=======================>......] - ETA: 1s - loss: 0.0061 - accuracy: 0.9985
1398/1688 [=======================>......] - ETA: 1s - loss: 0.0061 - accuracy: 0.9985
1408/1688 [========================>.....] - ETA: 1s - loss: 0.0061 - accuracy: 0.9985
1418/1688 [========================>.....] - ETA: 1s - loss: 0.0061 - accuracy: 0.9985
1428/1688 [========================>.....] - ETA: 1s - loss: 0.0061 - accuracy: 0.9985
1438/1688 [========================>.....] - ETA: 1s - loss: 0.0061 - accuracy: 0.9985
1449/1688 [========================>.....] - ETA: 1s - loss: 0.0060 - accuracy: 0.9985
1460/1688 [========================>.....] - ETA: 1s - loss: 0.0060 - accuracy: 0.9985
1470/1688 [=========================>....] - ETA: 1s - loss: 0.0060 - accuracy: 0.9985
1481/1688 [=========================>....] - ETA: 1s - loss: 0.0060 - accuracy: 0.9985
1492/1688 [=========================>....] - ETA: 0s - loss: 0.0059 - accuracy: 0.9986
1503/1688 [=========================>....] - ETA: 0s - loss: 0.0060 - accuracy: 0.9985
1514/1688 [=========================>....] - ETA: 0s - loss: 0.0059 - accuracy: 0.9985
1524/1688 [==========================>...] - ETA: 0s - loss: 0.0059 - accuracy: 0.9985
1534/1688 [==========================>...] - ETA: 0s - loss: 0.0059 - accuracy: 0.9985
1544/1688 [==========================>...] - ETA: 0s - loss: 0.0059 - accuracy: 0.9985
1555/1688 [==========================>...] - ETA: 0s - loss: 0.0059 - accuracy: 0.9985
1566/1688 [==========================>...] - ETA: 0s - loss: 0.0060 - accuracy: 0.9985
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0060 - accuracy: 0.9985
1587/1688 [===========================>..] - ETA: 0s - loss: 0.0061 - accuracy: 0.9985
1597/1688 [===========================>..] - ETA: 0s - loss: 0.0061 - accuracy: 0.9985
1607/1688 [===========================>..] - ETA: 0s - loss: 0.0061 - accuracy: 0.9985
1618/1688 [===========================>..] - ETA: 0s - loss: 0.0061 - accuracy: 0.9985
1629/1688 [===========================>..] - ETA: 0s - loss: 0.0061 - accuracy: 0.9985
1639/1688 [============================>.] - ETA: 0s - loss: 0.0061 - accuracy: 0.9985
1650/1688 [============================>.] - ETA: 0s - loss: 0.0061 - accuracy: 0.9985
1661/1688 [============================>.] - ETA: 0s - loss: 0.0062 - accuracy: 0.9984
1672/1688 [============================>.] - ETA: 0s - loss: 0.0063 - accuracy: 0.9984
1682/1688 [============================>.] - ETA: 0s - loss: 0.0063 - accuracy: 0.9984
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0063 - accuracy: 0.9984 - val_loss: 0.0499 - val_accuracy: 0.9892
Epoch 3/5
1/1688 [..............................] - ETA: 8s - loss: 2.8948e-04 - accuracy: 1.0000
12/1688 [..............................] - ETA: 8s - loss: 0.0026 - accuracy: 1.0000
22/1688 [..............................] - ETA: 8s - loss: 0.0034 - accuracy: 1.0000
32/1688 [..............................] - ETA: 8s - loss: 0.0036 - accuracy: 1.0000
42/1688 [..............................] - ETA: 8s - loss: 0.0029 - accuracy: 1.0000
52/1688 [..............................] - ETA: 8s - loss: 0.0035 - accuracy: 0.9994
62/1688 [>.............................] - ETA: 8s - loss: 0.0036 - accuracy: 0.9995
73/1688 [>.............................] - ETA: 8s - loss: 0.0037 - accuracy: 0.9991
84/1688 [>.............................] - ETA: 8s - loss: 0.0044 - accuracy: 0.9989
95/1688 [>.............................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9990
105/1688 [>.............................] - ETA: 7s - loss: 0.0047 - accuracy: 0.9988
115/1688 [=>............................] - ETA: 7s - loss: 0.0046 - accuracy: 0.9989
125/1688 [=>............................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9987
136/1688 [=>............................] - ETA: 7s - loss: 0.0052 - accuracy: 0.9989
147/1688 [=>............................] - ETA: 7s - loss: 0.0052 - accuracy: 0.9989
157/1688 [=>............................] - ETA: 7s - loss: 0.0051 - accuracy: 0.9990
168/1688 [=>............................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9987
179/1688 [==>...........................] - ETA: 7s - loss: 0.0052 - accuracy: 0.9988
190/1688 [==>...........................] - ETA: 7s - loss: 0.0052 - accuracy: 0.9988
201/1688 [==>...........................] - ETA: 7s - loss: 0.0051 - accuracy: 0.9989
212/1688 [==>...........................] - ETA: 7s - loss: 0.0050 - accuracy: 0.9988
222/1688 [==>...........................] - ETA: 7s - loss: 0.0051 - accuracy: 0.9987
232/1688 [===>..........................] - ETA: 7s - loss: 0.0051 - accuracy: 0.9988
243/1688 [===>..........................] - ETA: 7s - loss: 0.0050 - accuracy: 0.9988
253/1688 [===>..........................] - ETA: 7s - loss: 0.0049 - accuracy: 0.9989
264/1688 [===>..........................] - ETA: 7s - loss: 0.0051 - accuracy: 0.9987
275/1688 [===>..........................] - ETA: 7s - loss: 0.0050 - accuracy: 0.9987
285/1688 [====>.........................] - ETA: 7s - loss: 0.0050 - accuracy: 0.9988
296/1688 [====>.........................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9988
307/1688 [====>.........................] - ETA: 6s - loss: 0.0049 - accuracy: 0.9989
318/1688 [====>.........................] - ETA: 6s - loss: 0.0051 - accuracy: 0.9988
328/1688 [====>.........................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9989
339/1688 [=====>........................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9989
349/1688 [=====>........................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9989
360/1688 [=====>........................] - ETA: 6s - loss: 0.0049 - accuracy: 0.9990
371/1688 [=====>........................] - ETA: 6s - loss: 0.0051 - accuracy: 0.9989
381/1688 [=====>........................] - ETA: 6s - loss: 0.0052 - accuracy: 0.9989
392/1688 [=====>........................] - ETA: 6s - loss: 0.0051 - accuracy: 0.9989
402/1688 [======>.......................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9989
412/1688 [======>.......................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9989
423/1688 [======>.......................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9989
434/1688 [======>.......................] - ETA: 6s - loss: 0.0051 - accuracy: 0.9989
444/1688 [======>.......................] - ETA: 6s - loss: 0.0051 - accuracy: 0.9989
454/1688 [=======>......................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9989
465/1688 [=======>......................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9989
476/1688 [=======>......................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9989
487/1688 [=======>......................] - ETA: 6s - loss: 0.0051 - accuracy: 0.9988
498/1688 [=======>......................] - ETA: 5s - loss: 0.0051 - accuracy: 0.9988
508/1688 [========>.....................] - ETA: 5s - loss: 0.0051 - accuracy: 0.9988
519/1688 [========>.....................] - ETA: 5s - loss: 0.0050 - accuracy: 0.9988
529/1688 [========>.....................] - ETA: 5s - loss: 0.0050 - accuracy: 0.9988
539/1688 [========>.....................] - ETA: 5s - loss: 0.0051 - accuracy: 0.9988
550/1688 [========>.....................] - ETA: 5s - loss: 0.0050 - accuracy: 0.9988
560/1688 [========>.....................] - ETA: 5s - loss: 0.0051 - accuracy: 0.9988
571/1688 [=========>....................] - ETA: 5s - loss: 0.0050 - accuracy: 0.9988
581/1688 [=========>....................] - ETA: 5s - loss: 0.0050 - accuracy: 0.9988
592/1688 [=========>....................] - ETA: 5s - loss: 0.0050 - accuracy: 0.9988
603/1688 [=========>....................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
613/1688 [=========>....................] - ETA: 5s - loss: 0.0050 - accuracy: 0.9989
624/1688 [==========>...................] - ETA: 5s - loss: 0.0050 - accuracy: 0.9988
635/1688 [==========>...................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
646/1688 [==========>...................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
657/1688 [==========>...................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
667/1688 [==========>...................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
677/1688 [===========>..................] - ETA: 5s - loss: 0.0048 - accuracy: 0.9989
687/1688 [===========>..................] - ETA: 5s - loss: 0.0048 - accuracy: 0.9989
698/1688 [===========>..................] - ETA: 4s - loss: 0.0048 - accuracy: 0.9989
709/1688 [===========>..................] - ETA: 4s - loss: 0.0049 - accuracy: 0.9988
720/1688 [===========>..................] - ETA: 4s - loss: 0.0049 - accuracy: 0.9988
730/1688 [===========>..................] - ETA: 4s - loss: 0.0049 - accuracy: 0.9988
740/1688 [============>.................] - ETA: 4s - loss: 0.0049 - accuracy: 0.9989
751/1688 [============>.................] - ETA: 4s - loss: 0.0049 - accuracy: 0.9989
761/1688 [============>.................] - ETA: 4s - loss: 0.0049 - accuracy: 0.9989
772/1688 [============>.................] - ETA: 4s - loss: 0.0049 - accuracy: 0.9989
783/1688 [============>.................] - ETA: 4s - loss: 0.0049 - accuracy: 0.9989
793/1688 [=============>................] - ETA: 4s - loss: 0.0048 - accuracy: 0.9989
803/1688 [=============>................] - ETA: 4s - loss: 0.0050 - accuracy: 0.9989
813/1688 [=============>................] - ETA: 4s - loss: 0.0050 - accuracy: 0.9988
824/1688 [=============>................] - ETA: 4s - loss: 0.0052 - accuracy: 0.9988
835/1688 [=============>................] - ETA: 4s - loss: 0.0052 - accuracy: 0.9988
845/1688 [==============>...............] - ETA: 4s - loss: 0.0052 - accuracy: 0.9988
856/1688 [==============>...............] - ETA: 4s - loss: 0.0052 - accuracy: 0.9988
866/1688 [==============>...............] - ETA: 4s - loss: 0.0051 - accuracy: 0.9988
877/1688 [==============>...............] - ETA: 4s - loss: 0.0051 - accuracy: 0.9989
888/1688 [==============>...............] - ETA: 3s - loss: 0.0051 - accuracy: 0.9988
898/1688 [==============>...............] - ETA: 3s - loss: 0.0051 - accuracy: 0.9989
909/1688 [===============>..............] - ETA: 3s - loss: 0.0051 - accuracy: 0.9988
920/1688 [===============>..............] - ETA: 3s - loss: 0.0051 - accuracy: 0.9988
930/1688 [===============>..............] - ETA: 3s - loss: 0.0051 - accuracy: 0.9989
940/1688 [===============>..............] - ETA: 3s - loss: 0.0050 - accuracy: 0.9989
950/1688 [===============>..............] - ETA: 3s - loss: 0.0051 - accuracy: 0.9988
960/1688 [================>.............] - ETA: 3s - loss: 0.0051 - accuracy: 0.9988
970/1688 [================>.............] - ETA: 3s - loss: 0.0051 - accuracy: 0.9988
981/1688 [================>.............] - ETA: 3s - loss: 0.0050 - accuracy: 0.9989
991/1688 [================>.............] - ETA: 3s - loss: 0.0050 - accuracy: 0.9989
1001/1688 [================>.............] - ETA: 3s - loss: 0.0050 - accuracy: 0.9988
1012/1688 [================>.............] - ETA: 3s - loss: 0.0050 - accuracy: 0.9989
1023/1688 [=================>............] - ETA: 3s - loss: 0.0050 - accuracy: 0.9988
1034/1688 [=================>............] - ETA: 3s - loss: 0.0051 - accuracy: 0.9988
1045/1688 [=================>............] - ETA: 3s - loss: 0.0051 - accuracy: 0.9988
1056/1688 [=================>............] - ETA: 3s - loss: 0.0052 - accuracy: 0.9988
1067/1688 [=================>............] - ETA: 3s - loss: 0.0051 - accuracy: 0.9988
1077/1688 [==================>...........] - ETA: 3s - loss: 0.0052 - accuracy: 0.9988
1088/1688 [==================>...........] - ETA: 2s - loss: 0.0052 - accuracy: 0.9988
1098/1688 [==================>...........] - ETA: 2s - loss: 0.0052 - accuracy: 0.9987
1108/1688 [==================>...........] - ETA: 2s - loss: 0.0052 - accuracy: 0.9988
1119/1688 [==================>...........] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1129/1688 [===================>..........] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1139/1688 [===================>..........] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1150/1688 [===================>..........] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1161/1688 [===================>..........] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1171/1688 [===================>..........] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1181/1688 [===================>..........] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1191/1688 [====================>.........] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1202/1688 [====================>.........] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1212/1688 [====================>.........] - ETA: 2s - loss: 0.0051 - accuracy: 0.9988
1223/1688 [====================>.........] - ETA: 2s - loss: 0.0052 - accuracy: 0.9987
1233/1688 [====================>.........] - ETA: 2s - loss: 0.0052 - accuracy: 0.9988
1244/1688 [=====================>........] - ETA: 2s - loss: 0.0052 - accuracy: 0.9987
1255/1688 [=====================>........] - ETA: 2s - loss: 0.0052 - accuracy: 0.9987
1266/1688 [=====================>........] - ETA: 2s - loss: 0.0052 - accuracy: 0.9987
1276/1688 [=====================>........] - ETA: 2s - loss: 0.0052 - accuracy: 0.9987
1286/1688 [=====================>........] - ETA: 2s - loss: 0.0052 - accuracy: 0.9987
1296/1688 [======================>.......] - ETA: 1s - loss: 0.0052 - accuracy: 0.9987
1307/1688 [======================>.......] - ETA: 1s - loss: 0.0053 - accuracy: 0.9987
1317/1688 [======================>.......] - ETA: 1s - loss: 0.0052 - accuracy: 0.9987
1327/1688 [======================>.......] - ETA: 1s - loss: 0.0052 - accuracy: 0.9987
1338/1688 [======================>.......] - ETA: 1s - loss: 0.0052 - accuracy: 0.9987
1348/1688 [======================>.......] - ETA: 1s - loss: 0.0052 - accuracy: 0.9987
1358/1688 [=======================>......] - ETA: 1s - loss: 0.0052 - accuracy: 0.9987
1369/1688 [=======================>......] - ETA: 1s - loss: 0.0053 - accuracy: 0.9987
1379/1688 [=======================>......] - ETA: 1s - loss: 0.0053 - accuracy: 0.9987
1389/1688 [=======================>......] - ETA: 1s - loss: 0.0053 - accuracy: 0.9987
1400/1688 [=======================>......] - ETA: 1s - loss: 0.0053 - accuracy: 0.9987
1410/1688 [========================>.....] - ETA: 1s - loss: 0.0052 - accuracy: 0.9987
1421/1688 [========================>.....] - ETA: 1s - loss: 0.0052 - accuracy: 0.9987
1432/1688 [========================>.....] - ETA: 1s - loss: 0.0053 - accuracy: 0.9987
1442/1688 [========================>.....] - ETA: 1s - loss: 0.0053 - accuracy: 0.9987
1453/1688 [========================>.....] - ETA: 1s - loss: 0.0053 - accuracy: 0.9987
1464/1688 [=========================>....] - ETA: 1s - loss: 0.0053 - accuracy: 0.9987
1475/1688 [=========================>....] - ETA: 1s - loss: 0.0052 - accuracy: 0.9987
1486/1688 [=========================>....] - ETA: 1s - loss: 0.0053 - accuracy: 0.9987
1497/1688 [=========================>....] - ETA: 0s - loss: 0.0053 - accuracy: 0.9987
1508/1688 [=========================>....] - ETA: 0s - loss: 0.0052 - accuracy: 0.9987
1518/1688 [=========================>....] - ETA: 0s - loss: 0.0053 - accuracy: 0.9987
1528/1688 [==========================>...] - ETA: 0s - loss: 0.0053 - accuracy: 0.9987
1539/1688 [==========================>...] - ETA: 0s - loss: 0.0053 - accuracy: 0.9987
1549/1688 [==========================>...] - ETA: 0s - loss: 0.0053 - accuracy: 0.9987
1559/1688 [==========================>...] - ETA: 0s - loss: 0.0053 - accuracy: 0.9987
1570/1688 [==========================>...] - ETA: 0s - loss: 0.0052 - accuracy: 0.9987
1580/1688 [===========================>..] - ETA: 0s - loss: 0.0053 - accuracy: 0.9987
1591/1688 [===========================>..] - ETA: 0s - loss: 0.0052 - accuracy: 0.9987
1602/1688 [===========================>..] - ETA: 0s - loss: 0.0052 - accuracy: 0.9987
1613/1688 [===========================>..] - ETA: 0s - loss: 0.0052 - accuracy: 0.9987
1623/1688 [===========================>..] - ETA: 0s - loss: 0.0052 - accuracy: 0.9987
1634/1688 [============================>.] - ETA: 0s - loss: 0.0052 - accuracy: 0.9987
1645/1688 [============================>.] - ETA: 0s - loss: 0.0052 - accuracy: 0.9987
1655/1688 [============================>.] - ETA: 0s - loss: 0.0052 - accuracy: 0.9987
1665/1688 [============================>.] - ETA: 0s - loss: 0.0052 - accuracy: 0.9987
1675/1688 [============================>.] - ETA: 0s - loss: 0.0052 - accuracy: 0.9987
1685/1688 [============================>.] - ETA: 0s - loss: 0.0052 - accuracy: 0.9987
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0052 - accuracy: 0.9987 - val_loss: 0.0502 - val_accuracy: 0.9885
Epoch 4/5
1/1688 [..............................] - ETA: 8s - loss: 0.0014 - accuracy: 1.0000
12/1688 [..............................] - ETA: 8s - loss: 0.0037 - accuracy: 1.0000
23/1688 [..............................] - ETA: 8s - loss: 0.0033 - accuracy: 1.0000
33/1688 [..............................] - ETA: 8s - loss: 0.0030 - accuracy: 1.0000
43/1688 [..............................] - ETA: 8s - loss: 0.0065 - accuracy: 0.9993
54/1688 [..............................] - ETA: 8s - loss: 0.0057 - accuracy: 0.9994
65/1688 [>.............................] - ETA: 8s - loss: 0.0051 - accuracy: 0.9995
76/1688 [>.............................] - ETA: 8s - loss: 0.0049 - accuracy: 0.9996
87/1688 [>.............................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9996
98/1688 [>.............................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9997
109/1688 [>.............................] - ETA: 7s - loss: 0.0041 - accuracy: 0.9997
120/1688 [=>............................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9995
130/1688 [=>............................] - ETA: 7s - loss: 0.0042 - accuracy: 0.9995
140/1688 [=>............................] - ETA: 7s - loss: 0.0042 - accuracy: 0.9996
151/1688 [=>............................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9996
161/1688 [=>............................] - ETA: 7s - loss: 0.0041 - accuracy: 0.9996
172/1688 [==>...........................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9996
183/1688 [==>...........................] - ETA: 7s - loss: 0.0039 - accuracy: 0.9997
193/1688 [==>...........................] - ETA: 7s - loss: 0.0038 - accuracy: 0.9997
204/1688 [==>...........................] - ETA: 7s - loss: 0.0037 - accuracy: 0.9997
215/1688 [==>...........................] - ETA: 7s - loss: 0.0041 - accuracy: 0.9996
226/1688 [===>..........................] - ETA: 7s - loss: 0.0041 - accuracy: 0.9996
236/1688 [===>..........................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9995
247/1688 [===>..........................] - ETA: 7s - loss: 0.0045 - accuracy: 0.9994
257/1688 [===>..........................] - ETA: 7s - loss: 0.0047 - accuracy: 0.9993
268/1688 [===>..........................] - ETA: 7s - loss: 0.0045 - accuracy: 0.9993
278/1688 [===>..........................] - ETA: 7s - loss: 0.0046 - accuracy: 0.9993
289/1688 [====>.........................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9994
299/1688 [====>.........................] - ETA: 6s - loss: 0.0043 - accuracy: 0.9994
310/1688 [====>.........................] - ETA: 6s - loss: 0.0043 - accuracy: 0.9994
320/1688 [====>.........................] - ETA: 6s - loss: 0.0042 - accuracy: 0.9994
330/1688 [====>.........................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9993
341/1688 [=====>........................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9994
352/1688 [=====>........................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9993
363/1688 [=====>........................] - ETA: 6s - loss: 0.0047 - accuracy: 0.9991
374/1688 [=====>........................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9992
385/1688 [=====>........................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9992
396/1688 [======>.......................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9992
407/1688 [======>.......................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9992
418/1688 [======>.......................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9992
429/1688 [======>.......................] - ETA: 6s - loss: 0.0044 - accuracy: 0.9992
440/1688 [======>.......................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9991
450/1688 [======>.......................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9991
461/1688 [=======>......................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9991
471/1688 [=======>......................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9991
481/1688 [=======>......................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9991
491/1688 [=======>......................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9991
501/1688 [=======>......................] - ETA: 5s - loss: 0.0043 - accuracy: 0.9991
512/1688 [========>.....................] - ETA: 5s - loss: 0.0043 - accuracy: 0.9991
522/1688 [========>.....................] - ETA: 5s - loss: 0.0043 - accuracy: 0.9992
532/1688 [========>.....................] - ETA: 5s - loss: 0.0042 - accuracy: 0.9992
542/1688 [========>.....................] - ETA: 5s - loss: 0.0042 - accuracy: 0.9992
552/1688 [========>.....................] - ETA: 5s - loss: 0.0042 - accuracy: 0.9992
562/1688 [========>.....................] - ETA: 5s - loss: 0.0042 - accuracy: 0.9992
573/1688 [=========>....................] - ETA: 5s - loss: 0.0042 - accuracy: 0.9992
584/1688 [=========>....................] - ETA: 5s - loss: 0.0041 - accuracy: 0.9992
595/1688 [=========>....................] - ETA: 5s - loss: 0.0041 - accuracy: 0.9992
606/1688 [=========>....................] - ETA: 5s - loss: 0.0041 - accuracy: 0.9992
616/1688 [=========>....................] - ETA: 5s - loss: 0.0041 - accuracy: 0.9992
626/1688 [==========>...................] - ETA: 5s - loss: 0.0041 - accuracy: 0.9992
637/1688 [==========>...................] - ETA: 5s - loss: 0.0042 - accuracy: 0.9992
647/1688 [==========>...................] - ETA: 5s - loss: 0.0041 - accuracy: 0.9992
658/1688 [==========>...................] - ETA: 5s - loss: 0.0041 - accuracy: 0.9992
669/1688 [==========>...................] - ETA: 5s - loss: 0.0041 - accuracy: 0.9993
679/1688 [===========>..................] - ETA: 5s - loss: 0.0041 - accuracy: 0.9993
690/1688 [===========>..................] - ETA: 4s - loss: 0.0041 - accuracy: 0.9993
700/1688 [===========>..................] - ETA: 4s - loss: 0.0040 - accuracy: 0.9993
711/1688 [===========>..................] - ETA: 4s - loss: 0.0041 - accuracy: 0.9993
721/1688 [===========>..................] - ETA: 4s - loss: 0.0041 - accuracy: 0.9993
732/1688 [============>.................] - ETA: 4s - loss: 0.0041 - accuracy: 0.9993
742/1688 [============>.................] - ETA: 4s - loss: 0.0041 - accuracy: 0.9993
753/1688 [============>.................] - ETA: 4s - loss: 0.0041 - accuracy: 0.9993
764/1688 [============>.................] - ETA: 4s - loss: 0.0043 - accuracy: 0.9993
774/1688 [============>.................] - ETA: 4s - loss: 0.0043 - accuracy: 0.9993
785/1688 [============>.................] - ETA: 4s - loss: 0.0042 - accuracy: 0.9993
795/1688 [=============>................] - ETA: 4s - loss: 0.0042 - accuracy: 0.9993
805/1688 [=============>................] - ETA: 4s - loss: 0.0043 - accuracy: 0.9992
815/1688 [=============>................] - ETA: 4s - loss: 0.0043 - accuracy: 0.9992
825/1688 [=============>................] - ETA: 4s - loss: 0.0043 - accuracy: 0.9992
835/1688 [=============>................] - ETA: 4s - loss: 0.0043 - accuracy: 0.9993
846/1688 [==============>...............] - ETA: 4s - loss: 0.0043 - accuracy: 0.9992
856/1688 [==============>...............] - ETA: 4s - loss: 0.0043 - accuracy: 0.9992
867/1688 [==============>...............] - ETA: 4s - loss: 0.0044 - accuracy: 0.9992
878/1688 [==============>...............] - ETA: 4s - loss: 0.0044 - accuracy: 0.9991
889/1688 [==============>...............] - ETA: 3s - loss: 0.0043 - accuracy: 0.9992
899/1688 [==============>...............] - ETA: 3s - loss: 0.0043 - accuracy: 0.9992
910/1688 [===============>..............] - ETA: 3s - loss: 0.0043 - accuracy: 0.9991
920/1688 [===============>..............] - ETA: 3s - loss: 0.0043 - accuracy: 0.9991
930/1688 [===============>..............] - ETA: 3s - loss: 0.0043 - accuracy: 0.9991
940/1688 [===============>..............] - ETA: 3s - loss: 0.0043 - accuracy: 0.9991
951/1688 [===============>..............] - ETA: 3s - loss: 0.0043 - accuracy: 0.9991
962/1688 [================>.............] - ETA: 3s - loss: 0.0042 - accuracy: 0.9992
973/1688 [================>.............] - ETA: 3s - loss: 0.0042 - accuracy: 0.9992
983/1688 [================>.............] - ETA: 3s - loss: 0.0042 - accuracy: 0.9992
994/1688 [================>.............] - ETA: 3s - loss: 0.0042 - accuracy: 0.9992
1005/1688 [================>.............] - ETA: 3s - loss: 0.0042 - accuracy: 0.9992
1016/1688 [=================>............] - ETA: 3s - loss: 0.0042 - accuracy: 0.9992
1027/1688 [=================>............] - ETA: 3s - loss: 0.0042 - accuracy: 0.9992
1037/1688 [=================>............] - ETA: 3s - loss: 0.0042 - accuracy: 0.9992
1048/1688 [=================>............] - ETA: 3s - loss: 0.0042 - accuracy: 0.9992
1059/1688 [=================>............] - ETA: 3s - loss: 0.0042 - accuracy: 0.9992
1069/1688 [=================>............] - ETA: 3s - loss: 0.0042 - accuracy: 0.9992
1080/1688 [==================>...........] - ETA: 3s - loss: 0.0043 - accuracy: 0.9991
1090/1688 [==================>...........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9991
1101/1688 [==================>...........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9991
1112/1688 [==================>...........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9991
1122/1688 [==================>...........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9991
1132/1688 [===================>..........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9991
1143/1688 [===================>..........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9991
1154/1688 [===================>..........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9991
1165/1688 [===================>..........] - ETA: 2s - loss: 0.0044 - accuracy: 0.9991
1175/1688 [===================>..........] - ETA: 2s - loss: 0.0044 - accuracy: 0.9990
1185/1688 [====================>.........] - ETA: 2s - loss: 0.0044 - accuracy: 0.9991
1195/1688 [====================>.........] - ETA: 2s - loss: 0.0044 - accuracy: 0.9991
1206/1688 [====================>.........] - ETA: 2s - loss: 0.0044 - accuracy: 0.9991
1217/1688 [====================>.........] - ETA: 2s - loss: 0.0044 - accuracy: 0.9991
1228/1688 [====================>.........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9991
1239/1688 [=====================>........] - ETA: 2s - loss: 0.0044 - accuracy: 0.9991
1249/1688 [=====================>........] - ETA: 2s - loss: 0.0044 - accuracy: 0.9991
1259/1688 [=====================>........] - ETA: 2s - loss: 0.0044 - accuracy: 0.9991
1270/1688 [=====================>........] - ETA: 2s - loss: 0.0044 - accuracy: 0.9991
1281/1688 [=====================>........] - ETA: 2s - loss: 0.0044 - accuracy: 0.9991
1292/1688 [=====================>........] - ETA: 1s - loss: 0.0044 - accuracy: 0.9991
1303/1688 [======================>.......] - ETA: 1s - loss: 0.0044 - accuracy: 0.9991
1313/1688 [======================>.......] - ETA: 1s - loss: 0.0045 - accuracy: 0.9990
1324/1688 [======================>.......] - ETA: 1s - loss: 0.0045 - accuracy: 0.9990
1334/1688 [======================>.......] - ETA: 1s - loss: 0.0045 - accuracy: 0.9990
1344/1688 [======================>.......] - ETA: 1s - loss: 0.0045 - accuracy: 0.9990
1355/1688 [=======================>......] - ETA: 1s - loss: 0.0045 - accuracy: 0.9990
1365/1688 [=======================>......] - ETA: 1s - loss: 0.0045 - accuracy: 0.9990
1376/1688 [=======================>......] - ETA: 1s - loss: 0.0045 - accuracy: 0.9990
1387/1688 [=======================>......] - ETA: 1s - loss: 0.0045 - accuracy: 0.9991
1397/1688 [=======================>......] - ETA: 1s - loss: 0.0045 - accuracy: 0.9991
1408/1688 [========================>.....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9991
1418/1688 [========================>.....] - ETA: 1s - loss: 0.0044 - accuracy: 0.9991
1429/1688 [========================>.....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9991
1439/1688 [========================>.....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9990
1449/1688 [========================>.....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9990
1460/1688 [========================>.....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9990
1471/1688 [=========================>....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9990
1481/1688 [=========================>....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9990
1492/1688 [=========================>....] - ETA: 0s - loss: 0.0045 - accuracy: 0.9990
1503/1688 [=========================>....] - ETA: 0s - loss: 0.0045 - accuracy: 0.9990
1514/1688 [=========================>....] - ETA: 0s - loss: 0.0045 - accuracy: 0.9991
1525/1688 [==========================>...] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1535/1688 [==========================>...] - ETA: 0s - loss: 0.0045 - accuracy: 0.9990
1545/1688 [==========================>...] - ETA: 0s - loss: 0.0044 - accuracy: 0.9990
1555/1688 [==========================>...] - ETA: 0s - loss: 0.0044 - accuracy: 0.9990
1566/1688 [==========================>...] - ETA: 0s - loss: 0.0044 - accuracy: 0.9990
1577/1688 [===========================>..] - ETA: 0s - loss: 0.0044 - accuracy: 0.9990
1587/1688 [===========================>..] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1597/1688 [===========================>..] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1607/1688 [===========================>..] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1618/1688 [===========================>..] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1629/1688 [===========================>..] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1639/1688 [============================>.] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1650/1688 [============================>.] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1661/1688 [============================>.] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1672/1688 [============================>.] - ETA: 0s - loss: 0.0043 - accuracy: 0.9991
1682/1688 [============================>.] - ETA: 0s - loss: 0.0043 - accuracy: 0.9991
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0043 - accuracy: 0.9991 - val_loss: 0.0505 - val_accuracy: 0.9888
Epoch 5/5
1/1688 [..............................] - ETA: 8s - loss: 0.0045 - accuracy: 1.0000
12/1688 [..............................] - ETA: 8s - loss: 0.0034 - accuracy: 1.0000
22/1688 [..............................] - ETA: 8s - loss: 0.0031 - accuracy: 1.0000
32/1688 [..............................] - ETA: 8s - loss: 0.0028 - accuracy: 1.0000
43/1688 [..............................] - ETA: 8s - loss: 0.0025 - accuracy: 1.0000
53/1688 [..............................] - ETA: 8s - loss: 0.0036 - accuracy: 0.9994
63/1688 [>.............................] - ETA: 8s - loss: 0.0036 - accuracy: 0.9995
73/1688 [>.............................] - ETA: 8s - loss: 0.0039 - accuracy: 0.9996
83/1688 [>.............................] - ETA: 8s - loss: 0.0037 - accuracy: 0.9996
94/1688 [>.............................] - ETA: 8s - loss: 0.0037 - accuracy: 0.9997
105/1688 [>.............................] - ETA: 7s - loss: 0.0039 - accuracy: 0.9997
115/1688 [=>............................] - ETA: 7s - loss: 0.0039 - accuracy: 0.9997
126/1688 [=>............................] - ETA: 7s - loss: 0.0038 - accuracy: 0.9998
136/1688 [=>............................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9998
147/1688 [=>............................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9998
158/1688 [=>............................] - ETA: 7s - loss: 0.0037 - accuracy: 0.9996
169/1688 [==>...........................] - ETA: 7s - loss: 0.0038 - accuracy: 0.9994
180/1688 [==>...........................] - ETA: 7s - loss: 0.0039 - accuracy: 0.9993
190/1688 [==>...........................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9993
200/1688 [==>...........................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9994
211/1688 [==>...........................] - ETA: 7s - loss: 0.0039 - accuracy: 0.9994
221/1688 [==>...........................] - ETA: 7s - loss: 0.0038 - accuracy: 0.9994
232/1688 [===>..........................] - ETA: 7s - loss: 0.0040 - accuracy: 0.9993
242/1688 [===>..........................] - ETA: 7s - loss: 0.0039 - accuracy: 0.9994
253/1688 [===>..........................] - ETA: 7s - loss: 0.0039 - accuracy: 0.9994
263/1688 [===>..........................] - ETA: 7s - loss: 0.0038 - accuracy: 0.9994
274/1688 [===>..........................] - ETA: 7s - loss: 0.0037 - accuracy: 0.9994
284/1688 [====>.........................] - ETA: 7s - loss: 0.0037 - accuracy: 0.9994
295/1688 [====>.........................] - ETA: 6s - loss: 0.0037 - accuracy: 0.9995
306/1688 [====>.........................] - ETA: 6s - loss: 0.0036 - accuracy: 0.9995
317/1688 [====>.........................] - ETA: 6s - loss: 0.0036 - accuracy: 0.9995
327/1688 [====>.........................] - ETA: 6s - loss: 0.0036 - accuracy: 0.9995
337/1688 [====>.........................] - ETA: 6s - loss: 0.0036 - accuracy: 0.9995
347/1688 [=====>........................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9995
357/1688 [=====>........................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9996
367/1688 [=====>........................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9996
378/1688 [=====>........................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9996
389/1688 [=====>........................] - ETA: 6s - loss: 0.0036 - accuracy: 0.9994
400/1688 [======>.......................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9995
411/1688 [======>.......................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9995
422/1688 [======>.......................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9994
433/1688 [======>.......................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9994
443/1688 [======>.......................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9994
453/1688 [=======>......................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9994
464/1688 [=======>......................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9995
474/1688 [=======>......................] - ETA: 6s - loss: 0.0034 - accuracy: 0.9995
485/1688 [=======>......................] - ETA: 6s - loss: 0.0034 - accuracy: 0.9995
495/1688 [=======>......................] - ETA: 5s - loss: 0.0034 - accuracy: 0.9994
506/1688 [=======>......................] - ETA: 5s - loss: 0.0034 - accuracy: 0.9994
516/1688 [========>.....................] - ETA: 5s - loss: 0.0034 - accuracy: 0.9995
527/1688 [========>.....................] - ETA: 5s - loss: 0.0035 - accuracy: 0.9994
537/1688 [========>.....................] - ETA: 5s - loss: 0.0035 - accuracy: 0.9994
548/1688 [========>.....................] - ETA: 5s - loss: 0.0035 - accuracy: 0.9994
559/1688 [========>.....................] - ETA: 5s - loss: 0.0035 - accuracy: 0.9994
569/1688 [=========>....................] - ETA: 5s - loss: 0.0035 - accuracy: 0.9994
580/1688 [=========>....................] - ETA: 5s - loss: 0.0036 - accuracy: 0.9994
590/1688 [=========>....................] - ETA: 5s - loss: 0.0035 - accuracy: 0.9994
600/1688 [=========>....................] - ETA: 5s - loss: 0.0035 - accuracy: 0.9994
611/1688 [=========>....................] - ETA: 5s - loss: 0.0036 - accuracy: 0.9993
621/1688 [==========>...................] - ETA: 5s - loss: 0.0036 - accuracy: 0.9993
632/1688 [==========>...................] - ETA: 5s - loss: 0.0036 - accuracy: 0.9993
643/1688 [==========>...................] - ETA: 5s - loss: 0.0036 - accuracy: 0.9993
654/1688 [==========>...................] - ETA: 5s - loss: 0.0037 - accuracy: 0.9992
664/1688 [==========>...................] - ETA: 5s - loss: 0.0038 - accuracy: 0.9992
674/1688 [==========>...................] - ETA: 5s - loss: 0.0038 - accuracy: 0.9992
685/1688 [===========>..................] - ETA: 5s - loss: 0.0038 - accuracy: 0.9992
695/1688 [===========>..................] - ETA: 4s - loss: 0.0038 - accuracy: 0.9992
705/1688 [===========>..................] - ETA: 4s - loss: 0.0038 - accuracy: 0.9992
715/1688 [===========>..................] - ETA: 4s - loss: 0.0038 - accuracy: 0.9992
725/1688 [===========>..................] - ETA: 4s - loss: 0.0038 - accuracy: 0.9992
735/1688 [============>.................] - ETA: 4s - loss: 0.0038 - accuracy: 0.9992
745/1688 [============>.................] - ETA: 4s - loss: 0.0037 - accuracy: 0.9992
755/1688 [============>.................] - ETA: 4s - loss: 0.0037 - accuracy: 0.9993
765/1688 [============>.................] - ETA: 4s - loss: 0.0037 - accuracy: 0.9993
775/1688 [============>.................] - ETA: 4s - loss: 0.0037 - accuracy: 0.9993
785/1688 [============>.................] - ETA: 4s - loss: 0.0037 - accuracy: 0.9993
796/1688 [=============>................] - ETA: 4s - loss: 0.0037 - accuracy: 0.9993
806/1688 [=============>................] - ETA: 4s - loss: 0.0037 - accuracy: 0.9993
817/1688 [=============>................] - ETA: 4s - loss: 0.0037 - accuracy: 0.9993
827/1688 [=============>................] - ETA: 4s - loss: 0.0037 - accuracy: 0.9993
838/1688 [=============>................] - ETA: 4s - loss: 0.0037 - accuracy: 0.9993
849/1688 [==============>...............] - ETA: 4s - loss: 0.0037 - accuracy: 0.9993
859/1688 [==============>...............] - ETA: 4s - loss: 0.0037 - accuracy: 0.9993
870/1688 [==============>...............] - ETA: 4s - loss: 0.0037 - accuracy: 0.9993
880/1688 [==============>...............] - ETA: 4s - loss: 0.0037 - accuracy: 0.9993
890/1688 [==============>...............] - ETA: 3s - loss: 0.0037 - accuracy: 0.9993
900/1688 [==============>...............] - ETA: 3s - loss: 0.0038 - accuracy: 0.9993
910/1688 [===============>..............] - ETA: 3s - loss: 0.0038 - accuracy: 0.9993
921/1688 [===============>..............] - ETA: 3s - loss: 0.0038 - accuracy: 0.9993
931/1688 [===============>..............] - ETA: 3s - loss: 0.0038 - accuracy: 0.9993
942/1688 [===============>..............] - ETA: 3s - loss: 0.0038 - accuracy: 0.9993
952/1688 [===============>..............] - ETA: 3s - loss: 0.0039 - accuracy: 0.9992
962/1688 [================>.............] - ETA: 3s - loss: 0.0039 - accuracy: 0.9993
973/1688 [================>.............] - ETA: 3s - loss: 0.0038 - accuracy: 0.9993
983/1688 [================>.............] - ETA: 3s - loss: 0.0038 - accuracy: 0.9993
994/1688 [================>.............] - ETA: 3s - loss: 0.0038 - accuracy: 0.9992
1005/1688 [================>.............] - ETA: 3s - loss: 0.0039 - accuracy: 0.9993
1016/1688 [=================>............] - ETA: 3s - loss: 0.0039 - accuracy: 0.9992
1027/1688 [=================>............] - ETA: 3s - loss: 0.0039 - accuracy: 0.9992
1037/1688 [=================>............] - ETA: 3s - loss: 0.0039 - accuracy: 0.9992
1048/1688 [=================>............] - ETA: 3s - loss: 0.0039 - accuracy: 0.9993
1059/1688 [=================>............] - ETA: 3s - loss: 0.0038 - accuracy: 0.9993
1069/1688 [=================>............] - ETA: 3s - loss: 0.0039 - accuracy: 0.9993
1079/1688 [==================>...........] - ETA: 3s - loss: 0.0038 - accuracy: 0.9993
1090/1688 [==================>...........] - ETA: 2s - loss: 0.0039 - accuracy: 0.9993
1100/1688 [==================>...........] - ETA: 2s - loss: 0.0039 - accuracy: 0.9993
1111/1688 [==================>...........] - ETA: 2s - loss: 0.0039 - accuracy: 0.9993
1122/1688 [==================>...........] - ETA: 2s - loss: 0.0039 - accuracy: 0.9993
1133/1688 [===================>..........] - ETA: 2s - loss: 0.0039 - accuracy: 0.9993
1144/1688 [===================>..........] - ETA: 2s - loss: 0.0039 - accuracy: 0.9993
1154/1688 [===================>..........] - ETA: 2s - loss: 0.0038 - accuracy: 0.9993
1164/1688 [===================>..........] - ETA: 2s - loss: 0.0038 - accuracy: 0.9993
1175/1688 [===================>..........] - ETA: 2s - loss: 0.0038 - accuracy: 0.9993
1185/1688 [====================>.........] - ETA: 2s - loss: 0.0038 - accuracy: 0.9993
1196/1688 [====================>.........] - ETA: 2s - loss: 0.0038 - accuracy: 0.9993
1207/1688 [====================>.........] - ETA: 2s - loss: 0.0038 - accuracy: 0.9993
1218/1688 [====================>.........] - ETA: 2s - loss: 0.0038 - accuracy: 0.9993
1228/1688 [====================>.........] - ETA: 2s - loss: 0.0038 - accuracy: 0.9993
1238/1688 [=====================>........] - ETA: 2s - loss: 0.0038 - accuracy: 0.9993
1248/1688 [=====================>........] - ETA: 2s - loss: 0.0038 - accuracy: 0.9993
1258/1688 [=====================>........] - ETA: 2s - loss: 0.0038 - accuracy: 0.9993
1268/1688 [=====================>........] - ETA: 2s - loss: 0.0038 - accuracy: 0.9993
1278/1688 [=====================>........] - ETA: 2s - loss: 0.0038 - accuracy: 0.9993
1289/1688 [=====================>........] - ETA: 1s - loss: 0.0038 - accuracy: 0.9993
1300/1688 [======================>.......] - ETA: 1s - loss: 0.0038 - accuracy: 0.9993
1311/1688 [======================>.......] - ETA: 1s - loss: 0.0038 - accuracy: 0.9993
1322/1688 [======================>.......] - ETA: 1s - loss: 0.0038 - accuracy: 0.9993
1332/1688 [======================>.......] - ETA: 1s - loss: 0.0038 - accuracy: 0.9993
1343/1688 [======================>.......] - ETA: 1s - loss: 0.0038 - accuracy: 0.9993
1353/1688 [=======================>......] - ETA: 1s - loss: 0.0038 - accuracy: 0.9993
1363/1688 [=======================>......] - ETA: 1s - loss: 0.0038 - accuracy: 0.9993
1373/1688 [=======================>......] - ETA: 1s - loss: 0.0038 - accuracy: 0.9993
1383/1688 [=======================>......] - ETA: 1s - loss: 0.0038 - accuracy: 0.9993
1394/1688 [=======================>......] - ETA: 1s - loss: 0.0038 - accuracy: 0.9992
1405/1688 [=======================>......] - ETA: 1s - loss: 0.0038 - accuracy: 0.9992
1416/1688 [========================>.....] - ETA: 1s - loss: 0.0038 - accuracy: 0.9992
1426/1688 [========================>.....] - ETA: 1s - loss: 0.0038 - accuracy: 0.9993
1437/1688 [========================>.....] - ETA: 1s - loss: 0.0038 - accuracy: 0.9992
1447/1688 [========================>.....] - ETA: 1s - loss: 0.0038 - accuracy: 0.9992
1457/1688 [========================>.....] - ETA: 1s - loss: 0.0038 - accuracy: 0.9992
1468/1688 [=========================>....] - ETA: 1s - loss: 0.0038 - accuracy: 0.9992
1478/1688 [=========================>....] - ETA: 1s - loss: 0.0038 - accuracy: 0.9992
1488/1688 [=========================>....] - ETA: 1s - loss: 0.0038 - accuracy: 0.9992
1499/1688 [=========================>....] - ETA: 0s - loss: 0.0038 - accuracy: 0.9992
1509/1688 [=========================>....] - ETA: 0s - loss: 0.0038 - accuracy: 0.9992
1519/1688 [=========================>....] - ETA: 0s - loss: 0.0038 - accuracy: 0.9992
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1540/1688 [==========================>...] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1550/1688 [==========================>...] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1560/1688 [==========================>...] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1571/1688 [==========================>...] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1581/1688 [===========================>..] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1591/1688 [===========================>..] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1602/1688 [===========================>..] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1613/1688 [===========================>..] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1624/1688 [===========================>..] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1635/1688 [============================>.] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1646/1688 [============================>.] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1656/1688 [============================>.] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1666/1688 [============================>.] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1677/1688 [============================>.] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1688/1688 [==============================] - ETA: 0s - loss: 0.0039 - accuracy: 0.9992
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0039 - accuracy: 0.9992 - val_loss: 0.0509 - val_accuracy: 0.9885
<keras.src.callbacks.History object at 0x7fe414001490>
score = model_quantized.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after fine tuning:', score)
Test accuracy after fine tuning: 0.9854999780654907
3. Convert
3.1 Convert to Akida model
When the quantized model produces satisfactory performance, it can be converted to the native Akida format. The convert function returns a model in Akida format ready for inference.
As with Keras, the summary() method provides a textual representation of the Akida model.
from cnn2snn import convert
model_akida = convert(model_quantized)
model_akida.summary()
Model Summary
______________________________________________
Input shape Output shape Sequences Layers
==============================================
[28, 28, 1] [1, 1, 10] 1 5
______________________________________________
__________________________________________________________________
Layer (type) Output shape Kernel shape
=============== SW/conv2d-dequantizer_2 (Software) ===============
conv2d (InputConv2D) [13, 13, 32] (3, 3, 1, 32)
__________________________________________________________________
depthwise_conv2d (DepthwiseConv2D) [7, 7, 32] (3, 3, 32, 1)
__________________________________________________________________
conv2d_1 (Conv2D) [7, 7, 64] (1, 1, 32, 64)
__________________________________________________________________
dense (Dense1D) [1, 1, 10] (3136, 10)
__________________________________________________________________
dequantizer_2 (Dequantizer) [1, 1, 10] N/A
__________________________________________________________________
3.2. Check performance
accuracy = model_akida.evaluate(x_test, y_test)
print('Test accuracy after conversion:', accuracy)
# For non-regression purposes
assert accuracy > 0.96
Test accuracy after conversion: 0.9839000105857849
3.3 Show predictions for a single image
Display one of the test images, such as the first image in the dataset from above, to visualize the output of the model.
# Test a single example
sample_image = 0
image = x_test[sample_image]
outputs = model_akida.predict(image.reshape(1, 28, 28, 1))
print('Input Label: %i' % y_test[sample_image])
f, axarr = plt.subplots(1, 2)
axarr[0].imshow(x_test[sample_image].reshape((28, 28)), cmap=cm.Greys_r)
axarr[0].set_title('Class %d' % y_test[sample_image])
axarr[1].bar(range(10), outputs.squeeze())
axarr[1].set_xticks(range(10))
plt.show()
print(outputs.squeeze())
Input Label: 7
[-16.429174 -12.029522 -8.657636 0.73896545 -20.699781
-9.316402 -34.37597 10.484481 -6.573269 -2.5740962 ]
Consider the output from the model above. As is typical in backprop-trained models, the final layer is a Dense layer with one neuron for each of the 10 classes in the dataset. The goal of training is to maximize the response of the neuron corresponding to the label of each training sample while minimizing the responses of the other neurons.
In the bar chart above, you can see the outputs from all 10 neurons. It is easy to see that neuron 7 responds much more strongly than the others. The first sample is indeed a number 7.
Total running time of the script: (2 minutes 7.436 seconds)