Note
Go to the end to download the full example code.
Global Akida workflow
Using the MNIST dataset, this example shows the definition and training of a keras floating point model, its quantization to 8-bit with the help of calibration, its quantization to 4-bit using QAT and its conversion to Akida. Notice that the performance of the original keras floating point model is maintained throughout the Akida flow. Please refer to the Akida user guide for further information.
Note
Please refer to the TensorFlow tf.keras.models module for model creation/import details and the TensorFlow Guide for TensorFlow usage.
The MNIST example below is light enough so that a GPU is not needed for training.
Global Akida workflow
1. Create and train
1.1. Load and reshape MNIST dataset
import numpy as np
import matplotlib.cm as cm
import matplotlib.pyplot as plt
from keras.datasets import mnist
# Load MNIST dataset
(x_train, y_train), (x_test, y_test) = mnist.load_data()
# Add a channels dimension to the image sets as Akida expects 4-D inputs (corresponding to
# (num_samples, width, height, channels). Note: MNIST is a grayscale dataset and is unusual
# in this respect - most image data already includes a channel dimension, and this step will
# not be necessary.
x_train = np.expand_dims(x_train, -1)
x_test = np.expand_dims(x_test, -1)
# Display a few images from the test set
f, axarr = plt.subplots(1, 4)
for i in range(0, 4):
axarr[i].imshow(x_test[i].reshape((28, 28)), cmap=cm.Greys_r)
axarr[i].set_title('Class %d' % y_test[i])
plt.show()
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
8192/11490434 [..............................] - ETA: 0s
32768/11490434 [..............................] - ETA: 28s
49152/11490434 [..............................] - ETA: 31s
81920/11490434 [..............................] - ETA: 31s
147456/11490434 [..............................] - ETA: 22s
212992/11490434 [..............................] - ETA: 18s
327680/11490434 [..............................] - ETA: 13s
466944/11490434 [>.............................] - ETA: 10s
720896/11490434 [>.............................] - ETA: 7s
1081344/11490434 [=>............................] - ETA: 5s
1589248/11490434 [===>..........................] - ETA: 3s
2588672/11490434 [=====>........................] - ETA: 2s
3891200/11490434 [=========>....................] - ETA: 1s
4562944/11490434 [==========>...................] - ETA: 1s
5873664/11490434 [==============>...............] - ETA: 0s
7168000/11490434 [=================>............] - ETA: 0s
8560640/11490434 [=====================>........] - ETA: 0s
9863168/11490434 [========================>.....] - ETA: 0s
10805248/11490434 [===========================>..] - ETA: 0s
11490434/11490434 [==============================] - 1s 0us/step
1.2. Model definition
Note that at this stage, there is nothing specific to the Akida IP. The model constructed below, as inspired by this example, is a completely standard Keras CNN model.
import keras
model_keras = keras.models.Sequential([
keras.layers.Rescaling(1. / 255, input_shape=(28, 28, 1)),
keras.layers.Conv2D(filters=32, kernel_size=3, strides=2),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
# Separable layer
keras.layers.DepthwiseConv2D(kernel_size=3, padding='same', strides=2),
keras.layers.Conv2D(filters=64, kernel_size=1, padding='same'),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
keras.layers.Flatten(),
keras.layers.Dense(10)
], 'mnistnet')
model_keras.summary()
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling (Rescaling) (None, 28, 28, 1) 0
conv2d (Conv2D) (None, 13, 13, 32) 320
batch_normalization (Batch (None, 13, 13, 32) 128
Normalization)
re_lu (ReLU) (None, 13, 13, 32) 0
depthwise_conv2d (Depthwis (None, 7, 7, 32) 320
eConv2D)
conv2d_1 (Conv2D) (None, 7, 7, 64) 2112
batch_normalization_1 (Bat (None, 7, 7, 64) 256
chNormalization)
re_lu_1 (ReLU) (None, 7, 7, 64) 0
flatten (Flatten) (None, 3136) 0
dense (Dense) (None, 10) 31370
=================================================================
Total params: 34506 (134.79 KB)
Trainable params: 34314 (134.04 KB)
Non-trainable params: 192 (768.00 Byte)
_________________________________________________________________
1.3. Model training
Given the model created above, train the model and check its accuracy. The model should achieve a test accuracy over 98% after 10 epochs.
from keras.optimizers import Adam
model_keras.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-3),
metrics=['accuracy'])
_ = model_keras.fit(x_train, y_train, epochs=10, validation_split=0.1)
Epoch 1/10
1/1688 [..............................] - ETA: 1:04:25 - loss: 2.8720 - accuracy: 0.0625
23/1688 [..............................] - ETA: 3s - loss: 1.4817 - accuracy: 0.5054
45/1688 [..............................] - ETA: 3s - loss: 1.0820 - accuracy: 0.6444
68/1688 [>.............................] - ETA: 3s - loss: 0.8993 - accuracy: 0.7109
91/1688 [>.............................] - ETA: 3s - loss: 0.7605 - accuracy: 0.7541
114/1688 [=>............................] - ETA: 3s - loss: 0.6820 - accuracy: 0.7818
137/1688 [=>............................] - ETA: 3s - loss: 0.6226 - accuracy: 0.8025
160/1688 [=>............................] - ETA: 3s - loss: 0.5770 - accuracy: 0.8172
183/1688 [==>...........................] - ETA: 3s - loss: 0.5461 - accuracy: 0.8282
205/1688 [==>...........................] - ETA: 3s - loss: 0.5146 - accuracy: 0.8396
228/1688 [===>..........................] - ETA: 3s - loss: 0.4904 - accuracy: 0.8480
251/1688 [===>..........................] - ETA: 3s - loss: 0.4657 - accuracy: 0.8568
274/1688 [===>..........................] - ETA: 3s - loss: 0.4423 - accuracy: 0.8645
297/1688 [====>.........................] - ETA: 3s - loss: 0.4246 - accuracy: 0.8695
320/1688 [====>.........................] - ETA: 3s - loss: 0.4060 - accuracy: 0.8755
342/1688 [=====>........................] - ETA: 3s - loss: 0.3939 - accuracy: 0.8788
365/1688 [=====>........................] - ETA: 2s - loss: 0.3784 - accuracy: 0.8836
388/1688 [=====>........................] - ETA: 2s - loss: 0.3664 - accuracy: 0.8876
411/1688 [======>.......................] - ETA: 2s - loss: 0.3559 - accuracy: 0.8909
434/1688 [======>.......................] - ETA: 2s - loss: 0.3470 - accuracy: 0.8937
456/1688 [=======>......................] - ETA: 2s - loss: 0.3369 - accuracy: 0.8967
479/1688 [=======>......................] - ETA: 2s - loss: 0.3277 - accuracy: 0.8994
502/1688 [=======>......................] - ETA: 2s - loss: 0.3193 - accuracy: 0.9020
524/1688 [========>.....................] - ETA: 2s - loss: 0.3129 - accuracy: 0.9040
547/1688 [========>.....................] - ETA: 2s - loss: 0.3053 - accuracy: 0.9062
569/1688 [=========>....................] - ETA: 2s - loss: 0.2979 - accuracy: 0.9084
592/1688 [=========>....................] - ETA: 2s - loss: 0.2910 - accuracy: 0.9108
616/1688 [=========>....................] - ETA: 2s - loss: 0.2841 - accuracy: 0.9128
639/1688 [==========>...................] - ETA: 2s - loss: 0.2787 - accuracy: 0.9143
662/1688 [==========>...................] - ETA: 2s - loss: 0.2747 - accuracy: 0.9154
685/1688 [===========>..................] - ETA: 2s - loss: 0.2693 - accuracy: 0.9170
708/1688 [===========>..................] - ETA: 2s - loss: 0.2644 - accuracy: 0.9186
731/1688 [===========>..................] - ETA: 2s - loss: 0.2604 - accuracy: 0.9199
754/1688 [============>.................] - ETA: 2s - loss: 0.2565 - accuracy: 0.9210
777/1688 [============>.................] - ETA: 2s - loss: 0.2520 - accuracy: 0.9224
800/1688 [=============>................] - ETA: 1s - loss: 0.2480 - accuracy: 0.9237
822/1688 [=============>................] - ETA: 1s - loss: 0.2436 - accuracy: 0.9250
844/1688 [==============>...............] - ETA: 1s - loss: 0.2398 - accuracy: 0.9262
868/1688 [==============>...............] - ETA: 1s - loss: 0.2365 - accuracy: 0.9273
891/1688 [==============>...............] - ETA: 1s - loss: 0.2336 - accuracy: 0.9282
914/1688 [===============>..............] - ETA: 1s - loss: 0.2300 - accuracy: 0.9292
935/1688 [===============>..............] - ETA: 1s - loss: 0.2275 - accuracy: 0.9299
956/1688 [===============>..............] - ETA: 1s - loss: 0.2259 - accuracy: 0.9307
976/1688 [================>.............] - ETA: 1s - loss: 0.2238 - accuracy: 0.9313
998/1688 [================>.............] - ETA: 1s - loss: 0.2209 - accuracy: 0.9320
1019/1688 [=================>............] - ETA: 1s - loss: 0.2181 - accuracy: 0.9328
1040/1688 [=================>............] - ETA: 1s - loss: 0.2162 - accuracy: 0.9335
1061/1688 [=================>............] - ETA: 1s - loss: 0.2141 - accuracy: 0.9342
1082/1688 [==================>...........] - ETA: 1s - loss: 0.2117 - accuracy: 0.9350
1103/1688 [==================>...........] - ETA: 1s - loss: 0.2100 - accuracy: 0.9357
1124/1688 [==================>...........] - ETA: 1s - loss: 0.2077 - accuracy: 0.9363
1145/1688 [===================>..........] - ETA: 1s - loss: 0.2062 - accuracy: 0.9368
1166/1688 [===================>..........] - ETA: 1s - loss: 0.2046 - accuracy: 0.9372
1188/1688 [====================>.........] - ETA: 1s - loss: 0.2027 - accuracy: 0.9379
1209/1688 [====================>.........] - ETA: 1s - loss: 0.2010 - accuracy: 0.9383
1231/1688 [====================>.........] - ETA: 1s - loss: 0.1993 - accuracy: 0.9388
1252/1688 [=====================>........] - ETA: 0s - loss: 0.1977 - accuracy: 0.9393
1273/1688 [=====================>........] - ETA: 0s - loss: 0.1963 - accuracy: 0.9397
1294/1688 [=====================>........] - ETA: 0s - loss: 0.1946 - accuracy: 0.9401
1316/1688 [======================>.......] - ETA: 0s - loss: 0.1926 - accuracy: 0.9408
1338/1688 [======================>.......] - ETA: 0s - loss: 0.1911 - accuracy: 0.9412
1358/1688 [=======================>......] - ETA: 0s - loss: 0.1895 - accuracy: 0.9417
1378/1688 [=======================>......] - ETA: 0s - loss: 0.1883 - accuracy: 0.9420
1398/1688 [=======================>......] - ETA: 0s - loss: 0.1870 - accuracy: 0.9425
1418/1688 [========================>.....] - ETA: 0s - loss: 0.1860 - accuracy: 0.9428
1438/1688 [========================>.....] - ETA: 0s - loss: 0.1853 - accuracy: 0.9431
1458/1688 [========================>.....] - ETA: 0s - loss: 0.1842 - accuracy: 0.9435
1478/1688 [=========================>....] - ETA: 0s - loss: 0.1828 - accuracy: 0.9439
1498/1688 [=========================>....] - ETA: 0s - loss: 0.1809 - accuracy: 0.9445
1517/1688 [=========================>....] - ETA: 0s - loss: 0.1794 - accuracy: 0.9449
1536/1688 [==========================>...] - ETA: 0s - loss: 0.1783 - accuracy: 0.9453
1556/1688 [==========================>...] - ETA: 0s - loss: 0.1772 - accuracy: 0.9456
1576/1688 [===========================>..] - ETA: 0s - loss: 0.1768 - accuracy: 0.9459
1596/1688 [===========================>..] - ETA: 0s - loss: 0.1756 - accuracy: 0.9462
1616/1688 [===========================>..] - ETA: 0s - loss: 0.1746 - accuracy: 0.9465
1636/1688 [============================>.] - ETA: 0s - loss: 0.1741 - accuracy: 0.9468
1656/1688 [============================>.] - ETA: 0s - loss: 0.1730 - accuracy: 0.9471
1676/1688 [============================>.] - ETA: 0s - loss: 0.1727 - accuracy: 0.9473
1688/1688 [==============================] - ETA: 0s - loss: 0.1722 - accuracy: 0.9475
1688/1688 [==============================] - 7s 3ms/step - loss: 0.1722 - accuracy: 0.9475 - val_loss: 0.0983 - val_accuracy: 0.9758
Epoch 2/10
1/1688 [..............................] - ETA: 3s - loss: 0.1260 - accuracy: 0.9688
25/1688 [..............................] - ETA: 3s - loss: 0.0863 - accuracy: 0.9737
47/1688 [..............................] - ETA: 3s - loss: 0.0799 - accuracy: 0.9747
70/1688 [>.............................] - ETA: 3s - loss: 0.0721 - accuracy: 0.9759
93/1688 [>.............................] - ETA: 3s - loss: 0.0729 - accuracy: 0.9741
116/1688 [=>............................] - ETA: 3s - loss: 0.0745 - accuracy: 0.9739
138/1688 [=>............................] - ETA: 3s - loss: 0.0717 - accuracy: 0.9744
161/1688 [=>............................] - ETA: 3s - loss: 0.0719 - accuracy: 0.9746
184/1688 [==>...........................] - ETA: 3s - loss: 0.0736 - accuracy: 0.9745
207/1688 [==>...........................] - ETA: 3s - loss: 0.0727 - accuracy: 0.9746
230/1688 [===>..........................] - ETA: 3s - loss: 0.0721 - accuracy: 0.9751
253/1688 [===>..........................] - ETA: 3s - loss: 0.0712 - accuracy: 0.9753
276/1688 [===>..........................] - ETA: 3s - loss: 0.0709 - accuracy: 0.9753
299/1688 [====>.........................] - ETA: 3s - loss: 0.0688 - accuracy: 0.9764
321/1688 [====>.........................] - ETA: 3s - loss: 0.0685 - accuracy: 0.9764
344/1688 [=====>........................] - ETA: 3s - loss: 0.0690 - accuracy: 0.9761
367/1688 [=====>........................] - ETA: 2s - loss: 0.0688 - accuracy: 0.9764
390/1688 [=====>........................] - ETA: 2s - loss: 0.0690 - accuracy: 0.9761
413/1688 [======>.......................] - ETA: 2s - loss: 0.0697 - accuracy: 0.9762
436/1688 [======>.......................] - ETA: 2s - loss: 0.0683 - accuracy: 0.9769
459/1688 [=======>......................] - ETA: 2s - loss: 0.0678 - accuracy: 0.9773
483/1688 [=======>......................] - ETA: 2s - loss: 0.0670 - accuracy: 0.9774
506/1688 [=======>......................] - ETA: 2s - loss: 0.0662 - accuracy: 0.9778
528/1688 [========>.....................] - ETA: 2s - loss: 0.0657 - accuracy: 0.9780
551/1688 [========>.....................] - ETA: 2s - loss: 0.0648 - accuracy: 0.9784
574/1688 [=========>....................] - ETA: 2s - loss: 0.0656 - accuracy: 0.9782
597/1688 [=========>....................] - ETA: 2s - loss: 0.0669 - accuracy: 0.9778
620/1688 [==========>...................] - ETA: 2s - loss: 0.0667 - accuracy: 0.9781
643/1688 [==========>...................] - ETA: 2s - loss: 0.0663 - accuracy: 0.9783
666/1688 [==========>...................] - ETA: 2s - loss: 0.0654 - accuracy: 0.9787
689/1688 [===========>..................] - ETA: 2s - loss: 0.0658 - accuracy: 0.9786
712/1688 [===========>..................] - ETA: 2s - loss: 0.0661 - accuracy: 0.9785
735/1688 [============>.................] - ETA: 2s - loss: 0.0662 - accuracy: 0.9783
758/1688 [============>.................] - ETA: 2s - loss: 0.0663 - accuracy: 0.9785
780/1688 [============>.................] - ETA: 2s - loss: 0.0662 - accuracy: 0.9785
802/1688 [=============>................] - ETA: 1s - loss: 0.0664 - accuracy: 0.9785
825/1688 [=============>................] - ETA: 1s - loss: 0.0664 - accuracy: 0.9786
848/1688 [==============>...............] - ETA: 1s - loss: 0.0662 - accuracy: 0.9787
870/1688 [==============>...............] - ETA: 1s - loss: 0.0662 - accuracy: 0.9788
893/1688 [==============>...............] - ETA: 1s - loss: 0.0662 - accuracy: 0.9788
917/1688 [===============>..............] - ETA: 1s - loss: 0.0671 - accuracy: 0.9786
940/1688 [===============>..............] - ETA: 1s - loss: 0.0681 - accuracy: 0.9785
962/1688 [================>.............] - ETA: 1s - loss: 0.0682 - accuracy: 0.9784
985/1688 [================>.............] - ETA: 1s - loss: 0.0676 - accuracy: 0.9786
1007/1688 [================>.............] - ETA: 1s - loss: 0.0674 - accuracy: 0.9786
1030/1688 [=================>............] - ETA: 1s - loss: 0.0677 - accuracy: 0.9785
1053/1688 [=================>............] - ETA: 1s - loss: 0.0677 - accuracy: 0.9786
1076/1688 [==================>...........] - ETA: 1s - loss: 0.0681 - accuracy: 0.9785
1097/1688 [==================>...........] - ETA: 1s - loss: 0.0682 - accuracy: 0.9784
1118/1688 [==================>...........] - ETA: 1s - loss: 0.0683 - accuracy: 0.9783
1139/1688 [===================>..........] - ETA: 1s - loss: 0.0685 - accuracy: 0.9781
1160/1688 [===================>..........] - ETA: 1s - loss: 0.0691 - accuracy: 0.9780
1181/1688 [===================>..........] - ETA: 1s - loss: 0.0689 - accuracy: 0.9780
1202/1688 [====================>.........] - ETA: 1s - loss: 0.0695 - accuracy: 0.9779
1223/1688 [====================>.........] - ETA: 1s - loss: 0.0701 - accuracy: 0.9777
1244/1688 [=====================>........] - ETA: 1s - loss: 0.0697 - accuracy: 0.9777
1265/1688 [=====================>........] - ETA: 0s - loss: 0.0692 - accuracy: 0.9780
1287/1688 [=====================>........] - ETA: 0s - loss: 0.0689 - accuracy: 0.9780
1309/1688 [======================>.......] - ETA: 0s - loss: 0.0689 - accuracy: 0.9780
1330/1688 [======================>.......] - ETA: 0s - loss: 0.0690 - accuracy: 0.9780
1351/1688 [=======================>......] - ETA: 0s - loss: 0.0690 - accuracy: 0.9779
1373/1688 [=======================>......] - ETA: 0s - loss: 0.0683 - accuracy: 0.9781
1395/1688 [=======================>......] - ETA: 0s - loss: 0.0683 - accuracy: 0.9782
1416/1688 [========================>.....] - ETA: 0s - loss: 0.0683 - accuracy: 0.9782
1437/1688 [========================>.....] - ETA: 0s - loss: 0.0680 - accuracy: 0.9783
1458/1688 [========================>.....] - ETA: 0s - loss: 0.0681 - accuracy: 0.9783
1480/1688 [=========================>....] - ETA: 0s - loss: 0.0680 - accuracy: 0.9783
1502/1688 [=========================>....] - ETA: 0s - loss: 0.0685 - accuracy: 0.9783
1523/1688 [==========================>...] - ETA: 0s - loss: 0.0684 - accuracy: 0.9783
1545/1688 [==========================>...] - ETA: 0s - loss: 0.0685 - accuracy: 0.9783
1567/1688 [==========================>...] - ETA: 0s - loss: 0.0686 - accuracy: 0.9783
1589/1688 [===========================>..] - ETA: 0s - loss: 0.0689 - accuracy: 0.9783
1610/1688 [===========================>..] - ETA: 0s - loss: 0.0693 - accuracy: 0.9782
1631/1688 [===========================>..] - ETA: 0s - loss: 0.0694 - accuracy: 0.9782
1652/1688 [============================>.] - ETA: 0s - loss: 0.0692 - accuracy: 0.9783
1673/1688 [============================>.] - ETA: 0s - loss: 0.0694 - accuracy: 0.9782
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0693 - accuracy: 0.9782 - val_loss: 0.0714 - val_accuracy: 0.9798
Epoch 3/10
1/1688 [..............................] - ETA: 3s - loss: 0.0209 - accuracy: 1.0000
25/1688 [..............................] - ETA: 3s - loss: 0.0666 - accuracy: 0.9800
48/1688 [..............................] - ETA: 3s - loss: 0.0601 - accuracy: 0.9798
71/1688 [>.............................] - ETA: 3s - loss: 0.0508 - accuracy: 0.9824
94/1688 [>.............................] - ETA: 3s - loss: 0.0497 - accuracy: 0.9840
117/1688 [=>............................] - ETA: 3s - loss: 0.0512 - accuracy: 0.9845
140/1688 [=>............................] - ETA: 3s - loss: 0.0489 - accuracy: 0.9844
163/1688 [=>............................] - ETA: 3s - loss: 0.0452 - accuracy: 0.9858
185/1688 [==>...........................] - ETA: 3s - loss: 0.0478 - accuracy: 0.9853
207/1688 [==>...........................] - ETA: 3s - loss: 0.0467 - accuracy: 0.9857
229/1688 [===>..........................] - ETA: 3s - loss: 0.0449 - accuracy: 0.9865
251/1688 [===>..........................] - ETA: 3s - loss: 0.0433 - accuracy: 0.9869
274/1688 [===>..........................] - ETA: 3s - loss: 0.0427 - accuracy: 0.9871
297/1688 [====>.........................] - ETA: 3s - loss: 0.0440 - accuracy: 0.9871
320/1688 [====>.........................] - ETA: 3s - loss: 0.0452 - accuracy: 0.9866
343/1688 [=====>........................] - ETA: 3s - loss: 0.0456 - accuracy: 0.9866
366/1688 [=====>........................] - ETA: 2s - loss: 0.0456 - accuracy: 0.9864
389/1688 [=====>........................] - ETA: 2s - loss: 0.0449 - accuracy: 0.9867
411/1688 [======>.......................] - ETA: 2s - loss: 0.0458 - accuracy: 0.9865
432/1688 [======>.......................] - ETA: 2s - loss: 0.0452 - accuracy: 0.9868
453/1688 [=======>......................] - ETA: 2s - loss: 0.0442 - accuracy: 0.9872
474/1688 [=======>......................] - ETA: 2s - loss: 0.0455 - accuracy: 0.9868
495/1688 [=======>......................] - ETA: 2s - loss: 0.0449 - accuracy: 0.9869
516/1688 [========>.....................] - ETA: 2s - loss: 0.0459 - accuracy: 0.9866
537/1688 [========>.....................] - ETA: 2s - loss: 0.0469 - accuracy: 0.9864
558/1688 [========>.....................] - ETA: 2s - loss: 0.0471 - accuracy: 0.9863
580/1688 [=========>....................] - ETA: 2s - loss: 0.0468 - accuracy: 0.9861
602/1688 [=========>....................] - ETA: 2s - loss: 0.0466 - accuracy: 0.9862
623/1688 [==========>...................] - ETA: 2s - loss: 0.0466 - accuracy: 0.9862
644/1688 [==========>...................] - ETA: 2s - loss: 0.0465 - accuracy: 0.9862
665/1688 [==========>...................] - ETA: 2s - loss: 0.0465 - accuracy: 0.9862
687/1688 [===========>..................] - ETA: 2s - loss: 0.0458 - accuracy: 0.9864
708/1688 [===========>..................] - ETA: 2s - loss: 0.0458 - accuracy: 0.9862
729/1688 [===========>..................] - ETA: 2s - loss: 0.0464 - accuracy: 0.9859
751/1688 [============>.................] - ETA: 2s - loss: 0.0472 - accuracy: 0.9857
772/1688 [============>.................] - ETA: 2s - loss: 0.0471 - accuracy: 0.9856
794/1688 [=============>................] - ETA: 2s - loss: 0.0474 - accuracy: 0.9856
815/1688 [=============>................] - ETA: 2s - loss: 0.0473 - accuracy: 0.9857
837/1688 [=============>................] - ETA: 1s - loss: 0.0479 - accuracy: 0.9856
858/1688 [==============>...............] - ETA: 1s - loss: 0.0479 - accuracy: 0.9856
879/1688 [==============>...............] - ETA: 1s - loss: 0.0478 - accuracy: 0.9857
900/1688 [==============>...............] - ETA: 1s - loss: 0.0481 - accuracy: 0.9856
921/1688 [===============>..............] - ETA: 1s - loss: 0.0483 - accuracy: 0.9852
943/1688 [===============>..............] - ETA: 1s - loss: 0.0486 - accuracy: 0.9851
965/1688 [================>.............] - ETA: 1s - loss: 0.0488 - accuracy: 0.9849
986/1688 [================>.............] - ETA: 1s - loss: 0.0486 - accuracy: 0.9849
1007/1688 [================>.............] - ETA: 1s - loss: 0.0493 - accuracy: 0.9847
1028/1688 [=================>............] - ETA: 1s - loss: 0.0495 - accuracy: 0.9846
1049/1688 [=================>............] - ETA: 1s - loss: 0.0500 - accuracy: 0.9844
1070/1688 [==================>...........] - ETA: 1s - loss: 0.0498 - accuracy: 0.9843
1091/1688 [==================>...........] - ETA: 1s - loss: 0.0495 - accuracy: 0.9845
1112/1688 [==================>...........] - ETA: 1s - loss: 0.0492 - accuracy: 0.9846
1133/1688 [===================>..........] - ETA: 1s - loss: 0.0492 - accuracy: 0.9846
1155/1688 [===================>..........] - ETA: 1s - loss: 0.0490 - accuracy: 0.9846
1176/1688 [===================>..........] - ETA: 1s - loss: 0.0491 - accuracy: 0.9846
1198/1688 [====================>.........] - ETA: 1s - loss: 0.0489 - accuracy: 0.9846
1219/1688 [====================>.........] - ETA: 1s - loss: 0.0490 - accuracy: 0.9845
1241/1688 [=====================>........] - ETA: 1s - loss: 0.0489 - accuracy: 0.9846
1263/1688 [=====================>........] - ETA: 0s - loss: 0.0494 - accuracy: 0.9844
1284/1688 [=====================>........] - ETA: 0s - loss: 0.0498 - accuracy: 0.9843
1306/1688 [======================>.......] - ETA: 0s - loss: 0.0502 - accuracy: 0.9841
1327/1688 [======================>.......] - ETA: 0s - loss: 0.0506 - accuracy: 0.9840
1348/1688 [======================>.......] - ETA: 0s - loss: 0.0504 - accuracy: 0.9840
1370/1688 [=======================>......] - ETA: 0s - loss: 0.0503 - accuracy: 0.9841
1391/1688 [=======================>......] - ETA: 0s - loss: 0.0503 - accuracy: 0.9842
1412/1688 [========================>.....] - ETA: 0s - loss: 0.0501 - accuracy: 0.9843
1434/1688 [========================>.....] - ETA: 0s - loss: 0.0504 - accuracy: 0.9844
1455/1688 [========================>.....] - ETA: 0s - loss: 0.0503 - accuracy: 0.9844
1477/1688 [=========================>....] - ETA: 0s - loss: 0.0502 - accuracy: 0.9844
1498/1688 [=========================>....] - ETA: 0s - loss: 0.0502 - accuracy: 0.9844
1519/1688 [=========================>....] - ETA: 0s - loss: 0.0501 - accuracy: 0.9844
1540/1688 [==========================>...] - ETA: 0s - loss: 0.0499 - accuracy: 0.9844
1561/1688 [==========================>...] - ETA: 0s - loss: 0.0501 - accuracy: 0.9845
1582/1688 [===========================>..] - ETA: 0s - loss: 0.0498 - accuracy: 0.9845
1604/1688 [===========================>..] - ETA: 0s - loss: 0.0497 - accuracy: 0.9846
1626/1688 [===========================>..] - ETA: 0s - loss: 0.0501 - accuracy: 0.9845
1648/1688 [============================>.] - ETA: 0s - loss: 0.0500 - accuracy: 0.9845
1670/1688 [============================>.] - ETA: 0s - loss: 0.0501 - accuracy: 0.9844
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0501 - accuracy: 0.9844 - val_loss: 0.0704 - val_accuracy: 0.9810
Epoch 4/10
1/1688 [..............................] - ETA: 4s - loss: 0.0355 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0436 - accuracy: 0.9883
47/1688 [..............................] - ETA: 3s - loss: 0.0437 - accuracy: 0.9847
70/1688 [>.............................] - ETA: 3s - loss: 0.0360 - accuracy: 0.9879
93/1688 [>.............................] - ETA: 3s - loss: 0.0321 - accuracy: 0.9892
116/1688 [=>............................] - ETA: 3s - loss: 0.0305 - accuracy: 0.9903
139/1688 [=>............................] - ETA: 3s - loss: 0.0310 - accuracy: 0.9901
162/1688 [=>............................] - ETA: 3s - loss: 0.0316 - accuracy: 0.9892
185/1688 [==>...........................] - ETA: 3s - loss: 0.0322 - accuracy: 0.9890
208/1688 [==>...........................] - ETA: 3s - loss: 0.0345 - accuracy: 0.9884
231/1688 [===>..........................] - ETA: 3s - loss: 0.0339 - accuracy: 0.9884
254/1688 [===>..........................] - ETA: 3s - loss: 0.0342 - accuracy: 0.9883
277/1688 [===>..........................] - ETA: 3s - loss: 0.0348 - accuracy: 0.9883
300/1688 [====>.........................] - ETA: 3s - loss: 0.0352 - accuracy: 0.9882
323/1688 [====>.........................] - ETA: 3s - loss: 0.0363 - accuracy: 0.9882
345/1688 [=====>........................] - ETA: 2s - loss: 0.0360 - accuracy: 0.9882
368/1688 [=====>........................] - ETA: 2s - loss: 0.0360 - accuracy: 0.9883
391/1688 [=====>........................] - ETA: 2s - loss: 0.0366 - accuracy: 0.9881
414/1688 [======>.......................] - ETA: 2s - loss: 0.0367 - accuracy: 0.9881
437/1688 [======>.......................] - ETA: 2s - loss: 0.0363 - accuracy: 0.9883
460/1688 [=======>......................] - ETA: 2s - loss: 0.0359 - accuracy: 0.9882
482/1688 [=======>......................] - ETA: 2s - loss: 0.0366 - accuracy: 0.9879
505/1688 [=======>......................] - ETA: 2s - loss: 0.0358 - accuracy: 0.9882
528/1688 [========>.....................] - ETA: 2s - loss: 0.0361 - accuracy: 0.9882
551/1688 [========>.....................] - ETA: 2s - loss: 0.0359 - accuracy: 0.9881
574/1688 [=========>....................] - ETA: 2s - loss: 0.0369 - accuracy: 0.9879
596/1688 [=========>....................] - ETA: 2s - loss: 0.0375 - accuracy: 0.9878
619/1688 [==========>...................] - ETA: 2s - loss: 0.0385 - accuracy: 0.9875
642/1688 [==========>...................] - ETA: 2s - loss: 0.0388 - accuracy: 0.9875
665/1688 [==========>...................] - ETA: 2s - loss: 0.0396 - accuracy: 0.9873
688/1688 [===========>..................] - ETA: 2s - loss: 0.0395 - accuracy: 0.9874
711/1688 [===========>..................] - ETA: 2s - loss: 0.0387 - accuracy: 0.9877
734/1688 [============>.................] - ETA: 2s - loss: 0.0387 - accuracy: 0.9878
757/1688 [============>.................] - ETA: 2s - loss: 0.0403 - accuracy: 0.9874
780/1688 [============>.................] - ETA: 2s - loss: 0.0412 - accuracy: 0.9871
803/1688 [=============>................] - ETA: 1s - loss: 0.0408 - accuracy: 0.9873
826/1688 [=============>................] - ETA: 1s - loss: 0.0404 - accuracy: 0.9874
849/1688 [==============>...............] - ETA: 1s - loss: 0.0402 - accuracy: 0.9875
872/1688 [==============>...............] - ETA: 1s - loss: 0.0400 - accuracy: 0.9875
894/1688 [==============>...............] - ETA: 1s - loss: 0.0398 - accuracy: 0.9875
917/1688 [===============>..............] - ETA: 1s - loss: 0.0404 - accuracy: 0.9874
940/1688 [===============>..............] - ETA: 1s - loss: 0.0405 - accuracy: 0.9874
963/1688 [================>.............] - ETA: 1s - loss: 0.0404 - accuracy: 0.9872
985/1688 [================>.............] - ETA: 1s - loss: 0.0400 - accuracy: 0.9874
1007/1688 [================>.............] - ETA: 1s - loss: 0.0398 - accuracy: 0.9876
1028/1688 [=================>............] - ETA: 1s - loss: 0.0404 - accuracy: 0.9874
1049/1688 [=================>............] - ETA: 1s - loss: 0.0400 - accuracy: 0.9875
1071/1688 [==================>...........] - ETA: 1s - loss: 0.0400 - accuracy: 0.9875
1093/1688 [==================>...........] - ETA: 1s - loss: 0.0397 - accuracy: 0.9875
1114/1688 [==================>...........] - ETA: 1s - loss: 0.0397 - accuracy: 0.9875
1135/1688 [===================>..........] - ETA: 1s - loss: 0.0402 - accuracy: 0.9874
1157/1688 [===================>..........] - ETA: 1s - loss: 0.0401 - accuracy: 0.9874
1179/1688 [===================>..........] - ETA: 1s - loss: 0.0400 - accuracy: 0.9873
1201/1688 [====================>.........] - ETA: 1s - loss: 0.0403 - accuracy: 0.9874
1222/1688 [====================>.........] - ETA: 1s - loss: 0.0403 - accuracy: 0.9874
1243/1688 [=====================>........] - ETA: 1s - loss: 0.0401 - accuracy: 0.9875
1265/1688 [=====================>........] - ETA: 0s - loss: 0.0402 - accuracy: 0.9875
1286/1688 [=====================>........] - ETA: 0s - loss: 0.0403 - accuracy: 0.9873
1307/1688 [======================>.......] - ETA: 0s - loss: 0.0408 - accuracy: 0.9872
1328/1688 [======================>.......] - ETA: 0s - loss: 0.0408 - accuracy: 0.9872
1351/1688 [=======================>......] - ETA: 0s - loss: 0.0411 - accuracy: 0.9870
1374/1688 [=======================>......] - ETA: 0s - loss: 0.0412 - accuracy: 0.9870
1397/1688 [=======================>......] - ETA: 0s - loss: 0.0415 - accuracy: 0.9868
1420/1688 [========================>.....] - ETA: 0s - loss: 0.0414 - accuracy: 0.9868
1443/1688 [========================>.....] - ETA: 0s - loss: 0.0418 - accuracy: 0.9866
1466/1688 [=========================>....] - ETA: 0s - loss: 0.0423 - accuracy: 0.9865
1489/1688 [=========================>....] - ETA: 0s - loss: 0.0422 - accuracy: 0.9864
1512/1688 [=========================>....] - ETA: 0s - loss: 0.0421 - accuracy: 0.9865
1535/1688 [==========================>...] - ETA: 0s - loss: 0.0419 - accuracy: 0.9866
1558/1688 [==========================>...] - ETA: 0s - loss: 0.0415 - accuracy: 0.9867
1581/1688 [===========================>..] - ETA: 0s - loss: 0.0416 - accuracy: 0.9866
1603/1688 [===========================>..] - ETA: 0s - loss: 0.0417 - accuracy: 0.9867
1626/1688 [===========================>..] - ETA: 0s - loss: 0.0415 - accuracy: 0.9867
1648/1688 [============================>.] - ETA: 0s - loss: 0.0412 - accuracy: 0.9868
1671/1688 [============================>.] - ETA: 0s - loss: 0.0412 - accuracy: 0.9868
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0411 - accuracy: 0.9868 - val_loss: 0.0522 - val_accuracy: 0.9850
Epoch 5/10
1/1688 [..............................] - ETA: 3s - loss: 0.0025 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0350 - accuracy: 0.9896
47/1688 [..............................] - ETA: 3s - loss: 0.0307 - accuracy: 0.9894
70/1688 [>.............................] - ETA: 3s - loss: 0.0259 - accuracy: 0.9911
91/1688 [>.............................] - ETA: 3s - loss: 0.0224 - accuracy: 0.9924
112/1688 [>.............................] - ETA: 3s - loss: 0.0221 - accuracy: 0.9933
133/1688 [=>............................] - ETA: 3s - loss: 0.0246 - accuracy: 0.9925
154/1688 [=>............................] - ETA: 3s - loss: 0.0278 - accuracy: 0.9909
176/1688 [==>...........................] - ETA: 3s - loss: 0.0264 - accuracy: 0.9913
197/1688 [==>...........................] - ETA: 3s - loss: 0.0264 - accuracy: 0.9913
218/1688 [==>...........................] - ETA: 3s - loss: 0.0266 - accuracy: 0.9914
238/1688 [===>..........................] - ETA: 3s - loss: 0.0267 - accuracy: 0.9913
259/1688 [===>..........................] - ETA: 3s - loss: 0.0267 - accuracy: 0.9913
280/1688 [===>..........................] - ETA: 3s - loss: 0.0264 - accuracy: 0.9916
301/1688 [====>.........................] - ETA: 3s - loss: 0.0262 - accuracy: 0.9915
322/1688 [====>.........................] - ETA: 3s - loss: 0.0253 - accuracy: 0.9919
343/1688 [=====>........................] - ETA: 3s - loss: 0.0250 - accuracy: 0.9918
364/1688 [=====>........................] - ETA: 3s - loss: 0.0254 - accuracy: 0.9918
385/1688 [=====>........................] - ETA: 3s - loss: 0.0261 - accuracy: 0.9916
406/1688 [======>.......................] - ETA: 3s - loss: 0.0261 - accuracy: 0.9915
427/1688 [======>.......................] - ETA: 3s - loss: 0.0257 - accuracy: 0.9915
449/1688 [======>.......................] - ETA: 2s - loss: 0.0256 - accuracy: 0.9914
470/1688 [=======>......................] - ETA: 2s - loss: 0.0261 - accuracy: 0.9914
492/1688 [=======>......................] - ETA: 2s - loss: 0.0266 - accuracy: 0.9910
513/1688 [========>.....................] - ETA: 2s - loss: 0.0272 - accuracy: 0.9910
534/1688 [========>.....................] - ETA: 2s - loss: 0.0271 - accuracy: 0.9910
555/1688 [========>.....................] - ETA: 2s - loss: 0.0279 - accuracy: 0.9909
576/1688 [=========>....................] - ETA: 2s - loss: 0.0280 - accuracy: 0.9908
598/1688 [=========>....................] - ETA: 2s - loss: 0.0286 - accuracy: 0.9906
620/1688 [==========>...................] - ETA: 2s - loss: 0.0290 - accuracy: 0.9904
642/1688 [==========>...................] - ETA: 2s - loss: 0.0286 - accuracy: 0.9905
664/1688 [==========>...................] - ETA: 2s - loss: 0.0292 - accuracy: 0.9902
685/1688 [===========>..................] - ETA: 2s - loss: 0.0297 - accuracy: 0.9899
706/1688 [===========>..................] - ETA: 2s - loss: 0.0302 - accuracy: 0.9898
727/1688 [===========>..................] - ETA: 2s - loss: 0.0306 - accuracy: 0.9898
748/1688 [============>.................] - ETA: 2s - loss: 0.0308 - accuracy: 0.9897
770/1688 [============>.................] - ETA: 2s - loss: 0.0311 - accuracy: 0.9894
791/1688 [=============>................] - ETA: 2s - loss: 0.0307 - accuracy: 0.9896
812/1688 [=============>................] - ETA: 2s - loss: 0.0310 - accuracy: 0.9895
834/1688 [=============>................] - ETA: 2s - loss: 0.0316 - accuracy: 0.9894
855/1688 [==============>...............] - ETA: 1s - loss: 0.0321 - accuracy: 0.9894
876/1688 [==============>...............] - ETA: 1s - loss: 0.0324 - accuracy: 0.9892
897/1688 [==============>...............] - ETA: 1s - loss: 0.0327 - accuracy: 0.9891
918/1688 [===============>..............] - ETA: 1s - loss: 0.0330 - accuracy: 0.9890
939/1688 [===============>..............] - ETA: 1s - loss: 0.0330 - accuracy: 0.9890
960/1688 [================>.............] - ETA: 1s - loss: 0.0329 - accuracy: 0.9891
981/1688 [================>.............] - ETA: 1s - loss: 0.0334 - accuracy: 0.9889
1002/1688 [================>.............] - ETA: 1s - loss: 0.0332 - accuracy: 0.9890
1023/1688 [=================>............] - ETA: 1s - loss: 0.0334 - accuracy: 0.9889
1044/1688 [=================>............] - ETA: 1s - loss: 0.0335 - accuracy: 0.9887
1065/1688 [=================>............] - ETA: 1s - loss: 0.0335 - accuracy: 0.9887
1086/1688 [==================>...........] - ETA: 1s - loss: 0.0341 - accuracy: 0.9885
1107/1688 [==================>...........] - ETA: 1s - loss: 0.0343 - accuracy: 0.9884
1128/1688 [===================>..........] - ETA: 1s - loss: 0.0342 - accuracy: 0.9885
1150/1688 [===================>..........] - ETA: 1s - loss: 0.0340 - accuracy: 0.9885
1172/1688 [===================>..........] - ETA: 1s - loss: 0.0341 - accuracy: 0.9886
1193/1688 [====================>.........] - ETA: 1s - loss: 0.0346 - accuracy: 0.9885
1214/1688 [====================>.........] - ETA: 1s - loss: 0.0343 - accuracy: 0.9886
1235/1688 [====================>.........] - ETA: 1s - loss: 0.0342 - accuracy: 0.9886
1256/1688 [=====================>........] - ETA: 1s - loss: 0.0341 - accuracy: 0.9886
1277/1688 [=====================>........] - ETA: 0s - loss: 0.0341 - accuracy: 0.9886
1298/1688 [======================>.......] - ETA: 0s - loss: 0.0339 - accuracy: 0.9887
1320/1688 [======================>.......] - ETA: 0s - loss: 0.0336 - accuracy: 0.9888
1341/1688 [======================>.......] - ETA: 0s - loss: 0.0335 - accuracy: 0.9889
1362/1688 [=======================>......] - ETA: 0s - loss: 0.0334 - accuracy: 0.9889
1383/1688 [=======================>......] - ETA: 0s - loss: 0.0337 - accuracy: 0.9888
1404/1688 [=======================>......] - ETA: 0s - loss: 0.0339 - accuracy: 0.9887
1425/1688 [========================>.....] - ETA: 0s - loss: 0.0341 - accuracy: 0.9887
1446/1688 [========================>.....] - ETA: 0s - loss: 0.0339 - accuracy: 0.9888
1468/1688 [=========================>....] - ETA: 0s - loss: 0.0338 - accuracy: 0.9888
1490/1688 [=========================>....] - ETA: 0s - loss: 0.0338 - accuracy: 0.9888
1511/1688 [=========================>....] - ETA: 0s - loss: 0.0340 - accuracy: 0.9887
1532/1688 [==========================>...] - ETA: 0s - loss: 0.0340 - accuracy: 0.9887
1553/1688 [==========================>...] - ETA: 0s - loss: 0.0344 - accuracy: 0.9887
1574/1688 [==========================>...] - ETA: 0s - loss: 0.0345 - accuracy: 0.9887
1595/1688 [===========================>..] - ETA: 0s - loss: 0.0343 - accuracy: 0.9888
1616/1688 [===========================>..] - ETA: 0s - loss: 0.0344 - accuracy: 0.9887
1637/1688 [============================>.] - ETA: 0s - loss: 0.0346 - accuracy: 0.9887
1658/1688 [============================>.] - ETA: 0s - loss: 0.0349 - accuracy: 0.9887
1679/1688 [============================>.] - ETA: 0s - loss: 0.0349 - accuracy: 0.9886
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0349 - accuracy: 0.9887 - val_loss: 0.0542 - val_accuracy: 0.9855
Epoch 6/10
1/1688 [..............................] - ETA: 4s - loss: 0.0038 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0255 - accuracy: 0.9891
45/1688 [..............................] - ETA: 3s - loss: 0.0259 - accuracy: 0.9910
68/1688 [>.............................] - ETA: 3s - loss: 0.0255 - accuracy: 0.9926
91/1688 [>.............................] - ETA: 3s - loss: 0.0261 - accuracy: 0.9924
114/1688 [=>............................] - ETA: 3s - loss: 0.0254 - accuracy: 0.9923
137/1688 [=>............................] - ETA: 3s - loss: 0.0238 - accuracy: 0.9925
160/1688 [=>............................] - ETA: 3s - loss: 0.0237 - accuracy: 0.9922
184/1688 [==>...........................] - ETA: 3s - loss: 0.0237 - accuracy: 0.9927
207/1688 [==>...........................] - ETA: 3s - loss: 0.0230 - accuracy: 0.9929
230/1688 [===>..........................] - ETA: 3s - loss: 0.0228 - accuracy: 0.9931
253/1688 [===>..........................] - ETA: 3s - loss: 0.0233 - accuracy: 0.9925
276/1688 [===>..........................] - ETA: 3s - loss: 0.0236 - accuracy: 0.9926
298/1688 [====>.........................] - ETA: 3s - loss: 0.0231 - accuracy: 0.9930
321/1688 [====>.........................] - ETA: 3s - loss: 0.0238 - accuracy: 0.9929
344/1688 [=====>........................] - ETA: 3s - loss: 0.0235 - accuracy: 0.9932
366/1688 [=====>........................] - ETA: 2s - loss: 0.0236 - accuracy: 0.9931
389/1688 [=====>........................] - ETA: 2s - loss: 0.0233 - accuracy: 0.9932
412/1688 [======>.......................] - ETA: 2s - loss: 0.0235 - accuracy: 0.9930
435/1688 [======>.......................] - ETA: 2s - loss: 0.0244 - accuracy: 0.9925
458/1688 [=======>......................] - ETA: 2s - loss: 0.0242 - accuracy: 0.9924
481/1688 [=======>......................] - ETA: 2s - loss: 0.0239 - accuracy: 0.9923
504/1688 [=======>......................] - ETA: 2s - loss: 0.0242 - accuracy: 0.9922
527/1688 [========>.....................] - ETA: 2s - loss: 0.0245 - accuracy: 0.9922
550/1688 [========>.....................] - ETA: 2s - loss: 0.0243 - accuracy: 0.9921
573/1688 [=========>....................] - ETA: 2s - loss: 0.0250 - accuracy: 0.9915
596/1688 [=========>....................] - ETA: 2s - loss: 0.0254 - accuracy: 0.9915
619/1688 [==========>...................] - ETA: 2s - loss: 0.0252 - accuracy: 0.9915
640/1688 [==========>...................] - ETA: 2s - loss: 0.0251 - accuracy: 0.9915
661/1688 [==========>...................] - ETA: 2s - loss: 0.0253 - accuracy: 0.9915
682/1688 [===========>..................] - ETA: 2s - loss: 0.0259 - accuracy: 0.9914
703/1688 [===========>..................] - ETA: 2s - loss: 0.0262 - accuracy: 0.9913
724/1688 [===========>..................] - ETA: 2s - loss: 0.0267 - accuracy: 0.9912
745/1688 [============>.................] - ETA: 2s - loss: 0.0269 - accuracy: 0.9911
766/1688 [============>.................] - ETA: 2s - loss: 0.0272 - accuracy: 0.9909
787/1688 [============>.................] - ETA: 2s - loss: 0.0271 - accuracy: 0.9910
808/1688 [=============>................] - ETA: 2s - loss: 0.0270 - accuracy: 0.9910
829/1688 [=============>................] - ETA: 1s - loss: 0.0274 - accuracy: 0.9908
850/1688 [==============>...............] - ETA: 1s - loss: 0.0276 - accuracy: 0.9908
871/1688 [==============>...............] - ETA: 1s - loss: 0.0276 - accuracy: 0.9907
893/1688 [==============>...............] - ETA: 1s - loss: 0.0276 - accuracy: 0.9907
915/1688 [===============>..............] - ETA: 1s - loss: 0.0276 - accuracy: 0.9907
936/1688 [===============>..............] - ETA: 1s - loss: 0.0280 - accuracy: 0.9906
957/1688 [================>.............] - ETA: 1s - loss: 0.0279 - accuracy: 0.9906
978/1688 [================>.............] - ETA: 1s - loss: 0.0279 - accuracy: 0.9906
999/1688 [================>.............] - ETA: 1s - loss: 0.0279 - accuracy: 0.9906
1020/1688 [=================>............] - ETA: 1s - loss: 0.0279 - accuracy: 0.9906
1041/1688 [=================>............] - ETA: 1s - loss: 0.0278 - accuracy: 0.9906
1062/1688 [=================>............] - ETA: 1s - loss: 0.0276 - accuracy: 0.9905
1084/1688 [==================>...........] - ETA: 1s - loss: 0.0275 - accuracy: 0.9906
1105/1688 [==================>...........] - ETA: 1s - loss: 0.0273 - accuracy: 0.9906
1126/1688 [===================>..........] - ETA: 1s - loss: 0.0277 - accuracy: 0.9906
1147/1688 [===================>..........] - ETA: 1s - loss: 0.0279 - accuracy: 0.9905
1169/1688 [===================>..........] - ETA: 1s - loss: 0.0279 - accuracy: 0.9904
1190/1688 [====================>.........] - ETA: 1s - loss: 0.0279 - accuracy: 0.9904
1211/1688 [====================>.........] - ETA: 1s - loss: 0.0279 - accuracy: 0.9904
1232/1688 [====================>.........] - ETA: 1s - loss: 0.0279 - accuracy: 0.9904
1254/1688 [=====================>........] - ETA: 1s - loss: 0.0281 - accuracy: 0.9904
1275/1688 [=====================>........] - ETA: 0s - loss: 0.0282 - accuracy: 0.9904
1296/1688 [======================>.......] - ETA: 0s - loss: 0.0281 - accuracy: 0.9905
1317/1688 [======================>.......] - ETA: 0s - loss: 0.0281 - accuracy: 0.9905
1339/1688 [======================>.......] - ETA: 0s - loss: 0.0282 - accuracy: 0.9905
1360/1688 [=======================>......] - ETA: 0s - loss: 0.0284 - accuracy: 0.9904
1381/1688 [=======================>......] - ETA: 0s - loss: 0.0286 - accuracy: 0.9904
1402/1688 [=======================>......] - ETA: 0s - loss: 0.0288 - accuracy: 0.9903
1424/1688 [========================>.....] - ETA: 0s - loss: 0.0287 - accuracy: 0.9904
1445/1688 [========================>.....] - ETA: 0s - loss: 0.0286 - accuracy: 0.9904
1467/1688 [=========================>....] - ETA: 0s - loss: 0.0286 - accuracy: 0.9904
1488/1688 [=========================>....] - ETA: 0s - loss: 0.0285 - accuracy: 0.9904
1509/1688 [=========================>....] - ETA: 0s - loss: 0.0284 - accuracy: 0.9905
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0283 - accuracy: 0.9905
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0284 - accuracy: 0.9904
1572/1688 [==========================>...] - ETA: 0s - loss: 0.0290 - accuracy: 0.9902
1593/1688 [===========================>..] - ETA: 0s - loss: 0.0291 - accuracy: 0.9902
1614/1688 [===========================>..] - ETA: 0s - loss: 0.0290 - accuracy: 0.9902
1635/1688 [============================>.] - ETA: 0s - loss: 0.0290 - accuracy: 0.9903
1657/1688 [============================>.] - ETA: 0s - loss: 0.0290 - accuracy: 0.9902
1678/1688 [============================>.] - ETA: 0s - loss: 0.0289 - accuracy: 0.9902
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0289 - accuracy: 0.9902 - val_loss: 0.0613 - val_accuracy: 0.9830
Epoch 7/10
1/1688 [..............................] - ETA: 3s - loss: 0.0034 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0221 - accuracy: 0.9909
47/1688 [..............................] - ETA: 3s - loss: 0.0186 - accuracy: 0.9940
70/1688 [>.............................] - ETA: 3s - loss: 0.0190 - accuracy: 0.9942
93/1688 [>.............................] - ETA: 3s - loss: 0.0167 - accuracy: 0.9943
115/1688 [=>............................] - ETA: 3s - loss: 0.0172 - accuracy: 0.9935
136/1688 [=>............................] - ETA: 3s - loss: 0.0180 - accuracy: 0.9936
157/1688 [=>............................] - ETA: 3s - loss: 0.0170 - accuracy: 0.9940
178/1688 [==>...........................] - ETA: 3s - loss: 0.0181 - accuracy: 0.9930
199/1688 [==>...........................] - ETA: 3s - loss: 0.0182 - accuracy: 0.9934
220/1688 [==>...........................] - ETA: 3s - loss: 0.0193 - accuracy: 0.9929
242/1688 [===>..........................] - ETA: 3s - loss: 0.0199 - accuracy: 0.9928
264/1688 [===>..........................] - ETA: 3s - loss: 0.0196 - accuracy: 0.9931
286/1688 [====>.........................] - ETA: 3s - loss: 0.0197 - accuracy: 0.9931
308/1688 [====>.........................] - ETA: 3s - loss: 0.0202 - accuracy: 0.9929
329/1688 [====>.........................] - ETA: 3s - loss: 0.0205 - accuracy: 0.9929
350/1688 [=====>........................] - ETA: 3s - loss: 0.0207 - accuracy: 0.9927
371/1688 [=====>........................] - ETA: 3s - loss: 0.0208 - accuracy: 0.9926
392/1688 [=====>........................] - ETA: 3s - loss: 0.0220 - accuracy: 0.9923
413/1688 [======>.......................] - ETA: 3s - loss: 0.0225 - accuracy: 0.9921
434/1688 [======>.......................] - ETA: 2s - loss: 0.0225 - accuracy: 0.9921
454/1688 [=======>......................] - ETA: 2s - loss: 0.0229 - accuracy: 0.9922
472/1688 [=======>......................] - ETA: 2s - loss: 0.0229 - accuracy: 0.9921
492/1688 [=======>......................] - ETA: 2s - loss: 0.0230 - accuracy: 0.9920
512/1688 [========>.....................] - ETA: 2s - loss: 0.0235 - accuracy: 0.9921
532/1688 [========>.....................] - ETA: 2s - loss: 0.0233 - accuracy: 0.9922
552/1688 [========>.....................] - ETA: 2s - loss: 0.0234 - accuracy: 0.9922
572/1688 [=========>....................] - ETA: 2s - loss: 0.0237 - accuracy: 0.9922
592/1688 [=========>....................] - ETA: 2s - loss: 0.0238 - accuracy: 0.9922
612/1688 [=========>....................] - ETA: 2s - loss: 0.0238 - accuracy: 0.9921
632/1688 [==========>...................] - ETA: 2s - loss: 0.0237 - accuracy: 0.9921
652/1688 [==========>...................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9919
672/1688 [==========>...................] - ETA: 2s - loss: 0.0248 - accuracy: 0.9918
692/1688 [===========>..................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9918
712/1688 [===========>..................] - ETA: 2s - loss: 0.0249 - accuracy: 0.9917
732/1688 [============>.................] - ETA: 2s - loss: 0.0248 - accuracy: 0.9918
752/1688 [============>.................] - ETA: 2s - loss: 0.0249 - accuracy: 0.9916
772/1688 [============>.................] - ETA: 2s - loss: 0.0248 - accuracy: 0.9917
792/1688 [=============>................] - ETA: 2s - loss: 0.0253 - accuracy: 0.9916
811/1688 [=============>................] - ETA: 2s - loss: 0.0252 - accuracy: 0.9916
831/1688 [=============>................] - ETA: 2s - loss: 0.0248 - accuracy: 0.9917
851/1688 [==============>...............] - ETA: 2s - loss: 0.0246 - accuracy: 0.9918
871/1688 [==============>...............] - ETA: 2s - loss: 0.0243 - accuracy: 0.9919
891/1688 [==============>...............] - ETA: 1s - loss: 0.0243 - accuracy: 0.9919
911/1688 [===============>..............] - ETA: 1s - loss: 0.0242 - accuracy: 0.9919
931/1688 [===============>..............] - ETA: 1s - loss: 0.0243 - accuracy: 0.9919
951/1688 [===============>..............] - ETA: 1s - loss: 0.0242 - accuracy: 0.9919
971/1688 [================>.............] - ETA: 1s - loss: 0.0243 - accuracy: 0.9920
991/1688 [================>.............] - ETA: 1s - loss: 0.0243 - accuracy: 0.9919
1011/1688 [================>.............] - ETA: 1s - loss: 0.0246 - accuracy: 0.9917
1031/1688 [=================>............] - ETA: 1s - loss: 0.0245 - accuracy: 0.9918
1051/1688 [=================>............] - ETA: 1s - loss: 0.0245 - accuracy: 0.9918
1071/1688 [==================>...........] - ETA: 1s - loss: 0.0247 - accuracy: 0.9917
1091/1688 [==================>...........] - ETA: 1s - loss: 0.0245 - accuracy: 0.9916
1111/1688 [==================>...........] - ETA: 1s - loss: 0.0244 - accuracy: 0.9917
1131/1688 [===================>..........] - ETA: 1s - loss: 0.0243 - accuracy: 0.9918
1151/1688 [===================>..........] - ETA: 1s - loss: 0.0245 - accuracy: 0.9917
1171/1688 [===================>..........] - ETA: 1s - loss: 0.0246 - accuracy: 0.9916
1191/1688 [====================>.........] - ETA: 1s - loss: 0.0245 - accuracy: 0.9917
1210/1688 [====================>.........] - ETA: 1s - loss: 0.0244 - accuracy: 0.9917
1230/1688 [====================>.........] - ETA: 1s - loss: 0.0243 - accuracy: 0.9917
1250/1688 [=====================>........] - ETA: 1s - loss: 0.0244 - accuracy: 0.9917
1270/1688 [=====================>........] - ETA: 1s - loss: 0.0244 - accuracy: 0.9917
1290/1688 [=====================>........] - ETA: 0s - loss: 0.0242 - accuracy: 0.9917
1310/1688 [======================>.......] - ETA: 0s - loss: 0.0242 - accuracy: 0.9917
1330/1688 [======================>.......] - ETA: 0s - loss: 0.0240 - accuracy: 0.9918
1350/1688 [======================>.......] - ETA: 0s - loss: 0.0241 - accuracy: 0.9918
1370/1688 [=======================>......] - ETA: 0s - loss: 0.0241 - accuracy: 0.9917
1390/1688 [=======================>......] - ETA: 0s - loss: 0.0242 - accuracy: 0.9917
1410/1688 [========================>.....] - ETA: 0s - loss: 0.0243 - accuracy: 0.9916
1430/1688 [========================>.....] - ETA: 0s - loss: 0.0246 - accuracy: 0.9915
1450/1688 [========================>.....] - ETA: 0s - loss: 0.0244 - accuracy: 0.9916
1470/1688 [=========================>....] - ETA: 0s - loss: 0.0246 - accuracy: 0.9915
1490/1688 [=========================>....] - ETA: 0s - loss: 0.0245 - accuracy: 0.9915
1510/1688 [=========================>....] - ETA: 0s - loss: 0.0245 - accuracy: 0.9915
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0247 - accuracy: 0.9915
1550/1688 [==========================>...] - ETA: 0s - loss: 0.0247 - accuracy: 0.9914
1570/1688 [==========================>...] - ETA: 0s - loss: 0.0247 - accuracy: 0.9914
1590/1688 [===========================>..] - ETA: 0s - loss: 0.0247 - accuracy: 0.9914
1610/1688 [===========================>..] - ETA: 0s - loss: 0.0246 - accuracy: 0.9914
1630/1688 [===========================>..] - ETA: 0s - loss: 0.0245 - accuracy: 0.9914
1650/1688 [============================>.] - ETA: 0s - loss: 0.0249 - accuracy: 0.9914
1670/1688 [============================>.] - ETA: 0s - loss: 0.0251 - accuracy: 0.9913
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0250 - accuracy: 0.9913 - val_loss: 0.1103 - val_accuracy: 0.9710
Epoch 8/10
1/1688 [..............................] - ETA: 4s - loss: 0.0082 - accuracy: 1.0000
21/1688 [..............................] - ETA: 4s - loss: 0.0196 - accuracy: 0.9926
41/1688 [..............................] - ETA: 4s - loss: 0.0202 - accuracy: 0.9931
61/1688 [>.............................] - ETA: 4s - loss: 0.0177 - accuracy: 0.9939
81/1688 [>.............................] - ETA: 4s - loss: 0.0156 - accuracy: 0.9950
101/1688 [>.............................] - ETA: 4s - loss: 0.0158 - accuracy: 0.9944
121/1688 [=>............................] - ETA: 4s - loss: 0.0170 - accuracy: 0.9943
141/1688 [=>............................] - ETA: 3s - loss: 0.0184 - accuracy: 0.9936
161/1688 [=>............................] - ETA: 3s - loss: 0.0190 - accuracy: 0.9930
181/1688 [==>...........................] - ETA: 3s - loss: 0.0195 - accuracy: 0.9929
201/1688 [==>...........................] - ETA: 3s - loss: 0.0197 - accuracy: 0.9930
221/1688 [==>...........................] - ETA: 3s - loss: 0.0196 - accuracy: 0.9929
241/1688 [===>..........................] - ETA: 3s - loss: 0.0204 - accuracy: 0.9926
261/1688 [===>..........................] - ETA: 3s - loss: 0.0200 - accuracy: 0.9928
281/1688 [===>..........................] - ETA: 3s - loss: 0.0198 - accuracy: 0.9929
301/1688 [====>.........................] - ETA: 3s - loss: 0.0199 - accuracy: 0.9928
321/1688 [====>.........................] - ETA: 3s - loss: 0.0200 - accuracy: 0.9929
341/1688 [=====>........................] - ETA: 3s - loss: 0.0208 - accuracy: 0.9924
361/1688 [=====>........................] - ETA: 3s - loss: 0.0207 - accuracy: 0.9925
381/1688 [=====>........................] - ETA: 3s - loss: 0.0210 - accuracy: 0.9923
401/1688 [======>.......................] - ETA: 3s - loss: 0.0207 - accuracy: 0.9923
421/1688 [======>.......................] - ETA: 3s - loss: 0.0209 - accuracy: 0.9922
441/1688 [======>.......................] - ETA: 3s - loss: 0.0208 - accuracy: 0.9923
461/1688 [=======>......................] - ETA: 3s - loss: 0.0206 - accuracy: 0.9923
481/1688 [=======>......................] - ETA: 3s - loss: 0.0205 - accuracy: 0.9922
501/1688 [=======>......................] - ETA: 3s - loss: 0.0202 - accuracy: 0.9923
521/1688 [========>.....................] - ETA: 3s - loss: 0.0200 - accuracy: 0.9924
541/1688 [========>.....................] - ETA: 2s - loss: 0.0198 - accuracy: 0.9927
561/1688 [========>.....................] - ETA: 2s - loss: 0.0196 - accuracy: 0.9928
581/1688 [=========>....................] - ETA: 2s - loss: 0.0193 - accuracy: 0.9929
601/1688 [=========>....................] - ETA: 2s - loss: 0.0192 - accuracy: 0.9930
620/1688 [==========>...................] - ETA: 2s - loss: 0.0194 - accuracy: 0.9930
640/1688 [==========>...................] - ETA: 2s - loss: 0.0193 - accuracy: 0.9931
660/1688 [==========>...................] - ETA: 2s - loss: 0.0191 - accuracy: 0.9932
680/1688 [===========>..................] - ETA: 2s - loss: 0.0191 - accuracy: 0.9932
700/1688 [===========>..................] - ETA: 2s - loss: 0.0191 - accuracy: 0.9932
720/1688 [===========>..................] - ETA: 2s - loss: 0.0189 - accuracy: 0.9934
740/1688 [============>.................] - ETA: 2s - loss: 0.0191 - accuracy: 0.9933
760/1688 [============>.................] - ETA: 2s - loss: 0.0189 - accuracy: 0.9933
780/1688 [============>.................] - ETA: 2s - loss: 0.0189 - accuracy: 0.9933
800/1688 [=============>................] - ETA: 2s - loss: 0.0186 - accuracy: 0.9934
820/1688 [=============>................] - ETA: 2s - loss: 0.0187 - accuracy: 0.9933
840/1688 [=============>................] - ETA: 2s - loss: 0.0188 - accuracy: 0.9933
860/1688 [==============>...............] - ETA: 2s - loss: 0.0188 - accuracy: 0.9932
880/1688 [==============>...............] - ETA: 2s - loss: 0.0185 - accuracy: 0.9934
900/1688 [==============>...............] - ETA: 2s - loss: 0.0188 - accuracy: 0.9932
920/1688 [===============>..............] - ETA: 1s - loss: 0.0190 - accuracy: 0.9931
940/1688 [===============>..............] - ETA: 1s - loss: 0.0190 - accuracy: 0.9931
959/1688 [================>.............] - ETA: 1s - loss: 0.0189 - accuracy: 0.9932
979/1688 [================>.............] - ETA: 1s - loss: 0.0192 - accuracy: 0.9931
999/1688 [================>.............] - ETA: 1s - loss: 0.0192 - accuracy: 0.9931
1019/1688 [=================>............] - ETA: 1s - loss: 0.0193 - accuracy: 0.9931
1039/1688 [=================>............] - ETA: 1s - loss: 0.0192 - accuracy: 0.9932
1059/1688 [=================>............] - ETA: 1s - loss: 0.0191 - accuracy: 0.9932
1079/1688 [==================>...........] - ETA: 1s - loss: 0.0192 - accuracy: 0.9932
1099/1688 [==================>...........] - ETA: 1s - loss: 0.0190 - accuracy: 0.9933
1119/1688 [==================>...........] - ETA: 1s - loss: 0.0190 - accuracy: 0.9933
1139/1688 [===================>..........] - ETA: 1s - loss: 0.0188 - accuracy: 0.9934
1160/1688 [===================>..........] - ETA: 1s - loss: 0.0188 - accuracy: 0.9934
1180/1688 [===================>..........] - ETA: 1s - loss: 0.0189 - accuracy: 0.9933
1200/1688 [====================>.........] - ETA: 1s - loss: 0.0190 - accuracy: 0.9933
1220/1688 [====================>.........] - ETA: 1s - loss: 0.0190 - accuracy: 0.9933
1240/1688 [=====================>........] - ETA: 1s - loss: 0.0194 - accuracy: 0.9931
1260/1688 [=====================>........] - ETA: 1s - loss: 0.0195 - accuracy: 0.9930
1280/1688 [=====================>........] - ETA: 1s - loss: 0.0196 - accuracy: 0.9930
1300/1688 [======================>.......] - ETA: 1s - loss: 0.0198 - accuracy: 0.9930
1320/1688 [======================>.......] - ETA: 0s - loss: 0.0198 - accuracy: 0.9930
1340/1688 [======================>.......] - ETA: 0s - loss: 0.0198 - accuracy: 0.9930
1360/1688 [=======================>......] - ETA: 0s - loss: 0.0198 - accuracy: 0.9930
1380/1688 [=======================>......] - ETA: 0s - loss: 0.0201 - accuracy: 0.9928
1400/1688 [=======================>......] - ETA: 0s - loss: 0.0201 - accuracy: 0.9929
1420/1688 [========================>.....] - ETA: 0s - loss: 0.0202 - accuracy: 0.9928
1440/1688 [========================>.....] - ETA: 0s - loss: 0.0202 - accuracy: 0.9928
1460/1688 [========================>.....] - ETA: 0s - loss: 0.0206 - accuracy: 0.9927
1480/1688 [=========================>....] - ETA: 0s - loss: 0.0206 - accuracy: 0.9927
1500/1688 [=========================>....] - ETA: 0s - loss: 0.0207 - accuracy: 0.9927
1519/1688 [=========================>....] - ETA: 0s - loss: 0.0206 - accuracy: 0.9927
1539/1688 [==========================>...] - ETA: 0s - loss: 0.0206 - accuracy: 0.9926
1559/1688 [==========================>...] - ETA: 0s - loss: 0.0206 - accuracy: 0.9927
1579/1688 [===========================>..] - ETA: 0s - loss: 0.0205 - accuracy: 0.9928
1599/1688 [===========================>..] - ETA: 0s - loss: 0.0204 - accuracy: 0.9928
1619/1688 [===========================>..] - ETA: 0s - loss: 0.0205 - accuracy: 0.9928
1639/1688 [============================>.] - ETA: 0s - loss: 0.0204 - accuracy: 0.9928
1659/1688 [============================>.] - ETA: 0s - loss: 0.0205 - accuracy: 0.9927
1679/1688 [============================>.] - ETA: 0s - loss: 0.0205 - accuracy: 0.9927
1688/1688 [==============================] - 5s 3ms/step - loss: 0.0208 - accuracy: 0.9926 - val_loss: 0.0502 - val_accuracy: 0.9875
Epoch 9/10
1/1688 [..............................] - ETA: 3s - loss: 0.0036 - accuracy: 1.0000
25/1688 [..............................] - ETA: 3s - loss: 0.0275 - accuracy: 0.9912
48/1688 [..............................] - ETA: 3s - loss: 0.0253 - accuracy: 0.9922
70/1688 [>.............................] - ETA: 3s - loss: 0.0201 - accuracy: 0.9937
92/1688 [>.............................] - ETA: 3s - loss: 0.0202 - accuracy: 0.9942
115/1688 [=>............................] - ETA: 3s - loss: 0.0182 - accuracy: 0.9946
138/1688 [=>............................] - ETA: 3s - loss: 0.0175 - accuracy: 0.9943
161/1688 [=>............................] - ETA: 3s - loss: 0.0168 - accuracy: 0.9944
184/1688 [==>...........................] - ETA: 3s - loss: 0.0159 - accuracy: 0.9947
207/1688 [==>...........................] - ETA: 3s - loss: 0.0151 - accuracy: 0.9952
230/1688 [===>..........................] - ETA: 3s - loss: 0.0144 - accuracy: 0.9952
253/1688 [===>..........................] - ETA: 3s - loss: 0.0138 - accuracy: 0.9954
275/1688 [===>..........................] - ETA: 3s - loss: 0.0134 - accuracy: 0.9956
298/1688 [====>.........................] - ETA: 3s - loss: 0.0138 - accuracy: 0.9954
322/1688 [====>.........................] - ETA: 3s - loss: 0.0136 - accuracy: 0.9955
345/1688 [=====>........................] - ETA: 2s - loss: 0.0137 - accuracy: 0.9955
366/1688 [=====>........................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9950
387/1688 [=====>........................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9946
409/1688 [======>.......................] - ETA: 2s - loss: 0.0162 - accuracy: 0.9943
430/1688 [======>.......................] - ETA: 2s - loss: 0.0158 - accuracy: 0.9945
452/1688 [=======>......................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9944
474/1688 [=======>......................] - ETA: 2s - loss: 0.0162 - accuracy: 0.9941
495/1688 [=======>......................] - ETA: 2s - loss: 0.0164 - accuracy: 0.9941
516/1688 [========>.....................] - ETA: 2s - loss: 0.0162 - accuracy: 0.9942
537/1688 [========>.....................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9943
558/1688 [========>.....................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9945
579/1688 [=========>....................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9945
600/1688 [=========>....................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9945
622/1688 [==========>...................] - ETA: 2s - loss: 0.0163 - accuracy: 0.9944
644/1688 [==========>...................] - ETA: 2s - loss: 0.0161 - accuracy: 0.9945
665/1688 [==========>...................] - ETA: 2s - loss: 0.0164 - accuracy: 0.9944
687/1688 [===========>..................] - ETA: 2s - loss: 0.0166 - accuracy: 0.9943
708/1688 [===========>..................] - ETA: 2s - loss: 0.0167 - accuracy: 0.9942
729/1688 [===========>..................] - ETA: 2s - loss: 0.0170 - accuracy: 0.9942
750/1688 [============>.................] - ETA: 2s - loss: 0.0169 - accuracy: 0.9942
771/1688 [============>.................] - ETA: 2s - loss: 0.0170 - accuracy: 0.9941
793/1688 [=============>................] - ETA: 2s - loss: 0.0173 - accuracy: 0.9940
814/1688 [=============>................] - ETA: 2s - loss: 0.0174 - accuracy: 0.9939
835/1688 [=============>................] - ETA: 1s - loss: 0.0178 - accuracy: 0.9938
857/1688 [==============>...............] - ETA: 1s - loss: 0.0175 - accuracy: 0.9940
879/1688 [==============>...............] - ETA: 1s - loss: 0.0173 - accuracy: 0.9940
901/1688 [===============>..............] - ETA: 1s - loss: 0.0174 - accuracy: 0.9940
922/1688 [===============>..............] - ETA: 1s - loss: 0.0172 - accuracy: 0.9941
943/1688 [===============>..............] - ETA: 1s - loss: 0.0171 - accuracy: 0.9941
964/1688 [================>.............] - ETA: 1s - loss: 0.0169 - accuracy: 0.9941
985/1688 [================>.............] - ETA: 1s - loss: 0.0169 - accuracy: 0.9942
1007/1688 [================>.............] - ETA: 1s - loss: 0.0174 - accuracy: 0.9941
1028/1688 [=================>............] - ETA: 1s - loss: 0.0173 - accuracy: 0.9942
1050/1688 [=================>............] - ETA: 1s - loss: 0.0172 - accuracy: 0.9942
1071/1688 [==================>...........] - ETA: 1s - loss: 0.0171 - accuracy: 0.9942
1092/1688 [==================>...........] - ETA: 1s - loss: 0.0171 - accuracy: 0.9942
1113/1688 [==================>...........] - ETA: 1s - loss: 0.0169 - accuracy: 0.9943
1135/1688 [===================>..........] - ETA: 1s - loss: 0.0168 - accuracy: 0.9944
1156/1688 [===================>..........] - ETA: 1s - loss: 0.0168 - accuracy: 0.9944
1177/1688 [===================>..........] - ETA: 1s - loss: 0.0167 - accuracy: 0.9944
1199/1688 [====================>.........] - ETA: 1s - loss: 0.0167 - accuracy: 0.9944
1220/1688 [====================>.........] - ETA: 1s - loss: 0.0170 - accuracy: 0.9943
1241/1688 [=====================>........] - ETA: 1s - loss: 0.0170 - accuracy: 0.9943
1262/1688 [=====================>........] - ETA: 1s - loss: 0.0172 - accuracy: 0.9943
1284/1688 [=====================>........] - ETA: 0s - loss: 0.0173 - accuracy: 0.9943
1305/1688 [======================>.......] - ETA: 0s - loss: 0.0173 - accuracy: 0.9943
1327/1688 [======================>.......] - ETA: 0s - loss: 0.0172 - accuracy: 0.9943
1349/1688 [======================>.......] - ETA: 0s - loss: 0.0171 - accuracy: 0.9943
1370/1688 [=======================>......] - ETA: 0s - loss: 0.0173 - accuracy: 0.9943
1391/1688 [=======================>......] - ETA: 0s - loss: 0.0172 - accuracy: 0.9943
1412/1688 [========================>.....] - ETA: 0s - loss: 0.0171 - accuracy: 0.9943
1432/1688 [========================>.....] - ETA: 0s - loss: 0.0174 - accuracy: 0.9942
1452/1688 [========================>.....] - ETA: 0s - loss: 0.0175 - accuracy: 0.9942
1472/1688 [=========================>....] - ETA: 0s - loss: 0.0176 - accuracy: 0.9941
1491/1688 [=========================>....] - ETA: 0s - loss: 0.0176 - accuracy: 0.9941
1511/1688 [=========================>....] - ETA: 0s - loss: 0.0179 - accuracy: 0.9940
1531/1688 [==========================>...] - ETA: 0s - loss: 0.0182 - accuracy: 0.9939
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0182 - accuracy: 0.9939
1571/1688 [==========================>...] - ETA: 0s - loss: 0.0183 - accuracy: 0.9939
1591/1688 [===========================>..] - ETA: 0s - loss: 0.0184 - accuracy: 0.9938
1611/1688 [===========================>..] - ETA: 0s - loss: 0.0186 - accuracy: 0.9938
1631/1688 [===========================>..] - ETA: 0s - loss: 0.0187 - accuracy: 0.9937
1651/1688 [============================>.] - ETA: 0s - loss: 0.0186 - accuracy: 0.9937
1672/1688 [============================>.] - ETA: 0s - loss: 0.0189 - accuracy: 0.9936
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0192 - accuracy: 0.9935 - val_loss: 0.0591 - val_accuracy: 0.9852
Epoch 10/10
1/1688 [..............................] - ETA: 3s - loss: 5.5429e-05 - accuracy: 1.0000
25/1688 [..............................] - ETA: 3s - loss: 0.0114 - accuracy: 0.9975
48/1688 [..............................] - ETA: 3s - loss: 0.0107 - accuracy: 0.9967
71/1688 [>.............................] - ETA: 3s - loss: 0.0103 - accuracy: 0.9965
94/1688 [>.............................] - ETA: 3s - loss: 0.0105 - accuracy: 0.9957
117/1688 [=>............................] - ETA: 3s - loss: 0.0142 - accuracy: 0.9944
140/1688 [=>............................] - ETA: 3s - loss: 0.0164 - accuracy: 0.9940
163/1688 [=>............................] - ETA: 3s - loss: 0.0169 - accuracy: 0.9937
186/1688 [==>...........................] - ETA: 3s - loss: 0.0169 - accuracy: 0.9936
209/1688 [==>...........................] - ETA: 3s - loss: 0.0160 - accuracy: 0.9942
231/1688 [===>..........................] - ETA: 3s - loss: 0.0151 - accuracy: 0.9946
252/1688 [===>..........................] - ETA: 3s - loss: 0.0153 - accuracy: 0.9947
273/1688 [===>..........................] - ETA: 3s - loss: 0.0146 - accuracy: 0.9950
294/1688 [====>.........................] - ETA: 3s - loss: 0.0153 - accuracy: 0.9947
315/1688 [====>.........................] - ETA: 3s - loss: 0.0153 - accuracy: 0.9947
336/1688 [====>.........................] - ETA: 3s - loss: 0.0155 - accuracy: 0.9947
358/1688 [=====>........................] - ETA: 3s - loss: 0.0153 - accuracy: 0.9948
378/1688 [=====>........................] - ETA: 3s - loss: 0.0147 - accuracy: 0.9950
398/1688 [======>.......................] - ETA: 3s - loss: 0.0144 - accuracy: 0.9953
418/1688 [======>.......................] - ETA: 2s - loss: 0.0141 - accuracy: 0.9954
438/1688 [======>.......................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9953
458/1688 [=======>......................] - ETA: 2s - loss: 0.0141 - accuracy: 0.9954
478/1688 [=======>......................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9952
498/1688 [=======>......................] - ETA: 2s - loss: 0.0143 - accuracy: 0.9954
518/1688 [========>.....................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9954
538/1688 [========>.....................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9952
558/1688 [========>.....................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9951
578/1688 [=========>....................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9951
598/1688 [=========>....................] - ETA: 2s - loss: 0.0148 - accuracy: 0.9951
618/1688 [=========>....................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9951
638/1688 [==========>...................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9952
658/1688 [==========>...................] - ETA: 2s - loss: 0.0148 - accuracy: 0.9951
679/1688 [===========>..................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9949
699/1688 [===========>..................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9949
719/1688 [===========>..................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9950
739/1688 [============>.................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9950
759/1688 [============>.................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9949
779/1688 [============>.................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9949
799/1688 [=============>................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9950
819/1688 [=============>................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9950
839/1688 [=============>................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9948
859/1688 [==============>...............] - ETA: 2s - loss: 0.0152 - accuracy: 0.9949
879/1688 [==============>...............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9948
899/1688 [==============>...............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9949
918/1688 [===============>..............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9948
938/1688 [===============>..............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9948
958/1688 [================>.............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9949
978/1688 [================>.............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9949
998/1688 [================>.............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9949
1018/1688 [=================>............] - ETA: 1s - loss: 0.0151 - accuracy: 0.9949
1038/1688 [=================>............] - ETA: 1s - loss: 0.0153 - accuracy: 0.9949
1058/1688 [=================>............] - ETA: 1s - loss: 0.0151 - accuracy: 0.9949
1078/1688 [==================>...........] - ETA: 1s - loss: 0.0151 - accuracy: 0.9949
1098/1688 [==================>...........] - ETA: 1s - loss: 0.0151 - accuracy: 0.9949
1118/1688 [==================>...........] - ETA: 1s - loss: 0.0151 - accuracy: 0.9949
1138/1688 [===================>..........] - ETA: 1s - loss: 0.0151 - accuracy: 0.9949
1158/1688 [===================>..........] - ETA: 1s - loss: 0.0150 - accuracy: 0.9949
1178/1688 [===================>..........] - ETA: 1s - loss: 0.0150 - accuracy: 0.9948
1198/1688 [====================>.........] - ETA: 1s - loss: 0.0149 - accuracy: 0.9949
1217/1688 [====================>.........] - ETA: 1s - loss: 0.0149 - accuracy: 0.9949
1237/1688 [====================>.........] - ETA: 1s - loss: 0.0148 - accuracy: 0.9950
1256/1688 [=====================>........] - ETA: 1s - loss: 0.0148 - accuracy: 0.9950
1276/1688 [=====================>........] - ETA: 1s - loss: 0.0149 - accuracy: 0.9950
1296/1688 [======================>.......] - ETA: 0s - loss: 0.0149 - accuracy: 0.9950
1315/1688 [======================>.......] - ETA: 0s - loss: 0.0150 - accuracy: 0.9950
1335/1688 [======================>.......] - ETA: 0s - loss: 0.0151 - accuracy: 0.9950
1355/1688 [=======================>......] - ETA: 0s - loss: 0.0151 - accuracy: 0.9949
1375/1688 [=======================>......] - ETA: 0s - loss: 0.0150 - accuracy: 0.9950
1395/1688 [=======================>......] - ETA: 0s - loss: 0.0151 - accuracy: 0.9949
1415/1688 [========================>.....] - ETA: 0s - loss: 0.0153 - accuracy: 0.9948
1435/1688 [========================>.....] - ETA: 0s - loss: 0.0152 - accuracy: 0.9948
1455/1688 [========================>.....] - ETA: 0s - loss: 0.0152 - accuracy: 0.9948
1475/1688 [=========================>....] - ETA: 0s - loss: 0.0153 - accuracy: 0.9947
1495/1688 [=========================>....] - ETA: 0s - loss: 0.0155 - accuracy: 0.9947
1515/1688 [=========================>....] - ETA: 0s - loss: 0.0154 - accuracy: 0.9947
1535/1688 [==========================>...] - ETA: 0s - loss: 0.0154 - accuracy: 0.9947
1555/1688 [==========================>...] - ETA: 0s - loss: 0.0156 - accuracy: 0.9946
1575/1688 [==========================>...] - ETA: 0s - loss: 0.0157 - accuracy: 0.9946
1595/1688 [===========================>..] - ETA: 0s - loss: 0.0157 - accuracy: 0.9945
1615/1688 [===========================>..] - ETA: 0s - loss: 0.0155 - accuracy: 0.9946
1634/1688 [============================>.] - ETA: 0s - loss: 0.0155 - accuracy: 0.9946
1654/1688 [============================>.] - ETA: 0s - loss: 0.0156 - accuracy: 0.9946
1674/1688 [============================>.] - ETA: 0s - loss: 0.0156 - accuracy: 0.9945
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0156 - accuracy: 0.9946 - val_loss: 0.0778 - val_accuracy: 0.9808
score = model_keras.evaluate(x_test, y_test, verbose=0)
print('Test accuracy:', score[1])
Test accuracy: 0.9760000109672546
2. Quantize
2.1. 8-bit quantization
An Akida accelerator processes 8 or 4-bits integer activations and weights. Therefore, the floating point Keras model must be quantized in preparation to run on an Akida accelerator.
The QuantizeML quantize function can be used to quantize a Keras model for Akida. For this step in this example, an “8/8/8” quantization scheme will be applied to the floating point Keras model to produce 8-bit weights in the first layer, 8-bit weights in all other layers, and 8-bit activations.
The quantization process results in a Keras model with custom QuantizeML quantized layers substituted for the original Keras layers.
All Keras API functions can be applied on this new model: summary(), compile(), fit(). etc.
Note
The quantize function applies several transformations to
the original model. For example, it folds the batch normalization layers into the
corresponding neural layers. The new weights are computed according to this folding
operation.
from quantizeml.models import quantize, QuantizationParams
qparams = QuantizationParams(input_weight_bits=8, weight_bits=8, activation_bits=8)
model_quantized = quantize(model_keras, qparams=qparams)
/usr/local/lib/python3.11/dist-packages/quantizeml/models/quantize.py:479: UserWarning: Quantizing per-axis with random calibration samples is not accurate. Set QuantizationParams.per_tensor_activations=True when calibrating with random samples.
warnings.warn("Quantizing per-axis with random calibration samples is not accurate. "
1/1024 [..............................] - ETA: 3:18
57/1024 [>.............................] - ETA: 0s
113/1024 [==>...........................] - ETA: 0s
145/1024 [===>..........................] - ETA: 1s
203/1024 [====>.........................] - ETA: 1s
262/1024 [======>.......................] - ETA: 1s
321/1024 [========>.....................] - ETA: 1s
378/1024 [==========>...................] - ETA: 0s
435/1024 [===========>..................] - ETA: 0s
493/1024 [=============>................] - ETA: 0s
549/1024 [===============>..............] - ETA: 0s
606/1024 [================>.............] - ETA: 0s
664/1024 [==================>...........] - ETA: 0s
721/1024 [====================>.........] - ETA: 0s
779/1024 [=====================>........] - ETA: 0s
835/1024 [=======================>......] - ETA: 0s
894/1024 [=========================>....] - ETA: 0s
951/1024 [==========================>...] - ETA: 0s
1008/1024 [============================>.] - ETA: 0s
1024/1024 [==============================] - 1s 1ms/step
model_quantized.summary()
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling_input (InputLaye [(None, 28, 28, 1)] 0
r)
rescaling (QuantizedRescal (None, 28, 28, 1) 0
ing)
conv2d (QuantizedConv2D) (None, 13, 13, 32) 320
re_lu (QuantizedReLU) (None, 13, 13, 32) 64
depthwise_conv2d (Quantize (None, 7, 7, 32) 384
dDepthwiseConv2D)
conv2d_1 (QuantizedConv2D) (None, 7, 7, 64) 2112
re_lu_1 (QuantizedReLU) (None, 7, 7, 64) 128
flatten (QuantizedFlatten) (None, 3136) 0
dense (QuantizedDense) (None, 10) 31370
dequantizer (Dequantizer) (None, 10) 0
=================================================================
Total params: 34378 (134.29 KB)
Trainable params: 34122 (133.29 KB)
Non-trainable params: 256 (1.00 KB)
_________________________________________________________________
Note
Note that the number of parameters for the floating and quantized models differs, a consequence of the BatchNormalization folding and the additional parameters added for quantization. For further details, please refer to their respective summary.
Check the quantized model accuracy.
def compile_evaluate(model):
""" Compiles and evaluates the model, then return accuracy score. """
model.compile(metrics=['accuracy'])
return model.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after 8-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 8-bit quantization: 0.972000002861023
2.2. Effect of calibration
The previous call to quantize was made with random samples for calibration
(default parameters). While the observed drop in accuracy is minimal, that is
around 1%, it can be worse on more complex models. Therefore, it is advised to
use a set of real samples from the training set for calibration during a call
to quantize.
Note that this remains a calibration step rather than a training step in that
no output labels are required. Furthermore, any relevant data could be used for
calibration. The recommended settings for calibration that are widely used to
obtain the zoo performance are:
1024 samples
a batch size of 100
2 epochs
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 985us/step
Check the accuracy for the quantized and calibrated model.
print('Test accuracy after calibration:', compile_evaluate(model_quantized))
Test accuracy after calibration: 0.9757000207901001
Calibrating with real samples on this model recovers the initial float accuracy.
2.3. 4-bit quantization
The accuracy of the 8/8/8 quantized model is equal to that of the Keras floating point model. In some cases, a smaller memory size for the model is required. This can be accomplished through quantization of the model to smaller bitwidths.
The model will now be quantized to 8/4/4, that is 8-bit weights in the first layer with 4-bit weights and activations in all other layers. Such a quantization scheme will usually introduce a performance drop.
qparams = QuantizationParams(input_weight_bits=8, weight_bits=4, activation_bits=4)
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 991us/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 1ms/step
Check the 4-bit quantized accuracy.
print('Test accuracy after 4-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 4-bit quantization: 0.9664000272750854
2.4. Model fine tuning (Quantization Aware Training)
When a model suffers from an accuracy drop after quantization, fine tuning or Quantization Aware Training (QAT) may recover some or all of the original performance.
Note that since this is a fine tuning step, both the number of epochs and learning rate are expected to be lower than during the initial float training.
model_quantized.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-4),
metrics=['accuracy'])
model_quantized.fit(x_train, y_train, epochs=5, validation_split=0.1)
Epoch 1/5
1/1688 [..............................] - ETA: 2:05:35 - loss: 0.1100 - accuracy: 0.9375
12/1688 [..............................] - ETA: 8s - loss: 0.0665 - accuracy: 0.9766
22/1688 [..............................] - ETA: 8s - loss: 0.0583 - accuracy: 0.9787
33/1688 [..............................] - ETA: 8s - loss: 0.0525 - accuracy: 0.9820
44/1688 [..............................] - ETA: 8s - loss: 0.0468 - accuracy: 0.9837
55/1688 [..............................] - ETA: 8s - loss: 0.0443 - accuracy: 0.9852
66/1688 [>.............................] - ETA: 8s - loss: 0.0402 - accuracy: 0.9867
76/1688 [>.............................] - ETA: 8s - loss: 0.0390 - accuracy: 0.9864
87/1688 [>.............................] - ETA: 7s - loss: 0.0407 - accuracy: 0.9860
98/1688 [>.............................] - ETA: 7s - loss: 0.0405 - accuracy: 0.9860
109/1688 [>.............................] - ETA: 7s - loss: 0.0401 - accuracy: 0.9857
120/1688 [=>............................] - ETA: 7s - loss: 0.0378 - accuracy: 0.9867
131/1688 [=>............................] - ETA: 7s - loss: 0.0365 - accuracy: 0.9869
142/1688 [=>............................] - ETA: 7s - loss: 0.0346 - accuracy: 0.9879
153/1688 [=>............................] - ETA: 7s - loss: 0.0338 - accuracy: 0.9879
164/1688 [=>............................] - ETA: 7s - loss: 0.0348 - accuracy: 0.9880
175/1688 [==>...........................] - ETA: 7s - loss: 0.0335 - accuracy: 0.9884
186/1688 [==>...........................] - ETA: 7s - loss: 0.0332 - accuracy: 0.9886
197/1688 [==>...........................] - ETA: 7s - loss: 0.0333 - accuracy: 0.9883
208/1688 [==>...........................] - ETA: 7s - loss: 0.0326 - accuracy: 0.9884
219/1688 [==>...........................] - ETA: 7s - loss: 0.0315 - accuracy: 0.9890
230/1688 [===>..........................] - ETA: 7s - loss: 0.0305 - accuracy: 0.9894
241/1688 [===>..........................] - ETA: 7s - loss: 0.0303 - accuracy: 0.9894
252/1688 [===>..........................] - ETA: 7s - loss: 0.0306 - accuracy: 0.9892
263/1688 [===>..........................] - ETA: 7s - loss: 0.0301 - accuracy: 0.9895
274/1688 [===>..........................] - ETA: 6s - loss: 0.0297 - accuracy: 0.9897
285/1688 [====>.........................] - ETA: 6s - loss: 0.0293 - accuracy: 0.9900
296/1688 [====>.........................] - ETA: 6s - loss: 0.0288 - accuracy: 0.9902
307/1688 [====>.........................] - ETA: 6s - loss: 0.0284 - accuracy: 0.9905
318/1688 [====>.........................] - ETA: 6s - loss: 0.0286 - accuracy: 0.9904
329/1688 [====>.........................] - ETA: 6s - loss: 0.0280 - accuracy: 0.9906
340/1688 [=====>........................] - ETA: 6s - loss: 0.0275 - accuracy: 0.9908
351/1688 [=====>........................] - ETA: 6s - loss: 0.0271 - accuracy: 0.9910
362/1688 [=====>........................] - ETA: 6s - loss: 0.0268 - accuracy: 0.9911
373/1688 [=====>........................] - ETA: 6s - loss: 0.0269 - accuracy: 0.9911
384/1688 [=====>........................] - ETA: 6s - loss: 0.0269 - accuracy: 0.9910
395/1688 [======>.......................] - ETA: 6s - loss: 0.0268 - accuracy: 0.9910
406/1688 [======>.......................] - ETA: 6s - loss: 0.0271 - accuracy: 0.9910
417/1688 [======>.......................] - ETA: 6s - loss: 0.0267 - accuracy: 0.9911
428/1688 [======>.......................] - ETA: 6s - loss: 0.0264 - accuracy: 0.9911
439/1688 [======>.......................] - ETA: 6s - loss: 0.0260 - accuracy: 0.9912
450/1688 [======>.......................] - ETA: 6s - loss: 0.0261 - accuracy: 0.9912
461/1688 [=======>......................] - ETA: 6s - loss: 0.0262 - accuracy: 0.9911
472/1688 [=======>......................] - ETA: 5s - loss: 0.0263 - accuracy: 0.9911
483/1688 [=======>......................] - ETA: 5s - loss: 0.0262 - accuracy: 0.9911
494/1688 [=======>......................] - ETA: 5s - loss: 0.0259 - accuracy: 0.9912
505/1688 [=======>......................] - ETA: 5s - loss: 0.0260 - accuracy: 0.9912
516/1688 [========>.....................] - ETA: 5s - loss: 0.0259 - accuracy: 0.9912
527/1688 [========>.....................] - ETA: 5s - loss: 0.0257 - accuracy: 0.9913
538/1688 [========>.....................] - ETA: 5s - loss: 0.0256 - accuracy: 0.9914
549/1688 [========>.....................] - ETA: 5s - loss: 0.0254 - accuracy: 0.9915
560/1688 [========>.....................] - ETA: 5s - loss: 0.0253 - accuracy: 0.9915
571/1688 [=========>....................] - ETA: 5s - loss: 0.0252 - accuracy: 0.9915
582/1688 [=========>....................] - ETA: 5s - loss: 0.0251 - accuracy: 0.9915
592/1688 [=========>....................] - ETA: 5s - loss: 0.0253 - accuracy: 0.9914
603/1688 [=========>....................] - ETA: 5s - loss: 0.0250 - accuracy: 0.9914
614/1688 [=========>....................] - ETA: 5s - loss: 0.0248 - accuracy: 0.9916
625/1688 [==========>...................] - ETA: 5s - loss: 0.0247 - accuracy: 0.9916
636/1688 [==========>...................] - ETA: 5s - loss: 0.0247 - accuracy: 0.9917
647/1688 [==========>...................] - ETA: 5s - loss: 0.0247 - accuracy: 0.9917
657/1688 [==========>...................] - ETA: 5s - loss: 0.0246 - accuracy: 0.9917
668/1688 [==========>...................] - ETA: 5s - loss: 0.0245 - accuracy: 0.9916
679/1688 [===========>..................] - ETA: 4s - loss: 0.0248 - accuracy: 0.9915
690/1688 [===========>..................] - ETA: 4s - loss: 0.0247 - accuracy: 0.9916
701/1688 [===========>..................] - ETA: 4s - loss: 0.0246 - accuracy: 0.9917
712/1688 [===========>..................] - ETA: 4s - loss: 0.0247 - accuracy: 0.9917
723/1688 [===========>..................] - ETA: 4s - loss: 0.0245 - accuracy: 0.9917
734/1688 [============>.................] - ETA: 4s - loss: 0.0243 - accuracy: 0.9918
745/1688 [============>.................] - ETA: 4s - loss: 0.0244 - accuracy: 0.9918
756/1688 [============>.................] - ETA: 4s - loss: 0.0245 - accuracy: 0.9918
767/1688 [============>.................] - ETA: 4s - loss: 0.0243 - accuracy: 0.9919
778/1688 [============>.................] - ETA: 4s - loss: 0.0241 - accuracy: 0.9919
789/1688 [=============>................] - ETA: 4s - loss: 0.0241 - accuracy: 0.9919
799/1688 [=============>................] - ETA: 4s - loss: 0.0243 - accuracy: 0.9917
810/1688 [=============>................] - ETA: 4s - loss: 0.0242 - accuracy: 0.9917
821/1688 [=============>................] - ETA: 4s - loss: 0.0240 - accuracy: 0.9919
832/1688 [=============>................] - ETA: 4s - loss: 0.0241 - accuracy: 0.9919
843/1688 [=============>................] - ETA: 4s - loss: 0.0240 - accuracy: 0.9920
854/1688 [==============>...............] - ETA: 4s - loss: 0.0239 - accuracy: 0.9920
865/1688 [==============>...............] - ETA: 4s - loss: 0.0239 - accuracy: 0.9920
876/1688 [==============>...............] - ETA: 3s - loss: 0.0239 - accuracy: 0.9920
887/1688 [==============>...............] - ETA: 3s - loss: 0.0238 - accuracy: 0.9921
898/1688 [==============>...............] - ETA: 3s - loss: 0.0237 - accuracy: 0.9922
909/1688 [===============>..............] - ETA: 3s - loss: 0.0237 - accuracy: 0.9921
920/1688 [===============>..............] - ETA: 3s - loss: 0.0236 - accuracy: 0.9921
931/1688 [===============>..............] - ETA: 3s - loss: 0.0235 - accuracy: 0.9922
942/1688 [===============>..............] - ETA: 3s - loss: 0.0234 - accuracy: 0.9922
953/1688 [===============>..............] - ETA: 3s - loss: 0.0233 - accuracy: 0.9923
964/1688 [================>.............] - ETA: 3s - loss: 0.0232 - accuracy: 0.9923
975/1688 [================>.............] - ETA: 3s - loss: 0.0233 - accuracy: 0.9922
986/1688 [================>.............] - ETA: 3s - loss: 0.0234 - accuracy: 0.9922
997/1688 [================>.............] - ETA: 3s - loss: 0.0232 - accuracy: 0.9923
1008/1688 [================>.............] - ETA: 3s - loss: 0.0232 - accuracy: 0.9923
1019/1688 [=================>............] - ETA: 3s - loss: 0.0231 - accuracy: 0.9923
1030/1688 [=================>............] - ETA: 3s - loss: 0.0230 - accuracy: 0.9924
1041/1688 [=================>............] - ETA: 3s - loss: 0.0229 - accuracy: 0.9925
1051/1688 [=================>............] - ETA: 3s - loss: 0.0228 - accuracy: 0.9925
1062/1688 [=================>............] - ETA: 3s - loss: 0.0227 - accuracy: 0.9926
1073/1688 [==================>...........] - ETA: 3s - loss: 0.0226 - accuracy: 0.9926
1084/1688 [==================>...........] - ETA: 2s - loss: 0.0228 - accuracy: 0.9925
1095/1688 [==================>...........] - ETA: 2s - loss: 0.0227 - accuracy: 0.9926
1106/1688 [==================>...........] - ETA: 2s - loss: 0.0227 - accuracy: 0.9926
1117/1688 [==================>...........] - ETA: 2s - loss: 0.0226 - accuracy: 0.9926
1128/1688 [===================>..........] - ETA: 2s - loss: 0.0225 - accuracy: 0.9926
1138/1688 [===================>..........] - ETA: 2s - loss: 0.0224 - accuracy: 0.9927
1149/1688 [===================>..........] - ETA: 2s - loss: 0.0224 - accuracy: 0.9927
1160/1688 [===================>..........] - ETA: 2s - loss: 0.0222 - accuracy: 0.9928
1171/1688 [===================>..........] - ETA: 2s - loss: 0.0222 - accuracy: 0.9928
1182/1688 [====================>.........] - ETA: 2s - loss: 0.0221 - accuracy: 0.9928
1193/1688 [====================>.........] - ETA: 2s - loss: 0.0220 - accuracy: 0.9928
1204/1688 [====================>.........] - ETA: 2s - loss: 0.0219 - accuracy: 0.9929
1215/1688 [====================>.........] - ETA: 2s - loss: 0.0219 - accuracy: 0.9928
1226/1688 [====================>.........] - ETA: 2s - loss: 0.0219 - accuracy: 0.9929
1237/1688 [====================>.........] - ETA: 2s - loss: 0.0219 - accuracy: 0.9929
1248/1688 [=====================>........] - ETA: 2s - loss: 0.0218 - accuracy: 0.9929
1259/1688 [=====================>........] - ETA: 2s - loss: 0.0217 - accuracy: 0.9929
1270/1688 [=====================>........] - ETA: 2s - loss: 0.0218 - accuracy: 0.9929
1281/1688 [=====================>........] - ETA: 1s - loss: 0.0218 - accuracy: 0.9929
1292/1688 [=====================>........] - ETA: 1s - loss: 0.0218 - accuracy: 0.9929
1303/1688 [======================>.......] - ETA: 1s - loss: 0.0217 - accuracy: 0.9929
1314/1688 [======================>.......] - ETA: 1s - loss: 0.0217 - accuracy: 0.9930
1325/1688 [======================>.......] - ETA: 1s - loss: 0.0216 - accuracy: 0.9930
1336/1688 [======================>.......] - ETA: 1s - loss: 0.0216 - accuracy: 0.9930
1347/1688 [======================>.......] - ETA: 1s - loss: 0.0215 - accuracy: 0.9930
1358/1688 [=======================>......] - ETA: 1s - loss: 0.0215 - accuracy: 0.9930
1369/1688 [=======================>......] - ETA: 1s - loss: 0.0215 - accuracy: 0.9930
1380/1688 [=======================>......] - ETA: 1s - loss: 0.0215 - accuracy: 0.9930
1391/1688 [=======================>......] - ETA: 1s - loss: 0.0215 - accuracy: 0.9930
1402/1688 [=======================>......] - ETA: 1s - loss: 0.0215 - accuracy: 0.9930
1413/1688 [========================>.....] - ETA: 1s - loss: 0.0214 - accuracy: 0.9930
1424/1688 [========================>.....] - ETA: 1s - loss: 0.0214 - accuracy: 0.9930
1435/1688 [========================>.....] - ETA: 1s - loss: 0.0213 - accuracy: 0.9931
1446/1688 [========================>.....] - ETA: 1s - loss: 0.0213 - accuracy: 0.9930
1457/1688 [========================>.....] - ETA: 1s - loss: 0.0214 - accuracy: 0.9930
1468/1688 [=========================>....] - ETA: 1s - loss: 0.0215 - accuracy: 0.9930
1479/1688 [=========================>....] - ETA: 1s - loss: 0.0215 - accuracy: 0.9930
1490/1688 [=========================>....] - ETA: 0s - loss: 0.0215 - accuracy: 0.9930
1501/1688 [=========================>....] - ETA: 0s - loss: 0.0215 - accuracy: 0.9929
1512/1688 [=========================>....] - ETA: 0s - loss: 0.0215 - accuracy: 0.9930
1523/1688 [==========================>...] - ETA: 0s - loss: 0.0214 - accuracy: 0.9930
1534/1688 [==========================>...] - ETA: 0s - loss: 0.0215 - accuracy: 0.9930
1545/1688 [==========================>...] - ETA: 0s - loss: 0.0215 - accuracy: 0.9929
1556/1688 [==========================>...] - ETA: 0s - loss: 0.0216 - accuracy: 0.9929
1567/1688 [==========================>...] - ETA: 0s - loss: 0.0216 - accuracy: 0.9929
1578/1688 [===========================>..] - ETA: 0s - loss: 0.0216 - accuracy: 0.9929
1589/1688 [===========================>..] - ETA: 0s - loss: 0.0216 - accuracy: 0.9929
1600/1688 [===========================>..] - ETA: 0s - loss: 0.0215 - accuracy: 0.9929
1611/1688 [===========================>..] - ETA: 0s - loss: 0.0215 - accuracy: 0.9929
1622/1688 [===========================>..] - ETA: 0s - loss: 0.0215 - accuracy: 0.9929
1633/1688 [============================>.] - ETA: 0s - loss: 0.0215 - accuracy: 0.9929
1644/1688 [============================>.] - ETA: 0s - loss: 0.0215 - accuracy: 0.9929
1655/1688 [============================>.] - ETA: 0s - loss: 0.0214 - accuracy: 0.9929
1666/1688 [============================>.] - ETA: 0s - loss: 0.0214 - accuracy: 0.9929
1677/1688 [============================>.] - ETA: 0s - loss: 0.0213 - accuracy: 0.9929
1688/1688 [==============================] - ETA: 0s - loss: 0.0212 - accuracy: 0.9929
1688/1688 [==============================] - 15s 6ms/step - loss: 0.0212 - accuracy: 0.9929 - val_loss: 0.0469 - val_accuracy: 0.9868
Epoch 2/5
1/1688 [..............................] - ETA: 8s - loss: 8.2493e-04 - accuracy: 1.0000
12/1688 [..............................] - ETA: 8s - loss: 0.0128 - accuracy: 1.0000
23/1688 [..............................] - ETA: 8s - loss: 0.0128 - accuracy: 0.9986
34/1688 [..............................] - ETA: 8s - loss: 0.0148 - accuracy: 0.9963
45/1688 [..............................] - ETA: 8s - loss: 0.0198 - accuracy: 0.9937
56/1688 [..............................] - ETA: 7s - loss: 0.0175 - accuracy: 0.9950
67/1688 [>.............................] - ETA: 7s - loss: 0.0180 - accuracy: 0.9953
78/1688 [>.............................] - ETA: 7s - loss: 0.0171 - accuracy: 0.9956
89/1688 [>.............................] - ETA: 7s - loss: 0.0163 - accuracy: 0.9961
100/1688 [>.............................] - ETA: 7s - loss: 0.0161 - accuracy: 0.9959
111/1688 [>.............................] - ETA: 7s - loss: 0.0159 - accuracy: 0.9958
122/1688 [=>............................] - ETA: 7s - loss: 0.0152 - accuracy: 0.9962
133/1688 [=>............................] - ETA: 7s - loss: 0.0156 - accuracy: 0.9960
144/1688 [=>............................] - ETA: 7s - loss: 0.0165 - accuracy: 0.9959
155/1688 [=>............................] - ETA: 7s - loss: 0.0160 - accuracy: 0.9960
166/1688 [=>............................] - ETA: 7s - loss: 0.0157 - accuracy: 0.9962
177/1688 [==>...........................] - ETA: 7s - loss: 0.0154 - accuracy: 0.9963
188/1688 [==>...........................] - ETA: 7s - loss: 0.0157 - accuracy: 0.9963
199/1688 [==>...........................] - ETA: 7s - loss: 0.0153 - accuracy: 0.9965
210/1688 [==>...........................] - ETA: 7s - loss: 0.0153 - accuracy: 0.9966
221/1688 [==>...........................] - ETA: 7s - loss: 0.0151 - accuracy: 0.9966
232/1688 [===>..........................] - ETA: 7s - loss: 0.0150 - accuracy: 0.9966
243/1688 [===>..........................] - ETA: 7s - loss: 0.0149 - accuracy: 0.9965
254/1688 [===>..........................] - ETA: 7s - loss: 0.0155 - accuracy: 0.9963
265/1688 [===>..........................] - ETA: 6s - loss: 0.0153 - accuracy: 0.9965
276/1688 [===>..........................] - ETA: 6s - loss: 0.0151 - accuracy: 0.9965
287/1688 [====>.........................] - ETA: 6s - loss: 0.0152 - accuracy: 0.9965
298/1688 [====>.........................] - ETA: 6s - loss: 0.0152 - accuracy: 0.9964
308/1688 [====>.........................] - ETA: 6s - loss: 0.0152 - accuracy: 0.9964
319/1688 [====>.........................] - ETA: 6s - loss: 0.0155 - accuracy: 0.9964
330/1688 [====>.........................] - ETA: 6s - loss: 0.0154 - accuracy: 0.9965
341/1688 [=====>........................] - ETA: 6s - loss: 0.0156 - accuracy: 0.9963
352/1688 [=====>........................] - ETA: 6s - loss: 0.0158 - accuracy: 0.9961
363/1688 [=====>........................] - ETA: 6s - loss: 0.0158 - accuracy: 0.9961
373/1688 [=====>........................] - ETA: 6s - loss: 0.0157 - accuracy: 0.9961
384/1688 [=====>........................] - ETA: 6s - loss: 0.0158 - accuracy: 0.9961
395/1688 [======>.......................] - ETA: 6s - loss: 0.0161 - accuracy: 0.9959
406/1688 [======>.......................] - ETA: 6s - loss: 0.0162 - accuracy: 0.9958
417/1688 [======>.......................] - ETA: 6s - loss: 0.0160 - accuracy: 0.9959
428/1688 [======>.......................] - ETA: 6s - loss: 0.0158 - accuracy: 0.9960
439/1688 [======>.......................] - ETA: 6s - loss: 0.0156 - accuracy: 0.9961
450/1688 [======>.......................] - ETA: 6s - loss: 0.0158 - accuracy: 0.9960
461/1688 [=======>......................] - ETA: 6s - loss: 0.0158 - accuracy: 0.9959
472/1688 [=======>......................] - ETA: 5s - loss: 0.0159 - accuracy: 0.9958
483/1688 [=======>......................] - ETA: 5s - loss: 0.0158 - accuracy: 0.9959
494/1688 [=======>......................] - ETA: 5s - loss: 0.0157 - accuracy: 0.9960
505/1688 [=======>......................] - ETA: 5s - loss: 0.0157 - accuracy: 0.9959
515/1688 [========>.....................] - ETA: 5s - loss: 0.0156 - accuracy: 0.9959
526/1688 [========>.....................] - ETA: 5s - loss: 0.0156 - accuracy: 0.9959
537/1688 [========>.....................] - ETA: 5s - loss: 0.0155 - accuracy: 0.9959
548/1688 [========>.....................] - ETA: 5s - loss: 0.0155 - accuracy: 0.9960
559/1688 [========>.....................] - ETA: 5s - loss: 0.0159 - accuracy: 0.9958
570/1688 [=========>....................] - ETA: 5s - loss: 0.0159 - accuracy: 0.9958
581/1688 [=========>....................] - ETA: 5s - loss: 0.0162 - accuracy: 0.9955
592/1688 [=========>....................] - ETA: 5s - loss: 0.0161 - accuracy: 0.9956
603/1688 [=========>....................] - ETA: 5s - loss: 0.0160 - accuracy: 0.9956
614/1688 [=========>....................] - ETA: 5s - loss: 0.0159 - accuracy: 0.9956
625/1688 [==========>...................] - ETA: 5s - loss: 0.0159 - accuracy: 0.9956
636/1688 [==========>...................] - ETA: 5s - loss: 0.0158 - accuracy: 0.9956
647/1688 [==========>...................] - ETA: 5s - loss: 0.0157 - accuracy: 0.9957
658/1688 [==========>...................] - ETA: 5s - loss: 0.0156 - accuracy: 0.9956
669/1688 [==========>...................] - ETA: 4s - loss: 0.0157 - accuracy: 0.9956
679/1688 [===========>..................] - ETA: 4s - loss: 0.0157 - accuracy: 0.9955
690/1688 [===========>..................] - ETA: 4s - loss: 0.0158 - accuracy: 0.9954
701/1688 [===========>..................] - ETA: 4s - loss: 0.0156 - accuracy: 0.9955
712/1688 [===========>..................] - ETA: 4s - loss: 0.0155 - accuracy: 0.9955
723/1688 [===========>..................] - ETA: 4s - loss: 0.0154 - accuracy: 0.9956
734/1688 [============>.................] - ETA: 4s - loss: 0.0154 - accuracy: 0.9957
744/1688 [============>.................] - ETA: 4s - loss: 0.0153 - accuracy: 0.9957
755/1688 [============>.................] - ETA: 4s - loss: 0.0151 - accuracy: 0.9958
766/1688 [============>.................] - ETA: 4s - loss: 0.0152 - accuracy: 0.9957
777/1688 [============>.................] - ETA: 4s - loss: 0.0151 - accuracy: 0.9957
788/1688 [=============>................] - ETA: 4s - loss: 0.0152 - accuracy: 0.9957
799/1688 [=============>................] - ETA: 4s - loss: 0.0151 - accuracy: 0.9957
810/1688 [=============>................] - ETA: 4s - loss: 0.0150 - accuracy: 0.9958
821/1688 [=============>................] - ETA: 4s - loss: 0.0149 - accuracy: 0.9958
832/1688 [=============>................] - ETA: 4s - loss: 0.0151 - accuracy: 0.9958
843/1688 [=============>................] - ETA: 4s - loss: 0.0151 - accuracy: 0.9957
853/1688 [==============>...............] - ETA: 4s - loss: 0.0153 - accuracy: 0.9957
864/1688 [==============>...............] - ETA: 4s - loss: 0.0153 - accuracy: 0.9957
875/1688 [==============>...............] - ETA: 3s - loss: 0.0153 - accuracy: 0.9957
886/1688 [==============>...............] - ETA: 3s - loss: 0.0153 - accuracy: 0.9957
897/1688 [==============>...............] - ETA: 3s - loss: 0.0153 - accuracy: 0.9957
908/1688 [===============>..............] - ETA: 3s - loss: 0.0153 - accuracy: 0.9957
919/1688 [===============>..............] - ETA: 3s - loss: 0.0153 - accuracy: 0.9956
930/1688 [===============>..............] - ETA: 3s - loss: 0.0152 - accuracy: 0.9956
941/1688 [===============>..............] - ETA: 3s - loss: 0.0151 - accuracy: 0.9956
952/1688 [===============>..............] - ETA: 3s - loss: 0.0151 - accuracy: 0.9956
963/1688 [================>.............] - ETA: 3s - loss: 0.0151 - accuracy: 0.9957
974/1688 [================>.............] - ETA: 3s - loss: 0.0151 - accuracy: 0.9956
984/1688 [================>.............] - ETA: 3s - loss: 0.0151 - accuracy: 0.9956
995/1688 [================>.............] - ETA: 3s - loss: 0.0152 - accuracy: 0.9956
1006/1688 [================>.............] - ETA: 3s - loss: 0.0151 - accuracy: 0.9956
1017/1688 [=================>............] - ETA: 3s - loss: 0.0151 - accuracy: 0.9957
1028/1688 [=================>............] - ETA: 3s - loss: 0.0150 - accuracy: 0.9957
1038/1688 [=================>............] - ETA: 3s - loss: 0.0150 - accuracy: 0.9957
1049/1688 [=================>............] - ETA: 3s - loss: 0.0150 - accuracy: 0.9957
1060/1688 [=================>............] - ETA: 3s - loss: 0.0149 - accuracy: 0.9957
1071/1688 [==================>...........] - ETA: 3s - loss: 0.0149 - accuracy: 0.9957
1082/1688 [==================>...........] - ETA: 2s - loss: 0.0150 - accuracy: 0.9957
1093/1688 [==================>...........] - ETA: 2s - loss: 0.0149 - accuracy: 0.9957
1104/1688 [==================>...........] - ETA: 2s - loss: 0.0149 - accuracy: 0.9958
1115/1688 [==================>...........] - ETA: 2s - loss: 0.0148 - accuracy: 0.9958
1125/1688 [==================>...........] - ETA: 2s - loss: 0.0148 - accuracy: 0.9958
1136/1688 [===================>..........] - ETA: 2s - loss: 0.0149 - accuracy: 0.9958
1147/1688 [===================>..........] - ETA: 2s - loss: 0.0148 - accuracy: 0.9958
1158/1688 [===================>..........] - ETA: 2s - loss: 0.0148 - accuracy: 0.9958
1169/1688 [===================>..........] - ETA: 2s - loss: 0.0148 - accuracy: 0.9958
1180/1688 [===================>..........] - ETA: 2s - loss: 0.0147 - accuracy: 0.9958
1190/1688 [====================>.........] - ETA: 2s - loss: 0.0147 - accuracy: 0.9959
1201/1688 [====================>.........] - ETA: 2s - loss: 0.0148 - accuracy: 0.9958
1212/1688 [====================>.........] - ETA: 2s - loss: 0.0148 - accuracy: 0.9958
1223/1688 [====================>.........] - ETA: 2s - loss: 0.0149 - accuracy: 0.9957
1234/1688 [====================>.........] - ETA: 2s - loss: 0.0148 - accuracy: 0.9958
1245/1688 [=====================>........] - ETA: 2s - loss: 0.0148 - accuracy: 0.9958
1256/1688 [=====================>........] - ETA: 2s - loss: 0.0147 - accuracy: 0.9958
1267/1688 [=====================>........] - ETA: 2s - loss: 0.0147 - accuracy: 0.9958
1278/1688 [=====================>........] - ETA: 2s - loss: 0.0147 - accuracy: 0.9958
1289/1688 [=====================>........] - ETA: 1s - loss: 0.0147 - accuracy: 0.9958
1300/1688 [======================>.......] - ETA: 1s - loss: 0.0147 - accuracy: 0.9958
1311/1688 [======================>.......] - ETA: 1s - loss: 0.0146 - accuracy: 0.9958
1322/1688 [======================>.......] - ETA: 1s - loss: 0.0146 - accuracy: 0.9958
1332/1688 [======================>.......] - ETA: 1s - loss: 0.0146 - accuracy: 0.9958
1343/1688 [======================>.......] - ETA: 1s - loss: 0.0147 - accuracy: 0.9958
1354/1688 [=======================>......] - ETA: 1s - loss: 0.0147 - accuracy: 0.9958
1365/1688 [=======================>......] - ETA: 1s - loss: 0.0146 - accuracy: 0.9958
1376/1688 [=======================>......] - ETA: 1s - loss: 0.0147 - accuracy: 0.9958
1387/1688 [=======================>......] - ETA: 1s - loss: 0.0147 - accuracy: 0.9958
1398/1688 [=======================>......] - ETA: 1s - loss: 0.0146 - accuracy: 0.9958
1409/1688 [========================>.....] - ETA: 1s - loss: 0.0146 - accuracy: 0.9958
1420/1688 [========================>.....] - ETA: 1s - loss: 0.0146 - accuracy: 0.9958
1431/1688 [========================>.....] - ETA: 1s - loss: 0.0147 - accuracy: 0.9957
1442/1688 [========================>.....] - ETA: 1s - loss: 0.0146 - accuracy: 0.9958
1453/1688 [========================>.....] - ETA: 1s - loss: 0.0145 - accuracy: 0.9958
1464/1688 [=========================>....] - ETA: 1s - loss: 0.0146 - accuracy: 0.9958
1475/1688 [=========================>....] - ETA: 1s - loss: 0.0147 - accuracy: 0.9957
1486/1688 [=========================>....] - ETA: 0s - loss: 0.0146 - accuracy: 0.9958
1497/1688 [=========================>....] - ETA: 0s - loss: 0.0146 - accuracy: 0.9958
1508/1688 [=========================>....] - ETA: 0s - loss: 0.0146 - accuracy: 0.9958
1519/1688 [=========================>....] - ETA: 0s - loss: 0.0146 - accuracy: 0.9958
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0146 - accuracy: 0.9958
1541/1688 [==========================>...] - ETA: 0s - loss: 0.0146 - accuracy: 0.9958
1552/1688 [==========================>...] - ETA: 0s - loss: 0.0146 - accuracy: 0.9958
1563/1688 [==========================>...] - ETA: 0s - loss: 0.0146 - accuracy: 0.9958
1574/1688 [==========================>...] - ETA: 0s - loss: 0.0146 - accuracy: 0.9958
1585/1688 [===========================>..] - ETA: 0s - loss: 0.0145 - accuracy: 0.9958
1596/1688 [===========================>..] - ETA: 0s - loss: 0.0146 - accuracy: 0.9958
1607/1688 [===========================>..] - ETA: 0s - loss: 0.0146 - accuracy: 0.9958
1618/1688 [===========================>..] - ETA: 0s - loss: 0.0146 - accuracy: 0.9957
1629/1688 [===========================>..] - ETA: 0s - loss: 0.0146 - accuracy: 0.9957
1640/1688 [============================>.] - ETA: 0s - loss: 0.0146 - accuracy: 0.9957
1651/1688 [============================>.] - ETA: 0s - loss: 0.0147 - accuracy: 0.9957
1662/1688 [============================>.] - ETA: 0s - loss: 0.0147 - accuracy: 0.9957
1673/1688 [============================>.] - ETA: 0s - loss: 0.0147 - accuracy: 0.9957
1684/1688 [============================>.] - ETA: 0s - loss: 0.0147 - accuracy: 0.9957
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0147 - accuracy: 0.9957 - val_loss: 0.0456 - val_accuracy: 0.9872
Epoch 3/5
1/1688 [..............................] - ETA: 8s - loss: 0.0171 - accuracy: 1.0000
12/1688 [..............................] - ETA: 8s - loss: 0.0072 - accuracy: 0.9974
22/1688 [..............................] - ETA: 8s - loss: 0.0121 - accuracy: 0.9943
33/1688 [..............................] - ETA: 8s - loss: 0.0134 - accuracy: 0.9943
44/1688 [..............................] - ETA: 8s - loss: 0.0119 - accuracy: 0.9957
55/1688 [..............................] - ETA: 8s - loss: 0.0116 - accuracy: 0.9966
66/1688 [>.............................] - ETA: 7s - loss: 0.0116 - accuracy: 0.9972
77/1688 [>.............................] - ETA: 7s - loss: 0.0111 - accuracy: 0.9972
88/1688 [>.............................] - ETA: 7s - loss: 0.0119 - accuracy: 0.9964
99/1688 [>.............................] - ETA: 7s - loss: 0.0122 - accuracy: 0.9965
110/1688 [>.............................] - ETA: 7s - loss: 0.0118 - accuracy: 0.9969
121/1688 [=>............................] - ETA: 7s - loss: 0.0120 - accuracy: 0.9969
132/1688 [=>............................] - ETA: 7s - loss: 0.0119 - accuracy: 0.9969
143/1688 [=>............................] - ETA: 7s - loss: 0.0129 - accuracy: 0.9963
154/1688 [=>............................] - ETA: 7s - loss: 0.0131 - accuracy: 0.9963
165/1688 [=>............................] - ETA: 7s - loss: 0.0132 - accuracy: 0.9964
176/1688 [==>...........................] - ETA: 7s - loss: 0.0128 - accuracy: 0.9966
187/1688 [==>...........................] - ETA: 7s - loss: 0.0125 - accuracy: 0.9968
198/1688 [==>...........................] - ETA: 7s - loss: 0.0123 - accuracy: 0.9968
209/1688 [==>...........................] - ETA: 7s - loss: 0.0121 - accuracy: 0.9970
220/1688 [==>...........................] - ETA: 7s - loss: 0.0119 - accuracy: 0.9972
231/1688 [===>..........................] - ETA: 7s - loss: 0.0117 - accuracy: 0.9973
242/1688 [===>..........................] - ETA: 7s - loss: 0.0118 - accuracy: 0.9972
253/1688 [===>..........................] - ETA: 7s - loss: 0.0118 - accuracy: 0.9972
264/1688 [===>..........................] - ETA: 7s - loss: 0.0120 - accuracy: 0.9970
275/1688 [===>..........................] - ETA: 6s - loss: 0.0118 - accuracy: 0.9972
286/1688 [====>.........................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9970
297/1688 [====>.........................] - ETA: 6s - loss: 0.0118 - accuracy: 0.9972
308/1688 [====>.........................] - ETA: 6s - loss: 0.0117 - accuracy: 0.9973
319/1688 [====>.........................] - ETA: 6s - loss: 0.0118 - accuracy: 0.9971
330/1688 [====>.........................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9970
341/1688 [=====>........................] - ETA: 6s - loss: 0.0122 - accuracy: 0.9969
352/1688 [=====>........................] - ETA: 6s - loss: 0.0122 - accuracy: 0.9969
363/1688 [=====>........................] - ETA: 6s - loss: 0.0121 - accuracy: 0.9969
374/1688 [=====>........................] - ETA: 6s - loss: 0.0123 - accuracy: 0.9968
385/1688 [=====>........................] - ETA: 6s - loss: 0.0121 - accuracy: 0.9969
396/1688 [======>.......................] - ETA: 6s - loss: 0.0126 - accuracy: 0.9967
406/1688 [======>.......................] - ETA: 6s - loss: 0.0131 - accuracy: 0.9965
417/1688 [======>.......................] - ETA: 6s - loss: 0.0129 - accuracy: 0.9966
428/1688 [======>.......................] - ETA: 6s - loss: 0.0127 - accuracy: 0.9967
438/1688 [======>.......................] - ETA: 6s - loss: 0.0126 - accuracy: 0.9968
449/1688 [======>.......................] - ETA: 6s - loss: 0.0125 - accuracy: 0.9969
460/1688 [=======>......................] - ETA: 6s - loss: 0.0123 - accuracy: 0.9969
471/1688 [=======>......................] - ETA: 5s - loss: 0.0123 - accuracy: 0.9969
481/1688 [=======>......................] - ETA: 5s - loss: 0.0125 - accuracy: 0.9969
492/1688 [=======>......................] - ETA: 5s - loss: 0.0125 - accuracy: 0.9969
503/1688 [=======>......................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9970
514/1688 [========>.....................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9969
525/1688 [========>.....................] - ETA: 5s - loss: 0.0128 - accuracy: 0.9968
536/1688 [========>.....................] - ETA: 5s - loss: 0.0129 - accuracy: 0.9967
547/1688 [========>.....................] - ETA: 5s - loss: 0.0129 - accuracy: 0.9966
558/1688 [========>.....................] - ETA: 5s - loss: 0.0130 - accuracy: 0.9966
569/1688 [=========>....................] - ETA: 5s - loss: 0.0130 - accuracy: 0.9966
580/1688 [=========>....................] - ETA: 5s - loss: 0.0132 - accuracy: 0.9966
591/1688 [=========>....................] - ETA: 5s - loss: 0.0131 - accuracy: 0.9966
602/1688 [=========>....................] - ETA: 5s - loss: 0.0131 - accuracy: 0.9966
613/1688 [=========>....................] - ETA: 5s - loss: 0.0130 - accuracy: 0.9967
624/1688 [==========>...................] - ETA: 5s - loss: 0.0131 - accuracy: 0.9965
635/1688 [==========>...................] - ETA: 5s - loss: 0.0132 - accuracy: 0.9965
646/1688 [==========>...................] - ETA: 5s - loss: 0.0132 - accuracy: 0.9964
657/1688 [==========>...................] - ETA: 5s - loss: 0.0132 - accuracy: 0.9964
668/1688 [==========>...................] - ETA: 5s - loss: 0.0130 - accuracy: 0.9964
679/1688 [===========>..................] - ETA: 4s - loss: 0.0130 - accuracy: 0.9965
689/1688 [===========>..................] - ETA: 4s - loss: 0.0129 - accuracy: 0.9965
700/1688 [===========>..................] - ETA: 4s - loss: 0.0130 - accuracy: 0.9965
711/1688 [===========>..................] - ETA: 4s - loss: 0.0133 - accuracy: 0.9964
722/1688 [===========>..................] - ETA: 4s - loss: 0.0132 - accuracy: 0.9964
733/1688 [============>.................] - ETA: 4s - loss: 0.0133 - accuracy: 0.9964
744/1688 [============>.................] - ETA: 4s - loss: 0.0133 - accuracy: 0.9964
755/1688 [============>.................] - ETA: 4s - loss: 0.0132 - accuracy: 0.9964
766/1688 [============>.................] - ETA: 4s - loss: 0.0132 - accuracy: 0.9964
777/1688 [============>.................] - ETA: 4s - loss: 0.0131 - accuracy: 0.9965
788/1688 [=============>................] - ETA: 4s - loss: 0.0130 - accuracy: 0.9965
799/1688 [=============>................] - ETA: 4s - loss: 0.0130 - accuracy: 0.9966
810/1688 [=============>................] - ETA: 4s - loss: 0.0129 - accuracy: 0.9966
821/1688 [=============>................] - ETA: 4s - loss: 0.0128 - accuracy: 0.9966
832/1688 [=============>................] - ETA: 4s - loss: 0.0128 - accuracy: 0.9967
843/1688 [=============>................] - ETA: 4s - loss: 0.0127 - accuracy: 0.9967
854/1688 [==============>...............] - ETA: 4s - loss: 0.0126 - accuracy: 0.9967
864/1688 [==============>...............] - ETA: 4s - loss: 0.0126 - accuracy: 0.9967
875/1688 [==============>...............] - ETA: 4s - loss: 0.0126 - accuracy: 0.9967
886/1688 [==============>...............] - ETA: 3s - loss: 0.0126 - accuracy: 0.9967
897/1688 [==============>...............] - ETA: 3s - loss: 0.0125 - accuracy: 0.9968
908/1688 [===============>..............] - ETA: 3s - loss: 0.0124 - accuracy: 0.9968
918/1688 [===============>..............] - ETA: 3s - loss: 0.0124 - accuracy: 0.9968
929/1688 [===============>..............] - ETA: 3s - loss: 0.0124 - accuracy: 0.9967
940/1688 [===============>..............] - ETA: 3s - loss: 0.0123 - accuracy: 0.9968
951/1688 [===============>..............] - ETA: 3s - loss: 0.0124 - accuracy: 0.9968
962/1688 [================>.............] - ETA: 3s - loss: 0.0124 - accuracy: 0.9967
973/1688 [================>.............] - ETA: 3s - loss: 0.0124 - accuracy: 0.9967
984/1688 [================>.............] - ETA: 3s - loss: 0.0125 - accuracy: 0.9966
995/1688 [================>.............] - ETA: 3s - loss: 0.0125 - accuracy: 0.9966
1006/1688 [================>.............] - ETA: 3s - loss: 0.0126 - accuracy: 0.9966
1017/1688 [=================>............] - ETA: 3s - loss: 0.0125 - accuracy: 0.9966
1028/1688 [=================>............] - ETA: 3s - loss: 0.0125 - accuracy: 0.9967
1039/1688 [=================>............] - ETA: 3s - loss: 0.0125 - accuracy: 0.9967
1050/1688 [=================>............] - ETA: 3s - loss: 0.0124 - accuracy: 0.9967
1061/1688 [=================>............] - ETA: 3s - loss: 0.0124 - accuracy: 0.9967
1072/1688 [==================>...........] - ETA: 3s - loss: 0.0126 - accuracy: 0.9965
1083/1688 [==================>...........] - ETA: 2s - loss: 0.0127 - accuracy: 0.9965
1094/1688 [==================>...........] - ETA: 2s - loss: 0.0127 - accuracy: 0.9965
1105/1688 [==================>...........] - ETA: 2s - loss: 0.0128 - accuracy: 0.9964
1116/1688 [==================>...........] - ETA: 2s - loss: 0.0128 - accuracy: 0.9964
1127/1688 [===================>..........] - ETA: 2s - loss: 0.0128 - accuracy: 0.9965
1138/1688 [===================>..........] - ETA: 2s - loss: 0.0128 - accuracy: 0.9965
1149/1688 [===================>..........] - ETA: 2s - loss: 0.0128 - accuracy: 0.9965
1160/1688 [===================>..........] - ETA: 2s - loss: 0.0128 - accuracy: 0.9965
1171/1688 [===================>..........] - ETA: 2s - loss: 0.0128 - accuracy: 0.9965
1182/1688 [====================>.........] - ETA: 2s - loss: 0.0128 - accuracy: 0.9965
1193/1688 [====================>.........] - ETA: 2s - loss: 0.0128 - accuracy: 0.9965
1204/1688 [====================>.........] - ETA: 2s - loss: 0.0128 - accuracy: 0.9965
1215/1688 [====================>.........] - ETA: 2s - loss: 0.0127 - accuracy: 0.9965
1226/1688 [====================>.........] - ETA: 2s - loss: 0.0126 - accuracy: 0.9965
1237/1688 [====================>.........] - ETA: 2s - loss: 0.0127 - accuracy: 0.9965
1248/1688 [=====================>........] - ETA: 2s - loss: 0.0127 - accuracy: 0.9965
1259/1688 [=====================>........] - ETA: 2s - loss: 0.0127 - accuracy: 0.9965
1270/1688 [=====================>........] - ETA: 2s - loss: 0.0127 - accuracy: 0.9965
1281/1688 [=====================>........] - ETA: 2s - loss: 0.0127 - accuracy: 0.9965
1292/1688 [=====================>........] - ETA: 1s - loss: 0.0129 - accuracy: 0.9964
1303/1688 [======================>.......] - ETA: 1s - loss: 0.0129 - accuracy: 0.9963
1314/1688 [======================>.......] - ETA: 1s - loss: 0.0128 - accuracy: 0.9964
1325/1688 [======================>.......] - ETA: 1s - loss: 0.0127 - accuracy: 0.9964
1336/1688 [======================>.......] - ETA: 1s - loss: 0.0127 - accuracy: 0.9964
1347/1688 [======================>.......] - ETA: 1s - loss: 0.0126 - accuracy: 0.9965
1358/1688 [=======================>......] - ETA: 1s - loss: 0.0127 - accuracy: 0.9964
1369/1688 [=======================>......] - ETA: 1s - loss: 0.0126 - accuracy: 0.9965
1380/1688 [=======================>......] - ETA: 1s - loss: 0.0126 - accuracy: 0.9965
1391/1688 [=======================>......] - ETA: 1s - loss: 0.0126 - accuracy: 0.9965
1402/1688 [=======================>......] - ETA: 1s - loss: 0.0126 - accuracy: 0.9965
1413/1688 [========================>.....] - ETA: 1s - loss: 0.0126 - accuracy: 0.9965
1423/1688 [========================>.....] - ETA: 1s - loss: 0.0126 - accuracy: 0.9965
1434/1688 [========================>.....] - ETA: 1s - loss: 0.0125 - accuracy: 0.9965
1445/1688 [========================>.....] - ETA: 1s - loss: 0.0125 - accuracy: 0.9965
1456/1688 [========================>.....] - ETA: 1s - loss: 0.0126 - accuracy: 0.9965
1467/1688 [=========================>....] - ETA: 1s - loss: 0.0127 - accuracy: 0.9965
1478/1688 [=========================>....] - ETA: 1s - loss: 0.0126 - accuracy: 0.9965
1489/1688 [=========================>....] - ETA: 0s - loss: 0.0126 - accuracy: 0.9965
1500/1688 [=========================>....] - ETA: 0s - loss: 0.0126 - accuracy: 0.9965
1511/1688 [=========================>....] - ETA: 0s - loss: 0.0126 - accuracy: 0.9965
1521/1688 [==========================>...] - ETA: 0s - loss: 0.0125 - accuracy: 0.9965
1532/1688 [==========================>...] - ETA: 0s - loss: 0.0126 - accuracy: 0.9965
1543/1688 [==========================>...] - ETA: 0s - loss: 0.0125 - accuracy: 0.9965
1554/1688 [==========================>...] - ETA: 0s - loss: 0.0125 - accuracy: 0.9965
1565/1688 [==========================>...] - ETA: 0s - loss: 0.0125 - accuracy: 0.9965
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0125 - accuracy: 0.9965
1587/1688 [===========================>..] - ETA: 0s - loss: 0.0125 - accuracy: 0.9965
1598/1688 [===========================>..] - ETA: 0s - loss: 0.0125 - accuracy: 0.9965
1609/1688 [===========================>..] - ETA: 0s - loss: 0.0125 - accuracy: 0.9965
1620/1688 [===========================>..] - ETA: 0s - loss: 0.0126 - accuracy: 0.9965
1631/1688 [===========================>..] - ETA: 0s - loss: 0.0125 - accuracy: 0.9965
1642/1688 [============================>.] - ETA: 0s - loss: 0.0126 - accuracy: 0.9965
1653/1688 [============================>.] - ETA: 0s - loss: 0.0126 - accuracy: 0.9965
1664/1688 [============================>.] - ETA: 0s - loss: 0.0126 - accuracy: 0.9965
1675/1688 [============================>.] - ETA: 0s - loss: 0.0126 - accuracy: 0.9965
1686/1688 [============================>.] - ETA: 0s - loss: 0.0126 - accuracy: 0.9964
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0126 - accuracy: 0.9964 - val_loss: 0.0460 - val_accuracy: 0.9873
Epoch 4/5
1/1688 [..............................] - ETA: 8s - loss: 0.1153 - accuracy: 0.9688
12/1688 [..............................] - ETA: 8s - loss: 0.0160 - accuracy: 0.9974
23/1688 [..............................] - ETA: 8s - loss: 0.0151 - accuracy: 0.9959
34/1688 [..............................] - ETA: 8s - loss: 0.0131 - accuracy: 0.9972
44/1688 [..............................] - ETA: 8s - loss: 0.0127 - accuracy: 0.9972
55/1688 [..............................] - ETA: 8s - loss: 0.0141 - accuracy: 0.9966
66/1688 [>.............................] - ETA: 8s - loss: 0.0130 - accuracy: 0.9972
77/1688 [>.............................] - ETA: 7s - loss: 0.0117 - accuracy: 0.9976
88/1688 [>.............................] - ETA: 7s - loss: 0.0118 - accuracy: 0.9968
99/1688 [>.............................] - ETA: 7s - loss: 0.0109 - accuracy: 0.9972
110/1688 [>.............................] - ETA: 7s - loss: 0.0118 - accuracy: 0.9966
121/1688 [=>............................] - ETA: 7s - loss: 0.0114 - accuracy: 0.9969
132/1688 [=>............................] - ETA: 7s - loss: 0.0115 - accuracy: 0.9967
142/1688 [=>............................] - ETA: 7s - loss: 0.0117 - accuracy: 0.9965
153/1688 [=>............................] - ETA: 7s - loss: 0.0120 - accuracy: 0.9963
164/1688 [=>............................] - ETA: 7s - loss: 0.0118 - accuracy: 0.9964
175/1688 [==>...........................] - ETA: 7s - loss: 0.0115 - accuracy: 0.9966
186/1688 [==>...........................] - ETA: 7s - loss: 0.0119 - accuracy: 0.9965
197/1688 [==>...........................] - ETA: 7s - loss: 0.0117 - accuracy: 0.9965
208/1688 [==>...........................] - ETA: 7s - loss: 0.0117 - accuracy: 0.9965
219/1688 [==>...........................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9967
230/1688 [===>..........................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9969
241/1688 [===>..........................] - ETA: 7s - loss: 0.0112 - accuracy: 0.9969
252/1688 [===>..........................] - ETA: 7s - loss: 0.0114 - accuracy: 0.9968
263/1688 [===>..........................] - ETA: 7s - loss: 0.0110 - accuracy: 0.9969
274/1688 [===>..........................] - ETA: 6s - loss: 0.0111 - accuracy: 0.9968
285/1688 [====>.........................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9967
296/1688 [====>.........................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9966
307/1688 [====>.........................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9966
318/1688 [====>.........................] - ETA: 6s - loss: 0.0118 - accuracy: 0.9968
329/1688 [====>.........................] - ETA: 6s - loss: 0.0115 - accuracy: 0.9969
340/1688 [=====>........................] - ETA: 6s - loss: 0.0115 - accuracy: 0.9969
351/1688 [=====>........................] - ETA: 6s - loss: 0.0115 - accuracy: 0.9969
362/1688 [=====>........................] - ETA: 6s - loss: 0.0114 - accuracy: 0.9970
373/1688 [=====>........................] - ETA: 6s - loss: 0.0113 - accuracy: 0.9970
384/1688 [=====>........................] - ETA: 6s - loss: 0.0117 - accuracy: 0.9967
395/1688 [======>.......................] - ETA: 6s - loss: 0.0117 - accuracy: 0.9968
406/1688 [======>.......................] - ETA: 6s - loss: 0.0117 - accuracy: 0.9968
417/1688 [======>.......................] - ETA: 6s - loss: 0.0118 - accuracy: 0.9966
428/1688 [======>.......................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9966
438/1688 [======>.......................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9966
449/1688 [======>.......................] - ETA: 6s - loss: 0.0123 - accuracy: 0.9965
460/1688 [=======>......................] - ETA: 6s - loss: 0.0123 - accuracy: 0.9965
471/1688 [=======>......................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9964
482/1688 [=======>......................] - ETA: 5s - loss: 0.0125 - accuracy: 0.9964
493/1688 [=======>......................] - ETA: 5s - loss: 0.0126 - accuracy: 0.9963
504/1688 [=======>......................] - ETA: 5s - loss: 0.0125 - accuracy: 0.9963
515/1688 [========>.....................] - ETA: 5s - loss: 0.0125 - accuracy: 0.9963
526/1688 [========>.....................] - ETA: 5s - loss: 0.0127 - accuracy: 0.9962
537/1688 [========>.....................] - ETA: 5s - loss: 0.0126 - accuracy: 0.9962
548/1688 [========>.....................] - ETA: 5s - loss: 0.0125 - accuracy: 0.9963
559/1688 [========>.....................] - ETA: 5s - loss: 0.0125 - accuracy: 0.9963
570/1688 [=========>....................] - ETA: 5s - loss: 0.0125 - accuracy: 0.9963
581/1688 [=========>....................] - ETA: 5s - loss: 0.0127 - accuracy: 0.9962
592/1688 [=========>....................] - ETA: 5s - loss: 0.0127 - accuracy: 0.9962
603/1688 [=========>....................] - ETA: 5s - loss: 0.0126 - accuracy: 0.9963
614/1688 [=========>....................] - ETA: 5s - loss: 0.0125 - accuracy: 0.9963
625/1688 [==========>...................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9963
636/1688 [==========>...................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9963
646/1688 [==========>...................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9963
657/1688 [==========>...................] - ETA: 5s - loss: 0.0123 - accuracy: 0.9964
668/1688 [==========>...................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9964
678/1688 [===========>..................] - ETA: 4s - loss: 0.0124 - accuracy: 0.9964
689/1688 [===========>..................] - ETA: 4s - loss: 0.0123 - accuracy: 0.9964
700/1688 [===========>..................] - ETA: 4s - loss: 0.0123 - accuracy: 0.9964
711/1688 [===========>..................] - ETA: 4s - loss: 0.0123 - accuracy: 0.9964
722/1688 [===========>..................] - ETA: 4s - loss: 0.0123 - accuracy: 0.9964
733/1688 [============>.................] - ETA: 4s - loss: 0.0122 - accuracy: 0.9964
744/1688 [============>.................] - ETA: 4s - loss: 0.0122 - accuracy: 0.9965
755/1688 [============>.................] - ETA: 4s - loss: 0.0122 - accuracy: 0.9964
766/1688 [============>.................] - ETA: 4s - loss: 0.0122 - accuracy: 0.9965
777/1688 [============>.................] - ETA: 4s - loss: 0.0121 - accuracy: 0.9965
788/1688 [=============>................] - ETA: 4s - loss: 0.0121 - accuracy: 0.9965
799/1688 [=============>................] - ETA: 4s - loss: 0.0122 - accuracy: 0.9965
809/1688 [=============>................] - ETA: 4s - loss: 0.0121 - accuracy: 0.9966
820/1688 [=============>................] - ETA: 4s - loss: 0.0121 - accuracy: 0.9965
831/1688 [=============>................] - ETA: 4s - loss: 0.0121 - accuracy: 0.9965
842/1688 [=============>................] - ETA: 4s - loss: 0.0120 - accuracy: 0.9966
853/1688 [==============>...............] - ETA: 4s - loss: 0.0119 - accuracy: 0.9966
864/1688 [==============>...............] - ETA: 4s - loss: 0.0119 - accuracy: 0.9966
875/1688 [==============>...............] - ETA: 4s - loss: 0.0118 - accuracy: 0.9966
886/1688 [==============>...............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9967
897/1688 [==============>...............] - ETA: 3s - loss: 0.0117 - accuracy: 0.9967
908/1688 [===============>..............] - ETA: 3s - loss: 0.0117 - accuracy: 0.9967
919/1688 [===============>..............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9967
930/1688 [===============>..............] - ETA: 3s - loss: 0.0119 - accuracy: 0.9966
940/1688 [===============>..............] - ETA: 3s - loss: 0.0119 - accuracy: 0.9966
951/1688 [===============>..............] - ETA: 3s - loss: 0.0119 - accuracy: 0.9966
962/1688 [================>.............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9967
973/1688 [================>.............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9967
984/1688 [================>.............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9967
994/1688 [================>.............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9966
1005/1688 [================>.............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9966
1016/1688 [=================>............] - ETA: 3s - loss: 0.0117 - accuracy: 0.9967
1027/1688 [=================>............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9967
1038/1688 [=================>............] - ETA: 3s - loss: 0.0117 - accuracy: 0.9967
1048/1688 [=================>............] - ETA: 3s - loss: 0.0117 - accuracy: 0.9967
1059/1688 [=================>............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9966
1070/1688 [==================>...........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9966
1081/1688 [==================>...........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9966
1092/1688 [==================>...........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9966
1103/1688 [==================>...........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9966
1114/1688 [==================>...........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9966
1125/1688 [==================>...........] - ETA: 2s - loss: 0.0117 - accuracy: 0.9966
1136/1688 [===================>..........] - ETA: 2s - loss: 0.0117 - accuracy: 0.9967
1147/1688 [===================>..........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9967
1158/1688 [===================>..........] - ETA: 2s - loss: 0.0117 - accuracy: 0.9967
1169/1688 [===================>..........] - ETA: 2s - loss: 0.0117 - accuracy: 0.9967
1180/1688 [===================>..........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9967
1191/1688 [====================>.........] - ETA: 2s - loss: 0.0117 - accuracy: 0.9966
1202/1688 [====================>.........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9966
1213/1688 [====================>.........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9965
1223/1688 [====================>.........] - ETA: 2s - loss: 0.0119 - accuracy: 0.9966
1234/1688 [====================>.........] - ETA: 2s - loss: 0.0119 - accuracy: 0.9965
1245/1688 [=====================>........] - ETA: 2s - loss: 0.0119 - accuracy: 0.9965
1255/1688 [=====================>........] - ETA: 2s - loss: 0.0119 - accuracy: 0.9965
1266/1688 [=====================>........] - ETA: 2s - loss: 0.0120 - accuracy: 0.9965
1277/1688 [=====================>........] - ETA: 2s - loss: 0.0120 - accuracy: 0.9965
1288/1688 [=====================>........] - ETA: 1s - loss: 0.0119 - accuracy: 0.9966
1299/1688 [======================>.......] - ETA: 1s - loss: 0.0119 - accuracy: 0.9966
1310/1688 [======================>.......] - ETA: 1s - loss: 0.0119 - accuracy: 0.9966
1321/1688 [======================>.......] - ETA: 1s - loss: 0.0118 - accuracy: 0.9966
1332/1688 [======================>.......] - ETA: 1s - loss: 0.0119 - accuracy: 0.9966
1343/1688 [======================>.......] - ETA: 1s - loss: 0.0120 - accuracy: 0.9965
1354/1688 [=======================>......] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1365/1688 [=======================>......] - ETA: 1s - loss: 0.0121 - accuracy: 0.9965
1376/1688 [=======================>......] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1387/1688 [=======================>......] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1398/1688 [=======================>......] - ETA: 1s - loss: 0.0122 - accuracy: 0.9964
1409/1688 [========================>.....] - ETA: 1s - loss: 0.0122 - accuracy: 0.9964
1420/1688 [========================>.....] - ETA: 1s - loss: 0.0122 - accuracy: 0.9964
1431/1688 [========================>.....] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1442/1688 [========================>.....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9965
1453/1688 [========================>.....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9965
1464/1688 [=========================>....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9965
1475/1688 [=========================>....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9965
1486/1688 [=========================>....] - ETA: 0s - loss: 0.0120 - accuracy: 0.9965
1497/1688 [=========================>....] - ETA: 0s - loss: 0.0121 - accuracy: 0.9965
1508/1688 [=========================>....] - ETA: 0s - loss: 0.0121 - accuracy: 0.9965
1519/1688 [=========================>....] - ETA: 0s - loss: 0.0120 - accuracy: 0.9965
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0121 - accuracy: 0.9965
1541/1688 [==========================>...] - ETA: 0s - loss: 0.0121 - accuracy: 0.9965
1552/1688 [==========================>...] - ETA: 0s - loss: 0.0121 - accuracy: 0.9965
1563/1688 [==========================>...] - ETA: 0s - loss: 0.0122 - accuracy: 0.9964
1574/1688 [==========================>...] - ETA: 0s - loss: 0.0122 - accuracy: 0.9964
1585/1688 [===========================>..] - ETA: 0s - loss: 0.0122 - accuracy: 0.9965
1596/1688 [===========================>..] - ETA: 0s - loss: 0.0122 - accuracy: 0.9964
1607/1688 [===========================>..] - ETA: 0s - loss: 0.0122 - accuracy: 0.9964
1618/1688 [===========================>..] - ETA: 0s - loss: 0.0121 - accuracy: 0.9965
1629/1688 [===========================>..] - ETA: 0s - loss: 0.0121 - accuracy: 0.9965
1640/1688 [============================>.] - ETA: 0s - loss: 0.0121 - accuracy: 0.9965
1651/1688 [============================>.] - ETA: 0s - loss: 0.0121 - accuracy: 0.9965
1661/1688 [============================>.] - ETA: 0s - loss: 0.0121 - accuracy: 0.9965
1672/1688 [============================>.] - ETA: 0s - loss: 0.0121 - accuracy: 0.9965
1683/1688 [============================>.] - ETA: 0s - loss: 0.0121 - accuracy: 0.9965
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0121 - accuracy: 0.9965 - val_loss: 0.0448 - val_accuracy: 0.9885
Epoch 5/5
1/1688 [..............................] - ETA: 8s - loss: 0.0094 - accuracy: 1.0000
12/1688 [..............................] - ETA: 8s - loss: 0.0047 - accuracy: 1.0000
23/1688 [..............................] - ETA: 8s - loss: 0.0070 - accuracy: 1.0000
34/1688 [..............................] - ETA: 8s - loss: 0.0082 - accuracy: 0.9991
45/1688 [..............................] - ETA: 8s - loss: 0.0074 - accuracy: 0.9993
56/1688 [..............................] - ETA: 8s - loss: 0.0088 - accuracy: 0.9978
67/1688 [>.............................] - ETA: 7s - loss: 0.0087 - accuracy: 0.9977
78/1688 [>.............................] - ETA: 7s - loss: 0.0081 - accuracy: 0.9980
89/1688 [>.............................] - ETA: 7s - loss: 0.0074 - accuracy: 0.9982
100/1688 [>.............................] - ETA: 7s - loss: 0.0071 - accuracy: 0.9984
111/1688 [>.............................] - ETA: 7s - loss: 0.0072 - accuracy: 0.9983
122/1688 [=>............................] - ETA: 7s - loss: 0.0078 - accuracy: 0.9980
133/1688 [=>............................] - ETA: 7s - loss: 0.0087 - accuracy: 0.9977
144/1688 [=>............................] - ETA: 7s - loss: 0.0089 - accuracy: 0.9974
155/1688 [=>............................] - ETA: 7s - loss: 0.0090 - accuracy: 0.9972
166/1688 [=>............................] - ETA: 7s - loss: 0.0091 - accuracy: 0.9972
177/1688 [==>...........................] - ETA: 7s - loss: 0.0104 - accuracy: 0.9968
188/1688 [==>...........................] - ETA: 7s - loss: 0.0105 - accuracy: 0.9968
199/1688 [==>...........................] - ETA: 7s - loss: 0.0110 - accuracy: 0.9965
210/1688 [==>...........................] - ETA: 7s - loss: 0.0111 - accuracy: 0.9966
221/1688 [==>...........................] - ETA: 7s - loss: 0.0108 - accuracy: 0.9967
232/1688 [===>..........................] - ETA: 7s - loss: 0.0108 - accuracy: 0.9966
243/1688 [===>..........................] - ETA: 7s - loss: 0.0106 - accuracy: 0.9967
254/1688 [===>..........................] - ETA: 7s - loss: 0.0110 - accuracy: 0.9966
265/1688 [===>..........................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9963
276/1688 [===>..........................] - ETA: 6s - loss: 0.0114 - accuracy: 0.9964
287/1688 [====>.........................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9965
298/1688 [====>.........................] - ETA: 6s - loss: 0.0108 - accuracy: 0.9966
308/1688 [====>.........................] - ETA: 6s - loss: 0.0107 - accuracy: 0.9967
319/1688 [====>.........................] - ETA: 6s - loss: 0.0105 - accuracy: 0.9968
330/1688 [====>.........................] - ETA: 6s - loss: 0.0104 - accuracy: 0.9969
341/1688 [=====>........................] - ETA: 6s - loss: 0.0104 - accuracy: 0.9969
352/1688 [=====>........................] - ETA: 6s - loss: 0.0104 - accuracy: 0.9969
363/1688 [=====>........................] - ETA: 6s - loss: 0.0104 - accuracy: 0.9968
374/1688 [=====>........................] - ETA: 6s - loss: 0.0103 - accuracy: 0.9969
385/1688 [=====>........................] - ETA: 6s - loss: 0.0103 - accuracy: 0.9969
395/1688 [======>.......................] - ETA: 6s - loss: 0.0105 - accuracy: 0.9968
406/1688 [======>.......................] - ETA: 6s - loss: 0.0104 - accuracy: 0.9968
417/1688 [======>.......................] - ETA: 6s - loss: 0.0105 - accuracy: 0.9967
428/1688 [======>.......................] - ETA: 6s - loss: 0.0104 - accuracy: 0.9968
439/1688 [======>.......................] - ETA: 6s - loss: 0.0103 - accuracy: 0.9968
450/1688 [======>.......................] - ETA: 6s - loss: 0.0103 - accuracy: 0.9968
461/1688 [=======>......................] - ETA: 6s - loss: 0.0104 - accuracy: 0.9968
471/1688 [=======>......................] - ETA: 5s - loss: 0.0103 - accuracy: 0.9969
482/1688 [=======>......................] - ETA: 5s - loss: 0.0104 - accuracy: 0.9969
493/1688 [=======>......................] - ETA: 5s - loss: 0.0104 - accuracy: 0.9968
504/1688 [=======>......................] - ETA: 5s - loss: 0.0104 - accuracy: 0.9968
515/1688 [========>.....................] - ETA: 5s - loss: 0.0105 - accuracy: 0.9968
526/1688 [========>.....................] - ETA: 5s - loss: 0.0105 - accuracy: 0.9968
537/1688 [========>.....................] - ETA: 5s - loss: 0.0106 - accuracy: 0.9967
548/1688 [========>.....................] - ETA: 5s - loss: 0.0106 - accuracy: 0.9967
559/1688 [========>.....................] - ETA: 5s - loss: 0.0106 - accuracy: 0.9968
570/1688 [=========>....................] - ETA: 5s - loss: 0.0105 - accuracy: 0.9968
581/1688 [=========>....................] - ETA: 5s - loss: 0.0105 - accuracy: 0.9968
592/1688 [=========>....................] - ETA: 5s - loss: 0.0105 - accuracy: 0.9968
603/1688 [=========>....................] - ETA: 5s - loss: 0.0104 - accuracy: 0.9968
614/1688 [=========>....................] - ETA: 5s - loss: 0.0105 - accuracy: 0.9968
625/1688 [==========>...................] - ETA: 5s - loss: 0.0105 - accuracy: 0.9969
636/1688 [==========>...................] - ETA: 5s - loss: 0.0104 - accuracy: 0.9969
647/1688 [==========>...................] - ETA: 5s - loss: 0.0103 - accuracy: 0.9970
658/1688 [==========>...................] - ETA: 5s - loss: 0.0104 - accuracy: 0.9969
669/1688 [==========>...................] - ETA: 5s - loss: 0.0104 - accuracy: 0.9969
680/1688 [===========>..................] - ETA: 4s - loss: 0.0104 - accuracy: 0.9970
691/1688 [===========>..................] - ETA: 4s - loss: 0.0104 - accuracy: 0.9970
701/1688 [===========>..................] - ETA: 4s - loss: 0.0104 - accuracy: 0.9970
712/1688 [===========>..................] - ETA: 4s - loss: 0.0103 - accuracy: 0.9971
723/1688 [===========>..................] - ETA: 4s - loss: 0.0103 - accuracy: 0.9971
734/1688 [============>.................] - ETA: 4s - loss: 0.0104 - accuracy: 0.9971
745/1688 [============>.................] - ETA: 4s - loss: 0.0103 - accuracy: 0.9971
756/1688 [============>.................] - ETA: 4s - loss: 0.0103 - accuracy: 0.9971
767/1688 [============>.................] - ETA: 4s - loss: 0.0103 - accuracy: 0.9971
778/1688 [============>.................] - ETA: 4s - loss: 0.0103 - accuracy: 0.9971
789/1688 [=============>................] - ETA: 4s - loss: 0.0103 - accuracy: 0.9971
800/1688 [=============>................] - ETA: 4s - loss: 0.0102 - accuracy: 0.9971
811/1688 [=============>................] - ETA: 4s - loss: 0.0103 - accuracy: 0.9971
822/1688 [=============>................] - ETA: 4s - loss: 0.0103 - accuracy: 0.9971
833/1688 [=============>................] - ETA: 4s - loss: 0.0102 - accuracy: 0.9971
844/1688 [==============>...............] - ETA: 4s - loss: 0.0103 - accuracy: 0.9971
855/1688 [==============>...............] - ETA: 4s - loss: 0.0103 - accuracy: 0.9971
866/1688 [==============>...............] - ETA: 4s - loss: 0.0103 - accuracy: 0.9971
877/1688 [==============>...............] - ETA: 3s - loss: 0.0103 - accuracy: 0.9971
888/1688 [==============>...............] - ETA: 3s - loss: 0.0103 - accuracy: 0.9971
899/1688 [==============>...............] - ETA: 3s - loss: 0.0103 - accuracy: 0.9971
910/1688 [===============>..............] - ETA: 3s - loss: 0.0103 - accuracy: 0.9971
921/1688 [===============>..............] - ETA: 3s - loss: 0.0103 - accuracy: 0.9971
932/1688 [===============>..............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9971
943/1688 [===============>..............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9971
954/1688 [===============>..............] - ETA: 3s - loss: 0.0103 - accuracy: 0.9972
965/1688 [================>.............] - ETA: 3s - loss: 0.0103 - accuracy: 0.9972
976/1688 [================>.............] - ETA: 3s - loss: 0.0103 - accuracy: 0.9972
987/1688 [================>.............] - ETA: 3s - loss: 0.0103 - accuracy: 0.9972
998/1688 [================>.............] - ETA: 3s - loss: 0.0103 - accuracy: 0.9972
1009/1688 [================>.............] - ETA: 3s - loss: 0.0102 - accuracy: 0.9972
1020/1688 [=================>............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9972
1031/1688 [=================>............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9972
1042/1688 [=================>............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9972
1053/1688 [=================>............] - ETA: 3s - loss: 0.0103 - accuracy: 0.9972
1064/1688 [=================>............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9972
1075/1688 [==================>...........] - ETA: 3s - loss: 0.0104 - accuracy: 0.9972
1086/1688 [==================>...........] - ETA: 2s - loss: 0.0104 - accuracy: 0.9972
1097/1688 [==================>...........] - ETA: 2s - loss: 0.0105 - accuracy: 0.9971
1108/1688 [==================>...........] - ETA: 2s - loss: 0.0105 - accuracy: 0.9971
1119/1688 [==================>...........] - ETA: 2s - loss: 0.0105 - accuracy: 0.9971
1129/1688 [===================>..........] - ETA: 2s - loss: 0.0105 - accuracy: 0.9971
1140/1688 [===================>..........] - ETA: 2s - loss: 0.0105 - accuracy: 0.9971
1151/1688 [===================>..........] - ETA: 2s - loss: 0.0105 - accuracy: 0.9971
1162/1688 [===================>..........] - ETA: 2s - loss: 0.0105 - accuracy: 0.9971
1173/1688 [===================>..........] - ETA: 2s - loss: 0.0105 - accuracy: 0.9971
1184/1688 [====================>.........] - ETA: 2s - loss: 0.0105 - accuracy: 0.9972
1195/1688 [====================>.........] - ETA: 2s - loss: 0.0105 - accuracy: 0.9971
1206/1688 [====================>.........] - ETA: 2s - loss: 0.0105 - accuracy: 0.9971
1217/1688 [====================>.........] - ETA: 2s - loss: 0.0106 - accuracy: 0.9971
1228/1688 [====================>.........] - ETA: 2s - loss: 0.0107 - accuracy: 0.9971
1239/1688 [=====================>........] - ETA: 2s - loss: 0.0108 - accuracy: 0.9970
1250/1688 [=====================>........] - ETA: 2s - loss: 0.0108 - accuracy: 0.9970
1261/1688 [=====================>........] - ETA: 2s - loss: 0.0108 - accuracy: 0.9971
1272/1688 [=====================>........] - ETA: 2s - loss: 0.0108 - accuracy: 0.9971
1282/1688 [=====================>........] - ETA: 1s - loss: 0.0108 - accuracy: 0.9971
1293/1688 [=====================>........] - ETA: 1s - loss: 0.0108 - accuracy: 0.9971
1304/1688 [======================>.......] - ETA: 1s - loss: 0.0108 - accuracy: 0.9971
1315/1688 [======================>.......] - ETA: 1s - loss: 0.0109 - accuracy: 0.9971
1326/1688 [======================>.......] - ETA: 1s - loss: 0.0109 - accuracy: 0.9971
1337/1688 [======================>.......] - ETA: 1s - loss: 0.0109 - accuracy: 0.9971
1348/1688 [======================>.......] - ETA: 1s - loss: 0.0108 - accuracy: 0.9971
1359/1688 [=======================>......] - ETA: 1s - loss: 0.0108 - accuracy: 0.9971
1370/1688 [=======================>......] - ETA: 1s - loss: 0.0108 - accuracy: 0.9971
1381/1688 [=======================>......] - ETA: 1s - loss: 0.0108 - accuracy: 0.9971
1392/1688 [=======================>......] - ETA: 1s - loss: 0.0108 - accuracy: 0.9971
1402/1688 [=======================>......] - ETA: 1s - loss: 0.0108 - accuracy: 0.9971
1413/1688 [========================>.....] - ETA: 1s - loss: 0.0108 - accuracy: 0.9971
1424/1688 [========================>.....] - ETA: 1s - loss: 0.0108 - accuracy: 0.9971
1435/1688 [========================>.....] - ETA: 1s - loss: 0.0107 - accuracy: 0.9971
1446/1688 [========================>.....] - ETA: 1s - loss: 0.0107 - accuracy: 0.9971
1457/1688 [========================>.....] - ETA: 1s - loss: 0.0107 - accuracy: 0.9971
1468/1688 [=========================>....] - ETA: 1s - loss: 0.0108 - accuracy: 0.9971
1479/1688 [=========================>....] - ETA: 1s - loss: 0.0108 - accuracy: 0.9971
1489/1688 [=========================>....] - ETA: 0s - loss: 0.0108 - accuracy: 0.9971
1500/1688 [=========================>....] - ETA: 0s - loss: 0.0108 - accuracy: 0.9970
1511/1688 [=========================>....] - ETA: 0s - loss: 0.0109 - accuracy: 0.9970
1522/1688 [==========================>...] - ETA: 0s - loss: 0.0109 - accuracy: 0.9970
1533/1688 [==========================>...] - ETA: 0s - loss: 0.0109 - accuracy: 0.9970
1544/1688 [==========================>...] - ETA: 0s - loss: 0.0109 - accuracy: 0.9970
1555/1688 [==========================>...] - ETA: 0s - loss: 0.0108 - accuracy: 0.9970
1566/1688 [==========================>...] - ETA: 0s - loss: 0.0108 - accuracy: 0.9970
1577/1688 [===========================>..] - ETA: 0s - loss: 0.0108 - accuracy: 0.9970
1588/1688 [===========================>..] - ETA: 0s - loss: 0.0108 - accuracy: 0.9971
1599/1688 [===========================>..] - ETA: 0s - loss: 0.0107 - accuracy: 0.9971
1610/1688 [===========================>..] - ETA: 0s - loss: 0.0109 - accuracy: 0.9970
1621/1688 [===========================>..] - ETA: 0s - loss: 0.0109 - accuracy: 0.9970
1632/1688 [============================>.] - ETA: 0s - loss: 0.0109 - accuracy: 0.9970
1643/1688 [============================>.] - ETA: 0s - loss: 0.0109 - accuracy: 0.9970
1653/1688 [============================>.] - ETA: 0s - loss: 0.0109 - accuracy: 0.9970
1664/1688 [============================>.] - ETA: 0s - loss: 0.0110 - accuracy: 0.9970
1674/1688 [============================>.] - ETA: 0s - loss: 0.0110 - accuracy: 0.9969
1685/1688 [============================>.] - ETA: 0s - loss: 0.0110 - accuracy: 0.9969
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0110 - accuracy: 0.9969 - val_loss: 0.0429 - val_accuracy: 0.9890
<keras.src.callbacks.History object at 0x7f390971e450>
score = model_quantized.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after fine tuning:', score)
Test accuracy after fine tuning: 0.9871000051498413
3. Convert
3.1 Convert to Akida model
When the quantized model produces satisfactory performance, it can be converted to the native Akida format. The convert function returns a model in Akida format ready for inference.
As with Keras, the summary() method provides a textual representation of the Akida model.
from cnn2snn import convert
model_akida = convert(model_quantized)
model_akida.summary()
Model Summary
______________________________________________
Input shape Output shape Sequences Layers
==============================================
[28, 28, 1] [1, 1, 10] 1 5
______________________________________________
__________________________________________________________________
Layer (type) Output shape Kernel shape
=============== SW/conv2d-dequantizer_2 (Software) ===============
conv2d (InputConv2D) [13, 13, 32] (3, 3, 1, 32)
__________________________________________________________________
depthwise_conv2d (DepthwiseConv2D) [7, 7, 32] (3, 3, 32, 1)
__________________________________________________________________
conv2d_1 (Conv2D) [7, 7, 64] (1, 1, 32, 64)
__________________________________________________________________
dense (Dense1D) [1, 1, 10] (3136, 10)
__________________________________________________________________
dequantizer_2 (Dequantizer) [1, 1, 10] N/A
__________________________________________________________________
3.2. Check performance
accuracy = model_akida.evaluate(x_test, y_test)
print('Test accuracy after conversion:', accuracy)
# For non-regression purposes
assert accuracy > 0.96
Test accuracy after conversion: 0.9848999977111816
3.3 Show predictions for a single image
Display one of the test images, such as the first image in the dataset from above, to visualize the output of the model.
# Test a single example
sample_image = 0
image = x_test[sample_image]
outputs = model_akida.predict(image.reshape(1, 28, 28, 1))
print('Input Label: %i' % y_test[sample_image])
f, axarr = plt.subplots(1, 2)
axarr[0].imshow(x_test[sample_image].reshape((28, 28)), cmap=cm.Greys_r)
axarr[0].set_title('Class %d' % y_test[sample_image])
axarr[1].bar(range(10), outputs.squeeze())
axarr[1].set_xticks(range(10))
plt.show()
print(outputs.squeeze())
Input Label: 7
[ -9.631428 -8.194003 -4.3365364 -1.1605749 -10.2073555 -5.5714784
-18.182737 7.1073794 -5.191756 -0.8129991]
Consider the output from the model above. As is typical in backprop-trained models, the final layer is a Dense layer with one neuron for each of the 10 classes in the dataset. The goal of training is to maximize the response of the neuron corresponding to the label of each training sample while minimizing the responses of the other neurons.
In the bar chart above, you can see the outputs from all 10 neurons. It is easy to see that neuron 7 responds much more strongly than the others. The first sample is indeed a number 7.
Total running time of the script: (2 minutes 5.785 seconds)