Note
Go to the end to download the full example code.
Global Akida workflow
Using the MNIST dataset, this example shows the definition and training of a keras floating point model, its quantization to 8-bit with the help of calibration, its quantization to 4-bit using QAT and its conversion to Akida. Notice that the performance of the original keras floating point model is maintained throughout the Akida flow. Please refer to the Akida user guide for further information.
Note
Please refer to the TensorFlow tf.keras.models module for model creation/import details and the TensorFlow Guide for TensorFlow usage.
The MNIST example below is light enough so that a GPU is not needed for training.
Global Akida workflow
1. Create and train
1.1. Load and reshape MNIST dataset
import numpy as np
import matplotlib.cm as cm
import matplotlib.pyplot as plt
from keras.datasets import mnist
# Load MNIST dataset
(x_train, y_train), (x_test, y_test) = mnist.load_data()
# Add a channels dimension to the image sets as Akida expects 4-D inputs (corresponding to
# (num_samples, width, height, channels). Note: MNIST is a grayscale dataset and is unusual
# in this respect - most image data already includes a channel dimension, and this step will
# not be necessary.
x_train = np.expand_dims(x_train, -1)
x_test = np.expand_dims(x_test, -1)
# Display a few images from the test set
f, axarr = plt.subplots(1, 4)
for i in range(0, 4):
axarr[i].imshow(x_test[i].reshape((28, 28)), cmap=cm.Greys_r)
axarr[i].set_title('Class %d' % y_test[i])
plt.show()
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
8192/11490434 [..............................] - ETA: 0s
49152/11490434 [..............................] - ETA: 19s
81920/11490434 [..............................] - ETA: 22s
262144/11490434 [..............................] - ETA: 9s
393216/11490434 [>.............................] - ETA: 7s
606208/11490434 [>.............................] - ETA: 5s
983040/11490434 [=>............................] - ETA: 3s
1474560/11490434 [==>...........................] - ETA: 2s
2080768/11490434 [====>.........................] - ETA: 2s
3022848/11490434 [======>.......................] - ETA: 1s
3956736/11490434 [=========>....................] - ETA: 1s
5242880/11490434 [============>.................] - ETA: 0s
6520832/11490434 [================>.............] - ETA: 0s
7258112/11490434 [=================>............] - ETA: 0s
8437760/11490434 [=====================>........] - ETA: 0s
9388032/11490434 [=======================>......] - ETA: 0s
10665984/11490434 [==========================>...] - ETA: 0s
11490434/11490434 [==============================] - 1s 0us/step
1.2. Model definition
Note that at this stage, there is nothing specific to the Akida IP. The model constructed below, as inspired by this example, is a completely standard Keras CNN model.
import keras
model_keras = keras.models.Sequential([
keras.layers.Rescaling(1. / 255, input_shape=(28, 28, 1)),
keras.layers.Conv2D(filters=32, kernel_size=3, strides=2),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
# Separable layer
keras.layers.DepthwiseConv2D(kernel_size=3, padding='same', strides=2),
keras.layers.Conv2D(filters=64, kernel_size=1, padding='same'),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
keras.layers.Flatten(),
keras.layers.Dense(10)
], 'mnistnet')
model_keras.summary()
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling (Rescaling) (None, 28, 28, 1) 0
conv2d (Conv2D) (None, 13, 13, 32) 320
batch_normalization (Batch (None, 13, 13, 32) 128
Normalization)
re_lu (ReLU) (None, 13, 13, 32) 0
depthwise_conv2d (Depthwis (None, 7, 7, 32) 320
eConv2D)
conv2d_1 (Conv2D) (None, 7, 7, 64) 2112
batch_normalization_1 (Bat (None, 7, 7, 64) 256
chNormalization)
re_lu_1 (ReLU) (None, 7, 7, 64) 0
flatten (Flatten) (None, 3136) 0
dense (Dense) (None, 10) 31370
=================================================================
Total params: 34506 (134.79 KB)
Trainable params: 34314 (134.04 KB)
Non-trainable params: 192 (768.00 Byte)
_________________________________________________________________
1.3. Model training
Given the model created above, train the model and check its accuracy. The model should achieve a test accuracy over 98% after 10 epochs.
from keras.optimizers import Adam
model_keras.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-3),
metrics=['accuracy'])
_ = model_keras.fit(x_train, y_train, epochs=10, validation_split=0.1)
Epoch 1/10
1/1688 [..............................] - ETA: 50:40 - loss: 2.6358 - accuracy: 0.0938
23/1688 [..............................] - ETA: 3s - loss: 1.4258 - accuracy: 0.5312
46/1688 [..............................] - ETA: 3s - loss: 1.0084 - accuracy: 0.6773
69/1688 [>.............................] - ETA: 3s - loss: 0.8227 - accuracy: 0.7378
92/1688 [>.............................] - ETA: 3s - loss: 0.7094 - accuracy: 0.7738
115/1688 [=>............................] - ETA: 3s - loss: 0.6485 - accuracy: 0.7940
138/1688 [=>............................] - ETA: 3s - loss: 0.5908 - accuracy: 0.8136
161/1688 [=>............................] - ETA: 3s - loss: 0.5476 - accuracy: 0.8273
183/1688 [==>...........................] - ETA: 3s - loss: 0.5089 - accuracy: 0.8402
207/1688 [==>...........................] - ETA: 3s - loss: 0.4783 - accuracy: 0.8513
229/1688 [===>..........................] - ETA: 3s - loss: 0.4560 - accuracy: 0.8589
252/1688 [===>..........................] - ETA: 3s - loss: 0.4360 - accuracy: 0.8650
275/1688 [===>..........................] - ETA: 3s - loss: 0.4177 - accuracy: 0.8711
298/1688 [====>.........................] - ETA: 3s - loss: 0.4017 - accuracy: 0.8768
321/1688 [====>.........................] - ETA: 3s - loss: 0.3842 - accuracy: 0.8824
344/1688 [=====>........................] - ETA: 2s - loss: 0.3722 - accuracy: 0.8861
367/1688 [=====>........................] - ETA: 2s - loss: 0.3590 - accuracy: 0.8900
390/1688 [=====>........................] - ETA: 2s - loss: 0.3470 - accuracy: 0.8933
413/1688 [======>.......................] - ETA: 2s - loss: 0.3338 - accuracy: 0.8976
436/1688 [======>.......................] - ETA: 2s - loss: 0.3238 - accuracy: 0.9006
459/1688 [=======>......................] - ETA: 2s - loss: 0.3153 - accuracy: 0.9033
482/1688 [=======>......................] - ETA: 2s - loss: 0.3095 - accuracy: 0.9048
505/1688 [=======>......................] - ETA: 2s - loss: 0.3017 - accuracy: 0.9072
528/1688 [========>.....................] - ETA: 2s - loss: 0.2935 - accuracy: 0.9100
551/1688 [========>.....................] - ETA: 2s - loss: 0.2858 - accuracy: 0.9123
574/1688 [=========>....................] - ETA: 2s - loss: 0.2793 - accuracy: 0.9143
597/1688 [=========>....................] - ETA: 2s - loss: 0.2735 - accuracy: 0.9160
620/1688 [==========>...................] - ETA: 2s - loss: 0.2669 - accuracy: 0.9181
643/1688 [==========>...................] - ETA: 2s - loss: 0.2625 - accuracy: 0.9196
666/1688 [==========>...................] - ETA: 2s - loss: 0.2590 - accuracy: 0.9205
689/1688 [===========>..................] - ETA: 2s - loss: 0.2558 - accuracy: 0.9212
712/1688 [===========>..................] - ETA: 2s - loss: 0.2512 - accuracy: 0.9227
735/1688 [============>.................] - ETA: 2s - loss: 0.2467 - accuracy: 0.9241
758/1688 [============>.................] - ETA: 2s - loss: 0.2422 - accuracy: 0.9253
781/1688 [============>.................] - ETA: 2s - loss: 0.2383 - accuracy: 0.9265
803/1688 [=============>................] - ETA: 1s - loss: 0.2341 - accuracy: 0.9278
826/1688 [=============>................] - ETA: 1s - loss: 0.2310 - accuracy: 0.9288
849/1688 [==============>...............] - ETA: 1s - loss: 0.2274 - accuracy: 0.9299
872/1688 [==============>...............] - ETA: 1s - loss: 0.2248 - accuracy: 0.9306
895/1688 [==============>...............] - ETA: 1s - loss: 0.2224 - accuracy: 0.9314
918/1688 [===============>..............] - ETA: 1s - loss: 0.2197 - accuracy: 0.9323
941/1688 [===============>..............] - ETA: 1s - loss: 0.2170 - accuracy: 0.9331
963/1688 [================>.............] - ETA: 1s - loss: 0.2140 - accuracy: 0.9341
986/1688 [================>.............] - ETA: 1s - loss: 0.2114 - accuracy: 0.9346
1009/1688 [================>.............] - ETA: 1s - loss: 0.2091 - accuracy: 0.9355
1032/1688 [=================>............] - ETA: 1s - loss: 0.2067 - accuracy: 0.9361
1055/1688 [=================>............] - ETA: 1s - loss: 0.2046 - accuracy: 0.9366
1078/1688 [==================>...........] - ETA: 1s - loss: 0.2023 - accuracy: 0.9375
1100/1688 [==================>...........] - ETA: 1s - loss: 0.2009 - accuracy: 0.9380
1123/1688 [==================>...........] - ETA: 1s - loss: 0.1986 - accuracy: 0.9387
1146/1688 [===================>..........] - ETA: 1s - loss: 0.1966 - accuracy: 0.9391
1169/1688 [===================>..........] - ETA: 1s - loss: 0.1949 - accuracy: 0.9396
1192/1688 [====================>.........] - ETA: 1s - loss: 0.1932 - accuracy: 0.9403
1214/1688 [====================>.........] - ETA: 1s - loss: 0.1908 - accuracy: 0.9409
1237/1688 [====================>.........] - ETA: 1s - loss: 0.1886 - accuracy: 0.9418
1260/1688 [=====================>........] - ETA: 0s - loss: 0.1866 - accuracy: 0.9424
1283/1688 [=====================>........] - ETA: 0s - loss: 0.1851 - accuracy: 0.9431
1306/1688 [======================>.......] - ETA: 0s - loss: 0.1839 - accuracy: 0.9434
1329/1688 [======================>.......] - ETA: 0s - loss: 0.1823 - accuracy: 0.9439
1352/1688 [=======================>......] - ETA: 0s - loss: 0.1806 - accuracy: 0.9445
1375/1688 [=======================>......] - ETA: 0s - loss: 0.1790 - accuracy: 0.9449
1398/1688 [=======================>......] - ETA: 0s - loss: 0.1773 - accuracy: 0.9455
1421/1688 [========================>.....] - ETA: 0s - loss: 0.1758 - accuracy: 0.9460
1444/1688 [========================>.....] - ETA: 0s - loss: 0.1741 - accuracy: 0.9465
1467/1688 [=========================>....] - ETA: 0s - loss: 0.1733 - accuracy: 0.9467
1490/1688 [=========================>....] - ETA: 0s - loss: 0.1720 - accuracy: 0.9471
1513/1688 [=========================>....] - ETA: 0s - loss: 0.1705 - accuracy: 0.9475
1536/1688 [==========================>...] - ETA: 0s - loss: 0.1695 - accuracy: 0.9478
1559/1688 [==========================>...] - ETA: 0s - loss: 0.1685 - accuracy: 0.9481
1582/1688 [===========================>..] - ETA: 0s - loss: 0.1672 - accuracy: 0.9485
1605/1688 [===========================>..] - ETA: 0s - loss: 0.1662 - accuracy: 0.9488
1628/1688 [===========================>..] - ETA: 0s - loss: 0.1650 - accuracy: 0.9491
1651/1688 [============================>.] - ETA: 0s - loss: 0.1638 - accuracy: 0.9495
1674/1688 [============================>.] - ETA: 0s - loss: 0.1631 - accuracy: 0.9497
1688/1688 [==============================] - ETA: 0s - loss: 0.1626 - accuracy: 0.9498
1688/1688 [==============================] - 6s 2ms/step - loss: 0.1626 - accuracy: 0.9498 - val_loss: 0.0897 - val_accuracy: 0.9733
Epoch 2/10
1/1688 [..............................] - ETA: 3s - loss: 0.0110 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0674 - accuracy: 0.9727
47/1688 [..............................] - ETA: 3s - loss: 0.0684 - accuracy: 0.9741
69/1688 [>.............................] - ETA: 3s - loss: 0.0636 - accuracy: 0.9746
92/1688 [>.............................] - ETA: 3s - loss: 0.0598 - accuracy: 0.9769
115/1688 [=>............................] - ETA: 3s - loss: 0.0648 - accuracy: 0.9774
138/1688 [=>............................] - ETA: 3s - loss: 0.0663 - accuracy: 0.9767
161/1688 [=>............................] - ETA: 3s - loss: 0.0636 - accuracy: 0.9781
184/1688 [==>...........................] - ETA: 3s - loss: 0.0703 - accuracy: 0.9764
206/1688 [==>...........................] - ETA: 3s - loss: 0.0711 - accuracy: 0.9766
229/1688 [===>..........................] - ETA: 3s - loss: 0.0698 - accuracy: 0.9769
252/1688 [===>..........................] - ETA: 3s - loss: 0.0686 - accuracy: 0.9782
275/1688 [===>..........................] - ETA: 3s - loss: 0.0687 - accuracy: 0.9785
298/1688 [====>.........................] - ETA: 3s - loss: 0.0680 - accuracy: 0.9788
321/1688 [====>.........................] - ETA: 3s - loss: 0.0686 - accuracy: 0.9790
344/1688 [=====>........................] - ETA: 3s - loss: 0.0684 - accuracy: 0.9789
367/1688 [=====>........................] - ETA: 2s - loss: 0.0678 - accuracy: 0.9786
390/1688 [=====>........................] - ETA: 2s - loss: 0.0672 - accuracy: 0.9790
413/1688 [======>.......................] - ETA: 2s - loss: 0.0680 - accuracy: 0.9789
436/1688 [======>.......................] - ETA: 2s - loss: 0.0682 - accuracy: 0.9787
459/1688 [=======>......................] - ETA: 2s - loss: 0.0691 - accuracy: 0.9785
482/1688 [=======>......................] - ETA: 2s - loss: 0.0698 - accuracy: 0.9783
505/1688 [=======>......................] - ETA: 2s - loss: 0.0699 - accuracy: 0.9786
528/1688 [========>.....................] - ETA: 2s - loss: 0.0695 - accuracy: 0.9785
551/1688 [========>.....................] - ETA: 2s - loss: 0.0697 - accuracy: 0.9783
574/1688 [=========>....................] - ETA: 2s - loss: 0.0688 - accuracy: 0.9784
597/1688 [=========>....................] - ETA: 2s - loss: 0.0693 - accuracy: 0.9783
620/1688 [==========>...................] - ETA: 2s - loss: 0.0690 - accuracy: 0.9782
643/1688 [==========>...................] - ETA: 2s - loss: 0.0694 - accuracy: 0.9780
665/1688 [==========>...................] - ETA: 2s - loss: 0.0686 - accuracy: 0.9783
688/1688 [===========>..................] - ETA: 2s - loss: 0.0692 - accuracy: 0.9782
711/1688 [===========>..................] - ETA: 2s - loss: 0.0692 - accuracy: 0.9781
733/1688 [============>.................] - ETA: 2s - loss: 0.0693 - accuracy: 0.9780
756/1688 [============>.................] - ETA: 2s - loss: 0.0696 - accuracy: 0.9778
779/1688 [============>.................] - ETA: 2s - loss: 0.0697 - accuracy: 0.9779
802/1688 [=============>................] - ETA: 1s - loss: 0.0696 - accuracy: 0.9779
825/1688 [=============>................] - ETA: 1s - loss: 0.0695 - accuracy: 0.9778
848/1688 [==============>...............] - ETA: 1s - loss: 0.0691 - accuracy: 0.9780
871/1688 [==============>...............] - ETA: 1s - loss: 0.0691 - accuracy: 0.9781
894/1688 [==============>...............] - ETA: 1s - loss: 0.0691 - accuracy: 0.9782
917/1688 [===============>..............] - ETA: 1s - loss: 0.0694 - accuracy: 0.9781
940/1688 [===============>..............] - ETA: 1s - loss: 0.0703 - accuracy: 0.9780
963/1688 [================>.............] - ETA: 1s - loss: 0.0699 - accuracy: 0.9781
986/1688 [================>.............] - ETA: 1s - loss: 0.0700 - accuracy: 0.9782
1009/1688 [================>.............] - ETA: 1s - loss: 0.0700 - accuracy: 0.9782
1032/1688 [=================>............] - ETA: 1s - loss: 0.0698 - accuracy: 0.9783
1055/1688 [=================>............] - ETA: 1s - loss: 0.0698 - accuracy: 0.9782
1078/1688 [==================>...........] - ETA: 1s - loss: 0.0697 - accuracy: 0.9783
1101/1688 [==================>...........] - ETA: 1s - loss: 0.0695 - accuracy: 0.9783
1124/1688 [==================>...........] - ETA: 1s - loss: 0.0691 - accuracy: 0.9785
1147/1688 [===================>..........] - ETA: 1s - loss: 0.0686 - accuracy: 0.9785
1170/1688 [===================>..........] - ETA: 1s - loss: 0.0679 - accuracy: 0.9788
1193/1688 [====================>.........] - ETA: 1s - loss: 0.0675 - accuracy: 0.9788
1216/1688 [====================>.........] - ETA: 1s - loss: 0.0673 - accuracy: 0.9790
1238/1688 [=====================>........] - ETA: 1s - loss: 0.0674 - accuracy: 0.9790
1260/1688 [=====================>........] - ETA: 0s - loss: 0.0676 - accuracy: 0.9789
1281/1688 [=====================>........] - ETA: 0s - loss: 0.0674 - accuracy: 0.9789
1303/1688 [======================>.......] - ETA: 0s - loss: 0.0674 - accuracy: 0.9789
1324/1688 [======================>.......] - ETA: 0s - loss: 0.0673 - accuracy: 0.9789
1346/1688 [======================>.......] - ETA: 0s - loss: 0.0673 - accuracy: 0.9789
1367/1688 [=======================>......] - ETA: 0s - loss: 0.0671 - accuracy: 0.9789
1389/1688 [=======================>......] - ETA: 0s - loss: 0.0668 - accuracy: 0.9791
1410/1688 [========================>.....] - ETA: 0s - loss: 0.0665 - accuracy: 0.9791
1432/1688 [========================>.....] - ETA: 0s - loss: 0.0668 - accuracy: 0.9791
1453/1688 [========================>.....] - ETA: 0s - loss: 0.0668 - accuracy: 0.9791
1474/1688 [=========================>....] - ETA: 0s - loss: 0.0668 - accuracy: 0.9792
1495/1688 [=========================>....] - ETA: 0s - loss: 0.0670 - accuracy: 0.9792
1516/1688 [=========================>....] - ETA: 0s - loss: 0.0672 - accuracy: 0.9791
1538/1688 [==========================>...] - ETA: 0s - loss: 0.0671 - accuracy: 0.9792
1560/1688 [==========================>...] - ETA: 0s - loss: 0.0672 - accuracy: 0.9791
1581/1688 [===========================>..] - ETA: 0s - loss: 0.0673 - accuracy: 0.9791
1602/1688 [===========================>..] - ETA: 0s - loss: 0.0677 - accuracy: 0.9790
1623/1688 [===========================>..] - ETA: 0s - loss: 0.0677 - accuracy: 0.9790
1644/1688 [============================>.] - ETA: 0s - loss: 0.0674 - accuracy: 0.9791
1665/1688 [============================>.] - ETA: 0s - loss: 0.0678 - accuracy: 0.9790
1687/1688 [============================>.] - ETA: 0s - loss: 0.0676 - accuracy: 0.9790
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0676 - accuracy: 0.9790 - val_loss: 0.0857 - val_accuracy: 0.9772
Epoch 3/10
1/1688 [..............................] - ETA: 3s - loss: 0.0089 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0567 - accuracy: 0.9792
46/1688 [..............................] - ETA: 3s - loss: 0.0480 - accuracy: 0.9803
69/1688 [>.............................] - ETA: 3s - loss: 0.0462 - accuracy: 0.9828
92/1688 [>.............................] - ETA: 3s - loss: 0.0425 - accuracy: 0.9854
115/1688 [=>............................] - ETA: 3s - loss: 0.0435 - accuracy: 0.9845
138/1688 [=>............................] - ETA: 3s - loss: 0.0427 - accuracy: 0.9844
161/1688 [=>............................] - ETA: 3s - loss: 0.0425 - accuracy: 0.9845
184/1688 [==>...........................] - ETA: 3s - loss: 0.0437 - accuracy: 0.9844
207/1688 [==>...........................] - ETA: 3s - loss: 0.0452 - accuracy: 0.9837
230/1688 [===>..........................] - ETA: 3s - loss: 0.0432 - accuracy: 0.9842
252/1688 [===>..........................] - ETA: 3s - loss: 0.0419 - accuracy: 0.9847
275/1688 [===>..........................] - ETA: 3s - loss: 0.0442 - accuracy: 0.9844
298/1688 [====>.........................] - ETA: 3s - loss: 0.0438 - accuracy: 0.9846
321/1688 [====>.........................] - ETA: 3s - loss: 0.0447 - accuracy: 0.9840
343/1688 [=====>........................] - ETA: 3s - loss: 0.0470 - accuracy: 0.9836
366/1688 [=====>........................] - ETA: 2s - loss: 0.0469 - accuracy: 0.9836
389/1688 [=====>........................] - ETA: 2s - loss: 0.0466 - accuracy: 0.9839
412/1688 [======>.......................] - ETA: 2s - loss: 0.0467 - accuracy: 0.9841
435/1688 [======>.......................] - ETA: 2s - loss: 0.0465 - accuracy: 0.9843
458/1688 [=======>......................] - ETA: 2s - loss: 0.0479 - accuracy: 0.9842
481/1688 [=======>......................] - ETA: 2s - loss: 0.0488 - accuracy: 0.9841
504/1688 [=======>......................] - ETA: 2s - loss: 0.0491 - accuracy: 0.9841
527/1688 [========>.....................] - ETA: 2s - loss: 0.0495 - accuracy: 0.9840
550/1688 [========>.....................] - ETA: 2s - loss: 0.0497 - accuracy: 0.9840
573/1688 [=========>....................] - ETA: 2s - loss: 0.0504 - accuracy: 0.9837
596/1688 [=========>....................] - ETA: 2s - loss: 0.0514 - accuracy: 0.9834
619/1688 [==========>...................] - ETA: 2s - loss: 0.0510 - accuracy: 0.9835
642/1688 [==========>...................] - ETA: 2s - loss: 0.0497 - accuracy: 0.9838
665/1688 [==========>...................] - ETA: 2s - loss: 0.0495 - accuracy: 0.9839
688/1688 [===========>..................] - ETA: 2s - loss: 0.0493 - accuracy: 0.9839
711/1688 [===========>..................] - ETA: 2s - loss: 0.0496 - accuracy: 0.9839
735/1688 [============>.................] - ETA: 2s - loss: 0.0496 - accuracy: 0.9839
758/1688 [============>.................] - ETA: 2s - loss: 0.0497 - accuracy: 0.9838
781/1688 [============>.................] - ETA: 2s - loss: 0.0491 - accuracy: 0.9840
804/1688 [=============>................] - ETA: 1s - loss: 0.0491 - accuracy: 0.9839
827/1688 [=============>................] - ETA: 1s - loss: 0.0489 - accuracy: 0.9840
850/1688 [==============>...............] - ETA: 1s - loss: 0.0487 - accuracy: 0.9842
873/1688 [==============>...............] - ETA: 1s - loss: 0.0482 - accuracy: 0.9843
896/1688 [==============>...............] - ETA: 1s - loss: 0.0480 - accuracy: 0.9842
919/1688 [===============>..............] - ETA: 1s - loss: 0.0479 - accuracy: 0.9844
942/1688 [===============>..............] - ETA: 1s - loss: 0.0483 - accuracy: 0.9843
965/1688 [================>.............] - ETA: 1s - loss: 0.0489 - accuracy: 0.9844
987/1688 [================>.............] - ETA: 1s - loss: 0.0493 - accuracy: 0.9843
1010/1688 [================>.............] - ETA: 1s - loss: 0.0489 - accuracy: 0.9844
1032/1688 [=================>............] - ETA: 1s - loss: 0.0488 - accuracy: 0.9844
1055/1688 [=================>............] - ETA: 1s - loss: 0.0488 - accuracy: 0.9844
1078/1688 [==================>...........] - ETA: 1s - loss: 0.0485 - accuracy: 0.9845
1101/1688 [==================>...........] - ETA: 1s - loss: 0.0482 - accuracy: 0.9846
1124/1688 [==================>...........] - ETA: 1s - loss: 0.0479 - accuracy: 0.9847
1147/1688 [===================>..........] - ETA: 1s - loss: 0.0475 - accuracy: 0.9848
1170/1688 [===================>..........] - ETA: 1s - loss: 0.0475 - accuracy: 0.9848
1193/1688 [====================>.........] - ETA: 1s - loss: 0.0474 - accuracy: 0.9849
1215/1688 [====================>.........] - ETA: 1s - loss: 0.0475 - accuracy: 0.9849
1238/1688 [=====================>........] - ETA: 1s - loss: 0.0476 - accuracy: 0.9847
1261/1688 [=====================>........] - ETA: 0s - loss: 0.0483 - accuracy: 0.9846
1284/1688 [=====================>........] - ETA: 0s - loss: 0.0483 - accuracy: 0.9845
1307/1688 [======================>.......] - ETA: 0s - loss: 0.0486 - accuracy: 0.9845
1329/1688 [======================>.......] - ETA: 0s - loss: 0.0488 - accuracy: 0.9844
1350/1688 [======================>.......] - ETA: 0s - loss: 0.0488 - accuracy: 0.9844
1371/1688 [=======================>......] - ETA: 0s - loss: 0.0491 - accuracy: 0.9842
1391/1688 [=======================>......] - ETA: 0s - loss: 0.0492 - accuracy: 0.9842
1411/1688 [========================>.....] - ETA: 0s - loss: 0.0493 - accuracy: 0.9842
1431/1688 [========================>.....] - ETA: 0s - loss: 0.0493 - accuracy: 0.9841
1451/1688 [========================>.....] - ETA: 0s - loss: 0.0493 - accuracy: 0.9841
1471/1688 [=========================>....] - ETA: 0s - loss: 0.0492 - accuracy: 0.9841
1491/1688 [=========================>....] - ETA: 0s - loss: 0.0493 - accuracy: 0.9841
1511/1688 [=========================>....] - ETA: 0s - loss: 0.0494 - accuracy: 0.9841
1531/1688 [==========================>...] - ETA: 0s - loss: 0.0494 - accuracy: 0.9841
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0493 - accuracy: 0.9841
1571/1688 [==========================>...] - ETA: 0s - loss: 0.0493 - accuracy: 0.9841
1591/1688 [===========================>..] - ETA: 0s - loss: 0.0492 - accuracy: 0.9842
1611/1688 [===========================>..] - ETA: 0s - loss: 0.0496 - accuracy: 0.9842
1631/1688 [===========================>..] - ETA: 0s - loss: 0.0497 - accuracy: 0.9842
1651/1688 [============================>.] - ETA: 0s - loss: 0.0497 - accuracy: 0.9842
1671/1688 [============================>.] - ETA: 0s - loss: 0.0495 - accuracy: 0.9842
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0497 - accuracy: 0.9843 - val_loss: 0.0566 - val_accuracy: 0.9843
Epoch 4/10
1/1688 [..............................] - ETA: 4s - loss: 0.0581 - accuracy: 0.9688
21/1688 [..............................] - ETA: 4s - loss: 0.0676 - accuracy: 0.9777
41/1688 [..............................] - ETA: 4s - loss: 0.0527 - accuracy: 0.9809
61/1688 [>.............................] - ETA: 4s - loss: 0.0479 - accuracy: 0.9836
81/1688 [>.............................] - ETA: 4s - loss: 0.0462 - accuracy: 0.9838
101/1688 [>.............................] - ETA: 4s - loss: 0.0412 - accuracy: 0.9858
121/1688 [=>............................] - ETA: 4s - loss: 0.0410 - accuracy: 0.9861
141/1688 [=>............................] - ETA: 3s - loss: 0.0379 - accuracy: 0.9874
161/1688 [=>............................] - ETA: 3s - loss: 0.0370 - accuracy: 0.9872
181/1688 [==>...........................] - ETA: 3s - loss: 0.0372 - accuracy: 0.9872
201/1688 [==>...........................] - ETA: 3s - loss: 0.0350 - accuracy: 0.9883
221/1688 [==>...........................] - ETA: 3s - loss: 0.0346 - accuracy: 0.9883
241/1688 [===>..........................] - ETA: 3s - loss: 0.0344 - accuracy: 0.9886
261/1688 [===>..........................] - ETA: 3s - loss: 0.0345 - accuracy: 0.9887
281/1688 [===>..........................] - ETA: 3s - loss: 0.0346 - accuracy: 0.9887
301/1688 [====>.........................] - ETA: 3s - loss: 0.0346 - accuracy: 0.9887
321/1688 [====>.........................] - ETA: 3s - loss: 0.0350 - accuracy: 0.9886
340/1688 [=====>........................] - ETA: 3s - loss: 0.0362 - accuracy: 0.9881
360/1688 [=====>........................] - ETA: 3s - loss: 0.0374 - accuracy: 0.9879
380/1688 [=====>........................] - ETA: 3s - loss: 0.0377 - accuracy: 0.9876
400/1688 [======>.......................] - ETA: 3s - loss: 0.0381 - accuracy: 0.9877
420/1688 [======>.......................] - ETA: 3s - loss: 0.0381 - accuracy: 0.9874
440/1688 [======>.......................] - ETA: 3s - loss: 0.0390 - accuracy: 0.9871
460/1688 [=======>......................] - ETA: 3s - loss: 0.0398 - accuracy: 0.9870
479/1688 [=======>......................] - ETA: 3s - loss: 0.0399 - accuracy: 0.9871
499/1688 [=======>......................] - ETA: 3s - loss: 0.0412 - accuracy: 0.9868
519/1688 [========>.....................] - ETA: 3s - loss: 0.0414 - accuracy: 0.9868
539/1688 [========>.....................] - ETA: 2s - loss: 0.0412 - accuracy: 0.9868
559/1688 [========>.....................] - ETA: 2s - loss: 0.0404 - accuracy: 0.9870
579/1688 [=========>....................] - ETA: 2s - loss: 0.0398 - accuracy: 0.9873
598/1688 [=========>....................] - ETA: 2s - loss: 0.0401 - accuracy: 0.9873
618/1688 [=========>....................] - ETA: 2s - loss: 0.0402 - accuracy: 0.9871
638/1688 [==========>...................] - ETA: 2s - loss: 0.0404 - accuracy: 0.9871
658/1688 [==========>...................] - ETA: 2s - loss: 0.0403 - accuracy: 0.9872
678/1688 [===========>..................] - ETA: 2s - loss: 0.0409 - accuracy: 0.9871
698/1688 [===========>..................] - ETA: 2s - loss: 0.0409 - accuracy: 0.9871
718/1688 [===========>..................] - ETA: 2s - loss: 0.0411 - accuracy: 0.9870
738/1688 [============>.................] - ETA: 2s - loss: 0.0415 - accuracy: 0.9870
758/1688 [============>.................] - ETA: 2s - loss: 0.0413 - accuracy: 0.9871
778/1688 [============>.................] - ETA: 2s - loss: 0.0415 - accuracy: 0.9869
798/1688 [=============>................] - ETA: 2s - loss: 0.0415 - accuracy: 0.9872
818/1688 [=============>................] - ETA: 2s - loss: 0.0416 - accuracy: 0.9871
838/1688 [=============>................] - ETA: 2s - loss: 0.0418 - accuracy: 0.9871
858/1688 [==============>...............] - ETA: 2s - loss: 0.0419 - accuracy: 0.9870
878/1688 [==============>...............] - ETA: 2s - loss: 0.0420 - accuracy: 0.9870
898/1688 [==============>...............] - ETA: 2s - loss: 0.0422 - accuracy: 0.9869
918/1688 [===============>..............] - ETA: 1s - loss: 0.0422 - accuracy: 0.9869
938/1688 [===============>..............] - ETA: 1s - loss: 0.0419 - accuracy: 0.9870
958/1688 [================>.............] - ETA: 1s - loss: 0.0417 - accuracy: 0.9870
978/1688 [================>.............] - ETA: 1s - loss: 0.0415 - accuracy: 0.9871
998/1688 [================>.............] - ETA: 1s - loss: 0.0414 - accuracy: 0.9872
1018/1688 [=================>............] - ETA: 1s - loss: 0.0412 - accuracy: 0.9872
1038/1688 [=================>............] - ETA: 1s - loss: 0.0417 - accuracy: 0.9870
1058/1688 [=================>............] - ETA: 1s - loss: 0.0416 - accuracy: 0.9870
1078/1688 [==================>...........] - ETA: 1s - loss: 0.0414 - accuracy: 0.9870
1098/1688 [==================>...........] - ETA: 1s - loss: 0.0412 - accuracy: 0.9869
1118/1688 [==================>...........] - ETA: 1s - loss: 0.0415 - accuracy: 0.9868
1138/1688 [===================>..........] - ETA: 1s - loss: 0.0412 - accuracy: 0.9868
1158/1688 [===================>..........] - ETA: 1s - loss: 0.0410 - accuracy: 0.9869
1178/1688 [===================>..........] - ETA: 1s - loss: 0.0410 - accuracy: 0.9869
1198/1688 [====================>.........] - ETA: 1s - loss: 0.0408 - accuracy: 0.9869
1218/1688 [====================>.........] - ETA: 1s - loss: 0.0411 - accuracy: 0.9869
1238/1688 [=====================>........] - ETA: 1s - loss: 0.0410 - accuracy: 0.9869
1257/1688 [=====================>........] - ETA: 1s - loss: 0.0410 - accuracy: 0.9869
1277/1688 [=====================>........] - ETA: 1s - loss: 0.0410 - accuracy: 0.9870
1297/1688 [======================>.......] - ETA: 1s - loss: 0.0408 - accuracy: 0.9871
1317/1688 [======================>.......] - ETA: 0s - loss: 0.0407 - accuracy: 0.9872
1337/1688 [======================>.......] - ETA: 0s - loss: 0.0409 - accuracy: 0.9871
1357/1688 [=======================>......] - ETA: 0s - loss: 0.0412 - accuracy: 0.9870
1377/1688 [=======================>......] - ETA: 0s - loss: 0.0412 - accuracy: 0.9870
1397/1688 [=======================>......] - ETA: 0s - loss: 0.0411 - accuracy: 0.9870
1417/1688 [========================>.....] - ETA: 0s - loss: 0.0411 - accuracy: 0.9870
1437/1688 [========================>.....] - ETA: 0s - loss: 0.0411 - accuracy: 0.9870
1457/1688 [========================>.....] - ETA: 0s - loss: 0.0411 - accuracy: 0.9870
1476/1688 [=========================>....] - ETA: 0s - loss: 0.0410 - accuracy: 0.9870
1496/1688 [=========================>....] - ETA: 0s - loss: 0.0408 - accuracy: 0.9870
1516/1688 [=========================>....] - ETA: 0s - loss: 0.0407 - accuracy: 0.9870
1536/1688 [==========================>...] - ETA: 0s - loss: 0.0408 - accuracy: 0.9870
1556/1688 [==========================>...] - ETA: 0s - loss: 0.0409 - accuracy: 0.9869
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0409 - accuracy: 0.9870
1596/1688 [===========================>..] - ETA: 0s - loss: 0.0409 - accuracy: 0.9870
1616/1688 [===========================>..] - ETA: 0s - loss: 0.0409 - accuracy: 0.9870
1636/1688 [============================>.] - ETA: 0s - loss: 0.0408 - accuracy: 0.9870
1656/1688 [============================>.] - ETA: 0s - loss: 0.0406 - accuracy: 0.9871
1676/1688 [============================>.] - ETA: 0s - loss: 0.0405 - accuracy: 0.9871
1688/1688 [==============================] - 5s 3ms/step - loss: 0.0404 - accuracy: 0.9871 - val_loss: 0.0679 - val_accuracy: 0.9837
Epoch 5/10
1/1688 [..............................] - ETA: 4s - loss: 0.0115 - accuracy: 1.0000
21/1688 [..............................] - ETA: 4s - loss: 0.0137 - accuracy: 0.9970
41/1688 [..............................] - ETA: 4s - loss: 0.0222 - accuracy: 0.9939
61/1688 [>.............................] - ETA: 4s - loss: 0.0284 - accuracy: 0.9918
81/1688 [>.............................] - ETA: 4s - loss: 0.0266 - accuracy: 0.9927
101/1688 [>.............................] - ETA: 4s - loss: 0.0250 - accuracy: 0.9926
121/1688 [=>............................] - ETA: 4s - loss: 0.0256 - accuracy: 0.9925
141/1688 [=>............................] - ETA: 3s - loss: 0.0258 - accuracy: 0.9929
161/1688 [=>............................] - ETA: 3s - loss: 0.0249 - accuracy: 0.9932
181/1688 [==>...........................] - ETA: 3s - loss: 0.0251 - accuracy: 0.9933
201/1688 [==>...........................] - ETA: 3s - loss: 0.0268 - accuracy: 0.9927
220/1688 [==>...........................] - ETA: 3s - loss: 0.0276 - accuracy: 0.9920
240/1688 [===>..........................] - ETA: 3s - loss: 0.0268 - accuracy: 0.9923
260/1688 [===>..........................] - ETA: 3s - loss: 0.0274 - accuracy: 0.9919
280/1688 [===>..........................] - ETA: 3s - loss: 0.0282 - accuracy: 0.9917
300/1688 [====>.........................] - ETA: 3s - loss: 0.0279 - accuracy: 0.9915
320/1688 [====>.........................] - ETA: 3s - loss: 0.0285 - accuracy: 0.9915
340/1688 [=====>........................] - ETA: 3s - loss: 0.0282 - accuracy: 0.9915
360/1688 [=====>........................] - ETA: 3s - loss: 0.0276 - accuracy: 0.9915
380/1688 [=====>........................] - ETA: 3s - loss: 0.0280 - accuracy: 0.9913
400/1688 [======>.......................] - ETA: 3s - loss: 0.0279 - accuracy: 0.9914
420/1688 [======>.......................] - ETA: 3s - loss: 0.0286 - accuracy: 0.9911
440/1688 [======>.......................] - ETA: 3s - loss: 0.0287 - accuracy: 0.9911
460/1688 [=======>......................] - ETA: 3s - loss: 0.0296 - accuracy: 0.9905
479/1688 [=======>......................] - ETA: 3s - loss: 0.0298 - accuracy: 0.9905
499/1688 [=======>......................] - ETA: 3s - loss: 0.0296 - accuracy: 0.9906
519/1688 [========>.....................] - ETA: 3s - loss: 0.0294 - accuracy: 0.9905
539/1688 [========>.....................] - ETA: 2s - loss: 0.0293 - accuracy: 0.9905
559/1688 [========>.....................] - ETA: 2s - loss: 0.0295 - accuracy: 0.9906
579/1688 [=========>....................] - ETA: 2s - loss: 0.0299 - accuracy: 0.9906
599/1688 [=========>....................] - ETA: 2s - loss: 0.0306 - accuracy: 0.9902
619/1688 [==========>...................] - ETA: 2s - loss: 0.0304 - accuracy: 0.9903
638/1688 [==========>...................] - ETA: 2s - loss: 0.0305 - accuracy: 0.9902
658/1688 [==========>...................] - ETA: 2s - loss: 0.0308 - accuracy: 0.9902
678/1688 [===========>..................] - ETA: 2s - loss: 0.0305 - accuracy: 0.9902
698/1688 [===========>..................] - ETA: 2s - loss: 0.0300 - accuracy: 0.9905
718/1688 [===========>..................] - ETA: 2s - loss: 0.0302 - accuracy: 0.9905
738/1688 [============>.................] - ETA: 2s - loss: 0.0301 - accuracy: 0.9906
758/1688 [============>.................] - ETA: 2s - loss: 0.0305 - accuracy: 0.9905
778/1688 [============>.................] - ETA: 2s - loss: 0.0302 - accuracy: 0.9906
797/1688 [=============>................] - ETA: 2s - loss: 0.0305 - accuracy: 0.9905
818/1688 [=============>................] - ETA: 2s - loss: 0.0305 - accuracy: 0.9904
838/1688 [=============>................] - ETA: 2s - loss: 0.0305 - accuracy: 0.9903
858/1688 [==============>...............] - ETA: 2s - loss: 0.0304 - accuracy: 0.9902
877/1688 [==============>...............] - ETA: 2s - loss: 0.0306 - accuracy: 0.9902
897/1688 [==============>...............] - ETA: 2s - loss: 0.0304 - accuracy: 0.9902
917/1688 [===============>..............] - ETA: 1s - loss: 0.0307 - accuracy: 0.9901
937/1688 [===============>..............] - ETA: 1s - loss: 0.0307 - accuracy: 0.9901
957/1688 [================>.............] - ETA: 1s - loss: 0.0307 - accuracy: 0.9901
977/1688 [================>.............] - ETA: 1s - loss: 0.0306 - accuracy: 0.9902
997/1688 [================>.............] - ETA: 1s - loss: 0.0305 - accuracy: 0.9903
1017/1688 [=================>............] - ETA: 1s - loss: 0.0308 - accuracy: 0.9902
1037/1688 [=================>............] - ETA: 1s - loss: 0.0312 - accuracy: 0.9900
1057/1688 [=================>............] - ETA: 1s - loss: 0.0310 - accuracy: 0.9901
1077/1688 [==================>...........] - ETA: 1s - loss: 0.0313 - accuracy: 0.9900
1097/1688 [==================>...........] - ETA: 1s - loss: 0.0317 - accuracy: 0.9899
1117/1688 [==================>...........] - ETA: 1s - loss: 0.0317 - accuracy: 0.9900
1137/1688 [===================>..........] - ETA: 1s - loss: 0.0320 - accuracy: 0.9899
1157/1688 [===================>..........] - ETA: 1s - loss: 0.0320 - accuracy: 0.9898
1177/1688 [===================>..........] - ETA: 1s - loss: 0.0328 - accuracy: 0.9896
1197/1688 [====================>.........] - ETA: 1s - loss: 0.0327 - accuracy: 0.9896
1217/1688 [====================>.........] - ETA: 1s - loss: 0.0330 - accuracy: 0.9894
1237/1688 [====================>.........] - ETA: 1s - loss: 0.0330 - accuracy: 0.9894
1257/1688 [=====================>........] - ETA: 1s - loss: 0.0332 - accuracy: 0.9893
1276/1688 [=====================>........] - ETA: 1s - loss: 0.0333 - accuracy: 0.9893
1296/1688 [======================>.......] - ETA: 1s - loss: 0.0332 - accuracy: 0.9894
1316/1688 [======================>.......] - ETA: 0s - loss: 0.0334 - accuracy: 0.9894
1336/1688 [======================>.......] - ETA: 0s - loss: 0.0334 - accuracy: 0.9894
1356/1688 [=======================>......] - ETA: 0s - loss: 0.0334 - accuracy: 0.9894
1376/1688 [=======================>......] - ETA: 0s - loss: 0.0334 - accuracy: 0.9893
1395/1688 [=======================>......] - ETA: 0s - loss: 0.0333 - accuracy: 0.9894
1415/1688 [========================>.....] - ETA: 0s - loss: 0.0337 - accuracy: 0.9894
1435/1688 [========================>.....] - ETA: 0s - loss: 0.0339 - accuracy: 0.9893
1455/1688 [========================>.....] - ETA: 0s - loss: 0.0341 - accuracy: 0.9892
1475/1688 [=========================>....] - ETA: 0s - loss: 0.0343 - accuracy: 0.9891
1495/1688 [=========================>....] - ETA: 0s - loss: 0.0342 - accuracy: 0.9892
1515/1688 [=========================>....] - ETA: 0s - loss: 0.0340 - accuracy: 0.9892
1535/1688 [==========================>...] - ETA: 0s - loss: 0.0340 - accuracy: 0.9892
1555/1688 [==========================>...] - ETA: 0s - loss: 0.0341 - accuracy: 0.9891
1575/1688 [==========================>...] - ETA: 0s - loss: 0.0342 - accuracy: 0.9891
1595/1688 [===========================>..] - ETA: 0s - loss: 0.0343 - accuracy: 0.9890
1615/1688 [===========================>..] - ETA: 0s - loss: 0.0344 - accuracy: 0.9890
1635/1688 [============================>.] - ETA: 0s - loss: 0.0345 - accuracy: 0.9890
1653/1688 [============================>.] - ETA: 0s - loss: 0.0345 - accuracy: 0.9889
1672/1688 [============================>.] - ETA: 0s - loss: 0.0342 - accuracy: 0.9890
1688/1688 [==============================] - 5s 3ms/step - loss: 0.0342 - accuracy: 0.9890 - val_loss: 0.0516 - val_accuracy: 0.9878
Epoch 6/10
1/1688 [..............................] - ETA: 3s - loss: 0.0205 - accuracy: 1.0000
25/1688 [..............................] - ETA: 3s - loss: 0.0105 - accuracy: 0.9962
48/1688 [..............................] - ETA: 3s - loss: 0.0171 - accuracy: 0.9948
71/1688 [>.............................] - ETA: 3s - loss: 0.0193 - accuracy: 0.9952
93/1688 [>.............................] - ETA: 3s - loss: 0.0182 - accuracy: 0.9960
116/1688 [=>............................] - ETA: 3s - loss: 0.0177 - accuracy: 0.9960
139/1688 [=>............................] - ETA: 3s - loss: 0.0177 - accuracy: 0.9957
161/1688 [=>............................] - ETA: 3s - loss: 0.0171 - accuracy: 0.9961
182/1688 [==>...........................] - ETA: 3s - loss: 0.0175 - accuracy: 0.9954
203/1688 [==>...........................] - ETA: 3s - loss: 0.0172 - accuracy: 0.9949
224/1688 [==>...........................] - ETA: 3s - loss: 0.0176 - accuracy: 0.9943
246/1688 [===>..........................] - ETA: 3s - loss: 0.0179 - accuracy: 0.9944
267/1688 [===>..........................] - ETA: 3s - loss: 0.0180 - accuracy: 0.9943
288/1688 [====>.........................] - ETA: 3s - loss: 0.0179 - accuracy: 0.9941
310/1688 [====>.........................] - ETA: 3s - loss: 0.0181 - accuracy: 0.9940
331/1688 [====>.........................] - ETA: 3s - loss: 0.0184 - accuracy: 0.9940
352/1688 [=====>........................] - ETA: 3s - loss: 0.0181 - accuracy: 0.9941
374/1688 [=====>........................] - ETA: 3s - loss: 0.0191 - accuracy: 0.9939
395/1688 [======>.......................] - ETA: 3s - loss: 0.0192 - accuracy: 0.9936
416/1688 [======>.......................] - ETA: 2s - loss: 0.0197 - accuracy: 0.9935
438/1688 [======>.......................] - ETA: 2s - loss: 0.0197 - accuracy: 0.9934
459/1688 [=======>......................] - ETA: 2s - loss: 0.0194 - accuracy: 0.9935
479/1688 [=======>......................] - ETA: 2s - loss: 0.0196 - accuracy: 0.9934
499/1688 [=======>......................] - ETA: 2s - loss: 0.0205 - accuracy: 0.9934
519/1688 [========>.....................] - ETA: 2s - loss: 0.0211 - accuracy: 0.9931
538/1688 [========>.....................] - ETA: 2s - loss: 0.0215 - accuracy: 0.9931
558/1688 [========>.....................] - ETA: 2s - loss: 0.0216 - accuracy: 0.9931
578/1688 [=========>....................] - ETA: 2s - loss: 0.0217 - accuracy: 0.9930
598/1688 [=========>....................] - ETA: 2s - loss: 0.0220 - accuracy: 0.9928
618/1688 [=========>....................] - ETA: 2s - loss: 0.0224 - accuracy: 0.9928
638/1688 [==========>...................] - ETA: 2s - loss: 0.0224 - accuracy: 0.9928
658/1688 [==========>...................] - ETA: 2s - loss: 0.0227 - accuracy: 0.9928
678/1688 [===========>..................] - ETA: 2s - loss: 0.0231 - accuracy: 0.9926
698/1688 [===========>..................] - ETA: 2s - loss: 0.0232 - accuracy: 0.9926
718/1688 [===========>..................] - ETA: 2s - loss: 0.0229 - accuracy: 0.9926
738/1688 [============>.................] - ETA: 2s - loss: 0.0228 - accuracy: 0.9927
758/1688 [============>.................] - ETA: 2s - loss: 0.0227 - accuracy: 0.9927
778/1688 [============>.................] - ETA: 2s - loss: 0.0231 - accuracy: 0.9925
798/1688 [=============>................] - ETA: 2s - loss: 0.0234 - accuracy: 0.9924
818/1688 [=============>................] - ETA: 2s - loss: 0.0235 - accuracy: 0.9924
838/1688 [=============>................] - ETA: 2s - loss: 0.0237 - accuracy: 0.9923
858/1688 [==============>...............] - ETA: 2s - loss: 0.0241 - accuracy: 0.9922
878/1688 [==============>...............] - ETA: 1s - loss: 0.0246 - accuracy: 0.9920
898/1688 [==============>...............] - ETA: 1s - loss: 0.0248 - accuracy: 0.9920
918/1688 [===============>..............] - ETA: 1s - loss: 0.0246 - accuracy: 0.9920
938/1688 [===============>..............] - ETA: 1s - loss: 0.0248 - accuracy: 0.9918
958/1688 [================>.............] - ETA: 1s - loss: 0.0250 - accuracy: 0.9917
978/1688 [================>.............] - ETA: 1s - loss: 0.0255 - accuracy: 0.9915
998/1688 [================>.............] - ETA: 1s - loss: 0.0253 - accuracy: 0.9916
1018/1688 [=================>............] - ETA: 1s - loss: 0.0256 - accuracy: 0.9915
1038/1688 [=================>............] - ETA: 1s - loss: 0.0254 - accuracy: 0.9915
1058/1688 [=================>............] - ETA: 1s - loss: 0.0254 - accuracy: 0.9914
1077/1688 [==================>...........] - ETA: 1s - loss: 0.0258 - accuracy: 0.9914
1097/1688 [==================>...........] - ETA: 1s - loss: 0.0266 - accuracy: 0.9912
1117/1688 [==================>...........] - ETA: 1s - loss: 0.0265 - accuracy: 0.9912
1137/1688 [===================>..........] - ETA: 1s - loss: 0.0265 - accuracy: 0.9912
1157/1688 [===================>..........] - ETA: 1s - loss: 0.0263 - accuracy: 0.9912
1177/1688 [===================>..........] - ETA: 1s - loss: 0.0263 - accuracy: 0.9913
1197/1688 [====================>.........] - ETA: 1s - loss: 0.0265 - accuracy: 0.9911
1217/1688 [====================>.........] - ETA: 1s - loss: 0.0266 - accuracy: 0.9911
1237/1688 [====================>.........] - ETA: 1s - loss: 0.0265 - accuracy: 0.9911
1257/1688 [=====================>........] - ETA: 1s - loss: 0.0271 - accuracy: 0.9909
1277/1688 [=====================>........] - ETA: 1s - loss: 0.0276 - accuracy: 0.9907
1297/1688 [======================>.......] - ETA: 0s - loss: 0.0277 - accuracy: 0.9907
1317/1688 [======================>.......] - ETA: 0s - loss: 0.0278 - accuracy: 0.9907
1337/1688 [======================>.......] - ETA: 0s - loss: 0.0277 - accuracy: 0.9906
1357/1688 [=======================>......] - ETA: 0s - loss: 0.0278 - accuracy: 0.9906
1377/1688 [=======================>......] - ETA: 0s - loss: 0.0282 - accuracy: 0.9905
1397/1688 [=======================>......] - ETA: 0s - loss: 0.0285 - accuracy: 0.9905
1417/1688 [========================>.....] - ETA: 0s - loss: 0.0283 - accuracy: 0.9905
1436/1688 [========================>.....] - ETA: 0s - loss: 0.0282 - accuracy: 0.9906
1456/1688 [========================>.....] - ETA: 0s - loss: 0.0282 - accuracy: 0.9905
1476/1688 [=========================>....] - ETA: 0s - loss: 0.0284 - accuracy: 0.9904
1496/1688 [=========================>....] - ETA: 0s - loss: 0.0283 - accuracy: 0.9905
1516/1688 [=========================>....] - ETA: 0s - loss: 0.0282 - accuracy: 0.9905
1536/1688 [==========================>...] - ETA: 0s - loss: 0.0280 - accuracy: 0.9906
1556/1688 [==========================>...] - ETA: 0s - loss: 0.0280 - accuracy: 0.9906
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0282 - accuracy: 0.9906
1596/1688 [===========================>..] - ETA: 0s - loss: 0.0282 - accuracy: 0.9906
1616/1688 [===========================>..] - ETA: 0s - loss: 0.0282 - accuracy: 0.9905
1635/1688 [============================>.] - ETA: 0s - loss: 0.0285 - accuracy: 0.9905
1655/1688 [============================>.] - ETA: 0s - loss: 0.0285 - accuracy: 0.9905
1675/1688 [============================>.] - ETA: 0s - loss: 0.0284 - accuracy: 0.9905
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0285 - accuracy: 0.9905 - val_loss: 0.0576 - val_accuracy: 0.9845
Epoch 7/10
1/1688 [..............................] - ETA: 3s - loss: 0.0042 - accuracy: 1.0000
25/1688 [..............................] - ETA: 3s - loss: 0.0195 - accuracy: 0.9925
48/1688 [..............................] - ETA: 3s - loss: 0.0172 - accuracy: 0.9935
71/1688 [>.............................] - ETA: 3s - loss: 0.0195 - accuracy: 0.9921
94/1688 [>.............................] - ETA: 3s - loss: 0.0210 - accuracy: 0.9924
117/1688 [=>............................] - ETA: 3s - loss: 0.0222 - accuracy: 0.9920
139/1688 [=>............................] - ETA: 3s - loss: 0.0211 - accuracy: 0.9921
162/1688 [=>............................] - ETA: 3s - loss: 0.0208 - accuracy: 0.9925
185/1688 [==>...........................] - ETA: 3s - loss: 0.0211 - accuracy: 0.9917
208/1688 [==>...........................] - ETA: 3s - loss: 0.0202 - accuracy: 0.9920
231/1688 [===>..........................] - ETA: 3s - loss: 0.0205 - accuracy: 0.9924
253/1688 [===>..........................] - ETA: 3s - loss: 0.0201 - accuracy: 0.9927
275/1688 [===>..........................] - ETA: 3s - loss: 0.0203 - accuracy: 0.9927
297/1688 [====>.........................] - ETA: 3s - loss: 0.0215 - accuracy: 0.9922
319/1688 [====>.........................] - ETA: 3s - loss: 0.0212 - accuracy: 0.9923
342/1688 [=====>........................] - ETA: 3s - loss: 0.0205 - accuracy: 0.9926
365/1688 [=====>........................] - ETA: 2s - loss: 0.0207 - accuracy: 0.9926
388/1688 [=====>........................] - ETA: 2s - loss: 0.0206 - accuracy: 0.9925
410/1688 [======>.......................] - ETA: 2s - loss: 0.0206 - accuracy: 0.9925
433/1688 [======>.......................] - ETA: 2s - loss: 0.0215 - accuracy: 0.9924
456/1688 [=======>......................] - ETA: 2s - loss: 0.0213 - accuracy: 0.9925
479/1688 [=======>......................] - ETA: 2s - loss: 0.0212 - accuracy: 0.9925
502/1688 [=======>......................] - ETA: 2s - loss: 0.0217 - accuracy: 0.9923
524/1688 [========>.....................] - ETA: 2s - loss: 0.0219 - accuracy: 0.9923
547/1688 [========>.....................] - ETA: 2s - loss: 0.0221 - accuracy: 0.9923
570/1688 [=========>....................] - ETA: 2s - loss: 0.0225 - accuracy: 0.9923
593/1688 [=========>....................] - ETA: 2s - loss: 0.0226 - accuracy: 0.9924
616/1688 [=========>....................] - ETA: 2s - loss: 0.0228 - accuracy: 0.9923
639/1688 [==========>...................] - ETA: 2s - loss: 0.0228 - accuracy: 0.9924
662/1688 [==========>...................] - ETA: 2s - loss: 0.0232 - accuracy: 0.9924
685/1688 [===========>..................] - ETA: 2s - loss: 0.0229 - accuracy: 0.9924
708/1688 [===========>..................] - ETA: 2s - loss: 0.0230 - accuracy: 0.9924
731/1688 [===========>..................] - ETA: 2s - loss: 0.0231 - accuracy: 0.9923
754/1688 [============>.................] - ETA: 2s - loss: 0.0232 - accuracy: 0.9924
776/1688 [============>.................] - ETA: 2s - loss: 0.0229 - accuracy: 0.9925
799/1688 [=============>................] - ETA: 1s - loss: 0.0231 - accuracy: 0.9925
822/1688 [=============>................] - ETA: 1s - loss: 0.0229 - accuracy: 0.9925
845/1688 [==============>...............] - ETA: 1s - loss: 0.0229 - accuracy: 0.9926
868/1688 [==============>...............] - ETA: 1s - loss: 0.0227 - accuracy: 0.9927
891/1688 [==============>...............] - ETA: 1s - loss: 0.0227 - accuracy: 0.9926
914/1688 [===============>..............] - ETA: 1s - loss: 0.0228 - accuracy: 0.9925
936/1688 [===============>..............] - ETA: 1s - loss: 0.0230 - accuracy: 0.9925
957/1688 [================>.............] - ETA: 1s - loss: 0.0230 - accuracy: 0.9925
978/1688 [================>.............] - ETA: 1s - loss: 0.0229 - accuracy: 0.9926
999/1688 [================>.............] - ETA: 1s - loss: 0.0232 - accuracy: 0.9925
1020/1688 [=================>............] - ETA: 1s - loss: 0.0232 - accuracy: 0.9925
1041/1688 [=================>............] - ETA: 1s - loss: 0.0231 - accuracy: 0.9924
1062/1688 [=================>............] - ETA: 1s - loss: 0.0233 - accuracy: 0.9925
1083/1688 [==================>...........] - ETA: 1s - loss: 0.0232 - accuracy: 0.9925
1105/1688 [==================>...........] - ETA: 1s - loss: 0.0233 - accuracy: 0.9924
1127/1688 [===================>..........] - ETA: 1s - loss: 0.0234 - accuracy: 0.9924
1148/1688 [===================>..........] - ETA: 1s - loss: 0.0233 - accuracy: 0.9924
1169/1688 [===================>..........] - ETA: 1s - loss: 0.0237 - accuracy: 0.9923
1189/1688 [====================>.........] - ETA: 1s - loss: 0.0240 - accuracy: 0.9922
1208/1688 [====================>.........] - ETA: 1s - loss: 0.0242 - accuracy: 0.9922
1228/1688 [====================>.........] - ETA: 1s - loss: 0.0242 - accuracy: 0.9921
1248/1688 [=====================>........] - ETA: 1s - loss: 0.0241 - accuracy: 0.9922
1268/1688 [=====================>........] - ETA: 0s - loss: 0.0242 - accuracy: 0.9921
1288/1688 [=====================>........] - ETA: 0s - loss: 0.0247 - accuracy: 0.9920
1308/1688 [======================>.......] - ETA: 0s - loss: 0.0246 - accuracy: 0.9921
1328/1688 [======================>.......] - ETA: 0s - loss: 0.0246 - accuracy: 0.9920
1348/1688 [======================>.......] - ETA: 0s - loss: 0.0245 - accuracy: 0.9920
1368/1688 [=======================>......] - ETA: 0s - loss: 0.0243 - accuracy: 0.9921
1388/1688 [=======================>......] - ETA: 0s - loss: 0.0245 - accuracy: 0.9920
1408/1688 [========================>.....] - ETA: 0s - loss: 0.0246 - accuracy: 0.9920
1428/1688 [========================>.....] - ETA: 0s - loss: 0.0247 - accuracy: 0.9919
1448/1688 [========================>.....] - ETA: 0s - loss: 0.0249 - accuracy: 0.9918
1468/1688 [=========================>....] - ETA: 0s - loss: 0.0249 - accuracy: 0.9918
1488/1688 [=========================>....] - ETA: 0s - loss: 0.0250 - accuracy: 0.9918
1508/1688 [=========================>....] - ETA: 0s - loss: 0.0255 - accuracy: 0.9916
1528/1688 [==========================>...] - ETA: 0s - loss: 0.0255 - accuracy: 0.9916
1548/1688 [==========================>...] - ETA: 0s - loss: 0.0255 - accuracy: 0.9915
1568/1688 [==========================>...] - ETA: 0s - loss: 0.0254 - accuracy: 0.9915
1588/1688 [===========================>..] - ETA: 0s - loss: 0.0255 - accuracy: 0.9915
1608/1688 [===========================>..] - ETA: 0s - loss: 0.0255 - accuracy: 0.9915
1628/1688 [===========================>..] - ETA: 0s - loss: 0.0256 - accuracy: 0.9914
1648/1688 [============================>.] - ETA: 0s - loss: 0.0256 - accuracy: 0.9914
1668/1688 [============================>.] - ETA: 0s - loss: 0.0255 - accuracy: 0.9915
1688/1688 [==============================] - ETA: 0s - loss: 0.0255 - accuracy: 0.9914
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0255 - accuracy: 0.9914 - val_loss: 0.0577 - val_accuracy: 0.9852
Epoch 8/10
1/1688 [..............................] - ETA: 3s - loss: 0.0172 - accuracy: 1.0000
25/1688 [..............................] - ETA: 3s - loss: 0.0157 - accuracy: 0.9962
48/1688 [..............................] - ETA: 3s - loss: 0.0134 - accuracy: 0.9954
70/1688 [>.............................] - ETA: 3s - loss: 0.0152 - accuracy: 0.9946
91/1688 [>.............................] - ETA: 3s - loss: 0.0202 - accuracy: 0.9935
113/1688 [=>............................] - ETA: 3s - loss: 0.0202 - accuracy: 0.9934
134/1688 [=>............................] - ETA: 3s - loss: 0.0206 - accuracy: 0.9935
155/1688 [=>............................] - ETA: 3s - loss: 0.0202 - accuracy: 0.9935
176/1688 [==>...........................] - ETA: 3s - loss: 0.0197 - accuracy: 0.9936
197/1688 [==>...........................] - ETA: 3s - loss: 0.0194 - accuracy: 0.9935
218/1688 [==>...........................] - ETA: 3s - loss: 0.0187 - accuracy: 0.9940
239/1688 [===>..........................] - ETA: 3s - loss: 0.0188 - accuracy: 0.9939
260/1688 [===>..........................] - ETA: 3s - loss: 0.0186 - accuracy: 0.9939
281/1688 [===>..........................] - ETA: 3s - loss: 0.0189 - accuracy: 0.9937
302/1688 [====>.........................] - ETA: 3s - loss: 0.0191 - accuracy: 0.9938
323/1688 [====>.........................] - ETA: 3s - loss: 0.0190 - accuracy: 0.9935
344/1688 [=====>........................] - ETA: 3s - loss: 0.0199 - accuracy: 0.9935
365/1688 [=====>........................] - ETA: 3s - loss: 0.0195 - accuracy: 0.9937
387/1688 [=====>........................] - ETA: 3s - loss: 0.0193 - accuracy: 0.9936
409/1688 [======>.......................] - ETA: 3s - loss: 0.0188 - accuracy: 0.9937
430/1688 [======>.......................] - ETA: 2s - loss: 0.0185 - accuracy: 0.9939
452/1688 [=======>......................] - ETA: 2s - loss: 0.0184 - accuracy: 0.9940
474/1688 [=======>......................] - ETA: 2s - loss: 0.0183 - accuracy: 0.9939
496/1688 [=======>......................] - ETA: 2s - loss: 0.0184 - accuracy: 0.9938
517/1688 [========>.....................] - ETA: 2s - loss: 0.0183 - accuracy: 0.9940
539/1688 [========>.....................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9941
560/1688 [========>.....................] - ETA: 2s - loss: 0.0176 - accuracy: 0.9941
582/1688 [=========>....................] - ETA: 2s - loss: 0.0174 - accuracy: 0.9941
603/1688 [=========>....................] - ETA: 2s - loss: 0.0175 - accuracy: 0.9941
624/1688 [==========>...................] - ETA: 2s - loss: 0.0178 - accuracy: 0.9939
646/1688 [==========>...................] - ETA: 2s - loss: 0.0184 - accuracy: 0.9937
667/1688 [==========>...................] - ETA: 2s - loss: 0.0182 - accuracy: 0.9939
689/1688 [===========>..................] - ETA: 2s - loss: 0.0185 - accuracy: 0.9938
710/1688 [===========>..................] - ETA: 2s - loss: 0.0185 - accuracy: 0.9938
731/1688 [===========>..................] - ETA: 2s - loss: 0.0184 - accuracy: 0.9937
752/1688 [============>.................] - ETA: 2s - loss: 0.0186 - accuracy: 0.9936
773/1688 [============>.................] - ETA: 2s - loss: 0.0185 - accuracy: 0.9936
795/1688 [=============>................] - ETA: 2s - loss: 0.0190 - accuracy: 0.9936
816/1688 [=============>................] - ETA: 2s - loss: 0.0189 - accuracy: 0.9935
837/1688 [=============>................] - ETA: 2s - loss: 0.0186 - accuracy: 0.9937
858/1688 [==============>...............] - ETA: 1s - loss: 0.0185 - accuracy: 0.9937
880/1688 [==============>...............] - ETA: 1s - loss: 0.0187 - accuracy: 0.9936
901/1688 [===============>..............] - ETA: 1s - loss: 0.0190 - accuracy: 0.9935
922/1688 [===============>..............] - ETA: 1s - loss: 0.0191 - accuracy: 0.9935
943/1688 [===============>..............] - ETA: 1s - loss: 0.0194 - accuracy: 0.9934
964/1688 [================>.............] - ETA: 1s - loss: 0.0197 - accuracy: 0.9932
985/1688 [================>.............] - ETA: 1s - loss: 0.0197 - accuracy: 0.9931
1006/1688 [================>.............] - ETA: 1s - loss: 0.0196 - accuracy: 0.9932
1027/1688 [=================>............] - ETA: 1s - loss: 0.0197 - accuracy: 0.9932
1048/1688 [=================>............] - ETA: 1s - loss: 0.0194 - accuracy: 0.9933
1069/1688 [=================>............] - ETA: 1s - loss: 0.0192 - accuracy: 0.9934
1090/1688 [==================>...........] - ETA: 1s - loss: 0.0192 - accuracy: 0.9934
1111/1688 [==================>...........] - ETA: 1s - loss: 0.0193 - accuracy: 0.9934
1132/1688 [===================>..........] - ETA: 1s - loss: 0.0194 - accuracy: 0.9933
1153/1688 [===================>..........] - ETA: 1s - loss: 0.0193 - accuracy: 0.9934
1174/1688 [===================>..........] - ETA: 1s - loss: 0.0194 - accuracy: 0.9933
1196/1688 [====================>.........] - ETA: 1s - loss: 0.0195 - accuracy: 0.9933
1218/1688 [====================>.........] - ETA: 1s - loss: 0.0195 - accuracy: 0.9933
1240/1688 [=====================>........] - ETA: 1s - loss: 0.0194 - accuracy: 0.9934
1261/1688 [=====================>........] - ETA: 1s - loss: 0.0194 - accuracy: 0.9934
1282/1688 [=====================>........] - ETA: 0s - loss: 0.0195 - accuracy: 0.9933
1303/1688 [======================>.......] - ETA: 0s - loss: 0.0197 - accuracy: 0.9933
1325/1688 [======================>.......] - ETA: 0s - loss: 0.0199 - accuracy: 0.9932
1347/1688 [======================>.......] - ETA: 0s - loss: 0.0199 - accuracy: 0.9932
1369/1688 [=======================>......] - ETA: 0s - loss: 0.0199 - accuracy: 0.9932
1391/1688 [=======================>......] - ETA: 0s - loss: 0.0197 - accuracy: 0.9932
1412/1688 [========================>.....] - ETA: 0s - loss: 0.0197 - accuracy: 0.9933
1434/1688 [========================>.....] - ETA: 0s - loss: 0.0196 - accuracy: 0.9933
1456/1688 [========================>.....] - ETA: 0s - loss: 0.0195 - accuracy: 0.9933
1477/1688 [=========================>....] - ETA: 0s - loss: 0.0196 - accuracy: 0.9933
1498/1688 [=========================>....] - ETA: 0s - loss: 0.0198 - accuracy: 0.9932
1519/1688 [=========================>....] - ETA: 0s - loss: 0.0202 - accuracy: 0.9930
1540/1688 [==========================>...] - ETA: 0s - loss: 0.0205 - accuracy: 0.9929
1561/1688 [==========================>...] - ETA: 0s - loss: 0.0205 - accuracy: 0.9929
1583/1688 [===========================>..] - ETA: 0s - loss: 0.0205 - accuracy: 0.9929
1604/1688 [===========================>..] - ETA: 0s - loss: 0.0204 - accuracy: 0.9929
1626/1688 [===========================>..] - ETA: 0s - loss: 0.0203 - accuracy: 0.9930
1648/1688 [============================>.] - ETA: 0s - loss: 0.0203 - accuracy: 0.9930
1669/1688 [============================>.] - ETA: 0s - loss: 0.0203 - accuracy: 0.9930
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0202 - accuracy: 0.9930 - val_loss: 0.0523 - val_accuracy: 0.9878
Epoch 9/10
1/1688 [..............................] - ETA: 4s - loss: 1.0111e-04 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0177 - accuracy: 0.9935
46/1688 [..............................] - ETA: 3s - loss: 0.0141 - accuracy: 0.9959
69/1688 [>.............................] - ETA: 3s - loss: 0.0161 - accuracy: 0.9946
92/1688 [>.............................] - ETA: 3s - loss: 0.0136 - accuracy: 0.9956
115/1688 [=>............................] - ETA: 3s - loss: 0.0127 - accuracy: 0.9954
137/1688 [=>............................] - ETA: 3s - loss: 0.0126 - accuracy: 0.9957
160/1688 [=>............................] - ETA: 3s - loss: 0.0118 - accuracy: 0.9961
183/1688 [==>...........................] - ETA: 3s - loss: 0.0123 - accuracy: 0.9961
205/1688 [==>...........................] - ETA: 3s - loss: 0.0123 - accuracy: 0.9960
227/1688 [===>..........................] - ETA: 3s - loss: 0.0131 - accuracy: 0.9956
248/1688 [===>..........................] - ETA: 3s - loss: 0.0133 - accuracy: 0.9956
269/1688 [===>..........................] - ETA: 3s - loss: 0.0126 - accuracy: 0.9959
290/1688 [====>.........................] - ETA: 3s - loss: 0.0133 - accuracy: 0.9956
311/1688 [====>.........................] - ETA: 3s - loss: 0.0129 - accuracy: 0.9958
333/1688 [====>.........................] - ETA: 3s - loss: 0.0134 - accuracy: 0.9957
355/1688 [=====>........................] - ETA: 3s - loss: 0.0138 - accuracy: 0.9955
376/1688 [=====>........................] - ETA: 3s - loss: 0.0138 - accuracy: 0.9956
398/1688 [======>.......................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9953
420/1688 [======>.......................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9953
442/1688 [======>.......................] - ETA: 2s - loss: 0.0140 - accuracy: 0.9955
463/1688 [=======>......................] - ETA: 2s - loss: 0.0141 - accuracy: 0.9955
484/1688 [=======>......................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9952
505/1688 [=======>......................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9951
527/1688 [========>.....................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9951
548/1688 [========>.....................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9951
569/1688 [=========>....................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9951
590/1688 [=========>....................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9950
612/1688 [=========>....................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9949
634/1688 [==========>...................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9949
655/1688 [==========>...................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9948
676/1688 [===========>..................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9950
697/1688 [===========>..................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9950
718/1688 [===========>..................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9951
739/1688 [============>.................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9949
760/1688 [============>.................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9949
781/1688 [============>.................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9949
803/1688 [=============>................] - ETA: 2s - loss: 0.0162 - accuracy: 0.9949
824/1688 [=============>................] - ETA: 2s - loss: 0.0161 - accuracy: 0.9950
846/1688 [==============>...............] - ETA: 1s - loss: 0.0161 - accuracy: 0.9950
868/1688 [==============>...............] - ETA: 1s - loss: 0.0160 - accuracy: 0.9950
889/1688 [==============>...............] - ETA: 1s - loss: 0.0160 - accuracy: 0.9950
909/1688 [===============>..............] - ETA: 1s - loss: 0.0162 - accuracy: 0.9948
930/1688 [===============>..............] - ETA: 1s - loss: 0.0166 - accuracy: 0.9947
951/1688 [===============>..............] - ETA: 1s - loss: 0.0165 - accuracy: 0.9948
972/1688 [================>.............] - ETA: 1s - loss: 0.0164 - accuracy: 0.9948
994/1688 [================>.............] - ETA: 1s - loss: 0.0164 - accuracy: 0.9948
1015/1688 [=================>............] - ETA: 1s - loss: 0.0163 - accuracy: 0.9948
1036/1688 [=================>............] - ETA: 1s - loss: 0.0160 - accuracy: 0.9949
1057/1688 [=================>............] - ETA: 1s - loss: 0.0161 - accuracy: 0.9949
1078/1688 [==================>...........] - ETA: 1s - loss: 0.0163 - accuracy: 0.9948
1099/1688 [==================>...........] - ETA: 1s - loss: 0.0166 - accuracy: 0.9947
1120/1688 [==================>...........] - ETA: 1s - loss: 0.0167 - accuracy: 0.9946
1141/1688 [===================>..........] - ETA: 1s - loss: 0.0170 - accuracy: 0.9945
1162/1688 [===================>..........] - ETA: 1s - loss: 0.0172 - accuracy: 0.9945
1183/1688 [====================>.........] - ETA: 1s - loss: 0.0175 - accuracy: 0.9944
1204/1688 [====================>.........] - ETA: 1s - loss: 0.0177 - accuracy: 0.9943
1225/1688 [====================>.........] - ETA: 1s - loss: 0.0180 - accuracy: 0.9942
1247/1688 [=====================>........] - ETA: 1s - loss: 0.0180 - accuracy: 0.9942
1269/1688 [=====================>........] - ETA: 0s - loss: 0.0180 - accuracy: 0.9942
1290/1688 [=====================>........] - ETA: 0s - loss: 0.0182 - accuracy: 0.9941
1312/1688 [======================>.......] - ETA: 0s - loss: 0.0183 - accuracy: 0.9941
1333/1688 [======================>.......] - ETA: 0s - loss: 0.0183 - accuracy: 0.9941
1354/1688 [=======================>......] - ETA: 0s - loss: 0.0183 - accuracy: 0.9941
1375/1688 [=======================>......] - ETA: 0s - loss: 0.0181 - accuracy: 0.9941
1396/1688 [=======================>......] - ETA: 0s - loss: 0.0183 - accuracy: 0.9940
1418/1688 [========================>.....] - ETA: 0s - loss: 0.0182 - accuracy: 0.9941
1439/1688 [========================>.....] - ETA: 0s - loss: 0.0183 - accuracy: 0.9940
1460/1688 [========================>.....] - ETA: 0s - loss: 0.0182 - accuracy: 0.9941
1481/1688 [=========================>....] - ETA: 0s - loss: 0.0185 - accuracy: 0.9941
1502/1688 [=========================>....] - ETA: 0s - loss: 0.0184 - accuracy: 0.9941
1524/1688 [==========================>...] - ETA: 0s - loss: 0.0183 - accuracy: 0.9941
1545/1688 [==========================>...] - ETA: 0s - loss: 0.0184 - accuracy: 0.9941
1566/1688 [==========================>...] - ETA: 0s - loss: 0.0183 - accuracy: 0.9941
1586/1688 [===========================>..] - ETA: 0s - loss: 0.0182 - accuracy: 0.9941
1606/1688 [===========================>..] - ETA: 0s - loss: 0.0181 - accuracy: 0.9941
1626/1688 [===========================>..] - ETA: 0s - loss: 0.0184 - accuracy: 0.9940
1646/1688 [============================>.] - ETA: 0s - loss: 0.0184 - accuracy: 0.9940
1666/1688 [============================>.] - ETA: 0s - loss: 0.0184 - accuracy: 0.9940
1686/1688 [============================>.] - ETA: 0s - loss: 0.0185 - accuracy: 0.9940
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0185 - accuracy: 0.9940 - val_loss: 0.0515 - val_accuracy: 0.9882
Epoch 10/10
1/1688 [..............................] - ETA: 4s - loss: 5.2184e-04 - accuracy: 1.0000
21/1688 [..............................] - ETA: 4s - loss: 0.0152 - accuracy: 0.9970
41/1688 [..............................] - ETA: 4s - loss: 0.0167 - accuracy: 0.9954
61/1688 [>.............................] - ETA: 4s - loss: 0.0199 - accuracy: 0.9944
81/1688 [>.............................] - ETA: 4s - loss: 0.0193 - accuracy: 0.9946
101/1688 [>.............................] - ETA: 4s - loss: 0.0184 - accuracy: 0.9947
121/1688 [=>............................] - ETA: 4s - loss: 0.0163 - accuracy: 0.9954
141/1688 [=>............................] - ETA: 3s - loss: 0.0151 - accuracy: 0.9958
161/1688 [=>............................] - ETA: 3s - loss: 0.0141 - accuracy: 0.9961
181/1688 [==>...........................] - ETA: 3s - loss: 0.0135 - accuracy: 0.9964
201/1688 [==>...........................] - ETA: 3s - loss: 0.0130 - accuracy: 0.9964
221/1688 [==>...........................] - ETA: 3s - loss: 0.0126 - accuracy: 0.9965
241/1688 [===>..........................] - ETA: 3s - loss: 0.0127 - accuracy: 0.9964
261/1688 [===>..........................] - ETA: 3s - loss: 0.0122 - accuracy: 0.9965
281/1688 [===>..........................] - ETA: 3s - loss: 0.0118 - accuracy: 0.9967
301/1688 [====>.........................] - ETA: 3s - loss: 0.0114 - accuracy: 0.9968
321/1688 [====>.........................] - ETA: 3s - loss: 0.0110 - accuracy: 0.9969
341/1688 [=====>........................] - ETA: 3s - loss: 0.0108 - accuracy: 0.9969
361/1688 [=====>........................] - ETA: 3s - loss: 0.0106 - accuracy: 0.9971
381/1688 [=====>........................] - ETA: 3s - loss: 0.0105 - accuracy: 0.9970
401/1688 [======>.......................] - ETA: 3s - loss: 0.0102 - accuracy: 0.9971
421/1688 [======>.......................] - ETA: 3s - loss: 0.0101 - accuracy: 0.9972
441/1688 [======>.......................] - ETA: 3s - loss: 0.0097 - accuracy: 0.9973
461/1688 [=======>......................] - ETA: 3s - loss: 0.0094 - accuracy: 0.9974
481/1688 [=======>......................] - ETA: 3s - loss: 0.0094 - accuracy: 0.9973
501/1688 [=======>......................] - ETA: 3s - loss: 0.0094 - accuracy: 0.9973
521/1688 [========>.....................] - ETA: 3s - loss: 0.0095 - accuracy: 0.9972
541/1688 [========>.....................] - ETA: 2s - loss: 0.0093 - accuracy: 0.9973
560/1688 [========>.....................] - ETA: 2s - loss: 0.0094 - accuracy: 0.9973
580/1688 [=========>....................] - ETA: 2s - loss: 0.0095 - accuracy: 0.9973
600/1688 [=========>....................] - ETA: 2s - loss: 0.0096 - accuracy: 0.9972
620/1688 [==========>...................] - ETA: 2s - loss: 0.0095 - accuracy: 0.9973
640/1688 [==========>...................] - ETA: 2s - loss: 0.0095 - accuracy: 0.9973
660/1688 [==========>...................] - ETA: 2s - loss: 0.0098 - accuracy: 0.9971
680/1688 [===========>..................] - ETA: 2s - loss: 0.0099 - accuracy: 0.9971
700/1688 [===========>..................] - ETA: 2s - loss: 0.0100 - accuracy: 0.9971
719/1688 [===========>..................] - ETA: 2s - loss: 0.0099 - accuracy: 0.9971
739/1688 [============>.................] - ETA: 2s - loss: 0.0100 - accuracy: 0.9971
759/1688 [============>.................] - ETA: 2s - loss: 0.0102 - accuracy: 0.9971
779/1688 [============>.................] - ETA: 2s - loss: 0.0112 - accuracy: 0.9968
799/1688 [=============>................] - ETA: 2s - loss: 0.0118 - accuracy: 0.9965
819/1688 [=============>................] - ETA: 2s - loss: 0.0123 - accuracy: 0.9963
839/1688 [=============>................] - ETA: 2s - loss: 0.0124 - accuracy: 0.9962
859/1688 [==============>...............] - ETA: 2s - loss: 0.0126 - accuracy: 0.9961
879/1688 [==============>...............] - ETA: 2s - loss: 0.0127 - accuracy: 0.9961
898/1688 [==============>...............] - ETA: 2s - loss: 0.0132 - accuracy: 0.9959
918/1688 [===============>..............] - ETA: 1s - loss: 0.0134 - accuracy: 0.9957
938/1688 [===============>..............] - ETA: 1s - loss: 0.0135 - accuracy: 0.9956
958/1688 [================>.............] - ETA: 1s - loss: 0.0138 - accuracy: 0.9956
978/1688 [================>.............] - ETA: 1s - loss: 0.0139 - accuracy: 0.9955
998/1688 [================>.............] - ETA: 1s - loss: 0.0138 - accuracy: 0.9956
1018/1688 [=================>............] - ETA: 1s - loss: 0.0142 - accuracy: 0.9955
1038/1688 [=================>............] - ETA: 1s - loss: 0.0142 - accuracy: 0.9955
1058/1688 [=================>............] - ETA: 1s - loss: 0.0141 - accuracy: 0.9955
1078/1688 [==================>...........] - ETA: 1s - loss: 0.0141 - accuracy: 0.9955
1098/1688 [==================>...........] - ETA: 1s - loss: 0.0139 - accuracy: 0.9956
1118/1688 [==================>...........] - ETA: 1s - loss: 0.0141 - accuracy: 0.9955
1138/1688 [===================>..........] - ETA: 1s - loss: 0.0142 - accuracy: 0.9955
1157/1688 [===================>..........] - ETA: 1s - loss: 0.0143 - accuracy: 0.9954
1177/1688 [===================>..........] - ETA: 1s - loss: 0.0144 - accuracy: 0.9955
1197/1688 [====================>.........] - ETA: 1s - loss: 0.0146 - accuracy: 0.9954
1217/1688 [====================>.........] - ETA: 1s - loss: 0.0145 - accuracy: 0.9954
1237/1688 [====================>.........] - ETA: 1s - loss: 0.0146 - accuracy: 0.9953
1257/1688 [=====================>........] - ETA: 1s - loss: 0.0149 - accuracy: 0.9953
1277/1688 [=====================>........] - ETA: 1s - loss: 0.0149 - accuracy: 0.9952
1297/1688 [======================>.......] - ETA: 1s - loss: 0.0153 - accuracy: 0.9951
1317/1688 [======================>.......] - ETA: 0s - loss: 0.0152 - accuracy: 0.9952
1337/1688 [======================>.......] - ETA: 0s - loss: 0.0155 - accuracy: 0.9951
1357/1688 [=======================>......] - ETA: 0s - loss: 0.0155 - accuracy: 0.9951
1377/1688 [=======================>......] - ETA: 0s - loss: 0.0156 - accuracy: 0.9951
1397/1688 [=======================>......] - ETA: 0s - loss: 0.0157 - accuracy: 0.9950
1417/1688 [========================>.....] - ETA: 0s - loss: 0.0159 - accuracy: 0.9950
1437/1688 [========================>.....] - ETA: 0s - loss: 0.0159 - accuracy: 0.9949
1456/1688 [========================>.....] - ETA: 0s - loss: 0.0159 - accuracy: 0.9949
1476/1688 [=========================>....] - ETA: 0s - loss: 0.0160 - accuracy: 0.9949
1495/1688 [=========================>....] - ETA: 0s - loss: 0.0159 - accuracy: 0.9949
1515/1688 [=========================>....] - ETA: 0s - loss: 0.0161 - accuracy: 0.9949
1535/1688 [==========================>...] - ETA: 0s - loss: 0.0161 - accuracy: 0.9949
1555/1688 [==========================>...] - ETA: 0s - loss: 0.0161 - accuracy: 0.9949
1575/1688 [==========================>...] - ETA: 0s - loss: 0.0161 - accuracy: 0.9948
1594/1688 [===========================>..] - ETA: 0s - loss: 0.0161 - accuracy: 0.9948
1614/1688 [===========================>..] - ETA: 0s - loss: 0.0161 - accuracy: 0.9948
1634/1688 [============================>.] - ETA: 0s - loss: 0.0161 - accuracy: 0.9948
1655/1688 [============================>.] - ETA: 0s - loss: 0.0161 - accuracy: 0.9948
1675/1688 [============================>.] - ETA: 0s - loss: 0.0162 - accuracy: 0.9948
1688/1688 [==============================] - 5s 3ms/step - loss: 0.0162 - accuracy: 0.9947 - val_loss: 0.0630 - val_accuracy: 0.9845
score = model_keras.evaluate(x_test, y_test, verbose=0)
print('Test accuracy:', score[1])
Test accuracy: 0.9818999767303467
2. Quantize
2.1. 8-bit quantization
An Akida accelerator processes 8 or 4-bits integer activations and weights. Therefore, the floating point Keras model must be quantized in preparation to run on an Akida accelerator.
The QuantizeML quantize function can be used to quantize a Keras model for Akida. For this step in this example, an “8/8/8” quantization scheme will be applied to the floating point Keras model to produce 8-bit weights in the first layer, 8-bit weights in all other layers, and 8-bit activations.
The quantization process results in a Keras model with custom QuantizeML quantized layers substituted for the original Keras layers.
All Keras API functions can be applied on this new model: summary(), compile(), fit(). etc.
Note
The quantize function applies several transformations to
the original model. For example, it folds the batch normalization layers into the
corresponding neural layers. The new weights are computed according to this folding
operation.
from quantizeml.models import quantize, QuantizationParams
qparams = QuantizationParams(input_weight_bits=8, weight_bits=8, activation_bits=8)
model_quantized = quantize(model_keras, qparams=qparams)
model_quantized.summary()
/usr/local/lib/python3.11/dist-packages/onnxscript/converter.py:823: FutureWarning: 'onnxscript.values.Op.param_schemas' is deprecated in version 0.1 and will be removed in the future. Please use '.op_signature' instead.
param_schemas = callee.param_schemas()
/usr/local/lib/python3.11/dist-packages/quantizeml/models/quantize.py:479: UserWarning: Quantizing per-axis with random calibration samples is not accurate. Set QuantizationParams.per_tensor_activations=True when calibrating with random samples.
warnings.warn("Quantizing per-axis with random calibration samples is not accurate.\
1/1024 [..............................] - ETA: 3:11
56/1024 [>.............................] - ETA: 0s
113/1024 [==>...........................] - ETA: 0s
171/1024 [====>.........................] - ETA: 0s
228/1024 [=====>........................] - ETA: 0s
285/1024 [=======>......................] - ETA: 0s
343/1024 [=========>....................] - ETA: 0s
401/1024 [==========>...................] - ETA: 0s
458/1024 [============>.................] - ETA: 0s
515/1024 [==============>...............] - ETA: 0s
573/1024 [===============>..............] - ETA: 0s
629/1024 [=================>............] - ETA: 0s
686/1024 [===================>..........] - ETA: 0s
743/1024 [====================>.........] - ETA: 0s
800/1024 [======================>.......] - ETA: 0s
857/1024 [========================>.....] - ETA: 0s
914/1024 [=========================>....] - ETA: 0s
972/1024 [===========================>..] - ETA: 0s
1024/1024 [==============================] - 1s 883us/step
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling_input (InputLaye [(None, 28, 28, 1)] 0
r)
rescaling (QuantizedRescal (None, 28, 28, 1) 0
ing)
conv2d (QuantizedConv2D) (None, 13, 13, 32) 320
re_lu (QuantizedReLU) (None, 13, 13, 32) 64
depthwise_conv2d (Quantize (None, 7, 7, 32) 384
dDepthwiseConv2D)
conv2d_1 (QuantizedConv2D) (None, 7, 7, 64) 2112
re_lu_1 (QuantizedReLU) (None, 7, 7, 64) 128
flatten (QuantizedFlatten) (None, 3136) 0
dense (QuantizedDense) (None, 10) 31370
dequantizer (Dequantizer) (None, 10) 0
=================================================================
Total params: 34378 (134.29 KB)
Trainable params: 34122 (133.29 KB)
Non-trainable params: 256 (1.00 KB)
_________________________________________________________________
Note
Note that the number of parameters for the floating and quantized models differs, a consequence of the BatchNormalization folding and the additional parameters added for quantization. For further details, please refer to their respective summary.
Check the quantized model accuracy.
def compile_evaluate(model):
""" Compiles and evaluates the model, then return accuracy score. """
model.compile(metrics=['accuracy'])
return model.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after 8-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 8-bit quantization: 0.9793000221252441
2.2. Effect of calibration
The previous call to quantize was made with random samples for calibration
(default parameters). While the observed drop in accuracy is minimal, that is
around 1%, it can be worse on more complex models. Therefore, it is advised to
use a set of real samples from the training set for calibration during a call
to quantize.
Note that this remains a calibration step rather than a training step in that
no output labels are required. Furthermore, any relevant data could be used for
calibration. The recommended settings for calibration that are widely used to
obtain the zoo performance are:
1024 samples
a batch size of 100
2 epochs
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 3s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 1ms/step
Check the accuracy for the quantized and calibrated model.
print('Test accuracy after calibration:', compile_evaluate(model_quantized))
Test accuracy after calibration: 0.9805999994277954
Calibrating with real samples on this model recovers the initial float accuracy.
2.3. 4-bit quantization
The accuracy of the 8/8/8 quantized model is equal to that of the Keras floating point model. In some cases, a smaller memory size for the model is required. This can be accomplished through quantization of the model to smaller bitwidths.
The model will now be quantized to 8/4/4, that is 8-bit weights in the first layer with 4-bit weights and activations in all other layers. Such a quantization scheme will usually introduce a performance drop.
qparams = QuantizationParams(input_weight_bits=8, weight_bits=4, activation_bits=4)
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 1ms/step
Check the 4-bit quantized accuracy.
print('Test accuracy after 4-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 4-bit quantization: 0.977400004863739
2.4. Model fine tuning (Quantization Aware Training)
When a model suffers from an accuracy drop after quantization, fine tuning or Quantization Aware Training (QAT) may recover some or all of the original performance.
Note that since this is a fine tuning step, both the number of epochs and learning rate are expected to be lower than during the initial float training.
model_quantized.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-4),
metrics=['accuracy'])
model_quantized.fit(x_train, y_train, epochs=5, validation_split=0.1)
Epoch 1/5
1/1688 [..............................] - ETA: 2:05:28 - loss: 0.0344 - accuracy: 1.0000
11/1688 [..............................] - ETA: 8s - loss: 0.0442 - accuracy: 0.9858
21/1688 [..............................] - ETA: 8s - loss: 0.0623 - accuracy: 0.9792
31/1688 [..............................] - ETA: 8s - loss: 0.0496 - accuracy: 0.9829
42/1688 [..............................] - ETA: 8s - loss: 0.0460 - accuracy: 0.9829
53/1688 [..............................] - ETA: 8s - loss: 0.0414 - accuracy: 0.9858
63/1688 [>.............................] - ETA: 8s - loss: 0.0438 - accuracy: 0.9846
73/1688 [>.............................] - ETA: 8s - loss: 0.0455 - accuracy: 0.9837
83/1688 [>.............................] - ETA: 8s - loss: 0.0414 - accuracy: 0.9849
93/1688 [>.............................] - ETA: 8s - loss: 0.0398 - accuracy: 0.9856
104/1688 [>.............................] - ETA: 7s - loss: 0.0381 - accuracy: 0.9865
114/1688 [=>............................] - ETA: 7s - loss: 0.0376 - accuracy: 0.9868
124/1688 [=>............................] - ETA: 7s - loss: 0.0361 - accuracy: 0.9877
134/1688 [=>............................] - ETA: 7s - loss: 0.0371 - accuracy: 0.9876
145/1688 [=>............................] - ETA: 7s - loss: 0.0355 - accuracy: 0.9879
155/1688 [=>............................] - ETA: 7s - loss: 0.0340 - accuracy: 0.9885
165/1688 [=>............................] - ETA: 7s - loss: 0.0333 - accuracy: 0.9888
176/1688 [==>...........................] - ETA: 7s - loss: 0.0328 - accuracy: 0.9888
187/1688 [==>...........................] - ETA: 7s - loss: 0.0329 - accuracy: 0.9886
197/1688 [==>...........................] - ETA: 7s - loss: 0.0326 - accuracy: 0.9884
208/1688 [==>...........................] - ETA: 7s - loss: 0.0316 - accuracy: 0.9887
219/1688 [==>...........................] - ETA: 7s - loss: 0.0314 - accuracy: 0.9890
229/1688 [===>..........................] - ETA: 7s - loss: 0.0314 - accuracy: 0.9889
239/1688 [===>..........................] - ETA: 7s - loss: 0.0317 - accuracy: 0.9889
250/1688 [===>..........................] - ETA: 7s - loss: 0.0308 - accuracy: 0.9893
260/1688 [===>..........................] - ETA: 7s - loss: 0.0309 - accuracy: 0.9892
270/1688 [===>..........................] - ETA: 7s - loss: 0.0305 - accuracy: 0.9892
280/1688 [===>..........................] - ETA: 7s - loss: 0.0299 - accuracy: 0.9893
290/1688 [====>.........................] - ETA: 7s - loss: 0.0294 - accuracy: 0.9894
300/1688 [====>.........................] - ETA: 6s - loss: 0.0288 - accuracy: 0.9897
311/1688 [====>.........................] - ETA: 6s - loss: 0.0286 - accuracy: 0.9899
321/1688 [====>.........................] - ETA: 6s - loss: 0.0291 - accuracy: 0.9896
331/1688 [====>.........................] - ETA: 6s - loss: 0.0287 - accuracy: 0.9897
341/1688 [=====>........................] - ETA: 6s - loss: 0.0281 - accuracy: 0.9900
351/1688 [=====>........................] - ETA: 6s - loss: 0.0280 - accuracy: 0.9900
362/1688 [=====>........................] - ETA: 6s - loss: 0.0277 - accuracy: 0.9901
372/1688 [=====>........................] - ETA: 6s - loss: 0.0277 - accuracy: 0.9900
382/1688 [=====>........................] - ETA: 6s - loss: 0.0272 - accuracy: 0.9903
392/1688 [=====>........................] - ETA: 6s - loss: 0.0269 - accuracy: 0.9904
402/1688 [======>.......................] - ETA: 6s - loss: 0.0271 - accuracy: 0.9903
413/1688 [======>.......................] - ETA: 6s - loss: 0.0268 - accuracy: 0.9904
423/1688 [======>.......................] - ETA: 6s - loss: 0.0271 - accuracy: 0.9903
434/1688 [======>.......................] - ETA: 6s - loss: 0.0268 - accuracy: 0.9904
444/1688 [======>.......................] - ETA: 6s - loss: 0.0265 - accuracy: 0.9906
454/1688 [=======>......................] - ETA: 6s - loss: 0.0262 - accuracy: 0.9907
465/1688 [=======>......................] - ETA: 6s - loss: 0.0260 - accuracy: 0.9907
475/1688 [=======>......................] - ETA: 6s - loss: 0.0258 - accuracy: 0.9908
486/1688 [=======>......................] - ETA: 6s - loss: 0.0258 - accuracy: 0.9908
496/1688 [=======>......................] - ETA: 5s - loss: 0.0258 - accuracy: 0.9909
506/1688 [=======>......................] - ETA: 5s - loss: 0.0256 - accuracy: 0.9909
516/1688 [========>.....................] - ETA: 5s - loss: 0.0254 - accuracy: 0.9910
526/1688 [========>.....................] - ETA: 5s - loss: 0.0255 - accuracy: 0.9909
536/1688 [========>.....................] - ETA: 5s - loss: 0.0254 - accuracy: 0.9910
547/1688 [========>.....................] - ETA: 5s - loss: 0.0254 - accuracy: 0.9910
558/1688 [========>.....................] - ETA: 5s - loss: 0.0254 - accuracy: 0.9909
569/1688 [=========>....................] - ETA: 5s - loss: 0.0252 - accuracy: 0.9910
579/1688 [=========>....................] - ETA: 5s - loss: 0.0250 - accuracy: 0.9911
589/1688 [=========>....................] - ETA: 5s - loss: 0.0249 - accuracy: 0.9912
599/1688 [=========>....................] - ETA: 5s - loss: 0.0248 - accuracy: 0.9912
610/1688 [=========>....................] - ETA: 5s - loss: 0.0248 - accuracy: 0.9913
621/1688 [==========>...................] - ETA: 5s - loss: 0.0246 - accuracy: 0.9914
632/1688 [==========>...................] - ETA: 5s - loss: 0.0245 - accuracy: 0.9915
643/1688 [==========>...................] - ETA: 5s - loss: 0.0244 - accuracy: 0.9915
654/1688 [==========>...................] - ETA: 5s - loss: 0.0243 - accuracy: 0.9915
665/1688 [==========>...................] - ETA: 5s - loss: 0.0240 - accuracy: 0.9917
675/1688 [==========>...................] - ETA: 5s - loss: 0.0239 - accuracy: 0.9918
685/1688 [===========>..................] - ETA: 5s - loss: 0.0237 - accuracy: 0.9919
695/1688 [===========>..................] - ETA: 4s - loss: 0.0236 - accuracy: 0.9920
706/1688 [===========>..................] - ETA: 4s - loss: 0.0236 - accuracy: 0.9920
717/1688 [===========>..................] - ETA: 4s - loss: 0.0234 - accuracy: 0.9921
727/1688 [===========>..................] - ETA: 4s - loss: 0.0233 - accuracy: 0.9921
737/1688 [============>.................] - ETA: 4s - loss: 0.0233 - accuracy: 0.9922
747/1688 [============>.................] - ETA: 4s - loss: 0.0232 - accuracy: 0.9921
757/1688 [============>.................] - ETA: 4s - loss: 0.0231 - accuracy: 0.9921
768/1688 [============>.................] - ETA: 4s - loss: 0.0231 - accuracy: 0.9921
778/1688 [============>.................] - ETA: 4s - loss: 0.0230 - accuracy: 0.9921
788/1688 [=============>................] - ETA: 4s - loss: 0.0230 - accuracy: 0.9921
799/1688 [=============>................] - ETA: 4s - loss: 0.0229 - accuracy: 0.9921
810/1688 [=============>................] - ETA: 4s - loss: 0.0227 - accuracy: 0.9921
821/1688 [=============>................] - ETA: 4s - loss: 0.0226 - accuracy: 0.9922
831/1688 [=============>................] - ETA: 4s - loss: 0.0224 - accuracy: 0.9923
841/1688 [=============>................] - ETA: 4s - loss: 0.0223 - accuracy: 0.9924
851/1688 [==============>...............] - ETA: 4s - loss: 0.0224 - accuracy: 0.9924
862/1688 [==============>...............] - ETA: 4s - loss: 0.0224 - accuracy: 0.9924
872/1688 [==============>...............] - ETA: 4s - loss: 0.0223 - accuracy: 0.9924
882/1688 [==============>...............] - ETA: 4s - loss: 0.0222 - accuracy: 0.9924
892/1688 [==============>...............] - ETA: 3s - loss: 0.0224 - accuracy: 0.9923
903/1688 [===============>..............] - ETA: 3s - loss: 0.0223 - accuracy: 0.9923
913/1688 [===============>..............] - ETA: 3s - loss: 0.0222 - accuracy: 0.9924
923/1688 [===============>..............] - ETA: 3s - loss: 0.0220 - accuracy: 0.9925
933/1688 [===============>..............] - ETA: 3s - loss: 0.0220 - accuracy: 0.9925
943/1688 [===============>..............] - ETA: 3s - loss: 0.0221 - accuracy: 0.9925
953/1688 [===============>..............] - ETA: 3s - loss: 0.0221 - accuracy: 0.9925
963/1688 [================>.............] - ETA: 3s - loss: 0.0221 - accuracy: 0.9925
974/1688 [================>.............] - ETA: 3s - loss: 0.0219 - accuracy: 0.9926
984/1688 [================>.............] - ETA: 3s - loss: 0.0218 - accuracy: 0.9926
994/1688 [================>.............] - ETA: 3s - loss: 0.0217 - accuracy: 0.9927
1004/1688 [================>.............] - ETA: 3s - loss: 0.0216 - accuracy: 0.9927
1014/1688 [=================>............] - ETA: 3s - loss: 0.0215 - accuracy: 0.9928
1024/1688 [=================>............] - ETA: 3s - loss: 0.0215 - accuracy: 0.9928
1034/1688 [=================>............] - ETA: 3s - loss: 0.0214 - accuracy: 0.9929
1045/1688 [=================>............] - ETA: 3s - loss: 0.0213 - accuracy: 0.9929
1056/1688 [=================>............] - ETA: 3s - loss: 0.0211 - accuracy: 0.9930
1066/1688 [=================>............] - ETA: 3s - loss: 0.0212 - accuracy: 0.9930
1076/1688 [==================>...........] - ETA: 3s - loss: 0.0210 - accuracy: 0.9930
1087/1688 [==================>...........] - ETA: 3s - loss: 0.0210 - accuracy: 0.9931
1098/1688 [==================>...........] - ETA: 2s - loss: 0.0210 - accuracy: 0.9931
1108/1688 [==================>...........] - ETA: 2s - loss: 0.0210 - accuracy: 0.9931
1118/1688 [==================>...........] - ETA: 2s - loss: 0.0209 - accuracy: 0.9932
1129/1688 [===================>..........] - ETA: 2s - loss: 0.0208 - accuracy: 0.9932
1140/1688 [===================>..........] - ETA: 2s - loss: 0.0207 - accuracy: 0.9933
1151/1688 [===================>..........] - ETA: 2s - loss: 0.0206 - accuracy: 0.9933
1161/1688 [===================>..........] - ETA: 2s - loss: 0.0205 - accuracy: 0.9934
1172/1688 [===================>..........] - ETA: 2s - loss: 0.0206 - accuracy: 0.9933
1182/1688 [====================>.........] - ETA: 2s - loss: 0.0205 - accuracy: 0.9934
1193/1688 [====================>.........] - ETA: 2s - loss: 0.0205 - accuracy: 0.9934
1203/1688 [====================>.........] - ETA: 2s - loss: 0.0206 - accuracy: 0.9933
1214/1688 [====================>.........] - ETA: 2s - loss: 0.0205 - accuracy: 0.9934
1224/1688 [====================>.........] - ETA: 2s - loss: 0.0204 - accuracy: 0.9934
1234/1688 [====================>.........] - ETA: 2s - loss: 0.0204 - accuracy: 0.9934
1245/1688 [=====================>........] - ETA: 2s - loss: 0.0204 - accuracy: 0.9935
1256/1688 [=====================>........] - ETA: 2s - loss: 0.0204 - accuracy: 0.9935
1266/1688 [=====================>........] - ETA: 2s - loss: 0.0204 - accuracy: 0.9935
1276/1688 [=====================>........] - ETA: 2s - loss: 0.0204 - accuracy: 0.9935
1286/1688 [=====================>........] - ETA: 2s - loss: 0.0203 - accuracy: 0.9935
1296/1688 [======================>.......] - ETA: 1s - loss: 0.0202 - accuracy: 0.9935
1306/1688 [======================>.......] - ETA: 1s - loss: 0.0202 - accuracy: 0.9935
1317/1688 [======================>.......] - ETA: 1s - loss: 0.0202 - accuracy: 0.9935
1327/1688 [======================>.......] - ETA: 1s - loss: 0.0201 - accuracy: 0.9935
1338/1688 [======================>.......] - ETA: 1s - loss: 0.0201 - accuracy: 0.9936
1349/1688 [======================>.......] - ETA: 1s - loss: 0.0200 - accuracy: 0.9936
1360/1688 [=======================>......] - ETA: 1s - loss: 0.0199 - accuracy: 0.9937
1371/1688 [=======================>......] - ETA: 1s - loss: 0.0199 - accuracy: 0.9936
1381/1688 [=======================>......] - ETA: 1s - loss: 0.0199 - accuracy: 0.9936
1391/1688 [=======================>......] - ETA: 1s - loss: 0.0199 - accuracy: 0.9936
1402/1688 [=======================>......] - ETA: 1s - loss: 0.0200 - accuracy: 0.9936
1412/1688 [========================>.....] - ETA: 1s - loss: 0.0200 - accuracy: 0.9936
1423/1688 [========================>.....] - ETA: 1s - loss: 0.0199 - accuracy: 0.9937
1433/1688 [========================>.....] - ETA: 1s - loss: 0.0199 - accuracy: 0.9936
1444/1688 [========================>.....] - ETA: 1s - loss: 0.0199 - accuracy: 0.9936
1455/1688 [========================>.....] - ETA: 1s - loss: 0.0198 - accuracy: 0.9937
1466/1688 [=========================>....] - ETA: 1s - loss: 0.0198 - accuracy: 0.9937
1476/1688 [=========================>....] - ETA: 1s - loss: 0.0197 - accuracy: 0.9938
1486/1688 [=========================>....] - ETA: 1s - loss: 0.0196 - accuracy: 0.9938
1497/1688 [=========================>....] - ETA: 0s - loss: 0.0195 - accuracy: 0.9938
1507/1688 [=========================>....] - ETA: 0s - loss: 0.0195 - accuracy: 0.9938
1518/1688 [=========================>....] - ETA: 0s - loss: 0.0194 - accuracy: 0.9938
1528/1688 [==========================>...] - ETA: 0s - loss: 0.0194 - accuracy: 0.9939
1538/1688 [==========================>...] - ETA: 0s - loss: 0.0193 - accuracy: 0.9939
1549/1688 [==========================>...] - ETA: 0s - loss: 0.0192 - accuracy: 0.9939
1559/1688 [==========================>...] - ETA: 0s - loss: 0.0192 - accuracy: 0.9939
1569/1688 [==========================>...] - ETA: 0s - loss: 0.0191 - accuracy: 0.9939
1579/1688 [===========================>..] - ETA: 0s - loss: 0.0191 - accuracy: 0.9940
1589/1688 [===========================>..] - ETA: 0s - loss: 0.0190 - accuracy: 0.9940
1599/1688 [===========================>..] - ETA: 0s - loss: 0.0190 - accuracy: 0.9940
1609/1688 [===========================>..] - ETA: 0s - loss: 0.0189 - accuracy: 0.9940
1620/1688 [===========================>..] - ETA: 0s - loss: 0.0189 - accuracy: 0.9940
1631/1688 [===========================>..] - ETA: 0s - loss: 0.0188 - accuracy: 0.9941
1642/1688 [============================>.] - ETA: 0s - loss: 0.0188 - accuracy: 0.9941
1652/1688 [============================>.] - ETA: 0s - loss: 0.0187 - accuracy: 0.9941
1662/1688 [============================>.] - ETA: 0s - loss: 0.0187 - accuracy: 0.9941
1672/1688 [============================>.] - ETA: 0s - loss: 0.0187 - accuracy: 0.9941
1682/1688 [============================>.] - ETA: 0s - loss: 0.0186 - accuracy: 0.9941
1688/1688 [==============================] - 15s 6ms/step - loss: 0.0186 - accuracy: 0.9941 - val_loss: 0.0549 - val_accuracy: 0.9863
Epoch 2/5
1/1688 [..............................] - ETA: 8s - loss: 0.0107 - accuracy: 1.0000
11/1688 [..............................] - ETA: 8s - loss: 0.0238 - accuracy: 0.9972
21/1688 [..............................] - ETA: 8s - loss: 0.0218 - accuracy: 0.9940
31/1688 [..............................] - ETA: 8s - loss: 0.0196 - accuracy: 0.9940
41/1688 [..............................] - ETA: 8s - loss: 0.0170 - accuracy: 0.9954
51/1688 [..............................] - ETA: 8s - loss: 0.0164 - accuracy: 0.9957
61/1688 [>.............................] - ETA: 8s - loss: 0.0158 - accuracy: 0.9959
71/1688 [>.............................] - ETA: 8s - loss: 0.0155 - accuracy: 0.9956
81/1688 [>.............................] - ETA: 8s - loss: 0.0146 - accuracy: 0.9961
92/1688 [>.............................] - ETA: 8s - loss: 0.0139 - accuracy: 0.9966
102/1688 [>.............................] - ETA: 7s - loss: 0.0132 - accuracy: 0.9969
113/1688 [=>............................] - ETA: 7s - loss: 0.0127 - accuracy: 0.9972
123/1688 [=>............................] - ETA: 7s - loss: 0.0128 - accuracy: 0.9972
134/1688 [=>............................] - ETA: 7s - loss: 0.0136 - accuracy: 0.9967
144/1688 [=>............................] - ETA: 7s - loss: 0.0138 - accuracy: 0.9967
154/1688 [=>............................] - ETA: 7s - loss: 0.0141 - accuracy: 0.9966
165/1688 [=>............................] - ETA: 7s - loss: 0.0142 - accuracy: 0.9964
176/1688 [==>...........................] - ETA: 7s - loss: 0.0139 - accuracy: 0.9964
186/1688 [==>...........................] - ETA: 7s - loss: 0.0139 - accuracy: 0.9963
197/1688 [==>...........................] - ETA: 7s - loss: 0.0136 - accuracy: 0.9965
208/1688 [==>...........................] - ETA: 7s - loss: 0.0134 - accuracy: 0.9967
218/1688 [==>...........................] - ETA: 7s - loss: 0.0138 - accuracy: 0.9966
228/1688 [===>..........................] - ETA: 7s - loss: 0.0141 - accuracy: 0.9964
238/1688 [===>..........................] - ETA: 7s - loss: 0.0139 - accuracy: 0.9965
249/1688 [===>..........................] - ETA: 7s - loss: 0.0141 - accuracy: 0.9962
259/1688 [===>..........................] - ETA: 7s - loss: 0.0142 - accuracy: 0.9961
269/1688 [===>..........................] - ETA: 7s - loss: 0.0143 - accuracy: 0.9959
280/1688 [===>..........................] - ETA: 7s - loss: 0.0139 - accuracy: 0.9961
291/1688 [====>.........................] - ETA: 7s - loss: 0.0140 - accuracy: 0.9961
301/1688 [====>.........................] - ETA: 6s - loss: 0.0141 - accuracy: 0.9962
311/1688 [====>.........................] - ETA: 6s - loss: 0.0144 - accuracy: 0.9961
321/1688 [====>.........................] - ETA: 6s - loss: 0.0147 - accuracy: 0.9960
331/1688 [====>.........................] - ETA: 6s - loss: 0.0145 - accuracy: 0.9961
342/1688 [=====>........................] - ETA: 6s - loss: 0.0143 - accuracy: 0.9962
352/1688 [=====>........................] - ETA: 6s - loss: 0.0143 - accuracy: 0.9962
362/1688 [=====>........................] - ETA: 6s - loss: 0.0143 - accuracy: 0.9961
373/1688 [=====>........................] - ETA: 6s - loss: 0.0141 - accuracy: 0.9962
383/1688 [=====>........................] - ETA: 6s - loss: 0.0142 - accuracy: 0.9961
394/1688 [======>.......................] - ETA: 6s - loss: 0.0142 - accuracy: 0.9960
404/1688 [======>.......................] - ETA: 6s - loss: 0.0140 - accuracy: 0.9961
415/1688 [======>.......................] - ETA: 6s - loss: 0.0139 - accuracy: 0.9962
425/1688 [======>.......................] - ETA: 6s - loss: 0.0141 - accuracy: 0.9961
435/1688 [======>.......................] - ETA: 6s - loss: 0.0140 - accuracy: 0.9961
446/1688 [======>.......................] - ETA: 6s - loss: 0.0140 - accuracy: 0.9961
456/1688 [=======>......................] - ETA: 6s - loss: 0.0140 - accuracy: 0.9962
466/1688 [=======>......................] - ETA: 6s - loss: 0.0140 - accuracy: 0.9962
477/1688 [=======>......................] - ETA: 6s - loss: 0.0139 - accuracy: 0.9963
488/1688 [=======>......................] - ETA: 6s - loss: 0.0138 - accuracy: 0.9963
498/1688 [=======>......................] - ETA: 5s - loss: 0.0139 - accuracy: 0.9962
509/1688 [========>.....................] - ETA: 5s - loss: 0.0138 - accuracy: 0.9963
520/1688 [========>.....................] - ETA: 5s - loss: 0.0137 - accuracy: 0.9963
530/1688 [========>.....................] - ETA: 5s - loss: 0.0138 - accuracy: 0.9963
540/1688 [========>.....................] - ETA: 5s - loss: 0.0137 - accuracy: 0.9963
550/1688 [========>.....................] - ETA: 5s - loss: 0.0137 - accuracy: 0.9962
560/1688 [========>.....................] - ETA: 5s - loss: 0.0138 - accuracy: 0.9961
570/1688 [=========>....................] - ETA: 5s - loss: 0.0139 - accuracy: 0.9962
581/1688 [=========>....................] - ETA: 5s - loss: 0.0142 - accuracy: 0.9961
591/1688 [=========>....................] - ETA: 5s - loss: 0.0142 - accuracy: 0.9959
602/1688 [=========>....................] - ETA: 5s - loss: 0.0142 - accuracy: 0.9960
612/1688 [=========>....................] - ETA: 5s - loss: 0.0142 - accuracy: 0.9959
622/1688 [==========>...................] - ETA: 5s - loss: 0.0141 - accuracy: 0.9959
632/1688 [==========>...................] - ETA: 5s - loss: 0.0141 - accuracy: 0.9960
642/1688 [==========>...................] - ETA: 5s - loss: 0.0140 - accuracy: 0.9960
653/1688 [==========>...................] - ETA: 5s - loss: 0.0141 - accuracy: 0.9960
663/1688 [==========>...................] - ETA: 5s - loss: 0.0144 - accuracy: 0.9959
673/1688 [==========>...................] - ETA: 5s - loss: 0.0145 - accuracy: 0.9959
683/1688 [===========>..................] - ETA: 5s - loss: 0.0144 - accuracy: 0.9959
693/1688 [===========>..................] - ETA: 5s - loss: 0.0143 - accuracy: 0.9959
703/1688 [===========>..................] - ETA: 4s - loss: 0.0144 - accuracy: 0.9959
714/1688 [===========>..................] - ETA: 4s - loss: 0.0143 - accuracy: 0.9959
724/1688 [===========>..................] - ETA: 4s - loss: 0.0143 - accuracy: 0.9959
734/1688 [============>.................] - ETA: 4s - loss: 0.0143 - accuracy: 0.9960
744/1688 [============>.................] - ETA: 4s - loss: 0.0142 - accuracy: 0.9960
754/1688 [============>.................] - ETA: 4s - loss: 0.0141 - accuracy: 0.9960
764/1688 [============>.................] - ETA: 4s - loss: 0.0141 - accuracy: 0.9961
775/1688 [============>.................] - ETA: 4s - loss: 0.0140 - accuracy: 0.9961
785/1688 [============>.................] - ETA: 4s - loss: 0.0140 - accuracy: 0.9961
796/1688 [=============>................] - ETA: 4s - loss: 0.0141 - accuracy: 0.9959
806/1688 [=============>................] - ETA: 4s - loss: 0.0143 - accuracy: 0.9959
817/1688 [=============>................] - ETA: 4s - loss: 0.0142 - accuracy: 0.9959
827/1688 [=============>................] - ETA: 4s - loss: 0.0143 - accuracy: 0.9959
837/1688 [=============>................] - ETA: 4s - loss: 0.0143 - accuracy: 0.9959
848/1688 [==============>...............] - ETA: 4s - loss: 0.0143 - accuracy: 0.9959
858/1688 [==============>...............] - ETA: 4s - loss: 0.0142 - accuracy: 0.9959
868/1688 [==============>...............] - ETA: 4s - loss: 0.0141 - accuracy: 0.9960
879/1688 [==============>...............] - ETA: 4s - loss: 0.0141 - accuracy: 0.9959
889/1688 [==============>...............] - ETA: 4s - loss: 0.0140 - accuracy: 0.9960
899/1688 [==============>...............] - ETA: 3s - loss: 0.0140 - accuracy: 0.9960
909/1688 [===============>..............] - ETA: 3s - loss: 0.0140 - accuracy: 0.9960
919/1688 [===============>..............] - ETA: 3s - loss: 0.0139 - accuracy: 0.9960
929/1688 [===============>..............] - ETA: 3s - loss: 0.0139 - accuracy: 0.9960
939/1688 [===============>..............] - ETA: 3s - loss: 0.0139 - accuracy: 0.9960
949/1688 [===============>..............] - ETA: 3s - loss: 0.0139 - accuracy: 0.9960
959/1688 [================>.............] - ETA: 3s - loss: 0.0139 - accuracy: 0.9960
970/1688 [================>.............] - ETA: 3s - loss: 0.0140 - accuracy: 0.9960
980/1688 [================>.............] - ETA: 3s - loss: 0.0139 - accuracy: 0.9960
991/1688 [================>.............] - ETA: 3s - loss: 0.0140 - accuracy: 0.9960
1001/1688 [================>.............] - ETA: 3s - loss: 0.0140 - accuracy: 0.9961
1011/1688 [================>.............] - ETA: 3s - loss: 0.0139 - accuracy: 0.9961
1022/1688 [=================>............] - ETA: 3s - loss: 0.0139 - accuracy: 0.9961
1032/1688 [=================>............] - ETA: 3s - loss: 0.0139 - accuracy: 0.9961
1043/1688 [=================>............] - ETA: 3s - loss: 0.0139 - accuracy: 0.9961
1053/1688 [=================>............] - ETA: 3s - loss: 0.0139 - accuracy: 0.9961
1064/1688 [=================>............] - ETA: 3s - loss: 0.0139 - accuracy: 0.9961
1074/1688 [==================>...........] - ETA: 3s - loss: 0.0140 - accuracy: 0.9961
1084/1688 [==================>...........] - ETA: 3s - loss: 0.0139 - accuracy: 0.9962
1094/1688 [==================>...........] - ETA: 2s - loss: 0.0139 - accuracy: 0.9962
1104/1688 [==================>...........] - ETA: 2s - loss: 0.0139 - accuracy: 0.9962
1114/1688 [==================>...........] - ETA: 2s - loss: 0.0140 - accuracy: 0.9961
1124/1688 [==================>...........] - ETA: 2s - loss: 0.0140 - accuracy: 0.9961
1134/1688 [===================>..........] - ETA: 2s - loss: 0.0139 - accuracy: 0.9961
1144/1688 [===================>..........] - ETA: 2s - loss: 0.0138 - accuracy: 0.9962
1155/1688 [===================>..........] - ETA: 2s - loss: 0.0139 - accuracy: 0.9961
1165/1688 [===================>..........] - ETA: 2s - loss: 0.0140 - accuracy: 0.9961
1176/1688 [===================>..........] - ETA: 2s - loss: 0.0139 - accuracy: 0.9961
1186/1688 [====================>.........] - ETA: 2s - loss: 0.0139 - accuracy: 0.9962
1197/1688 [====================>.........] - ETA: 2s - loss: 0.0139 - accuracy: 0.9962
1207/1688 [====================>.........] - ETA: 2s - loss: 0.0138 - accuracy: 0.9962
1217/1688 [====================>.........] - ETA: 2s - loss: 0.0138 - accuracy: 0.9962
1227/1688 [====================>.........] - ETA: 2s - loss: 0.0138 - accuracy: 0.9962
1237/1688 [====================>.........] - ETA: 2s - loss: 0.0138 - accuracy: 0.9962
1248/1688 [=====================>........] - ETA: 2s - loss: 0.0137 - accuracy: 0.9962
1258/1688 [=====================>........] - ETA: 2s - loss: 0.0137 - accuracy: 0.9962
1269/1688 [=====================>........] - ETA: 2s - loss: 0.0137 - accuracy: 0.9963
1280/1688 [=====================>........] - ETA: 2s - loss: 0.0136 - accuracy: 0.9963
1290/1688 [=====================>........] - ETA: 1s - loss: 0.0137 - accuracy: 0.9962
1300/1688 [======================>.......] - ETA: 1s - loss: 0.0136 - accuracy: 0.9962
1311/1688 [======================>.......] - ETA: 1s - loss: 0.0137 - accuracy: 0.9962
1321/1688 [======================>.......] - ETA: 1s - loss: 0.0136 - accuracy: 0.9962
1331/1688 [======================>.......] - ETA: 1s - loss: 0.0136 - accuracy: 0.9962
1341/1688 [======================>.......] - ETA: 1s - loss: 0.0137 - accuracy: 0.9962
1351/1688 [=======================>......] - ETA: 1s - loss: 0.0137 - accuracy: 0.9962
1361/1688 [=======================>......] - ETA: 1s - loss: 0.0136 - accuracy: 0.9962
1371/1688 [=======================>......] - ETA: 1s - loss: 0.0137 - accuracy: 0.9962
1382/1688 [=======================>......] - ETA: 1s - loss: 0.0136 - accuracy: 0.9962
1392/1688 [=======================>......] - ETA: 1s - loss: 0.0136 - accuracy: 0.9963
1403/1688 [=======================>......] - ETA: 1s - loss: 0.0135 - accuracy: 0.9963
1413/1688 [========================>.....] - ETA: 1s - loss: 0.0135 - accuracy: 0.9963
1423/1688 [========================>.....] - ETA: 1s - loss: 0.0135 - accuracy: 0.9963
1433/1688 [========================>.....] - ETA: 1s - loss: 0.0134 - accuracy: 0.9963
1443/1688 [========================>.....] - ETA: 1s - loss: 0.0134 - accuracy: 0.9964
1453/1688 [========================>.....] - ETA: 1s - loss: 0.0134 - accuracy: 0.9964
1464/1688 [=========================>....] - ETA: 1s - loss: 0.0133 - accuracy: 0.9964
1475/1688 [=========================>....] - ETA: 1s - loss: 0.0133 - accuracy: 0.9964
1485/1688 [=========================>....] - ETA: 1s - loss: 0.0133 - accuracy: 0.9964
1495/1688 [=========================>....] - ETA: 0s - loss: 0.0133 - accuracy: 0.9964
1506/1688 [=========================>....] - ETA: 0s - loss: 0.0132 - accuracy: 0.9964
1516/1688 [=========================>....] - ETA: 0s - loss: 0.0133 - accuracy: 0.9964
1526/1688 [==========================>...] - ETA: 0s - loss: 0.0134 - accuracy: 0.9964
1536/1688 [==========================>...] - ETA: 0s - loss: 0.0134 - accuracy: 0.9964
1547/1688 [==========================>...] - ETA: 0s - loss: 0.0134 - accuracy: 0.9964
1557/1688 [==========================>...] - ETA: 0s - loss: 0.0134 - accuracy: 0.9964
1568/1688 [==========================>...] - ETA: 0s - loss: 0.0133 - accuracy: 0.9964
1578/1688 [===========================>..] - ETA: 0s - loss: 0.0134 - accuracy: 0.9964
1589/1688 [===========================>..] - ETA: 0s - loss: 0.0134 - accuracy: 0.9963
1599/1688 [===========================>..] - ETA: 0s - loss: 0.0135 - accuracy: 0.9963
1610/1688 [===========================>..] - ETA: 0s - loss: 0.0135 - accuracy: 0.9963
1620/1688 [===========================>..] - ETA: 0s - loss: 0.0135 - accuracy: 0.9963
1630/1688 [===========================>..] - ETA: 0s - loss: 0.0134 - accuracy: 0.9963
1640/1688 [============================>.] - ETA: 0s - loss: 0.0134 - accuracy: 0.9963
1651/1688 [============================>.] - ETA: 0s - loss: 0.0134 - accuracy: 0.9963
1661/1688 [============================>.] - ETA: 0s - loss: 0.0134 - accuracy: 0.9963
1671/1688 [============================>.] - ETA: 0s - loss: 0.0134 - accuracy: 0.9963
1682/1688 [============================>.] - ETA: 0s - loss: 0.0134 - accuracy: 0.9963
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0133 - accuracy: 0.9963 - val_loss: 0.0547 - val_accuracy: 0.9860
Epoch 3/5
1/1688 [..............................] - ETA: 8s - loss: 0.0057 - accuracy: 1.0000
12/1688 [..............................] - ETA: 8s - loss: 0.0188 - accuracy: 0.9922
22/1688 [..............................] - ETA: 8s - loss: 0.0168 - accuracy: 0.9929
32/1688 [..............................] - ETA: 8s - loss: 0.0164 - accuracy: 0.9941
43/1688 [..............................] - ETA: 8s - loss: 0.0178 - accuracy: 0.9935
53/1688 [..............................] - ETA: 8s - loss: 0.0158 - accuracy: 0.9941
63/1688 [>.............................] - ETA: 8s - loss: 0.0148 - accuracy: 0.9945
73/1688 [>.............................] - ETA: 8s - loss: 0.0135 - accuracy: 0.9953
84/1688 [>.............................] - ETA: 8s - loss: 0.0134 - accuracy: 0.9952
94/1688 [>.............................] - ETA: 8s - loss: 0.0129 - accuracy: 0.9953
104/1688 [>.............................] - ETA: 7s - loss: 0.0129 - accuracy: 0.9955
114/1688 [=>............................] - ETA: 7s - loss: 0.0120 - accuracy: 0.9959
125/1688 [=>............................] - ETA: 7s - loss: 0.0115 - accuracy: 0.9962
135/1688 [=>............................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9965
145/1688 [=>............................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9966
156/1688 [=>............................] - ETA: 7s - loss: 0.0112 - accuracy: 0.9966
167/1688 [=>............................] - ETA: 7s - loss: 0.0109 - accuracy: 0.9968
177/1688 [==>...........................] - ETA: 7s - loss: 0.0110 - accuracy: 0.9968
188/1688 [==>...........................] - ETA: 7s - loss: 0.0112 - accuracy: 0.9968
199/1688 [==>...........................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9969
210/1688 [==>...........................] - ETA: 7s - loss: 0.0109 - accuracy: 0.9970
220/1688 [==>...........................] - ETA: 7s - loss: 0.0109 - accuracy: 0.9970
230/1688 [===>..........................] - ETA: 7s - loss: 0.0107 - accuracy: 0.9971
241/1688 [===>..........................] - ETA: 7s - loss: 0.0110 - accuracy: 0.9970
251/1688 [===>..........................] - ETA: 7s - loss: 0.0109 - accuracy: 0.9970
262/1688 [===>..........................] - ETA: 7s - loss: 0.0110 - accuracy: 0.9970
273/1688 [===>..........................] - ETA: 7s - loss: 0.0110 - accuracy: 0.9970
283/1688 [====>.........................] - ETA: 7s - loss: 0.0115 - accuracy: 0.9969
293/1688 [====>.........................] - ETA: 7s - loss: 0.0117 - accuracy: 0.9967
303/1688 [====>.........................] - ETA: 6s - loss: 0.0116 - accuracy: 0.9967
313/1688 [====>.........................] - ETA: 6s - loss: 0.0114 - accuracy: 0.9968
324/1688 [====>.........................] - ETA: 6s - loss: 0.0112 - accuracy: 0.9968
334/1688 [====>.........................] - ETA: 6s - loss: 0.0113 - accuracy: 0.9968
344/1688 [=====>........................] - ETA: 6s - loss: 0.0112 - accuracy: 0.9968
354/1688 [=====>........................] - ETA: 6s - loss: 0.0112 - accuracy: 0.9969
364/1688 [=====>........................] - ETA: 6s - loss: 0.0112 - accuracy: 0.9969
374/1688 [=====>........................] - ETA: 6s - loss: 0.0112 - accuracy: 0.9969
384/1688 [=====>........................] - ETA: 6s - loss: 0.0112 - accuracy: 0.9969
395/1688 [======>.......................] - ETA: 6s - loss: 0.0111 - accuracy: 0.9970
406/1688 [======>.......................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9971
416/1688 [======>.......................] - ETA: 6s - loss: 0.0109 - accuracy: 0.9971
426/1688 [======>.......................] - ETA: 6s - loss: 0.0109 - accuracy: 0.9971
436/1688 [======>.......................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9970
447/1688 [======>.......................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9969
457/1688 [=======>......................] - ETA: 6s - loss: 0.0109 - accuracy: 0.9970
467/1688 [=======>......................] - ETA: 6s - loss: 0.0111 - accuracy: 0.9970
478/1688 [=======>......................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9971
488/1688 [=======>......................] - ETA: 6s - loss: 0.0112 - accuracy: 0.9970
499/1688 [=======>......................] - ETA: 5s - loss: 0.0111 - accuracy: 0.9971
509/1688 [========>.....................] - ETA: 5s - loss: 0.0112 - accuracy: 0.9971
519/1688 [========>.....................] - ETA: 5s - loss: 0.0113 - accuracy: 0.9970
530/1688 [========>.....................] - ETA: 5s - loss: 0.0111 - accuracy: 0.9971
540/1688 [========>.....................] - ETA: 5s - loss: 0.0110 - accuracy: 0.9972
551/1688 [========>.....................] - ETA: 5s - loss: 0.0111 - accuracy: 0.9971
561/1688 [========>.....................] - ETA: 5s - loss: 0.0112 - accuracy: 0.9970
572/1688 [=========>....................] - ETA: 5s - loss: 0.0111 - accuracy: 0.9970
583/1688 [=========>....................] - ETA: 5s - loss: 0.0111 - accuracy: 0.9971
594/1688 [=========>....................] - ETA: 5s - loss: 0.0110 - accuracy: 0.9971
605/1688 [=========>....................] - ETA: 5s - loss: 0.0110 - accuracy: 0.9971
615/1688 [=========>....................] - ETA: 5s - loss: 0.0109 - accuracy: 0.9971
625/1688 [==========>...................] - ETA: 5s - loss: 0.0109 - accuracy: 0.9971
635/1688 [==========>...................] - ETA: 5s - loss: 0.0110 - accuracy: 0.9970
645/1688 [==========>...................] - ETA: 5s - loss: 0.0112 - accuracy: 0.9970
656/1688 [==========>...................] - ETA: 5s - loss: 0.0112 - accuracy: 0.9970
667/1688 [==========>...................] - ETA: 5s - loss: 0.0111 - accuracy: 0.9970
678/1688 [===========>..................] - ETA: 5s - loss: 0.0110 - accuracy: 0.9971
688/1688 [===========>..................] - ETA: 5s - loss: 0.0110 - accuracy: 0.9971
698/1688 [===========>..................] - ETA: 4s - loss: 0.0111 - accuracy: 0.9971
708/1688 [===========>..................] - ETA: 4s - loss: 0.0110 - accuracy: 0.9971
719/1688 [===========>..................] - ETA: 4s - loss: 0.0112 - accuracy: 0.9970
730/1688 [===========>..................] - ETA: 4s - loss: 0.0113 - accuracy: 0.9970
740/1688 [============>.................] - ETA: 4s - loss: 0.0113 - accuracy: 0.9970
750/1688 [============>.................] - ETA: 4s - loss: 0.0114 - accuracy: 0.9970
761/1688 [============>.................] - ETA: 4s - loss: 0.0113 - accuracy: 0.9970
771/1688 [============>.................] - ETA: 4s - loss: 0.0115 - accuracy: 0.9970
781/1688 [============>.................] - ETA: 4s - loss: 0.0114 - accuracy: 0.9970
791/1688 [=============>................] - ETA: 4s - loss: 0.0114 - accuracy: 0.9970
801/1688 [=============>................] - ETA: 4s - loss: 0.0116 - accuracy: 0.9969
812/1688 [=============>................] - ETA: 4s - loss: 0.0117 - accuracy: 0.9969
823/1688 [=============>................] - ETA: 4s - loss: 0.0117 - accuracy: 0.9969
834/1688 [=============>................] - ETA: 4s - loss: 0.0117 - accuracy: 0.9969
844/1688 [==============>...............] - ETA: 4s - loss: 0.0117 - accuracy: 0.9969
855/1688 [==============>...............] - ETA: 4s - loss: 0.0117 - accuracy: 0.9969
865/1688 [==============>...............] - ETA: 4s - loss: 0.0119 - accuracy: 0.9967
875/1688 [==============>...............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9967
885/1688 [==============>...............] - ETA: 4s - loss: 0.0119 - accuracy: 0.9968
896/1688 [==============>...............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9968
906/1688 [===============>..............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9968
916/1688 [===============>..............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9968
926/1688 [===============>..............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9968
936/1688 [===============>..............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9968
946/1688 [===============>..............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9968
957/1688 [================>.............] - ETA: 3s - loss: 0.0119 - accuracy: 0.9967
967/1688 [================>.............] - ETA: 3s - loss: 0.0119 - accuracy: 0.9967
977/1688 [================>.............] - ETA: 3s - loss: 0.0119 - accuracy: 0.9967
987/1688 [================>.............] - ETA: 3s - loss: 0.0119 - accuracy: 0.9967
998/1688 [================>.............] - ETA: 3s - loss: 0.0120 - accuracy: 0.9967
1009/1688 [================>.............] - ETA: 3s - loss: 0.0120 - accuracy: 0.9966
1019/1688 [=================>............] - ETA: 3s - loss: 0.0121 - accuracy: 0.9966
1029/1688 [=================>............] - ETA: 3s - loss: 0.0121 - accuracy: 0.9966
1040/1688 [=================>............] - ETA: 3s - loss: 0.0122 - accuracy: 0.9965
1050/1688 [=================>............] - ETA: 3s - loss: 0.0121 - accuracy: 0.9965
1060/1688 [=================>............] - ETA: 3s - loss: 0.0121 - accuracy: 0.9966
1071/1688 [==================>...........] - ETA: 3s - loss: 0.0121 - accuracy: 0.9966
1081/1688 [==================>...........] - ETA: 3s - loss: 0.0121 - accuracy: 0.9965
1092/1688 [==================>...........] - ETA: 2s - loss: 0.0120 - accuracy: 0.9966
1102/1688 [==================>...........] - ETA: 2s - loss: 0.0121 - accuracy: 0.9965
1112/1688 [==================>...........] - ETA: 2s - loss: 0.0123 - accuracy: 0.9964
1122/1688 [==================>...........] - ETA: 2s - loss: 0.0124 - accuracy: 0.9964
1132/1688 [===================>..........] - ETA: 2s - loss: 0.0124 - accuracy: 0.9964
1143/1688 [===================>..........] - ETA: 2s - loss: 0.0123 - accuracy: 0.9964
1153/1688 [===================>..........] - ETA: 2s - loss: 0.0123 - accuracy: 0.9965
1164/1688 [===================>..........] - ETA: 2s - loss: 0.0123 - accuracy: 0.9965
1175/1688 [===================>..........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9965
1185/1688 [====================>.........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9964
1195/1688 [====================>.........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9964
1205/1688 [====================>.........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9964
1215/1688 [====================>.........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9964
1225/1688 [====================>.........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9964
1235/1688 [====================>.........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9964
1245/1688 [=====================>........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9964
1255/1688 [=====================>........] - ETA: 2s - loss: 0.0124 - accuracy: 0.9964
1266/1688 [=====================>........] - ETA: 2s - loss: 0.0123 - accuracy: 0.9964
1276/1688 [=====================>........] - ETA: 2s - loss: 0.0123 - accuracy: 0.9964
1287/1688 [=====================>........] - ETA: 2s - loss: 0.0123 - accuracy: 0.9965
1297/1688 [======================>.......] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1308/1688 [======================>.......] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1318/1688 [======================>.......] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1329/1688 [======================>.......] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1339/1688 [======================>.......] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1350/1688 [======================>.......] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1360/1688 [=======================>......] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1370/1688 [=======================>......] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1381/1688 [=======================>......] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1392/1688 [=======================>......] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1403/1688 [=======================>......] - ETA: 1s - loss: 0.0121 - accuracy: 0.9965
1414/1688 [========================>.....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9966
1424/1688 [========================>.....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9966
1435/1688 [========================>.....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9966
1445/1688 [========================>.....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9966
1456/1688 [========================>.....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9966
1466/1688 [=========================>....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9966
1477/1688 [=========================>....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9966
1487/1688 [=========================>....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9966
1498/1688 [=========================>....] - ETA: 0s - loss: 0.0121 - accuracy: 0.9966
1509/1688 [=========================>....] - ETA: 0s - loss: 0.0121 - accuracy: 0.9966
1519/1688 [=========================>....] - ETA: 0s - loss: 0.0120 - accuracy: 0.9966
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0120 - accuracy: 0.9966
1540/1688 [==========================>...] - ETA: 0s - loss: 0.0121 - accuracy: 0.9966
1550/1688 [==========================>...] - ETA: 0s - loss: 0.0120 - accuracy: 0.9966
1561/1688 [==========================>...] - ETA: 0s - loss: 0.0120 - accuracy: 0.9966
1571/1688 [==========================>...] - ETA: 0s - loss: 0.0119 - accuracy: 0.9966
1582/1688 [===========================>..] - ETA: 0s - loss: 0.0119 - accuracy: 0.9966
1592/1688 [===========================>..] - ETA: 0s - loss: 0.0119 - accuracy: 0.9966
1602/1688 [===========================>..] - ETA: 0s - loss: 0.0119 - accuracy: 0.9966
1613/1688 [===========================>..] - ETA: 0s - loss: 0.0120 - accuracy: 0.9966
1623/1688 [===========================>..] - ETA: 0s - loss: 0.0119 - accuracy: 0.9966
1634/1688 [============================>.] - ETA: 0s - loss: 0.0119 - accuracy: 0.9966
1645/1688 [============================>.] - ETA: 0s - loss: 0.0119 - accuracy: 0.9967
1656/1688 [============================>.] - ETA: 0s - loss: 0.0119 - accuracy: 0.9966
1666/1688 [============================>.] - ETA: 0s - loss: 0.0119 - accuracy: 0.9966
1676/1688 [============================>.] - ETA: 0s - loss: 0.0119 - accuracy: 0.9966
1686/1688 [============================>.] - ETA: 0s - loss: 0.0118 - accuracy: 0.9967
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0118 - accuracy: 0.9967 - val_loss: 0.0549 - val_accuracy: 0.9857
Epoch 4/5
1/1688 [..............................] - ETA: 8s - loss: 0.0022 - accuracy: 1.0000
12/1688 [..............................] - ETA: 8s - loss: 0.0122 - accuracy: 0.9922
22/1688 [..............................] - ETA: 8s - loss: 0.0088 - accuracy: 0.9957
32/1688 [..............................] - ETA: 8s - loss: 0.0076 - accuracy: 0.9971
42/1688 [..............................] - ETA: 8s - loss: 0.0086 - accuracy: 0.9970
52/1688 [..............................] - ETA: 8s - loss: 0.0090 - accuracy: 0.9970
62/1688 [>.............................] - ETA: 8s - loss: 0.0086 - accuracy: 0.9975
73/1688 [>.............................] - ETA: 8s - loss: 0.0083 - accuracy: 0.9979
83/1688 [>.............................] - ETA: 8s - loss: 0.0079 - accuracy: 0.9981
94/1688 [>.............................] - ETA: 7s - loss: 0.0084 - accuracy: 0.9977
105/1688 [>.............................] - ETA: 7s - loss: 0.0089 - accuracy: 0.9973
115/1688 [=>............................] - ETA: 7s - loss: 0.0096 - accuracy: 0.9973
125/1688 [=>............................] - ETA: 7s - loss: 0.0092 - accuracy: 0.9975
136/1688 [=>............................] - ETA: 7s - loss: 0.0089 - accuracy: 0.9977
146/1688 [=>............................] - ETA: 7s - loss: 0.0088 - accuracy: 0.9979
156/1688 [=>............................] - ETA: 7s - loss: 0.0090 - accuracy: 0.9978
166/1688 [=>............................] - ETA: 7s - loss: 0.0090 - accuracy: 0.9977
177/1688 [==>...........................] - ETA: 7s - loss: 0.0090 - accuracy: 0.9977
187/1688 [==>...........................] - ETA: 7s - loss: 0.0090 - accuracy: 0.9978
197/1688 [==>...........................] - ETA: 7s - loss: 0.0095 - accuracy: 0.9976
207/1688 [==>...........................] - ETA: 7s - loss: 0.0098 - accuracy: 0.9976
217/1688 [==>...........................] - ETA: 7s - loss: 0.0099 - accuracy: 0.9974
227/1688 [===>..........................] - ETA: 7s - loss: 0.0099 - accuracy: 0.9975
237/1688 [===>..........................] - ETA: 7s - loss: 0.0098 - accuracy: 0.9975
247/1688 [===>..........................] - ETA: 7s - loss: 0.0099 - accuracy: 0.9975
257/1688 [===>..........................] - ETA: 7s - loss: 0.0101 - accuracy: 0.9974
267/1688 [===>..........................] - ETA: 7s - loss: 0.0099 - accuracy: 0.9975
277/1688 [===>..........................] - ETA: 7s - loss: 0.0100 - accuracy: 0.9975
287/1688 [====>.........................] - ETA: 7s - loss: 0.0097 - accuracy: 0.9976
298/1688 [====>.........................] - ETA: 7s - loss: 0.0097 - accuracy: 0.9976
309/1688 [====>.........................] - ETA: 6s - loss: 0.0099 - accuracy: 0.9975
319/1688 [====>.........................] - ETA: 6s - loss: 0.0100 - accuracy: 0.9975
329/1688 [====>.........................] - ETA: 6s - loss: 0.0099 - accuracy: 0.9975
339/1688 [=====>........................] - ETA: 6s - loss: 0.0099 - accuracy: 0.9975
350/1688 [=====>........................] - ETA: 6s - loss: 0.0098 - accuracy: 0.9976
360/1688 [=====>........................] - ETA: 6s - loss: 0.0099 - accuracy: 0.9976
371/1688 [=====>........................] - ETA: 6s - loss: 0.0098 - accuracy: 0.9975
381/1688 [=====>........................] - ETA: 6s - loss: 0.0099 - accuracy: 0.9974
391/1688 [=====>........................] - ETA: 6s - loss: 0.0100 - accuracy: 0.9974
401/1688 [======>.......................] - ETA: 6s - loss: 0.0099 - accuracy: 0.9974
411/1688 [======>.......................] - ETA: 6s - loss: 0.0100 - accuracy: 0.9973
421/1688 [======>.......................] - ETA: 6s - loss: 0.0102 - accuracy: 0.9972
431/1688 [======>.......................] - ETA: 6s - loss: 0.0101 - accuracy: 0.9972
441/1688 [======>.......................] - ETA: 6s - loss: 0.0101 - accuracy: 0.9972
452/1688 [=======>......................] - ETA: 6s - loss: 0.0102 - accuracy: 0.9971
463/1688 [=======>......................] - ETA: 6s - loss: 0.0100 - accuracy: 0.9972
473/1688 [=======>......................] - ETA: 6s - loss: 0.0100 - accuracy: 0.9972
483/1688 [=======>......................] - ETA: 6s - loss: 0.0099 - accuracy: 0.9972
494/1688 [=======>......................] - ETA: 6s - loss: 0.0100 - accuracy: 0.9972
505/1688 [=======>......................] - ETA: 5s - loss: 0.0100 - accuracy: 0.9972
515/1688 [========>.....................] - ETA: 5s - loss: 0.0100 - accuracy: 0.9973
525/1688 [========>.....................] - ETA: 5s - loss: 0.0101 - accuracy: 0.9972
535/1688 [========>.....................] - ETA: 5s - loss: 0.0101 - accuracy: 0.9973
545/1688 [========>.....................] - ETA: 5s - loss: 0.0100 - accuracy: 0.9973
556/1688 [========>.....................] - ETA: 5s - loss: 0.0099 - accuracy: 0.9973
567/1688 [=========>....................] - ETA: 5s - loss: 0.0100 - accuracy: 0.9973
577/1688 [=========>....................] - ETA: 5s - loss: 0.0099 - accuracy: 0.9973
587/1688 [=========>....................] - ETA: 5s - loss: 0.0099 - accuracy: 0.9974
597/1688 [=========>....................] - ETA: 5s - loss: 0.0098 - accuracy: 0.9974
607/1688 [=========>....................] - ETA: 5s - loss: 0.0100 - accuracy: 0.9974
617/1688 [=========>....................] - ETA: 5s - loss: 0.0102 - accuracy: 0.9973
628/1688 [==========>...................] - ETA: 5s - loss: 0.0102 - accuracy: 0.9973
638/1688 [==========>...................] - ETA: 5s - loss: 0.0103 - accuracy: 0.9972
649/1688 [==========>...................] - ETA: 5s - loss: 0.0103 - accuracy: 0.9972
659/1688 [==========>...................] - ETA: 5s - loss: 0.0102 - accuracy: 0.9972
670/1688 [==========>...................] - ETA: 5s - loss: 0.0101 - accuracy: 0.9972
680/1688 [===========>..................] - ETA: 5s - loss: 0.0102 - accuracy: 0.9972
690/1688 [===========>..................] - ETA: 5s - loss: 0.0103 - accuracy: 0.9972
700/1688 [===========>..................] - ETA: 4s - loss: 0.0103 - accuracy: 0.9973
710/1688 [===========>..................] - ETA: 4s - loss: 0.0104 - accuracy: 0.9972
721/1688 [===========>..................] - ETA: 4s - loss: 0.0103 - accuracy: 0.9973
731/1688 [===========>..................] - ETA: 4s - loss: 0.0104 - accuracy: 0.9972
741/1688 [============>.................] - ETA: 4s - loss: 0.0106 - accuracy: 0.9972
752/1688 [============>.................] - ETA: 4s - loss: 0.0106 - accuracy: 0.9973
762/1688 [============>.................] - ETA: 4s - loss: 0.0106 - accuracy: 0.9973
772/1688 [============>.................] - ETA: 4s - loss: 0.0106 - accuracy: 0.9972
782/1688 [============>.................] - ETA: 4s - loss: 0.0106 - accuracy: 0.9972
792/1688 [=============>................] - ETA: 4s - loss: 0.0105 - accuracy: 0.9973
802/1688 [=============>................] - ETA: 4s - loss: 0.0105 - accuracy: 0.9973
812/1688 [=============>................] - ETA: 4s - loss: 0.0105 - accuracy: 0.9973
822/1688 [=============>................] - ETA: 4s - loss: 0.0104 - accuracy: 0.9973
832/1688 [=============>................] - ETA: 4s - loss: 0.0104 - accuracy: 0.9973
843/1688 [=============>................] - ETA: 4s - loss: 0.0104 - accuracy: 0.9973
853/1688 [==============>...............] - ETA: 4s - loss: 0.0103 - accuracy: 0.9973
863/1688 [==============>...............] - ETA: 4s - loss: 0.0103 - accuracy: 0.9973
873/1688 [==============>...............] - ETA: 4s - loss: 0.0104 - accuracy: 0.9973
883/1688 [==============>...............] - ETA: 4s - loss: 0.0104 - accuracy: 0.9973
893/1688 [==============>...............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9973
904/1688 [===============>..............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9973
914/1688 [===============>..............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9973
924/1688 [===============>..............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9973
935/1688 [===============>..............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9973
946/1688 [===============>..............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9973
956/1688 [===============>..............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9973
967/1688 [================>.............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9973
977/1688 [================>.............] - ETA: 3s - loss: 0.0103 - accuracy: 0.9973
987/1688 [================>.............] - ETA: 3s - loss: 0.0103 - accuracy: 0.9973
997/1688 [================>.............] - ETA: 3s - loss: 0.0103 - accuracy: 0.9973
1007/1688 [================>.............] - ETA: 3s - loss: 0.0103 - accuracy: 0.9973
1017/1688 [=================>............] - ETA: 3s - loss: 0.0103 - accuracy: 0.9974
1027/1688 [=================>............] - ETA: 3s - loss: 0.0103 - accuracy: 0.9973
1037/1688 [=================>............] - ETA: 3s - loss: 0.0103 - accuracy: 0.9973
1047/1688 [=================>............] - ETA: 3s - loss: 0.0105 - accuracy: 0.9972
1057/1688 [=================>............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9973
1067/1688 [=================>............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9973
1077/1688 [==================>...........] - ETA: 3s - loss: 0.0104 - accuracy: 0.9973
1087/1688 [==================>...........] - ETA: 3s - loss: 0.0104 - accuracy: 0.9973
1098/1688 [==================>...........] - ETA: 2s - loss: 0.0104 - accuracy: 0.9973
1109/1688 [==================>...........] - ETA: 2s - loss: 0.0104 - accuracy: 0.9973
1119/1688 [==================>...........] - ETA: 2s - loss: 0.0104 - accuracy: 0.9973
1129/1688 [===================>..........] - ETA: 2s - loss: 0.0103 - accuracy: 0.9973
1140/1688 [===================>..........] - ETA: 2s - loss: 0.0104 - accuracy: 0.9973
1151/1688 [===================>..........] - ETA: 2s - loss: 0.0106 - accuracy: 0.9973
1162/1688 [===================>..........] - ETA: 2s - loss: 0.0105 - accuracy: 0.9973
1172/1688 [===================>..........] - ETA: 2s - loss: 0.0105 - accuracy: 0.9973
1182/1688 [====================>.........] - ETA: 2s - loss: 0.0106 - accuracy: 0.9973
1193/1688 [====================>.........] - ETA: 2s - loss: 0.0106 - accuracy: 0.9973
1203/1688 [====================>.........] - ETA: 2s - loss: 0.0106 - accuracy: 0.9973
1213/1688 [====================>.........] - ETA: 2s - loss: 0.0106 - accuracy: 0.9973
1224/1688 [====================>.........] - ETA: 2s - loss: 0.0106 - accuracy: 0.9973
1235/1688 [====================>.........] - ETA: 2s - loss: 0.0106 - accuracy: 0.9973
1246/1688 [=====================>........] - ETA: 2s - loss: 0.0106 - accuracy: 0.9973
1256/1688 [=====================>........] - ETA: 2s - loss: 0.0105 - accuracy: 0.9973
1266/1688 [=====================>........] - ETA: 2s - loss: 0.0105 - accuracy: 0.9973
1276/1688 [=====================>........] - ETA: 2s - loss: 0.0105 - accuracy: 0.9973
1286/1688 [=====================>........] - ETA: 2s - loss: 0.0106 - accuracy: 0.9973
1297/1688 [======================>.......] - ETA: 1s - loss: 0.0106 - accuracy: 0.9973
1308/1688 [======================>.......] - ETA: 1s - loss: 0.0105 - accuracy: 0.9973
1318/1688 [======================>.......] - ETA: 1s - loss: 0.0105 - accuracy: 0.9973
1328/1688 [======================>.......] - ETA: 1s - loss: 0.0105 - accuracy: 0.9973
1338/1688 [======================>.......] - ETA: 1s - loss: 0.0105 - accuracy: 0.9973
1348/1688 [======================>.......] - ETA: 1s - loss: 0.0105 - accuracy: 0.9973
1358/1688 [=======================>......] - ETA: 1s - loss: 0.0105 - accuracy: 0.9973
1369/1688 [=======================>......] - ETA: 1s - loss: 0.0105 - accuracy: 0.9974
1380/1688 [=======================>......] - ETA: 1s - loss: 0.0105 - accuracy: 0.9974
1390/1688 [=======================>......] - ETA: 1s - loss: 0.0105 - accuracy: 0.9973
1400/1688 [=======================>......] - ETA: 1s - loss: 0.0105 - accuracy: 0.9973
1410/1688 [========================>.....] - ETA: 1s - loss: 0.0105 - accuracy: 0.9973
1420/1688 [========================>.....] - ETA: 1s - loss: 0.0105 - accuracy: 0.9973
1431/1688 [========================>.....] - ETA: 1s - loss: 0.0106 - accuracy: 0.9973
1441/1688 [========================>.....] - ETA: 1s - loss: 0.0105 - accuracy: 0.9973
1451/1688 [========================>.....] - ETA: 1s - loss: 0.0106 - accuracy: 0.9973
1462/1688 [========================>.....] - ETA: 1s - loss: 0.0106 - accuracy: 0.9973
1472/1688 [=========================>....] - ETA: 1s - loss: 0.0106 - accuracy: 0.9972
1482/1688 [=========================>....] - ETA: 1s - loss: 0.0106 - accuracy: 0.9973
1492/1688 [=========================>....] - ETA: 0s - loss: 0.0107 - accuracy: 0.9972
1502/1688 [=========================>....] - ETA: 0s - loss: 0.0107 - accuracy: 0.9972
1512/1688 [=========================>....] - ETA: 0s - loss: 0.0107 - accuracy: 0.9972
1523/1688 [==========================>...] - ETA: 0s - loss: 0.0106 - accuracy: 0.9973
1534/1688 [==========================>...] - ETA: 0s - loss: 0.0106 - accuracy: 0.9972
1545/1688 [==========================>...] - ETA: 0s - loss: 0.0106 - accuracy: 0.9973
1555/1688 [==========================>...] - ETA: 0s - loss: 0.0106 - accuracy: 0.9973
1565/1688 [==========================>...] - ETA: 0s - loss: 0.0106 - accuracy: 0.9973
1575/1688 [==========================>...] - ETA: 0s - loss: 0.0106 - accuracy: 0.9973
1586/1688 [===========================>..] - ETA: 0s - loss: 0.0106 - accuracy: 0.9973
1596/1688 [===========================>..] - ETA: 0s - loss: 0.0105 - accuracy: 0.9973
1607/1688 [===========================>..] - ETA: 0s - loss: 0.0105 - accuracy: 0.9973
1617/1688 [===========================>..] - ETA: 0s - loss: 0.0105 - accuracy: 0.9974
1627/1688 [===========================>..] - ETA: 0s - loss: 0.0105 - accuracy: 0.9973
1638/1688 [============================>.] - ETA: 0s - loss: 0.0105 - accuracy: 0.9973
1648/1688 [============================>.] - ETA: 0s - loss: 0.0105 - accuracy: 0.9973
1659/1688 [============================>.] - ETA: 0s - loss: 0.0105 - accuracy: 0.9973
1669/1688 [============================>.] - ETA: 0s - loss: 0.0105 - accuracy: 0.9974
1679/1688 [============================>.] - ETA: 0s - loss: 0.0104 - accuracy: 0.9974
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0104 - accuracy: 0.9974 - val_loss: 0.0525 - val_accuracy: 0.9868
Epoch 5/5
1/1688 [..............................] - ETA: 8s - loss: 0.0016 - accuracy: 1.0000
11/1688 [..............................] - ETA: 8s - loss: 0.0116 - accuracy: 0.9972
21/1688 [..............................] - ETA: 8s - loss: 0.0113 - accuracy: 0.9970
31/1688 [..............................] - ETA: 8s - loss: 0.0106 - accuracy: 0.9980
41/1688 [..............................] - ETA: 8s - loss: 0.0096 - accuracy: 0.9985
51/1688 [..............................] - ETA: 8s - loss: 0.0100 - accuracy: 0.9982
62/1688 [>.............................] - ETA: 8s - loss: 0.0089 - accuracy: 0.9985
73/1688 [>.............................] - ETA: 8s - loss: 0.0088 - accuracy: 0.9987
83/1688 [>.............................] - ETA: 8s - loss: 0.0089 - accuracy: 0.9985
93/1688 [>.............................] - ETA: 8s - loss: 0.0087 - accuracy: 0.9983
103/1688 [>.............................] - ETA: 7s - loss: 0.0093 - accuracy: 0.9976
113/1688 [=>............................] - ETA: 7s - loss: 0.0090 - accuracy: 0.9978
124/1688 [=>............................] - ETA: 7s - loss: 0.0099 - accuracy: 0.9975
134/1688 [=>............................] - ETA: 7s - loss: 0.0095 - accuracy: 0.9977
144/1688 [=>............................] - ETA: 7s - loss: 0.0097 - accuracy: 0.9974
155/1688 [=>............................] - ETA: 7s - loss: 0.0100 - accuracy: 0.9972
165/1688 [=>............................] - ETA: 7s - loss: 0.0103 - accuracy: 0.9972
176/1688 [==>...........................] - ETA: 7s - loss: 0.0100 - accuracy: 0.9973
186/1688 [==>...........................] - ETA: 7s - loss: 0.0097 - accuracy: 0.9975
197/1688 [==>...........................] - ETA: 7s - loss: 0.0103 - accuracy: 0.9973
207/1688 [==>...........................] - ETA: 7s - loss: 0.0101 - accuracy: 0.9974
217/1688 [==>...........................] - ETA: 7s - loss: 0.0101 - accuracy: 0.9973
228/1688 [===>..........................] - ETA: 7s - loss: 0.0101 - accuracy: 0.9974
239/1688 [===>..........................] - ETA: 7s - loss: 0.0098 - accuracy: 0.9975
249/1688 [===>..........................] - ETA: 7s - loss: 0.0096 - accuracy: 0.9976
259/1688 [===>..........................] - ETA: 7s - loss: 0.0095 - accuracy: 0.9976
269/1688 [===>..........................] - ETA: 7s - loss: 0.0096 - accuracy: 0.9976
279/1688 [===>..........................] - ETA: 7s - loss: 0.0097 - accuracy: 0.9975
289/1688 [====>.........................] - ETA: 7s - loss: 0.0095 - accuracy: 0.9976
299/1688 [====>.........................] - ETA: 6s - loss: 0.0095 - accuracy: 0.9977
309/1688 [====>.........................] - ETA: 6s - loss: 0.0094 - accuracy: 0.9978
319/1688 [====>.........................] - ETA: 6s - loss: 0.0093 - accuracy: 0.9978
330/1688 [====>.........................] - ETA: 6s - loss: 0.0092 - accuracy: 0.9978
341/1688 [=====>........................] - ETA: 6s - loss: 0.0091 - accuracy: 0.9979
352/1688 [=====>........................] - ETA: 6s - loss: 0.0091 - accuracy: 0.9979
363/1688 [=====>........................] - ETA: 6s - loss: 0.0093 - accuracy: 0.9978
373/1688 [=====>........................] - ETA: 6s - loss: 0.0092 - accuracy: 0.9978
383/1688 [=====>........................] - ETA: 6s - loss: 0.0091 - accuracy: 0.9979
393/1688 [=====>........................] - ETA: 6s - loss: 0.0091 - accuracy: 0.9979
403/1688 [======>.......................] - ETA: 6s - loss: 0.0094 - accuracy: 0.9978
414/1688 [======>.......................] - ETA: 6s - loss: 0.0094 - accuracy: 0.9977
424/1688 [======>.......................] - ETA: 6s - loss: 0.0095 - accuracy: 0.9976
434/1688 [======>.......................] - ETA: 6s - loss: 0.0096 - accuracy: 0.9976
444/1688 [======>.......................] - ETA: 6s - loss: 0.0095 - accuracy: 0.9976
455/1688 [=======>......................] - ETA: 6s - loss: 0.0097 - accuracy: 0.9976
466/1688 [=======>......................] - ETA: 6s - loss: 0.0097 - accuracy: 0.9976
476/1688 [=======>......................] - ETA: 6s - loss: 0.0097 - accuracy: 0.9976
487/1688 [=======>......................] - ETA: 6s - loss: 0.0098 - accuracy: 0.9976
497/1688 [=======>......................] - ETA: 5s - loss: 0.0097 - accuracy: 0.9976
507/1688 [========>.....................] - ETA: 5s - loss: 0.0098 - accuracy: 0.9976
518/1688 [========>.....................] - ETA: 5s - loss: 0.0097 - accuracy: 0.9976
528/1688 [========>.....................] - ETA: 5s - loss: 0.0099 - accuracy: 0.9976
539/1688 [========>.....................] - ETA: 5s - loss: 0.0099 - accuracy: 0.9976
550/1688 [========>.....................] - ETA: 5s - loss: 0.0099 - accuracy: 0.9976
560/1688 [========>.....................] - ETA: 5s - loss: 0.0099 - accuracy: 0.9976
570/1688 [=========>....................] - ETA: 5s - loss: 0.0099 - accuracy: 0.9976
580/1688 [=========>....................] - ETA: 5s - loss: 0.0099 - accuracy: 0.9977
590/1688 [=========>....................] - ETA: 5s - loss: 0.0098 - accuracy: 0.9977
600/1688 [=========>....................] - ETA: 5s - loss: 0.0099 - accuracy: 0.9977
610/1688 [=========>....................] - ETA: 5s - loss: 0.0098 - accuracy: 0.9977
621/1688 [==========>...................] - ETA: 5s - loss: 0.0097 - accuracy: 0.9978
631/1688 [==========>...................] - ETA: 5s - loss: 0.0097 - accuracy: 0.9978
641/1688 [==========>...................] - ETA: 5s - loss: 0.0097 - accuracy: 0.9978
652/1688 [==========>...................] - ETA: 5s - loss: 0.0098 - accuracy: 0.9978
662/1688 [==========>...................] - ETA: 5s - loss: 0.0098 - accuracy: 0.9978
673/1688 [==========>...................] - ETA: 5s - loss: 0.0099 - accuracy: 0.9978
683/1688 [===========>..................] - ETA: 5s - loss: 0.0099 - accuracy: 0.9978
693/1688 [===========>..................] - ETA: 4s - loss: 0.0099 - accuracy: 0.9977
704/1688 [===========>..................] - ETA: 4s - loss: 0.0100 - accuracy: 0.9977
714/1688 [===========>..................] - ETA: 4s - loss: 0.0100 - accuracy: 0.9977
724/1688 [===========>..................] - ETA: 4s - loss: 0.0101 - accuracy: 0.9977
735/1688 [============>.................] - ETA: 4s - loss: 0.0101 - accuracy: 0.9977
746/1688 [============>.................] - ETA: 4s - loss: 0.0102 - accuracy: 0.9976
756/1688 [============>.................] - ETA: 4s - loss: 0.0103 - accuracy: 0.9975
766/1688 [============>.................] - ETA: 4s - loss: 0.0105 - accuracy: 0.9974
776/1688 [============>.................] - ETA: 4s - loss: 0.0105 - accuracy: 0.9975
786/1688 [============>.................] - ETA: 4s - loss: 0.0106 - accuracy: 0.9974
797/1688 [=============>................] - ETA: 4s - loss: 0.0105 - accuracy: 0.9975
807/1688 [=============>................] - ETA: 4s - loss: 0.0105 - accuracy: 0.9974
817/1688 [=============>................] - ETA: 4s - loss: 0.0105 - accuracy: 0.9974
827/1688 [=============>................] - ETA: 4s - loss: 0.0105 - accuracy: 0.9974
838/1688 [=============>................] - ETA: 4s - loss: 0.0104 - accuracy: 0.9974
848/1688 [==============>...............] - ETA: 4s - loss: 0.0104 - accuracy: 0.9975
858/1688 [==============>...............] - ETA: 4s - loss: 0.0104 - accuracy: 0.9975
869/1688 [==============>...............] - ETA: 4s - loss: 0.0105 - accuracy: 0.9974
879/1688 [==============>...............] - ETA: 4s - loss: 0.0105 - accuracy: 0.9975
890/1688 [==============>...............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9975
901/1688 [===============>..............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9975
911/1688 [===============>..............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9976
921/1688 [===============>..............] - ETA: 3s - loss: 0.0103 - accuracy: 0.9976
931/1688 [===============>..............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9975
941/1688 [===============>..............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9975
951/1688 [===============>..............] - ETA: 3s - loss: 0.0103 - accuracy: 0.9976
961/1688 [================>.............] - ETA: 3s - loss: 0.0102 - accuracy: 0.9976
972/1688 [================>.............] - ETA: 3s - loss: 0.0102 - accuracy: 0.9976
982/1688 [================>.............] - ETA: 3s - loss: 0.0102 - accuracy: 0.9976
992/1688 [================>.............] - ETA: 3s - loss: 0.0103 - accuracy: 0.9976
1002/1688 [================>.............] - ETA: 3s - loss: 0.0103 - accuracy: 0.9976
1012/1688 [================>.............] - ETA: 3s - loss: 0.0102 - accuracy: 0.9976
1023/1688 [=================>............] - ETA: 3s - loss: 0.0102 - accuracy: 0.9976
1033/1688 [=================>............] - ETA: 3s - loss: 0.0102 - accuracy: 0.9976
1044/1688 [=================>............] - ETA: 3s - loss: 0.0101 - accuracy: 0.9976
1054/1688 [=================>............] - ETA: 3s - loss: 0.0101 - accuracy: 0.9976
1064/1688 [=================>............] - ETA: 3s - loss: 0.0101 - accuracy: 0.9976
1074/1688 [==================>...........] - ETA: 3s - loss: 0.0101 - accuracy: 0.9976
1084/1688 [==================>...........] - ETA: 3s - loss: 0.0102 - accuracy: 0.9975
1094/1688 [==================>...........] - ETA: 2s - loss: 0.0101 - accuracy: 0.9976
1105/1688 [==================>...........] - ETA: 2s - loss: 0.0101 - accuracy: 0.9976
1115/1688 [==================>...........] - ETA: 2s - loss: 0.0102 - accuracy: 0.9976
1126/1688 [===================>..........] - ETA: 2s - loss: 0.0101 - accuracy: 0.9976
1136/1688 [===================>..........] - ETA: 2s - loss: 0.0101 - accuracy: 0.9976
1146/1688 [===================>..........] - ETA: 2s - loss: 0.0101 - accuracy: 0.9976
1156/1688 [===================>..........] - ETA: 2s - loss: 0.0101 - accuracy: 0.9976
1167/1688 [===================>..........] - ETA: 2s - loss: 0.0100 - accuracy: 0.9977
1178/1688 [===================>..........] - ETA: 2s - loss: 0.0101 - accuracy: 0.9977
1188/1688 [====================>.........] - ETA: 2s - loss: 0.0101 - accuracy: 0.9976
1198/1688 [====================>.........] - ETA: 2s - loss: 0.0101 - accuracy: 0.9976
1208/1688 [====================>.........] - ETA: 2s - loss: 0.0101 - accuracy: 0.9976
1219/1688 [====================>.........] - ETA: 2s - loss: 0.0101 - accuracy: 0.9976
1229/1688 [====================>.........] - ETA: 2s - loss: 0.0100 - accuracy: 0.9976
1239/1688 [=====================>........] - ETA: 2s - loss: 0.0100 - accuracy: 0.9977
1250/1688 [=====================>........] - ETA: 2s - loss: 0.0100 - accuracy: 0.9977
1261/1688 [=====================>........] - ETA: 2s - loss: 0.0100 - accuracy: 0.9977
1271/1688 [=====================>........] - ETA: 2s - loss: 0.0100 - accuracy: 0.9977
1281/1688 [=====================>........] - ETA: 2s - loss: 0.0100 - accuracy: 0.9977
1291/1688 [=====================>........] - ETA: 1s - loss: 0.0100 - accuracy: 0.9977
1301/1688 [======================>.......] - ETA: 1s - loss: 0.0100 - accuracy: 0.9977
1311/1688 [======================>.......] - ETA: 1s - loss: 0.0100 - accuracy: 0.9977
1321/1688 [======================>.......] - ETA: 1s - loss: 0.0100 - accuracy: 0.9977
1331/1688 [======================>.......] - ETA: 1s - loss: 0.0100 - accuracy: 0.9976
1342/1688 [======================>.......] - ETA: 1s - loss: 0.0100 - accuracy: 0.9976
1352/1688 [=======================>......] - ETA: 1s - loss: 0.0100 - accuracy: 0.9976
1362/1688 [=======================>......] - ETA: 1s - loss: 0.0100 - accuracy: 0.9976
1372/1688 [=======================>......] - ETA: 1s - loss: 0.0101 - accuracy: 0.9976
1382/1688 [=======================>......] - ETA: 1s - loss: 0.0101 - accuracy: 0.9976
1392/1688 [=======================>......] - ETA: 1s - loss: 0.0100 - accuracy: 0.9976
1402/1688 [=======================>......] - ETA: 1s - loss: 0.0100 - accuracy: 0.9976
1412/1688 [========================>.....] - ETA: 1s - loss: 0.0100 - accuracy: 0.9976
1422/1688 [========================>.....] - ETA: 1s - loss: 0.0100 - accuracy: 0.9976
1432/1688 [========================>.....] - ETA: 1s - loss: 0.0100 - accuracy: 0.9976
1443/1688 [========================>.....] - ETA: 1s - loss: 0.0101 - accuracy: 0.9976
1454/1688 [========================>.....] - ETA: 1s - loss: 0.0101 - accuracy: 0.9976
1465/1688 [=========================>....] - ETA: 1s - loss: 0.0101 - accuracy: 0.9976
1475/1688 [=========================>....] - ETA: 1s - loss: 0.0100 - accuracy: 0.9976
1485/1688 [=========================>....] - ETA: 1s - loss: 0.0100 - accuracy: 0.9976
1496/1688 [=========================>....] - ETA: 0s - loss: 0.0100 - accuracy: 0.9976
1506/1688 [=========================>....] - ETA: 0s - loss: 0.0100 - accuracy: 0.9976
1516/1688 [=========================>....] - ETA: 0s - loss: 0.0100 - accuracy: 0.9976
1527/1688 [==========================>...] - ETA: 0s - loss: 0.0100 - accuracy: 0.9976
1538/1688 [==========================>...] - ETA: 0s - loss: 0.0100 - accuracy: 0.9976
1549/1688 [==========================>...] - ETA: 0s - loss: 0.0100 - accuracy: 0.9975
1559/1688 [==========================>...] - ETA: 0s - loss: 0.0101 - accuracy: 0.9975
1569/1688 [==========================>...] - ETA: 0s - loss: 0.0101 - accuracy: 0.9975
1579/1688 [===========================>..] - ETA: 0s - loss: 0.0101 - accuracy: 0.9975
1590/1688 [===========================>..] - ETA: 0s - loss: 0.0101 - accuracy: 0.9975
1600/1688 [===========================>..] - ETA: 0s - loss: 0.0101 - accuracy: 0.9975
1610/1688 [===========================>..] - ETA: 0s - loss: 0.0100 - accuracy: 0.9976
1620/1688 [===========================>..] - ETA: 0s - loss: 0.0100 - accuracy: 0.9976
1630/1688 [===========================>..] - ETA: 0s - loss: 0.0100 - accuracy: 0.9976
1640/1688 [============================>.] - ETA: 0s - loss: 0.0100 - accuracy: 0.9976
1650/1688 [============================>.] - ETA: 0s - loss: 0.0100 - accuracy: 0.9976
1661/1688 [============================>.] - ETA: 0s - loss: 0.0101 - accuracy: 0.9976
1672/1688 [============================>.] - ETA: 0s - loss: 0.0101 - accuracy: 0.9976
1682/1688 [============================>.] - ETA: 0s - loss: 0.0100 - accuracy: 0.9976
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0100 - accuracy: 0.9976 - val_loss: 0.0537 - val_accuracy: 0.9870
<keras.src.callbacks.History object at 0x7ff8cc96e4d0>
score = model_quantized.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after fine tuning:', score)
Test accuracy after fine tuning: 0.986299991607666
3. Convert
3.1 Convert to Akida model
When the quantized model produces satisfactory performance, it can be converted to the native Akida format. The convert function returns a model in Akida format ready for inference.
As with Keras, the summary() method provides a textual representation of the Akida model.
from cnn2snn import convert
model_akida = convert(model_quantized)
model_akida.summary()
Model Summary
______________________________________________
Input shape Output shape Sequences Layers
==============================================
[28, 28, 1] [1, 1, 10] 1 5
______________________________________________
__________________________________________________________________
Layer (type) Output shape Kernel shape
=============== SW/conv2d-dequantizer_2 (Software) ===============
conv2d (InputConv2D) [13, 13, 32] (3, 3, 1, 32)
__________________________________________________________________
depthwise_conv2d (DepthwiseConv2D) [7, 7, 32] (3, 3, 32, 1)
__________________________________________________________________
conv2d_1 (Conv2D) [7, 7, 64] (1, 1, 32, 64)
__________________________________________________________________
dense (Dense1D) [1, 1, 10] (3136, 10)
__________________________________________________________________
dequantizer_2 (Dequantizer) [1, 1, 10] N/A
__________________________________________________________________
3.2. Check performance
accuracy = model_akida.evaluate(x_test, y_test)
print('Test accuracy after conversion:', accuracy)
# For non-regression purposes
assert accuracy > 0.96
Test accuracy after conversion: 0.9858999848365784
3.3 Show predictions for a single image
Display one of the test images, such as the first image in the dataset from above, to visualize the output of the model.
# Test a single example
sample_image = 0
image = x_test[sample_image]
outputs = model_akida.predict(image.reshape(1, 28, 28, 1))
print('Input Label: %i' % y_test[sample_image])
f, axarr = plt.subplots(1, 2)
axarr[0].imshow(x_test[sample_image].reshape((28, 28)), cmap=cm.Greys_r)
axarr[0].set_title('Class %d' % y_test[sample_image])
axarr[1].bar(range(10), outputs.squeeze())
axarr[1].set_xticks(range(10))
plt.show()
print(outputs.squeeze())
Input Label: 7
[ -9.1838665 -6.34692 -3.1393735 -1.8285385 -9.289068
-4.736754 -17.68652 7.017484 -4.695699 -0.92606574]
Consider the output from the model above. As is typical in backprop-trained models, the final layer is a Dense layer with one neuron for each of the 10 classes in the dataset. The goal of training is to maximize the response of the neuron corresponding to the label of each training sample while minimizing the responses of the other neurons.
In the bar chart above, you can see the outputs from all 10 neurons. It is easy to see that neuron 7 responds much more strongly than the others. The first sample is indeed a number 7.
Total running time of the script: (2 minutes 5.790 seconds)