CNN conversion flow tutorial

This tutorial illustrates how to use the CNN2SNN toolkit to convert CNN networks to SNN networks compatible with the Akida NSoC in a few steps. You can refer to our CNN2SNN toolkit user guide for further explanation.

The CNN2SNN tool is based on Keras, TensorFlow high-level API for building and training deep learning models.

Note

Please refer to TensorFlow tf.keras.models module for model creation/import details and TensorFlow Guide for details of how TensorFlow works.

MNIST example below is light enough so you do not need a GPU to run the CNN2SNN tool.

../_images/cnn2snn_flow_small.jpg

1. Load and reshape MNIST dataset

After loading, we make 2 transformations on the dataset:

  1. Reshape the sample content data (x values) into a num_samples x width x height x channels matrix.

Note

At this point, we’ll set aside the raw data for testing our converted model in the Akida Execution Engine later.

  1. Rescale the 8-bit loaded data to the range 0-to-1 for training.

Note

Input data normalization is a common step dealing with CNN (rationale is to keep data in a range that works with selected optimizers, some reading can be found here.

This shift makes almost no difference in the current example, but for some datasets rescaling the absolute values (and also shifting to zero-mean) can make a really major difference.

Also note that we store the scaling values input_scaling for use when preparing the model for the Akida Execution Engine. The implementation of the Akida neural network allows us to completely skip the rescaling step (i.e. the Akida model should be fed with the raw 8-bit values) but that does require information about what scaling was applied prior to training - see below for more details.

import tensorflow as tf
from tensorflow import keras

# Load MNIST dataset
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()

# Reshape x-data
x_train = x_train.reshape(60000, 28, 28, 1)
x_test = x_test.reshape(10000, 28, 28, 1)

# Set aside raw test data for use with Akida Execution Engine later
raw_x_test = x_test.astype('uint8')
raw_y_test = y_test

# Rescale x-data
a = 255
b = 0
input_scaling = (a, b)
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train = (x_train - b) / a
x_test = (x_test - b) / a

2. Model definition

Note that at this stage, there is nothing specific to the Akida NSoC. This start point is very much a completely standard CNN as defined within Keras.

An appropriate model for MNIST (inspired by this example) might look something like the following:

model_keras = keras.models.Sequential([
    keras.layers.Conv2D(filters=32, kernel_size=3, input_shape=(28, 28, 1)),
    keras.layers.MaxPool2D(),
    keras.layers.BatchNormalization(),
    keras.layers.ReLU(),
    keras.layers.Conv2D(filters=64, kernel_size=3, padding='same'),
    keras.layers.MaxPool2D(padding='same'),
    keras.layers.BatchNormalization(),
    keras.layers.ReLU(),
    keras.layers.Flatten(),
    keras.layers.Dense(10)
], 'mnistnet')

model_keras.summary()

Out:

Model: "mnistnet"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
conv2d (Conv2D)              (None, 26, 26, 32)        320
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 13, 13, 32)        0
_________________________________________________________________
batch_normalization (BatchNo (None, 13, 13, 32)        128
_________________________________________________________________
re_lu (ReLU)                 (None, 13, 13, 32)        0
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 13, 13, 64)        18496
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 7, 7, 64)          0
_________________________________________________________________
batch_normalization_1 (Batch (None, 7, 7, 64)          256
_________________________________________________________________
re_lu_1 (ReLU)               (None, 7, 7, 64)          0
_________________________________________________________________
flatten (Flatten)            (None, 3136)              0
_________________________________________________________________
dense (Dense)                (None, 10)                31370
=================================================================
Total params: 50,570
Trainable params: 50,378
Non-trainable params: 192
_________________________________________________________________

The model defined above is compatible for conversion into an Akida model, i.e. the model doesn’t include any layers or operations that aren’t Akida-compatible (please refer to the CNN2SNN toolkit documentation for full details):

  • Standard Conv2D and Dense layers are supported

  • Hidden layers must be followed by a ReLU layer.

  • BatchNormalization must always happen before activations.

  • Convolutional blocks can optionally be followed by a MaxPooling.

3. Model training

Before going any further, train the model and get its performance. The created model should have achieved a test accuracy a little over 99% after 10 epochs.

model_keras.compile(
    loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
    optimizer='adam',
    metrics=['accuracy'])

model_keras.fit(x_train, y_train, epochs=10, validation_split=0.1)

score = model_keras.evaluate(x_test, y_test, verbose=0)
print('Test score:', score[0])
print('Test accuracy:', score[1])

Out:

Epoch 1/10

   1/1688 [..............................] - ETA: 0s - loss: 3.0471 - accuracy: 0.0938
  10/1688 [..............................] - ETA: 8s - loss: 1.5898 - accuracy: 0.4875
  19/1688 [..............................] - ETA: 8s - loss: 1.1297 - accuracy: 0.6414
  28/1688 [..............................] - ETA: 9s - loss: 0.9270 - accuracy: 0.7121
  37/1688 [..............................] - ETA: 9s - loss: 0.8084 - accuracy: 0.7508
  46/1688 [..............................] - ETA: 9s - loss: 0.7124 - accuracy: 0.7792
  55/1688 [..............................] - ETA: 9s - loss: 0.6263 - accuracy: 0.8085
  64/1688 [>.............................] - ETA: 8s - loss: 0.5763 - accuracy: 0.8237
  73/1688 [>.............................] - ETA: 8s - loss: 0.5403 - accuracy: 0.8322
  82/1688 [>.............................] - ETA: 8s - loss: 0.5042 - accuracy: 0.8449
  91/1688 [>.............................] - ETA: 8s - loss: 0.4768 - accuracy: 0.8537
 100/1688 [>.............................] - ETA: 8s - loss: 0.4514 - accuracy: 0.8631
 110/1688 [>.............................] - ETA: 8s - loss: 0.4255 - accuracy: 0.8710
 120/1688 [=>............................] - ETA: 8s - loss: 0.4104 - accuracy: 0.8750
 130/1688 [=>............................] - ETA: 8s - loss: 0.3926 - accuracy: 0.8805
 140/1688 [=>............................] - ETA: 8s - loss: 0.3776 - accuracy: 0.8857
 150/1688 [=>............................] - ETA: 8s - loss: 0.3632 - accuracy: 0.8906
 159/1688 [=>............................] - ETA: 8s - loss: 0.3477 - accuracy: 0.8954
 168/1688 [=>............................] - ETA: 8s - loss: 0.3396 - accuracy: 0.8983
 178/1688 [==>...........................] - ETA: 8s - loss: 0.3296 - accuracy: 0.9013
 188/1688 [==>...........................] - ETA: 8s - loss: 0.3184 - accuracy: 0.9046
 198/1688 [==>...........................] - ETA: 8s - loss: 0.3087 - accuracy: 0.9072
 208/1688 [==>...........................] - ETA: 8s - loss: 0.2989 - accuracy: 0.9102
 218/1688 [==>...........................] - ETA: 8s - loss: 0.2899 - accuracy: 0.9130
 228/1688 [===>..........................] - ETA: 8s - loss: 0.2829 - accuracy: 0.9152
 238/1688 [===>..........................] - ETA: 8s - loss: 0.2738 - accuracy: 0.9183
 248/1688 [===>..........................] - ETA: 7s - loss: 0.2680 - accuracy: 0.9199
 258/1688 [===>..........................] - ETA: 7s - loss: 0.2628 - accuracy: 0.9213
 268/1688 [===>..........................] - ETA: 7s - loss: 0.2551 - accuracy: 0.9237
 278/1688 [===>..........................] - ETA: 7s - loss: 0.2493 - accuracy: 0.9251
 288/1688 [====>.........................] - ETA: 7s - loss: 0.2457 - accuracy: 0.9263
 297/1688 [====>.........................] - ETA: 7s - loss: 0.2411 - accuracy: 0.9275
 307/1688 [====>.........................] - ETA: 7s - loss: 0.2367 - accuracy: 0.9285
 317/1688 [====>.........................] - ETA: 7s - loss: 0.2315 - accuracy: 0.9299
 327/1688 [====>.........................] - ETA: 7s - loss: 0.2284 - accuracy: 0.9305
 336/1688 [====>.........................] - ETA: 7s - loss: 0.2254 - accuracy: 0.9312
 345/1688 [=====>........................] - ETA: 7s - loss: 0.2231 - accuracy: 0.9317
 355/1688 [=====>........................] - ETA: 7s - loss: 0.2192 - accuracy: 0.9329
 365/1688 [=====>........................] - ETA: 7s - loss: 0.2163 - accuracy: 0.9338
 375/1688 [=====>........................] - ETA: 7s - loss: 0.2136 - accuracy: 0.9346
 385/1688 [=====>........................] - ETA: 7s - loss: 0.2103 - accuracy: 0.9353
 395/1688 [======>.......................] - ETA: 7s - loss: 0.2073 - accuracy: 0.9363
 405/1688 [======>.......................] - ETA: 7s - loss: 0.2047 - accuracy: 0.9370
 415/1688 [======>.......................] - ETA: 7s - loss: 0.2014 - accuracy: 0.9381
 425/1688 [======>.......................] - ETA: 6s - loss: 0.1997 - accuracy: 0.9388
 435/1688 [======>.......................] - ETA: 6s - loss: 0.1985 - accuracy: 0.9391
 444/1688 [======>.......................] - ETA: 6s - loss: 0.1963 - accuracy: 0.9400
 454/1688 [=======>......................] - ETA: 6s - loss: 0.1937 - accuracy: 0.9407
 464/1688 [=======>......................] - ETA: 6s - loss: 0.1915 - accuracy: 0.9413
 474/1688 [=======>......................] - ETA: 6s - loss: 0.1888 - accuracy: 0.9422
 483/1688 [=======>......................] - ETA: 6s - loss: 0.1867 - accuracy: 0.9428
 493/1688 [=======>......................] - ETA: 6s - loss: 0.1840 - accuracy: 0.9436
 503/1688 [=======>......................] - ETA: 6s - loss: 0.1818 - accuracy: 0.9443
 513/1688 [========>.....................] - ETA: 6s - loss: 0.1802 - accuracy: 0.9448
 523/1688 [========>.....................] - ETA: 6s - loss: 0.1784 - accuracy: 0.9454
 533/1688 [========>.....................] - ETA: 6s - loss: 0.1762 - accuracy: 0.9461
 543/1688 [========>.....................] - ETA: 6s - loss: 0.1737 - accuracy: 0.9469
 553/1688 [========>.....................] - ETA: 6s - loss: 0.1718 - accuracy: 0.9475
 563/1688 [=========>....................] - ETA: 6s - loss: 0.1701 - accuracy: 0.9480
 572/1688 [=========>....................] - ETA: 6s - loss: 0.1695 - accuracy: 0.9484
 582/1688 [=========>....................] - ETA: 6s - loss: 0.1679 - accuracy: 0.9488
 592/1688 [=========>....................] - ETA: 6s - loss: 0.1658 - accuracy: 0.9494
 601/1688 [=========>....................] - ETA: 6s - loss: 0.1651 - accuracy: 0.9496
 611/1688 [=========>....................] - ETA: 5s - loss: 0.1649 - accuracy: 0.9499
 621/1688 [==========>...................] - ETA: 5s - loss: 0.1632 - accuracy: 0.9503
 631/1688 [==========>...................] - ETA: 5s - loss: 0.1621 - accuracy: 0.9506
 640/1688 [==========>...................] - ETA: 5s - loss: 0.1610 - accuracy: 0.9509
 649/1688 [==========>...................] - ETA: 5s - loss: 0.1595 - accuracy: 0.9513
 658/1688 [==========>...................] - ETA: 5s - loss: 0.1586 - accuracy: 0.9516
 668/1688 [==========>...................] - ETA: 5s - loss: 0.1575 - accuracy: 0.9519
 678/1688 [===========>..................] - ETA: 5s - loss: 0.1560 - accuracy: 0.9524
 687/1688 [===========>..................] - ETA: 5s - loss: 0.1550 - accuracy: 0.9526
 696/1688 [===========>..................] - ETA: 5s - loss: 0.1543 - accuracy: 0.9528
 705/1688 [===========>..................] - ETA: 5s - loss: 0.1527 - accuracy: 0.9532
 714/1688 [===========>..................] - ETA: 5s - loss: 0.1518 - accuracy: 0.9536
 724/1688 [===========>..................] - ETA: 5s - loss: 0.1509 - accuracy: 0.9539
 733/1688 [============>.................] - ETA: 5s - loss: 0.1504 - accuracy: 0.9542
 743/1688 [============>.................] - ETA: 5s - loss: 0.1501 - accuracy: 0.9543
 753/1688 [============>.................] - ETA: 5s - loss: 0.1491 - accuracy: 0.9544
 763/1688 [============>.................] - ETA: 5s - loss: 0.1483 - accuracy: 0.9546
 773/1688 [============>.................] - ETA: 5s - loss: 0.1470 - accuracy: 0.9550
 783/1688 [============>.................] - ETA: 5s - loss: 0.1460 - accuracy: 0.9552
 792/1688 [=============>................] - ETA: 4s - loss: 0.1450 - accuracy: 0.9554
 802/1688 [=============>................] - ETA: 4s - loss: 0.1444 - accuracy: 0.9557
 812/1688 [=============>................] - ETA: 4s - loss: 0.1435 - accuracy: 0.9559
 822/1688 [=============>................] - ETA: 4s - loss: 0.1434 - accuracy: 0.9559
 832/1688 [=============>................] - ETA: 4s - loss: 0.1425 - accuracy: 0.9560
 842/1688 [=============>................] - ETA: 4s - loss: 0.1423 - accuracy: 0.9562
 852/1688 [==============>...............] - ETA: 4s - loss: 0.1418 - accuracy: 0.9564
 862/1688 [==============>...............] - ETA: 4s - loss: 0.1410 - accuracy: 0.9567
 871/1688 [==============>...............] - ETA: 4s - loss: 0.1399 - accuracy: 0.9570
 881/1688 [==============>...............] - ETA: 4s - loss: 0.1396 - accuracy: 0.9572
 890/1688 [==============>...............] - ETA: 4s - loss: 0.1390 - accuracy: 0.9574
 900/1688 [==============>...............] - ETA: 4s - loss: 0.1378 - accuracy: 0.9577
 910/1688 [===============>..............] - ETA: 4s - loss: 0.1369 - accuracy: 0.9581
 920/1688 [===============>..............] - ETA: 4s - loss: 0.1361 - accuracy: 0.9584
 930/1688 [===============>..............] - ETA: 4s - loss: 0.1358 - accuracy: 0.9586
 940/1688 [===============>..............] - ETA: 4s - loss: 0.1349 - accuracy: 0.9589
 950/1688 [===============>..............] - ETA: 4s - loss: 0.1344 - accuracy: 0.9590
 960/1688 [================>.............] - ETA: 4s - loss: 0.1338 - accuracy: 0.9591
 970/1688 [================>.............] - ETA: 3s - loss: 0.1336 - accuracy: 0.9592
 980/1688 [================>.............] - ETA: 3s - loss: 0.1333 - accuracy: 0.9594
 990/1688 [================>.............] - ETA: 3s - loss: 0.1326 - accuracy: 0.9597
1000/1688 [================>.............] - ETA: 3s - loss: 0.1316 - accuracy: 0.9599
1010/1688 [================>.............] - ETA: 3s - loss: 0.1310 - accuracy: 0.9601
1020/1688 [=================>............] - ETA: 3s - loss: 0.1302 - accuracy: 0.9603
1030/1688 [=================>............] - ETA: 3s - loss: 0.1293 - accuracy: 0.9606
1039/1688 [=================>............] - ETA: 3s - loss: 0.1286 - accuracy: 0.9608
1049/1688 [=================>............] - ETA: 3s - loss: 0.1278 - accuracy: 0.9611
1059/1688 [=================>............] - ETA: 3s - loss: 0.1269 - accuracy: 0.9614
1069/1688 [=================>............] - ETA: 3s - loss: 0.1264 - accuracy: 0.9616
1078/1688 [==================>...........] - ETA: 3s - loss: 0.1258 - accuracy: 0.9618
1088/1688 [==================>...........] - ETA: 3s - loss: 0.1254 - accuracy: 0.9619
1098/1688 [==================>...........] - ETA: 3s - loss: 0.1247 - accuracy: 0.9621
1108/1688 [==================>...........] - ETA: 3s - loss: 0.1242 - accuracy: 0.9623
1118/1688 [==================>...........] - ETA: 3s - loss: 0.1235 - accuracy: 0.9625
1128/1688 [===================>..........] - ETA: 3s - loss: 0.1230 - accuracy: 0.9627
1138/1688 [===================>..........] - ETA: 3s - loss: 0.1223 - accuracy: 0.9629
1148/1688 [===================>..........] - ETA: 2s - loss: 0.1219 - accuracy: 0.9630
1158/1688 [===================>..........] - ETA: 2s - loss: 0.1211 - accuracy: 0.9632
1168/1688 [===================>..........] - ETA: 2s - loss: 0.1208 - accuracy: 0.9633
1178/1688 [===================>..........] - ETA: 2s - loss: 0.1203 - accuracy: 0.9634
1188/1688 [====================>.........] - ETA: 2s - loss: 0.1197 - accuracy: 0.9636
1198/1688 [====================>.........] - ETA: 2s - loss: 0.1193 - accuracy: 0.9637
1208/1688 [====================>.........] - ETA: 2s - loss: 0.1189 - accuracy: 0.9639
1218/1688 [====================>.........] - ETA: 2s - loss: 0.1185 - accuracy: 0.9640
1228/1688 [====================>.........] - ETA: 2s - loss: 0.1182 - accuracy: 0.9641
1237/1688 [====================>.........] - ETA: 2s - loss: 0.1175 - accuracy: 0.9643
1246/1688 [=====================>........] - ETA: 2s - loss: 0.1173 - accuracy: 0.9643
1256/1688 [=====================>........] - ETA: 2s - loss: 0.1167 - accuracy: 0.9645
1266/1688 [=====================>........] - ETA: 2s - loss: 0.1164 - accuracy: 0.9646
1276/1688 [=====================>........] - ETA: 2s - loss: 0.1158 - accuracy: 0.9647
1285/1688 [=====================>........] - ETA: 2s - loss: 0.1153 - accuracy: 0.9649
1294/1688 [=====================>........] - ETA: 2s - loss: 0.1148 - accuracy: 0.9650
1304/1688 [======================>.......] - ETA: 2s - loss: 0.1144 - accuracy: 0.9651
1314/1688 [======================>.......] - ETA: 2s - loss: 0.1142 - accuracy: 0.9652
1324/1688 [======================>.......] - ETA: 2s - loss: 0.1139 - accuracy: 0.9654
1334/1688 [======================>.......] - ETA: 1s - loss: 0.1135 - accuracy: 0.9654
1344/1688 [======================>.......] - ETA: 1s - loss: 0.1128 - accuracy: 0.9656
1354/1688 [=======================>......] - ETA: 1s - loss: 0.1123 - accuracy: 0.9658
1364/1688 [=======================>......] - ETA: 1s - loss: 0.1122 - accuracy: 0.9658
1374/1688 [=======================>......] - ETA: 1s - loss: 0.1116 - accuracy: 0.9660
1384/1688 [=======================>......] - ETA: 1s - loss: 0.1112 - accuracy: 0.9661
1394/1688 [=======================>......] - ETA: 1s - loss: 0.1111 - accuracy: 0.9661
1403/1688 [=======================>......] - ETA: 1s - loss: 0.1108 - accuracy: 0.9661
1413/1688 [========================>.....] - ETA: 1s - loss: 0.1101 - accuracy: 0.9663
1422/1688 [========================>.....] - ETA: 1s - loss: 0.1100 - accuracy: 0.9664
1432/1688 [========================>.....] - ETA: 1s - loss: 0.1095 - accuracy: 0.9665
1442/1688 [========================>.....] - ETA: 1s - loss: 0.1090 - accuracy: 0.9668
1452/1688 [========================>.....] - ETA: 1s - loss: 0.1084 - accuracy: 0.9669
1462/1688 [========================>.....] - ETA: 1s - loss: 0.1082 - accuracy: 0.9670
1472/1688 [=========================>....] - ETA: 1s - loss: 0.1081 - accuracy: 0.9671
1482/1688 [=========================>....] - ETA: 1s - loss: 0.1082 - accuracy: 0.9671
1492/1688 [=========================>....] - ETA: 1s - loss: 0.1081 - accuracy: 0.9672
1502/1688 [=========================>....] - ETA: 1s - loss: 0.1078 - accuracy: 0.9673
1512/1688 [=========================>....] - ETA: 0s - loss: 0.1074 - accuracy: 0.9674
1522/1688 [==========================>...] - ETA: 0s - loss: 0.1071 - accuracy: 0.9675
1531/1688 [==========================>...] - ETA: 0s - loss: 0.1069 - accuracy: 0.9676
1541/1688 [==========================>...] - ETA: 0s - loss: 0.1069 - accuracy: 0.9677
1551/1688 [==========================>...] - ETA: 0s - loss: 0.1067 - accuracy: 0.9678
1561/1688 [==========================>...] - ETA: 0s - loss: 0.1064 - accuracy: 0.9678
1571/1688 [==========================>...] - ETA: 0s - loss: 0.1064 - accuracy: 0.9679
1581/1688 [===========================>..] - ETA: 0s - loss: 0.1063 - accuracy: 0.9680
1591/1688 [===========================>..] - ETA: 0s - loss: 0.1060 - accuracy: 0.9681
1601/1688 [===========================>..] - ETA: 0s - loss: 0.1057 - accuracy: 0.9681
1611/1688 [===========================>..] - ETA: 0s - loss: 0.1055 - accuracy: 0.9682
1620/1688 [===========================>..] - ETA: 0s - loss: 0.1053 - accuracy: 0.9682
1630/1688 [===========================>..] - ETA: 0s - loss: 0.1049 - accuracy: 0.9683
1640/1688 [============================>.] - ETA: 0s - loss: 0.1046 - accuracy: 0.9683
1650/1688 [============================>.] - ETA: 0s - loss: 0.1045 - accuracy: 0.9684
1660/1688 [============================>.] - ETA: 0s - loss: 0.1043 - accuracy: 0.9684
1670/1688 [============================>.] - ETA: 0s - loss: 0.1041 - accuracy: 0.9685
1680/1688 [============================>.] - ETA: 0s - loss: 0.1040 - accuracy: 0.9684
1688/1688 [==============================] - 10s 6ms/step - loss: 0.1037 - accuracy: 0.9685 - val_loss: 0.0444 - val_accuracy: 0.9867
Epoch 2/10

   1/1688 [..............................] - ETA: 0s - loss: 0.0025 - accuracy: 1.0000
  11/1688 [..............................] - ETA: 8s - loss: 0.0590 - accuracy: 0.9858
  21/1688 [..............................] - ETA: 8s - loss: 0.0500 - accuracy: 0.9866
  31/1688 [..............................] - ETA: 8s - loss: 0.0426 - accuracy: 0.9879
  40/1688 [..............................] - ETA: 8s - loss: 0.0585 - accuracy: 0.9844
  50/1688 [..............................] - ETA: 8s - loss: 0.0515 - accuracy: 0.9862
  60/1688 [>.............................] - ETA: 8s - loss: 0.0562 - accuracy: 0.9844
  70/1688 [>.............................] - ETA: 8s - loss: 0.0538 - accuracy: 0.9848
  80/1688 [>.............................] - ETA: 8s - loss: 0.0505 - accuracy: 0.9852
  90/1688 [>.............................] - ETA: 8s - loss: 0.0494 - accuracy: 0.9851
 100/1688 [>.............................] - ETA: 8s - loss: 0.0486 - accuracy: 0.9853
 110/1688 [>.............................] - ETA: 8s - loss: 0.0494 - accuracy: 0.9855
 120/1688 [=>............................] - ETA: 8s - loss: 0.0525 - accuracy: 0.9849
 129/1688 [=>............................] - ETA: 8s - loss: 0.0494 - accuracy: 0.9859
 139/1688 [=>............................] - ETA: 8s - loss: 0.0484 - accuracy: 0.9861
 149/1688 [=>............................] - ETA: 8s - loss: 0.0477 - accuracy: 0.9859
 159/1688 [=>............................] - ETA: 8s - loss: 0.0457 - accuracy: 0.9864
 169/1688 [==>...........................] - ETA: 8s - loss: 0.0438 - accuracy: 0.9871
 179/1688 [==>...........................] - ETA: 8s - loss: 0.0425 - accuracy: 0.9874
 189/1688 [==>...........................] - ETA: 8s - loss: 0.0422 - accuracy: 0.9876
 198/1688 [==>...........................] - ETA: 8s - loss: 0.0410 - accuracy: 0.9880
 208/1688 [==>...........................] - ETA: 8s - loss: 0.0402 - accuracy: 0.9881
 217/1688 [==>...........................] - ETA: 8s - loss: 0.0413 - accuracy: 0.9878
 226/1688 [===>..........................] - ETA: 8s - loss: 0.0431 - accuracy: 0.9873
 236/1688 [===>..........................] - ETA: 7s - loss: 0.0424 - accuracy: 0.9873
 246/1688 [===>..........................] - ETA: 7s - loss: 0.0417 - accuracy: 0.9874
 256/1688 [===>..........................] - ETA: 7s - loss: 0.0427 - accuracy: 0.9871
 266/1688 [===>..........................] - ETA: 7s - loss: 0.0425 - accuracy: 0.9871
 276/1688 [===>..........................] - ETA: 7s - loss: 0.0428 - accuracy: 0.9870
 286/1688 [====>.........................] - ETA: 7s - loss: 0.0425 - accuracy: 0.9868
 296/1688 [====>.........................] - ETA: 7s - loss: 0.0418 - accuracy: 0.9869
 306/1688 [====>.........................] - ETA: 7s - loss: 0.0429 - accuracy: 0.9865
 316/1688 [====>.........................] - ETA: 7s - loss: 0.0435 - accuracy: 0.9865
 326/1688 [====>.........................] - ETA: 7s - loss: 0.0442 - accuracy: 0.9865
 336/1688 [====>.........................] - ETA: 7s - loss: 0.0448 - accuracy: 0.9862
 346/1688 [=====>........................] - ETA: 7s - loss: 0.0442 - accuracy: 0.9865
 356/1688 [=====>........................] - ETA: 7s - loss: 0.0443 - accuracy: 0.9865
 366/1688 [=====>........................] - ETA: 7s - loss: 0.0450 - accuracy: 0.9863
 376/1688 [=====>........................] - ETA: 7s - loss: 0.0442 - accuracy: 0.9865
 386/1688 [=====>........................] - ETA: 7s - loss: 0.0441 - accuracy: 0.9866
 396/1688 [======>.......................] - ETA: 7s - loss: 0.0441 - accuracy: 0.9866
 406/1688 [======>.......................] - ETA: 7s - loss: 0.0444 - accuracy: 0.9866
 415/1688 [======>.......................] - ETA: 7s - loss: 0.0437 - accuracy: 0.9868
 425/1688 [======>.......................] - ETA: 6s - loss: 0.0434 - accuracy: 0.9869
 435/1688 [======>.......................] - ETA: 6s - loss: 0.0432 - accuracy: 0.9869
 444/1688 [======>.......................] - ETA: 6s - loss: 0.0425 - accuracy: 0.9870
 454/1688 [=======>......................] - ETA: 6s - loss: 0.0421 - accuracy: 0.9869
 464/1688 [=======>......................] - ETA: 6s - loss: 0.0425 - accuracy: 0.9868
 473/1688 [=======>......................] - ETA: 6s - loss: 0.0431 - accuracy: 0.9867
 483/1688 [=======>......................] - ETA: 6s - loss: 0.0439 - accuracy: 0.9865
 493/1688 [=======>......................] - ETA: 6s - loss: 0.0442 - accuracy: 0.9864
 502/1688 [=======>......................] - ETA: 6s - loss: 0.0443 - accuracy: 0.9864
 512/1688 [========>.....................] - ETA: 6s - loss: 0.0444 - accuracy: 0.9864
 521/1688 [========>.....................] - ETA: 6s - loss: 0.0440 - accuracy: 0.9864
 531/1688 [========>.....................] - ETA: 6s - loss: 0.0442 - accuracy: 0.9862
 541/1688 [========>.....................] - ETA: 6s - loss: 0.0437 - accuracy: 0.9864
 551/1688 [========>.....................] - ETA: 6s - loss: 0.0437 - accuracy: 0.9863
 561/1688 [========>.....................] - ETA: 6s - loss: 0.0437 - accuracy: 0.9864
 571/1688 [=========>....................] - ETA: 6s - loss: 0.0434 - accuracy: 0.9864
 580/1688 [=========>....................] - ETA: 6s - loss: 0.0434 - accuracy: 0.9864
 590/1688 [=========>....................] - ETA: 6s - loss: 0.0434 - accuracy: 0.9862
 599/1688 [=========>....................] - ETA: 6s - loss: 0.0432 - accuracy: 0.9863
 609/1688 [=========>....................] - ETA: 5s - loss: 0.0432 - accuracy: 0.9864
 619/1688 [==========>...................] - ETA: 5s - loss: 0.0431 - accuracy: 0.9863
 628/1688 [==========>...................] - ETA: 5s - loss: 0.0431 - accuracy: 0.9862
 638/1688 [==========>...................] - ETA: 5s - loss: 0.0430 - accuracy: 0.9863
 648/1688 [==========>...................] - ETA: 5s - loss: 0.0432 - accuracy: 0.9863
 658/1688 [==========>...................] - ETA: 5s - loss: 0.0428 - accuracy: 0.9864
 668/1688 [==========>...................] - ETA: 5s - loss: 0.0434 - accuracy: 0.9864
 678/1688 [===========>..................] - ETA: 5s - loss: 0.0430 - accuracy: 0.9864
 688/1688 [===========>..................] - ETA: 5s - loss: 0.0428 - accuracy: 0.9866
 698/1688 [===========>..................] - ETA: 5s - loss: 0.0423 - accuracy: 0.9867
 708/1688 [===========>..................] - ETA: 5s - loss: 0.0419 - accuracy: 0.9868
 718/1688 [===========>..................] - ETA: 5s - loss: 0.0417 - accuracy: 0.9869
 728/1688 [===========>..................] - ETA: 5s - loss: 0.0418 - accuracy: 0.9870
 738/1688 [============>.................] - ETA: 5s - loss: 0.0419 - accuracy: 0.9869
 748/1688 [============>.................] - ETA: 5s - loss: 0.0421 - accuracy: 0.9868
 758/1688 [============>.................] - ETA: 5s - loss: 0.0421 - accuracy: 0.9868
 768/1688 [============>.................] - ETA: 5s - loss: 0.0420 - accuracy: 0.9868
 778/1688 [============>.................] - ETA: 5s - loss: 0.0421 - accuracy: 0.9868
 788/1688 [=============>................] - ETA: 4s - loss: 0.0421 - accuracy: 0.9868
 798/1688 [=============>................] - ETA: 4s - loss: 0.0423 - accuracy: 0.9867
 808/1688 [=============>................] - ETA: 4s - loss: 0.0426 - accuracy: 0.9865
 818/1688 [=============>................] - ETA: 4s - loss: 0.0423 - accuracy: 0.9866
 828/1688 [=============>................] - ETA: 4s - loss: 0.0428 - accuracy: 0.9866
 838/1688 [=============>................] - ETA: 4s - loss: 0.0428 - accuracy: 0.9866
 848/1688 [==============>...............] - ETA: 4s - loss: 0.0430 - accuracy: 0.9865
 857/1688 [==============>...............] - ETA: 4s - loss: 0.0432 - accuracy: 0.9865
 867/1688 [==============>...............] - ETA: 4s - loss: 0.0432 - accuracy: 0.9866
 877/1688 [==============>...............] - ETA: 4s - loss: 0.0432 - accuracy: 0.9866
 886/1688 [==============>...............] - ETA: 4s - loss: 0.0430 - accuracy: 0.9866
 896/1688 [==============>...............] - ETA: 4s - loss: 0.0426 - accuracy: 0.9867
 905/1688 [===============>..............] - ETA: 4s - loss: 0.0425 - accuracy: 0.9867
 915/1688 [===============>..............] - ETA: 4s - loss: 0.0425 - accuracy: 0.9867
 925/1688 [===============>..............] - ETA: 4s - loss: 0.0426 - accuracy: 0.9866
 934/1688 [===============>..............] - ETA: 4s - loss: 0.0426 - accuracy: 0.9865
 944/1688 [===============>..............] - ETA: 4s - loss: 0.0427 - accuracy: 0.9865
 954/1688 [===============>..............] - ETA: 4s - loss: 0.0427 - accuracy: 0.9865
 964/1688 [================>.............] - ETA: 3s - loss: 0.0427 - accuracy: 0.9865
 974/1688 [================>.............] - ETA: 3s - loss: 0.0425 - accuracy: 0.9866
 984/1688 [================>.............] - ETA: 3s - loss: 0.0426 - accuracy: 0.9866
 994/1688 [================>.............] - ETA: 3s - loss: 0.0429 - accuracy: 0.9865
1004/1688 [================>.............] - ETA: 3s - loss: 0.0427 - accuracy: 0.9865
1014/1688 [=================>............] - ETA: 3s - loss: 0.0430 - accuracy: 0.9864
1024/1688 [=================>............] - ETA: 3s - loss: 0.0431 - accuracy: 0.9864
1034/1688 [=================>............] - ETA: 3s - loss: 0.0430 - accuracy: 0.9864
1044/1688 [=================>............] - ETA: 3s - loss: 0.0431 - accuracy: 0.9864
1054/1688 [=================>............] - ETA: 3s - loss: 0.0430 - accuracy: 0.9864
1064/1688 [=================>............] - ETA: 3s - loss: 0.0431 - accuracy: 0.9863
1074/1688 [==================>...........] - ETA: 3s - loss: 0.0432 - accuracy: 0.9863
1084/1688 [==================>...........] - ETA: 3s - loss: 0.0433 - accuracy: 0.9862
1094/1688 [==================>...........] - ETA: 3s - loss: 0.0438 - accuracy: 0.9862
1104/1688 [==================>...........] - ETA: 3s - loss: 0.0437 - accuracy: 0.9862
1114/1688 [==================>...........] - ETA: 3s - loss: 0.0434 - accuracy: 0.9864
1124/1688 [==================>...........] - ETA: 3s - loss: 0.0435 - accuracy: 0.9864
1133/1688 [===================>..........] - ETA: 3s - loss: 0.0436 - accuracy: 0.9864
1143/1688 [===================>..........] - ETA: 3s - loss: 0.0434 - accuracy: 0.9864
1153/1688 [===================>..........] - ETA: 2s - loss: 0.0433 - accuracy: 0.9865
1163/1688 [===================>..........] - ETA: 2s - loss: 0.0433 - accuracy: 0.9865
1173/1688 [===================>..........] - ETA: 2s - loss: 0.0432 - accuracy: 0.9865
1183/1688 [====================>.........] - ETA: 2s - loss: 0.0434 - accuracy: 0.9865
1193/1688 [====================>.........] - ETA: 2s - loss: 0.0435 - accuracy: 0.9864
1203/1688 [====================>.........] - ETA: 2s - loss: 0.0438 - accuracy: 0.9863
1213/1688 [====================>.........] - ETA: 2s - loss: 0.0437 - accuracy: 0.9863
1222/1688 [====================>.........] - ETA: 2s - loss: 0.0436 - accuracy: 0.9864
1231/1688 [====================>.........] - ETA: 2s - loss: 0.0435 - accuracy: 0.9863
1241/1688 [=====================>........] - ETA: 2s - loss: 0.0434 - accuracy: 0.9864
1251/1688 [=====================>........] - ETA: 2s - loss: 0.0435 - accuracy: 0.9863
1261/1688 [=====================>........] - ETA: 2s - loss: 0.0434 - accuracy: 0.9863
1271/1688 [=====================>........] - ETA: 2s - loss: 0.0433 - accuracy: 0.9864
1281/1688 [=====================>........] - ETA: 2s - loss: 0.0436 - accuracy: 0.9863
1291/1688 [=====================>........] - ETA: 2s - loss: 0.0435 - accuracy: 0.9864
1301/1688 [======================>.......] - ETA: 2s - loss: 0.0434 - accuracy: 0.9865
1310/1688 [======================>.......] - ETA: 2s - loss: 0.0437 - accuracy: 0.9863
1319/1688 [======================>.......] - ETA: 2s - loss: 0.0440 - accuracy: 0.9862
1329/1688 [======================>.......] - ETA: 1s - loss: 0.0440 - accuracy: 0.9863
1339/1688 [======================>.......] - ETA: 1s - loss: 0.0439 - accuracy: 0.9863
1349/1688 [======================>.......] - ETA: 1s - loss: 0.0442 - accuracy: 0.9861
1358/1688 [=======================>......] - ETA: 1s - loss: 0.0444 - accuracy: 0.9861
1368/1688 [=======================>......] - ETA: 1s - loss: 0.0445 - accuracy: 0.9860
1377/1688 [=======================>......] - ETA: 1s - loss: 0.0445 - accuracy: 0.9860
1387/1688 [=======================>......] - ETA: 1s - loss: 0.0448 - accuracy: 0.9859
1397/1688 [=======================>......] - ETA: 1s - loss: 0.0450 - accuracy: 0.9858
1406/1688 [=======================>......] - ETA: 1s - loss: 0.0451 - accuracy: 0.9858
1416/1688 [========================>.....] - ETA: 1s - loss: 0.0451 - accuracy: 0.9859
1426/1688 [========================>.....] - ETA: 1s - loss: 0.0451 - accuracy: 0.9858
1436/1688 [========================>.....] - ETA: 1s - loss: 0.0452 - accuracy: 0.9858
1446/1688 [========================>.....] - ETA: 1s - loss: 0.0452 - accuracy: 0.9858
1456/1688 [========================>.....] - ETA: 1s - loss: 0.0454 - accuracy: 0.9858
1466/1688 [=========================>....] - ETA: 1s - loss: 0.0453 - accuracy: 0.9858
1476/1688 [=========================>....] - ETA: 1s - loss: 0.0452 - accuracy: 0.9858
1486/1688 [=========================>....] - ETA: 1s - loss: 0.0451 - accuracy: 0.9859
1496/1688 [=========================>....] - ETA: 1s - loss: 0.0452 - accuracy: 0.9858
1505/1688 [=========================>....] - ETA: 1s - loss: 0.0451 - accuracy: 0.9859
1515/1688 [=========================>....] - ETA: 0s - loss: 0.0450 - accuracy: 0.9859
1525/1688 [==========================>...] - ETA: 0s - loss: 0.0450 - accuracy: 0.9859
1535/1688 [==========================>...] - ETA: 0s - loss: 0.0448 - accuracy: 0.9859
1545/1688 [==========================>...] - ETA: 0s - loss: 0.0449 - accuracy: 0.9858
1555/1688 [==========================>...] - ETA: 0s - loss: 0.0449 - accuracy: 0.9858
1564/1688 [==========================>...] - ETA: 0s - loss: 0.0449 - accuracy: 0.9858
1574/1688 [==========================>...] - ETA: 0s - loss: 0.0448 - accuracy: 0.9859
1583/1688 [===========================>..] - ETA: 0s - loss: 0.0447 - accuracy: 0.9859
1593/1688 [===========================>..] - ETA: 0s - loss: 0.0448 - accuracy: 0.9858
1603/1688 [===========================>..] - ETA: 0s - loss: 0.0446 - accuracy: 0.9859
1613/1688 [===========================>..] - ETA: 0s - loss: 0.0444 - accuracy: 0.9859
1623/1688 [===========================>..] - ETA: 0s - loss: 0.0443 - accuracy: 0.9860
1633/1688 [============================>.] - ETA: 0s - loss: 0.0441 - accuracy: 0.9860
1643/1688 [============================>.] - ETA: 0s - loss: 0.0439 - accuracy: 0.9861
1653/1688 [============================>.] - ETA: 0s - loss: 0.0440 - accuracy: 0.9861
1663/1688 [============================>.] - ETA: 0s - loss: 0.0440 - accuracy: 0.9860
1673/1688 [============================>.] - ETA: 0s - loss: 0.0438 - accuracy: 0.9861
1682/1688 [============================>.] - ETA: 0s - loss: 0.0441 - accuracy: 0.9861
1688/1688 [==============================] - 10s 6ms/step - loss: 0.0441 - accuracy: 0.9861 - val_loss: 0.0402 - val_accuracy: 0.9872
Epoch 3/10

   1/1688 [..............................] - ETA: 0s - loss: 0.0288 - accuracy: 0.9688
  11/1688 [..............................] - ETA: 8s - loss: 0.0326 - accuracy: 0.9915
  21/1688 [..............................] - ETA: 8s - loss: 0.0319 - accuracy: 0.9911
  31/1688 [..............................] - ETA: 8s - loss: 0.0295 - accuracy: 0.9919
  40/1688 [..............................] - ETA: 8s - loss: 0.0270 - accuracy: 0.9922
  49/1688 [..............................] - ETA: 8s - loss: 0.0290 - accuracy: 0.9917
  59/1688 [>.............................] - ETA: 8s - loss: 0.0267 - accuracy: 0.9921
  69/1688 [>.............................] - ETA: 8s - loss: 0.0268 - accuracy: 0.9923
  78/1688 [>.............................] - ETA: 8s - loss: 0.0282 - accuracy: 0.9912
  88/1688 [>.............................] - ETA: 8s - loss: 0.0279 - accuracy: 0.9911
  98/1688 [>.............................] - ETA: 8s - loss: 0.0263 - accuracy: 0.9920
 108/1688 [>.............................] - ETA: 8s - loss: 0.0270 - accuracy: 0.9919
 118/1688 [=>............................] - ETA: 8s - loss: 0.0265 - accuracy: 0.9921
 127/1688 [=>............................] - ETA: 8s - loss: 0.0290 - accuracy: 0.9914
 137/1688 [=>............................] - ETA: 8s - loss: 0.0305 - accuracy: 0.9904
 147/1688 [=>............................] - ETA: 8s - loss: 0.0317 - accuracy: 0.9898
 157/1688 [=>............................] - ETA: 8s - loss: 0.0305 - accuracy: 0.9902
 167/1688 [=>............................] - ETA: 8s - loss: 0.0310 - accuracy: 0.9899
 176/1688 [==>...........................] - ETA: 8s - loss: 0.0305 - accuracy: 0.9899
 186/1688 [==>...........................] - ETA: 8s - loss: 0.0297 - accuracy: 0.9901
 196/1688 [==>...........................] - ETA: 8s - loss: 0.0289 - accuracy: 0.9904
 205/1688 [==>...........................] - ETA: 8s - loss: 0.0287 - accuracy: 0.9905
 215/1688 [==>...........................] - ETA: 8s - loss: 0.0285 - accuracy: 0.9906
 225/1688 [==>...........................] - ETA: 8s - loss: 0.0289 - accuracy: 0.9906
 235/1688 [===>..........................] - ETA: 7s - loss: 0.0287 - accuracy: 0.9906
 245/1688 [===>..........................] - ETA: 7s - loss: 0.0292 - accuracy: 0.9907
 255/1688 [===>..........................] - ETA: 7s - loss: 0.0300 - accuracy: 0.9908
 265/1688 [===>..........................] - ETA: 7s - loss: 0.0296 - accuracy: 0.9909
 275/1688 [===>..........................] - ETA: 7s - loss: 0.0291 - accuracy: 0.9911
 285/1688 [====>.........................] - ETA: 7s - loss: 0.0285 - accuracy: 0.9913
 295/1688 [====>.........................] - ETA: 7s - loss: 0.0285 - accuracy: 0.9912
 304/1688 [====>.........................] - ETA: 7s - loss: 0.0286 - accuracy: 0.9911
 314/1688 [====>.........................] - ETA: 7s - loss: 0.0289 - accuracy: 0.9911
 324/1688 [====>.........................] - ETA: 7s - loss: 0.0284 - accuracy: 0.9911
 334/1688 [====>.........................] - ETA: 7s - loss: 0.0278 - accuracy: 0.9913
 344/1688 [=====>........................] - ETA: 7s - loss: 0.0277 - accuracy: 0.9913
 354/1688 [=====>........................] - ETA: 7s - loss: 0.0272 - accuracy: 0.9914
 364/1688 [=====>........................] - ETA: 7s - loss: 0.0272 - accuracy: 0.9914
 374/1688 [=====>........................] - ETA: 7s - loss: 0.0267 - accuracy: 0.9916
 384/1688 [=====>........................] - ETA: 7s - loss: 0.0277 - accuracy: 0.9915
 394/1688 [======>.......................] - ETA: 7s - loss: 0.0275 - accuracy: 0.9916
 404/1688 [======>.......................] - ETA: 7s - loss: 0.0272 - accuracy: 0.9916
 414/1688 [======>.......................] - ETA: 7s - loss: 0.0271 - accuracy: 0.9915
 424/1688 [======>.......................] - ETA: 6s - loss: 0.0267 - accuracy: 0.9915
 434/1688 [======>.......................] - ETA: 6s - loss: 0.0268 - accuracy: 0.9916
 444/1688 [======>.......................] - ETA: 6s - loss: 0.0264 - accuracy: 0.9916
 454/1688 [=======>......................] - ETA: 6s - loss: 0.0264 - accuracy: 0.9916
 464/1688 [=======>......................] - ETA: 6s - loss: 0.0259 - accuracy: 0.9917
 474/1688 [=======>......................] - ETA: 6s - loss: 0.0260 - accuracy: 0.9916
 483/1688 [=======>......................] - ETA: 6s - loss: 0.0261 - accuracy: 0.9915
 493/1688 [=======>......................] - ETA: 6s - loss: 0.0261 - accuracy: 0.9916
 503/1688 [=======>......................] - ETA: 6s - loss: 0.0258 - accuracy: 0.9917
 513/1688 [========>.....................] - ETA: 6s - loss: 0.0261 - accuracy: 0.9916
 523/1688 [========>.....................] - ETA: 6s - loss: 0.0262 - accuracy: 0.9916
 533/1688 [========>.....................] - ETA: 6s - loss: 0.0265 - accuracy: 0.9915
 543/1688 [========>.....................] - ETA: 6s - loss: 0.0264 - accuracy: 0.9915
 552/1688 [========>.....................] - ETA: 6s - loss: 0.0266 - accuracy: 0.9913
 561/1688 [========>.....................] - ETA: 6s - loss: 0.0263 - accuracy: 0.9915
 570/1688 [=========>....................] - ETA: 6s - loss: 0.0264 - accuracy: 0.9915
 579/1688 [=========>....................] - ETA: 6s - loss: 0.0265 - accuracy: 0.9914
 588/1688 [=========>....................] - ETA: 6s - loss: 0.0268 - accuracy: 0.9913
 597/1688 [=========>....................] - ETA: 6s - loss: 0.0265 - accuracy: 0.9914
 606/1688 [=========>....................] - ETA: 5s - loss: 0.0264 - accuracy: 0.9914
 616/1688 [=========>....................] - ETA: 5s - loss: 0.0262 - accuracy: 0.9914
 626/1688 [==========>...................] - ETA: 5s - loss: 0.0263 - accuracy: 0.9912
 636/1688 [==========>...................] - ETA: 5s - loss: 0.0262 - accuracy: 0.9912
 646/1688 [==========>...................] - ETA: 5s - loss: 0.0260 - accuracy: 0.9912
 656/1688 [==========>...................] - ETA: 5s - loss: 0.0261 - accuracy: 0.9912
 666/1688 [==========>...................] - ETA: 5s - loss: 0.0261 - accuracy: 0.9913
 676/1688 [===========>..................] - ETA: 5s - loss: 0.0262 - accuracy: 0.9913
 686/1688 [===========>..................] - ETA: 5s - loss: 0.0270 - accuracy: 0.9910
 696/1688 [===========>..................] - ETA: 5s - loss: 0.0271 - accuracy: 0.9909
 706/1688 [===========>..................] - ETA: 5s - loss: 0.0271 - accuracy: 0.9908
 716/1688 [===========>..................] - ETA: 5s - loss: 0.0272 - accuracy: 0.9909
 726/1688 [===========>..................] - ETA: 5s - loss: 0.0273 - accuracy: 0.9909
 735/1688 [============>.................] - ETA: 5s - loss: 0.0271 - accuracy: 0.9910
 744/1688 [============>.................] - ETA: 5s - loss: 0.0271 - accuracy: 0.9910
 754/1688 [============>.................] - ETA: 5s - loss: 0.0271 - accuracy: 0.9910
 763/1688 [============>.................] - ETA: 5s - loss: 0.0270 - accuracy: 0.9910
 772/1688 [============>.................] - ETA: 5s - loss: 0.0272 - accuracy: 0.9909
 782/1688 [============>.................] - ETA: 5s - loss: 0.0270 - accuracy: 0.9910
 792/1688 [=============>................] - ETA: 4s - loss: 0.0271 - accuracy: 0.9910
 802/1688 [=============>................] - ETA: 4s - loss: 0.0275 - accuracy: 0.9910
 812/1688 [=============>................] - ETA: 4s - loss: 0.0277 - accuracy: 0.9909
 822/1688 [=============>................] - ETA: 4s - loss: 0.0277 - accuracy: 0.9909
 832/1688 [=============>................] - ETA: 4s - loss: 0.0276 - accuracy: 0.9909
 842/1688 [=============>................] - ETA: 4s - loss: 0.0276 - accuracy: 0.9909
 851/1688 [==============>...............] - ETA: 4s - loss: 0.0274 - accuracy: 0.9910
 861/1688 [==============>...............] - ETA: 4s - loss: 0.0274 - accuracy: 0.9910
 871/1688 [==============>...............] - ETA: 4s - loss: 0.0277 - accuracy: 0.9910
 881/1688 [==============>...............] - ETA: 4s - loss: 0.0277 - accuracy: 0.9910
 891/1688 [==============>...............] - ETA: 4s - loss: 0.0280 - accuracy: 0.9910
 901/1688 [===============>..............] - ETA: 4s - loss: 0.0282 - accuracy: 0.9908
 910/1688 [===============>..............] - ETA: 4s - loss: 0.0284 - accuracy: 0.9908
 919/1688 [===============>..............] - ETA: 4s - loss: 0.0284 - accuracy: 0.9908
 929/1688 [===============>..............] - ETA: 4s - loss: 0.0282 - accuracy: 0.9909
 939/1688 [===============>..............] - ETA: 4s - loss: 0.0280 - accuracy: 0.9909
 948/1688 [===============>..............] - ETA: 4s - loss: 0.0279 - accuracy: 0.9909
 958/1688 [================>.............] - ETA: 4s - loss: 0.0281 - accuracy: 0.9909
 968/1688 [================>.............] - ETA: 3s - loss: 0.0280 - accuracy: 0.9910
 978/1688 [================>.............] - ETA: 3s - loss: 0.0278 - accuracy: 0.9910
 988/1688 [================>.............] - ETA: 3s - loss: 0.0277 - accuracy: 0.9910
 998/1688 [================>.............] - ETA: 3s - loss: 0.0277 - accuracy: 0.9911
1008/1688 [================>.............] - ETA: 3s - loss: 0.0280 - accuracy: 0.9909
1018/1688 [=================>............] - ETA: 3s - loss: 0.0281 - accuracy: 0.9909
1027/1688 [=================>............] - ETA: 3s - loss: 0.0283 - accuracy: 0.9908
1037/1688 [=================>............] - ETA: 3s - loss: 0.0282 - accuracy: 0.9908
1047/1688 [=================>............] - ETA: 3s - loss: 0.0283 - accuracy: 0.9907
1057/1688 [=================>............] - ETA: 3s - loss: 0.0282 - accuracy: 0.9908
1067/1688 [=================>............] - ETA: 3s - loss: 0.0280 - accuracy: 0.9908
1076/1688 [==================>...........] - ETA: 3s - loss: 0.0280 - accuracy: 0.9909
1086/1688 [==================>...........] - ETA: 3s - loss: 0.0280 - accuracy: 0.9909
1096/1688 [==================>...........] - ETA: 3s - loss: 0.0279 - accuracy: 0.9909
1106/1688 [==================>...........] - ETA: 3s - loss: 0.0280 - accuracy: 0.9908
1116/1688 [==================>...........] - ETA: 3s - loss: 0.0280 - accuracy: 0.9908
1126/1688 [===================>..........] - ETA: 3s - loss: 0.0278 - accuracy: 0.9909
1136/1688 [===================>..........] - ETA: 3s - loss: 0.0277 - accuracy: 0.9909
1146/1688 [===================>..........] - ETA: 2s - loss: 0.0277 - accuracy: 0.9909
1156/1688 [===================>..........] - ETA: 2s - loss: 0.0277 - accuracy: 0.9909
1165/1688 [===================>..........] - ETA: 2s - loss: 0.0280 - accuracy: 0.9910
1175/1688 [===================>..........] - ETA: 2s - loss: 0.0281 - accuracy: 0.9909
1185/1688 [====================>.........] - ETA: 2s - loss: 0.0281 - accuracy: 0.9909
1194/1688 [====================>.........] - ETA: 2s - loss: 0.0282 - accuracy: 0.9909
1204/1688 [====================>.........] - ETA: 2s - loss: 0.0282 - accuracy: 0.9909
1213/1688 [====================>.........] - ETA: 2s - loss: 0.0281 - accuracy: 0.9909
1223/1688 [====================>.........] - ETA: 2s - loss: 0.0281 - accuracy: 0.9909
1233/1688 [====================>.........] - ETA: 2s - loss: 0.0282 - accuracy: 0.9909
1243/1688 [=====================>........] - ETA: 2s - loss: 0.0283 - accuracy: 0.9908
1253/1688 [=====================>........] - ETA: 2s - loss: 0.0282 - accuracy: 0.9909
1263/1688 [=====================>........] - ETA: 2s - loss: 0.0282 - accuracy: 0.9908
1273/1688 [=====================>........] - ETA: 2s - loss: 0.0282 - accuracy: 0.9909
1283/1688 [=====================>........] - ETA: 2s - loss: 0.0283 - accuracy: 0.9909
1292/1688 [=====================>........] - ETA: 2s - loss: 0.0285 - accuracy: 0.9909
1301/1688 [======================>.......] - ETA: 2s - loss: 0.0284 - accuracy: 0.9909
1311/1688 [======================>.......] - ETA: 2s - loss: 0.0284 - accuracy: 0.9909
1321/1688 [======================>.......] - ETA: 2s - loss: 0.0284 - accuracy: 0.9909
1330/1688 [======================>.......] - ETA: 1s - loss: 0.0286 - accuracy: 0.9908
1339/1688 [======================>.......] - ETA: 1s - loss: 0.0285 - accuracy: 0.9909
1349/1688 [======================>.......] - ETA: 1s - loss: 0.0284 - accuracy: 0.9909
1359/1688 [=======================>......] - ETA: 1s - loss: 0.0284 - accuracy: 0.9909
1369/1688 [=======================>......] - ETA: 1s - loss: 0.0285 - accuracy: 0.9909
1379/1688 [=======================>......] - ETA: 1s - loss: 0.0285 - accuracy: 0.9909
1389/1688 [=======================>......] - ETA: 1s - loss: 0.0287 - accuracy: 0.9908
1398/1688 [=======================>......] - ETA: 1s - loss: 0.0288 - accuracy: 0.9908
1408/1688 [========================>.....] - ETA: 1s - loss: 0.0288 - accuracy: 0.9908
1418/1688 [========================>.....] - ETA: 1s - loss: 0.0287 - accuracy: 0.9909
1427/1688 [========================>.....] - ETA: 1s - loss: 0.0289 - accuracy: 0.9909
1437/1688 [========================>.....] - ETA: 1s - loss: 0.0290 - accuracy: 0.9908
1447/1688 [========================>.....] - ETA: 1s - loss: 0.0290 - accuracy: 0.9909
1457/1688 [========================>.....] - ETA: 1s - loss: 0.0290 - accuracy: 0.9909
1467/1688 [=========================>....] - ETA: 1s - loss: 0.0291 - accuracy: 0.9909
1477/1688 [=========================>....] - ETA: 1s - loss: 0.0290 - accuracy: 0.9909
1487/1688 [=========================>....] - ETA: 1s - loss: 0.0290 - accuracy: 0.9909
1497/1688 [=========================>....] - ETA: 1s - loss: 0.0288 - accuracy: 0.9909
1507/1688 [=========================>....] - ETA: 1s - loss: 0.0290 - accuracy: 0.9909
1517/1688 [=========================>....] - ETA: 0s - loss: 0.0291 - accuracy: 0.9908
1527/1688 [==========================>...] - ETA: 0s - loss: 0.0291 - accuracy: 0.9909
1536/1688 [==========================>...] - ETA: 0s - loss: 0.0291 - accuracy: 0.9908
1546/1688 [==========================>...] - ETA: 0s - loss: 0.0290 - accuracy: 0.9908
1555/1688 [==========================>...] - ETA: 0s - loss: 0.0290 - accuracy: 0.9908
1565/1688 [==========================>...] - ETA: 0s - loss: 0.0291 - accuracy: 0.9908
1574/1688 [==========================>...] - ETA: 0s - loss: 0.0290 - accuracy: 0.9908
1584/1688 [===========================>..] - ETA: 0s - loss: 0.0291 - accuracy: 0.9908
1594/1688 [===========================>..] - ETA: 0s - loss: 0.0291 - accuracy: 0.9908
1604/1688 [===========================>..] - ETA: 0s - loss: 0.0292 - accuracy: 0.9908
1614/1688 [===========================>..] - ETA: 0s - loss: 0.0291 - accuracy: 0.9908
1624/1688 [===========================>..] - ETA: 0s - loss: 0.0290 - accuracy: 0.9908
1634/1688 [============================>.] - ETA: 0s - loss: 0.0290 - accuracy: 0.9908
1644/1688 [============================>.] - ETA: 0s - loss: 0.0289 - accuracy: 0.9908
1654/1688 [============================>.] - ETA: 0s - loss: 0.0288 - accuracy: 0.9908
1663/1688 [============================>.] - ETA: 0s - loss: 0.0289 - accuracy: 0.9909
1672/1688 [============================>.] - ETA: 0s - loss: 0.0290 - accuracy: 0.9908
1682/1688 [============================>.] - ETA: 0s - loss: 0.0289 - accuracy: 0.9908
1688/1688 [==============================] - 10s 6ms/step - loss: 0.0289 - accuracy: 0.9908 - val_loss: 0.0426 - val_accuracy: 0.9885
Epoch 4/10

   1/1688 [..............................] - ETA: 0s - loss: 0.0346 - accuracy: 1.0000
  11/1688 [..............................] - ETA: 8s - loss: 0.0231 - accuracy: 0.9915
  21/1688 [..............................] - ETA: 8s - loss: 0.0183 - accuracy: 0.9926
  31/1688 [..............................] - ETA: 8s - loss: 0.0258 - accuracy: 0.9929
  40/1688 [..............................] - ETA: 8s - loss: 0.0280 - accuracy: 0.9922
  49/1688 [..............................] - ETA: 8s - loss: 0.0236 - accuracy: 0.9936
  59/1688 [>.............................] - ETA: 8s - loss: 0.0232 - accuracy: 0.9931
  69/1688 [>.............................] - ETA: 8s - loss: 0.0213 - accuracy: 0.9941
  79/1688 [>.............................] - ETA: 8s - loss: 0.0221 - accuracy: 0.9941
  89/1688 [>.............................] - ETA: 8s - loss: 0.0205 - accuracy: 0.9947
  99/1688 [>.............................] - ETA: 8s - loss: 0.0197 - accuracy: 0.9946
 109/1688 [>.............................] - ETA: 8s - loss: 0.0181 - accuracy: 0.9951
 119/1688 [=>............................] - ETA: 8s - loss: 0.0173 - accuracy: 0.9953
 129/1688 [=>............................] - ETA: 8s - loss: 0.0163 - accuracy: 0.9956
 139/1688 [=>............................] - ETA: 8s - loss: 0.0154 - accuracy: 0.9960
 149/1688 [=>............................] - ETA: 8s - loss: 0.0149 - accuracy: 0.9960
 159/1688 [=>............................] - ETA: 8s - loss: 0.0150 - accuracy: 0.9959
 168/1688 [=>............................] - ETA: 8s - loss: 0.0150 - accuracy: 0.9957
 178/1688 [==>...........................] - ETA: 8s - loss: 0.0152 - accuracy: 0.9958
 188/1688 [==>...........................] - ETA: 8s - loss: 0.0147 - accuracy: 0.9958
 197/1688 [==>...........................] - ETA: 8s - loss: 0.0147 - accuracy: 0.9959
 207/1688 [==>...........................] - ETA: 8s - loss: 0.0143 - accuracy: 0.9959
 217/1688 [==>...........................] - ETA: 8s - loss: 0.0146 - accuracy: 0.9955
 227/1688 [===>..........................] - ETA: 8s - loss: 0.0141 - accuracy: 0.9957
 237/1688 [===>..........................] - ETA: 7s - loss: 0.0140 - accuracy: 0.9959
 247/1688 [===>..........................] - ETA: 7s - loss: 0.0141 - accuracy: 0.9960
 257/1688 [===>..........................] - ETA: 7s - loss: 0.0148 - accuracy: 0.9957
 266/1688 [===>..........................] - ETA: 7s - loss: 0.0146 - accuracy: 0.9959
 276/1688 [===>..........................] - ETA: 7s - loss: 0.0154 - accuracy: 0.9958
 286/1688 [====>.........................] - ETA: 7s - loss: 0.0158 - accuracy: 0.9957
 296/1688 [====>.........................] - ETA: 7s - loss: 0.0162 - accuracy: 0.9957
 306/1688 [====>.........................] - ETA: 7s - loss: 0.0158 - accuracy: 0.9957
 316/1688 [====>.........................] - ETA: 7s - loss: 0.0160 - accuracy: 0.9956
 326/1688 [====>.........................] - ETA: 7s - loss: 0.0156 - accuracy: 0.9958
 336/1688 [====>.........................] - ETA: 7s - loss: 0.0160 - accuracy: 0.9955
 346/1688 [=====>........................] - ETA: 7s - loss: 0.0159 - accuracy: 0.9956
 356/1688 [=====>........................] - ETA: 7s - loss: 0.0162 - accuracy: 0.9954
 365/1688 [=====>........................] - ETA: 7s - loss: 0.0167 - accuracy: 0.9952
 374/1688 [=====>........................] - ETA: 7s - loss: 0.0164 - accuracy: 0.9953
 384/1688 [=====>........................] - ETA: 7s - loss: 0.0162 - accuracy: 0.9953
 394/1688 [======>.......................] - ETA: 7s - loss: 0.0161 - accuracy: 0.9952
 404/1688 [======>.......................] - ETA: 7s - loss: 0.0159 - accuracy: 0.9953
 414/1688 [======>.......................] - ETA: 7s - loss: 0.0159 - accuracy: 0.9953
 423/1688 [======>.......................] - ETA: 6s - loss: 0.0159 - accuracy: 0.9953
 433/1688 [======>.......................] - ETA: 6s - loss: 0.0163 - accuracy: 0.9951
 443/1688 [======>.......................] - ETA: 6s - loss: 0.0161 - accuracy: 0.9952
 452/1688 [=======>......................] - ETA: 6s - loss: 0.0159 - accuracy: 0.9953
 462/1688 [=======>......................] - ETA: 6s - loss: 0.0159 - accuracy: 0.9953
 472/1688 [=======>......................] - ETA: 6s - loss: 0.0161 - accuracy: 0.9952
 482/1688 [=======>......................] - ETA: 6s - loss: 0.0161 - accuracy: 0.9952
 492/1688 [=======>......................] - ETA: 6s - loss: 0.0163 - accuracy: 0.9951
 502/1688 [=======>......................] - ETA: 6s - loss: 0.0167 - accuracy: 0.9951
 512/1688 [========>.....................] - ETA: 6s - loss: 0.0169 - accuracy: 0.9951
 522/1688 [========>.....................] - ETA: 6s - loss: 0.0172 - accuracy: 0.9950
 532/1688 [========>.....................] - ETA: 6s - loss: 0.0170 - accuracy: 0.9951
 542/1688 [========>.....................] - ETA: 6s - loss: 0.0168 - accuracy: 0.9952
 552/1688 [========>.....................] - ETA: 6s - loss: 0.0169 - accuracy: 0.9951
 561/1688 [========>.....................] - ETA: 6s - loss: 0.0167 - accuracy: 0.9951
 571/1688 [=========>....................] - ETA: 6s - loss: 0.0167 - accuracy: 0.9951
 581/1688 [=========>....................] - ETA: 6s - loss: 0.0171 - accuracy: 0.9950
 591/1688 [=========>....................] - ETA: 6s - loss: 0.0171 - accuracy: 0.9950
 601/1688 [=========>....................] - ETA: 6s - loss: 0.0171 - accuracy: 0.9949
 610/1688 [=========>....................] - ETA: 5s - loss: 0.0169 - accuracy: 0.9949
 620/1688 [==========>...................] - ETA: 5s - loss: 0.0172 - accuracy: 0.9949
 630/1688 [==========>...................] - ETA: 5s - loss: 0.0173 - accuracy: 0.9948
 640/1688 [==========>...................] - ETA: 5s - loss: 0.0172 - accuracy: 0.9948
 649/1688 [==========>...................] - ETA: 5s - loss: 0.0173 - accuracy: 0.9948
 659/1688 [==========>...................] - ETA: 5s - loss: 0.0171 - accuracy: 0.9948
 669/1688 [==========>...................] - ETA: 5s - loss: 0.0169 - accuracy: 0.9949
 679/1688 [===========>..................] - ETA: 5s - loss: 0.0173 - accuracy: 0.9948
 689/1688 [===========>..................] - ETA: 5s - loss: 0.0174 - accuracy: 0.9947
 699/1688 [===========>..................] - ETA: 5s - loss: 0.0175 - accuracy: 0.9947
 709/1688 [===========>..................] - ETA: 5s - loss: 0.0175 - accuracy: 0.9946
 718/1688 [===========>..................] - ETA: 5s - loss: 0.0175 - accuracy: 0.9946
 728/1688 [===========>..................] - ETA: 5s - loss: 0.0175 - accuracy: 0.9946
 738/1688 [============>.................] - ETA: 5s - loss: 0.0180 - accuracy: 0.9944
 748/1688 [============>.................] - ETA: 5s - loss: 0.0182 - accuracy: 0.9943
 758/1688 [============>.................] - ETA: 5s - loss: 0.0180 - accuracy: 0.9943
 768/1688 [============>.................] - ETA: 5s - loss: 0.0183 - accuracy: 0.9942
 778/1688 [============>.................] - ETA: 5s - loss: 0.0184 - accuracy: 0.9941
 788/1688 [=============>................] - ETA: 4s - loss: 0.0185 - accuracy: 0.9940
 798/1688 [=============>................] - ETA: 4s - loss: 0.0187 - accuracy: 0.9939
 808/1688 [=============>................] - ETA: 4s - loss: 0.0186 - accuracy: 0.9939
 818/1688 [=============>................] - ETA: 4s - loss: 0.0188 - accuracy: 0.9938
 828/1688 [=============>................] - ETA: 4s - loss: 0.0188 - accuracy: 0.9938
 838/1688 [=============>................] - ETA: 4s - loss: 0.0189 - accuracy: 0.9938
 848/1688 [==============>...............] - ETA: 4s - loss: 0.0191 - accuracy: 0.9937
 858/1688 [==============>...............] - ETA: 4s - loss: 0.0193 - accuracy: 0.9936
 867/1688 [==============>...............] - ETA: 4s - loss: 0.0196 - accuracy: 0.9935
 876/1688 [==============>...............] - ETA: 4s - loss: 0.0196 - accuracy: 0.9935
 886/1688 [==============>...............] - ETA: 4s - loss: 0.0200 - accuracy: 0.9933
 896/1688 [==============>...............] - ETA: 4s - loss: 0.0202 - accuracy: 0.9933
 906/1688 [===============>..............] - ETA: 4s - loss: 0.0207 - accuracy: 0.9931
 916/1688 [===============>..............] - ETA: 4s - loss: 0.0209 - accuracy: 0.9930
 925/1688 [===============>..............] - ETA: 4s - loss: 0.0209 - accuracy: 0.9931
 935/1688 [===============>..............] - ETA: 4s - loss: 0.0210 - accuracy: 0.9930
 945/1688 [===============>..............] - ETA: 4s - loss: 0.0209 - accuracy: 0.9930
 954/1688 [===============>..............] - ETA: 4s - loss: 0.0209 - accuracy: 0.9930
 964/1688 [================>.............] - ETA: 3s - loss: 0.0208 - accuracy: 0.9931
 974/1688 [================>.............] - ETA: 3s - loss: 0.0208 - accuracy: 0.9931
 984/1688 [================>.............] - ETA: 3s - loss: 0.0211 - accuracy: 0.9930
 994/1688 [================>.............] - ETA: 3s - loss: 0.0209 - accuracy: 0.9931
1004/1688 [================>.............] - ETA: 3s - loss: 0.0209 - accuracy: 0.9930
1014/1688 [=================>............] - ETA: 3s - loss: 0.0207 - accuracy: 0.9931
1024/1688 [=================>............] - ETA: 3s - loss: 0.0211 - accuracy: 0.9930
1034/1688 [=================>............] - ETA: 3s - loss: 0.0211 - accuracy: 0.9930
1044/1688 [=================>............] - ETA: 3s - loss: 0.0211 - accuracy: 0.9931
1054/1688 [=================>............] - ETA: 3s - loss: 0.0213 - accuracy: 0.9930
1064/1688 [=================>............] - ETA: 3s - loss: 0.0212 - accuracy: 0.9931
1074/1688 [==================>...........] - ETA: 3s - loss: 0.0211 - accuracy: 0.9931
1084/1688 [==================>...........] - ETA: 3s - loss: 0.0211 - accuracy: 0.9931
1093/1688 [==================>...........] - ETA: 3s - loss: 0.0211 - accuracy: 0.9931
1102/1688 [==================>...........] - ETA: 3s - loss: 0.0211 - accuracy: 0.9931
1111/1688 [==================>...........] - ETA: 3s - loss: 0.0211 - accuracy: 0.9931
1121/1688 [==================>...........] - ETA: 3s - loss: 0.0210 - accuracy: 0.9931
1131/1688 [===================>..........] - ETA: 3s - loss: 0.0209 - accuracy: 0.9932
1141/1688 [===================>..........] - ETA: 3s - loss: 0.0208 - accuracy: 0.9932
1151/1688 [===================>..........] - ETA: 2s - loss: 0.0208 - accuracy: 0.9932
1161/1688 [===================>..........] - ETA: 2s - loss: 0.0207 - accuracy: 0.9932
1171/1688 [===================>..........] - ETA: 2s - loss: 0.0208 - accuracy: 0.9931
1181/1688 [===================>..........] - ETA: 2s - loss: 0.0210 - accuracy: 0.9930
1190/1688 [====================>.........] - ETA: 2s - loss: 0.0210 - accuracy: 0.9930
1200/1688 [====================>.........] - ETA: 2s - loss: 0.0210 - accuracy: 0.9930
1210/1688 [====================>.........] - ETA: 2s - loss: 0.0210 - accuracy: 0.9931
1220/1688 [====================>.........] - ETA: 2s - loss: 0.0211 - accuracy: 0.9931
1230/1688 [====================>.........] - ETA: 2s - loss: 0.0212 - accuracy: 0.9931
1240/1688 [=====================>........] - ETA: 2s - loss: 0.0212 - accuracy: 0.9930
1250/1688 [=====================>........] - ETA: 2s - loss: 0.0212 - accuracy: 0.9931
1260/1688 [=====================>........] - ETA: 2s - loss: 0.0211 - accuracy: 0.9931
1270/1688 [=====================>........] - ETA: 2s - loss: 0.0212 - accuracy: 0.9931
1280/1688 [=====================>........] - ETA: 2s - loss: 0.0212 - accuracy: 0.9931
1290/1688 [=====================>........] - ETA: 2s - loss: 0.0214 - accuracy: 0.9931
1300/1688 [======================>.......] - ETA: 2s - loss: 0.0214 - accuracy: 0.9931
1310/1688 [======================>.......] - ETA: 2s - loss: 0.0213 - accuracy: 0.9931
1320/1688 [======================>.......] - ETA: 2s - loss: 0.0215 - accuracy: 0.9931
1330/1688 [======================>.......] - ETA: 1s - loss: 0.0215 - accuracy: 0.9931
1340/1688 [======================>.......] - ETA: 1s - loss: 0.0215 - accuracy: 0.9931
1350/1688 [======================>.......] - ETA: 1s - loss: 0.0214 - accuracy: 0.9931
1360/1688 [=======================>......] - ETA: 1s - loss: 0.0214 - accuracy: 0.9931
1369/1688 [=======================>......] - ETA: 1s - loss: 0.0215 - accuracy: 0.9930
1379/1688 [=======================>......] - ETA: 1s - loss: 0.0213 - accuracy: 0.9931
1389/1688 [=======================>......] - ETA: 1s - loss: 0.0212 - accuracy: 0.9931
1399/1688 [=======================>......] - ETA: 1s - loss: 0.0212 - accuracy: 0.9931
1409/1688 [========================>.....] - ETA: 1s - loss: 0.0211 - accuracy: 0.9932
1419/1688 [========================>.....] - ETA: 1s - loss: 0.0211 - accuracy: 0.9932
1429/1688 [========================>.....] - ETA: 1s - loss: 0.0212 - accuracy: 0.9932
1439/1688 [========================>.....] - ETA: 1s - loss: 0.0213 - accuracy: 0.9932
1449/1688 [========================>.....] - ETA: 1s - loss: 0.0214 - accuracy: 0.9931
1459/1688 [========================>.....] - ETA: 1s - loss: 0.0214 - accuracy: 0.9931
1468/1688 [=========================>....] - ETA: 1s - loss: 0.0213 - accuracy: 0.9932
1478/1688 [=========================>....] - ETA: 1s - loss: 0.0213 - accuracy: 0.9932
1488/1688 [=========================>....] - ETA: 1s - loss: 0.0213 - accuracy: 0.9932
1498/1688 [=========================>....] - ETA: 1s - loss: 0.0213 - accuracy: 0.9932
1508/1688 [=========================>....] - ETA: 0s - loss: 0.0213 - accuracy: 0.9932
1518/1688 [=========================>....] - ETA: 0s - loss: 0.0212 - accuracy: 0.9932
1527/1688 [==========================>...] - ETA: 0s - loss: 0.0213 - accuracy: 0.9932
1536/1688 [==========================>...] - ETA: 0s - loss: 0.0214 - accuracy: 0.9932
1545/1688 [==========================>...] - ETA: 0s - loss: 0.0216 - accuracy: 0.9931
1555/1688 [==========================>...] - ETA: 0s - loss: 0.0217 - accuracy: 0.9931
1564/1688 [==========================>...] - ETA: 0s - loss: 0.0217 - accuracy: 0.9931
1573/1688 [==========================>...] - ETA: 0s - loss: 0.0217 - accuracy: 0.9930
1583/1688 [===========================>..] - ETA: 0s - loss: 0.0217 - accuracy: 0.9930
1593/1688 [===========================>..] - ETA: 0s - loss: 0.0217 - accuracy: 0.9930
1602/1688 [===========================>..] - ETA: 0s - loss: 0.0217 - accuracy: 0.9930
1612/1688 [===========================>..] - ETA: 0s - loss: 0.0218 - accuracy: 0.9929
1622/1688 [===========================>..] - ETA: 0s - loss: 0.0219 - accuracy: 0.9929
1632/1688 [============================>.] - ETA: 0s - loss: 0.0222 - accuracy: 0.9929
1642/1688 [============================>.] - ETA: 0s - loss: 0.0221 - accuracy: 0.9929
1652/1688 [============================>.] - ETA: 0s - loss: 0.0221 - accuracy: 0.9929
1662/1688 [============================>.] - ETA: 0s - loss: 0.0221 - accuracy: 0.9929
1672/1688 [============================>.] - ETA: 0s - loss: 0.0221 - accuracy: 0.9929
1682/1688 [============================>.] - ETA: 0s - loss: 0.0222 - accuracy: 0.9928
1688/1688 [==============================] - 10s 6ms/step - loss: 0.0222 - accuracy: 0.9929 - val_loss: 0.0491 - val_accuracy: 0.9880
Epoch 5/10

   1/1688 [..............................] - ETA: 0s - loss: 1.5401e-04 - accuracy: 1.0000
  11/1688 [..............................] - ETA: 8s - loss: 0.0044 - accuracy: 1.0000    
  21/1688 [..............................] - ETA: 8s - loss: 0.0112 - accuracy: 0.9955
  31/1688 [..............................] - ETA: 8s - loss: 0.0113 - accuracy: 0.9950
  41/1688 [..............................] - ETA: 8s - loss: 0.0104 - accuracy: 0.9954
  51/1688 [..............................] - ETA: 8s - loss: 0.0112 - accuracy: 0.9957
  61/1688 [>.............................] - ETA: 8s - loss: 0.0138 - accuracy: 0.9954
  71/1688 [>.............................] - ETA: 8s - loss: 0.0124 - accuracy: 0.9960
  80/1688 [>.............................] - ETA: 8s - loss: 0.0124 - accuracy: 0.9957
  89/1688 [>.............................] - ETA: 8s - loss: 0.0127 - accuracy: 0.9954
  99/1688 [>.............................] - ETA: 8s - loss: 0.0120 - accuracy: 0.9959
 109/1688 [>.............................] - ETA: 8s - loss: 0.0123 - accuracy: 0.9960
 119/1688 [=>............................] - ETA: 8s - loss: 0.0119 - accuracy: 0.9961
 129/1688 [=>............................] - ETA: 8s - loss: 0.0145 - accuracy: 0.9954
 139/1688 [=>............................] - ETA: 8s - loss: 0.0136 - accuracy: 0.9957
 149/1688 [=>............................] - ETA: 8s - loss: 0.0141 - accuracy: 0.9954
 159/1688 [=>............................] - ETA: 8s - loss: 0.0137 - accuracy: 0.9955
 169/1688 [==>...........................] - ETA: 8s - loss: 0.0137 - accuracy: 0.9956
 179/1688 [==>...........................] - ETA: 8s - loss: 0.0136 - accuracy: 0.9955
 189/1688 [==>...........................] - ETA: 8s - loss: 0.0131 - accuracy: 0.9955
 199/1688 [==>...........................] - ETA: 8s - loss: 0.0127 - accuracy: 0.9958
 209/1688 [==>...........................] - ETA: 8s - loss: 0.0131 - accuracy: 0.9955
 219/1688 [==>...........................] - ETA: 8s - loss: 0.0126 - accuracy: 0.9957
 229/1688 [===>..........................] - ETA: 7s - loss: 0.0126 - accuracy: 0.9958
 239/1688 [===>..........................] - ETA: 7s - loss: 0.0124 - accuracy: 0.9959
 249/1688 [===>..........................] - ETA: 7s - loss: 0.0121 - accuracy: 0.9960
 259/1688 [===>..........................] - ETA: 7s - loss: 0.0119 - accuracy: 0.9961
 268/1688 [===>..........................] - ETA: 7s - loss: 0.0118 - accuracy: 0.9962
 278/1688 [===>..........................] - ETA: 7s - loss: 0.0116 - accuracy: 0.9962
 288/1688 [====>.........................] - ETA: 7s - loss: 0.0114 - accuracy: 0.9962
 298/1688 [====>.........................] - ETA: 7s - loss: 0.0116 - accuracy: 0.9962
 308/1688 [====>.........................] - ETA: 7s - loss: 0.0122 - accuracy: 0.9961
 318/1688 [====>.........................] - ETA: 7s - loss: 0.0120 - accuracy: 0.9962
 328/1688 [====>.........................] - ETA: 7s - loss: 0.0124 - accuracy: 0.9959
 338/1688 [=====>........................] - ETA: 7s - loss: 0.0125 - accuracy: 0.9958
 347/1688 [=====>........................] - ETA: 7s - loss: 0.0127 - accuracy: 0.9958
 357/1688 [=====>........................] - ETA: 7s - loss: 0.0127 - accuracy: 0.9958
 367/1688 [=====>........................] - ETA: 7s - loss: 0.0126 - accuracy: 0.9958
 377/1688 [=====>........................] - ETA: 7s - loss: 0.0125 - accuracy: 0.9959
 387/1688 [=====>........................] - ETA: 7s - loss: 0.0137 - accuracy: 0.9956
 397/1688 [======>.......................] - ETA: 7s - loss: 0.0146 - accuracy: 0.9955
 407/1688 [======>.......................] - ETA: 7s - loss: 0.0151 - accuracy: 0.9954
 417/1688 [======>.......................] - ETA: 6s - loss: 0.0151 - accuracy: 0.9954
 426/1688 [======>.......................] - ETA: 6s - loss: 0.0150 - accuracy: 0.9953
 436/1688 [======>.......................] - ETA: 6s - loss: 0.0151 - accuracy: 0.9953
 445/1688 [======>.......................] - ETA: 6s - loss: 0.0151 - accuracy: 0.9952
 454/1688 [=======>......................] - ETA: 6s - loss: 0.0150 - accuracy: 0.9953
 464/1688 [=======>......................] - ETA: 6s - loss: 0.0151 - accuracy: 0.9953
 474/1688 [=======>......................] - ETA: 6s - loss: 0.0150 - accuracy: 0.9953
 484/1688 [=======>......................] - ETA: 6s - loss: 0.0148 - accuracy: 0.9954
 494/1688 [=======>......................] - ETA: 6s - loss: 0.0150 - accuracy: 0.9952
 504/1688 [=======>......................] - ETA: 6s - loss: 0.0148 - accuracy: 0.9953
 513/1688 [========>.....................] - ETA: 6s - loss: 0.0150 - accuracy: 0.9952
 522/1688 [========>.....................] - ETA: 6s - loss: 0.0148 - accuracy: 0.9953
 532/1688 [========>.....................] - ETA: 6s - loss: 0.0146 - accuracy: 0.9953
 542/1688 [========>.....................] - ETA: 6s - loss: 0.0145 - accuracy: 0.9953
 552/1688 [========>.....................] - ETA: 6s - loss: 0.0144 - accuracy: 0.9952
 561/1688 [========>.....................] - ETA: 6s - loss: 0.0143 - accuracy: 0.9953
 570/1688 [=========>....................] - ETA: 6s - loss: 0.0142 - accuracy: 0.9953
 580/1688 [=========>....................] - ETA: 6s - loss: 0.0140 - accuracy: 0.9954
 590/1688 [=========>....................] - ETA: 6s - loss: 0.0141 - accuracy: 0.9953
 599/1688 [=========>....................] - ETA: 5s - loss: 0.0140 - accuracy: 0.9954
 609/1688 [=========>....................] - ETA: 5s - loss: 0.0139 - accuracy: 0.9953
 619/1688 [==========>...................] - ETA: 5s - loss: 0.0139 - accuracy: 0.9953
 629/1688 [==========>...................] - ETA: 5s - loss: 0.0139 - accuracy: 0.9953
 638/1688 [==========>...................] - ETA: 5s - loss: 0.0138 - accuracy: 0.9953
 648/1688 [==========>...................] - ETA: 5s - loss: 0.0139 - accuracy: 0.9953
 658/1688 [==========>...................] - ETA: 5s - loss: 0.0138 - accuracy: 0.9953
 668/1688 [==========>...................] - ETA: 5s - loss: 0.0141 - accuracy: 0.9952
 678/1688 [===========>..................] - ETA: 5s - loss: 0.0139 - accuracy: 0.9953
 688/1688 [===========>..................] - ETA: 5s - loss: 0.0141 - accuracy: 0.9951
 698/1688 [===========>..................] - ETA: 5s - loss: 0.0142 - accuracy: 0.9951
 708/1688 [===========>..................] - ETA: 5s - loss: 0.0143 - accuracy: 0.9951
 718/1688 [===========>..................] - ETA: 5s - loss: 0.0144 - accuracy: 0.9950
 728/1688 [===========>..................] - ETA: 5s - loss: 0.0145 - accuracy: 0.9950
 738/1688 [============>.................] - ETA: 5s - loss: 0.0150 - accuracy: 0.9950
 748/1688 [============>.................] - ETA: 5s - loss: 0.0148 - accuracy: 0.9950
 758/1688 [============>.................] - ETA: 5s - loss: 0.0152 - accuracy: 0.9949
 768/1688 [============>.................] - ETA: 5s - loss: 0.0151 - accuracy: 0.9950
 778/1688 [============>.................] - ETA: 5s - loss: 0.0156 - accuracy: 0.9949
 788/1688 [=============>................] - ETA: 4s - loss: 0.0155 - accuracy: 0.9950
 798/1688 [=============>................] - ETA: 4s - loss: 0.0154 - accuracy: 0.9950
 808/1688 [=============>................] - ETA: 4s - loss: 0.0153 - accuracy: 0.9950
 817/1688 [=============>................] - ETA: 4s - loss: 0.0154 - accuracy: 0.9949
 827/1688 [=============>................] - ETA: 4s - loss: 0.0156 - accuracy: 0.9949
 837/1688 [=============>................] - ETA: 4s - loss: 0.0155 - accuracy: 0.9949
 847/1688 [==============>...............] - ETA: 4s - loss: 0.0158 - accuracy: 0.9949
 857/1688 [==============>...............] - ETA: 4s - loss: 0.0159 - accuracy: 0.9948
 867/1688 [==============>...............] - ETA: 4s - loss: 0.0159 - accuracy: 0.9948
 877/1688 [==============>...............] - ETA: 4s - loss: 0.0162 - accuracy: 0.9947
 887/1688 [==============>...............] - ETA: 4s - loss: 0.0162 - accuracy: 0.9947
 897/1688 [==============>...............] - ETA: 4s - loss: 0.0163 - accuracy: 0.9947
 907/1688 [===============>..............] - ETA: 4s - loss: 0.0163 - accuracy: 0.9947
 917/1688 [===============>..............] - ETA: 4s - loss: 0.0163 - accuracy: 0.9947
 927/1688 [===============>..............] - ETA: 4s - loss: 0.0164 - accuracy: 0.9946
 937/1688 [===============>..............] - ETA: 4s - loss: 0.0166 - accuracy: 0.9945
 947/1688 [===============>..............] - ETA: 4s - loss: 0.0165 - accuracy: 0.9945
 957/1688 [================>.............] - ETA: 4s - loss: 0.0166 - accuracy: 0.9944
 967/1688 [================>.............] - ETA: 3s - loss: 0.0168 - accuracy: 0.9944
 976/1688 [================>.............] - ETA: 3s - loss: 0.0168 - accuracy: 0.9944
 986/1688 [================>.............] - ETA: 3s - loss: 0.0169 - accuracy: 0.9943
 996/1688 [================>.............] - ETA: 3s - loss: 0.0169 - accuracy: 0.9942
1006/1688 [================>.............] - ETA: 3s - loss: 0.0169 - accuracy: 0.9943
1016/1688 [=================>............] - ETA: 3s - loss: 0.0169 - accuracy: 0.9942
1025/1688 [=================>............] - ETA: 3s - loss: 0.0169 - accuracy: 0.9942
1035/1688 [=================>............] - ETA: 3s - loss: 0.0168 - accuracy: 0.9943
1044/1688 [=================>............] - ETA: 3s - loss: 0.0168 - accuracy: 0.9943
1053/1688 [=================>............] - ETA: 3s - loss: 0.0167 - accuracy: 0.9943
1063/1688 [=================>............] - ETA: 3s - loss: 0.0166 - accuracy: 0.9944
1073/1688 [==================>...........] - ETA: 3s - loss: 0.0165 - accuracy: 0.9944
1082/1688 [==================>...........] - ETA: 3s - loss: 0.0164 - accuracy: 0.9944
1092/1688 [==================>...........] - ETA: 3s - loss: 0.0163 - accuracy: 0.9944
1102/1688 [==================>...........] - ETA: 3s - loss: 0.0163 - accuracy: 0.9944
1111/1688 [==================>...........] - ETA: 3s - loss: 0.0164 - accuracy: 0.9944
1120/1688 [==================>...........] - ETA: 3s - loss: 0.0164 - accuracy: 0.9944
1130/1688 [===================>..........] - ETA: 3s - loss: 0.0164 - accuracy: 0.9945
1140/1688 [===================>..........] - ETA: 3s - loss: 0.0163 - accuracy: 0.9945
1150/1688 [===================>..........] - ETA: 2s - loss: 0.0161 - accuracy: 0.9946
1160/1688 [===================>..........] - ETA: 2s - loss: 0.0161 - accuracy: 0.9945
1170/1688 [===================>..........] - ETA: 2s - loss: 0.0164 - accuracy: 0.9944
1180/1688 [===================>..........] - ETA: 2s - loss: 0.0166 - accuracy: 0.9944
1190/1688 [====================>.........] - ETA: 2s - loss: 0.0166 - accuracy: 0.9944
1200/1688 [====================>.........] - ETA: 2s - loss: 0.0166 - accuracy: 0.9944
1209/1688 [====================>.........] - ETA: 2s - loss: 0.0168 - accuracy: 0.9943
1219/1688 [====================>.........] - ETA: 2s - loss: 0.0168 - accuracy: 0.9943
1229/1688 [====================>.........] - ETA: 2s - loss: 0.0168 - accuracy: 0.9943
1239/1688 [=====================>........] - ETA: 2s - loss: 0.0168 - accuracy: 0.9943
1249/1688 [=====================>........] - ETA: 2s - loss: 0.0170 - accuracy: 0.9942
1259/1688 [=====================>........] - ETA: 2s - loss: 0.0171 - accuracy: 0.9941
1268/1688 [=====================>........] - ETA: 2s - loss: 0.0172 - accuracy: 0.9941
1278/1688 [=====================>........] - ETA: 2s - loss: 0.0171 - accuracy: 0.9941
1288/1688 [=====================>........] - ETA: 2s - loss: 0.0172 - accuracy: 0.9941
1297/1688 [======================>.......] - ETA: 2s - loss: 0.0173 - accuracy: 0.9941
1307/1688 [======================>.......] - ETA: 2s - loss: 0.0174 - accuracy: 0.9941
1317/1688 [======================>.......] - ETA: 2s - loss: 0.0173 - accuracy: 0.9941
1327/1688 [======================>.......] - ETA: 1s - loss: 0.0173 - accuracy: 0.9941
1337/1688 [======================>.......] - ETA: 1s - loss: 0.0174 - accuracy: 0.9941
1346/1688 [======================>.......] - ETA: 1s - loss: 0.0174 - accuracy: 0.9941
1356/1688 [=======================>......] - ETA: 1s - loss: 0.0176 - accuracy: 0.9941
1365/1688 [=======================>......] - ETA: 1s - loss: 0.0175 - accuracy: 0.9941
1375/1688 [=======================>......] - ETA: 1s - loss: 0.0175 - accuracy: 0.9941
1385/1688 [=======================>......] - ETA: 1s - loss: 0.0174 - accuracy: 0.9942
1395/1688 [=======================>......] - ETA: 1s - loss: 0.0173 - accuracy: 0.9942
1405/1688 [=======================>......] - ETA: 1s - loss: 0.0172 - accuracy: 0.9942
1415/1688 [========================>.....] - ETA: 1s - loss: 0.0171 - accuracy: 0.9943
1425/1688 [========================>.....] - ETA: 1s - loss: 0.0171 - accuracy: 0.9943
1435/1688 [========================>.....] - ETA: 1s - loss: 0.0173 - accuracy: 0.9942
1445/1688 [========================>.....] - ETA: 1s - loss: 0.0173 - accuracy: 0.9942
1455/1688 [========================>.....] - ETA: 1s - loss: 0.0174 - accuracy: 0.9942
1465/1688 [=========================>....] - ETA: 1s - loss: 0.0175 - accuracy: 0.9941
1474/1688 [=========================>....] - ETA: 1s - loss: 0.0175 - accuracy: 0.9941
1484/1688 [=========================>....] - ETA: 1s - loss: 0.0176 - accuracy: 0.9941
1494/1688 [=========================>....] - ETA: 1s - loss: 0.0175 - accuracy: 0.9941
1504/1688 [=========================>....] - ETA: 1s - loss: 0.0177 - accuracy: 0.9941
1514/1688 [=========================>....] - ETA: 0s - loss: 0.0176 - accuracy: 0.9941
1524/1688 [==========================>...] - ETA: 0s - loss: 0.0176 - accuracy: 0.9941
1534/1688 [==========================>...] - ETA: 0s - loss: 0.0176 - accuracy: 0.9941
1544/1688 [==========================>...] - ETA: 0s - loss: 0.0175 - accuracy: 0.9941
1554/1688 [==========================>...] - ETA: 0s - loss: 0.0175 - accuracy: 0.9941
1564/1688 [==========================>...] - ETA: 0s - loss: 0.0175 - accuracy: 0.9941
1574/1688 [==========================>...] - ETA: 0s - loss: 0.0175 - accuracy: 0.9941
1584/1688 [===========================>..] - ETA: 0s - loss: 0.0175 - accuracy: 0.9940
1594/1688 [===========================>..] - ETA: 0s - loss: 0.0176 - accuracy: 0.9940
1604/1688 [===========================>..] - ETA: 0s - loss: 0.0175 - accuracy: 0.9940
1614/1688 [===========================>..] - ETA: 0s - loss: 0.0176 - accuracy: 0.9940
1624/1688 [===========================>..] - ETA: 0s - loss: 0.0177 - accuracy: 0.9940
1634/1688 [============================>.] - ETA: 0s - loss: 0.0178 - accuracy: 0.9940
1643/1688 [============================>.] - ETA: 0s - loss: 0.0177 - accuracy: 0.9940
1652/1688 [============================>.] - ETA: 0s - loss: 0.0178 - accuracy: 0.9940
1661/1688 [============================>.] - ETA: 0s - loss: 0.0179 - accuracy: 0.9940
1671/1688 [============================>.] - ETA: 0s - loss: 0.0179 - accuracy: 0.9939
1680/1688 [============================>.] - ETA: 0s - loss: 0.0180 - accuracy: 0.9939
1688/1688 [==============================] - 10s 6ms/step - loss: 0.0180 - accuracy: 0.9939 - val_loss: 0.0372 - val_accuracy: 0.9917
Epoch 6/10

   1/1688 [..............................] - ETA: 0s - loss: 0.0024 - accuracy: 1.0000
  11/1688 [..............................] - ETA: 8s - loss: 0.0106 - accuracy: 0.9943
  21/1688 [..............................] - ETA: 8s - loss: 0.0271 - accuracy: 0.9926
  31/1688 [..............................] - ETA: 8s - loss: 0.0205 - accuracy: 0.9940
  41/1688 [..............................] - ETA: 8s - loss: 0.0184 - accuracy: 0.9947
  51/1688 [..............................] - ETA: 8s - loss: 0.0176 - accuracy: 0.9939
  61/1688 [>.............................] - ETA: 8s - loss: 0.0152 - accuracy: 0.9949
  71/1688 [>.............................] - ETA: 8s - loss: 0.0145 - accuracy: 0.9952
  81/1688 [>.............................] - ETA: 8s - loss: 0.0129 - accuracy: 0.9958
  91/1688 [>.............................] - ETA: 8s - loss: 0.0133 - accuracy: 0.9952
 101/1688 [>.............................] - ETA: 8s - loss: 0.0127 - accuracy: 0.9954
 111/1688 [>.............................] - ETA: 8s - loss: 0.0130 - accuracy: 0.9952
 121/1688 [=>............................] - ETA: 8s - loss: 0.0137 - accuracy: 0.9948
 131/1688 [=>............................] - ETA: 8s - loss: 0.0134 - accuracy: 0.9948
 141/1688 [=>............................] - ETA: 8s - loss: 0.0128 - accuracy: 0.9951
 151/1688 [=>............................] - ETA: 8s - loss: 0.0128 - accuracy: 0.9950
 160/1688 [=>............................] - ETA: 8s - loss: 0.0127 - accuracy: 0.9951
 169/1688 [==>...........................] - ETA: 8s - loss: 0.0124 - accuracy: 0.9952
 179/1688 [==>...........................] - ETA: 8s - loss: 0.0118 - accuracy: 0.9955
 189/1688 [==>...........................] - ETA: 8s - loss: 0.0117 - accuracy: 0.9955
 199/1688 [==>...........................] - ETA: 8s - loss: 0.0113 - accuracy: 0.9958
 209/1688 [==>...........................] - ETA: 8s - loss: 0.0108 - accuracy: 0.9960
 219/1688 [==>...........................] - ETA: 8s - loss: 0.0111 - accuracy: 0.9959
 229/1688 [===>..........................] - ETA: 7s - loss: 0.0106 - accuracy: 0.9960
 239/1688 [===>..........................] - ETA: 7s - loss: 0.0104 - accuracy: 0.9961
 248/1688 [===>..........................] - ETA: 7s - loss: 0.0110 - accuracy: 0.9960
 257/1688 [===>..........................] - ETA: 7s - loss: 0.0111 - accuracy: 0.9959
 267/1688 [===>..........................] - ETA: 7s - loss: 0.0117 - accuracy: 0.9957
 277/1688 [===>..........................] - ETA: 7s - loss: 0.0117 - accuracy: 0.9958
 287/1688 [====>.........................] - ETA: 7s - loss: 0.0124 - accuracy: 0.9956
 297/1688 [====>.........................] - ETA: 7s - loss: 0.0128 - accuracy: 0.9957
 307/1688 [====>.........................] - ETA: 7s - loss: 0.0128 - accuracy: 0.9956
 317/1688 [====>.........................] - ETA: 7s - loss: 0.0126 - accuracy: 0.9957
 327/1688 [====>.........................] - ETA: 7s - loss: 0.0123 - accuracy: 0.9958
 337/1688 [====>.........................] - ETA: 7s - loss: 0.0122 - accuracy: 0.9957
 347/1688 [=====>........................] - ETA: 7s - loss: 0.0120 - accuracy: 0.9958
 357/1688 [=====>........................] - ETA: 7s - loss: 0.0117 - accuracy: 0.9959
 367/1688 [=====>........................] - ETA: 7s - loss: 0.0117 - accuracy: 0.9958
 377/1688 [=====>........................] - ETA: 7s - loss: 0.0118 - accuracy: 0.9959
 387/1688 [=====>........................] - ETA: 7s - loss: 0.0118 - accuracy: 0.9958
 397/1688 [======>.......................] - ETA: 7s - loss: 0.0117 - accuracy: 0.9958
 407/1688 [======>.......................] - ETA: 7s - loss: 0.0119 - accuracy: 0.9958
 417/1688 [======>.......................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9957
 427/1688 [======>.......................] - ETA: 6s - loss: 0.0124 - accuracy: 0.9955
 437/1688 [======>.......................] - ETA: 6s - loss: 0.0123 - accuracy: 0.9956
 447/1688 [======>.......................] - ETA: 6s - loss: 0.0122 - accuracy: 0.9956
 456/1688 [=======>......................] - ETA: 6s - loss: 0.0124 - accuracy: 0.9955
 466/1688 [=======>......................] - ETA: 6s - loss: 0.0122 - accuracy: 0.9955
 476/1688 [=======>......................] - ETA: 6s - loss: 0.0127 - accuracy: 0.9955
 486/1688 [=======>......................] - ETA: 6s - loss: 0.0134 - accuracy: 0.9954
 496/1688 [=======>......................] - ETA: 6s - loss: 0.0133 - accuracy: 0.9955
 506/1688 [=======>......................] - ETA: 6s - loss: 0.0134 - accuracy: 0.9954
 516/1688 [========>.....................] - ETA: 6s - loss: 0.0132 - accuracy: 0.9955
 526/1688 [========>.....................] - ETA: 6s - loss: 0.0131 - accuracy: 0.9956
 536/1688 [========>.....................] - ETA: 6s - loss: 0.0130 - accuracy: 0.9957
 546/1688 [========>.....................] - ETA: 6s - loss: 0.0128 - accuracy: 0.9958
 556/1688 [========>.....................] - ETA: 6s - loss: 0.0128 - accuracy: 0.9957
 565/1688 [=========>....................] - ETA: 6s - loss: 0.0128 - accuracy: 0.9957
 574/1688 [=========>....................] - ETA: 6s - loss: 0.0126 - accuracy: 0.9958
 584/1688 [=========>....................] - ETA: 6s - loss: 0.0126 - accuracy: 0.9958
 594/1688 [=========>....................] - ETA: 6s - loss: 0.0126 - accuracy: 0.9957
 604/1688 [=========>....................] - ETA: 5s - loss: 0.0126 - accuracy: 0.9958
 614/1688 [=========>....................] - ETA: 5s - loss: 0.0125 - accuracy: 0.9958
 624/1688 [==========>...................] - ETA: 5s - loss: 0.0123 - accuracy: 0.9958
 634/1688 [==========>...................] - ETA: 5s - loss: 0.0123 - accuracy: 0.9959
 644/1688 [==========>...................] - ETA: 5s - loss: 0.0122 - accuracy: 0.9959
 654/1688 [==========>...................] - ETA: 5s - loss: 0.0122 - accuracy: 0.9959
 664/1688 [==========>...................] - ETA: 5s - loss: 0.0121 - accuracy: 0.9959
 674/1688 [==========>...................] - ETA: 5s - loss: 0.0120 - accuracy: 0.9959
 684/1688 [===========>..................] - ETA: 5s - loss: 0.0121 - accuracy: 0.9959
 694/1688 [===========>..................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9960
 704/1688 [===========>..................] - ETA: 5s - loss: 0.0118 - accuracy: 0.9960
 713/1688 [===========>..................] - ETA: 5s - loss: 0.0117 - accuracy: 0.9961
 723/1688 [===========>..................] - ETA: 5s - loss: 0.0116 - accuracy: 0.9962
 733/1688 [============>.................] - ETA: 5s - loss: 0.0115 - accuracy: 0.9962
 743/1688 [============>.................] - ETA: 5s - loss: 0.0115 - accuracy: 0.9961
 752/1688 [============>.................] - ETA: 5s - loss: 0.0115 - accuracy: 0.9962
 762/1688 [============>.................] - ETA: 5s - loss: 0.0115 - accuracy: 0.9962
 772/1688 [============>.................] - ETA: 5s - loss: 0.0115 - accuracy: 0.9962
 782/1688 [============>.................] - ETA: 4s - loss: 0.0114 - accuracy: 0.9962
 791/1688 [=============>................] - ETA: 4s - loss: 0.0114 - accuracy: 0.9962
 801/1688 [=============>................] - ETA: 4s - loss: 0.0113 - accuracy: 0.9962
 811/1688 [=============>................] - ETA: 4s - loss: 0.0112 - accuracy: 0.9963
 821/1688 [=============>................] - ETA: 4s - loss: 0.0112 - accuracy: 0.9963
 831/1688 [=============>................] - ETA: 4s - loss: 0.0111 - accuracy: 0.9963
 841/1688 [=============>................] - ETA: 4s - loss: 0.0112 - accuracy: 0.9963
 851/1688 [==============>...............] - ETA: 4s - loss: 0.0113 - accuracy: 0.9963
 861/1688 [==============>...............] - ETA: 4s - loss: 0.0113 - accuracy: 0.9963
 871/1688 [==============>...............] - ETA: 4s - loss: 0.0113 - accuracy: 0.9963
 881/1688 [==============>...............] - ETA: 4s - loss: 0.0114 - accuracy: 0.9963
 890/1688 [==============>...............] - ETA: 4s - loss: 0.0114 - accuracy: 0.9963
 900/1688 [==============>...............] - ETA: 4s - loss: 0.0114 - accuracy: 0.9963
 910/1688 [===============>..............] - ETA: 4s - loss: 0.0114 - accuracy: 0.9962
 920/1688 [===============>..............] - ETA: 4s - loss: 0.0115 - accuracy: 0.9962
 930/1688 [===============>..............] - ETA: 4s - loss: 0.0115 - accuracy: 0.9962
 940/1688 [===============>..............] - ETA: 4s - loss: 0.0114 - accuracy: 0.9961
 950/1688 [===============>..............] - ETA: 4s - loss: 0.0114 - accuracy: 0.9961
 960/1688 [================>.............] - ETA: 4s - loss: 0.0114 - accuracy: 0.9961
 970/1688 [================>.............] - ETA: 3s - loss: 0.0116 - accuracy: 0.9960
 980/1688 [================>.............] - ETA: 3s - loss: 0.0116 - accuracy: 0.9960
 990/1688 [================>.............] - ETA: 3s - loss: 0.0115 - accuracy: 0.9961
1000/1688 [================>.............] - ETA: 3s - loss: 0.0116 - accuracy: 0.9960
1010/1688 [================>.............] - ETA: 3s - loss: 0.0120 - accuracy: 0.9959
1019/1688 [=================>............] - ETA: 3s - loss: 0.0120 - accuracy: 0.9960
1029/1688 [=================>............] - ETA: 3s - loss: 0.0119 - accuracy: 0.9960
1039/1688 [=================>............] - ETA: 3s - loss: 0.0121 - accuracy: 0.9959
1049/1688 [=================>............] - ETA: 3s - loss: 0.0121 - accuracy: 0.9959
1059/1688 [=================>............] - ETA: 3s - loss: 0.0123 - accuracy: 0.9958
1069/1688 [=================>............] - ETA: 3s - loss: 0.0123 - accuracy: 0.9958
1079/1688 [==================>...........] - ETA: 3s - loss: 0.0123 - accuracy: 0.9958
1089/1688 [==================>...........] - ETA: 3s - loss: 0.0123 - accuracy: 0.9958
1099/1688 [==================>...........] - ETA: 3s - loss: 0.0125 - accuracy: 0.9957
1109/1688 [==================>...........] - ETA: 3s - loss: 0.0126 - accuracy: 0.9957
1118/1688 [==================>...........] - ETA: 3s - loss: 0.0126 - accuracy: 0.9956
1128/1688 [===================>..........] - ETA: 3s - loss: 0.0126 - accuracy: 0.9956
1138/1688 [===================>..........] - ETA: 3s - loss: 0.0127 - accuracy: 0.9955
1148/1688 [===================>..........] - ETA: 2s - loss: 0.0128 - accuracy: 0.9955
1158/1688 [===================>..........] - ETA: 2s - loss: 0.0127 - accuracy: 0.9955
1168/1688 [===================>..........] - ETA: 2s - loss: 0.0127 - accuracy: 0.9956
1178/1688 [===================>..........] - ETA: 2s - loss: 0.0128 - accuracy: 0.9956
1188/1688 [====================>.........] - ETA: 2s - loss: 0.0128 - accuracy: 0.9956
1198/1688 [====================>.........] - ETA: 2s - loss: 0.0127 - accuracy: 0.9956
1208/1688 [====================>.........] - ETA: 2s - loss: 0.0127 - accuracy: 0.9956
1218/1688 [====================>.........] - ETA: 2s - loss: 0.0127 - accuracy: 0.9955
1228/1688 [====================>.........] - ETA: 2s - loss: 0.0128 - accuracy: 0.9955
1238/1688 [=====================>........] - ETA: 2s - loss: 0.0127 - accuracy: 0.9955
1248/1688 [=====================>........] - ETA: 2s - loss: 0.0127 - accuracy: 0.9955
1257/1688 [=====================>........] - ETA: 2s - loss: 0.0126 - accuracy: 0.9955
1266/1688 [=====================>........] - ETA: 2s - loss: 0.0126 - accuracy: 0.9956
1276/1688 [=====================>........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9956
1286/1688 [=====================>........] - ETA: 2s - loss: 0.0125 - accuracy: 0.9956
1296/1688 [======================>.......] - ETA: 2s - loss: 0.0124 - accuracy: 0.9957
1306/1688 [======================>.......] - ETA: 2s - loss: 0.0124 - accuracy: 0.9956
1316/1688 [======================>.......] - ETA: 2s - loss: 0.0123 - accuracy: 0.9957
1326/1688 [======================>.......] - ETA: 1s - loss: 0.0123 - accuracy: 0.9957
1336/1688 [======================>.......] - ETA: 1s - loss: 0.0123 - accuracy: 0.9957
1345/1688 [======================>.......] - ETA: 1s - loss: 0.0123 - accuracy: 0.9957
1354/1688 [=======================>......] - ETA: 1s - loss: 0.0122 - accuracy: 0.9957
1364/1688 [=======================>......] - ETA: 1s - loss: 0.0122 - accuracy: 0.9957
1374/1688 [=======================>......] - ETA: 1s - loss: 0.0122 - accuracy: 0.9957
1384/1688 [=======================>......] - ETA: 1s - loss: 0.0121 - accuracy: 0.9957
1394/1688 [=======================>......] - ETA: 1s - loss: 0.0121 - accuracy: 0.9957
1404/1688 [=======================>......] - ETA: 1s - loss: 0.0124 - accuracy: 0.9957
1414/1688 [========================>.....] - ETA: 1s - loss: 0.0123 - accuracy: 0.9957
1424/1688 [========================>.....] - ETA: 1s - loss: 0.0123 - accuracy: 0.9957
1434/1688 [========================>.....] - ETA: 1s - loss: 0.0124 - accuracy: 0.9957
1444/1688 [========================>.....] - ETA: 1s - loss: 0.0123 - accuracy: 0.9957
1454/1688 [========================>.....] - ETA: 1s - loss: 0.0124 - accuracy: 0.9957
1464/1688 [=========================>....] - ETA: 1s - loss: 0.0123 - accuracy: 0.9957
1473/1688 [=========================>....] - ETA: 1s - loss: 0.0123 - accuracy: 0.9957
1482/1688 [=========================>....] - ETA: 1s - loss: 0.0123 - accuracy: 0.9957
1492/1688 [=========================>....] - ETA: 1s - loss: 0.0123 - accuracy: 0.9957
1502/1688 [=========================>....] - ETA: 1s - loss: 0.0122 - accuracy: 0.9957
1512/1688 [=========================>....] - ETA: 0s - loss: 0.0122 - accuracy: 0.9958
1522/1688 [==========================>...] - ETA: 0s - loss: 0.0122 - accuracy: 0.9958
1532/1688 [==========================>...] - ETA: 0s - loss: 0.0122 - accuracy: 0.9958
1542/1688 [==========================>...] - ETA: 0s - loss: 0.0122 - accuracy: 0.9958
1552/1688 [==========================>...] - ETA: 0s - loss: 0.0121 - accuracy: 0.9958
1562/1688 [==========================>...] - ETA: 0s - loss: 0.0122 - accuracy: 0.9958
1572/1688 [==========================>...] - ETA: 0s - loss: 0.0122 - accuracy: 0.9958
1582/1688 [===========================>..] - ETA: 0s - loss: 0.0121 - accuracy: 0.9958
1592/1688 [===========================>..] - ETA: 0s - loss: 0.0121 - accuracy: 0.9958
1602/1688 [===========================>..] - ETA: 0s - loss: 0.0121 - accuracy: 0.9958
1611/1688 [===========================>..] - ETA: 0s - loss: 0.0121 - accuracy: 0.9958
1621/1688 [===========================>..] - ETA: 0s - loss: 0.0121 - accuracy: 0.9959
1630/1688 [===========================>..] - ETA: 0s - loss: 0.0120 - accuracy: 0.9959
1640/1688 [============================>.] - ETA: 0s - loss: 0.0121 - accuracy: 0.9958
1650/1688 [============================>.] - ETA: 0s - loss: 0.0121 - accuracy: 0.9958
1659/1688 [============================>.] - ETA: 0s - loss: 0.0120 - accuracy: 0.9958
1669/1688 [============================>.] - ETA: 0s - loss: 0.0120 - accuracy: 0.9958
1678/1688 [============================>.] - ETA: 0s - loss: 0.0121 - accuracy: 0.9958
1687/1688 [============================>.] - ETA: 0s - loss: 0.0121 - accuracy: 0.9958
1688/1688 [==============================] - 10s 6ms/step - loss: 0.0121 - accuracy: 0.9958 - val_loss: 0.0512 - val_accuracy: 0.9877
Epoch 7/10

   1/1688 [..............................] - ETA: 0s - loss: 0.0484 - accuracy: 0.9688
  11/1688 [..............................] - ETA: 8s - loss: 0.0132 - accuracy: 0.9943
  20/1688 [..............................] - ETA: 8s - loss: 0.0094 - accuracy: 0.9969
  30/1688 [..............................] - ETA: 8s - loss: 0.0104 - accuracy: 0.9958
  40/1688 [..............................] - ETA: 8s - loss: 0.0134 - accuracy: 0.9953
  49/1688 [..............................] - ETA: 8s - loss: 0.0155 - accuracy: 0.9949
  59/1688 [>.............................] - ETA: 8s - loss: 0.0141 - accuracy: 0.9952
  68/1688 [>.............................] - ETA: 8s - loss: 0.0131 - accuracy: 0.9959
  78/1688 [>.............................] - ETA: 8s - loss: 0.0124 - accuracy: 0.9960
  88/1688 [>.............................] - ETA: 8s - loss: 0.0118 - accuracy: 0.9961
  97/1688 [>.............................] - ETA: 8s - loss: 0.0116 - accuracy: 0.9961
 107/1688 [>.............................] - ETA: 8s - loss: 0.0120 - accuracy: 0.9962
 116/1688 [=>............................] - ETA: 8s - loss: 0.0118 - accuracy: 0.9962
 125/1688 [=>............................] - ETA: 8s - loss: 0.0121 - accuracy: 0.9962
 134/1688 [=>............................] - ETA: 8s - loss: 0.0121 - accuracy: 0.9963
 144/1688 [=>............................] - ETA: 8s - loss: 0.0120 - accuracy: 0.9963
 154/1688 [=>............................] - ETA: 8s - loss: 0.0113 - accuracy: 0.9966
 164/1688 [=>............................] - ETA: 8s - loss: 0.0109 - accuracy: 0.9968
 174/1688 [==>...........................] - ETA: 8s - loss: 0.0104 - accuracy: 0.9969
 184/1688 [==>...........................] - ETA: 8s - loss: 0.0099 - accuracy: 0.9971
 194/1688 [==>...........................] - ETA: 8s - loss: 0.0102 - accuracy: 0.9969
 204/1688 [==>...........................] - ETA: 8s - loss: 0.0104 - accuracy: 0.9968
 213/1688 [==>...........................] - ETA: 8s - loss: 0.0101 - accuracy: 0.9969
 223/1688 [==>...........................] - ETA: 8s - loss: 0.0105 - accuracy: 0.9968
 232/1688 [===>..........................] - ETA: 8s - loss: 0.0105 - accuracy: 0.9968
 242/1688 [===>..........................] - ETA: 7s - loss: 0.0107 - accuracy: 0.9965
 252/1688 [===>..........................] - ETA: 7s - loss: 0.0104 - accuracy: 0.9967
 262/1688 [===>..........................] - ETA: 7s - loss: 0.0104 - accuracy: 0.9965
 272/1688 [===>..........................] - ETA: 7s - loss: 0.0104 - accuracy: 0.9966
 281/1688 [===>..........................] - ETA: 7s - loss: 0.0110 - accuracy: 0.9963
 291/1688 [====>.........................] - ETA: 7s - loss: 0.0109 - accuracy: 0.9963
 300/1688 [====>.........................] - ETA: 7s - loss: 0.0108 - accuracy: 0.9964
 310/1688 [====>.........................] - ETA: 7s - loss: 0.0108 - accuracy: 0.9962
 320/1688 [====>.........................] - ETA: 7s - loss: 0.0107 - accuracy: 0.9962
 330/1688 [====>.........................] - ETA: 7s - loss: 0.0104 - accuracy: 0.9963
 340/1688 [=====>........................] - ETA: 7s - loss: 0.0102 - accuracy: 0.9964
 350/1688 [=====>........................] - ETA: 7s - loss: 0.0101 - accuracy: 0.9964
 360/1688 [=====>........................] - ETA: 7s - loss: 0.0104 - accuracy: 0.9963
 370/1688 [=====>........................] - ETA: 7s - loss: 0.0102 - accuracy: 0.9964
 380/1688 [=====>........................] - ETA: 7s - loss: 0.0100 - accuracy: 0.9965
 390/1688 [=====>........................] - ETA: 7s - loss: 0.0101 - accuracy: 0.9965
 400/1688 [======>.......................] - ETA: 7s - loss: 0.0102 - accuracy: 0.9964
 410/1688 [======>.......................] - ETA: 7s - loss: 0.0107 - accuracy: 0.9963
 420/1688 [======>.......................] - ETA: 6s - loss: 0.0106 - accuracy: 0.9963
 430/1688 [======>.......................] - ETA: 6s - loss: 0.0104 - accuracy: 0.9964
 440/1688 [======>.......................] - ETA: 6s - loss: 0.0103 - accuracy: 0.9964
 450/1688 [======>.......................] - ETA: 6s - loss: 0.0102 - accuracy: 0.9965
 459/1688 [=======>......................] - ETA: 6s - loss: 0.0100 - accuracy: 0.9965
 469/1688 [=======>......................] - ETA: 6s - loss: 0.0102 - accuracy: 0.9964
 479/1688 [=======>......................] - ETA: 6s - loss: 0.0103 - accuracy: 0.9964
 489/1688 [=======>......................] - ETA: 6s - loss: 0.0102 - accuracy: 0.9964
 498/1688 [=======>......................] - ETA: 6s - loss: 0.0103 - accuracy: 0.9964
 507/1688 [========>.....................] - ETA: 6s - loss: 0.0105 - accuracy: 0.9964
 517/1688 [========>.....................] - ETA: 6s - loss: 0.0105 - accuracy: 0.9964
 527/1688 [========>.....................] - ETA: 6s - loss: 0.0104 - accuracy: 0.9964
 537/1688 [========>.....................] - ETA: 6s - loss: 0.0104 - accuracy: 0.9964
 547/1688 [========>.....................] - ETA: 6s - loss: 0.0104 - accuracy: 0.9963
 557/1688 [========>.....................] - ETA: 6s - loss: 0.0104 - accuracy: 0.9964
 567/1688 [=========>....................] - ETA: 6s - loss: 0.0102 - accuracy: 0.9964
 577/1688 [=========>....................] - ETA: 6s - loss: 0.0102 - accuracy: 0.9964
 587/1688 [=========>....................] - ETA: 6s - loss: 0.0101 - accuracy: 0.9964
 597/1688 [=========>....................] - ETA: 6s - loss: 0.0101 - accuracy: 0.9964
 606/1688 [=========>....................] - ETA: 5s - loss: 0.0101 - accuracy: 0.9964
 615/1688 [=========>....................] - ETA: 5s - loss: 0.0101 - accuracy: 0.9964
 625/1688 [==========>...................] - ETA: 5s - loss: 0.0100 - accuracy: 0.9965
 635/1688 [==========>...................] - ETA: 5s - loss: 0.0100 - accuracy: 0.9965
 645/1688 [==========>...................] - ETA: 5s - loss: 0.0099 - accuracy: 0.9966
 655/1688 [==========>...................] - ETA: 5s - loss: 0.0099 - accuracy: 0.9965
 665/1688 [==========>...................] - ETA: 5s - loss: 0.0100 - accuracy: 0.9965
 675/1688 [==========>...................] - ETA: 5s - loss: 0.0101 - accuracy: 0.9965
 685/1688 [===========>..................] - ETA: 5s - loss: 0.0100 - accuracy: 0.9965
 695/1688 [===========>..................] - ETA: 5s - loss: 0.0099 - accuracy: 0.9965
 705/1688 [===========>..................] - ETA: 5s - loss: 0.0098 - accuracy: 0.9966
 715/1688 [===========>..................] - ETA: 5s - loss: 0.0098 - accuracy: 0.9966
 725/1688 [===========>..................] - ETA: 5s - loss: 0.0100 - accuracy: 0.9966
 735/1688 [============>.................] - ETA: 5s - loss: 0.0100 - accuracy: 0.9966
 744/1688 [============>.................] - ETA: 5s - loss: 0.0099 - accuracy: 0.9966
 754/1688 [============>.................] - ETA: 5s - loss: 0.0099 - accuracy: 0.9966
 764/1688 [============>.................] - ETA: 5s - loss: 0.0098 - accuracy: 0.9966
 774/1688 [============>.................] - ETA: 5s - loss: 0.0098 - accuracy: 0.9966
 784/1688 [============>.................] - ETA: 4s - loss: 0.0099 - accuracy: 0.9967
 794/1688 [=============>................] - ETA: 4s - loss: 0.0099 - accuracy: 0.9967
 804/1688 [=============>................] - ETA: 4s - loss: 0.0098 - accuracy: 0.9967
 814/1688 [=============>................] - ETA: 4s - loss: 0.0098 - accuracy: 0.9967
 824/1688 [=============>................] - ETA: 4s - loss: 0.0098 - accuracy: 0.9967
 834/1688 [=============>................] - ETA: 4s - loss: 0.0097 - accuracy: 0.9967
 844/1688 [==============>...............] - ETA: 4s - loss: 0.0097 - accuracy: 0.9967
 854/1688 [==============>...............] - ETA: 4s - loss: 0.0097 - accuracy: 0.9967
 864/1688 [==============>...............] - ETA: 4s - loss: 0.0097 - accuracy: 0.9967
 874/1688 [==============>...............] - ETA: 4s - loss: 0.0096 - accuracy: 0.9967
 884/1688 [==============>...............] - ETA: 4s - loss: 0.0098 - accuracy: 0.9967
 894/1688 [==============>...............] - ETA: 4s - loss: 0.0098 - accuracy: 0.9967
 904/1688 [===============>..............] - ETA: 4s - loss: 0.0098 - accuracy: 0.9967
 914/1688 [===============>..............] - ETA: 4s - loss: 0.0098 - accuracy: 0.9967
 924/1688 [===============>..............] - ETA: 4s - loss: 0.0098 - accuracy: 0.9968
 934/1688 [===============>..............] - ETA: 4s - loss: 0.0097 - accuracy: 0.9968
 944/1688 [===============>..............] - ETA: 4s - loss: 0.0097 - accuracy: 0.9968
 954/1688 [===============>..............] - ETA: 4s - loss: 0.0098 - accuracy: 0.9967
 964/1688 [================>.............] - ETA: 3s - loss: 0.0099 - accuracy: 0.9967
 973/1688 [================>.............] - ETA: 3s - loss: 0.0099 - accuracy: 0.9967
 983/1688 [================>.............] - ETA: 3s - loss: 0.0099 - accuracy: 0.9967
 993/1688 [================>.............] - ETA: 3s - loss: 0.0099 - accuracy: 0.9967
1003/1688 [================>.............] - ETA: 3s - loss: 0.0098 - accuracy: 0.9968
1013/1688 [=================>............] - ETA: 3s - loss: 0.0098 - accuracy: 0.9968
1023/1688 [=================>............] - ETA: 3s - loss: 0.0097 - accuracy: 0.9968
1033/1688 [=================>............] - ETA: 3s - loss: 0.0097 - accuracy: 0.9968
1043/1688 [=================>............] - ETA: 3s - loss: 0.0098 - accuracy: 0.9967
1053/1688 [=================>............] - ETA: 3s - loss: 0.0098 - accuracy: 0.9967
1063/1688 [=================>............] - ETA: 3s - loss: 0.0097 - accuracy: 0.9968
1072/1688 [==================>...........] - ETA: 3s - loss: 0.0097 - accuracy: 0.9968
1082/1688 [==================>...........] - ETA: 3s - loss: 0.0097 - accuracy: 0.9968
1092/1688 [==================>...........] - ETA: 3s - loss: 0.0097 - accuracy: 0.9968
1102/1688 [==================>...........] - ETA: 3s - loss: 0.0099 - accuracy: 0.9968
1112/1688 [==================>...........] - ETA: 3s - loss: 0.0099 - accuracy: 0.9968
1122/1688 [==================>...........] - ETA: 3s - loss: 0.0099 - accuracy: 0.9968
1132/1688 [===================>..........] - ETA: 3s - loss: 0.0100 - accuracy: 0.9967
1141/1688 [===================>..........] - ETA: 3s - loss: 0.0101 - accuracy: 0.9967
1151/1688 [===================>..........] - ETA: 2s - loss: 0.0102 - accuracy: 0.9967
1160/1688 [===================>..........] - ETA: 2s - loss: 0.0103 - accuracy: 0.9966
1170/1688 [===================>..........] - ETA: 2s - loss: 0.0103 - accuracy: 0.9966
1180/1688 [===================>..........] - ETA: 2s - loss: 0.0102 - accuracy: 0.9967
1189/1688 [====================>.........] - ETA: 2s - loss: 0.0102 - accuracy: 0.9967
1199/1688 [====================>.........] - ETA: 2s - loss: 0.0101 - accuracy: 0.9967
1209/1688 [====================>.........] - ETA: 2s - loss: 0.0102 - accuracy: 0.9966
1218/1688 [====================>.........] - ETA: 2s - loss: 0.0102 - accuracy: 0.9966
1228/1688 [====================>.........] - ETA: 2s - loss: 0.0102 - accuracy: 0.9966
1237/1688 [====================>.........] - ETA: 2s - loss: 0.0101 - accuracy: 0.9966
1247/1688 [=====================>........] - ETA: 2s - loss: 0.0101 - accuracy: 0.9967
1257/1688 [=====================>........] - ETA: 2s - loss: 0.0102 - accuracy: 0.9966
1267/1688 [=====================>........] - ETA: 2s - loss: 0.0102 - accuracy: 0.9966
1276/1688 [=====================>........] - ETA: 2s - loss: 0.0102 - accuracy: 0.9966
1286/1688 [=====================>........] - ETA: 2s - loss: 0.0103 - accuracy: 0.9966
1296/1688 [======================>.......] - ETA: 2s - loss: 0.0103 - accuracy: 0.9966
1306/1688 [======================>.......] - ETA: 2s - loss: 0.0103 - accuracy: 0.9967
1316/1688 [======================>.......] - ETA: 2s - loss: 0.0103 - accuracy: 0.9966
1326/1688 [======================>.......] - ETA: 1s - loss: 0.0103 - accuracy: 0.9966
1336/1688 [======================>.......] - ETA: 1s - loss: 0.0104 - accuracy: 0.9966
1346/1688 [======================>.......] - ETA: 1s - loss: 0.0104 - accuracy: 0.9966
1356/1688 [=======================>......] - ETA: 1s - loss: 0.0104 - accuracy: 0.9966
1366/1688 [=======================>......] - ETA: 1s - loss: 0.0103 - accuracy: 0.9966
1376/1688 [=======================>......] - ETA: 1s - loss: 0.0104 - accuracy: 0.9966
1385/1688 [=======================>......] - ETA: 1s - loss: 0.0104 - accuracy: 0.9966
1395/1688 [=======================>......] - ETA: 1s - loss: 0.0104 - accuracy: 0.9966
1405/1688 [=======================>......] - ETA: 1s - loss: 0.0105 - accuracy: 0.9966
1415/1688 [========================>.....] - ETA: 1s - loss: 0.0105 - accuracy: 0.9966
1425/1688 [========================>.....] - ETA: 1s - loss: 0.0106 - accuracy: 0.9966
1435/1688 [========================>.....] - ETA: 1s - loss: 0.0106 - accuracy: 0.9966
1444/1688 [========================>.....] - ETA: 1s - loss: 0.0106 - accuracy: 0.9966
1454/1688 [========================>.....] - ETA: 1s - loss: 0.0107 - accuracy: 0.9965
1464/1688 [=========================>....] - ETA: 1s - loss: 0.0107 - accuracy: 0.9965
1474/1688 [=========================>....] - ETA: 1s - loss: 0.0107 - accuracy: 0.9965
1484/1688 [=========================>....] - ETA: 1s - loss: 0.0108 - accuracy: 0.9965
1494/1688 [=========================>....] - ETA: 1s - loss: 0.0108 - accuracy: 0.9965
1503/1688 [=========================>....] - ETA: 1s - loss: 0.0108 - accuracy: 0.9965
1513/1688 [=========================>....] - ETA: 0s - loss: 0.0109 - accuracy: 0.9964
1523/1688 [==========================>...] - ETA: 0s - loss: 0.0111 - accuracy: 0.9964
1533/1688 [==========================>...] - ETA: 0s - loss: 0.0111 - accuracy: 0.9964
1543/1688 [==========================>...] - ETA: 0s - loss: 0.0111 - accuracy: 0.9964
1553/1688 [==========================>...] - ETA: 0s - loss: 0.0111 - accuracy: 0.9964
1563/1688 [==========================>...] - ETA: 0s - loss: 0.0110 - accuracy: 0.9964
1573/1688 [==========================>...] - ETA: 0s - loss: 0.0110 - accuracy: 0.9965
1583/1688 [===========================>..] - ETA: 0s - loss: 0.0109 - accuracy: 0.9965
1593/1688 [===========================>..] - ETA: 0s - loss: 0.0109 - accuracy: 0.9965
1603/1688 [===========================>..] - ETA: 0s - loss: 0.0108 - accuracy: 0.9965
1613/1688 [===========================>..] - ETA: 0s - loss: 0.0108 - accuracy: 0.9966
1623/1688 [===========================>..] - ETA: 0s - loss: 0.0108 - accuracy: 0.9966
1633/1688 [============================>.] - ETA: 0s - loss: 0.0108 - accuracy: 0.9965
1643/1688 [============================>.] - ETA: 0s - loss: 0.0107 - accuracy: 0.9966
1653/1688 [============================>.] - ETA: 0s - loss: 0.0109 - accuracy: 0.9965
1663/1688 [============================>.] - ETA: 0s - loss: 0.0111 - accuracy: 0.9965
1673/1688 [============================>.] - ETA: 0s - loss: 0.0112 - accuracy: 0.9964
1683/1688 [============================>.] - ETA: 0s - loss: 0.0112 - accuracy: 0.9964
1688/1688 [==============================] - 10s 6ms/step - loss: 0.0112 - accuracy: 0.9964 - val_loss: 0.0501 - val_accuracy: 0.9893
Epoch 8/10

   1/1688 [..............................] - ETA: 0s - loss: 0.0804 - accuracy: 0.9688
  11/1688 [..............................] - ETA: 8s - loss: 0.0097 - accuracy: 0.9972
  21/1688 [..............................] - ETA: 8s - loss: 0.0089 - accuracy: 0.9970
  31/1688 [..............................] - ETA: 8s - loss: 0.0070 - accuracy: 0.9980
  41/1688 [..............................] - ETA: 8s - loss: 0.0056 - accuracy: 0.9985
  51/1688 [..............................] - ETA: 8s - loss: 0.0048 - accuracy: 0.9988
  61/1688 [>.............................] - ETA: 8s - loss: 0.0072 - accuracy: 0.9980
  71/1688 [>.............................] - ETA: 8s - loss: 0.0067 - accuracy: 0.9982
  81/1688 [>.............................] - ETA: 8s - loss: 0.0063 - accuracy: 0.9985
  91/1688 [>.............................] - ETA: 8s - loss: 0.0058 - accuracy: 0.9986
 101/1688 [>.............................] - ETA: 8s - loss: 0.0054 - accuracy: 0.9988
 111/1688 [>.............................] - ETA: 8s - loss: 0.0053 - accuracy: 0.9986
 121/1688 [=>............................] - ETA: 8s - loss: 0.0050 - accuracy: 0.9987
 131/1688 [=>............................] - ETA: 8s - loss: 0.0046 - accuracy: 0.9988
 140/1688 [=>............................] - ETA: 8s - loss: 0.0046 - accuracy: 0.9989
 150/1688 [=>............................] - ETA: 8s - loss: 0.0044 - accuracy: 0.9990
 160/1688 [=>............................] - ETA: 8s - loss: 0.0051 - accuracy: 0.9988
 170/1688 [==>...........................] - ETA: 8s - loss: 0.0048 - accuracy: 0.9989
 180/1688 [==>...........................] - ETA: 8s - loss: 0.0047 - accuracy: 0.9990
 190/1688 [==>...........................] - ETA: 8s - loss: 0.0045 - accuracy: 0.9990
 200/1688 [==>...........................] - ETA: 8s - loss: 0.0045 - accuracy: 0.9991
 210/1688 [==>...........................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9991
 219/1688 [==>...........................] - ETA: 8s - loss: 0.0050 - accuracy: 0.9989
 229/1688 [===>..........................] - ETA: 7s - loss: 0.0052 - accuracy: 0.9988
 238/1688 [===>..........................] - ETA: 7s - loss: 0.0051 - accuracy: 0.9988
 248/1688 [===>..........................] - ETA: 7s - loss: 0.0056 - accuracy: 0.9987
 257/1688 [===>..........................] - ETA: 7s - loss: 0.0056 - accuracy: 0.9987
 267/1688 [===>..........................] - ETA: 7s - loss: 0.0056 - accuracy: 0.9987
 277/1688 [===>..........................] - ETA: 7s - loss: 0.0055 - accuracy: 0.9988
 287/1688 [====>.........................] - ETA: 7s - loss: 0.0054 - accuracy: 0.9988
 297/1688 [====>.........................] - ETA: 7s - loss: 0.0055 - accuracy: 0.9987
 307/1688 [====>.........................] - ETA: 7s - loss: 0.0056 - accuracy: 0.9987
 317/1688 [====>.........................] - ETA: 7s - loss: 0.0054 - accuracy: 0.9987
 326/1688 [====>.........................] - ETA: 7s - loss: 0.0054 - accuracy: 0.9988
 336/1688 [====>.........................] - ETA: 7s - loss: 0.0054 - accuracy: 0.9987
 346/1688 [=====>........................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9987
 356/1688 [=====>........................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9988
 366/1688 [=====>........................] - ETA: 7s - loss: 0.0052 - accuracy: 0.9988
 376/1688 [=====>........................] - ETA: 7s - loss: 0.0052 - accuracy: 0.9988
 386/1688 [=====>........................] - ETA: 7s - loss: 0.0052 - accuracy: 0.9987
 395/1688 [======>.......................] - ETA: 7s - loss: 0.0052 - accuracy: 0.9987
 405/1688 [======>.......................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9986
 415/1688 [======>.......................] - ETA: 6s - loss: 0.0052 - accuracy: 0.9986
 425/1688 [======>.......................] - ETA: 6s - loss: 0.0056 - accuracy: 0.9986
 434/1688 [======>.......................] - ETA: 6s - loss: 0.0055 - accuracy: 0.9986
 444/1688 [======>.......................] - ETA: 6s - loss: 0.0054 - accuracy: 0.9987
 454/1688 [=======>......................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9987
 464/1688 [=======>......................] - ETA: 6s - loss: 0.0052 - accuracy: 0.9987
 474/1688 [=======>......................] - ETA: 6s - loss: 0.0052 - accuracy: 0.9987
 484/1688 [=======>......................] - ETA: 6s - loss: 0.0051 - accuracy: 0.9988
 494/1688 [=======>......................] - ETA: 6s - loss: 0.0051 - accuracy: 0.9987
 503/1688 [=======>......................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9988
 512/1688 [========>.....................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9988
 522/1688 [========>.....................] - ETA: 6s - loss: 0.0049 - accuracy: 0.9988
 532/1688 [========>.....................] - ETA: 6s - loss: 0.0049 - accuracy: 0.9988
 542/1688 [========>.....................] - ETA: 6s - loss: 0.0049 - accuracy: 0.9988
 552/1688 [========>.....................] - ETA: 6s - loss: 0.0048 - accuracy: 0.9988
 562/1688 [========>.....................] - ETA: 6s - loss: 0.0048 - accuracy: 0.9988
 572/1688 [=========>....................] - ETA: 6s - loss: 0.0048 - accuracy: 0.9989
 581/1688 [=========>....................] - ETA: 6s - loss: 0.0049 - accuracy: 0.9988
 591/1688 [=========>....................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9988
 601/1688 [=========>....................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9988
 610/1688 [=========>....................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9988
 620/1688 [==========>...................] - ETA: 5s - loss: 0.0048 - accuracy: 0.9988
 630/1688 [==========>...................] - ETA: 5s - loss: 0.0048 - accuracy: 0.9989
 640/1688 [==========>...................] - ETA: 5s - loss: 0.0052 - accuracy: 0.9988
 650/1688 [==========>...................] - ETA: 5s - loss: 0.0052 - accuracy: 0.9988
 660/1688 [==========>...................] - ETA: 5s - loss: 0.0053 - accuracy: 0.9988
 669/1688 [==========>...................] - ETA: 5s - loss: 0.0052 - accuracy: 0.9988
 678/1688 [===========>..................] - ETA: 5s - loss: 0.0053 - accuracy: 0.9988
 687/1688 [===========>..................] - ETA: 5s - loss: 0.0052 - accuracy: 0.9988
 696/1688 [===========>..................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9987
 706/1688 [===========>..................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9987
 716/1688 [===========>..................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9986
 726/1688 [===========>..................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9986
 736/1688 [============>.................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9986
 746/1688 [============>.................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9986
 756/1688 [============>.................] - ETA: 5s - loss: 0.0056 - accuracy: 0.9985
 765/1688 [============>.................] - ETA: 5s - loss: 0.0056 - accuracy: 0.9985
 775/1688 [============>.................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9985
 785/1688 [============>.................] - ETA: 4s - loss: 0.0055 - accuracy: 0.9985
 795/1688 [=============>................] - ETA: 4s - loss: 0.0055 - accuracy: 0.9985
 805/1688 [=============>................] - ETA: 4s - loss: 0.0055 - accuracy: 0.9985
 815/1688 [=============>................] - ETA: 4s - loss: 0.0054 - accuracy: 0.9985
 825/1688 [=============>................] - ETA: 4s - loss: 0.0054 - accuracy: 0.9985
 835/1688 [=============>................] - ETA: 4s - loss: 0.0054 - accuracy: 0.9985
 845/1688 [==============>...............] - ETA: 4s - loss: 0.0057 - accuracy: 0.9985
 855/1688 [==============>...............] - ETA: 4s - loss: 0.0056 - accuracy: 0.9985
 865/1688 [==============>...............] - ETA: 4s - loss: 0.0057 - accuracy: 0.9984
 875/1688 [==============>...............] - ETA: 4s - loss: 0.0058 - accuracy: 0.9984
 885/1688 [==============>...............] - ETA: 4s - loss: 0.0062 - accuracy: 0.9983
 895/1688 [==============>...............] - ETA: 4s - loss: 0.0063 - accuracy: 0.9982
 905/1688 [===============>..............] - ETA: 4s - loss: 0.0065 - accuracy: 0.9982
 915/1688 [===============>..............] - ETA: 4s - loss: 0.0065 - accuracy: 0.9982
 924/1688 [===============>..............] - ETA: 4s - loss: 0.0065 - accuracy: 0.9982
 934/1688 [===============>..............] - ETA: 4s - loss: 0.0065 - accuracy: 0.9982
 944/1688 [===============>..............] - ETA: 4s - loss: 0.0067 - accuracy: 0.9982
 954/1688 [===============>..............] - ETA: 4s - loss: 0.0067 - accuracy: 0.9982
 964/1688 [================>.............] - ETA: 3s - loss: 0.0067 - accuracy: 0.9982
 974/1688 [================>.............] - ETA: 3s - loss: 0.0067 - accuracy: 0.9982
 984/1688 [================>.............] - ETA: 3s - loss: 0.0068 - accuracy: 0.9981
 994/1688 [================>.............] - ETA: 3s - loss: 0.0068 - accuracy: 0.9981
1004/1688 [================>.............] - ETA: 3s - loss: 0.0069 - accuracy: 0.9981
1014/1688 [=================>............] - ETA: 3s - loss: 0.0069 - accuracy: 0.9981
1024/1688 [=================>............] - ETA: 3s - loss: 0.0069 - accuracy: 0.9981
1033/1688 [=================>............] - ETA: 3s - loss: 0.0068 - accuracy: 0.9981
1042/1688 [=================>............] - ETA: 3s - loss: 0.0071 - accuracy: 0.9980
1052/1688 [=================>............] - ETA: 3s - loss: 0.0071 - accuracy: 0.9980
1061/1688 [=================>............] - ETA: 3s - loss: 0.0072 - accuracy: 0.9980
1071/1688 [==================>...........] - ETA: 3s - loss: 0.0073 - accuracy: 0.9979
1081/1688 [==================>...........] - ETA: 3s - loss: 0.0072 - accuracy: 0.9979
1091/1688 [==================>...........] - ETA: 3s - loss: 0.0077 - accuracy: 0.9978
1101/1688 [==================>...........] - ETA: 3s - loss: 0.0078 - accuracy: 0.9978
1111/1688 [==================>...........] - ETA: 3s - loss: 0.0078 - accuracy: 0.9977
1121/1688 [==================>...........] - ETA: 3s - loss: 0.0077 - accuracy: 0.9977
1130/1688 [===================>..........] - ETA: 3s - loss: 0.0077 - accuracy: 0.9977
1140/1688 [===================>..........] - ETA: 3s - loss: 0.0078 - accuracy: 0.9977
1150/1688 [===================>..........] - ETA: 2s - loss: 0.0078 - accuracy: 0.9977
1160/1688 [===================>..........] - ETA: 2s - loss: 0.0080 - accuracy: 0.9977
1170/1688 [===================>..........] - ETA: 2s - loss: 0.0080 - accuracy: 0.9976
1180/1688 [===================>..........] - ETA: 2s - loss: 0.0081 - accuracy: 0.9976
1190/1688 [====================>.........] - ETA: 2s - loss: 0.0081 - accuracy: 0.9976
1200/1688 [====================>.........] - ETA: 2s - loss: 0.0080 - accuracy: 0.9977
1210/1688 [====================>.........] - ETA: 2s - loss: 0.0081 - accuracy: 0.9976
1220/1688 [====================>.........] - ETA: 2s - loss: 0.0081 - accuracy: 0.9976
1230/1688 [====================>.........] - ETA: 2s - loss: 0.0080 - accuracy: 0.9977
1239/1688 [=====================>........] - ETA: 2s - loss: 0.0080 - accuracy: 0.9977
1249/1688 [=====================>........] - ETA: 2s - loss: 0.0081 - accuracy: 0.9976
1258/1688 [=====================>........] - ETA: 2s - loss: 0.0082 - accuracy: 0.9975
1268/1688 [=====================>........] - ETA: 2s - loss: 0.0082 - accuracy: 0.9975
1278/1688 [=====================>........] - ETA: 2s - loss: 0.0084 - accuracy: 0.9974
1288/1688 [=====================>........] - ETA: 2s - loss: 0.0086 - accuracy: 0.9973
1298/1688 [======================>.......] - ETA: 2s - loss: 0.0085 - accuracy: 0.9973
1308/1688 [======================>.......] - ETA: 2s - loss: 0.0087 - accuracy: 0.9973
1318/1688 [======================>.......] - ETA: 2s - loss: 0.0087 - accuracy: 0.9973
1328/1688 [======================>.......] - ETA: 1s - loss: 0.0086 - accuracy: 0.9973
1338/1688 [======================>.......] - ETA: 1s - loss: 0.0087 - accuracy: 0.9973
1348/1688 [======================>.......] - ETA: 1s - loss: 0.0087 - accuracy: 0.9973
1357/1688 [=======================>......] - ETA: 1s - loss: 0.0087 - accuracy: 0.9973
1367/1688 [=======================>......] - ETA: 1s - loss: 0.0087 - accuracy: 0.9973
1377/1688 [=======================>......] - ETA: 1s - loss: 0.0087 - accuracy: 0.9973
1386/1688 [=======================>......] - ETA: 1s - loss: 0.0087 - accuracy: 0.9972
1395/1688 [=======================>......] - ETA: 1s - loss: 0.0088 - accuracy: 0.9972
1405/1688 [=======================>......] - ETA: 1s - loss: 0.0088 - accuracy: 0.9972
1415/1688 [========================>.....] - ETA: 1s - loss: 0.0087 - accuracy: 0.9973
1425/1688 [========================>.....] - ETA: 1s - loss: 0.0087 - accuracy: 0.9973
1435/1688 [========================>.....] - ETA: 1s - loss: 0.0087 - accuracy: 0.9973
1444/1688 [========================>.....] - ETA: 1s - loss: 0.0088 - accuracy: 0.9972
1453/1688 [========================>.....] - ETA: 1s - loss: 0.0088 - accuracy: 0.9972
1462/1688 [========================>.....] - ETA: 1s - loss: 0.0088 - accuracy: 0.9972
1471/1688 [=========================>....] - ETA: 1s - loss: 0.0088 - accuracy: 0.9972
1481/1688 [=========================>....] - ETA: 1s - loss: 0.0090 - accuracy: 0.9971
1490/1688 [=========================>....] - ETA: 1s - loss: 0.0090 - accuracy: 0.9971
1499/1688 [=========================>....] - ETA: 1s - loss: 0.0090 - accuracy: 0.9971
1508/1688 [=========================>....] - ETA: 0s - loss: 0.0092 - accuracy: 0.9971
1517/1688 [=========================>....] - ETA: 0s - loss: 0.0092 - accuracy: 0.9971
1527/1688 [==========================>...] - ETA: 0s - loss: 0.0093 - accuracy: 0.9970
1537/1688 [==========================>...] - ETA: 0s - loss: 0.0093 - accuracy: 0.9970
1547/1688 [==========================>...] - ETA: 0s - loss: 0.0093 - accuracy: 0.9970
1557/1688 [==========================>...] - ETA: 0s - loss: 0.0092 - accuracy: 0.9970
1567/1688 [==========================>...] - ETA: 0s - loss: 0.0092 - accuracy: 0.9970
1577/1688 [===========================>..] - ETA: 0s - loss: 0.0092 - accuracy: 0.9970
1586/1688 [===========================>..] - ETA: 0s - loss: 0.0092 - accuracy: 0.9970
1595/1688 [===========================>..] - ETA: 0s - loss: 0.0092 - accuracy: 0.9970
1604/1688 [===========================>..] - ETA: 0s - loss: 0.0092 - accuracy: 0.9970
1614/1688 [===========================>..] - ETA: 0s - loss: 0.0092 - accuracy: 0.9970
1624/1688 [===========================>..] - ETA: 0s - loss: 0.0095 - accuracy: 0.9969
1634/1688 [============================>.] - ETA: 0s - loss: 0.0095 - accuracy: 0.9969
1644/1688 [============================>.] - ETA: 0s - loss: 0.0094 - accuracy: 0.9970
1654/1688 [============================>.] - ETA: 0s - loss: 0.0094 - accuracy: 0.9970
1663/1688 [============================>.] - ETA: 0s - loss: 0.0094 - accuracy: 0.9970
1673/1688 [============================>.] - ETA: 0s - loss: 0.0094 - accuracy: 0.9970
1683/1688 [============================>.] - ETA: 0s - loss: 0.0094 - accuracy: 0.9969
1688/1688 [==============================] - 10s 6ms/step - loss: 0.0094 - accuracy: 0.9969 - val_loss: 0.0419 - val_accuracy: 0.9898
Epoch 9/10

   1/1688 [..............................] - ETA: 0s - loss: 3.3739e-04 - accuracy: 1.0000
  11/1688 [..............................] - ETA: 8s - loss: 0.0030 - accuracy: 1.0000    
  21/1688 [..............................] - ETA: 8s - loss: 0.0058 - accuracy: 0.9970
  31/1688 [..............................] - ETA: 8s - loss: 0.0085 - accuracy: 0.9960
  41/1688 [..............................] - ETA: 8s - loss: 0.0067 - accuracy: 0.9970
  51/1688 [..............................] - ETA: 8s - loss: 0.0064 - accuracy: 0.9969
  60/1688 [>.............................] - ETA: 8s - loss: 0.0067 - accuracy: 0.9964
  70/1688 [>.............................] - ETA: 8s - loss: 0.0064 - accuracy: 0.9964
  80/1688 [>.............................] - ETA: 8s - loss: 0.0061 - accuracy: 0.9969
  90/1688 [>.............................] - ETA: 8s - loss: 0.0064 - accuracy: 0.9969
 100/1688 [>.............................] - ETA: 8s - loss: 0.0077 - accuracy: 0.9962
 110/1688 [>.............................] - ETA: 8s - loss: 0.0104 - accuracy: 0.9952
 120/1688 [=>............................] - ETA: 8s - loss: 0.0095 - accuracy: 0.9956
 130/1688 [=>............................] - ETA: 8s - loss: 0.0098 - accuracy: 0.9957
 140/1688 [=>............................] - ETA: 8s - loss: 0.0093 - accuracy: 0.9960
 150/1688 [=>............................] - ETA: 8s - loss: 0.0099 - accuracy: 0.9958
 160/1688 [=>............................] - ETA: 8s - loss: 0.0096 - accuracy: 0.9959
 169/1688 [==>...........................] - ETA: 8s - loss: 0.0092 - accuracy: 0.9961
 179/1688 [==>...........................] - ETA: 8s - loss: 0.0088 - accuracy: 0.9963
 189/1688 [==>...........................] - ETA: 8s - loss: 0.0091 - accuracy: 0.9962
 199/1688 [==>...........................] - ETA: 8s - loss: 0.0088 - accuracy: 0.9964
 209/1688 [==>...........................] - ETA: 8s - loss: 0.0084 - accuracy: 0.9966
 219/1688 [==>...........................] - ETA: 8s - loss: 0.0081 - accuracy: 0.9967
 229/1688 [===>..........................] - ETA: 7s - loss: 0.0079 - accuracy: 0.9969
 239/1688 [===>..........................] - ETA: 7s - loss: 0.0076 - accuracy: 0.9970
 249/1688 [===>..........................] - ETA: 7s - loss: 0.0076 - accuracy: 0.9969
 259/1688 [===>..........................] - ETA: 7s - loss: 0.0079 - accuracy: 0.9969
 269/1688 [===>..........................] - ETA: 7s - loss: 0.0077 - accuracy: 0.9970
 279/1688 [===>..........................] - ETA: 7s - loss: 0.0075 - accuracy: 0.9971
 288/1688 [====>.........................] - ETA: 7s - loss: 0.0073 - accuracy: 0.9972
 298/1688 [====>.........................] - ETA: 7s - loss: 0.0072 - accuracy: 0.9973
 307/1688 [====>.........................] - ETA: 7s - loss: 0.0071 - accuracy: 0.9973
 316/1688 [====>.........................] - ETA: 7s - loss: 0.0070 - accuracy: 0.9973
 326/1688 [====>.........................] - ETA: 7s - loss: 0.0068 - accuracy: 0.9974
 336/1688 [====>.........................] - ETA: 7s - loss: 0.0066 - accuracy: 0.9975
 346/1688 [=====>........................] - ETA: 7s - loss: 0.0066 - accuracy: 0.9975
 355/1688 [=====>........................] - ETA: 7s - loss: 0.0065 - accuracy: 0.9975
 365/1688 [=====>........................] - ETA: 7s - loss: 0.0064 - accuracy: 0.9976
 374/1688 [=====>........................] - ETA: 7s - loss: 0.0063 - accuracy: 0.9977
 384/1688 [=====>........................] - ETA: 7s - loss: 0.0064 - accuracy: 0.9976
 394/1688 [======>.......................] - ETA: 7s - loss: 0.0065 - accuracy: 0.9975
 403/1688 [======>.......................] - ETA: 7s - loss: 0.0064 - accuracy: 0.9976
 412/1688 [======>.......................] - ETA: 7s - loss: 0.0067 - accuracy: 0.9975
 421/1688 [======>.......................] - ETA: 6s - loss: 0.0066 - accuracy: 0.9976
 431/1688 [======>.......................] - ETA: 6s - loss: 0.0065 - accuracy: 0.9976
 441/1688 [======>.......................] - ETA: 6s - loss: 0.0065 - accuracy: 0.9976
 451/1688 [=======>......................] - ETA: 6s - loss: 0.0064 - accuracy: 0.9976
 461/1688 [=======>......................] - ETA: 6s - loss: 0.0063 - accuracy: 0.9976
 471/1688 [=======>......................] - ETA: 6s - loss: 0.0063 - accuracy: 0.9976
 481/1688 [=======>......................] - ETA: 6s - loss: 0.0062 - accuracy: 0.9977
 491/1688 [=======>......................] - ETA: 6s - loss: 0.0062 - accuracy: 0.9976
 501/1688 [=======>......................] - ETA: 6s - loss: 0.0061 - accuracy: 0.9976
 511/1688 [========>.....................] - ETA: 6s - loss: 0.0063 - accuracy: 0.9976
 521/1688 [========>.....................] - ETA: 6s - loss: 0.0062 - accuracy: 0.9976
 531/1688 [========>.....................] - ETA: 6s - loss: 0.0061 - accuracy: 0.9976
 541/1688 [========>.....................] - ETA: 6s - loss: 0.0060 - accuracy: 0.9977
 551/1688 [========>.....................] - ETA: 6s - loss: 0.0060 - accuracy: 0.9977
 561/1688 [========>.....................] - ETA: 6s - loss: 0.0059 - accuracy: 0.9977
 571/1688 [=========>....................] - ETA: 6s - loss: 0.0059 - accuracy: 0.9978
 580/1688 [=========>....................] - ETA: 6s - loss: 0.0058 - accuracy: 0.9978
 590/1688 [=========>....................] - ETA: 6s - loss: 0.0058 - accuracy: 0.9978
 600/1688 [=========>....................] - ETA: 5s - loss: 0.0058 - accuracy: 0.9978
 610/1688 [=========>....................] - ETA: 5s - loss: 0.0057 - accuracy: 0.9978
 620/1688 [==========>...................] - ETA: 5s - loss: 0.0057 - accuracy: 0.9979
 629/1688 [==========>...................] - ETA: 5s - loss: 0.0057 - accuracy: 0.9979
 639/1688 [==========>...................] - ETA: 5s - loss: 0.0056 - accuracy: 0.9979
 649/1688 [==========>...................] - ETA: 5s - loss: 0.0056 - accuracy: 0.9980
 659/1688 [==========>...................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9980
 669/1688 [==========>...................] - ETA: 5s - loss: 0.0057 - accuracy: 0.9979
 679/1688 [===========>..................] - ETA: 5s - loss: 0.0056 - accuracy: 0.9980
 689/1688 [===========>..................] - ETA: 5s - loss: 0.0057 - accuracy: 0.9979
 699/1688 [===========>..................] - ETA: 5s - loss: 0.0057 - accuracy: 0.9979
 709/1688 [===========>..................] - ETA: 5s - loss: 0.0058 - accuracy: 0.9979
 719/1688 [===========>..................] - ETA: 5s - loss: 0.0057 - accuracy: 0.9979
 729/1688 [===========>..................] - ETA: 5s - loss: 0.0057 - accuracy: 0.9979
 739/1688 [============>.................] - ETA: 5s - loss: 0.0057 - accuracy: 0.9979
 748/1688 [============>.................] - ETA: 5s - loss: 0.0060 - accuracy: 0.9978
 758/1688 [============>.................] - ETA: 5s - loss: 0.0059 - accuracy: 0.9979
 768/1688 [============>.................] - ETA: 5s - loss: 0.0065 - accuracy: 0.9978
 778/1688 [============>.................] - ETA: 5s - loss: 0.0065 - accuracy: 0.9978
 788/1688 [=============>................] - ETA: 4s - loss: 0.0065 - accuracy: 0.9978
 798/1688 [=============>................] - ETA: 4s - loss: 0.0065 - accuracy: 0.9978
 808/1688 [=============>................] - ETA: 4s - loss: 0.0066 - accuracy: 0.9978
 817/1688 [=============>................] - ETA: 4s - loss: 0.0066 - accuracy: 0.9977
 827/1688 [=============>................] - ETA: 4s - loss: 0.0065 - accuracy: 0.9978
 837/1688 [=============>................] - ETA: 4s - loss: 0.0068 - accuracy: 0.9977
 846/1688 [==============>...............] - ETA: 4s - loss: 0.0068 - accuracy: 0.9976
 855/1688 [==============>...............] - ETA: 4s - loss: 0.0068 - accuracy: 0.9977
 864/1688 [==============>...............] - ETA: 4s - loss: 0.0067 - accuracy: 0.9977
 873/1688 [==============>...............] - ETA: 4s - loss: 0.0069 - accuracy: 0.9976
 882/1688 [==============>...............] - ETA: 4s - loss: 0.0069 - accuracy: 0.9976
 892/1688 [==============>...............] - ETA: 4s - loss: 0.0069 - accuracy: 0.9976
 902/1688 [===============>..............] - ETA: 4s - loss: 0.0069 - accuracy: 0.9976
 912/1688 [===============>..............] - ETA: 4s - loss: 0.0070 - accuracy: 0.9975
 922/1688 [===============>..............] - ETA: 4s - loss: 0.0071 - accuracy: 0.9975
 932/1688 [===============>..............] - ETA: 4s - loss: 0.0071 - accuracy: 0.9975
 942/1688 [===============>..............] - ETA: 4s - loss: 0.0071 - accuracy: 0.9975
 951/1688 [===============>..............] - ETA: 4s - loss: 0.0072 - accuracy: 0.9975
 961/1688 [================>.............] - ETA: 4s - loss: 0.0074 - accuracy: 0.9974
 971/1688 [================>.............] - ETA: 3s - loss: 0.0073 - accuracy: 0.9975
 981/1688 [================>.............] - ETA: 3s - loss: 0.0073 - accuracy: 0.9975
 991/1688 [================>.............] - ETA: 3s - loss: 0.0074 - accuracy: 0.9974
1001/1688 [================>.............] - ETA: 3s - loss: 0.0074 - accuracy: 0.9974
1010/1688 [================>.............] - ETA: 3s - loss: 0.0074 - accuracy: 0.9974
1020/1688 [=================>............] - ETA: 3s - loss: 0.0074 - accuracy: 0.9974
1030/1688 [=================>............] - ETA: 3s - loss: 0.0074 - accuracy: 0.9974
1040/1688 [=================>............] - ETA: 3s - loss: 0.0074 - accuracy: 0.9974
1050/1688 [=================>............] - ETA: 3s - loss: 0.0074 - accuracy: 0.9974
1060/1688 [=================>............] - ETA: 3s - loss: 0.0073 - accuracy: 0.9974
1070/1688 [==================>...........] - ETA: 3s - loss: 0.0075 - accuracy: 0.9974
1080/1688 [==================>...........] - ETA: 3s - loss: 0.0075 - accuracy: 0.9974
1090/1688 [==================>...........] - ETA: 3s - loss: 0.0077 - accuracy: 0.9973
1100/1688 [==================>...........] - ETA: 3s - loss: 0.0076 - accuracy: 0.9973
1110/1688 [==================>...........] - ETA: 3s - loss: 0.0077 - accuracy: 0.9973
1120/1688 [==================>...........] - ETA: 3s - loss: 0.0077 - accuracy: 0.9973
1130/1688 [===================>..........] - ETA: 3s - loss: 0.0077 - accuracy: 0.9973
1140/1688 [===================>..........] - ETA: 3s - loss: 0.0077 - accuracy: 0.9973
1150/1688 [===================>..........] - ETA: 2s - loss: 0.0079 - accuracy: 0.9973
1160/1688 [===================>..........] - ETA: 2s - loss: 0.0078 - accuracy: 0.9973
1170/1688 [===================>..........] - ETA: 2s - loss: 0.0078 - accuracy: 0.9973
1180/1688 [===================>..........] - ETA: 2s - loss: 0.0078 - accuracy: 0.9973
1190/1688 [====================>.........] - ETA: 2s - loss: 0.0078 - accuracy: 0.9973
1200/1688 [====================>.........] - ETA: 2s - loss: 0.0079 - accuracy: 0.9973
1210/1688 [====================>.........] - ETA: 2s - loss: 0.0078 - accuracy: 0.9973
1220/1688 [====================>.........] - ETA: 2s - loss: 0.0079 - accuracy: 0.9973
1230/1688 [====================>.........] - ETA: 2s - loss: 0.0080 - accuracy: 0.9973
1240/1688 [=====================>........] - ETA: 2s - loss: 0.0080 - accuracy: 0.9973
1250/1688 [=====================>........] - ETA: 2s - loss: 0.0079 - accuracy: 0.9973
1260/1688 [=====================>........] - ETA: 2s - loss: 0.0079 - accuracy: 0.9973
1270/1688 [=====================>........] - ETA: 2s - loss: 0.0079 - accuracy: 0.9973
1280/1688 [=====================>........] - ETA: 2s - loss: 0.0079 - accuracy: 0.9973
1290/1688 [=====================>........] - ETA: 2s - loss: 0.0081 - accuracy: 0.9972
1300/1688 [======================>.......] - ETA: 2s - loss: 0.0081 - accuracy: 0.9972
1309/1688 [======================>.......] - ETA: 2s - loss: 0.0084 - accuracy: 0.9972
1319/1688 [======================>.......] - ETA: 2s - loss: 0.0083 - accuracy: 0.9972
1328/1688 [======================>.......] - ETA: 1s - loss: 0.0083 - accuracy: 0.9972
1338/1688 [======================>.......] - ETA: 1s - loss: 0.0084 - accuracy: 0.9972
1347/1688 [======================>.......] - ETA: 1s - loss: 0.0084 - accuracy: 0.9971
1357/1688 [=======================>......] - ETA: 1s - loss: 0.0083 - accuracy: 0.9971
1367/1688 [=======================>......] - ETA: 1s - loss: 0.0084 - accuracy: 0.9971
1377/1688 [=======================>......] - ETA: 1s - loss: 0.0084 - accuracy: 0.9971
1387/1688 [=======================>......] - ETA: 1s - loss: 0.0084 - accuracy: 0.9971
1397/1688 [=======================>......] - ETA: 1s - loss: 0.0084 - accuracy: 0.9971
1407/1688 [========================>.....] - ETA: 1s - loss: 0.0085 - accuracy: 0.9971
1417/1688 [========================>.....] - ETA: 1s - loss: 0.0084 - accuracy: 0.9971
1426/1688 [========================>.....] - ETA: 1s - loss: 0.0084 - accuracy: 0.9971
1436/1688 [========================>.....] - ETA: 1s - loss: 0.0084 - accuracy: 0.9971
1446/1688 [========================>.....] - ETA: 1s - loss: 0.0084 - accuracy: 0.9971
1456/1688 [========================>.....] - ETA: 1s - loss: 0.0084 - accuracy: 0.9971
1466/1688 [=========================>....] - ETA: 1s - loss: 0.0083 - accuracy: 0.9972
1476/1688 [=========================>....] - ETA: 1s - loss: 0.0083 - accuracy: 0.9972
1486/1688 [=========================>....] - ETA: 1s - loss: 0.0083 - accuracy: 0.9972
1495/1688 [=========================>....] - ETA: 1s - loss: 0.0083 - accuracy: 0.9972
1505/1688 [=========================>....] - ETA: 1s - loss: 0.0083 - accuracy: 0.9972
1515/1688 [=========================>....] - ETA: 0s - loss: 0.0082 - accuracy: 0.9972
1525/1688 [==========================>...] - ETA: 0s - loss: 0.0082 - accuracy: 0.9972
1535/1688 [==========================>...] - ETA: 0s - loss: 0.0082 - accuracy: 0.9972
1545/1688 [==========================>...] - ETA: 0s - loss: 0.0082 - accuracy: 0.9972
1555/1688 [==========================>...] - ETA: 0s - loss: 0.0081 - accuracy: 0.9972
1565/1688 [==========================>...] - ETA: 0s - loss: 0.0081 - accuracy: 0.9972
1575/1688 [==========================>...] - ETA: 0s - loss: 0.0081 - accuracy: 0.9972
1585/1688 [===========================>..] - ETA: 0s - loss: 0.0081 - accuracy: 0.9972
1595/1688 [===========================>..] - ETA: 0s - loss: 0.0082 - accuracy: 0.9972
1604/1688 [===========================>..] - ETA: 0s - loss: 0.0082 - accuracy: 0.9972
1614/1688 [===========================>..] - ETA: 0s - loss: 0.0082 - accuracy: 0.9972
1624/1688 [===========================>..] - ETA: 0s - loss: 0.0082 - accuracy: 0.9972
1634/1688 [============================>.] - ETA: 0s - loss: 0.0081 - accuracy: 0.9972
1644/1688 [============================>.] - ETA: 0s - loss: 0.0081 - accuracy: 0.9972
1653/1688 [============================>.] - ETA: 0s - loss: 0.0081 - accuracy: 0.9972
1663/1688 [============================>.] - ETA: 0s - loss: 0.0082 - accuracy: 0.9972
1672/1688 [============================>.] - ETA: 0s - loss: 0.0082 - accuracy: 0.9972
1681/1688 [============================>.] - ETA: 0s - loss: 0.0082 - accuracy: 0.9972
1688/1688 [==============================] - 10s 6ms/step - loss: 0.0082 - accuracy: 0.9972 - val_loss: 0.0420 - val_accuracy: 0.9915
Epoch 10/10

   1/1688 [..............................] - ETA: 0s - loss: 2.5694e-05 - accuracy: 1.0000
  11/1688 [..............................] - ETA: 8s - loss: 0.0013 - accuracy: 1.0000    
  21/1688 [..............................] - ETA: 8s - loss: 0.0020 - accuracy: 0.9985
  31/1688 [..............................] - ETA: 8s - loss: 0.0020 - accuracy: 0.9990
  41/1688 [..............................] - ETA: 8s - loss: 0.0026 - accuracy: 0.9985
  50/1688 [..............................] - ETA: 8s - loss: 0.0025 - accuracy: 0.9987
  60/1688 [>.............................] - ETA: 8s - loss: 0.0030 - accuracy: 0.9984
  70/1688 [>.............................] - ETA: 8s - loss: 0.0026 - accuracy: 0.9987
  80/1688 [>.............................] - ETA: 8s - loss: 0.0027 - accuracy: 0.9988
  90/1688 [>.............................] - ETA: 8s - loss: 0.0025 - accuracy: 0.9990
 100/1688 [>.............................] - ETA: 8s - loss: 0.0024 - accuracy: 0.9991
 110/1688 [>.............................] - ETA: 8s - loss: 0.0024 - accuracy: 0.9991
 120/1688 [=>............................] - ETA: 8s - loss: 0.0025 - accuracy: 0.9990
 130/1688 [=>............................] - ETA: 8s - loss: 0.0028 - accuracy: 0.9986
 140/1688 [=>............................] - ETA: 8s - loss: 0.0028 - accuracy: 0.9987
 150/1688 [=>............................] - ETA: 8s - loss: 0.0027 - accuracy: 0.9987
 160/1688 [=>............................] - ETA: 8s - loss: 0.0026 - accuracy: 0.9988
 170/1688 [==>...........................] - ETA: 8s - loss: 0.0025 - accuracy: 0.9989
 180/1688 [==>...........................] - ETA: 8s - loss: 0.0026 - accuracy: 0.9990
 190/1688 [==>...........................] - ETA: 8s - loss: 0.0030 - accuracy: 0.9988
 200/1688 [==>...........................] - ETA: 8s - loss: 0.0033 - accuracy: 0.9987
 210/1688 [==>...........................] - ETA: 8s - loss: 0.0034 - accuracy: 0.9987
 220/1688 [==>...........................] - ETA: 8s - loss: 0.0033 - accuracy: 0.9987
 230/1688 [===>..........................] - ETA: 7s - loss: 0.0033 - accuracy: 0.9988
 239/1688 [===>..........................] - ETA: 7s - loss: 0.0033 - accuracy: 0.9987
 249/1688 [===>..........................] - ETA: 7s - loss: 0.0034 - accuracy: 0.9986
 259/1688 [===>..........................] - ETA: 7s - loss: 0.0034 - accuracy: 0.9987
 269/1688 [===>..........................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9986
 279/1688 [===>..........................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9987
 289/1688 [====>.........................] - ETA: 7s - loss: 0.0035 - accuracy: 0.9987
 298/1688 [====>.........................] - ETA: 7s - loss: 0.0035 - accuracy: 0.9987
 308/1688 [====>.........................] - ETA: 7s - loss: 0.0034 - accuracy: 0.9988
 318/1688 [====>.........................] - ETA: 7s - loss: 0.0034 - accuracy: 0.9987
 328/1688 [====>.........................] - ETA: 7s - loss: 0.0034 - accuracy: 0.9988
 338/1688 [=====>........................] - ETA: 7s - loss: 0.0033 - accuracy: 0.9988
 348/1688 [=====>........................] - ETA: 7s - loss: 0.0033 - accuracy: 0.9988
 358/1688 [=====>........................] - ETA: 7s - loss: 0.0034 - accuracy: 0.9988
 368/1688 [=====>........................] - ETA: 7s - loss: 0.0035 - accuracy: 0.9986
 377/1688 [=====>........................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9987
 387/1688 [=====>........................] - ETA: 7s - loss: 0.0035 - accuracy: 0.9987
 397/1688 [======>.......................] - ETA: 7s - loss: 0.0035 - accuracy: 0.9987
 407/1688 [======>.......................] - ETA: 7s - loss: 0.0035 - accuracy: 0.9988
 417/1688 [======>.......................] - ETA: 6s - loss: 0.0034 - accuracy: 0.9988
 427/1688 [======>.......................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9988
 437/1688 [======>.......................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9988
 447/1688 [======>.......................] - ETA: 6s - loss: 0.0034 - accuracy: 0.9988
 457/1688 [=======>......................] - ETA: 6s - loss: 0.0034 - accuracy: 0.9988
 466/1688 [=======>......................] - ETA: 6s - loss: 0.0034 - accuracy: 0.9989
 476/1688 [=======>......................] - ETA: 6s - loss: 0.0033 - accuracy: 0.9989
 486/1688 [=======>......................] - ETA: 6s - loss: 0.0033 - accuracy: 0.9989
 496/1688 [=======>......................] - ETA: 6s - loss: 0.0032 - accuracy: 0.9989
 506/1688 [=======>......................] - ETA: 6s - loss: 0.0032 - accuracy: 0.9990
 516/1688 [========>.....................] - ETA: 6s - loss: 0.0031 - accuracy: 0.9990
 526/1688 [========>.....................] - ETA: 6s - loss: 0.0032 - accuracy: 0.9989
 536/1688 [========>.....................] - ETA: 6s - loss: 0.0032 - accuracy: 0.9989
 546/1688 [========>.....................] - ETA: 6s - loss: 0.0032 - accuracy: 0.9989
 556/1688 [========>.....................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9989
 565/1688 [=========>....................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9988
 574/1688 [=========>....................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9987
 584/1688 [=========>....................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9987
 594/1688 [=========>....................] - ETA: 6s - loss: 0.0041 - accuracy: 0.9985
 604/1688 [=========>....................] - ETA: 5s - loss: 0.0042 - accuracy: 0.9984
 614/1688 [=========>....................] - ETA: 5s - loss: 0.0042 - accuracy: 0.9984
 624/1688 [==========>...................] - ETA: 5s - loss: 0.0041 - accuracy: 0.9984
 634/1688 [==========>...................] - ETA: 5s - loss: 0.0041 - accuracy: 0.9985
 644/1688 [==========>...................] - ETA: 5s - loss: 0.0041 - accuracy: 0.9985
 654/1688 [==========>...................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9985
 663/1688 [==========>...................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9985
 673/1688 [==========>...................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9986
 683/1688 [===========>..................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9986
 693/1688 [===========>..................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9986
 703/1688 [===========>..................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9985
 713/1688 [===========>..................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9985
 723/1688 [===========>..................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9985
 733/1688 [============>.................] - ETA: 5s - loss: 0.0041 - accuracy: 0.9985
 743/1688 [============>.................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9985
 753/1688 [============>.................] - ETA: 5s - loss: 0.0041 - accuracy: 0.9985
 763/1688 [============>.................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9985
 773/1688 [============>.................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9985
 783/1688 [============>.................] - ETA: 4s - loss: 0.0042 - accuracy: 0.9984
 793/1688 [=============>................] - ETA: 4s - loss: 0.0044 - accuracy: 0.9983
 803/1688 [=============>................] - ETA: 4s - loss: 0.0044 - accuracy: 0.9984
 813/1688 [=============>................] - ETA: 4s - loss: 0.0045 - accuracy: 0.9983
 823/1688 [=============>................] - ETA: 4s - loss: 0.0046 - accuracy: 0.9982
 833/1688 [=============>................] - ETA: 4s - loss: 0.0048 - accuracy: 0.9982
 842/1688 [=============>................] - ETA: 4s - loss: 0.0050 - accuracy: 0.9981
 851/1688 [==============>...............] - ETA: 4s - loss: 0.0050 - accuracy: 0.9981
 860/1688 [==============>...............] - ETA: 4s - loss: 0.0050 - accuracy: 0.9981
 870/1688 [==============>...............] - ETA: 4s - loss: 0.0050 - accuracy: 0.9981
 880/1688 [==============>...............] - ETA: 4s - loss: 0.0052 - accuracy: 0.9979
 890/1688 [==============>...............] - ETA: 4s - loss: 0.0053 - accuracy: 0.9979
 900/1688 [==============>...............] - ETA: 4s - loss: 0.0053 - accuracy: 0.9979
 909/1688 [===============>..............] - ETA: 4s - loss: 0.0053 - accuracy: 0.9979
 919/1688 [===============>..............] - ETA: 4s - loss: 0.0054 - accuracy: 0.9979
 929/1688 [===============>..............] - ETA: 4s - loss: 0.0053 - accuracy: 0.9979
 939/1688 [===============>..............] - ETA: 4s - loss: 0.0054 - accuracy: 0.9979
 949/1688 [===============>..............] - ETA: 4s - loss: 0.0053 - accuracy: 0.9979
 959/1688 [================>.............] - ETA: 4s - loss: 0.0055 - accuracy: 0.9978
 969/1688 [================>.............] - ETA: 3s - loss: 0.0056 - accuracy: 0.9978
 979/1688 [================>.............] - ETA: 3s - loss: 0.0058 - accuracy: 0.9977
 989/1688 [================>.............] - ETA: 3s - loss: 0.0058 - accuracy: 0.9977
 999/1688 [================>.............] - ETA: 3s - loss: 0.0057 - accuracy: 0.9977
1009/1688 [================>.............] - ETA: 3s - loss: 0.0057 - accuracy: 0.9977
1019/1688 [=================>............] - ETA: 3s - loss: 0.0056 - accuracy: 0.9978
1028/1688 [=================>............] - ETA: 3s - loss: 0.0057 - accuracy: 0.9978
1038/1688 [=================>............] - ETA: 3s - loss: 0.0057 - accuracy: 0.9978
1047/1688 [=================>............] - ETA: 3s - loss: 0.0058 - accuracy: 0.9977
1057/1688 [=================>............] - ETA: 3s - loss: 0.0058 - accuracy: 0.9977
1067/1688 [=================>............] - ETA: 3s - loss: 0.0058 - accuracy: 0.9977
1077/1688 [==================>...........] - ETA: 3s - loss: 0.0058 - accuracy: 0.9977
1087/1688 [==================>...........] - ETA: 3s - loss: 0.0059 - accuracy: 0.9977
1097/1688 [==================>...........] - ETA: 3s - loss: 0.0059 - accuracy: 0.9977
1107/1688 [==================>...........] - ETA: 3s - loss: 0.0059 - accuracy: 0.9978
1117/1688 [==================>...........] - ETA: 3s - loss: 0.0059 - accuracy: 0.9978
1127/1688 [===================>..........] - ETA: 3s - loss: 0.0060 - accuracy: 0.9977
1137/1688 [===================>..........] - ETA: 3s - loss: 0.0059 - accuracy: 0.9977
1147/1688 [===================>..........] - ETA: 2s - loss: 0.0059 - accuracy: 0.9978
1157/1688 [===================>..........] - ETA: 2s - loss: 0.0060 - accuracy: 0.9977
1166/1688 [===================>..........] - ETA: 2s - loss: 0.0060 - accuracy: 0.9977
1176/1688 [===================>..........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9977
1186/1688 [====================>.........] - ETA: 2s - loss: 0.0060 - accuracy: 0.9977
1195/1688 [====================>.........] - ETA: 2s - loss: 0.0060 - accuracy: 0.9977
1205/1688 [====================>.........] - ETA: 2s - loss: 0.0060 - accuracy: 0.9977
1215/1688 [====================>.........] - ETA: 2s - loss: 0.0062 - accuracy: 0.9977
1225/1688 [====================>.........] - ETA: 2s - loss: 0.0062 - accuracy: 0.9977
1235/1688 [====================>.........] - ETA: 2s - loss: 0.0062 - accuracy: 0.9976
1245/1688 [=====================>........] - ETA: 2s - loss: 0.0064 - accuracy: 0.9976
1255/1688 [=====================>........] - ETA: 2s - loss: 0.0065 - accuracy: 0.9976
1265/1688 [=====================>........] - ETA: 2s - loss: 0.0065 - accuracy: 0.9976
1275/1688 [=====================>........] - ETA: 2s - loss: 0.0067 - accuracy: 0.9975
1285/1688 [=====================>........] - ETA: 2s - loss: 0.0067 - accuracy: 0.9975
1295/1688 [======================>.......] - ETA: 2s - loss: 0.0067 - accuracy: 0.9975
1305/1688 [======================>.......] - ETA: 2s - loss: 0.0067 - accuracy: 0.9975
1315/1688 [======================>.......] - ETA: 2s - loss: 0.0067 - accuracy: 0.9975
1325/1688 [======================>.......] - ETA: 1s - loss: 0.0067 - accuracy: 0.9975
1335/1688 [======================>.......] - ETA: 1s - loss: 0.0067 - accuracy: 0.9975
1345/1688 [======================>.......] - ETA: 1s - loss: 0.0067 - accuracy: 0.9976
1354/1688 [=======================>......] - ETA: 1s - loss: 0.0067 - accuracy: 0.9976
1364/1688 [=======================>......] - ETA: 1s - loss: 0.0066 - accuracy: 0.9976
1374/1688 [=======================>......] - ETA: 1s - loss: 0.0066 - accuracy: 0.9976
1384/1688 [=======================>......] - ETA: 1s - loss: 0.0066 - accuracy: 0.9976
1393/1688 [=======================>......] - ETA: 1s - loss: 0.0066 - accuracy: 0.9976
1402/1688 [=======================>......] - ETA: 1s - loss: 0.0066 - accuracy: 0.9976
1412/1688 [========================>.....] - ETA: 1s - loss: 0.0066 - accuracy: 0.9976
1422/1688 [========================>.....] - ETA: 1s - loss: 0.0065 - accuracy: 0.9976
1432/1688 [========================>.....] - ETA: 1s - loss: 0.0065 - accuracy: 0.9976
1442/1688 [========================>.....] - ETA: 1s - loss: 0.0065 - accuracy: 0.9976
1452/1688 [========================>.....] - ETA: 1s - loss: 0.0066 - accuracy: 0.9976
1462/1688 [========================>.....] - ETA: 1s - loss: 0.0065 - accuracy: 0.9976
1472/1688 [=========================>....] - ETA: 1s - loss: 0.0066 - accuracy: 0.9976
1482/1688 [=========================>....] - ETA: 1s - loss: 0.0066 - accuracy: 0.9977
1492/1688 [=========================>....] - ETA: 1s - loss: 0.0066 - accuracy: 0.9977
1502/1688 [=========================>....] - ETA: 1s - loss: 0.0065 - accuracy: 0.9977
1512/1688 [=========================>....] - ETA: 0s - loss: 0.0065 - accuracy: 0.9977
1522/1688 [==========================>...] - ETA: 0s - loss: 0.0065 - accuracy: 0.9977
1532/1688 [==========================>...] - ETA: 0s - loss: 0.0064 - accuracy: 0.9977
1542/1688 [==========================>...] - ETA: 0s - loss: 0.0065 - accuracy: 0.9977
1552/1688 [==========================>...] - ETA: 0s - loss: 0.0065 - accuracy: 0.9977
1562/1688 [==========================>...] - ETA: 0s - loss: 0.0065 - accuracy: 0.9977
1572/1688 [==========================>...] - ETA: 0s - loss: 0.0066 - accuracy: 0.9977
1582/1688 [===========================>..] - ETA: 0s - loss: 0.0065 - accuracy: 0.9977
1592/1688 [===========================>..] - ETA: 0s - loss: 0.0065 - accuracy: 0.9977
1602/1688 [===========================>..] - ETA: 0s - loss: 0.0065 - accuracy: 0.9977
1611/1688 [===========================>..] - ETA: 0s - loss: 0.0065 - accuracy: 0.9977
1621/1688 [===========================>..] - ETA: 0s - loss: 0.0065 - accuracy: 0.9977
1631/1688 [===========================>..] - ETA: 0s - loss: 0.0065 - accuracy: 0.9977
1641/1688 [============================>.] - ETA: 0s - loss: 0.0065 - accuracy: 0.9977
1651/1688 [============================>.] - ETA: 0s - loss: 0.0065 - accuracy: 0.9977
1660/1688 [============================>.] - ETA: 0s - loss: 0.0065 - accuracy: 0.9977
1670/1688 [============================>.] - ETA: 0s - loss: 0.0065 - accuracy: 0.9977
1680/1688 [============================>.] - ETA: 0s - loss: 0.0064 - accuracy: 0.9977
1688/1688 [==============================] - 10s 6ms/step - loss: 0.0064 - accuracy: 0.9977 - val_loss: 0.0371 - val_accuracy: 0.9918
Test score: 0.034738846123218536
Test accuracy: 0.9921000003814697

4. Model quantization

We can now turn to quantization to get a discretized version of the model, where the weights and activations are quantized so as to be suitable for implementation in the Akida NSoC.

For this, we just have to quantize the Keras model using the quantize function. Here, we decide to quantize to the maximum allowed bitwidths for the first layer weights (8-bit), the subsequent layer weights (4-bit) and the activations (4-bit).

The quantized model is a Keras model where the neural layers (Conv2D, Dense) and the ReLU layers are replaced with custom CNN2SNN quantized layers (QuantizedConv2D, QuantizedDense, QuantizedReLU). All Keras API functions can be applied on this new model: summary(), compile(), fit(). etc.

Note

The quantize function folds the batch normalization layers into the corresponding neural layer. The new weights are computed according to this folding operation.

Note

The CNN2SNN toolkit provides the check_model_compatibility function to ensure that the quantized model is compatible with the Akida NSoC. If the model is not fully compatible, substitutes will be needed for the relevant layers/operations (guidelines included in the documentation).

from cnn2snn import quantize, check_model_compatibility

model_quantized = quantize(model_keras,
                           input_weight_quantization=8,
                           weight_quantization=4,
                           activ_quantization=4)
model_quantized.summary()

print("Model compatible for Akida conversion:",
      check_model_compatibility(model_quantized, input_is_sparse=False))

Out:

Model: "model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
input_3 (InputLayer)         [(None, 28, 28, 1)]       0
_________________________________________________________________
conv2d (QuantizedConv2D)     (None, 26, 26, 32)        320
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 13, 13, 32)        0
_________________________________________________________________
activation_discrete_relu (Ac (None, 13, 13, 32)        0
_________________________________________________________________
conv2d_1 (QuantizedConv2D)   (None, 13, 13, 64)        18496
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 7, 7, 64)          0
_________________________________________________________________
activation_discrete_relu_1 ( (None, 7, 7, 64)          0
_________________________________________________________________
flatten (Flatten)            (None, 3136)              0
_________________________________________________________________
dense (QuantizedDense)       (None, 10)                31370
=================================================================
Total params: 50,186
Trainable params: 50,186
Non-trainable params: 0
_________________________________________________________________
Model compatible for Akida conversion: True

Check the quantized model accuracy.

model_quantized.compile(
    loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
    optimizer='adam',
    metrics=['accuracy'])

score = model_quantized.evaluate(x_test, y_test, verbose=0)
print('Test accuracy after 8-4-4 quantization:', score[1])

Out:

Test accuracy after 8-4-4 quantization: 0.9914000034332275

Since we used the maximum allowed bitwidths for weights and activations, the accuracy of the quantized model is equivalent to the one of the base model, but for lower bitwidth, the quantization usually introduces a performance drop.

Let’s try this time with 2-bit for weights and 1-bit for activations.

model_quantized = quantize(model_keras,
                           weight_quantization=2,
                           activ_quantization=1)

model_quantized.compile(
    loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
    optimizer='adam',
    metrics=['accuracy'])

score = model_quantized.evaluate(x_test, y_test, verbose=0)
print('Test accuracy after 2-2-1 quantization:', score[1])

# To recover the original model accuracy, a quantization-aware training phase
# is required.

Out:

Test accuracy after 2-2-1 quantization: 0.28029999136924744

5. Model fine tuning (quantization-aware training)

This quantization-aware training (fine tuning) allows to cover the performance drop due to the quantization step.

Note that since this step is a fine tuning, the number of epochs can be lowered, compared to the training from scratch of the standard model.

model_quantized.fit(x_train, y_train, epochs=5, validation_split=0.1)

score = model_quantized.evaluate(x_test, y_test, verbose=0)
print('Test accuracy after fine tuning:', score[1])

Out:

Epoch 1/5

   1/1688 [..............................] - ETA: 0s - loss: 1.8570 - accuracy: 0.3750
  10/1688 [..............................] - ETA: 9s - loss: 1.5440 - accuracy: 0.4812
  19/1688 [..............................] - ETA: 9s - loss: 1.2816 - accuracy: 0.5789
  28/1688 [..............................] - ETA: 9s - loss: 1.0478 - accuracy: 0.6618
  37/1688 [..............................] - ETA: 9s - loss: 0.8832 - accuracy: 0.7188
  46/1688 [..............................] - ETA: 9s - loss: 0.7651 - accuracy: 0.7582
  55/1688 [..............................] - ETA: 9s - loss: 0.6858 - accuracy: 0.7812
  64/1688 [>.............................] - ETA: 9s - loss: 0.6146 - accuracy: 0.8047
  73/1688 [>.............................] - ETA: 9s - loss: 0.5613 - accuracy: 0.8228
  82/1688 [>.............................] - ETA: 9s - loss: 0.5225 - accuracy: 0.8338
  91/1688 [>.............................] - ETA: 9s - loss: 0.4881 - accuracy: 0.8458
 100/1688 [>.............................] - ETA: 9s - loss: 0.4637 - accuracy: 0.8528
 109/1688 [>.............................] - ETA: 9s - loss: 0.4415 - accuracy: 0.8598
 118/1688 [=>............................] - ETA: 9s - loss: 0.4187 - accuracy: 0.8668
 127/1688 [=>............................] - ETA: 9s - loss: 0.3941 - accuracy: 0.8752
 136/1688 [=>............................] - ETA: 9s - loss: 0.3748 - accuracy: 0.8810
 145/1688 [=>............................] - ETA: 8s - loss: 0.3608 - accuracy: 0.8858
 154/1688 [=>............................] - ETA: 8s - loss: 0.3474 - accuracy: 0.8898
 163/1688 [=>............................] - ETA: 8s - loss: 0.3352 - accuracy: 0.8936
 172/1688 [==>...........................] - ETA: 8s - loss: 0.3234 - accuracy: 0.8972
 181/1688 [==>...........................] - ETA: 8s - loss: 0.3132 - accuracy: 0.9006
 190/1688 [==>...........................] - ETA: 8s - loss: 0.3043 - accuracy: 0.9030
 199/1688 [==>...........................] - ETA: 8s - loss: 0.2993 - accuracy: 0.9045
 208/1688 [==>...........................] - ETA: 8s - loss: 0.2921 - accuracy: 0.9072
 217/1688 [==>...........................] - ETA: 8s - loss: 0.2847 - accuracy: 0.9096
 226/1688 [===>..........................] - ETA: 8s - loss: 0.2779 - accuracy: 0.9114
 235/1688 [===>..........................] - ETA: 8s - loss: 0.2704 - accuracy: 0.9138
 244/1688 [===>..........................] - ETA: 8s - loss: 0.2658 - accuracy: 0.9152
 253/1688 [===>..........................] - ETA: 8s - loss: 0.2612 - accuracy: 0.9172
 262/1688 [===>..........................] - ETA: 8s - loss: 0.2538 - accuracy: 0.9196
 271/1688 [===>..........................] - ETA: 8s - loss: 0.2474 - accuracy: 0.9215
 280/1688 [===>..........................] - ETA: 8s - loss: 0.2419 - accuracy: 0.9231
 289/1688 [====>.........................] - ETA: 8s - loss: 0.2369 - accuracy: 0.9245
 298/1688 [====>.........................] - ETA: 8s - loss: 0.2333 - accuracy: 0.9258
 307/1688 [====>.........................] - ETA: 8s - loss: 0.2292 - accuracy: 0.9272
 316/1688 [====>.........................] - ETA: 8s - loss: 0.2251 - accuracy: 0.9285
 325/1688 [====>.........................] - ETA: 7s - loss: 0.2212 - accuracy: 0.9294
 334/1688 [====>.........................] - ETA: 7s - loss: 0.2193 - accuracy: 0.9301
 343/1688 [=====>........................] - ETA: 7s - loss: 0.2167 - accuracy: 0.9311
 352/1688 [=====>........................] - ETA: 7s - loss: 0.2143 - accuracy: 0.9317
 361/1688 [=====>........................] - ETA: 7s - loss: 0.2109 - accuracy: 0.9328
 370/1688 [=====>........................] - ETA: 7s - loss: 0.2080 - accuracy: 0.9335
 379/1688 [=====>........................] - ETA: 7s - loss: 0.2063 - accuracy: 0.9341
 388/1688 [=====>........................] - ETA: 7s - loss: 0.2035 - accuracy: 0.9349
 397/1688 [======>.......................] - ETA: 7s - loss: 0.2004 - accuracy: 0.9358
 406/1688 [======>.......................] - ETA: 7s - loss: 0.1978 - accuracy: 0.9366
 415/1688 [======>.......................] - ETA: 7s - loss: 0.1954 - accuracy: 0.9373
 424/1688 [======>.......................] - ETA: 7s - loss: 0.1924 - accuracy: 0.9382
 433/1688 [======>.......................] - ETA: 7s - loss: 0.1894 - accuracy: 0.9392
 442/1688 [======>.......................] - ETA: 7s - loss: 0.1874 - accuracy: 0.9400
 451/1688 [=======>......................] - ETA: 7s - loss: 0.1853 - accuracy: 0.9408
 460/1688 [=======>......................] - ETA: 7s - loss: 0.1832 - accuracy: 0.9414
 469/1688 [=======>......................] - ETA: 7s - loss: 0.1815 - accuracy: 0.9421
 478/1688 [=======>......................] - ETA: 7s - loss: 0.1796 - accuracy: 0.9428
 487/1688 [=======>......................] - ETA: 7s - loss: 0.1776 - accuracy: 0.9433
 496/1688 [=======>......................] - ETA: 6s - loss: 0.1756 - accuracy: 0.9440
 505/1688 [=======>......................] - ETA: 6s - loss: 0.1739 - accuracy: 0.9445
 514/1688 [========>.....................] - ETA: 6s - loss: 0.1718 - accuracy: 0.9453
 523/1688 [========>.....................] - ETA: 6s - loss: 0.1706 - accuracy: 0.9457
 532/1688 [========>.....................] - ETA: 6s - loss: 0.1697 - accuracy: 0.9462
 541/1688 [========>.....................] - ETA: 6s - loss: 0.1683 - accuracy: 0.9466
 550/1688 [========>.....................] - ETA: 6s - loss: 0.1669 - accuracy: 0.9471
 559/1688 [========>.....................] - ETA: 6s - loss: 0.1653 - accuracy: 0.9476
 568/1688 [=========>....................] - ETA: 6s - loss: 0.1640 - accuracy: 0.9479
 577/1688 [=========>....................] - ETA: 6s - loss: 0.1627 - accuracy: 0.9484
 586/1688 [=========>....................] - ETA: 6s - loss: 0.1605 - accuracy: 0.9491
 595/1688 [=========>....................] - ETA: 6s - loss: 0.1586 - accuracy: 0.9496
 604/1688 [=========>....................] - ETA: 6s - loss: 0.1569 - accuracy: 0.9502
 613/1688 [=========>....................] - ETA: 6s - loss: 0.1550 - accuracy: 0.9508
 622/1688 [==========>...................] - ETA: 6s - loss: 0.1533 - accuracy: 0.9514
 631/1688 [==========>...................] - ETA: 6s - loss: 0.1519 - accuracy: 0.9518
 640/1688 [==========>...................] - ETA: 6s - loss: 0.1500 - accuracy: 0.9524
 649/1688 [==========>...................] - ETA: 6s - loss: 0.1489 - accuracy: 0.9528
 658/1688 [==========>...................] - ETA: 6s - loss: 0.1481 - accuracy: 0.9532
 667/1688 [==========>...................] - ETA: 5s - loss: 0.1474 - accuracy: 0.9534
 676/1688 [===========>..................] - ETA: 5s - loss: 0.1469 - accuracy: 0.9537
 685/1688 [===========>..................] - ETA: 5s - loss: 0.1464 - accuracy: 0.9537
 694/1688 [===========>..................] - ETA: 5s - loss: 0.1452 - accuracy: 0.9541
 703/1688 [===========>..................] - ETA: 5s - loss: 0.1442 - accuracy: 0.9544
 712/1688 [===========>..................] - ETA: 5s - loss: 0.1429 - accuracy: 0.9548
 721/1688 [===========>..................] - ETA: 5s - loss: 0.1418 - accuracy: 0.9551
 730/1688 [===========>..................] - ETA: 5s - loss: 0.1411 - accuracy: 0.9552
 739/1688 [============>.................] - ETA: 5s - loss: 0.1401 - accuracy: 0.9556
 748/1688 [============>.................] - ETA: 5s - loss: 0.1390 - accuracy: 0.9558
 757/1688 [============>.................] - ETA: 5s - loss: 0.1381 - accuracy: 0.9560
 766/1688 [============>.................] - ETA: 5s - loss: 0.1370 - accuracy: 0.9564
 775/1688 [============>.................] - ETA: 5s - loss: 0.1362 - accuracy: 0.9565
 784/1688 [============>.................] - ETA: 5s - loss: 0.1352 - accuracy: 0.9569
 793/1688 [=============>................] - ETA: 5s - loss: 0.1348 - accuracy: 0.9570
 802/1688 [=============>................] - ETA: 5s - loss: 0.1346 - accuracy: 0.9571
 811/1688 [=============>................] - ETA: 5s - loss: 0.1338 - accuracy: 0.9574
 820/1688 [=============>................] - ETA: 5s - loss: 0.1330 - accuracy: 0.9577
 829/1688 [=============>................] - ETA: 5s - loss: 0.1319 - accuracy: 0.9580
 838/1688 [=============>................] - ETA: 4s - loss: 0.1312 - accuracy: 0.9582
 847/1688 [==============>...............] - ETA: 4s - loss: 0.1306 - accuracy: 0.9584
 856/1688 [==============>...............] - ETA: 4s - loss: 0.1305 - accuracy: 0.9586
 865/1688 [==============>...............] - ETA: 4s - loss: 0.1299 - accuracy: 0.9588
 874/1688 [==============>...............] - ETA: 4s - loss: 0.1292 - accuracy: 0.9590
 883/1688 [==============>...............] - ETA: 4s - loss: 0.1285 - accuracy: 0.9592
 892/1688 [==============>...............] - ETA: 4s - loss: 0.1276 - accuracy: 0.9595
 901/1688 [===============>..............] - ETA: 4s - loss: 0.1269 - accuracy: 0.9597
 910/1688 [===============>..............] - ETA: 4s - loss: 0.1268 - accuracy: 0.9599
 919/1688 [===============>..............] - ETA: 4s - loss: 0.1262 - accuracy: 0.9599
 928/1688 [===============>..............] - ETA: 4s - loss: 0.1255 - accuracy: 0.9602
 937/1688 [===============>..............] - ETA: 4s - loss: 0.1252 - accuracy: 0.9603
 946/1688 [===============>..............] - ETA: 4s - loss: 0.1245 - accuracy: 0.9606
 955/1688 [===============>..............] - ETA: 4s - loss: 0.1242 - accuracy: 0.9607
 964/1688 [================>.............] - ETA: 4s - loss: 0.1235 - accuracy: 0.9609
 973/1688 [================>.............] - ETA: 4s - loss: 0.1231 - accuracy: 0.9610
 982/1688 [================>.............] - ETA: 4s - loss: 0.1228 - accuracy: 0.9611
 991/1688 [================>.............] - ETA: 4s - loss: 0.1222 - accuracy: 0.9613
1000/1688 [================>.............] - ETA: 4s - loss: 0.1216 - accuracy: 0.9614
1009/1688 [================>.............] - ETA: 3s - loss: 0.1215 - accuracy: 0.9615
1018/1688 [=================>............] - ETA: 3s - loss: 0.1216 - accuracy: 0.9616
1027/1688 [=================>............] - ETA: 3s - loss: 0.1209 - accuracy: 0.9618
1036/1688 [=================>............] - ETA: 3s - loss: 0.1203 - accuracy: 0.9620
1045/1688 [=================>............] - ETA: 3s - loss: 0.1197 - accuracy: 0.9622
1054/1688 [=================>............] - ETA: 3s - loss: 0.1191 - accuracy: 0.9623
1063/1688 [=================>............] - ETA: 3s - loss: 0.1184 - accuracy: 0.9626
1072/1688 [==================>...........] - ETA: 3s - loss: 0.1183 - accuracy: 0.9626
1081/1688 [==================>...........] - ETA: 3s - loss: 0.1181 - accuracy: 0.9627
1090/1688 [==================>...........] - ETA: 3s - loss: 0.1179 - accuracy: 0.9628
1099/1688 [==================>...........] - ETA: 3s - loss: 0.1172 - accuracy: 0.9630
1108/1688 [==================>...........] - ETA: 3s - loss: 0.1168 - accuracy: 0.9631
1117/1688 [==================>...........] - ETA: 3s - loss: 0.1163 - accuracy: 0.9633
1126/1688 [===================>..........] - ETA: 3s - loss: 0.1158 - accuracy: 0.9634
1135/1688 [===================>..........] - ETA: 3s - loss: 0.1155 - accuracy: 0.9635
1144/1688 [===================>..........] - ETA: 3s - loss: 0.1153 - accuracy: 0.9636
1153/1688 [===================>..........] - ETA: 3s - loss: 0.1148 - accuracy: 0.9638
1162/1688 [===================>..........] - ETA: 3s - loss: 0.1143 - accuracy: 0.9639
1171/1688 [===================>..........] - ETA: 3s - loss: 0.1137 - accuracy: 0.9641
1180/1688 [===================>..........] - ETA: 2s - loss: 0.1133 - accuracy: 0.9642
1189/1688 [====================>.........] - ETA: 2s - loss: 0.1127 - accuracy: 0.9644
1198/1688 [====================>.........] - ETA: 2s - loss: 0.1122 - accuracy: 0.9646
1207/1688 [====================>.........] - ETA: 2s - loss: 0.1119 - accuracy: 0.9646
1216/1688 [====================>.........] - ETA: 2s - loss: 0.1113 - accuracy: 0.9648
1225/1688 [====================>.........] - ETA: 2s - loss: 0.1107 - accuracy: 0.9650
1234/1688 [====================>.........] - ETA: 2s - loss: 0.1105 - accuracy: 0.9652
1243/1688 [=====================>........] - ETA: 2s - loss: 0.1099 - accuracy: 0.9654
1252/1688 [=====================>........] - ETA: 2s - loss: 0.1096 - accuracy: 0.9655
1261/1688 [=====================>........] - ETA: 2s - loss: 0.1090 - accuracy: 0.9657
1270/1688 [=====================>........] - ETA: 2s - loss: 0.1086 - accuracy: 0.9658
1279/1688 [=====================>........] - ETA: 2s - loss: 0.1083 - accuracy: 0.9659
1288/1688 [=====================>........] - ETA: 2s - loss: 0.1077 - accuracy: 0.9661
1297/1688 [======================>.......] - ETA: 2s - loss: 0.1072 - accuracy: 0.9662
1306/1688 [======================>.......] - ETA: 2s - loss: 0.1069 - accuracy: 0.9664
1315/1688 [======================>.......] - ETA: 2s - loss: 0.1063 - accuracy: 0.9665
1324/1688 [======================>.......] - ETA: 2s - loss: 0.1058 - accuracy: 0.9667
1333/1688 [======================>.......] - ETA: 2s - loss: 0.1056 - accuracy: 0.9668
1342/1688 [======================>.......] - ETA: 2s - loss: 0.1054 - accuracy: 0.9668
1351/1688 [=======================>......] - ETA: 1s - loss: 0.1052 - accuracy: 0.9669
1360/1688 [=======================>......] - ETA: 1s - loss: 0.1050 - accuracy: 0.9670
1369/1688 [=======================>......] - ETA: 1s - loss: 0.1046 - accuracy: 0.9670
1378/1688 [=======================>......] - ETA: 1s - loss: 0.1044 - accuracy: 0.9671
1387/1688 [=======================>......] - ETA: 1s - loss: 0.1042 - accuracy: 0.9672
1396/1688 [=======================>......] - ETA: 1s - loss: 0.1040 - accuracy: 0.9673
1405/1688 [=======================>......] - ETA: 1s - loss: 0.1038 - accuracy: 0.9673
1414/1688 [========================>.....] - ETA: 1s - loss: 0.1033 - accuracy: 0.9674
1423/1688 [========================>.....] - ETA: 1s - loss: 0.1029 - accuracy: 0.9675
1432/1688 [========================>.....] - ETA: 1s - loss: 0.1024 - accuracy: 0.9677
1441/1688 [========================>.....] - ETA: 1s - loss: 0.1021 - accuracy: 0.9678
1450/1688 [========================>.....] - ETA: 1s - loss: 0.1019 - accuracy: 0.9679
1459/1688 [========================>.....] - ETA: 1s - loss: 0.1016 - accuracy: 0.9680
1468/1688 [=========================>....] - ETA: 1s - loss: 0.1013 - accuracy: 0.9681
1477/1688 [=========================>....] - ETA: 1s - loss: 0.1008 - accuracy: 0.9682
1486/1688 [=========================>....] - ETA: 1s - loss: 0.1007 - accuracy: 0.9683
1495/1688 [=========================>....] - ETA: 1s - loss: 0.1005 - accuracy: 0.9684
1504/1688 [=========================>....] - ETA: 1s - loss: 0.1004 - accuracy: 0.9685
1513/1688 [=========================>....] - ETA: 1s - loss: 0.1001 - accuracy: 0.9685
1522/1688 [==========================>...] - ETA: 0s - loss: 0.0997 - accuracy: 0.9686
1531/1688 [==========================>...] - ETA: 0s - loss: 0.0994 - accuracy: 0.9687
1540/1688 [==========================>...] - ETA: 0s - loss: 0.0991 - accuracy: 0.9688
1549/1688 [==========================>...] - ETA: 0s - loss: 0.0987 - accuracy: 0.9689
1558/1688 [==========================>...] - ETA: 0s - loss: 0.0985 - accuracy: 0.9690
1567/1688 [==========================>...] - ETA: 0s - loss: 0.0983 - accuracy: 0.9690
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0979 - accuracy: 0.9691
1585/1688 [===========================>..] - ETA: 0s - loss: 0.0978 - accuracy: 0.9692
1594/1688 [===========================>..] - ETA: 0s - loss: 0.0974 - accuracy: 0.9693
1603/1688 [===========================>..] - ETA: 0s - loss: 0.0972 - accuracy: 0.9693
1612/1688 [===========================>..] - ETA: 0s - loss: 0.0968 - accuracy: 0.9695
1621/1688 [===========================>..] - ETA: 0s - loss: 0.0968 - accuracy: 0.9695
1630/1688 [===========================>..] - ETA: 0s - loss: 0.0965 - accuracy: 0.9696
1639/1688 [============================>.] - ETA: 0s - loss: 0.0961 - accuracy: 0.9697
1648/1688 [============================>.] - ETA: 0s - loss: 0.0959 - accuracy: 0.9698
1657/1688 [============================>.] - ETA: 0s - loss: 0.0957 - accuracy: 0.9699
1666/1688 [============================>.] - ETA: 0s - loss: 0.0957 - accuracy: 0.9700
1675/1688 [============================>.] - ETA: 0s - loss: 0.0956 - accuracy: 0.9700
1684/1688 [============================>.] - ETA: 0s - loss: 0.0953 - accuracy: 0.9701
1688/1688 [==============================] - 10s 6ms/step - loss: 0.0952 - accuracy: 0.9701 - val_loss: 0.0555 - val_accuracy: 0.9828
Epoch 2/5

   1/1688 [..............................] - ETA: 0s - loss: 0.0015 - accuracy: 1.0000
  10/1688 [..............................] - ETA: 8s - loss: 0.0361 - accuracy: 0.9875
  19/1688 [..............................] - ETA: 9s - loss: 0.0334 - accuracy: 0.9918
  28/1688 [..............................] - ETA: 9s - loss: 0.0272 - accuracy: 0.9922
  37/1688 [..............................] - ETA: 9s - loss: 0.0414 - accuracy: 0.9890
  46/1688 [..............................] - ETA: 9s - loss: 0.0396 - accuracy: 0.9898
  55/1688 [..............................] - ETA: 9s - loss: 0.0403 - accuracy: 0.9886
  64/1688 [>.............................] - ETA: 9s - loss: 0.0415 - accuracy: 0.9873
  73/1688 [>.............................] - ETA: 9s - loss: 0.0399 - accuracy: 0.9880
  82/1688 [>.............................] - ETA: 9s - loss: 0.0413 - accuracy: 0.9878
  91/1688 [>.............................] - ETA: 9s - loss: 0.0406 - accuracy: 0.9880
 100/1688 [>.............................] - ETA: 9s - loss: 0.0375 - accuracy: 0.9891
 109/1688 [>.............................] - ETA: 9s - loss: 0.0352 - accuracy: 0.9900
 118/1688 [=>............................] - ETA: 9s - loss: 0.0333 - accuracy: 0.9905
 127/1688 [=>............................] - ETA: 9s - loss: 0.0335 - accuracy: 0.9906
 136/1688 [=>............................] - ETA: 8s - loss: 0.0356 - accuracy: 0.9892
 145/1688 [=>............................] - ETA: 8s - loss: 0.0365 - accuracy: 0.9888
 154/1688 [=>............................] - ETA: 8s - loss: 0.0362 - accuracy: 0.9892
 163/1688 [=>............................] - ETA: 8s - loss: 0.0362 - accuracy: 0.9893
 172/1688 [==>...........................] - ETA: 8s - loss: 0.0371 - accuracy: 0.9889
 181/1688 [==>...........................] - ETA: 8s - loss: 0.0379 - accuracy: 0.9888
 190/1688 [==>...........................] - ETA: 8s - loss: 0.0389 - accuracy: 0.9885
 199/1688 [==>...........................] - ETA: 8s - loss: 0.0383 - accuracy: 0.9887
 208/1688 [==>...........................] - ETA: 8s - loss: 0.0391 - accuracy: 0.9883
 217/1688 [==>...........................] - ETA: 8s - loss: 0.0389 - accuracy: 0.9882
 226/1688 [===>..........................] - ETA: 8s - loss: 0.0387 - accuracy: 0.9882
 235/1688 [===>..........................] - ETA: 8s - loss: 0.0398 - accuracy: 0.9879
 244/1688 [===>..........................] - ETA: 8s - loss: 0.0403 - accuracy: 0.9877
 253/1688 [===>..........................] - ETA: 8s - loss: 0.0399 - accuracy: 0.9878
 262/1688 [===>..........................] - ETA: 8s - loss: 0.0400 - accuracy: 0.9874
 271/1688 [===>..........................] - ETA: 8s - loss: 0.0408 - accuracy: 0.9870
 280/1688 [===>..........................] - ETA: 8s - loss: 0.0408 - accuracy: 0.9871
 289/1688 [====>.........................] - ETA: 8s - loss: 0.0414 - accuracy: 0.9868
 298/1688 [====>.........................] - ETA: 8s - loss: 0.0427 - accuracy: 0.9864
 307/1688 [====>.........................] - ETA: 8s - loss: 0.0427 - accuracy: 0.9863
 316/1688 [====>.........................] - ETA: 7s - loss: 0.0422 - accuracy: 0.9864
 325/1688 [====>.........................] - ETA: 7s - loss: 0.0430 - accuracy: 0.9862
 334/1688 [====>.........................] - ETA: 7s - loss: 0.0430 - accuracy: 0.9862
 343/1688 [=====>........................] - ETA: 7s - loss: 0.0428 - accuracy: 0.9862
 352/1688 [=====>........................] - ETA: 7s - loss: 0.0425 - accuracy: 0.9863
 361/1688 [=====>........................] - ETA: 7s - loss: 0.0421 - accuracy: 0.9864
 370/1688 [=====>........................] - ETA: 7s - loss: 0.0426 - accuracy: 0.9862
 379/1688 [=====>........................] - ETA: 7s - loss: 0.0437 - accuracy: 0.9858
 388/1688 [=====>........................] - ETA: 7s - loss: 0.0439 - accuracy: 0.9858
 397/1688 [======>.......................] - ETA: 7s - loss: 0.0446 - accuracy: 0.9858
 406/1688 [======>.......................] - ETA: 7s - loss: 0.0446 - accuracy: 0.9857
 415/1688 [======>.......................] - ETA: 7s - loss: 0.0460 - accuracy: 0.9855
 424/1688 [======>.......................] - ETA: 7s - loss: 0.0458 - accuracy: 0.9856
 433/1688 [======>.......................] - ETA: 7s - loss: 0.0454 - accuracy: 0.9857
 442/1688 [======>.......................] - ETA: 7s - loss: 0.0450 - accuracy: 0.9856
 451/1688 [=======>......................] - ETA: 7s - loss: 0.0451 - accuracy: 0.9855
 460/1688 [=======>......................] - ETA: 7s - loss: 0.0455 - accuracy: 0.9855
 469/1688 [=======>......................] - ETA: 7s - loss: 0.0459 - accuracy: 0.9853
 478/1688 [=======>......................] - ETA: 7s - loss: 0.0457 - accuracy: 0.9854
 487/1688 [=======>......................] - ETA: 7s - loss: 0.0455 - accuracy: 0.9854
 496/1688 [=======>......................] - ETA: 6s - loss: 0.0453 - accuracy: 0.9853
 505/1688 [=======>......................] - ETA: 6s - loss: 0.0448 - accuracy: 0.9855
 513/1688 [========>.....................] - ETA: 6s - loss: 0.0455 - accuracy: 0.9853
 522/1688 [========>.....................] - ETA: 6s - loss: 0.0451 - accuracy: 0.9855
 531/1688 [========>.....................] - ETA: 6s - loss: 0.0461 - accuracy: 0.9852
 540/1688 [========>.....................] - ETA: 6s - loss: 0.0458 - accuracy: 0.9852
 549/1688 [========>.....................] - ETA: 6s - loss: 0.0458 - accuracy: 0.9853
 558/1688 [========>.....................] - ETA: 6s - loss: 0.0459 - accuracy: 0.9853
 567/1688 [=========>....................] - ETA: 6s - loss: 0.0459 - accuracy: 0.9853
 576/1688 [=========>....................] - ETA: 6s - loss: 0.0455 - accuracy: 0.9853
 585/1688 [=========>....................] - ETA: 6s - loss: 0.0455 - accuracy: 0.9853
 594/1688 [=========>....................] - ETA: 6s - loss: 0.0451 - accuracy: 0.9853
 603/1688 [=========>....................] - ETA: 6s - loss: 0.0453 - accuracy: 0.9852
 612/1688 [=========>....................] - ETA: 6s - loss: 0.0450 - accuracy: 0.9853
 621/1688 [==========>...................] - ETA: 6s - loss: 0.0452 - accuracy: 0.9853
 630/1688 [==========>...................] - ETA: 6s - loss: 0.0454 - accuracy: 0.9853
 639/1688 [==========>...................] - ETA: 6s - loss: 0.0452 - accuracy: 0.9854
 648/1688 [==========>...................] - ETA: 6s - loss: 0.0454 - accuracy: 0.9854
 657/1688 [==========>...................] - ETA: 6s - loss: 0.0450 - accuracy: 0.9855
 666/1688 [==========>...................] - ETA: 5s - loss: 0.0448 - accuracy: 0.9856
 675/1688 [==========>...................] - ETA: 5s - loss: 0.0446 - accuracy: 0.9856
 684/1688 [===========>..................] - ETA: 5s - loss: 0.0444 - accuracy: 0.9857
 693/1688 [===========>..................] - ETA: 5s - loss: 0.0442 - accuracy: 0.9857
 702/1688 [===========>..................] - ETA: 5s - loss: 0.0439 - accuracy: 0.9858
 711/1688 [===========>..................] - ETA: 5s - loss: 0.0438 - accuracy: 0.9857
 720/1688 [===========>..................] - ETA: 5s - loss: 0.0442 - accuracy: 0.9857
 729/1688 [===========>..................] - ETA: 5s - loss: 0.0439 - accuracy: 0.9858
 738/1688 [============>.................] - ETA: 5s - loss: 0.0440 - accuracy: 0.9857
 747/1688 [============>.................] - ETA: 5s - loss: 0.0441 - accuracy: 0.9857
 756/1688 [============>.................] - ETA: 5s - loss: 0.0437 - accuracy: 0.9859
 765/1688 [============>.................] - ETA: 5s - loss: 0.0440 - accuracy: 0.9858
 774/1688 [============>.................] - ETA: 5s - loss: 0.0440 - accuracy: 0.9857
 783/1688 [============>.................] - ETA: 5s - loss: 0.0436 - accuracy: 0.9858
 792/1688 [=============>................] - ETA: 5s - loss: 0.0437 - accuracy: 0.9856
 801/1688 [=============>................] - ETA: 5s - loss: 0.0435 - accuracy: 0.9856
 810/1688 [=============>................] - ETA: 5s - loss: 0.0435 - accuracy: 0.9856
 819/1688 [=============>................] - ETA: 5s - loss: 0.0440 - accuracy: 0.9856
 828/1688 [=============>................] - ETA: 5s - loss: 0.0444 - accuracy: 0.9855
 837/1688 [=============>................] - ETA: 4s - loss: 0.0445 - accuracy: 0.9855
 846/1688 [==============>...............] - ETA: 4s - loss: 0.0443 - accuracy: 0.9855
 855/1688 [==============>...............] - ETA: 4s - loss: 0.0443 - accuracy: 0.9855
 864/1688 [==============>...............] - ETA: 4s - loss: 0.0442 - accuracy: 0.9855
 873/1688 [==============>...............] - ETA: 4s - loss: 0.0443 - accuracy: 0.9854
 882/1688 [==============>...............] - ETA: 4s - loss: 0.0440 - accuracy: 0.9855
 891/1688 [==============>...............] - ETA: 4s - loss: 0.0439 - accuracy: 0.9855
 900/1688 [==============>...............] - ETA: 4s - loss: 0.0440 - accuracy: 0.9856
 909/1688 [===============>..............] - ETA: 4s - loss: 0.0439 - accuracy: 0.9856
 918/1688 [===============>..............] - ETA: 4s - loss: 0.0440 - accuracy: 0.9856
 927/1688 [===============>..............] - ETA: 4s - loss: 0.0442 - accuracy: 0.9856
 936/1688 [===============>..............] - ETA: 4s - loss: 0.0448 - accuracy: 0.9854
 945/1688 [===============>..............] - ETA: 4s - loss: 0.0447 - accuracy: 0.9854
 954/1688 [===============>..............] - ETA: 4s - loss: 0.0446 - accuracy: 0.9855
 963/1688 [================>.............] - ETA: 4s - loss: 0.0445 - accuracy: 0.9855
 972/1688 [================>.............] - ETA: 4s - loss: 0.0444 - accuracy: 0.9855
 981/1688 [================>.............] - ETA: 4s - loss: 0.0446 - accuracy: 0.9854
 990/1688 [================>.............] - ETA: 4s - loss: 0.0445 - accuracy: 0.9854
 999/1688 [================>.............] - ETA: 4s - loss: 0.0443 - accuracy: 0.9855
1008/1688 [================>.............] - ETA: 3s - loss: 0.0443 - accuracy: 0.9854
1017/1688 [=================>............] - ETA: 3s - loss: 0.0440 - accuracy: 0.9855
1026/1688 [=================>............] - ETA: 3s - loss: 0.0441 - accuracy: 0.9855
1035/1688 [=================>............] - ETA: 3s - loss: 0.0441 - accuracy: 0.9854
1044/1688 [=================>............] - ETA: 3s - loss: 0.0441 - accuracy: 0.9854
1053/1688 [=================>............] - ETA: 3s - loss: 0.0440 - accuracy: 0.9854
1062/1688 [=================>............] - ETA: 3s - loss: 0.0442 - accuracy: 0.9854
1071/1688 [==================>...........] - ETA: 3s - loss: 0.0441 - accuracy: 0.9854
1080/1688 [==================>...........] - ETA: 3s - loss: 0.0441 - accuracy: 0.9853
1089/1688 [==================>...........] - ETA: 3s - loss: 0.0446 - accuracy: 0.9852
1098/1688 [==================>...........] - ETA: 3s - loss: 0.0446 - accuracy: 0.9852
1107/1688 [==================>...........] - ETA: 3s - loss: 0.0453 - accuracy: 0.9852
1116/1688 [==================>...........] - ETA: 3s - loss: 0.0459 - accuracy: 0.9850
1125/1688 [==================>...........] - ETA: 3s - loss: 0.0459 - accuracy: 0.9851
1134/1688 [===================>..........] - ETA: 3s - loss: 0.0460 - accuracy: 0.9851
1143/1688 [===================>..........] - ETA: 3s - loss: 0.0461 - accuracy: 0.9851
1152/1688 [===================>..........] - ETA: 3s - loss: 0.0463 - accuracy: 0.9850
1161/1688 [===================>..........] - ETA: 3s - loss: 0.0461 - accuracy: 0.9851
1170/1688 [===================>..........] - ETA: 3s - loss: 0.0461 - accuracy: 0.9851
1178/1688 [===================>..........] - ETA: 2s - loss: 0.0465 - accuracy: 0.9849
1187/1688 [====================>.........] - ETA: 2s - loss: 0.0466 - accuracy: 0.9848
1196/1688 [====================>.........] - ETA: 2s - loss: 0.0470 - accuracy: 0.9847
1205/1688 [====================>.........] - ETA: 2s - loss: 0.0467 - accuracy: 0.9848
1214/1688 [====================>.........] - ETA: 2s - loss: 0.0467 - accuracy: 0.9848
1223/1688 [====================>.........] - ETA: 2s - loss: 0.0467 - accuracy: 0.9848
1232/1688 [====================>.........] - ETA: 2s - loss: 0.0469 - accuracy: 0.9847
1241/1688 [=====================>........] - ETA: 2s - loss: 0.0471 - accuracy: 0.9846
1250/1688 [=====================>........] - ETA: 2s - loss: 0.0471 - accuracy: 0.9845
1259/1688 [=====================>........] - ETA: 2s - loss: 0.0476 - accuracy: 0.9845
1268/1688 [=====================>........] - ETA: 2s - loss: 0.0475 - accuracy: 0.9845
1277/1688 [=====================>........] - ETA: 2s - loss: 0.0476 - accuracy: 0.9845
1286/1688 [=====================>........] - ETA: 2s - loss: 0.0475 - accuracy: 0.9845
1295/1688 [======================>.......] - ETA: 2s - loss: 0.0473 - accuracy: 0.9845
1304/1688 [======================>.......] - ETA: 2s - loss: 0.0472 - accuracy: 0.9845
1313/1688 [======================>.......] - ETA: 2s - loss: 0.0476 - accuracy: 0.9845
1322/1688 [======================>.......] - ETA: 2s - loss: 0.0476 - accuracy: 0.9845
1331/1688 [======================>.......] - ETA: 2s - loss: 0.0476 - accuracy: 0.9844
1340/1688 [======================>.......] - ETA: 2s - loss: 0.0475 - accuracy: 0.9845
1349/1688 [======================>.......] - ETA: 1s - loss: 0.0475 - accuracy: 0.9844
1358/1688 [=======================>......] - ETA: 1s - loss: 0.0475 - accuracy: 0.9844
1367/1688 [=======================>......] - ETA: 1s - loss: 0.0473 - accuracy: 0.9845
1376/1688 [=======================>......] - ETA: 1s - loss: 0.0472 - accuracy: 0.9845
1385/1688 [=======================>......] - ETA: 1s - loss: 0.0470 - accuracy: 0.9845
1394/1688 [=======================>......] - ETA: 1s - loss: 0.0471 - accuracy: 0.9845
1403/1688 [=======================>......] - ETA: 1s - loss: 0.0470 - accuracy: 0.9845
1412/1688 [========================>.....] - ETA: 1s - loss: 0.0471 - accuracy: 0.9845
1421/1688 [========================>.....] - ETA: 1s - loss: 0.0473 - accuracy: 0.9844
1430/1688 [========================>.....] - ETA: 1s - loss: 0.0471 - accuracy: 0.9845
1439/1688 [========================>.....] - ETA: 1s - loss: 0.0469 - accuracy: 0.9845
1448/1688 [========================>.....] - ETA: 1s - loss: 0.0471 - accuracy: 0.9845
1457/1688 [========================>.....] - ETA: 1s - loss: 0.0470 - accuracy: 0.9846
1466/1688 [=========================>....] - ETA: 1s - loss: 0.0470 - accuracy: 0.9845
1475/1688 [=========================>....] - ETA: 1s - loss: 0.0470 - accuracy: 0.9846
1484/1688 [=========================>....] - ETA: 1s - loss: 0.0469 - accuracy: 0.9846
1493/1688 [=========================>....] - ETA: 1s - loss: 0.0469 - accuracy: 0.9847
1502/1688 [=========================>....] - ETA: 1s - loss: 0.0470 - accuracy: 0.9846
1511/1688 [=========================>....] - ETA: 1s - loss: 0.0474 - accuracy: 0.9845
1520/1688 [==========================>...] - ETA: 0s - loss: 0.0472 - accuracy: 0.9846
1529/1688 [==========================>...] - ETA: 0s - loss: 0.0471 - accuracy: 0.9846
1538/1688 [==========================>...] - ETA: 0s - loss: 0.0471 - accuracy: 0.9847
1547/1688 [==========================>...] - ETA: 0s - loss: 0.0472 - accuracy: 0.9846
1556/1688 [==========================>...] - ETA: 0s - loss: 0.0472 - accuracy: 0.9846
1565/1688 [==========================>...] - ETA: 0s - loss: 0.0471 - accuracy: 0.9847
1574/1688 [==========================>...] - ETA: 0s - loss: 0.0472 - accuracy: 0.9847
1583/1688 [===========================>..] - ETA: 0s - loss: 0.0471 - accuracy: 0.9846
1592/1688 [===========================>..] - ETA: 0s - loss: 0.0472 - accuracy: 0.9846
1601/1688 [===========================>..] - ETA: 0s - loss: 0.0471 - accuracy: 0.9846
1610/1688 [===========================>..] - ETA: 0s - loss: 0.0472 - accuracy: 0.9847
1619/1688 [===========================>..] - ETA: 0s - loss: 0.0470 - accuracy: 0.9848
1628/1688 [===========================>..] - ETA: 0s - loss: 0.0471 - accuracy: 0.9847
1637/1688 [============================>.] - ETA: 0s - loss: 0.0470 - accuracy: 0.9847
1646/1688 [============================>.] - ETA: 0s - loss: 0.0468 - accuracy: 0.9848
1655/1688 [============================>.] - ETA: 0s - loss: 0.0469 - accuracy: 0.9848
1664/1688 [============================>.] - ETA: 0s - loss: 0.0468 - accuracy: 0.9849
1673/1688 [============================>.] - ETA: 0s - loss: 0.0468 - accuracy: 0.9849
1682/1688 [============================>.] - ETA: 0s - loss: 0.0469 - accuracy: 0.9848
1688/1688 [==============================] - 10s 6ms/step - loss: 0.0469 - accuracy: 0.9849 - val_loss: 0.0517 - val_accuracy: 0.9860
Epoch 3/5

   1/1688 [..............................] - ETA: 0s - loss: 7.5884e-04 - accuracy: 1.0000
  10/1688 [..............................] - ETA: 8s - loss: 0.0674 - accuracy: 0.9844    
  19/1688 [..............................] - ETA: 9s - loss: 0.0467 - accuracy: 0.9885
  28/1688 [..............................] - ETA: 9s - loss: 0.0438 - accuracy: 0.9877
  37/1688 [..............................] - ETA: 9s - loss: 0.0417 - accuracy: 0.9882
  46/1688 [..............................] - ETA: 9s - loss: 0.0389 - accuracy: 0.9885
  55/1688 [..............................] - ETA: 9s - loss: 0.0400 - accuracy: 0.9864
  64/1688 [>.............................] - ETA: 9s - loss: 0.0378 - accuracy: 0.9873
  73/1688 [>.............................] - ETA: 9s - loss: 0.0371 - accuracy: 0.9872
  82/1688 [>.............................] - ETA: 9s - loss: 0.0356 - accuracy: 0.9878
  91/1688 [>.............................] - ETA: 9s - loss: 0.0371 - accuracy: 0.9876
 100/1688 [>.............................] - ETA: 9s - loss: 0.0351 - accuracy: 0.9884
 109/1688 [>.............................] - ETA: 9s - loss: 0.0355 - accuracy: 0.9885
 118/1688 [=>............................] - ETA: 9s - loss: 0.0365 - accuracy: 0.9886
 127/1688 [=>............................] - ETA: 9s - loss: 0.0364 - accuracy: 0.9884
 136/1688 [=>............................] - ETA: 8s - loss: 0.0366 - accuracy: 0.9883
 145/1688 [=>............................] - ETA: 8s - loss: 0.0371 - accuracy: 0.9886
 154/1688 [=>............................] - ETA: 8s - loss: 0.0413 - accuracy: 0.9870
 163/1688 [=>............................] - ETA: 8s - loss: 0.0396 - accuracy: 0.9877
 172/1688 [==>...........................] - ETA: 8s - loss: 0.0416 - accuracy: 0.9869
 181/1688 [==>...........................] - ETA: 8s - loss: 0.0422 - accuracy: 0.9867
 190/1688 [==>...........................] - ETA: 8s - loss: 0.0408 - accuracy: 0.9870
 199/1688 [==>...........................] - ETA: 8s - loss: 0.0402 - accuracy: 0.9873
 208/1688 [==>...........................] - ETA: 8s - loss: 0.0404 - accuracy: 0.9871
 217/1688 [==>...........................] - ETA: 8s - loss: 0.0394 - accuracy: 0.9873
 226/1688 [===>..........................] - ETA: 8s - loss: 0.0393 - accuracy: 0.9876
 235/1688 [===>..........................] - ETA: 8s - loss: 0.0399 - accuracy: 0.9875
 244/1688 [===>..........................] - ETA: 8s - loss: 0.0397 - accuracy: 0.9874
 253/1688 [===>..........................] - ETA: 8s - loss: 0.0391 - accuracy: 0.9876
 262/1688 [===>..........................] - ETA: 8s - loss: 0.0399 - accuracy: 0.9875
 271/1688 [===>..........................] - ETA: 8s - loss: 0.0396 - accuracy: 0.9878
 280/1688 [===>..........................] - ETA: 8s - loss: 0.0397 - accuracy: 0.9877
 289/1688 [====>.........................] - ETA: 8s - loss: 0.0392 - accuracy: 0.9878
 298/1688 [====>.........................] - ETA: 8s - loss: 0.0397 - accuracy: 0.9876
 307/1688 [====>.........................] - ETA: 8s - loss: 0.0393 - accuracy: 0.9879
 316/1688 [====>.........................] - ETA: 7s - loss: 0.0386 - accuracy: 0.9881
 325/1688 [====>.........................] - ETA: 7s - loss: 0.0382 - accuracy: 0.9883
 334/1688 [====>.........................] - ETA: 7s - loss: 0.0378 - accuracy: 0.9884
 343/1688 [=====>........................] - ETA: 7s - loss: 0.0377 - accuracy: 0.9884
 352/1688 [=====>........................] - ETA: 7s - loss: 0.0379 - accuracy: 0.9882
 361/1688 [=====>........................] - ETA: 7s - loss: 0.0373 - accuracy: 0.9884
 370/1688 [=====>........................] - ETA: 7s - loss: 0.0366 - accuracy: 0.9887
 379/1688 [=====>........................] - ETA: 7s - loss: 0.0361 - accuracy: 0.9886
 388/1688 [=====>........................] - ETA: 7s - loss: 0.0360 - accuracy: 0.9886
 397/1688 [======>.......................] - ETA: 7s - loss: 0.0361 - accuracy: 0.9885
 406/1688 [======>.......................] - ETA: 7s - loss: 0.0365 - accuracy: 0.9883
 415/1688 [======>.......................] - ETA: 7s - loss: 0.0365 - accuracy: 0.9883
 424/1688 [======>.......................] - ETA: 7s - loss: 0.0363 - accuracy: 0.9883
 433/1688 [======>.......................] - ETA: 7s - loss: 0.0364 - accuracy: 0.9882
 442/1688 [======>.......................] - ETA: 7s - loss: 0.0364 - accuracy: 0.9883
 451/1688 [=======>......................] - ETA: 7s - loss: 0.0362 - accuracy: 0.9884
 460/1688 [=======>......................] - ETA: 7s - loss: 0.0365 - accuracy: 0.9882
 469/1688 [=======>......................] - ETA: 7s - loss: 0.0363 - accuracy: 0.9883
 478/1688 [=======>......................] - ETA: 7s - loss: 0.0363 - accuracy: 0.9882
 487/1688 [=======>......................] - ETA: 6s - loss: 0.0359 - accuracy: 0.9883
 496/1688 [=======>......................] - ETA: 6s - loss: 0.0363 - accuracy: 0.9880
 505/1688 [=======>......................] - ETA: 6s - loss: 0.0369 - accuracy: 0.9878
 514/1688 [========>.....................] - ETA: 6s - loss: 0.0371 - accuracy: 0.9877
 523/1688 [========>.....................] - ETA: 6s - loss: 0.0372 - accuracy: 0.9877
 532/1688 [========>.....................] - ETA: 6s - loss: 0.0371 - accuracy: 0.9878
 541/1688 [========>.....................] - ETA: 6s - loss: 0.0375 - accuracy: 0.9876
 550/1688 [========>.....................] - ETA: 6s - loss: 0.0380 - accuracy: 0.9875
 559/1688 [========>.....................] - ETA: 6s - loss: 0.0386 - accuracy: 0.9874
 568/1688 [=========>....................] - ETA: 6s - loss: 0.0388 - accuracy: 0.9873
 577/1688 [=========>....................] - ETA: 6s - loss: 0.0391 - accuracy: 0.9873
 586/1688 [=========>....................] - ETA: 6s - loss: 0.0391 - accuracy: 0.9873
 595/1688 [=========>....................] - ETA: 6s - loss: 0.0391 - accuracy: 0.9873
 604/1688 [=========>....................] - ETA: 6s - loss: 0.0392 - accuracy: 0.9873
 613/1688 [=========>....................] - ETA: 6s - loss: 0.0395 - accuracy: 0.9873
 622/1688 [==========>...................] - ETA: 6s - loss: 0.0399 - accuracy: 0.9872
 631/1688 [==========>...................] - ETA: 6s - loss: 0.0399 - accuracy: 0.9873
 640/1688 [==========>...................] - ETA: 6s - loss: 0.0399 - accuracy: 0.9873
 649/1688 [==========>...................] - ETA: 6s - loss: 0.0407 - accuracy: 0.9870
 658/1688 [==========>...................] - ETA: 5s - loss: 0.0403 - accuracy: 0.9872
 667/1688 [==========>...................] - ETA: 5s - loss: 0.0407 - accuracy: 0.9870
 676/1688 [===========>..................] - ETA: 5s - loss: 0.0408 - accuracy: 0.9870
 685/1688 [===========>..................] - ETA: 5s - loss: 0.0412 - accuracy: 0.9869
 694/1688 [===========>..................] - ETA: 5s - loss: 0.0410 - accuracy: 0.9869
 703/1688 [===========>..................] - ETA: 5s - loss: 0.0407 - accuracy: 0.9870
 712/1688 [===========>..................] - ETA: 5s - loss: 0.0411 - accuracy: 0.9869
 721/1688 [===========>..................] - ETA: 5s - loss: 0.0412 - accuracy: 0.9869
 730/1688 [===========>..................] - ETA: 5s - loss: 0.0410 - accuracy: 0.9870
 739/1688 [============>.................] - ETA: 5s - loss: 0.0410 - accuracy: 0.9870
 748/1688 [============>.................] - ETA: 5s - loss: 0.0407 - accuracy: 0.9870
 757/1688 [============>.................] - ETA: 5s - loss: 0.0407 - accuracy: 0.9870
 766/1688 [============>.................] - ETA: 5s - loss: 0.0407 - accuracy: 0.9869
 775/1688 [============>.................] - ETA: 5s - loss: 0.0403 - accuracy: 0.9871
 784/1688 [============>.................] - ETA: 5s - loss: 0.0400 - accuracy: 0.9872
 793/1688 [=============>................] - ETA: 5s - loss: 0.0399 - accuracy: 0.9873
 802/1688 [=============>................] - ETA: 5s - loss: 0.0400 - accuracy: 0.9873
 811/1688 [=============>................] - ETA: 5s - loss: 0.0401 - accuracy: 0.9873
 820/1688 [=============>................] - ETA: 5s - loss: 0.0400 - accuracy: 0.9873
 829/1688 [=============>................] - ETA: 5s - loss: 0.0401 - accuracy: 0.9873
 838/1688 [=============>................] - ETA: 4s - loss: 0.0400 - accuracy: 0.9873
 847/1688 [==============>...............] - ETA: 4s - loss: 0.0399 - accuracy: 0.9873
 856/1688 [==============>...............] - ETA: 4s - loss: 0.0399 - accuracy: 0.9872
 865/1688 [==============>...............] - ETA: 4s - loss: 0.0396 - accuracy: 0.9873
 874/1688 [==============>...............] - ETA: 4s - loss: 0.0395 - accuracy: 0.9873
 883/1688 [==============>...............] - ETA: 4s - loss: 0.0395 - accuracy: 0.9873
 892/1688 [==============>...............] - ETA: 4s - loss: 0.0394 - accuracy: 0.9873
 901/1688 [===============>..............] - ETA: 4s - loss: 0.0391 - accuracy: 0.9874
 910/1688 [===============>..............] - ETA: 4s - loss: 0.0392 - accuracy: 0.9874
 919/1688 [===============>..............] - ETA: 4s - loss: 0.0391 - accuracy: 0.9875
 928/1688 [===============>..............] - ETA: 4s - loss: 0.0388 - accuracy: 0.9875
 937/1688 [===============>..............] - ETA: 4s - loss: 0.0386 - accuracy: 0.9876
 946/1688 [===============>..............] - ETA: 4s - loss: 0.0385 - accuracy: 0.9876
 955/1688 [===============>..............] - ETA: 4s - loss: 0.0382 - accuracy: 0.9877
 964/1688 [================>.............] - ETA: 4s - loss: 0.0380 - accuracy: 0.9878
 973/1688 [================>.............] - ETA: 4s - loss: 0.0378 - accuracy: 0.9879
 982/1688 [================>.............] - ETA: 4s - loss: 0.0379 - accuracy: 0.9878
 991/1688 [================>.............] - ETA: 4s - loss: 0.0377 - accuracy: 0.9879
1000/1688 [================>.............] - ETA: 4s - loss: 0.0378 - accuracy: 0.9878
1009/1688 [================>.............] - ETA: 3s - loss: 0.0387 - accuracy: 0.9877
1018/1688 [=================>............] - ETA: 3s - loss: 0.0386 - accuracy: 0.9877
1027/1688 [=================>............] - ETA: 3s - loss: 0.0387 - accuracy: 0.9876
1036/1688 [=================>............] - ETA: 3s - loss: 0.0385 - accuracy: 0.9877
1045/1688 [=================>............] - ETA: 3s - loss: 0.0384 - accuracy: 0.9877
1054/1688 [=================>............] - ETA: 3s - loss: 0.0388 - accuracy: 0.9877
1063/1688 [=================>............] - ETA: 3s - loss: 0.0392 - accuracy: 0.9876
1072/1688 [==================>...........] - ETA: 3s - loss: 0.0390 - accuracy: 0.9877
1081/1688 [==================>...........] - ETA: 3s - loss: 0.0389 - accuracy: 0.9877
1090/1688 [==================>...........] - ETA: 3s - loss: 0.0392 - accuracy: 0.9876
1099/1688 [==================>...........] - ETA: 3s - loss: 0.0393 - accuracy: 0.9875
1108/1688 [==================>...........] - ETA: 3s - loss: 0.0394 - accuracy: 0.9874
1117/1688 [==================>...........] - ETA: 3s - loss: 0.0393 - accuracy: 0.9874
1126/1688 [===================>..........] - ETA: 3s - loss: 0.0391 - accuracy: 0.9875
1135/1688 [===================>..........] - ETA: 3s - loss: 0.0391 - accuracy: 0.9875
1144/1688 [===================>..........] - ETA: 3s - loss: 0.0393 - accuracy: 0.9874
1153/1688 [===================>..........] - ETA: 3s - loss: 0.0398 - accuracy: 0.9873
1162/1688 [===================>..........] - ETA: 3s - loss: 0.0397 - accuracy: 0.9873
1171/1688 [===================>..........] - ETA: 3s - loss: 0.0396 - accuracy: 0.9874
1180/1688 [===================>..........] - ETA: 2s - loss: 0.0396 - accuracy: 0.9874
1189/1688 [====================>.........] - ETA: 2s - loss: 0.0394 - accuracy: 0.9874
1198/1688 [====================>.........] - ETA: 2s - loss: 0.0393 - accuracy: 0.9875
1207/1688 [====================>.........] - ETA: 2s - loss: 0.0392 - accuracy: 0.9875
1216/1688 [====================>.........] - ETA: 2s - loss: 0.0391 - accuracy: 0.9875
1225/1688 [====================>.........] - ETA: 2s - loss: 0.0391 - accuracy: 0.9875
1234/1688 [====================>.........] - ETA: 2s - loss: 0.0390 - accuracy: 0.9875
1243/1688 [=====================>........] - ETA: 2s - loss: 0.0389 - accuracy: 0.9875
1252/1688 [=====================>........] - ETA: 2s - loss: 0.0389 - accuracy: 0.9875
1261/1688 [=====================>........] - ETA: 2s - loss: 0.0387 - accuracy: 0.9876
1270/1688 [=====================>........] - ETA: 2s - loss: 0.0387 - accuracy: 0.9875
1279/1688 [=====================>........] - ETA: 2s - loss: 0.0387 - accuracy: 0.9875
1288/1688 [=====================>........] - ETA: 2s - loss: 0.0392 - accuracy: 0.9874
1297/1688 [======================>.......] - ETA: 2s - loss: 0.0393 - accuracy: 0.9874
1306/1688 [======================>.......] - ETA: 2s - loss: 0.0393 - accuracy: 0.9873
1315/1688 [======================>.......] - ETA: 2s - loss: 0.0392 - accuracy: 0.9874
1324/1688 [======================>.......] - ETA: 2s - loss: 0.0391 - accuracy: 0.9874
1333/1688 [======================>.......] - ETA: 2s - loss: 0.0390 - accuracy: 0.9875
1342/1688 [======================>.......] - ETA: 2s - loss: 0.0388 - accuracy: 0.9875
1351/1688 [=======================>......] - ETA: 1s - loss: 0.0387 - accuracy: 0.9875
1360/1688 [=======================>......] - ETA: 1s - loss: 0.0387 - accuracy: 0.9876
1369/1688 [=======================>......] - ETA: 1s - loss: 0.0386 - accuracy: 0.9877
1378/1688 [=======================>......] - ETA: 1s - loss: 0.0384 - accuracy: 0.9877
1387/1688 [=======================>......] - ETA: 1s - loss: 0.0383 - accuracy: 0.9877
1396/1688 [=======================>......] - ETA: 1s - loss: 0.0383 - accuracy: 0.9877
1405/1688 [=======================>......] - ETA: 1s - loss: 0.0383 - accuracy: 0.9878
1414/1688 [========================>.....] - ETA: 1s - loss: 0.0382 - accuracy: 0.9878
1423/1688 [========================>.....] - ETA: 1s - loss: 0.0381 - accuracy: 0.9878
1432/1688 [========================>.....] - ETA: 1s - loss: 0.0379 - accuracy: 0.9879
1441/1688 [========================>.....] - ETA: 1s - loss: 0.0378 - accuracy: 0.9879
1450/1688 [========================>.....] - ETA: 1s - loss: 0.0377 - accuracy: 0.9880
1459/1688 [========================>.....] - ETA: 1s - loss: 0.0379 - accuracy: 0.9879
1468/1688 [=========================>....] - ETA: 1s - loss: 0.0377 - accuracy: 0.9880
1477/1688 [=========================>....] - ETA: 1s - loss: 0.0376 - accuracy: 0.9880
1486/1688 [=========================>....] - ETA: 1s - loss: 0.0376 - accuracy: 0.9881
1495/1688 [=========================>....] - ETA: 1s - loss: 0.0375 - accuracy: 0.9881
1504/1688 [=========================>....] - ETA: 1s - loss: 0.0375 - accuracy: 0.9881
1513/1688 [=========================>....] - ETA: 1s - loss: 0.0375 - accuracy: 0.9881
1522/1688 [==========================>...] - ETA: 0s - loss: 0.0375 - accuracy: 0.9881
1531/1688 [==========================>...] - ETA: 0s - loss: 0.0373 - accuracy: 0.9881
1540/1688 [==========================>...] - ETA: 0s - loss: 0.0372 - accuracy: 0.9881
1549/1688 [==========================>...] - ETA: 0s - loss: 0.0372 - accuracy: 0.9882
1558/1688 [==========================>...] - ETA: 0s - loss: 0.0372 - accuracy: 0.9882
1567/1688 [==========================>...] - ETA: 0s - loss: 0.0370 - accuracy: 0.9882
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0372 - accuracy: 0.9882
1585/1688 [===========================>..] - ETA: 0s - loss: 0.0372 - accuracy: 0.9881
1594/1688 [===========================>..] - ETA: 0s - loss: 0.0371 - accuracy: 0.9882
1603/1688 [===========================>..] - ETA: 0s - loss: 0.0370 - accuracy: 0.9882
1612/1688 [===========================>..] - ETA: 0s - loss: 0.0369 - accuracy: 0.9882
1621/1688 [===========================>..] - ETA: 0s - loss: 0.0369 - accuracy: 0.9882
1630/1688 [===========================>..] - ETA: 0s - loss: 0.0368 - accuracy: 0.9882
1639/1688 [============================>.] - ETA: 0s - loss: 0.0368 - accuracy: 0.9882
1648/1688 [============================>.] - ETA: 0s - loss: 0.0369 - accuracy: 0.9882
1657/1688 [============================>.] - ETA: 0s - loss: 0.0368 - accuracy: 0.9883
1666/1688 [============================>.] - ETA: 0s - loss: 0.0369 - accuracy: 0.9882
1675/1688 [============================>.] - ETA: 0s - loss: 0.0370 - accuracy: 0.9882
1684/1688 [============================>.] - ETA: 0s - loss: 0.0369 - accuracy: 0.9882
1688/1688 [==============================] - 10s 6ms/step - loss: 0.0370 - accuracy: 0.9882 - val_loss: 0.0701 - val_accuracy: 0.9838
Epoch 4/5

   1/1688 [..............................] - ETA: 0s - loss: 4.8200e-04 - accuracy: 1.0000
  10/1688 [..............................] - ETA: 8s - loss: 0.0308 - accuracy: 0.9906    
  19/1688 [..............................] - ETA: 9s - loss: 0.0259 - accuracy: 0.9901
  28/1688 [..............................] - ETA: 9s - loss: 0.0232 - accuracy: 0.9922
  37/1688 [..............................] - ETA: 9s - loss: 0.0200 - accuracy: 0.9924
  46/1688 [..............................] - ETA: 9s - loss: 0.0201 - accuracy: 0.9918
  55/1688 [..............................] - ETA: 9s - loss: 0.0176 - accuracy: 0.9932
  64/1688 [>.............................] - ETA: 9s - loss: 0.0191 - accuracy: 0.9927
  73/1688 [>.............................] - ETA: 9s - loss: 0.0177 - accuracy: 0.9936
  82/1688 [>.............................] - ETA: 9s - loss: 0.0269 - accuracy: 0.9924
  91/1688 [>.............................] - ETA: 9s - loss: 0.0275 - accuracy: 0.9914
 100/1688 [>.............................] - ETA: 9s - loss: 0.0286 - accuracy: 0.9906
 109/1688 [>.............................] - ETA: 9s - loss: 0.0270 - accuracy: 0.9911
 118/1688 [=>............................] - ETA: 9s - loss: 0.0284 - accuracy: 0.9910
 127/1688 [=>............................] - ETA: 9s - loss: 0.0283 - accuracy: 0.9911
 136/1688 [=>............................] - ETA: 9s - loss: 0.0282 - accuracy: 0.9908
 145/1688 [=>............................] - ETA: 9s - loss: 0.0282 - accuracy: 0.9907
 154/1688 [=>............................] - ETA: 8s - loss: 0.0304 - accuracy: 0.9901
 163/1688 [=>............................] - ETA: 8s - loss: 0.0303 - accuracy: 0.9900
 172/1688 [==>...........................] - ETA: 8s - loss: 0.0297 - accuracy: 0.9900
 181/1688 [==>...........................] - ETA: 8s - loss: 0.0300 - accuracy: 0.9900
 190/1688 [==>...........................] - ETA: 8s - loss: 0.0303 - accuracy: 0.9901
 199/1688 [==>...........................] - ETA: 8s - loss: 0.0303 - accuracy: 0.9904
 208/1688 [==>...........................] - ETA: 8s - loss: 0.0300 - accuracy: 0.9904
 217/1688 [==>...........................] - ETA: 8s - loss: 0.0299 - accuracy: 0.9904
 226/1688 [===>..........................] - ETA: 8s - loss: 0.0315 - accuracy: 0.9899
 235/1688 [===>..........................] - ETA: 8s - loss: 0.0318 - accuracy: 0.9898
 244/1688 [===>..........................] - ETA: 8s - loss: 0.0323 - accuracy: 0.9892
 253/1688 [===>..........................] - ETA: 8s - loss: 0.0327 - accuracy: 0.9894
 262/1688 [===>..........................] - ETA: 8s - loss: 0.0331 - accuracy: 0.9894
 271/1688 [===>..........................] - ETA: 8s - loss: 0.0328 - accuracy: 0.9892
 280/1688 [===>..........................] - ETA: 8s - loss: 0.0327 - accuracy: 0.9893
 289/1688 [====>.........................] - ETA: 8s - loss: 0.0327 - accuracy: 0.9891
 298/1688 [====>.........................] - ETA: 8s - loss: 0.0320 - accuracy: 0.9893
 307/1688 [====>.........................] - ETA: 8s - loss: 0.0313 - accuracy: 0.9895
 316/1688 [====>.........................] - ETA: 8s - loss: 0.0316 - accuracy: 0.9894
 325/1688 [====>.........................] - ETA: 7s - loss: 0.0317 - accuracy: 0.9894
 334/1688 [====>.........................] - ETA: 7s - loss: 0.0315 - accuracy: 0.9894
 343/1688 [=====>........................] - ETA: 7s - loss: 0.0311 - accuracy: 0.9896
 352/1688 [=====>........................] - ETA: 7s - loss: 0.0313 - accuracy: 0.9895
 361/1688 [=====>........................] - ETA: 7s - loss: 0.0308 - accuracy: 0.9897
 370/1688 [=====>........................] - ETA: 7s - loss: 0.0317 - accuracy: 0.9895
 379/1688 [=====>........................] - ETA: 7s - loss: 0.0318 - accuracy: 0.9894
 388/1688 [=====>........................] - ETA: 7s - loss: 0.0312 - accuracy: 0.9897
 397/1688 [======>.......................] - ETA: 7s - loss: 0.0307 - accuracy: 0.9899
 406/1688 [======>.......................] - ETA: 7s - loss: 0.0303 - accuracy: 0.9901
 415/1688 [======>.......................] - ETA: 7s - loss: 0.0310 - accuracy: 0.9899
 424/1688 [======>.......................] - ETA: 7s - loss: 0.0306 - accuracy: 0.9900
 433/1688 [======>.......................] - ETA: 7s - loss: 0.0311 - accuracy: 0.9899
 442/1688 [======>.......................] - ETA: 7s - loss: 0.0306 - accuracy: 0.9900
 451/1688 [=======>......................] - ETA: 7s - loss: 0.0302 - accuracy: 0.9902
 460/1688 [=======>......................] - ETA: 7s - loss: 0.0312 - accuracy: 0.9900
 469/1688 [=======>......................] - ETA: 7s - loss: 0.0308 - accuracy: 0.9901
 478/1688 [=======>......................] - ETA: 7s - loss: 0.0305 - accuracy: 0.9902
 487/1688 [=======>......................] - ETA: 7s - loss: 0.0305 - accuracy: 0.9903
 496/1688 [=======>......................] - ETA: 6s - loss: 0.0310 - accuracy: 0.9901
 505/1688 [=======>......................] - ETA: 6s - loss: 0.0317 - accuracy: 0.9899
 514/1688 [========>.....................] - ETA: 6s - loss: 0.0314 - accuracy: 0.9901
 523/1688 [========>.....................] - ETA: 6s - loss: 0.0315 - accuracy: 0.9901
 532/1688 [========>.....................] - ETA: 6s - loss: 0.0314 - accuracy: 0.9901
 541/1688 [========>.....................] - ETA: 6s - loss: 0.0311 - accuracy: 0.9902
 550/1688 [========>.....................] - ETA: 6s - loss: 0.0316 - accuracy: 0.9899
 559/1688 [========>.....................] - ETA: 6s - loss: 0.0319 - accuracy: 0.9897
 568/1688 [=========>....................] - ETA: 6s - loss: 0.0321 - accuracy: 0.9897
 577/1688 [=========>....................] - ETA: 6s - loss: 0.0324 - accuracy: 0.9894
 586/1688 [=========>....................] - ETA: 6s - loss: 0.0321 - accuracy: 0.9895
 595/1688 [=========>....................] - ETA: 6s - loss: 0.0323 - accuracy: 0.9895
 604/1688 [=========>....................] - ETA: 6s - loss: 0.0324 - accuracy: 0.9895
 613/1688 [=========>....................] - ETA: 6s - loss: 0.0329 - accuracy: 0.9894
 621/1688 [==========>...................] - ETA: 6s - loss: 0.0332 - accuracy: 0.9894
 630/1688 [==========>...................] - ETA: 6s - loss: 0.0330 - accuracy: 0.9894
 639/1688 [==========>...................] - ETA: 6s - loss: 0.0336 - accuracy: 0.9894
 648/1688 [==========>...................] - ETA: 6s - loss: 0.0336 - accuracy: 0.9893
 657/1688 [==========>...................] - ETA: 6s - loss: 0.0335 - accuracy: 0.9893
 666/1688 [==========>...................] - ETA: 5s - loss: 0.0337 - accuracy: 0.9892
 675/1688 [==========>...................] - ETA: 5s - loss: 0.0341 - accuracy: 0.9891
 684/1688 [===========>..................] - ETA: 5s - loss: 0.0347 - accuracy: 0.9890
 693/1688 [===========>..................] - ETA: 5s - loss: 0.0352 - accuracy: 0.9888
 702/1688 [===========>..................] - ETA: 5s - loss: 0.0352 - accuracy: 0.9887
 711/1688 [===========>..................] - ETA: 5s - loss: 0.0352 - accuracy: 0.9887
 720/1688 [===========>..................] - ETA: 5s - loss: 0.0352 - accuracy: 0.9887
 729/1688 [===========>..................] - ETA: 5s - loss: 0.0351 - accuracy: 0.9886
 738/1688 [============>.................] - ETA: 5s - loss: 0.0349 - accuracy: 0.9887
 747/1688 [============>.................] - ETA: 5s - loss: 0.0349 - accuracy: 0.9886
 756/1688 [============>.................] - ETA: 5s - loss: 0.0352 - accuracy: 0.9885
 765/1688 [============>.................] - ETA: 5s - loss: 0.0353 - accuracy: 0.9886
 774/1688 [============>.................] - ETA: 5s - loss: 0.0355 - accuracy: 0.9886
 783/1688 [============>.................] - ETA: 5s - loss: 0.0360 - accuracy: 0.9886
 792/1688 [=============>................] - ETA: 5s - loss: 0.0359 - accuracy: 0.9886
 801/1688 [=============>................] - ETA: 5s - loss: 0.0359 - accuracy: 0.9886
 810/1688 [=============>................] - ETA: 5s - loss: 0.0358 - accuracy: 0.9886
 819/1688 [=============>................] - ETA: 5s - loss: 0.0356 - accuracy: 0.9886
 828/1688 [=============>................] - ETA: 5s - loss: 0.0359 - accuracy: 0.9885
 837/1688 [=============>................] - ETA: 4s - loss: 0.0360 - accuracy: 0.9885
 846/1688 [==============>...............] - ETA: 4s - loss: 0.0360 - accuracy: 0.9884
 855/1688 [==============>...............] - ETA: 4s - loss: 0.0360 - accuracy: 0.9884
 864/1688 [==============>...............] - ETA: 4s - loss: 0.0360 - accuracy: 0.9883
 873/1688 [==============>...............] - ETA: 4s - loss: 0.0363 - accuracy: 0.9882
 882/1688 [==============>...............] - ETA: 4s - loss: 0.0361 - accuracy: 0.9882
 891/1688 [==============>...............] - ETA: 4s - loss: 0.0361 - accuracy: 0.9883
 900/1688 [==============>...............] - ETA: 4s - loss: 0.0362 - accuracy: 0.9883
 909/1688 [===============>..............] - ETA: 4s - loss: 0.0361 - accuracy: 0.9882
 918/1688 [===============>..............] - ETA: 4s - loss: 0.0361 - accuracy: 0.9883
 927/1688 [===============>..............] - ETA: 4s - loss: 0.0360 - accuracy: 0.9883
 936/1688 [===============>..............] - ETA: 4s - loss: 0.0361 - accuracy: 0.9882
 945/1688 [===============>..............] - ETA: 4s - loss: 0.0362 - accuracy: 0.9882
 954/1688 [===============>..............] - ETA: 4s - loss: 0.0360 - accuracy: 0.9883
 963/1688 [================>.............] - ETA: 4s - loss: 0.0362 - accuracy: 0.9883
 972/1688 [================>.............] - ETA: 4s - loss: 0.0362 - accuracy: 0.9883
 981/1688 [================>.............] - ETA: 4s - loss: 0.0362 - accuracy: 0.9883
 990/1688 [================>.............] - ETA: 4s - loss: 0.0361 - accuracy: 0.9884
 999/1688 [================>.............] - ETA: 4s - loss: 0.0363 - accuracy: 0.9882
1008/1688 [================>.............] - ETA: 3s - loss: 0.0362 - accuracy: 0.9883
1017/1688 [=================>............] - ETA: 3s - loss: 0.0363 - accuracy: 0.9882
1026/1688 [=================>............] - ETA: 3s - loss: 0.0365 - accuracy: 0.9881
1035/1688 [=================>............] - ETA: 3s - loss: 0.0366 - accuracy: 0.9881
1044/1688 [=================>............] - ETA: 3s - loss: 0.0365 - accuracy: 0.9881
1053/1688 [=================>............] - ETA: 3s - loss: 0.0364 - accuracy: 0.9882
1062/1688 [=================>............] - ETA: 3s - loss: 0.0364 - accuracy: 0.9883
1071/1688 [==================>...........] - ETA: 3s - loss: 0.0362 - accuracy: 0.9883
1080/1688 [==================>...........] - ETA: 3s - loss: 0.0363 - accuracy: 0.9883
1089/1688 [==================>...........] - ETA: 3s - loss: 0.0361 - accuracy: 0.9883
1098/1688 [==================>...........] - ETA: 3s - loss: 0.0364 - accuracy: 0.9883
1107/1688 [==================>...........] - ETA: 3s - loss: 0.0365 - accuracy: 0.9882
1116/1688 [==================>...........] - ETA: 3s - loss: 0.0365 - accuracy: 0.9882
1125/1688 [==================>...........] - ETA: 3s - loss: 0.0366 - accuracy: 0.9882
1134/1688 [===================>..........] - ETA: 3s - loss: 0.0369 - accuracy: 0.9881
1143/1688 [===================>..........] - ETA: 3s - loss: 0.0367 - accuracy: 0.9881
1152/1688 [===================>..........] - ETA: 3s - loss: 0.0368 - accuracy: 0.9881
1161/1688 [===================>..........] - ETA: 3s - loss: 0.0369 - accuracy: 0.9880
1170/1688 [===================>..........] - ETA: 3s - loss: 0.0373 - accuracy: 0.9880
1179/1688 [===================>..........] - ETA: 2s - loss: 0.0372 - accuracy: 0.9880
1188/1688 [====================>.........] - ETA: 2s - loss: 0.0371 - accuracy: 0.9881
1197/1688 [====================>.........] - ETA: 2s - loss: 0.0372 - accuracy: 0.9881
1206/1688 [====================>.........] - ETA: 2s - loss: 0.0371 - accuracy: 0.9881
1215/1688 [====================>.........] - ETA: 2s - loss: 0.0371 - accuracy: 0.9880
1224/1688 [====================>.........] - ETA: 2s - loss: 0.0371 - accuracy: 0.9880
1233/1688 [====================>.........] - ETA: 2s - loss: 0.0369 - accuracy: 0.9881
1242/1688 [=====================>........] - ETA: 2s - loss: 0.0372 - accuracy: 0.9880
1251/1688 [=====================>........] - ETA: 2s - loss: 0.0373 - accuracy: 0.9880
1260/1688 [=====================>........] - ETA: 2s - loss: 0.0376 - accuracy: 0.9879
1269/1688 [=====================>........] - ETA: 2s - loss: 0.0374 - accuracy: 0.9880
1278/1688 [=====================>........] - ETA: 2s - loss: 0.0373 - accuracy: 0.9880
1287/1688 [=====================>........] - ETA: 2s - loss: 0.0378 - accuracy: 0.9879
1296/1688 [======================>.......] - ETA: 2s - loss: 0.0378 - accuracy: 0.9879
1305/1688 [======================>.......] - ETA: 2s - loss: 0.0378 - accuracy: 0.9880
1314/1688 [======================>.......] - ETA: 2s - loss: 0.0377 - accuracy: 0.9880
1323/1688 [======================>.......] - ETA: 2s - loss: 0.0377 - accuracy: 0.9879
1332/1688 [======================>.......] - ETA: 2s - loss: 0.0378 - accuracy: 0.9879
1341/1688 [======================>.......] - ETA: 2s - loss: 0.0379 - accuracy: 0.9878
1350/1688 [======================>.......] - ETA: 1s - loss: 0.0378 - accuracy: 0.9878
1359/1688 [=======================>......] - ETA: 1s - loss: 0.0378 - accuracy: 0.9878
1368/1688 [=======================>......] - ETA: 1s - loss: 0.0376 - accuracy: 0.9878
1377/1688 [=======================>......] - ETA: 1s - loss: 0.0374 - accuracy: 0.9879
1386/1688 [=======================>......] - ETA: 1s - loss: 0.0377 - accuracy: 0.9878
1395/1688 [=======================>......] - ETA: 1s - loss: 0.0377 - accuracy: 0.9878
1404/1688 [=======================>......] - ETA: 1s - loss: 0.0380 - accuracy: 0.9878
1413/1688 [========================>.....] - ETA: 1s - loss: 0.0383 - accuracy: 0.9877
1422/1688 [========================>.....] - ETA: 1s - loss: 0.0381 - accuracy: 0.9878
1431/1688 [========================>.....] - ETA: 1s - loss: 0.0382 - accuracy: 0.9877
1440/1688 [========================>.....] - ETA: 1s - loss: 0.0381 - accuracy: 0.9877
1449/1688 [========================>.....] - ETA: 1s - loss: 0.0380 - accuracy: 0.9878
1458/1688 [========================>.....] - ETA: 1s - loss: 0.0380 - accuracy: 0.9878
1467/1688 [=========================>....] - ETA: 1s - loss: 0.0380 - accuracy: 0.9878
1476/1688 [=========================>....] - ETA: 1s - loss: 0.0380 - accuracy: 0.9877
1485/1688 [=========================>....] - ETA: 1s - loss: 0.0384 - accuracy: 0.9876
1494/1688 [=========================>....] - ETA: 1s - loss: 0.0386 - accuracy: 0.9876
1503/1688 [=========================>....] - ETA: 1s - loss: 0.0387 - accuracy: 0.9875
1512/1688 [=========================>....] - ETA: 1s - loss: 0.0387 - accuracy: 0.9876
1521/1688 [==========================>...] - ETA: 0s - loss: 0.0388 - accuracy: 0.9875
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0388 - accuracy: 0.9876
1539/1688 [==========================>...] - ETA: 0s - loss: 0.0388 - accuracy: 0.9876
1548/1688 [==========================>...] - ETA: 0s - loss: 0.0391 - accuracy: 0.9875
1557/1688 [==========================>...] - ETA: 0s - loss: 0.0392 - accuracy: 0.9876
1566/1688 [==========================>...] - ETA: 0s - loss: 0.0392 - accuracy: 0.9875
1575/1688 [==========================>...] - ETA: 0s - loss: 0.0394 - accuracy: 0.9875
1584/1688 [===========================>..] - ETA: 0s - loss: 0.0394 - accuracy: 0.9875
1593/1688 [===========================>..] - ETA: 0s - loss: 0.0397 - accuracy: 0.9874
1602/1688 [===========================>..] - ETA: 0s - loss: 0.0398 - accuracy: 0.9874
1611/1688 [===========================>..] - ETA: 0s - loss: 0.0398 - accuracy: 0.9874
1620/1688 [===========================>..] - ETA: 0s - loss: 0.0398 - accuracy: 0.9874
1629/1688 [===========================>..] - ETA: 0s - loss: 0.0400 - accuracy: 0.9873
1638/1688 [============================>.] - ETA: 0s - loss: 0.0400 - accuracy: 0.9873
1647/1688 [============================>.] - ETA: 0s - loss: 0.0400 - accuracy: 0.9872
1656/1688 [============================>.] - ETA: 0s - loss: 0.0399 - accuracy: 0.9873
1665/1688 [============================>.] - ETA: 0s - loss: 0.0402 - accuracy: 0.9872
1674/1688 [============================>.] - ETA: 0s - loss: 0.0401 - accuracy: 0.9872
1683/1688 [============================>.] - ETA: 0s - loss: 0.0401 - accuracy: 0.9873
1688/1688 [==============================] - 10s 6ms/step - loss: 0.0400 - accuracy: 0.9873 - val_loss: 0.0537 - val_accuracy: 0.9867
Epoch 5/5

   1/1688 [..............................] - ETA: 0s - loss: 3.3946e-04 - accuracy: 1.0000
  10/1688 [..............................] - ETA: 8s - loss: 0.0187 - accuracy: 0.9937    
  19/1688 [..............................] - ETA: 9s - loss: 0.0203 - accuracy: 0.9951
  28/1688 [..............................] - ETA: 9s - loss: 0.0232 - accuracy: 0.9944
  37/1688 [..............................] - ETA: 9s - loss: 0.0222 - accuracy: 0.9941
  46/1688 [..............................] - ETA: 9s - loss: 0.0270 - accuracy: 0.9918
  55/1688 [..............................] - ETA: 9s - loss: 0.0265 - accuracy: 0.9920
  64/1688 [>.............................] - ETA: 9s - loss: 0.0274 - accuracy: 0.9922
  73/1688 [>.............................] - ETA: 9s - loss: 0.0254 - accuracy: 0.9927
  82/1688 [>.............................] - ETA: 9s - loss: 0.0266 - accuracy: 0.9924
  91/1688 [>.............................] - ETA: 9s - loss: 0.0249 - accuracy: 0.9928
 100/1688 [>.............................] - ETA: 9s - loss: 0.0241 - accuracy: 0.9925
 109/1688 [>.............................] - ETA: 9s - loss: 0.0223 - accuracy: 0.9931
 118/1688 [=>............................] - ETA: 9s - loss: 0.0227 - accuracy: 0.9934
 127/1688 [=>............................] - ETA: 9s - loss: 0.0243 - accuracy: 0.9929
 136/1688 [=>............................] - ETA: 8s - loss: 0.0244 - accuracy: 0.9926
 145/1688 [=>............................] - ETA: 8s - loss: 0.0249 - accuracy: 0.9922
 154/1688 [=>............................] - ETA: 8s - loss: 0.0256 - accuracy: 0.9923
 163/1688 [=>............................] - ETA: 8s - loss: 0.0268 - accuracy: 0.9921
 172/1688 [==>...........................] - ETA: 8s - loss: 0.0272 - accuracy: 0.9920
 181/1688 [==>...........................] - ETA: 8s - loss: 0.0279 - accuracy: 0.9917
 190/1688 [==>...........................] - ETA: 8s - loss: 0.0287 - accuracy: 0.9916
 199/1688 [==>...........................] - ETA: 8s - loss: 0.0278 - accuracy: 0.9920
 208/1688 [==>...........................] - ETA: 8s - loss: 0.0271 - accuracy: 0.9922
 217/1688 [==>...........................] - ETA: 8s - loss: 0.0264 - accuracy: 0.9924
 226/1688 [===>..........................] - ETA: 8s - loss: 0.0265 - accuracy: 0.9924
 235/1688 [===>..........................] - ETA: 8s - loss: 0.0261 - accuracy: 0.9924
 244/1688 [===>..........................] - ETA: 8s - loss: 0.0268 - accuracy: 0.9923
 253/1688 [===>..........................] - ETA: 8s - loss: 0.0263 - accuracy: 0.9925
 262/1688 [===>..........................] - ETA: 8s - loss: 0.0264 - accuracy: 0.9924
 271/1688 [===>..........................] - ETA: 8s - loss: 0.0286 - accuracy: 0.9920
 280/1688 [===>..........................] - ETA: 8s - loss: 0.0281 - accuracy: 0.9921
 289/1688 [====>.........................] - ETA: 8s - loss: 0.0274 - accuracy: 0.9922
 298/1688 [====>.........................] - ETA: 8s - loss: 0.0270 - accuracy: 0.9922
 307/1688 [====>.........................] - ETA: 8s - loss: 0.0274 - accuracy: 0.9919
 316/1688 [====>.........................] - ETA: 7s - loss: 0.0280 - accuracy: 0.9915
 325/1688 [====>.........................] - ETA: 7s - loss: 0.0276 - accuracy: 0.9915
 334/1688 [====>.........................] - ETA: 7s - loss: 0.0272 - accuracy: 0.9917
 343/1688 [=====>........................] - ETA: 7s - loss: 0.0268 - accuracy: 0.9917
 352/1688 [=====>........................] - ETA: 7s - loss: 0.0273 - accuracy: 0.9917
 361/1688 [=====>........................] - ETA: 7s - loss: 0.0269 - accuracy: 0.9918
 370/1688 [=====>........................] - ETA: 7s - loss: 0.0267 - accuracy: 0.9918
 379/1688 [=====>........................] - ETA: 7s - loss: 0.0269 - accuracy: 0.9918
 388/1688 [=====>........................] - ETA: 7s - loss: 0.0267 - accuracy: 0.9918
 397/1688 [======>.......................] - ETA: 7s - loss: 0.0262 - accuracy: 0.9920
 406/1688 [======>.......................] - ETA: 7s - loss: 0.0265 - accuracy: 0.9919
 415/1688 [======>.......................] - ETA: 7s - loss: 0.0266 - accuracy: 0.9918
 424/1688 [======>.......................] - ETA: 7s - loss: 0.0268 - accuracy: 0.9917
 433/1688 [======>.......................] - ETA: 7s - loss: 0.0263 - accuracy: 0.9918
 441/1688 [======>.......................] - ETA: 7s - loss: 0.0268 - accuracy: 0.9917
 450/1688 [======>.......................] - ETA: 7s - loss: 0.0266 - accuracy: 0.9917
 459/1688 [=======>......................] - ETA: 7s - loss: 0.0270 - accuracy: 0.9918
 468/1688 [=======>......................] - ETA: 7s - loss: 0.0267 - accuracy: 0.9918
 477/1688 [=======>......................] - ETA: 7s - loss: 0.0275 - accuracy: 0.9915
 486/1688 [=======>......................] - ETA: 6s - loss: 0.0273 - accuracy: 0.9916
 495/1688 [=======>......................] - ETA: 6s - loss: 0.0278 - accuracy: 0.9915
 504/1688 [=======>......................] - ETA: 6s - loss: 0.0280 - accuracy: 0.9914
 513/1688 [========>.....................] - ETA: 6s - loss: 0.0279 - accuracy: 0.9915
 522/1688 [========>.....................] - ETA: 6s - loss: 0.0275 - accuracy: 0.9916
 531/1688 [========>.....................] - ETA: 6s - loss: 0.0280 - accuracy: 0.9915
 540/1688 [========>.....................] - ETA: 6s - loss: 0.0278 - accuracy: 0.9914
 549/1688 [========>.....................] - ETA: 6s - loss: 0.0277 - accuracy: 0.9914
 558/1688 [========>.....................] - ETA: 6s - loss: 0.0276 - accuracy: 0.9913
 567/1688 [=========>....................] - ETA: 6s - loss: 0.0273 - accuracy: 0.9914
 576/1688 [=========>....................] - ETA: 6s - loss: 0.0270 - accuracy: 0.9915
 585/1688 [=========>....................] - ETA: 6s - loss: 0.0271 - accuracy: 0.9914
 594/1688 [=========>....................] - ETA: 6s - loss: 0.0271 - accuracy: 0.9914
 603/1688 [=========>....................] - ETA: 6s - loss: 0.0273 - accuracy: 0.9913
 612/1688 [=========>....................] - ETA: 6s - loss: 0.0271 - accuracy: 0.9914
 621/1688 [==========>...................] - ETA: 6s - loss: 0.0269 - accuracy: 0.9914
 630/1688 [==========>...................] - ETA: 6s - loss: 0.0273 - accuracy: 0.9912
 639/1688 [==========>...................] - ETA: 6s - loss: 0.0271 - accuracy: 0.9913
 648/1688 [==========>...................] - ETA: 6s - loss: 0.0273 - accuracy: 0.9913
 657/1688 [==========>...................] - ETA: 5s - loss: 0.0270 - accuracy: 0.9914
 666/1688 [==========>...................] - ETA: 5s - loss: 0.0272 - accuracy: 0.9913
 675/1688 [==========>...................] - ETA: 5s - loss: 0.0274 - accuracy: 0.9912
 684/1688 [===========>..................] - ETA: 5s - loss: 0.0278 - accuracy: 0.9910
 693/1688 [===========>..................] - ETA: 5s - loss: 0.0276 - accuracy: 0.9910
 702/1688 [===========>..................] - ETA: 5s - loss: 0.0274 - accuracy: 0.9911
 711/1688 [===========>..................] - ETA: 5s - loss: 0.0275 - accuracy: 0.9910
 720/1688 [===========>..................] - ETA: 5s - loss: 0.0278 - accuracy: 0.9909
 729/1688 [===========>..................] - ETA: 5s - loss: 0.0278 - accuracy: 0.9910
 738/1688 [============>.................] - ETA: 5s - loss: 0.0279 - accuracy: 0.9910
 747/1688 [============>.................] - ETA: 5s - loss: 0.0283 - accuracy: 0.9909
 756/1688 [============>.................] - ETA: 5s - loss: 0.0286 - accuracy: 0.9908
 765/1688 [============>.................] - ETA: 5s - loss: 0.0285 - accuracy: 0.9908
 774/1688 [============>.................] - ETA: 5s - loss: 0.0286 - accuracy: 0.9907
 783/1688 [============>.................] - ETA: 5s - loss: 0.0288 - accuracy: 0.9907
 792/1688 [=============>................] - ETA: 5s - loss: 0.0291 - accuracy: 0.9906
 801/1688 [=============>................] - ETA: 5s - loss: 0.0288 - accuracy: 0.9907
 810/1688 [=============>................] - ETA: 5s - loss: 0.0288 - accuracy: 0.9907
 819/1688 [=============>................] - ETA: 5s - loss: 0.0288 - accuracy: 0.9907
 828/1688 [=============>................] - ETA: 5s - loss: 0.0287 - accuracy: 0.9908
 837/1688 [=============>................] - ETA: 4s - loss: 0.0289 - accuracy: 0.9907
 846/1688 [==============>...............] - ETA: 4s - loss: 0.0288 - accuracy: 0.9907
 855/1688 [==============>...............] - ETA: 4s - loss: 0.0288 - accuracy: 0.9907
 864/1688 [==============>...............] - ETA: 4s - loss: 0.0287 - accuracy: 0.9907
 873/1688 [==============>...............] - ETA: 4s - loss: 0.0289 - accuracy: 0.9908
 882/1688 [==============>...............] - ETA: 4s - loss: 0.0289 - accuracy: 0.9908
 891/1688 [==============>...............] - ETA: 4s - loss: 0.0292 - accuracy: 0.9907
 900/1688 [==============>...............] - ETA: 4s - loss: 0.0290 - accuracy: 0.9907
 909/1688 [===============>..............] - ETA: 4s - loss: 0.0289 - accuracy: 0.9908
 918/1688 [===============>..............] - ETA: 4s - loss: 0.0291 - accuracy: 0.9907
 927/1688 [===============>..............] - ETA: 4s - loss: 0.0291 - accuracy: 0.9907
 936/1688 [===============>..............] - ETA: 4s - loss: 0.0291 - accuracy: 0.9907
 945/1688 [===============>..............] - ETA: 4s - loss: 0.0291 - accuracy: 0.9906
 954/1688 [===============>..............] - ETA: 4s - loss: 0.0297 - accuracy: 0.9905
 963/1688 [================>.............] - ETA: 4s - loss: 0.0299 - accuracy: 0.9904
 972/1688 [================>.............] - ETA: 4s - loss: 0.0302 - accuracy: 0.9904
 981/1688 [================>.............] - ETA: 4s - loss: 0.0300 - accuracy: 0.9905
 990/1688 [================>.............] - ETA: 4s - loss: 0.0302 - accuracy: 0.9904
 999/1688 [================>.............] - ETA: 4s - loss: 0.0303 - accuracy: 0.9903
1008/1688 [================>.............] - ETA: 3s - loss: 0.0302 - accuracy: 0.9904
1017/1688 [=================>............] - ETA: 3s - loss: 0.0301 - accuracy: 0.9904
1026/1688 [=================>............] - ETA: 3s - loss: 0.0300 - accuracy: 0.9904
1035/1688 [=================>............] - ETA: 3s - loss: 0.0299 - accuracy: 0.9905
1044/1688 [=================>............] - ETA: 3s - loss: 0.0300 - accuracy: 0.9904
1053/1688 [=================>............] - ETA: 3s - loss: 0.0301 - accuracy: 0.9903
1062/1688 [=================>............] - ETA: 3s - loss: 0.0302 - accuracy: 0.9903
1071/1688 [==================>...........] - ETA: 3s - loss: 0.0301 - accuracy: 0.9902
1080/1688 [==================>...........] - ETA: 3s - loss: 0.0300 - accuracy: 0.9902
1089/1688 [==================>...........] - ETA: 3s - loss: 0.0302 - accuracy: 0.9902
1098/1688 [==================>...........] - ETA: 3s - loss: 0.0303 - accuracy: 0.9902
1107/1688 [==================>...........] - ETA: 3s - loss: 0.0304 - accuracy: 0.9901
1116/1688 [==================>...........] - ETA: 3s - loss: 0.0307 - accuracy: 0.9901
1125/1688 [==================>...........] - ETA: 3s - loss: 0.0309 - accuracy: 0.9901
1134/1688 [===================>..........] - ETA: 3s - loss: 0.0312 - accuracy: 0.9901
1143/1688 [===================>..........] - ETA: 3s - loss: 0.0312 - accuracy: 0.9901
1152/1688 [===================>..........] - ETA: 3s - loss: 0.0312 - accuracy: 0.9901
1161/1688 [===================>..........] - ETA: 3s - loss: 0.0312 - accuracy: 0.9900
1170/1688 [===================>..........] - ETA: 3s - loss: 0.0312 - accuracy: 0.9900
1179/1688 [===================>..........] - ETA: 2s - loss: 0.0314 - accuracy: 0.9900
1188/1688 [====================>.........] - ETA: 2s - loss: 0.0313 - accuracy: 0.9900
1197/1688 [====================>.........] - ETA: 2s - loss: 0.0311 - accuracy: 0.9901
1206/1688 [====================>.........] - ETA: 2s - loss: 0.0313 - accuracy: 0.9900
1215/1688 [====================>.........] - ETA: 2s - loss: 0.0312 - accuracy: 0.9900
1224/1688 [====================>.........] - ETA: 2s - loss: 0.0312 - accuracy: 0.9900
1233/1688 [====================>.........] - ETA: 2s - loss: 0.0312 - accuracy: 0.9900
1242/1688 [=====================>........] - ETA: 2s - loss: 0.0315 - accuracy: 0.9900
1251/1688 [=====================>........] - ETA: 2s - loss: 0.0314 - accuracy: 0.9900
1260/1688 [=====================>........] - ETA: 2s - loss: 0.0313 - accuracy: 0.9900
1269/1688 [=====================>........] - ETA: 2s - loss: 0.0316 - accuracy: 0.9899
1278/1688 [=====================>........] - ETA: 2s - loss: 0.0318 - accuracy: 0.9898
1287/1688 [=====================>........] - ETA: 2s - loss: 0.0320 - accuracy: 0.9898
1296/1688 [======================>.......] - ETA: 2s - loss: 0.0321 - accuracy: 0.9898
1305/1688 [======================>.......] - ETA: 2s - loss: 0.0321 - accuracy: 0.9897
1314/1688 [======================>.......] - ETA: 2s - loss: 0.0321 - accuracy: 0.9897
1323/1688 [======================>.......] - ETA: 2s - loss: 0.0321 - accuracy: 0.9897
1332/1688 [======================>.......] - ETA: 2s - loss: 0.0328 - accuracy: 0.9896
1341/1688 [======================>.......] - ETA: 2s - loss: 0.0330 - accuracy: 0.9895
1350/1688 [======================>.......] - ETA: 1s - loss: 0.0331 - accuracy: 0.9895
1359/1688 [=======================>......] - ETA: 1s - loss: 0.0332 - accuracy: 0.9895
1368/1688 [=======================>......] - ETA: 1s - loss: 0.0333 - accuracy: 0.9895
1377/1688 [=======================>......] - ETA: 1s - loss: 0.0332 - accuracy: 0.9895
1386/1688 [=======================>......] - ETA: 1s - loss: 0.0334 - accuracy: 0.9895
1395/1688 [=======================>......] - ETA: 1s - loss: 0.0337 - accuracy: 0.9894
1404/1688 [=======================>......] - ETA: 1s - loss: 0.0338 - accuracy: 0.9894
1413/1688 [========================>.....] - ETA: 1s - loss: 0.0339 - accuracy: 0.9893
1422/1688 [========================>.....] - ETA: 1s - loss: 0.0341 - accuracy: 0.9893
1431/1688 [========================>.....] - ETA: 1s - loss: 0.0342 - accuracy: 0.9892
1440/1688 [========================>.....] - ETA: 1s - loss: 0.0342 - accuracy: 0.9892
1449/1688 [========================>.....] - ETA: 1s - loss: 0.0341 - accuracy: 0.9892
1458/1688 [========================>.....] - ETA: 1s - loss: 0.0342 - accuracy: 0.9892
1467/1688 [=========================>....] - ETA: 1s - loss: 0.0343 - accuracy: 0.9891
1476/1688 [=========================>....] - ETA: 1s - loss: 0.0344 - accuracy: 0.9891
1485/1688 [=========================>....] - ETA: 1s - loss: 0.0345 - accuracy: 0.9890
1494/1688 [=========================>....] - ETA: 1s - loss: 0.0347 - accuracy: 0.9890
1503/1688 [=========================>....] - ETA: 1s - loss: 0.0347 - accuracy: 0.9890
1512/1688 [=========================>....] - ETA: 1s - loss: 0.0346 - accuracy: 0.9890
1521/1688 [==========================>...] - ETA: 0s - loss: 0.0350 - accuracy: 0.9889
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0352 - accuracy: 0.9889
1539/1688 [==========================>...] - ETA: 0s - loss: 0.0354 - accuracy: 0.9888
1548/1688 [==========================>...] - ETA: 0s - loss: 0.0352 - accuracy: 0.9889
1557/1688 [==========================>...] - ETA: 0s - loss: 0.0352 - accuracy: 0.9889
1566/1688 [==========================>...] - ETA: 0s - loss: 0.0352 - accuracy: 0.9889
1575/1688 [==========================>...] - ETA: 0s - loss: 0.0353 - accuracy: 0.9889
1584/1688 [===========================>..] - ETA: 0s - loss: 0.0352 - accuracy: 0.9890
1593/1688 [===========================>..] - ETA: 0s - loss: 0.0352 - accuracy: 0.9890
1602/1688 [===========================>..] - ETA: 0s - loss: 0.0350 - accuracy: 0.9890
1611/1688 [===========================>..] - ETA: 0s - loss: 0.0354 - accuracy: 0.9889
1620/1688 [===========================>..] - ETA: 0s - loss: 0.0353 - accuracy: 0.9889
1629/1688 [===========================>..] - ETA: 0s - loss: 0.0353 - accuracy: 0.9890
1638/1688 [============================>.] - ETA: 0s - loss: 0.0352 - accuracy: 0.9890
1647/1688 [============================>.] - ETA: 0s - loss: 0.0351 - accuracy: 0.9890
1656/1688 [============================>.] - ETA: 0s - loss: 0.0352 - accuracy: 0.9890
1665/1688 [============================>.] - ETA: 0s - loss: 0.0351 - accuracy: 0.9890
1674/1688 [============================>.] - ETA: 0s - loss: 0.0351 - accuracy: 0.9890
1683/1688 [============================>.] - ETA: 0s - loss: 0.0353 - accuracy: 0.9889
1688/1688 [==============================] - 10s 6ms/step - loss: 0.0352 - accuracy: 0.9889 - val_loss: 0.0616 - val_accuracy: 0.9832
Test accuracy after fine tuning: 0.9851999878883362

6. Model conversion

After having obtained a quantized model with satisfactory performance, it can be converted to a model suitable to be used in the Akida NSoC in inference mode. The convert function returns a model in Akida format, ready for the Akida NSoC or the Akida Execution Engine.

Note

One needs to supply the coefficients used to rescale the input dataset before the training - here input_scaling.

As with Keras, the summary() method provides a textual representation of the Akida model.

from cnn2snn import convert

model_akida = convert(model_quantized, input_scaling=input_scaling)
model_akida.summary()

results = model_akida.predict(raw_x_test)
accuracy = (raw_y_test == results).mean()

print('Test accuracy after conversion:', accuracy)

# For non-regression purpose
assert accuracy > 0.97

Out:

                        Model Summary
_____________________________________________________________
Layer (type)                 Output shape  Kernel shape
=============================================================
conv2d (InputConvolutional)  [13, 13, 32]  (3, 3, 1, 32)
_____________________________________________________________
conv2d_1 (Convolutional)     [7, 7, 64]    (3, 3, 32, 64)
_____________________________________________________________
dense (FullyConnected)       [1, 1, 10]    (1, 1, 3136, 10)
_____________________________________________________________
Input shape: 28, 28, 1
Backend type: Software - 1.8.7


Test accuracy after conversion: 0.9853

Depending on the number of samples you run, you should find a performance of around 98% (better results can be achieved using more epochs for training).

Total running time of the script: ( 2 minutes 32.876 seconds)

Gallery generated by Sphinx-Gallery