Commit 19f41468 authored by Andrey Kutuzov's avatar Andrey Kutuzov
Browse files

Test script for benchmarking TensorFlow on CPU

parent e527ac1c
Starting job 1590175 on c1-25 at Sun Nov 29 18:02:51 CET 2020
The following modules were not unloaded:
(Use "module --force purge" to unload all):
1) StdEnv
2020-11-29 18:03:10.318731: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-11-29 18:03:49,486 : INFO : Loading the datasets...
2020-11-29 18:03:49,768 : INFO : Finished loading the datasets
2020-11-29 18:03:49,769 : INFO : 6557 train texts
2020-11-29 18:03:49,769 : INFO : 825 dev texts
2020-11-29 18:03:49,792 : INFO : Average train text length: 19.0 words
2020-11-29 18:03:49,795 : INFO : Average dev text length: 19.2 words
2020-11-29 18:03:49,795 : INFO : 2 classes
2020-11-29 18:03:49,795 : INFO : ===========================
2020-11-29 18:03:49,795 : INFO : Class distribution in the training data:
2020-11-29 18:03:49,827 : INFO : sent_id phrase_id tokens lemmatized
label
negative 3115 3115 3115 3115
positive 3442 3442 3442 3442
2020-11-29 18:03:49,833 : INFO : Class distribution in the development data:
2020-11-29 18:03:49,835 : INFO : sent_id phrase_id tokens lemmatized
label
negative 408 408 408 408
positive 417 417 417 417
2020-11-29 18:03:49,840 : INFO : ===========================
2020-11-29 18:03:49,876 : INFO : 16128 unique words
2020-11-29 18:03:49,876 : INFO : Vectorizing words...
2020-11-29 18:03:50,186 : INFO : Train labels shape: (6557, 2)
2020-11-29 18:03:50,186 : INFO : Dev labels shape: (825, 2)
WARNING:tensorflow:From /cluster/shared/nlpl/software/easybuild_ak/easybuild/install/software/NLPL-TensorFlow/1.15.2-gomkl-2019b-Python-3.7.4/lib/python3.7/site-packages/tensorflow_core/python/keras/initializers.py:119: calling RandomUniform.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
2020-11-29 18:03:50,187 : WARNING : From /cluster/shared/nlpl/software/easybuild_ak/easybuild/install/software/NLPL-TensorFlow/1.15.2-gomkl-2019b-Python-3.7.4/lib/python3.7/site-packages/tensorflow_core/python/keras/initializers.py:119: calling RandomUniform.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /cluster/shared/nlpl/software/easybuild_ak/easybuild/install/software/NLPL-TensorFlow/1.15.2-gomkl-2019b-Python-3.7.4/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
2020-11-29 18:03:50,202 : WARNING : From /cluster/shared/nlpl/software/easybuild_ak/easybuild/install/software/NLPL-TensorFlow/1.15.2-gomkl-2019b-Python-3.7.4/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
2020-11-29 18:03:50,620 : INFO : None
WARNING:tensorflow:OMP_NUM_THREADS is no longer used by the default Keras config. To configure the number of threads, use tf.config.threading APIs.
2020-11-29 18:03:51,081 : WARNING : OMP_NUM_THREADS is no longer used by the default Keras config. To configure the number of threads, use tf.config.threading APIs.
2020-11-29 18:03:51.116768: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /cluster/shared/nlpl/software/easybuild_ak/easybuild/install/software/NLPL-TensorFlow/1.15.2-gomkl-2019b-Python-3.7.4/lib:/cluster/software/NCCL/2.6.4-CUDA-10.1/lib:/cluster/software/HDF5/1.10.5-gompi-2019b/lib:/cluster/software/Szip/2.1.1-GCCcore-8.3.0/lib:/cluster/shared/nlpl/software/easybuild_ak/easybuild/install/software/NLPL-SciPy-bundle/2020.03-gomkl-2019b-Python-3.7.4/lib:/cluster/software/libyaml/0.2.2-GCCcore-8.3.0/lib:/cluster/software/Tkinter/3.7.4-GCCcore-8.3.0/lib:/cluster/software/Tk/8.6.9-GCCcore-8.3.0/lib:/cluster/software/X11/20190717-GCCcore-8.3.0/lib:/cluster/software/fontconfig/2.13.1-GCCcore-8.3.0/lib:/cluster/software/util-linux/2.34-GCCcore-8.3.0/lib:/cluster/software/expat/2.2.7-GCCcore-8.3.0/lib:/cluster/software/Tcl/8.6.9-GCCcore-8.3.0/lib:/cluster/software/freetype/2.10.1-GCCcore-8.3.0/lib:/cluster/software/libpng/1.6.37-GCCcore-8.3.0/lib:/cluster/software/Python/3.7.4-GCCcore-8.3.0/lib:/cluster/software/libffi/3.2.1-GCCcore-8.3.0/lib64:/cluster/software/libffi/3.2.1-GCCcore-8.3.0/lib:/cluster/software/GMP/6.1.2-GCCcore-8.3.0/lib:/cluster/software/XZ/5.2.4-GCCcore-8.3.0/lib:/cluster/software/libreadline/8.0-GCCcore-8.3.0/lib:/cluster/software/ncurses/6.1-GCCcore-8.3.0/lib:/cluster/software/bzip2/1.0.8-GCCcore-8.3.0/lib:/cluster/software/cuDNN/7.6.4.38/lib64:/cluster/software/CUDA/10.1.243/nvvm/lib64:/cluster/software/CUDA/10.1.243/extras/CUPTI/lib64:/cluster/software/CUDA/10.1.243/lib64:/cluster/shared/nlpl/software/easybuild_ak/easybuild/install/software/imkl/2019.1.144-gompi-2019b/mkl/lib/intel64:/cluster/shared/nlpl/software/easybuild_ak/easybuild/install/software/imkl/2019.1.144-gompi-2019b/lib/intel64:/cluster/software/OpenMPI/3.1.4-GCC-8.3.0/lib:/cluster/software/hpcx/2.4/sharp/lib:/cluster/software/hpcx/2.4/hcoll/lib:/cluster/software/hpcx/2.4/ucx/mt/lib:/cluster/software/binutils/2.32-GCCcore-8.3.0/lib:/cluster/software/zlib/1.2.11-GCCcore-8.3.0/lib:/cluster/software/GCCcore/8.3.0/lib/gcc/x86_64-pc-linux-gnu/8.3.0:/cluster/software/GCCcore/8.3.0/lib64:/cluster/software/GCCcore/8.3.0/lib
2020-11-29 18:03:51.116811: E tensorflow/stream_executor/cuda/cuda_driver.cc:318] failed call to cuInit: UNKNOWN ERROR (303)
2020-11-29 18:03:51.116992: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (c1-25): /proc/driver/nvidia/version does not exist
2020-11-29 18:03:51.142009: I tensorflow/core/common_runtime/process_util.cc:115] Creating new thread pool with default inter op setting: 16. Tune using inter_op_parallelism_threads for best performance.
Model: "model"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
Word_sequences (InputLayer) [(None, 50)] 0
__________________________________________________________________________________________________
Word_embeddings (Embedding) (None, 50, 50) 500050 Word_sequences[0][0]
__________________________________________________________________________________________________
conv1d (Conv1D) (None, 48, 100) 15100 Word_embeddings[0][0]
__________________________________________________________________________________________________
conv1d_1 (Conv1D) (None, 47, 100) 20100 Word_embeddings[0][0]
__________________________________________________________________________________________________
conv1d_2 (Conv1D) (None, 46, 100) 25100 Word_embeddings[0][0]
__________________________________________________________________________________________________
global_max_pooling1d (GlobalMax (None, 100) 0 conv1d[0][0]
__________________________________________________________________________________________________
global_max_pooling1d_1 (GlobalM (None, 100) 0 conv1d_1[0][0]
__________________________________________________________________________________________________
global_max_pooling1d_2 (GlobalM (None, 100) 0 conv1d_2[0][0]
__________________________________________________________________________________________________
concatenate (Concatenate) (None, 300) 0 global_max_pooling1d[0][0]
global_max_pooling1d_1[0][0]
global_max_pooling1d_2[0][0]
__________________________________________________________________________________________________
dropout (Dropout) (None, 300) 0 concatenate[0][0]
__________________________________________________________________________________________________
dense (Dense) (None, 200) 60200 dropout[0][0]
__________________________________________________________________________________________________
Output (Dense) (None, 2) 402 dense[0][0]
==================================================================================================
Total params: 620,952
Trainable params: 620,952
Non-trainable params: 0
__________________________________________________________________________________________________
Train on 6557 samples, validate on 825 samples
Epoch 1/20
6557/6557 - 4s - loss: 0.6209 - acc: 0.6344 - val_loss: 0.5015 - val_acc: 0.7564
Epoch 2/20
6557/6557 - 3s - loss: 0.3358 - acc: 0.8553 - val_loss: 0.4915 - val_acc: 0.7782
Epoch 3/20
6557/6557 - 3s - loss: 0.1332 - acc: 0.9524 - val_loss: 0.6720 - val_acc: 0.7576
Epoch 4/20
6557/6557 - 3s - loss: 0.0531 - acc: 0.9832 - val_loss: 0.9200 - val_acc: 0.7624
Epoch 5/20
6557/6557 - 3s - loss: 0.0331 - acc: 0.9890 - val_loss: 1.2654 - val_acc: 0.7612
Epoch 6/20
6557/6557 - 3s - loss: 0.0227 - acc: 0.9937 - val_loss: 1.2840 - val_acc: 0.7697
Epoch 7/20
6557/6557 - 2s - loss: 0.0235 - acc: 0.9908 - val_loss: 1.3331 - val_acc: 0.7685
Epoch 8/20
6557/6557 - 3s - loss: 0.0171 - acc: 0.9947 - val_loss: 1.6934 - val_acc: 0.7685
Epoch 9/20
6557/6557 - 3s - loss: 0.0174 - acc: 0.9950 - val_loss: 1.6789 - val_acc: 0.7515
Epoch 10/20
6557/6557 - 3s - loss: 0.0118 - acc: 0.9968 - val_loss: 1.6589 - val_acc: 0.7552
Epoch 11/20
6557/6557 - 3s - loss: 0.0073 - acc: 0.9980 - val_loss: 1.9871 - val_acc: 0.7564
Epoch 12/20
6557/6557 - 3s - loss: 0.0094 - acc: 0.9969 - val_loss: 2.0368 - val_acc: 0.7624
Epoch 13/20
6557/6557 - 3s - loss: 0.0073 - acc: 0.9977 - val_loss: 2.2841 - val_acc: 0.7624
Epoch 14/20
6557/6557 - 3s - loss: 0.0155 - acc: 0.9944 - val_loss: 1.6948 - val_acc: 0.7612
Epoch 15/20
6557/6557 - 3s - loss: 0.0078 - acc: 0.9977 - val_loss: 2.2416 - val_acc: 0.7527
Epoch 16/20
6557/6557 - 3s - loss: 0.0049 - acc: 0.9988 - val_loss: 2.4387 - val_acc: 0.7576
Epoch 17/20
6557/6557 - 3s - loss: 0.0027 - acc: 0.9994 - val_loss: 3.7951 - val_acc: 0.7382
Epoch 18/20
6557/6557 - 3s - loss: 0.0085 - acc: 0.9973 - val_loss: 2.5248 - val_acc: 0.7588
Epoch 19/20
6557/6557 - 3s - loss: 0.0074 - acc: 0.9977 - val_loss: 2.3628 - val_acc: 0.7539
Epoch 20/20
6557/6557 - 3s - loss: 0.0052 - acc: 0.9982 - val_loss: 2.4788 - val_acc: 0.7588
2020-11-29 18:04:45,791 : INFO : Training took 55 seconds
2020-11-29 18:04:45,791 : INFO : ===========================
2020-11-29 18:04:45,792 : INFO : Training is over! Evaluating.
2020-11-29 18:04:45,792 : INFO : ===========================
825/825 - 0s - loss: 2.4788 - acc: 0.7588
2020-11-29 18:04:45,887 : INFO : Validation loss: 2.4788
2020-11-29 18:04:45,887 : INFO : Validation accuracy: 0.7588000297546387
2020-11-29 18:04:46,105 : INFO : Classification report for the dev set (with weighted F1):
2020-11-29 18:04:46,115 : INFO : precision recall f1-score support
negative 0.75 0.76 0.76 408
positive 0.76 0.76 0.76 417
accuracy 0.76 825
macro avg 0.76 0.76 0.76 825
weighted avg 0.76 0.76 0.76 825
Task and CPU usage stats:
JobID JobName AllocCPUS NTasks MinCPU MinCPUTask AveCPU Elapsed ExitCode
------------ ---------- ---------- -------- ---------- ---------- ---------- ---------- --------
1590175 test_tf_c+ 8 00:01:57 0:0
1590175.bat+ batch 8 1 00:02:34 0 00:02:34 00:01:57 0:0
1590175.ext+ extern 8 1 00:00:00 0 00:00:00 00:01:57 0:0
Memory usage stats:
JobID MaxRSS MaxRSSTask AveRSS MaxPages MaxPagesTask AvePages
------------ ---------- ---------- ---------- -------- -------------- ----------
1590175
1590175.bat+ 478708K 0 478708K 1914 0 1914
1590175.ext+ 0 0 0 0 0 0
Disk usage stats:
JobID MaxDiskRead MaxDiskReadTask AveDiskRead MaxDiskWrite MaxDiskWriteTask AveDiskWrite
------------ ------------ --------------- -------------- ------------ ---------------- --------------
1590175
1590175.bat+ 9.79M 0 9.79M 0.06M 0 0.06M
1590175.ext+ 0.00M 0 0.00M 0 0 0
Job 1590175 completed at Sun Nov 29 18:04:48 CET 2020
Starting job 1590174 on c1-25 at Sun Nov 29 17:59:38 CET 2020
The following modules were not unloaded:
(Use "module --force purge" to unload all):
1) StdEnv
2020-11-29 17:59:57.100170: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-11-29 18:00:35,622 : INFO : Loading the datasets...
2020-11-29 18:00:36,036 : INFO : Finished loading the datasets
2020-11-29 18:00:36,037 : INFO : 6557 train texts
2020-11-29 18:00:36,037 : INFO : 825 dev texts
2020-11-29 18:00:36,060 : INFO : Average train text length: 19.0 words
2020-11-29 18:00:36,063 : INFO : Average dev text length: 19.2 words
2020-11-29 18:00:36,063 : INFO : 2 classes
2020-11-29 18:00:36,063 : INFO : ===========================
2020-11-29 18:00:36,063 : INFO : Class distribution in the training data:
2020-11-29 18:00:36,118 : INFO : sent_id phrase_id tokens lemmatized
label
negative 3115 3115 3115 3115
positive 3442 3442 3442 3442
2020-11-29 18:00:36,124 : INFO : Class distribution in the development data:
2020-11-29 18:00:36,126 : INFO : sent_id phrase_id tokens lemmatized
label
negative 408 408 408 408
positive 417 417 417 417
2020-11-29 18:00:36,131 : INFO : ===========================
2020-11-29 18:00:36,166 : INFO : 16128 unique words
2020-11-29 18:00:36,167 : INFO : Vectorizing words...
2020-11-29 18:00:36,475 : INFO : Train labels shape: (6557, 2)
2020-11-29 18:00:36,475 : INFO : Dev labels shape: (825, 2)
WARNING:tensorflow:From /cluster/shared/nlpl/software/easybuild_ak/easybuild/install/software/NLPL-TensorFlow/1.15.2-foss-2019b-Python-3.7.4/lib/python3.7/site-packages/tensorflow_core/python/keras/initializers.py:119: calling RandomUniform.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
2020-11-29 18:00:36,476 : WARNING : From /cluster/shared/nlpl/software/easybuild_ak/easybuild/install/software/NLPL-TensorFlow/1.15.2-foss-2019b-Python-3.7.4/lib/python3.7/site-packages/tensorflow_core/python/keras/initializers.py:119: calling RandomUniform.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /cluster/shared/nlpl/software/easybuild_ak/easybuild/install/software/NLPL-TensorFlow/1.15.2-foss-2019b-Python-3.7.4/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
2020-11-29 18:00:36,492 : WARNING : From /cluster/shared/nlpl/software/easybuild_ak/easybuild/install/software/NLPL-TensorFlow/1.15.2-foss-2019b-Python-3.7.4/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
2020-11-29 18:00:36,874 : INFO : None
WARNING:tensorflow:OMP_NUM_THREADS is no longer used by the default Keras config. To configure the number of threads, use tf.config.threading APIs.
2020-11-29 18:00:37,404 : WARNING : OMP_NUM_THREADS is no longer used by the default Keras config. To configure the number of threads, use tf.config.threading APIs.
2020-11-29 18:00:37.431765: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /cluster/shared/nlpl/software/easybuild_ak/easybuild/install/software/NLPL-TensorFlow/1.15.2-foss-2019b-Python-3.7.4/lib:/cluster/software/NCCL/2.6.4-CUDA-10.1/lib:/cluster/software/HDF5/1.10.5-gompi-2019b/lib:/cluster/software/Szip/2.1.1-GCCcore-8.3.0/lib:/cluster/shared/nlpl/software/easybuild_ak/easybuild/install/software/NLPL-SciPy-bundle/2020.03-foss-2019b-Python-3.7.4/lib:/cluster/software/libyaml/0.2.2-GCCcore-8.3.0/lib:/cluster/software/Tkinter/3.7.4-GCCcore-8.3.0/lib:/cluster/software/Tk/8.6.9-GCCcore-8.3.0/lib:/cluster/software/X11/20190717-GCCcore-8.3.0/lib:/cluster/software/fontconfig/2.13.1-GCCcore-8.3.0/lib:/cluster/software/util-linux/2.34-GCCcore-8.3.0/lib:/cluster/software/expat/2.2.7-GCCcore-8.3.0/lib:/cluster/software/Tcl/8.6.9-GCCcore-8.3.0/lib:/cluster/software/freetype/2.10.1-GCCcore-8.3.0/lib:/cluster/software/libpng/1.6.37-GCCcore-8.3.0/lib:/cluster/software/Python/3.7.4-GCCcore-8.3.0/lib:/cluster/software/libffi/3.2.1-GCCcore-8.3.0/lib64:/cluster/software/libffi/3.2.1-GCCcore-8.3.0/lib:/cluster/software/GMP/6.1.2-GCCcore-8.3.0/lib:/cluster/software/XZ/5.2.4-GCCcore-8.3.0/lib:/cluster/software/libreadline/8.0-GCCcore-8.3.0/lib:/cluster/software/ncurses/6.1-GCCcore-8.3.0/lib:/cluster/software/bzip2/1.0.8-GCCcore-8.3.0/lib:/cluster/software/cuDNN/7.6.4.38/lib64:/cluster/software/CUDA/10.1.243/nvvm/lib64:/cluster/software/CUDA/10.1.243/extras/CUPTI/lib64:/cluster/software/CUDA/10.1.243/lib64:/cluster/software/ScaLAPACK/2.0.2-gompi-2019b/lib:/cluster/software/FFTW/3.3.8-gompi-2019b/lib:/cluster/software/OpenBLAS/0.3.7-GCC-8.3.0/lib:/cluster/software/OpenMPI/3.1.4-GCC-8.3.0/lib:/cluster/software/hpcx/2.4/sharp/lib:/cluster/software/hpcx/2.4/hcoll/lib:/cluster/software/hpcx/2.4/ucx/mt/lib:/cluster/software/binutils/2.32-GCCcore-8.3.0/lib:/cluster/software/zlib/1.2.11-GCCcore-8.3.0/lib:/cluster/software/GCCcore/8.3.0/lib/gcc/x86_64-pc-linux-gnu/8.3.0:/cluster/software/GCCcore/8.3.0/lib64:/cluster/software/GCCcore/8.3.0/lib
2020-11-29 18:00:37.431818: E tensorflow/stream_executor/cuda/cuda_driver.cc:318] failed call to cuInit: UNKNOWN ERROR (303)
2020-11-29 18:00:37.432056: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (c1-25): /proc/driver/nvidia/version does not exist
2020-11-29 18:00:37.440962: I tensorflow/core/common_runtime/process_util.cc:115] Creating new thread pool with default inter op setting: 16. Tune using inter_op_parallelism_threads for best performance.
Model: "model"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
Word_sequences (InputLayer) [(None, 50)] 0
__________________________________________________________________________________________________
Word_embeddings (Embedding) (None, 50, 50) 500050 Word_sequences[0][0]
__________________________________________________________________________________________________
conv1d (Conv1D) (None, 48, 100) 15100 Word_embeddings[0][0]
__________________________________________________________________________________________________
conv1d_1 (Conv1D) (None, 47, 100) 20100 Word_embeddings[0][0]
__________________________________________________________________________________________________
conv1d_2 (Conv1D) (None, 46, 100) 25100 Word_embeddings[0][0]
__________________________________________________________________________________________________
global_max_pooling1d (GlobalMax (None, 100) 0 conv1d[0][0]
__________________________________________________________________________________________________
global_max_pooling1d_1 (GlobalM (None, 100) 0 conv1d_1[0][0]
__________________________________________________________________________________________________
global_max_pooling1d_2 (GlobalM (None, 100) 0 conv1d_2[0][0]
__________________________________________________________________________________________________
concatenate (Concatenate) (None, 300) 0 global_max_pooling1d[0][0]
global_max_pooling1d_1[0][0]
global_max_pooling1d_2[0][0]
__________________________________________________________________________________________________
dropout (Dropout) (None, 300) 0 concatenate[0][0]
__________________________________________________________________________________________________
dense (Dense) (None, 200) 60200 dropout[0][0]
__________________________________________________________________________________________________
Output (Dense) (None, 2) 402 dense[0][0]
==================================================================================================
Total params: 620,952
Trainable params: 620,952
Non-trainable params: 0
__________________________________________________________________________________________________
Train on 6557 samples, validate on 825 samples
Epoch 1/20
6557/6557 - 4s - loss: 0.6209 - acc: 0.6344 - val_loss: 0.5015 - val_acc: 0.7564
Epoch 2/20
6557/6557 - 3s - loss: 0.3358 - acc: 0.8553 - val_loss: 0.4915 - val_acc: 0.7782
Epoch 3/20
6557/6557 - 3s - loss: 0.1332 - acc: 0.9524 - val_loss: 0.6720 - val_acc: 0.7576
Epoch 4/20
6557/6557 - 3s - loss: 0.0531 - acc: 0.9832 - val_loss: 0.9200 - val_acc: 0.7624
Epoch 5/20
6557/6557 - 3s - loss: 0.0331 - acc: 0.9890 - val_loss: 1.2654 - val_acc: 0.7612
Epoch 6/20
6557/6557 - 3s - loss: 0.0227 - acc: 0.9937 - val_loss: 1.2840 - val_acc: 0.7697
Epoch 7/20
6557/6557 - 3s - loss: 0.0235 - acc: 0.9908 - val_loss: 1.3331 - val_acc: 0.7685
Epoch 8/20
6557/6557 - 3s - loss: 0.0171 - acc: 0.9947 - val_loss: 1.6934 - val_acc: 0.7685
Epoch 9/20
6557/6557 - 3s - loss: 0.0174 - acc: 0.9950 - val_loss: 1.6789 - val_acc: 0.7515
Epoch 10/20
6557/6557 - 3s - loss: 0.0118 - acc: 0.9968 - val_loss: 1.6589 - val_acc: 0.7552
Epoch 11/20
6557/6557 - 3s - loss: 0.0073 - acc: 0.9980 - val_loss: 1.9871 - val_acc: 0.7564
Epoch 12/20
6557/6557 - 3s - loss: 0.0094 - acc: 0.9969 - val_loss: 2.0368 - val_acc: 0.7624
Epoch 13/20
6557/6557 - 3s - loss: 0.0073 - acc: 0.9977 - val_loss: 2.2841 - val_acc: 0.7624
Epoch 14/20
6557/6557 - 2s - loss: 0.0155 - acc: 0.9944 - val_loss: 1.6948 - val_acc: 0.7612
Epoch 15/20
6557/6557 - 3s - loss: 0.0078 - acc: 0.9977 - val_loss: 2.2416 - val_acc: 0.7527
Epoch 16/20
6557/6557 - 3s - loss: 0.0049 - acc: 0.9988 - val_loss: 2.4387 - val_acc: 0.7576
Epoch 17/20
6557/6557 - 3s - loss: 0.0027 - acc: 0.9994 - val_loss: 3.7951 - val_acc: 0.7382
Epoch 18/20
6557/6557 - 3s - loss: 0.0085 - acc: 0.9973 - val_loss: 2.5248 - val_acc: 0.7588
Epoch 19/20
6557/6557 - 3s - loss: 0.0074 - acc: 0.9977 - val_loss: 2.3628 - val_acc: 0.7539
Epoch 20/20
6557/6557 - 3s - loss: 0.0052 - acc: 0.9982 - val_loss: 2.4788 - val_acc: 0.7588
2020-11-29 18:01:32,196 : INFO : Training took 55 seconds
2020-11-29 18:01:32,196 : INFO : ===========================
2020-11-29 18:01:32,196 : INFO : Training is over! Evaluating.
2020-11-29 18:01:32,196 : INFO : ===========================
825/825 - 0s - loss: 2.4788 - acc: 0.7588
2020-11-29 18:01:32,278 : INFO : Validation loss: 2.4788
2020-11-29 18:01:32,279 : INFO : Validation accuracy: 0.7588000297546387
2020-11-29 18:01:32,528 : INFO : Classification report for the dev set (with weighted F1):
2020-11-29 18:01:32,539 : INFO : precision recall f1-score support
negative 0.75 0.76 0.76 408
positive 0.76 0.76 0.76 417
accuracy 0.76 825
macro avg 0.76 0.76 0.76 825
weighted avg 0.76 0.76 0.76 825
Task and CPU usage stats:
JobID JobName AllocCPUS NTasks MinCPU MinCPUTask AveCPU Elapsed ExitCode
------------ ---------- ---------- -------- ---------- ---------- ---------- ---------- --------
1590174 test_tf_c+ 8 00:01:57 0:0
1590174.bat+ batch 8 1 00:02:39 0 00:02:39 00:01:57 0:0
1590174.ext+ extern 8 1 00:00:00 0 00:00:00 00:01:57 0:0
Memory usage stats:
JobID MaxRSS MaxRSSTask AveRSS MaxPages MaxPagesTask AvePages
------------ ---------- ---------- ---------- -------- -------------- ----------
1590174
1590174.bat+ 450700K 0 450700K 1800 0 1800
1590174.ext+ 0 0 0 0 0 0
Disk usage stats:
JobID MaxDiskRead MaxDiskReadTask AveDiskRead MaxDiskWrite MaxDiskWriteTask AveDiskWrite
------------ ------------ --------------- -------------- ------------ ---------------- --------------
1590174
1590174.bat+ 9.79M 0 9.79M 0.06M 0 0.06M
1590174.ext+ 0.00M 0 0.00M 0 0 0
Job 1590174 completed at Sun Nov 29 18:01:35 CET 2020
#!/bin/bash
#SBATCH --job-name=test_tf_cnn
#SBATCH --mail-type=FAIL
#SBATCH --account=nn9447k
#SBATCH --time=1:00:00 # Max walltime is 14 days.
#SBATCH --mem-per-cpu=4G
#SBATCH --ntasks=8
#SBATCH --nodes=1
set -o errexit # Recommended for easier debugging
# Replace with your module repository directory:
module use -a /cluster/shared/nlpl/software/easybuild_ak/easybuild/install/modules/all/
module purge
# module load NLPL-TensorFlow/1.15.2-foss-2019b-Python-3.7.4
# or...
module load NLPL-TensorFlow/1.15.2-gomkl-2019b-Python-3.7.4
python3 tensorflow_text_cnn.py -t SST/stanford_sentiment_binary_train.tsv.gz -d SST/stanford_sentiment_binary_dev.tsv.gz
\ No newline at end of file
#! /bin/env/ python3
# coding: utf-8
import argparse
import logging
import random
import time
import numpy as np
import pandas as pd
import tensorflow as tf
from sklearn.metrics import classification_report
from tensorflow.keras import backend
from tensorflow.keras import preprocessing
from tensorflow.keras.callbacks import TensorBoard
from tensorflow.keras.layers import Dense, Input, Conv1D, GlobalMaxPooling1D, Dropout, Embedding, \
concatenate
from tensorflow.keras.models import Model
from tensorflow.keras.utils import to_categorical
if __name__ == '__main__':
logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s', level=logging.INFO)
logger = logging.getLogger(__name__)
parser = argparse.ArgumentParser()
arg = parser.add_argument
arg('--trainfile', '-t', help='Annotated training dataset', required=True)
arg('--devfile', '-d', help='Annotated development dataset', required=True)
arg('--name', help='Human-readable name of the run.', default='testrun')
args = parser.parse_args()
trainfile = args.trainfile
devfile = args.devfile
run_name = args.name
# Fix random seeds for repeatability of experiments:
random.seed(42)
np.random.seed(42)
tf.compat.v1.set_random_seed(42)
seqlen = 50 # Maximum sentence length for padding
# We don't need a separate dev/validation dataset
# since it is created automatically by Keras from the train set.
logger.info('Loading the datasets...')
train_dataset = pd.read_csv(trainfile, sep='\t', header=0, compression="gzip")
dev_dataset = pd.read_csv(devfile, sep='\t', header=0, compression="gzip")
logger.info('Finished loading the datasets')
(x_train, x_tags_train, y_train) = \
train_dataset['tokens'], train_dataset['lemmatized'], train_dataset['label']
(x_dev, x_tags_dev, y_dev) = \
dev_dataset['tokens'], dev_dataset['lemmatized'], dev_dataset['label']
logger.info('%d train texts' % len(x_train))
logger.info('%d dev texts' % len(x_dev))
logger.info('Average train text length: {0:.{1}f} words'.
format(np.mean(list(map(len, x_train.str.split()))), 1))
logger.info('Average dev text length: {0:.{1}f} words'.
format(np.mean(list(map(len, x_dev.str.split()))), 1))
classes = sorted(list(set(y_train)))
num_classes = len(classes)
logger.info('%d classes' % num_classes)
logger.info('===========================')
logger.info('Class distribution in the training data:')
logger.info(train_dataset.groupby('label').count())
logger.info('Class distribution in the development data:')
logger.info(dev_dataset.groupby('label').count())
logger.info('===========================')
allwords = [el for sublist in x_tags_train for el in sublist.split()]
allwords_set = sorted(list(set(allwords)))
num_words = len(allwords_set)
logger.info('%d unique words' % num_words)
word_emb_size = 50
max_vocab_words = 10000
logger.info('Vectorizing words...')
tokenizer = preprocessing.text.Tokenizer(
num_words=max_vocab_words, filters='', lower=True, oov_token="UNKN")
tokenizer.fit_on_texts(x_tags_train)
vectorized_tags_train = tokenizer.texts_to_sequences(x_tags_train)
vectorized_tags_dev = tokenizer.texts_to_sequences(x_tags_dev)
# Padding:
vectorized_tags_train = preprocessing.sequence.pad_sequences(
vectorized_tags_train, maxlen=seqlen, truncating='post', padding='post')
vectorized_tags_dev = preprocessing.sequence.pad_sequences(
vectorized_tags_dev, maxlen=seqlen, truncating='post', padding='post')
# Converting text labels to indexes
y_train = [classes.index(i) for i in y_train]
y_dev = [classes.index(i) for i in y_dev]
# Convert indexes to binary class matrix (for use with categorical_crossentropy loss)
y_train = to_categorical(y_train, num_classes)
y_dev = to_categorical(y_dev, num_classes)
logger.info(f'Train labels shape: {y_train.shape}')
logger.info(f'Dev labels shape: {y_dev.shape}')
# Start building model
tag_embedding_layer = Embedding(max_vocab_words + 1, word_emb_size, name='Word_embeddings')
tags_input = Input(shape=(seqlen,), name='Word_sequences')
# Words in a sequence are transformed into their embeddings:
tags_embedded = tag_embedding_layer(tags_input)
filter_sizes = [3, 4, 5] # Window sizes for convolution filters
tag_convolutions = []
for filter_len in filter_sizes:
# Words convolutions:
conv = Conv1D(100, filter_len, activation='relu')(tags_embedded)
conv = GlobalMaxPooling1D()(conv)
tag_convolutions.append(conv)
training = concatenate(tag_convolutions)
training = Dropout(0.4)(training)
training = Dense(200, activation='relu')(training)
output = Dense(num_classes, activation='softmax', name='Output')(training) # Output
model = Model(inputs=[tags_input], outputs=[output]) # Keras Functional model
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
# Print out the model architecture
logger.info(model.summary())
loss_plot = TensorBoard(log_dir='logs/%s' % run_name)
# Train the compiled model on the training data
# See more at https://keras.io/models/sequential/#sequential-model-methods
start = time.time()
history = model.fit([vectorized_tags_train], y_train, epochs=20, verbose=2,
batch_size=16, validation_data=([vectorized_tags_dev], y_dev))
end = time.time()
training_time = int(end - start)
logger.info(f'Training took {training_time} seconds')
logger.info('===========================')
logger.info('Training is over! Evaluating.')
logger.info('===========================')
score = model.evaluate([vectorized_tags_dev], y_dev, verbose=2)
logger.info(f'Validation loss: {round(score[0], 4)}')
logger.info(f'Validation accuracy: {round(score[1], 4)}')
# We use sklearn function classification_report() to calculate per-class F1 score:
predictions = model.predict([vectorized_tags_dev])
# Convert predictions from integers back to text labels:
y_test_real = [classes[np.argmax(pred)] for pred in y_dev]
predictions = [classes[np.argmax(pred)] for pred in predictions]
logger.info('Classification report for the dev set (with weighted F1):')
logger.info(classification_report(y_test_real, predictions))
# Saving the model to file
# model_filename = run_name + '_' + str(i) + '.h5'
# model.save(model_filename)
# print('Model saved to', model_filename)
backend.clear_session()
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment