root@dsw-398300-6795fbc6cd-6752k:/mnt/workspace/bert-bilstm-in-sentiment-classification# python BiLSTM.py
2024-07-21 20:42:10.214152: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-07-21 20:42:10.251708: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-07-21 20:42:10.870408: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
test++++++++++A
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
Some weights of the model checkpoint at ./model/bert-base-chinese were not used when initializing BertModel: ['cls.seq_relationship.bias', 'cls.predictions.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.seq_relationship.weight', 'cls.predictions.transform.dense.weight']
- This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
test+_++++++++++++B
/opt/conda/lib/python3.8/site-packages/pytorch_lightning/callbacks/model_checkpoint.py:616: UserWarning: Checkpoint directory /mnt/workspace/bert-bilstm-in-sentiment-classification/model/checkpoints exists and is not empty.
rank_zero_warn(f"Checkpoint directory {dirpath} exists and is not empty.")
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
| Name | Type | Params
-----------------------------------------------
0 | model | BiLSTMClassifier | 125 M
1 | criterion | CrossEntropyLoss | 0
-----------------------------------------------
23.6 M Trainable params
102 M Non-trainable params
125 M Total params
503.553 Total estimated model params size (MB)
/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:236: PossibleUserWarning: The dataloader, train_dataloader, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 8 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
rank_zero_warn(
Epoch 4: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 487/487 [01:30<00:00, 5.38it/s, loss=0.171, v_num=8, train_loss_step=0.128, train_loss_epoch=0.185]`Trainer.fit` stopped: `max_epochs=5` reached.
Epoch 4: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 487/487 [01:30<00:00, 5.38it/s, loss=0.171, v_num=8, train_loss_step=0.128, train_loss_epoch=0.185]
GPU available: True (cuda), used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py:1764: PossibleUserWarning: GPU available but not used. Set `accelerator` and `devices` using `Trainer(accelerator='gpu', devices=1)`.
rank_zero_warn(
Testing DataLoader 0: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 98/98 [06:36<00:00, 4.05s/it]
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Test metric ┃ DataLoader 0 ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ acc │ 0.9047619104385376 │
│ avg_f1 │ 0.9017320871353149 │
│ avg_precision │ 0.9123411178588867 │
│ avg_recall │ 0.9047619104385376 │
│ f1_class0 │ 0.826066792011261 │
│ f1_class1 │ 0.9289177060127258 │
│ loss │ 0.2378724366426468 │
│ precision_class0 │ 0.8918467164039612 │
│ precision_class1 │ 0.9114090204238892 │
│ recall_class0 │ 0.7939081788063049 │
│ recall_class1 │ 0.9530112743377686 │
└───────────────────────────┴───────────────────────────┘
[{'loss': 0.2378724366426468, 'recall_class0': 0.7939081788063049, 'recall_class1': 0.9530112743377686, 'precision_class0': 0.8918467164039612, 'precision_class1': 0.9114090204238892, 'f1_class0': 0.826066792011261, 'f1_class1': 0.9289177060127258, 'acc': 0.9047619104385376, 'avg_recall': 0.9047619104385376, 'avg_precision': 0.9123411178588867, 'avg_f1': 0.9017320871353149}]
root@dsw-398300-6795fbc6cd-6752k:/mnt/workspace/bert-bilstm-in-sentiment-classification#