Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ATEPC训练指标显示不全 #411

Open
zwc-wc opened this issue Feb 28, 2025 · 0 comments
Open

ATEPC训练指标显示不全 #411

zwc-wc opened this issue Feb 28, 2025 · 0 comments
Labels
bug Something isn't working

Comments

@zwc-wc
Copy link

zwc-wc commented Feb 28, 2025

代码:
from pyabsa import DatasetItem, ModelSaveOption, DeviceTypeOption
import warnings

忽略警告信息

warnings.filterwarnings("ignore")

from pyabsa.framework.trainer_class.trainer_template import Trainer
from pyabsa.tasks.AspectTermExtraction.configuration.atepc_configuration import ATEPCConfigManager
from autocuda import auto_cuda

config = ATEPCConfigManager.get_atepc_config_english()
dataset = '137.all'

dataset = '100.CustomDataset'

添加sklearn的评估指标

from sklearn.metrics import precision_score, recall_score, f1_score, accuracy_score
import numpy as np

config.show_metric=True, #启用这个可以展示更多指标?
config.output_dim=3, #据说这个是方面分类的个数?
config.cross_validate_fold = 5 # 5折交叉验证
config.max_seq_length=512
config.batch_size = 8
config.learning_rate = 1e-5 # 降低学习率(当前为2e-5)
config.l2reg = 1e-4 # 增强L2正则化(当前为1e-5)
config.early_stop_by = "ate_f1" # 默认可能监控APC_ACC,改为监控ATE_F1
config.max_grad_norm = 1.0 # 防止梯度爆炸
config.use_crf = True # 改善序列标注的连贯性
config.patience = 2
config.log_step = -1
config.seed = [1]
config.verbose = False # If verbose == True, PyABSA will output the model strcture and seversal processed data examples
config.notice = (
"This is an training example for aspect term extraction" # for memos usage
)
config.checkpoint_save_mode = ModelSaveOption.SAVE_FULL_MODEL # 保存完整模型

trainer = ATEPC.ATEPCTrainer(
config=config,
dataset=dataset,
auto_device=DeviceTypeOption.AUTO, # use cuda if available
checkpoint_save_mode=ModelSaveOption.SAVE_FULL_MODEL, # save state dict only instead of the whole model
load_aug=False, # there are some augmentation dataset for integrated datasets, you use them by setting load_aug=True to improve performance
)

问题:

  1. 指标显示不全,如Epoch: 0| loss_apc:1.3189 | loss_ate:0.0577 |: 100%|█| 613/613 [06:13<00:00, 1.64it/s, APC_ACC: 84.49(max:84.49) | A
    Epoch: 1 | Smooth Loss: 0.7048: 100%|██████████████████████████████████████████████▉| 612/613 [05:39<00:00, 1.78it/s]
    根据教程,每个Epoch都应该显示Epoch: 0| loss_apc:0.0099 | loss_ate:0.0139 |: 100%|██████████| 226/226 [01:02<00:00, 3.59it/s, APC_ACC: 85.96(max:87.21) | APC_F1: 78.04(max:81.65) | ATE_F1: 82.87(max:84.58)]
    但是这里APC_F1和ATE_F1都没有显示出来,如图:
Image
  1. 为什么模型只能输出ATE Classification Report,而没有输出APC Classification Report的表格呢?应该怎样设置?我想知道APC任务的各项指标。如图:
Image
  1. 我曾设置了多个随机种子进行训练,如[1,2,3]但每个随机种子训练后似乎是独立输出指标,也就是说,不管连续训练几次,Std和IQR的值都是0,和教程中的情况不符,请问应该怎样设置,才能使这两个值正常计算!

您的解答将对我非常有帮助,提前感谢!

@zwc-wc zwc-wc added the bug Something isn't working label Feb 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant