(二十八)逐轮训练、选择与保存
1. 一轮训练之后再验证
# 续写同一文件:training_project/src/readingtorch/engine.py
# 先完成一轮训练,再只用验证组决定是否替换最佳快照。
for epoch in range(start, epochs):
training = one_epoch(
model, train_loader, loss_fn, target_device, task, scale, optimizer
)
validation = one_epoch(model, val_loader, loss_fn, target_device, task, scale)
history.append({"epoch": epoch + 1, "train": training, "val": validation})
if validation["loss"] < best_score:
best_score, best_epoch, best_state = (
validation["loss"],
epoch + 1,
snapshot(model),
)
# last 保存继续训练所需状态,best 保存验证集选中的推理权重。
common = {
"format": 1,
"task": task,
"seed": seed,
"preprocess": bundle.preprocess,
"source_hash": bundle.source_hash,
"splits": bundle.splits,
"classes": ["竖线", "横线", "十字"] if task == "images" else [],
}训练组每批更新参数,随后验证组只读取这一轮的模型。若验证损失更小,就复制当前权重,并记住对应轮数。类别名称也保存进检查点,恢复预测时按同一份“编号到名称”映射解释输出。
2. last 与 best 保存不同结果
# 续写同一文件:training_project/src/readingtorch/engine.py
# 本轮结束时把优化器、随机状态与批次状态一并写入 last。
save_checkpoint(
output / "last.pt",
{
**common,
"kind": "training",
"config": config,
"epoch": epoch + 1,
"model": snapshot(model),
"optimizer": optimizer.state_dict(),
"history": history,
"best_score": best_score,
"best_epoch": best_epoch,
"best_model": best_state,
"torch_rng": torch.get_rng_state(),
"loader_rng": generator.get_state(),
"cuda_rng": torch.cuda.get_rng_state_all() if device == "cuda" else [],
},
)
save_checkpoint(
output / "best.pt",
{
**common,
"kind": "inference",
"epoch": best_epoch,
"model": best_state,
},
)last.pt 包含本轮结束的完整训练状态,用于继续;best.pt 保存截至本轮验证集选中的权重与输入信息,用于预测。若第十二轮最好、继续训练到第十八轮,两个文件里的模型可能不同。
3. 输出可读的训练历史
# 续写同一文件:training_project/src/readingtorch/engine.py
# 历史只写普通数值,绘图时不再持有训练计算图。
report = {
"task": task,
"seed": seed,
"device": device,
"epochs": epochs,
"best_epoch": best_epoch,
"best_val_loss": best_score,
"counts": {name: len(ids) for name, ids in bundle.splits.items()},
"history": history,
"torch_version": torch.__version__,
}
(output / "history.json").write_text(
json.dumps(report, ensure_ascii=False, indent=2), encoding="utf-8"
)
return reporthistory.json 记录每轮的训练与验证指标、实际版本、种子和设备。里面没有张量计算图,画曲线只需读取这份文件。