Python 现代开发学习笔记

(二十八)逐轮训练、选择与保存

1. 一轮训练之后再验证

# 续写同一文件:training_project/src/readingtorch/engine.py
    # 先完成一轮训练,再只用验证组决定是否替换最佳快照。
    for epoch in range(start, epochs):
        training = one_epoch(
            model, train_loader, loss_fn, target_device, task, scale, optimizer
        )
        validation = one_epoch(model, val_loader, loss_fn, target_device, task, scale)
        history.append({"epoch": epoch + 1, "train": training, "val": validation})
        if validation["loss"] < best_score:
            best_score, best_epoch, best_state = (
                validation["loss"],
                epoch + 1,
                snapshot(model),
            )
        # last 保存继续训练所需状态,best 保存验证集选中的推理权重。
        common = {
            "format": 1,
            "task": task,
            "seed": seed,
            "preprocess": bundle.preprocess,
            "source_hash": bundle.source_hash,
            "splits": bundle.splits,
            "classes": ["竖线", "横线", "十字"] if task == "images" else [],
        }

训练组每批更新参数,随后验证组只读取这一轮的模型。若验证损失更小,就复制当前权重,并记住对应轮数。类别名称也保存进检查点,恢复预测时按同一份“编号到名称”映射解释输出。

2. last 与 best 保存不同结果

# 续写同一文件:training_project/src/readingtorch/engine.py
        # 本轮结束时把优化器、随机状态与批次状态一并写入 last。
        save_checkpoint(
            output / "last.pt",
            {
                **common,
                "kind": "training",
                "config": config,
                "epoch": epoch + 1,
                "model": snapshot(model),
                "optimizer": optimizer.state_dict(),
                "history": history,
                "best_score": best_score,
                "best_epoch": best_epoch,
                "best_model": best_state,
                "torch_rng": torch.get_rng_state(),
                "loader_rng": generator.get_state(),
                "cuda_rng": torch.cuda.get_rng_state_all() if device == "cuda" else [],
            },
        )
        save_checkpoint(
            output / "best.pt",
            {
                **common,
                "kind": "inference",
                "epoch": best_epoch,
                "model": best_state,
            },
        )

last.pt 包含本轮结束的完整训练状态,用于继续;best.pt 保存截至本轮验证集选中的权重与输入信息,用于预测。若第十二轮最好、继续训练到第十八轮,两个文件里的模型可能不同。

3. 输出可读的训练历史

# 续写同一文件:training_project/src/readingtorch/engine.py
    # 历史只写普通数值,绘图时不再持有训练计算图。
    report = {
        "task": task,
        "seed": seed,
        "device": device,
        "epochs": epochs,
        "best_epoch": best_epoch,
        "best_val_loss": best_score,
        "counts": {name: len(ids) for name, ids in bundle.splits.items()},
        "history": history,
        "torch_version": torch.__version__,
    }
    (output / "history.json").write_text(
        json.dumps(report, ensure_ascii=False, indent=2), encoding="utf-8"
    )
    return report

history.json 记录每轮的训练与验证指标、实际版本、种子和设备。里面没有张量计算图,画曲线只需读取这份文件。

每轮先训练再验证,改进时复制最佳权重,同时保存本轮 last;十八轮结束时 best 仍指向第十二轮,历史保留各轮数值

笔记正文与全部题解 · Ctrl / ⌘ K