ray.train.huggingface.transformers.RayTrainReportCallback#

class ray.train.huggingface.transformers.RayTrainReportCallback(*args: Any, **kwargs: Any)[source]#

Bases: transformers.trainer_callback.TrainerCallback

A simple callback to report checkpoints and metrics to Ray Tarin.

This callback overrides the TrainerCallback.on_save() method. After a new checkpoint get saved, it fetches the latest metric dictionary from TrainerState.log_history and reports it with the latest checkpoint to Ray Train.

If you want more customized reporting logics, please implement your own callbacks following the Transformers integration user guides.

Note that users should ensure that the logging, evaluation, and saving frequencies are properly configured so that the monitoring metric is always up-to-date when transformers.Trainer saves a checkpoint.

Suppose the monitoring metric is reported from evaluation stage:

Some valid configurations:

  • evaluation_strategy == save_strategy == “epoch”

  • evaluation_strategy == save_strategy == “steps”, save_steps % eval_steps == 0

Some invalid configurations:

  • evaluation_strategy != save_strategy

  • evaluation_strategy == save_strategy == “steps”, save_steps % eval_steps != 0

For more info, see: Saving and Loading Checkpoints.

PublicAPI (beta): This API is in beta and may change before becoming stable.

Methods

on_save(args, state, control, **kwargs)

Event called after a checkpoint save.