ray.train.SyncConfig#

class ray.train.SyncConfig(sync_period: int = 300, sync_timeout: int = 1800, sync_artifacts: bool = False, sync_artifacts_on_checkpoint: bool = True, upload_dir: Optional[str] = 'DEPRECATED', syncer: Optional[Union[str, ray.train._internal.syncer.Syncer]] = 'DEPRECATED', sync_on_checkpoint: bool = 'DEPRECATED')#

Bases: object

Configuration object for Train/Tune file syncing to RunConfig(storage_path).

See Appendix: Types of data stored by Tune for an overview of what data is synchronized.

In Ray Train/Tune, here is where syncing (mainly uploading) happens:

The experiment driver (on the head node) syncs the experiment directory to storage (which includes experiment state such as searcher state, the list of trials and their statuses, and trial metadata).

For a Ray Tune run with many trials, each trial will upload its trial directory to storage, which includes arbitrary files that you dumped during the run. For a Ray Train run doing distributed training, each remote worker will similarly upload its trial directory to storage.

See How to Configure Persistent Storage in Ray Tune for more details and examples.

Parameters

  • sync_period – Minimum time in seconds to wait between two sync operations. A smaller sync_period will have the data in storage updated more often but introduces more syncing overhead. Defaults to 5 minutes.

  • sync_timeout – Maximum time in seconds to wait for a sync process to finish running. A sync operation will run for at most this long before raising a TimeoutError. Defaults to 30 minutes.

  • sync_artifacts – [Beta] Whether or not to sync artifacts that are saved to the trial directory (accessed via train.get_context().get_trial_dir()) to the persistent storage configured via train.RunConfig(storage_path). The trial or remote worker will try to launch an artifact syncing operation every time train.report happens, subject to sync_period and sync_artifacts_on_checkpoint. Defaults to False – no artifacts are persisted by default.

  • sync_artifacts_on_checkpoint – If True, trial/worker artifacts are forcefully synced on every reported checkpoint. This only has an effect if sync_artifacts is True. Defaults to True.

PublicAPI: This API is stable across Ray releases.

Methods

Attributes

sync_artifacts

sync_artifacts_on_checkpoint

sync_on_checkpoint

sync_period

sync_timeout

syncer

upload_dir