ray.data.Dataset.union

Dataset.union(*other: List[ray.data.dataset.Dataset]) → ray.data.dataset.Dataset

Materialize and concatenate Datasets across rows.

The order of the blocks in the datasets is preserved, as is the relative ordering between the datasets passed in the argument list.

Caution

Unioned datasets aren’t lineage-serializable. As a result, they can’t be used as a tunable hyperparameter in Ray Tune.

Note

This operation will trigger execution of the lazy transformations performed on this dataset.

Examples

>>> import ray
>>> ds1 = ray.data.range(2)
>>> ds2 = ray.data.range(3)
>>> ds1.union(ds2).take_all()
[{'id': 0}, {'id': 1}, {'id': 0}, {'id': 1}, {'id': 2}]

Parameters

other – List of datasets to combine with this one. The datasets must have the same schema as this dataset, otherwise the behavior is undefined.

Returns

A new dataset holding the rows of the input datasets.