Module

split.py

import goldener.split
View source

Splitting of a dataset in multiple sets.

The GoldSplitter splits a dataset into multiple sets (e.g. train, validation and test) defined by GoldSet configurations. Instead of a random split, every set but the last one is filled by selecting a representative subset of the samples (goldener.select), so that the sets reproduce the variability of the task. The last set gathers all the remaining samples. The splitter chains the whole pipeline: description (goldener.describe), vectorization (goldener.vectorize), optional clustering (goldener.clusterize) and selection.

On this page 2 classes1 function
GoldSet check_sets_validity() GoldSplitter
logger = <Logger goldener.split (INFO)>
@dataclass
class GoldSet:

Configuration for a gold set used for splitting.

Attributes:
  • name: Name of the gold set (None if the set is for not selected data).
  • size: Size (ratio or actual number) of samples to assign to this set (between 0 and 1). If a ratio, this value
  • cannot be one of 0 or 1 (goal is to select a subset of the full dataset). If a number, it must be strictly
  • positive and less than the total number of samples in the dataset.
GoldSet(name: str | None, size: float | int)
name: str | None
size: float | int
def check_sets_validity( sets: list[GoldSet], total: int | None = None, force_max: bool = True) -> None:

Validate the sets configuration.

This function ensures that the sum of ratios is valid and that set names are unique.

Arguments:
  • sets: List of GoldSet configurations to validate.
  • total: Optional total number of samples (used if sizes are integers).
  • force_max: Whether to enforce that the sum of ratios equals 1.0 (for float sizes) or total (for int sizes).
Raises:
  • ValueError: If the set's sizes are not valid, or if set names are not unique.
class GoldSplitter(goldener.do.GoldDoer):

Split a dataset into multiple sets based on embeddings.

The GoldSplitter leverages a GoldDescriptor to compute embeddings from the dataset, a GoldVectorizer to vectorize these embeddings, a GoldClusterizer to cluster them and a GoldSelector to select samples for each set based on specified ratios.

The splitting can operate in a sequential (single-process) mode or a distributed mode (not implemented).

At least 2 sets are required to split the data. Every sample will be associated with exactly one set using a GoldSelector. The last set is filled with any remaining elements. Every set is at least associated with at least 1 points (the ratio might not be fully matched for small data regimes).

See GoldDescriptor, GoldVectorizer, and GoldSelector for more details on each component.

Attributes:
  • sets: List of GoldSet configurations defining the splits. They are required to split all the samples among them.
  • selector: GoldSelector used to select samples for each set.
  • descriptor: Optional GoldDescriptor used to describe the dataset.
  • vectorizer: Optional GoldVectorizer used to vectorize the described dataset.
  • clusterizer: Optional GoldClusterizer used to clusterize the described dataset.
  • n_clusters: Number of clusters to use for clusterized selection (if clusterizer is provided).
  • in_described_table: Whether to return the splitting in the described table or the selected table.
  • allow_existing: Whether to allow existing tables in all components.
  • drop_table: Whether to drop intermediate tables. Defaults to False.
  • max_batches: Optional maximum number of batches to process in both descriptor and selector. Useful for testing on a small subset of the dataset.
GoldSplitter( sets: list[GoldSet], selector: goldener.select.GoldSelector, descriptor: goldener.describe.GoldDescriptor | None = None, vectorizer: goldener.vectorize.GoldVectorizer | None = None, clusterizer: goldener.clusterize.GoldClusterizer | None = None, n_clusters: int = 0, in_described_table: bool = False, allow_existing: bool = True, drop_table: bool = False, max_batches: int | None = None)

Initialize the GoldSplitter.

Arguments:
  • sets: List of GoldSet configurations defining the splits. They are required to split all the samples among them.
  • selector: GoldSelector used to select samples for each set.
  • descriptor: Optional GoldDescriptor used to describe the dataset.
  • vectorizer: Optional GoldVectorizer used to vectorize the described dataset.
  • clusterizer: Optional GoldClusterizer used to clusterize the described dataset.
  • n_clusters: Number of clusters to use for clusterized selection (if clusterizer is provided).
  • in_described_table: Whether to return the splitting in the described table or the selected table.
  • allow_existing: Whether to allow existing tables in all components.
  • drop_table: Whether to drop intermediate tables. Defaults to False.
  • max_batches: Optional maximum number of batches to process for the selection.
  • Useful for testing on a small subset of the dataset.
Raises:
  • ValueError: If in_described_table is True but no descriptor is provided.
  • ValueError: If vectorizer.vectorized_key does not match selector.vectorized_key.
  • ValueError: If descriptor.description_key does not match vectorizer.data_key (when vectorizer is set).
  • ValueError: If descriptor.description_key does not match selector.vectorized_key (when no vectorizer).
  • ValueError: If clusterizer.vectorized_key does not match selector.vectorized_key.
  • ValueError: If the sets configuration is invalid (for example, invalid sum of sizes/ratios or duplicate set names).
sets: list[GoldSet]

Get the current sets configuration for the splitter.

descriptor
vectorizer
clusterizer
n_clusters
selector
in_described_table
max_batches: int | None

Get the maximum number of batches to process.

allow_existing: bool

Get whether existing tables are allowed in all components.

clustering_enable: bool
@staticmethod
def get_split_indices( split_data: pixeltable.catalog.table.Table | torch.utils.data.dataset.Dataset, selection_key: str = 'selected', idx_key: str = 'idx', batch_size: int = 1024, num_workers: int = 0, collate_fn: Optional[Callable] = None) -> dict[str, set[int]]:

Get the indices of samples for each set from a split table.

Arguments:
  • split_table: PixelTable Table containing the split information.
  • selection_key: Column name in the table indicating the set assignment.
  • idx_key: Column name in the table indicating the sample indices.
  • batch_size: Batch size to use when processing the input as dataset.
  • num_workers: Number of workers to use when processing the input as dataset.
  • collate_fn: Collate function to use when processing the input as dataset. It should return a dictionary with at least the keys specified by idx_key and selection_key. If None, collate_keeping_sequences_as_sequences is used.
Returns:

A dictionary mapping each set name to a set of sample indices.

def split_in_dataset( self, to_split: torch.utils.data.dataset.Dataset | pixeltable.catalog.table.Table) -> goldener.pxt_utils.GoldPxtTorchDataset:

Split the dataset into multiple sets in a PixelTable table.

The dataset is first described using the gold descriptor (computes embeddings), and then samples are selected for each set based on the specified ratios after vectorization.

At least 2 sets are required to split the data. Every sample will be associated with exactly one set using a GoldSelector. The last set is filled with any remaining elements. Every set is at least associated with at least 1 points (the ratio might not be fully matched for small data regimes).

This method is idempotent (i.e. failure proof), meaning that if it is called multiple times on the same dataset or table, it will not duplicate or recompute the splitting decisions already present in the PixelTable table.

Arguments:
  • to_split: Dataset or Table to be split. If a Dataset is provided, each item should be a
  • dictionary with at least the key specified by descriptor data_key after applying the collate_fn.
  • If a Table is provided,
  • it should contain both 'idx' and data_key column for the descriptor.
Returns:

A GoldPxtTorchDataset dataset containing at least the set assignation in the selection_key column. Then the other columns are either from the described table (if in_described_table is True) or from the selected table (if in_described_table is False).

def split_in_table( self, to_split: torch.utils.data.dataset.Dataset | pixeltable.catalog.table.Table) -> pixeltable.catalog.table.Table:

Split the dataset into multiple sets in a PixelTable table.

The dataset is first described using the gold descriptor (computes embeddings), and then samples are selected for each set based on the specified ratios after vectorization.

At least 2 sets are required to split the data. Every sample will be associated with exactly one set using a GoldSelector. The last set is filled with any remaining elements. Every set is at least associated with at least 1 points (the ratio might not be fully matched for small data regimes).

This method is idempotent (i.e. failure proof), meaning that if it is called multiple times on the same dataset or table, it will not duplicate or recompute the splitting decisions already present in the PixelTable table.

See describe_in_table, vectorize_in_table, and select_in_table for more details on each step.

Arguments:
  • to_split: Dataset or Table to be split. If a Dataset is provided, each item should be a
  • dictionary with at least the key specified by descriptor data_key after applying the collate_fn.
  • If a Table is provided,
  • it should contain both 'idx' and data_key column for the descriptor.
Returns:

A PixelTable Table containing at least the set assignation in the selection_key column. Then the other columns are either from the described table (if in_described_table is True) or from the selected table (if in_described_table is False).

Raises:
  • ValueError: If any set results in zero samples due to its ratio, if label_key is not found,
  • or if label stratification results in zero samples for any label in a set.
Inherited Members
goldener.do.GoldDoer
drop_table