Module

describe.py

import goldener.describe

Description of the samples of a dataset with embeddings.

This is the first step of most of the Goldener pipelines. The GoldDescriptor iterates over a dataset in batches, computes the embedding of the samples with a goldener.embed.GoldEmbeddingTool and stores the embeddings locally. The description is idempotent: samples already described are not recomputed.

The resulting embeddings are the input of the vectorization step (goldener.vectorize).

On this page 1 class

GoldDescriptor

logger = <Logger goldener.describe (INFO)>

class GoldDescriptor(goldener.do.GoldDoerWithTable):

Compute embeddings from dataset samples using a pretrained model.

The GoldDescriptor processes a dataset or PixelTable table to compute embeddings using a GoldEmbeddingTool. The computed embeddings are stored in a local PixelTable table (specified by table_path) so that the description process is idempotent: calling the same operation multiple times will not duplicate or recompute embeddings that are already present in the table.

Assuming all the data will not fit in memory, the dataset is processed in batches. All torch tensors will be converted to numpy arrays before saving.

The description can operate in a sequential (single-process) mode or a distributed mode (not implemented). The table schema is created/validated automatically and will include minimal indexing columns (idx) required to link descriptions back to their originating samples.

Attributes:

table_path: Path to the PixelTable table where descriptions will be saved locally.
embedder: GoldEmbeddingTool instance for embedding the data.
vectorizer: Optional GoldTensorVectorizationTool to further vectorize the computed embeddings before storing them in the table.
transform: Optional transformation to apply to the data before embedding computation if not already applied by the collate_fn.
collate_fn: Optional function to collate dataset samples into batches composed of dictionaries with at least the key specified by data_key returning a PyTorch Tensor. If None, collate_keeping_sequences_as_sequences is used, which preserves sequence-valued fields as one sequence per sample. It should format the value at data_key in the format expected by the embedder.
data_key: Key in the batch dictionary that contains the data to compute embeddings from. Default is "data".
target_key: Key in the batch dictionary that contains the target/label information. Default is "target".
label_key: Optional key for labels in the batch dictionary. Default is None.
target_to_label: Optional mapping from target values to label strings. Default is None.
exclude_full_zero_target: Whether to exclude samples with a target tensor containing only zeros (in case of multi target). Default is False.
merge_multilabels: Whether to merge multiple labels into a single label. Default is False.
exclude_labels: Optional set of label strings to exclude from vectorization. Default is None.
description_key: Column name to store the computed embeddings in the PixelTable table. Default is "embeddings".
force_fix_description: When True (default), the shape and dtype of the description are fixed in the column schema. When False, the spatial dimensions and type of the description can be different between rows.
to_keep_schema: Optional dictionary defining additional columns to keep from the original dataset/table into the description table. The keys are the column names and the values are the PixelTable types.
min_pxt_insert_size: Minimum number of rows to accumulate before inserting into PixelTable. Default is 100.
batch_size: Batch size used when iterating over the data.
num_workers: Number of workers for the PyTorch DataLoader during iteration on data.
allow_existing: If False, an error will be raised when the table already exists. Default is True.
distribute: Whether to use distributed processing for embedding computation and table population. Not implemented yet. Default is False.
drop_table: Whether to drop the description table after creating the dataset with descriptions. It is only applied when using describe_in_dataset. Default is False.
device: Torch device to use for embedding computation. If None, it will use 'cuda' if available, otherwise 'cpu'.
max_batches: Optional maximum number of batches to process. Useful for testing on a small subset of the dataset.

GoldDescriptor( table_path: str, embedder: goldener.embed.GoldEmbeddingTool, vectorizer: goldener.vectorize.GoldTensorVectorizationTool | None = None, transform: Optional[Callable] = None, collate_fn: Optional[Callable] = None, data_key: str = 'data', target_key: str = 'target', label_key: str | None = None, target_to_label: dict[tuple[int, ...], str] | None = None, exclude_full_zero_target: bool = False, merge_multilabels: bool = False, exclude_labels: set[str] | None = None, description_key: str = 'embeddings', force_fix_description: bool = True, to_keep_schema: dict[str, type] | None = None, min_pxt_insert_size: int = 100, batch_size: int = 1, num_workers: int = 0, allow_existing: bool = True, distribute: bool = False, drop_table: bool = False, device: torch.device | None = None, max_batches: int | None = None)

Initialize the GoldDescriptor.

Arguments:

table_path: Path to the PixelTable table where descriptions will be saved.
embedder: GoldEmbeddingTool instance for embedding data.
vectorizer: Optional GoldTensorVectorizationTool to further vectorize the computed embeddings.
transform: Optional transformation to apply before embedding computation.
collate_fn: Optional function to collate dataset samples into batches. If None, collate_keeping_sequences_as_sequences is used.
data_key: Key in the batch dictionary containing the data. Defaults to "data".
target_key: Key in the batch dictionary containing the target/label. Defaults to "target".
label_key: Optional key for labels in the batch dictionary. Default is None.
target_to_label: Optional mapping from target values to label strings. Default is None.
exclude_full_zero_target: Whether to exclude samples with a target tensor containing only zeros (in case of multi target). Default is False.
merge_multilabels: Whether to merge multiple labels into a single label. Default is False.
exclude_labels: Optional set of label strings to exclude from vectorization. Default is None.
description_key: Key for storing computed embeddings. Defaults to "embeddings".
force_fix_description: When True (default), the shape and dtype of the description are fixed in the column schema. When False, the spatial dimensions and type can differ between rows. Defaults to True.
to_keep_schema: Optional schema for additional columns to preserve.
min_pxt_insert_size: Minimum number of rows to accumulate before inserting into PixelTable. Defaults to 100.
batch_size: Batch size used when iterating over the data.
num_workers: Number of workers for the PyTorch DataLoader during iteration on data.
allow_existing: Whether to allow using an existing table. Defaults to True.
distribute: Whether to use distributed processing. Defaults to False.
drop_table: Whether to drop the table after dataset creation. Defaults to False.
device: Torch device for embedding computation. Auto-detected if None.
max_batches: Optional maximum number of batches to process.

Raises:

NotImplementedError: If distribute is True.

table_path

embedder

vectorizer

transform

data_key

target_key

label_key

target_to_label

exclude_full_zero_target

merge_multilabels

exclude_labels

description_key

force_fix_description

device

distribute: bool

Whether distributed processing is enabled.

def describe_in_dataset( self, to_describe: torch.utils.data.dataset.Dataset | pixeltable.catalog.table.Table, restrict_to: set[int] | None = None) -> goldener.pxt_utils.GoldPxtTorchDataset:

Compute embeddings from samples and return results as a GoldPxtTorchDataset.

The description process computes embeddings from the data using the provided embedder and stores them in a PixelTable table specified by table_path. The computed embeddings will be stored in the column specified by description_key.

This is a convenience wrapper that runs describe_in_table to populate (or resume populating) the PixelTable table, then wraps the table into a GoldPxtTorchDataset for downstream consumption. If drop_table is True, the table will be removed after the dataset is created.

Arguments:

to_describe: Dataset or Table to be described. If a Dataset is provided, each item should be a dictionary with at least the key specified by data_key after applying the collate_fn. If a Table is provided, it should contain both 'idx' and data_key columns.
restrict_to: Optional set of sample indices (idx) to restrict the description to. If provided, only the selected samples will be described.

Returns:

A GoldPxtTorchDataset containing at least the computed embeddings in the description_key key and an idx key as well.

def describe_in_table( self, to_describe: torch.utils.data.dataset.Dataset | pixeltable.catalog.table.Table, restrict_to: set[int] | None = None) -> pixeltable.catalog.table.Table:

Compute embeddings from samples and store results in a PixelTable table.

This method is idempotent (i.e., failure proof), meaning that if it is called multiple times on the same dataset or table, it will not duplicate or recompute the descriptions already present in the PixelTable table.

Arguments:

to_describe: Dataset or Table to be described. If a Dataset is provided, each item should be a dictionary with at least the key specified by data_key after applying the collate_fn. If a Table is provided, it should contain both 'idx' and data_key columns.
restrict_to: Optional set of sample indices (idx) to restrict the description to. If provided, only the selected samples will be described.

Returns:

A PixelTable Table containing at least the computed embeddings in the description_key column and an idx column as well.

Raises:

ValueError: If allow_existing is False and the table at table_path already exists, or if an existing description table does not have the expected primary key.

Inherited Members

goldener.do.GoldDoerWithTable: collate_fn; to_keep_schema; min_pxt_insert_size; batch_size; num_workers
goldener.do.GoldDoer: allow_existing; drop_table; max_batches