Module
View source
transform.py
import goldener.vision.transform
Transforms for vision data.
PatchifyImageMask patchifies a binary mask to align it with the tokens of a ViT model, so that the mask can be
used to filter the token vectors during the vectorization (see goldener.vectorize).
class
PatchifyImageMask(torchvision.transforms.v2._transform.Transform):
Patchify a binary mask in order to align it with the tokens of a ViT model.
Attributes:
- patch_size: The size of the patches to create.
- match_ratio: The ratio of non zero pixels in a patch to consider it as a valid token for the class.
PatchifyImageMask(patch_size: int = 16, match_ratio: float = 0.5)
Initialize internal Module state, shared by both nn.Module and ScriptModule.
def
transform(self, inpt: Any, params: dict[str, typing.Any] | None = None) -> Any:
Patchify a binary mask in order to align it with the tokens of a ViT model.
Arguments:
- inpt: A binary mask tensor of shape (B, 1, H, W) where B is the batch size, H and W are the height and width of the image.
- params: A dictionary of parameters (not used in this transform).
Returns:
A tensor of shape (B, N) where N is the number of patches (H // patch_size) * (W // patch_size) and each value is 1 if the ratio of non zero pixels in the patch is greater than match_ratio, otherwise 0.