ACTGAN

The ACTGAN sub-package contains an alternate implementation of the SDV CTGAN model. It provides some improvement and automation around automatic detection of datetime fields and optional usage of a binary encoder for discrete columns for better memory usage.

Please see the “ACTGAN_Demo” Notebook in the “examples” directory in the repository root.

Wrapper around ACTGAN model.

class gretel_synthetics.actgan.actgan_wrapper.ACTGAN(field_names: Optional[List[str]] = None, field_types: Optional[Dict[str, dict]] = None, field_transformers: Optional[Dict[str, Union[BaseTransformer, str]]] = None, auto_transform_datetimes: bool = False, anonymize_fields: Optional[Dict[str, str]] = None, primary_key: Optional[str] = None, constraints: Optional[Union[List[Constraint], List[dict]]] = None, table_metadata: Optional[Union[Metadata, dict]] = None, embedding_dim: int = 128, generator_dim: Sequence[int] = 256, 256, discriminator_dim: Sequence[int] = 256, 256, generator_lr: float = 0.0002, generator_decay: float = 1e-06, discriminator_lr: float = 0.0002, discriminator_decay: float = 1e-06, batch_size: int = 500, discriminator_steps: int = 1, binary_encoder_cutoff: int = 500, binary_encoder_nan_handler: Optional[str] = None, log_frequency: bool = True, verbose: bool = False, epochs: int = 300, epoch_callback: Optional[Callable[[EpochInfo], None]] = None, pac: int = 10, cuda: bool = True, learn_rounding_scheme: bool = True, enforce_min_max_values: bool = True)
Parameters

  • field_names – List of names of the fields that need to be modeled and included in the generated output data. Any additional fields found in the data will be ignored and will not be included in the generated output. If None, all the fields found in the data are used.

  • field_types – Dictinary specifying the data types and subtypes of the fields that will be modeled. Field types and subtypes combinations must be compatible with the SDV Metadata Schema.

  • field_transformers –

    Dictinary specifying which transformers to use for each field. Available transformers are:

    • FloatFormatter: Uses a FloatFormatter for numerical data.

    • FrequencyEncoder: Uses a FrequencyEncoder without gaussian noise.

    • FrequencyEncoder_noised: Uses a FrequencyEncoder adding gaussian noise.

    • OneHotEncoder: Uses a OneHotEncoder.

    • LabelEncoder: Uses a LabelEncoder without gaussian nose.

    • LabelEncoder_noised: Uses a LabelEncoder adding gaussian noise.

    • BinaryEncoder: Uses a BinaryEncoder.

    • UnixTimestampEncoder: Uses a UnixTimestampEncoder.

    NOTE: Specifically for ACTGAN, some attributes such as auto_transform_datetimes will automatically attempt to detect field types and will automatically set the field_transformers dictionary at construction time. However, autodetection of field_types and field_transformers will not be over-written by any concrete values that were provided to this constructor.

  • auto_transform_datetimes – If set, prior to fitting, each column will be checked for being a potential “datetime” type. For each column that is discovered as a “datetime” the field_types and field_transformers SDV metadata dicts will be automatically updated such that datetimes are transformed to Unix timestamps. NOTE: if fields are already specified in field_types or field_transformers these fields will be skipped by the auto detector.

  • anonymize_fields – Dict specifying which fields to anonymize and what faker category they belong to.

  • primary_key – Name of the field which is the primary key of the table.

  • constraints – List of Constraint objects or dicts.

  • table_metadata – Table metadata instance or dict representation. If given alongside any other metadata-related arguments, an exception will be raised. If not given at all, it will be built using the other arguments or learned from the data.

  • embedding_dim – Size of the random sample passed to the Generator. Defaults to 128.

  • generator_dim – Size of the output samples for each one of the Residuals. A Residual Layer will be created for each one of the values provided. Defaults to (256, 256).

  • discriminator_dim – Size of the output samples for each one of the Discriminator Layers. A Linear Layer will be created for each one of the values provided. Defaults to (256, 256).

  • generator_lr – Learning rate for the generator. Defaults to 2e-4.

  • generator_decay – Generator weight decay for the Adam Optimizer. Defaults to 1e-6.

  • discriminator_lr – Learning rate for the discriminator. Defaults to 2e-4.

  • discriminator_decay – Discriminator weight decay for the Adam Optimizer. Defaults to 1e-6.

  • batch_size – Number of data samples to process in each step.

  • discriminator_steps – Number of discriminator updates to do for each generator update. From the WGAN paper: https://arxiv.org/abs/1701.07875. WGAN paper default is 5. Default used is 1 to match original CTGAN implementation.

  • binary_encoder_cutoff – For any given column, the number of unique values that should exist before switching over to binary encoding instead of OHE. This will help reduce memory consumption for datasets with a lot of unique values.

  • binary_encoder_nan_handler – Binary encoding currently may produce errant NaN values during reverse transformation. By default these NaN’s will be left in place, however if this value is set to “mode” then those NaN’ will be replaced by a random value that is a known mode for a given column.

  • log_frequency – Whether to use log frequency of categorical levels in conditional sampling. Defaults to True.

  • verbose – Whether to have print statements for progress results. Defaults to False.

  • epochs – Number of training epochs. Defaults to 300.

  • epoch_callback – An optional function to call after each epoch, the argument will be a EpochInfo instance

  • pac – Number of samples to group together when applying the discriminator. Defaults to 10.

  • cuda – If True, use CUDA. If a str, use the indicated device. If False, do not use cuda at all.

  • learn_rounding_scheme – Define rounding scheme for FloatFormatter. If True, the data returned by reverse_transform will be rounded to that place. Defaults to True.

  • enforce_min_max_values – Specify whether or not to clip the data returned by reverse_transform of the numerical transformer, FloatFormatter, to the min and max values seen during fit. Defaults to True.

Complex datastructures for ACTGAN

class gretel_synthetics.actgan.structures.ActivationFn

An enumeration.

class gretel_synthetics.actgan.structures.ColumnIdInfo(discrete_column_id: ‘int’, column_id: ‘int’, value_id: ‘np.ndarray’)
class gretel_synthetics.actgan.structures.ColumnTransformInfo(column_name: ‘str’, column_type: ‘ColumnType’, transform: ‘BaseTransformer’, output_info: ‘List[SpanInfo]’, output_dimensions: ‘int’)
class gretel_synthetics.actgan.structures.ColumnType

An enumeration.

class gretel_synthetics.actgan.structures.EpochInfo(epoch: int, loss_g: float, loss_d: float)

When creating a model such as ACTGAN if the epoch_callback attribute is set to a callable, then after each epoch the provided callable will be called with an instance of this class as the only argument.

class gretel_synthetics.actgan.structures.SpanInfo(dim: ‘int’, activation_fn: ‘ActivationFn’)