The ACTGAN sub-package contains an alternate implementation of the SDV CTGAN model. It provides some improvement and automation around automatic detection of datetime fields and optional usage of a binary encoder for discrete columns for better memory usage.
Please see the “ACTGAN_Demo” Notebook in the “examples” directory in the repository root.
Wrapper around ACTGAN model.
ACTGAN(field_names: Optional[List[str]] = None, field_types: Optional[Dict[str, dict]] = None, field_transformers: Optional[Dict[str, Union[BaseTransformer, str]]] = None, auto_transform_datetimes: bool = False, anonymize_fields: Optional[Dict[str, str]] = None, primary_key: Optional[str] = None, constraints: Optional[Union[List[Constraint], List[dict]]] = None, table_metadata: Optional[Union[Metadata, dict]] = None, embedding_dim: int = 128, generator_dim: Sequence[int] = 256, 256, discriminator_dim: Sequence[int] = 256, 256, generator_lr: float = 0.0002, generator_decay: float = 1e-06, discriminator_lr: float = 0.0002, discriminator_decay: float = 1e-06, batch_size: int = 500, discriminator_steps: int = 1, binary_encoder_cutoff: int = 500, binary_encoder_nan_handler: Optional[str] = None, log_frequency: bool = True, verbose: bool = False, epochs: int = 300, epoch_callback: Optional[Callable[[EpochInfo], None]] = None, pac: int = 10, cuda: bool = True, learn_rounding_scheme: bool = True, enforce_min_max_values: bool = True)¶
field_names – List of names of the fields that need to be modeled and included in the generated output data. Any additional fields found in the data will be ignored and will not be included in the generated output. If
None, all the fields found in the data are used.
field_types – Dictinary specifying the data types and subtypes of the fields that will be modeled. Field types and subtypes combinations must be compatible with the SDV Metadata Schema.
Dictinary specifying which transformers to use for each field. Available transformers are:
FloatFormatter: Uses a
FloatFormatterfor numerical data.
FrequencyEncoder: Uses a
FrequencyEncoderwithout gaussian noise.
FrequencyEncoder_noised: Uses a
FrequencyEncoderadding gaussian noise.
OneHotEncoder: Uses a
LabelEncoder: Uses a
LabelEncoderwithout gaussian nose.
LabelEncoder_noised: Uses a
LabelEncoderadding gaussian noise.
BinaryEncoder: Uses a
UnixTimestampEncoder: Uses a
NOTE: Specifically for ACTGAN, some attributes such as
auto_transform_datetimeswill automatically attempt to detect field types and will automatically set the
field_transformersdictionary at construction time. However, autodetection of
field_transformerswill not be over-written by any concrete values that were provided to this constructor.
auto_transform_datetimes – If set, prior to fitting, each column will be checked for being a potential “datetime” type. For each column that is discovered as a “datetime” the field_types and field_transformers SDV metadata dicts will be automatically updated such that datetimes are transformed to Unix timestamps. NOTE: if fields are already specified in field_types or field_transformers these fields will be skipped by the auto detector.
anonymize_fields – Dict specifying which fields to anonymize and what faker category they belong to.
primary_key – Name of the field which is the primary key of the table.
constraints – List of Constraint objects or dicts.
table_metadata – Table metadata instance or dict representation. If given alongside any other metadata-related arguments, an exception will be raised. If not given at all, it will be built using the other arguments or learned from the data.
embedding_dim – Size of the random sample passed to the Generator. Defaults to 128.
generator_dim – Size of the output samples for each one of the Residuals. A Residual Layer will be created for each one of the values provided. Defaults to (256, 256).
discriminator_dim – Size of the output samples for each one of the Discriminator Layers. A Linear Layer will be created for each one of the values provided. Defaults to (256, 256).
generator_lr – Learning rate for the generator. Defaults to 2e-4.
generator_decay – Generator weight decay for the Adam Optimizer. Defaults to 1e-6.
discriminator_lr – Learning rate for the discriminator. Defaults to 2e-4.
discriminator_decay – Discriminator weight decay for the Adam Optimizer. Defaults to 1e-6.
batch_size – Number of data samples to process in each step.
discriminator_steps – Number of discriminator updates to do for each generator update. From the WGAN paper: https://arxiv.org/abs/1701.07875. WGAN paper default is 5. Default used is 1 to match original CTGAN implementation.
binary_encoder_cutoff – For any given column, the number of unique values that should exist before switching over to binary encoding instead of OHE. This will help reduce memory consumption for datasets with a lot of unique values.
binary_encoder_nan_handler – Binary encoding currently may produce errant NaN values during reverse transformation. By default these NaN’s will be left in place, however if this value is set to “mode” then those NaN’ will be replaced by a random value that is a known mode for a given column.
log_frequency – Whether to use log frequency of categorical levels in conditional sampling. Defaults to
verbose – Whether to have print statements for progress results. Defaults to
epochs – Number of training epochs. Defaults to 300.
epoch_callback – An optional function to call after each epoch, the argument will be a
pac – Number of samples to group together when applying the discriminator. Defaults to 10.
cuda – If
True, use CUDA. If a
str, use the indicated device. If
False, do not use cuda at all.
learn_rounding_scheme – Define rounding scheme for
True, the data returned by
reverse_transformwill be rounded to that place. Defaults to
enforce_min_max_values – Specify whether or not to clip the data returned by
reverse_transformof the numerical transformer,
FloatFormatter, to the min and max values seen during
fit. Defaults to
Complex datastructures for ACTGAN
ColumnIdInfo(discrete_column_id: ‘int’, column_id: ‘int’, value_id: ‘np.ndarray’)¶
ColumnTransformInfo(column_name: ‘str’, column_type: ‘ColumnType’, transform: ‘BaseTransformer’, output_info: ‘List[SpanInfo]’, output_dimensions: ‘int’)¶
EpochInfo(epoch: int, loss_g: float, loss_d: float)¶
When creating a model such as ACTGAN if the
epoch_callbackattribute is set to a callable, then after each epoch the provided callable will be called with an instance of this class as the only argument.
SpanInfo(dim: ‘int’, activation_fn: ‘ActivationFn’)¶