import_table#

DynamoDB import from S3 tool box.

Reference:

DynamoDB data import from Amazon S3: how it works: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/S3DataImport.HowItWorks.html

class aws_dynamodb_io.import_table.ImportStatusEnum(value)[source]#: Import job status enumeration.

class aws_dynamodb_io.import_table.ImportFormatEnum(value)[source]#: Import format enumeration.

class aws_dynamodb_io.import_table.ImportJob(arn: str, status: str, table_arn: Optional[str] = None, table_id: Optional[str] = None, client_token: Optional[str] = None, s3_bucket_owner: Optional[str] = None, s3_bucket: Optional[str] = None, s3_prefix: Optional[str] = None, error_count: Optional[int] = None, cloudwatch_log_group_arn: Optional[str] = None, input_format: Optional[str] = None, import_format_options: Optional[Dict[str, str]] = None, import_compression_type: Optional[str] = None, table_creation_parameters: Optional[Dict[str, str]] = None, start_time: Optional[datetime] = None, end_time: Optional[datetime] = None, processed_size_bytes: Optional[int] = None, processed_item_count: Optional[int] = None, imported_item_count: Optional[int] = None, failure_code: Optional[str] = None, failure_message: Optional[str] = None)[source]#

The DynamoDB import table job data model.

Ref:

classmethod list_imports(dynamodb_client: DynamoDBClient, table_arn: str, page_size: int = 25, max_results: int = 1000, get_details: bool = False) → List[ImportJob][source]#: Ref: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb/client/list_imports.html

classmethod from_import_description(desc: dict)[source]#

Parameters:: desc – The import description dictionary from import_table, describe_import or list_imports.

classmethod describe_import(dynamodb_client: DynamoDBClient, import_arn: str) → Optional[ImportJob][source]#: Ref: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb/client/describe_import.html

classmethod wait_until_complete(dynamodb_client: DynamoDBClient, import_arn: str, delays: int = 10, timeout: int = 900, verbose: bool = True) → ImportJob[source]#: Wait until the DynamoDB import is completed.

aws_dynamodb_io.import_table.write_amazon_ion(records: Iterable[Dict[str, Any]], s3uri: str, s3_client: S3Client) → PutObjectOutputTypeDef[source]#

Write records to S3, so that you can import DynamoDB table from S3 later.

This function uses the Amazon Ion format and gzip compression.

Ref:

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/S3DataImport.Format.html#S3DataImport.Requesting.Formats.Ion

Note

the amazon.ion python library doesn’t work well with the DynamoDB import table feature. The amazon.ion library will encode integer value like the standard json library. 1 to 1. However, the data for import table requires a “dot” after the integer, for example 1.. Looks like there’s no way we can let the amazon.ion to add the “dot”. If you want to construct dataset for DynamoDB import table, don’t use ION, use DynamoDB JSON instead.

Parameters:

records – The list of python dictionary representing the dynamodb item to write to S3.
s3uri – The S3 URI to write the data to.
s3_client – The S3 client to use.

aws_dynamodb_io.import_table.write_dynamodb_json(records: Iterable[Dict[str, Any]], s3uri: str, s3_client: S3Client) → PutObjectOutputTypeDef[source]#

Write records to S3, so that you can import DynamoDB table from S3 later.

This function uses the DynamoDB JSON format and gzip compression.

Ref:

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/S3DataImport.Format.html#S3DataImport.Requesting.Formats.DDBJson

Parameters:

records – The list of python dictionary representing the dynamodb item to write to S3.
s3uri – The S3 URI to write the data to.
s3_client – The S3 client to use.