Lambda - Stream to S3 Code

Code

lambdas.storepick_stream_to_s3.get_config(name: str) → dict

Get the config from the config service. Uses global variable for caching and reducing network calls.

Params:

name (string): Name of the config

Returns

Config dictionary

Return type

configuration (dict)

lambdas.storepick_stream_to_s3.get_current_datetime() → dict

This function generated the current date & time applying appropriate formatting for the s3 path

Returns

Dictionary containing current date & time

Example: {‘date’: ‘20210510’, ‘time’: ‘11:56:02’}

Return type

(dict)

lambdas.storepick_stream_to_s3.create_s3_path(root_folder, seq_num)

This function makes creates the path in which the file will be saved within the s3 bucket

Params:

root_folder (string) : root folder of s3 bucket seq_num (string): SequenceNumber from the event received from dynamodb

Returns

the full file path of where the data will be written to within s3

Return type

s3_path (string)

lambdas.storepick_stream_to_s3.set_component_logging(config)

This method is used to set the logging level for BigQuery Writer and Pymorri.

lambdas.storepick_stream_to_s3.lambda_handler(event, context)

This Lambda receives events from DynamoDB streams. The list of data from within the event will be saved to s3.

Params:

event: Event from DynamoDB streams context: Context from DynamoDB streams

Returns

The response meta data

Return type

response (json)