Sales to BigQuery

Overview

Passing sales records from RDS databse into BigQuery table

Repositories

• main repository: Sales to BigQuery <https://bitbucket.org/morrisonsplc/storepick-monitor-mordering-rds-triggers/src/master/>

• additional repositories:

  • Pymorri library:

    overview: python library with common functions and classes shared between multiple services

    repo: pymorri

    more info: pymorri docs

  • Common Serverless

    overview storepick_events_writer lambda is used from this repo for writing metrics into DataDog

    repo: common-serverless

    more info: common-serverless docs

Useful scripts

Useful scripts for debugging issues of this project are placed in Support Scripts repo.

Repo: Support Scripts

Especially script aws_insights_query.py is useful for analysis mixing, sorting by timestampe and analysing of logs of related multiple CloudWatch log groups.

Config file

It requires authentication using API KEY:

Path to Config File

Related CloudWatch Log Groups

"/aws/lambda/lmb-euw1-{env}-storepick_monitor_mordering_rds_triggers-001",
"clg-euw1-storepick-rds-trg-ecs-{env}-001",
"/aws/lambda/lmb-euw1-{env}-strpck-event_writer-001",

JIRA tickets

Main Story - SPDP-2864