AUI Processors Infrastructure¶
Components¶
Component |
Description |
Ref |
|---|---|---|
<<Config Service>> |
Configuration details used for OSP Event Handler Service, S3 to SQS lambda |
|
<<S3 Bucket>> |
S3 Bucket to store pick details received |
|
<<SQS>> |
SQS to add MINs in batches from file processor and then load processor will read that data for further processing |
|
<<SQS>> |
SQS to add MINs, maskFrom, maskUntil, stores in batches from product availability file processor and then product-availability load processor will read that data for further processing |
|
<<SQS>> |
SQS will read that data from product availability SQS |
|
<<SQS>> |
SQS to add MINs, fromDate, toDate, capAlertSet, productCap, alertEmail in batches from product capping file processor and then product-availability load processor will read that data for further processing |
|
<<RDS>> |
RDS to save product capping information , bulk_upload details, aui_audit, product masking details in the given RDS table |
|
<<Lambda>> |
Lambda to process file received in S3 bucket. This lambda will trigger based on product import file PUT into S3 bucket |
|
<<Lambda>> |
Lambda to process file received in S3 bucket. This lambda will trigger based on product availability file PUT into S3 bucket |
|
<<Lambda>> |
Lambda to process file received in S3 bucket. This lambda will trigger based on product capping file PUT into S3 bucket |
storepick-aui-lmb-product-capping-file |
<<Lambda>> |
Lambda to process SQS message to RDS. This lambda will trigger based on SQS event once message added into SQS this lambda will trigger |
storepick-aui-lmb-product-capping-load |
<<Lambda>> |
Lambda to process SQS message to RDS. This lambda will trigger based on SQS event once message added into SQS this lambda will trigger |
|
<<Lambda>> |
Lambda to process SQS message to RDS. This lambda will trigger based on SQS event once message added into SQS this lambda will trigger |
Configuration(from config service)¶
Config File Name |
|---|
storepick-aui-productavailability-service.json |
Value:
{
"S3_BUCKET_NAME": "m$ENV.xxwmm.storepick.aui.product-import.eeeeeeeeee",
"S3_PRODUCT_IMPORT_PATH": "productimport/pending",
"S3_PRODUCT_AVAILABILITY_PATH": "product-availability/pending",
"S3_PRODUCT_CAPPING_PATH": "product-capping/pending",
"PRODUCT_SQS_NAME": "sqs-euw1-$ENV-storepick-aui-product-import-002.fifo",
"AVAILABILITY_SQS_NAME": "sqs-euw1-$ENV-storepick-aui-product-availability-002.fifo",
"CAPPING_SQS_NAME": "sqs-euw1-$ENV-storepick-aui-product-capping-002.fifo",
"CHECKOUT_SQS_NAME": "sqs-euw1-$ENV-storepick-aui-checkout-002.fifo",
"PRIORITY_ONE_SQS_NAME": "sqs-euw1-$ENV-storepick-aui-oos-priority-one-002.fifo",
"PRIORITY_TWO_SQS_NAME": "sqs-euw1-$ENV-storepick-aui-oos-priority-two-002.fifo",
"NO_OF_THREADS": 1,
"BUFFER_SIZE": 1,
"VISIBILITY_TIMEOUT": 10,
"FLUSH_TIMEOUT": 120,
"WAIT_TIME": 20,
"WAIT_FOR_NEXT_SQS_CALL": 10,
"READ_NO_OF_MESSAGES": 10,
"DATABASE": "aui",
"SQS_MSG_LIMIT": 100,
"DEFAULT_FETCH_LIMIT": 1,
"FILE_TYPES": [
"csv",
"xlsx"
"MORI_BASE_URL": "https://sit-api.morrisons.com",
"PRODUCT_URL": "/product/v1/items/",
"OSP_BASE_URL": "https://mocks.dev.spt.digital.morconnect.com/osp",
"OSP_AVAILABILITY_URL": "v4/products",
"OSP_ORDER_SERVICE_URL": "v3/morrisons/orders",
"OSP_TIMEOUT_SECONDS": 61,
"OSP_API_CONCURRENT_THREADS": 12
}
Attribute |
Description |
|---|---|
S3_BUCKET_NAME |
S3 bucket to upload product import, capping, masking files for process and some predefined templates also |
S3_PRODUCT_IMPORT_PATH |
Import path for product import file |
S3_PRODUCT_AVAILABILITY_PATH |
Import path for product availability file |
S3_PRODUCT_CAPPING_PATH |
Import path for product capping file |
PRODUCT_SQS_NAME |
Product SQS name in this file-processor lambdas will add MINs from uploaded file into batches |
AVAILABILITY_SQS_NAME |
Product availability SQS name in this product-availability-file-processor lambdas will add MINs, maskfrom maskUntil, stores from uploaded file, in batches |
CAPPING_SQS_NAME |
Product capping SQS name in this product-availability-file-processor lambdas will add MINs, fromDate toDate, capAlertSet, productCap, alertEmail, from uploaded file, in batches |
SQS_MSG_LIMIT |
This is SQS message limit, number of MINs in one message |
FILE_TYPES |
Supported file types csv and xlsx |
DATABASE |
this is the RDS database that we are using to store information |
MORI_BASE_URL |
This is morrisons base URL |
PRODUCT_URL |
This is product service URL |
OSP_BASE_URL |
This is OCADO base URL |
OSP_AVAILABILITY_URL |
This is OCADO availability URL to GET/POST product overrides |
AWS S3 Bucket¶
Environment |
S3 Bucket |
|---|---|
dev |
mdev.xxwmm.storepick.aui.product-import.eeeeeeeeee |
sit |
msit.xxwmm.storepick.aui.product-import.eeeeeeeeee |
prd |
mprd.xxwmm.storepick.aui.product-import.zzzzzzzzzz |
AWS Product Import SQS¶
Environment |
SQS |
|---|---|
dev |
sqs-euw1-dev-storepick-aui-product-import-002.fifo |
sit |
sqs-euw1-sit-storepick-aui-product-import-002.fifo |
prd |
sqs-euw1-prd-storepick-aui-product-import-002.fifo |
AWS Product Availaility SQS¶
Environment |
SQS |
|---|---|
dev |
sqs-euw1-dev-storepick-aui-product-availability-002.fifo |
sit |
sqs-euw1-sit-storepick-aui-product-availability-002.fifo |
prd |
sqs-euw1-prd-storepick-aui-product-availability-002.fifo |
AWS Product capping SQS¶
Environment |
SQS |
|---|---|
dev |
sqs-euw1-$dev-storepick-aui-product-capping-002.fifo |
sit |
sqs-euw1-$sit-storepick-aui-product-capping-002.fifo |
prd |
sqs-euw1-$prd-storepick-aui-product-capping-002.fifo |
AWS SQS¶
Environment |
SQS |
|---|---|
dev |
sqs-euw1-dev-storepick-aui-oos-priority-two-002.fifo |
sit |
sqs-euw1-sit-storepick-aui-oos-priority-two-002.fifo |
prd |
sqs-euw1-prd-storepick-aui-oos-priority-two-002.fifo |
RDS TABLES¶
RDS Name - rds-euw1-dev-storepick-pgr-aui-001¶
RDS Name - rds-euw1-sit-storepick-pgr-aui-001¶
RDS Name - rds-euw1-prd-storepick-pgr-aui-001¶
Table Name |
|---|
storepick_aui.product_capping |
storepick_aui.bulk_upload |
storepick_aui.audit |
storepick_aui.product_masking |
AWS Lambda¶
Environment |
Lambda |
|---|---|
dev |
lmb-euw1-dev-storepick-aui-product-import-file-processor-001 |
sit |
lmb-euw1-sit-storepick-aui-product-import-file-processor-001 |
prd |
lmb-euw1-prd-storepick-aui-product-import-file-processor-001 |
AWS Lambda¶
Environment |
Lambda |
|---|---|
dev |
lmb-euw1-dev-storepick-aui-product-availability-file-processor-001 |
sit |
lmb-euw1-sit-storepick-aui-product-availability-file-processor-001 |
prd |
lmb-euw1-prd-storepick-aui-product-availability-file-processor-001 |
AWS Lambda¶
Environment |
Lambda |
|---|---|
dev |
lmb-euw1-dev-storepick-aui-product-import-load-processor-001 |
sit |
lmb-euw1-sit-storepick-aui-product-import-load-processor-001 |
prd |
lmb-euw1-prd-storepick-aui-product-import-load-processor-001 |
AWS Lambda¶
Environment |
Lambda |
|---|---|
dev |
lmb-euw1-dev-storepick-aui-product-avail-load-processor-001 |
sit |
lmb-euw1-sit-storepick-aui-product-avail-load-processor-001 |
prd |
lmb-euw1-prd-storepick-aui-product-avail-load-processor-001 |
AWS Lambda¶
Environment |
Lambda |
|---|---|
dev |
lmb-euw1-dev-storepick-aui-product-capping-file-processor-001 |
sit |
lmb-euw1-sit-storepick-aui-product-capping-file-processor-001 |
prd |
lmb-euw1-prd-storepick-aui-product-capping-file-processor-001 |
AWS Lambda¶
Environment |
Lambda |
|---|---|
dev |
lmb-euw1-dev-storepick-aui-product-capping-load-processor-001 |
sit |
lmb-euw1-sit-storepick-aui-product-capping-load-processor-001 |
prd |
lmb-euw1-prd-storepick-aui-product-capping-load-processor-001 |