OSP Stock Transfer Adapter Infrastructure¶

Components¶

Component	Description	Activity	Ref
<<SSM Attribute>>	New SSM key specifically for stock transfer adapter to protect the content	New	SSM Key
<<SSM Attribute - Location>>	New SSM key specifically for location service to protect the content	New	SSM Key
<<SSM Attribute - Product>>	New SSM key specifically for product service to protect the content	New	SSM Key
<<Config Service>>	New application configuration for monitoring RDS events	New	Configuration (from config service)
<<AWS Role>>	New AWS Role to CW log filter to work	New	AWS Role
<<AWS ECS Task>>	New ECS task for stock transfers. Handle reading messages from kafka, posting to stock transfer service and saving to dynamoDB	New	AWS ECS Task
<<AWS ECS Task>>	New ECS task for stock transfer aggregator.	New	AWS ECS Task
<<AWS ECS Task>>	New ECS task for stock transfer returns aggregator.	New	AWS ECS Task
<<AWS Kafka Cluster>>		New	AWS Kafka Cluster

SSM Key¶

The security credentials will be stored in the following security credentials in the digital Jenkins. These credentials are used to populate the SSM key below in the respective environments. This secure attribute is created from the common-infrastructure repository.

SSM Parameter Name
storepick_stocktransfer_secret

Value:

{
    "API_KEY": "1rQlOAe1PAWaWL.....",
    "AUTHORISATION": "Basic MXJRbE9BZTFQQVdhV0xI....."
}

SSM Key¶

The security credentials will be stored in the following security credentials in the digital Jenkins. These credentials are used to populate the SSM key below in the respective environments. This secure attribute is created from the common-infrastructure repository.

SSM Parameter Name
storepick_location_service_secret

Value:

{
    "API_KEY": "1rQlOAe1PAWaWL.....",
    "AUTHORISATION": "Basic MXJRbE9BZTFQQVdhV0xI....."
}

SSM Key¶

The security credentials will be stored in the following security credentials in the digital Jenkins. These credentials are used to populate the SSM key below in the respective environments. This secure attribute is created from the common-infrastructure repository.

SSM Parameter Name
storepick_product_service_secret

Value:

{
    "API_KEY": "1rQlOAe1PAWaWL.....",
    "AUTHORISATION": "Basic MXJRbE9BZTFQQVdhV0xI....."
}

Configuration (from config service)¶

Config File Name
stock-transfer-adapter.json

Value:

{
    "MORI_BASE_URL": "https://sit-api.morrisons.com",
    "STOCKTRANSFERSCREATE_URL": "stock/v2/transfers",
    "PRODUCTELASTICSEARCH_URL": "product/v1/items/",
    "GETLOCATION_URL": "location/v2/stores",
    "API_RETRY_COUNT": 5,
    "API_RETRY_EXPONENTIAL_BACKOFF": 1000,
    "CACHE_SIZE_STORES": 1000,
    "CACHE_SIZE_MINS": 5000,
    "STOCK_TRANSFER_REQUEST": {
        "VIRTUAL_LOC_PREFIX": "8",
        "USER_NAME": "StorePickUser",
        "CONTAINER_ID": 1
    },
    "DYNAMODB_AUDIT": {
        "TABLE_NAME": "$ENV.xxwmm.storepick-stock-transfers-audit",
        "TTL_NO_OF_DAYS": 30,
        "HASH_KEY": "siteDate"
    },
    "STOCK_TRANSFER_AGGREGATOR": {
        "TIME_OUT": 30,
        "BATCH_SZ": 50,
        "BROKERS": "b-1.picktask-test.k8bixh.c5.kafka.eu-west-1.amazonaws.com:9094,b-2.picktask-test.k8bixh.c5.kafka.eu-west-1.amazonaws.com:9094,b-3.picktask-test.k8bixh.c5.kafka.eu-west-1.amazonaws.com:9094",
        "PICKS_TOPIC": "picktasks-new",
        "PICKS_AGG_TOPIC": "picktasks-aggregates",
        "KFK_MAX_READ_MESSAGES": 1000,
        "KFK_TRX_ID_PREFIX": "PICKS-AGG",
        "GROUP_ID": "PICKS-AGG",
        "MAX_MESSAGES_IN_ONE_BATCH": 100000
    },
    "STOCK_TRANSFER_RETURNS_AGGREGATOR": {
        "TIME_OUT": 30,
        "BATCH_SZ": 50,
        "BROKERS": "b-1.picktask-test.k8bixh.c5.kafka.eu-west-1.amazonaws.com:9094,b-2.picktask-test.k8bixh.c5.kafka.eu-west-1.amazonaws.com:9094,b-3.picktask-test.k8bixh.c5.kafka.eu-west-1.amazonaws.com:9094",
        "PICKS_TOPIC": "picktasks-old",
        "PICKS_AGG_TOPIC": "picktasks-aggregates",
        "KFK_MAX_READ_MESSAGES": 1000,
        "KFK_TRX_ID_PREFIX": "PICKS-RET-AGG",
        "GROUP_ID": "PICKS-RET-AGG",
        "MAX_MESSAGES_IN_ONE_BATCH": 100000
    }
    "STOCK_TRANSFER_ADAPTER": {
        "BROKERS": "b-1.picktask-test.k8bixh.c5.kafka.eu-west-1.amazonaws.com:9094,b-2.picktask-test.k8bixh.c5.kafka.eu-west-1.amazonaws.com:9094,b-3.picktask-test.k8bixh.c5.kafka.eu-west-1.amazonaws.com:9094",
        "PICKS_AGG_TOPIC": "kafka-euw1-$ENV-digital-storepick-picktasks-aggregates",
        "KFK_MAX_READ_MESSAGES": 1,
        "GROUP_ID": "STOCK-TRANSFER-ADAPTER",
        "KFK_MAX_POLL_INTERVAL_MS": 900000,
        "KFK_AUTO_COMMIT_INTERVAL_MS": 900000,
        "KAFKA_MAX_RETRY_ATTEMPTS": 5,
        "PICKS_AGGREGATOR_DLQ_TOPIC": "kafka-euw1-$ENV-digital-storepick-picktasks-aggregates-dlq",
        "KFK_TRX_ID_PREFIX": "PICKS-AGG-DLQ"
    }
}

Attribute	Description
MORI_BASE_URL	The base endpoint used for Morrisons APIs, specific key name for pymorri library use
STOCKTRANSFERSCREATE_URL	The suffix for the endpoint used within the Stock transfer API, specific key name for pymorri library use
PRODUCTELASTICSEARCH_URL	The suffix for the endpoint used within the product elastic search API, specific key name for pymorri library use
GETLOCATION_URL	The suffix for the endpoint used within the location API, specific key name for pymorri library use
API_RETRY_COUNT	Retries of API calls.
API_RETRY_EXPONENTIAL_BACKOFF	Number of milliseconds in exponential backoff.
CACHE_SIZE_STORES	Number of stores to maintain in location service’s cache.
CACHE_SIZE_MINS	Number of MINs to maintain in product service’s cache.
VIRTUAL_LOC_PREFIX	Number added before previous siteID to make virtual store ID used for stock transfer
USER_NAME	Used in _prepare_request_for_stock_transfer function, determines the ‘who’ value of the ‘audit’ section of the stock transfer payload.
CONTAINER_ID	Added to the transfers payload along with the aggregated items of the stock transfer during preparation of the payload.
TABLE_NAME	The table name for the dynamodb table, this is where picktask details are stored
TTL_NO_OF_DAYS	The time to live within dynamodb in days. Messages will be deleted after this period of time.
HASH_KEY	Sort key used for querying dynamoDB table
TIME_OUT	The time to receive each message before it times out.
BATCH_SZ	This number indicates the item batch size to aggregate per pick
BROKERS	The Kafka broker endpoints to receive messages from.
PICKS_TOPIC	The Kafka topic in which the picks are located.
PICKS_AGG_TOPIC	The Kafka topic in which the aggregated picks are located.
KFK_MAX_READ_MESSAGES	The max amount of messages to read each time.
KFK_TRX_ID_PREFIX	Prefix for IDs of Kafka messages.
GROUP_ID	The ID of Kafka consumer group.
MAX_MESSAGES_IN_ONE_BATCH	The max messages to be read per batch.
KFK_MAX_POLL_INTERVAL_MS	Kafka maximum poll interval in milliseconds.
KFK_AUTO_COMMIT_INTERVAL_MS	Kafka auto commit interval in milliseconds.
KAFKA_MAX_RETRY_ATTEMPTS	The max retries to process a Kafka message thru stock transfer API.
PICKS_AGGREGATOR_DLQ_TOPIC	The DLQ topic.

AWS Role¶

Role Name
rol-glob-p-digital-ecsTaskSsmRestrictAccess

AWS ECS Task¶

Environment	Aws ECS Task
dev	storepick-osp-transfer-adapter-dev-001
sit	storepick-osp-transfer-adapter-sit-001
prd	storepick-osp-transfer-adapter-prd-001

AWS ECS Task¶

Environment	Aws ECS Task
dev	svc-euw1-dev-storepick-location-aggregator-001
sit	svc-euw1-sit-storepick-location-aggregator-001
prd	svc-euw1-prd-storepick-location-aggregator-001

AWS ECS Task¶

Environment	Aws ECS Task
dev	svc-euw1-dev-storepick-return-aggregator-001
sit	svc-euw1-sit-storepick-return-aggregator-001
prd	svc-euw1-prd-storepick-return-aggregator-001

AWS DynamoDB Table¶

Environment	Dynamo Table
dev	dev.xxwmm.storepick-stock-transfers-audit
sit	sit.xxwmm.storepick-stock-transfers-audit
prd	prd.xxwmm.storepick-stock-transfers-audit

AWS Kafka Cluster¶

Environment	Kafka Cluster
dev
sit
prd