Picktask-Order Transfer Adapter Runbook¶

Overview¶

This is the runbook for Storepick WOS Order Transfer Adapter component. This documents (and references) the component documentation. Below are a set of scenarios that may happen in production along with diagnosis and resolution information.

Status: Datadog Metrics Dashboard

Runtime Issues List¶

Issues	Related Error Codes in logs	Description
Issue 1		The specified queue does not exist for this wsdl version.
Issue 2		KeyError: config[‘SQS’][‘NO_OF_THREADS’]
Issue 3	404	Exception while Posting Orders to Wholesale Create Order Service
Issue 4	404	Exception while Posting Orders to Wholesale Ship Order Service
Issue 5	500	The specified bucket does not exist
Issue 6	500	DynamoDB Requested resource not found

Runtime Issues¶

Issue 1¶

Component¶

Wholesale Order Transfer Adapter

Scenario¶

Incorrect AWS SQS name in wholesale-order-transfer-service.json config

Error¶

“NonExistentQueue”

Symptoms¶

The log files for Wholesale Order Transfer Adapter would have an error message similar to below.

Example error:

botocore.errorfactory.QueueDoesNotExist:
An error occurred (AWS.SimpleQueueService.NonExistentQueue) when calling the GetQueueUrl operation:
The specified queue does not exist for this wsdl version.

Corrective Actions¶

Check the AWS SQS name in wholesale-order-transfer-service.json config

Use storepick-support-scripts componenet to move messages from DLQ to SQS(Details provided in README.md)

Issue 2¶

Component¶

Wholesale Order Transfer Adapter

Scenario¶

Incorrect values in wholesale-order-transfer-service.json config

Error¶

“KeyError”

Symptoms¶

The log files for Wholesale Order Transfer Adapter would have an error message similar to below.

Example error:

max_pool_connections=config['SQS']['NO_OF_THREADS'] * 2,
KeyError: 'NO_OF_THREADS'

Corrective Action¶

Check all the details in wholesale-order-transfer-service.json config

wholesale-order-transfer-service.json
{
    "MORI_BASE_URL": "https://$ENV-api.morrisons.com",
    "WHOLESALEORDERSHIP_URL": "wholesaleorder/v1/customers",
    "WHOLESALEORDERSERVICE_URL": "wholesaleorder/v1/customers",
    "CUSTOMER_ID": "homedelivery",
    "S3_BUCKET": "m$ENV.xxwmm.storepick.wos-ord-hdlr.zzzzzzzzzz",
    "DYNAMODB": {
        "TABLE_NAME": "$ENV.xxwmm.storepick.wos-ord-hdlr-orders",
        "HASH_KEY": "orderId",
        "SORT_KEY": "transferId"
    },
    "SQS": {
        "QUEUE_NAME": "sqs-euw1-$ENV-storepick-wos-ord-hdlr-ord-trf-002",
        "NO_OF_THREADS": 1,
        "BUFFER_SIZE": 1,
        "VISIBILITY_TIMEOUT": 5,
        "FLUSH_TIMEOUT": 120,
        "WAIT_TIME": 20
    },
    "SLEEP": 0.4
}

Use storepick-support-scripts componenet to move messages from DLQ to SQS(Details provided in README.md)

Issue 3¶

Component¶

Wholesale Order Transfer Adapter

Scenario¶

The Wholesale Order Service POST endpoint fails with 404 error.

Error¶

“http_status_code: 404”

Symptoms¶

The log files for Wholesale Order Transfer Adapter would have an error message similar to below.

Example error:

[ERROR] (MainThread) OSP_WOS_ORD_HNDLR_TRANSFER_ALARM Exception while Posting Order OSP13200135 to Wholesale Create Order Service
MoriException: exception_code: 404; detail: MoriException: WholesaleOrderService: 404 not found:
{'fault': {'faultstring': 'Unable to identify proxy for host: morrisons-ssl and
url: /wholesaleorder/Invalid/v1/customers/homedelivery_sit/orders',
'detail': {'errorcode': 'messaging.adaptors.http.flow.ApplicationNotFound'}}};
http_status_code: 404

Corrective Action¶

Check the Wholesale Order Service is working properly.

Check URL defined in wholesale-order-transfer-service.json config is correct.

Use storepick-support-scripts componenet to move messages from DLQ to SQS(Details provided in README.md)

Issue 4¶

Component¶

Wholesale Order Transfer Adapter

Scenario¶

The Wholesale Order Service POST endpoint fails with 404 error.

Error¶

“http_status_code: 404”

Symptoms¶

The log files for Wholesale Order Transfer Adapter would have an error message similar to below.

Example error:

[ERROR] (MainThread) OSP_WOS_ORD_HNDLR_TRANSFER_ALARM Exception while Posting Order OSP13200136 to Wholesale Order Shipment Service
MoriException: exception_code: 404; detail: MoriException: WholesaleOrderShip: 404 not found:
{'fault': {'faultstring': 'Unable to identify proxy for host: morrisons-ssl and
url: /wholesaleorder/Invalid/v1/customers/homedelivery_sit/orders/OSP13200136/@ship',
'detail': {'errorcode': 'messaging.adaptors.http.flow.ApplicationNotFound'}}};
 http_status_code: 404

Corrective Action¶

Check the Wholesale Order Service is working properly.

Check URL defined in wholesale-order-transfer-service.json config is correct.

Use storepick-support-scripts componenet to move messages from DLQ to SQS(Details provided in README.md)

Issue 5¶

Component¶

Wholesale Order Transfer Adapter

Scenario¶

AWS S3 Bucket name is incorrect.

There are multiple possible reasons like AccessDenied or the AWS S3 bucket being accessed does not exist.

Error¶

“http_status_code: 500”

Symptoms¶

The log files for Wholesale Order Transfer Adapter would have an error message similar to below.

Example error

[ERROR] (MainThread) OSP_WOS_ORD_HNDLR_TRANSFER_ALARM Error while reading file from AWS S3 MoriException: exception_code: 1500;
detail: MoriException: Exception while getting file wholesale-order-transfer/20210617/OSP13200135.json from AWS S3.
Exception: An error occurred (NoSuchBucket) when calling the GetObject operation: The specified bucket does not exist; http_status_code: 500

Corrective Action¶

Check the wholesale-order-transfer-service.json config for AWS S3 bucket name and path.

Check whether IAM role has permission to access AWS S3 bucket.

Use storepick-support-scripts componenet to move messages from DLQ to SQS(Details provided in README.md)

Issue 6¶

Component¶

Wholesale Order Transfer Adapter

Scenario¶

There are multiple possible reasons like AccessDenied or the Dynamo Table being accessed does not exist.

Error¶

“http_status_code: 500”

Symptoms¶

The log files for Wholesale Order Transfer Adapter would have an error message similar to below.

Example error

[ERROR] (MainThread) OSP_WOS_ORD_HNDLR_TRANSFER_ALARM Exception while saving records into DynamoDB table MoriException: exception_code: 1500;
detail: MoriException: Exception while updating table dev.xxwmm.storepick.invalid_wholesale-order-service.
Exception: An error occurred (ResourceNotFoundException) when calling the UpdateItem operation: Requested resource not found; http_status_code: 500

Corrective Action¶

Check the wholesale-order-transfer-service.json config for AWS DynamoDB table name and path.

Check whether IAM role has permission to access AWS DynamoDB table.

Use storepick-support-scripts componenet to move messages from DLQ to SQS(Details provided in README.md)