Picktask-Order Aggregator Runbook¶

Overview¶

This is the runbook for Storepick WOS Order Aggregator Component. This documents (and references) the component documentation. Below are a set of scenarios that may happen in production along with diagnosis and resolution information.

Status: Datadog Metrics Dashboard

Runtime Issues List¶

Issues	Related Error Codes in logs	Description
Issue 1	404	Exception while fetching products from Product service
Issue 2		Product not exist in Product service
Issue 3		The specified queue does not exist for this wsdl version
Issue 4	500	The specified bucket does not exist

Runtime Issues¶

Issue 1¶

Component¶

Wholesale Order Aggregator Task

Scenario¶

The Product Service POST endpoint fails with 404 error.

Error¶

“http_status_code: 404”

Symptoms¶

The log files for Wholesale Order Aggregator Task would have an error message similar to below.

Example error:

ERROR:ecstask.wos_order_aggregator.util.product_service_api:Product service api exception
MoriException: exception_code: 404; detail: MoriException: ProductElasticSearch: 404 not found:
{'fault': {'faultstring': 'Unable to identify proxy for host: morrisons-ssl and
url: /product/Invalid/v1/items/@search',
'detail': {'errorcode': 'messaging.adaptors.http.flow.ApplicationNotFound'}}};
http_status_code: 404

Corrective Actions¶

Check the Wholesale Product Service is working properly.

Check URL defined in storepick-wos-aggregator.json config is correct.

Use storepick-support-scripts componenet to move messages from DLQ to SQS(Details provided in README.md)

Issue 2¶

Component¶

Wholesale Order Aggregator Task

Scenario¶

Product not exist in product service

Error¶

“”

Symptoms¶

The log files for Wholesale Order Aggregator Task would have an error message similar to below.

Example error:

ERROR:__main__: OSP_WOS_ORD_HNDLR_ORD_AGGREGATOR_ALARM Product details not found for product list []

Corrective Action¶

Check Product Id mentioned in Order SQS message is correct.

Use storepick-support-scripts componenet to move messages from DLQ to SQS(Details provided in README.md)

Issue 3¶

Component¶

Wholesale Order Aggregator Task

Scenario¶

Incorrect AWS SQS name in storepick-wos-aggregator.json config

Error¶

“NonExistentQueue”

Symptoms¶

The log files for Wholesale Order Aggregator Task would have an error message similar to below.

Example error:

botocore.errorfactory.QueueDoesNotExist:
An error occurred (AWS.SimpleQueueService.NonExistentQueue) when calling the GetQueueUrl operation:
The specified queue does not exist for this wsdl version.

Corrective Action¶

Check the AWS SQS name in storepick-wos-aggregator.json config

Use storepick-support-scripts componenet to move messages from DLQ to SQS(Details provided in README.md)

Issue 4¶

Component¶

Wholesale Order Aggregator Task

Scenario¶

AWS S3 Bucket name is incorrect.

There are multiple possible reasons like AccessDenied or the AWS S3 bucket being accessed does not exist.

Error¶

“http_status_code: 500”

Symptoms¶

The log files for Wholesale Order Aggregator Task would have an error message similar to below.

Example error

ERROR:ecstask.wos_order_aggregator.util.picks_upload_util: OSP_WOS_ORD_HNDLR_ORD_AGGREGATOR_ALARM
Error processing uploading payload
{
      "message_id": "OSP0345210621556",
      "transfer_id": "OSP0345210621556",
      "ship_to_location_id": "345",
      "orders": ["65214"],
  "items": [
              {
                  "item_id": "100418009",
                  "item_line_id": 1,
                  "item_base_type": "skuMin",
                  "item_alternate_id": {
                      "sku_min": "M100418009",
                      "sku_legacy": "0478082"
                  },
                  "item_description": "B/DRAGON S/FRY SWT&SOUR SC120GSSPS-113",
                  "quantity_type": "KG",
                  "quantity_ordered": 2,
                  "quantity_shipped": 2
              }
              ]
  }
  to s3. Error is MoriException: exception_code: 1500;
  detail: MoriException: Exception while writing file wholesale-order-transfer/20210621/OSP0345210621556.json
  to AWS S3. Exception: An error occurred (NoSuchBucket) when calling the PutObject operation:
  The specified bucket does not exist; http_status_code: 500

Corrective Action¶

Check the storepick-wos-aggregator.json config for AWS S3 bucket name and path.

Check whether IAM role has permission to access AWS S3 bucket.

Use storepick-support-scripts componenet to move messages from DLQ to SQS(Details provided in README.md)