Picktask-Post Cut Off Aggregator Runbook¶

Overview¶

This is the runbook for Storepick WOS Post Cut Off Aggregator component. This documents (and references) the component documentation. Below are a set of scenarios that may happen in production along with diagnosis and resolution information.

Status: Datadog Metrics Dashboard

Runtime Issues List¶

Issues	Related Error Codes in logs	Description
Issue 1	404	Exception while fetching products from Product service
Issue 2		Product not exist in Product service
Issue 3		The specified queue does not exist for this wsdl version
Issue 4	500	The specified bucket does not exist

Runtime Issues¶

Issue 1¶

Component¶

Wholesale Post Cut Off Aggregator Task

Scenario¶

The Product Service POST endpoint fails with 404 error.

Error¶

“http_status_code: 404”

Symptoms¶

The log files for Wholesale Post Cut Off Aggregator Task would have an error message similar to below.

Example error:

ERROR:ecstask.wos_order_aggregator.util.product_service_api:Product service api exception
MoriException: exception_code: 404; detail: MoriException: ProductElasticSearch: 404 not found:
{'fault': {'faultstring': 'Unable to identify proxy for host: morrisons-ssl and
url: /product/Invalid/v1/items/@search',
'detail': {'errorcode': 'messaging.adaptors.http.flow.ApplicationNotFound'}}};
http_status_code: 404

Corrective Actions¶

Check the Wholesale Product Service is working properly.

Check URL defined in storepick-wos-aggregator.json config is correct.

Issue 2¶

Component¶

Wholesale Post Cut Off Aggregator Task

Scenario¶

Product not exist in product service

Error¶

“”

Symptoms¶

The log files for Wholesale Post Cut Off Aggregator Task would have an error message similar to below.

Example error:

ERROR:__main__: OSP_WOS_ORD_HNDLR_POST_CUT_OFF_ALARM Product details not found for product list []

Corrective Action¶

Check Product Id mentioned in Order SQS message is correct.

Issue 3¶

Component¶

Wholesale Post Cut Off Aggregator Task

Scenario¶

Incorrect AWS SQS name in storepick-wos-aggregator.json config

Error¶

“NonExistentQueue”

Symptoms¶

The log files for Wholesale Post Cut Off Aggregator Task would have an error message similar to below.

Example error:

botocore.errorfactory.QueueDoesNotExist:
An error occurred (AWS.SimpleQueueService.NonExistentQueue) when calling the GetQueueUrl operation:
The specified queue does not exist for this wsdl version.

Corrective Action¶

Check the AWS SQS name in storepick-wos-aggregator.json config

Issue 4¶

Component¶

Wholesale Post Cut Off Aggregator Task

Scenario¶

AWS S3 Bucket name is incorrect.

There are multiple possible reasons like AccessDenied or the AWS S3 bucket being accessed does not exist.

Error¶

“http_status_code: 500”

Symptoms¶

The log files for Wholesale Post Cut Off Aggregator Task would have an error message similar to below.

Example error

ERROR:ecstask.wos_order_aggregator.util.picks_upload_util: OSP_WOS_ORD_HNDLR_POST_CUT_OFF_ALARM
Error processing uploading payload
{
      "message_id": "OSP0345210621556",
      "transfer_id": "OSP0345210621556",
      "ship_to_location_id": "345",
      "orders": ["65214"],
  "items": [
              {
                  "item_id": "100418009",
                  "item_line_id": 1,
                  "item_base_type": "skuMin",
                  "item_alternate_id": {
                      "sku_min": "M100418009",
                      "sku_legacy": "0478082"
                  },
                  "item_description": "B/DRAGON S/FRY SWT&SOUR SC120GSSPS-113",
                  "quantity_type": "KG",
                  "quantity_ordered": 2,
                  "quantity_shipped": 2
              }
              ]
  }
  to s3. Error is MoriException: exception_code: 1500;
  detail: MoriException: Exception while writing file wholesale-order-transfer/20210621/OSP0345210621556.json
  to AWS S3. Exception: An error occurred (NoSuchBucket) when calling the PutObject operation:
  The specified bucket does not exist; http_status_code: 500

Corrective Action¶

Check the storepick-wos-aggregator.json config for AWS S3 bucket name and path.

Check whether IAM role has permission to access AWS S3 bucket.