Started At: May 28, 2023 8:02 PM (GMT-7)
Mitigated At: May 28, 2023 8:19 PM (GMT-7)
Resolved At: May 28, 2023 8:21 PM (GMT-7)
Jira issue : Link
Pagerduty incident : Link
Slack channel : Link
Incident Page: Link
Commander
Andre Yang
Users were unable to cart any items from the new summer collection. This turned out to be because the item_id of the new items were not updated with the nd- prefix. Without the prefix, the newly updated Cart API continues to reject any requests that includes items from the new collection.
The Cart API requires that all items with type = new to have an item_id that includes the nd- prefix. The API will reject any item that does not have that prefix.
This was missed because the new API request requirements were not communicated to all teams.
Currently, there isn’t any check in place to ensure that all API updates must communicated to all orgs.
To mitigate, we temporarily took all items from the new collection off of our site.
To resolve, we updated all items from the new collection with the nd- prefix.
Strict communication procedures need to be put in place for all technical teams.
Enforcement measures need to be taken as well, or else teams will continue to push updates without communicating to all impacted parties.