Summary:
From 8:40 pm ET on September 21st, 2022 to 11:52 AM ET on September 22nd, 2022, Shogun Frontend’s site builder was unavailable. When a new build was triggered, customers received an error message asking them to contact Shogun support. This was due to the underlying service that powers the site builder being unavailable for the duration of this incident.
Details:
Shogun partners with a global leader in infrastructure as a service as its infrastructure provider for storefront builds as well as storage of the resulting static assets output by the Frontend builder. Shogun maintains a geographically redundant builder and storage infrastructure within our provider.
At 8:40 PM ET Shogun was alerted to errors within the Frontend builder infrastructure. The error indicated Shogun's account could no longer access the service underlying the builder. Our provider did not provide Shogun with any prior communication or warning that access to the service would be affected in any way.
Shogun’s Engineering team immediately engaged with support at our provider to re-establish access to the underlying service.
In parallel, Shogun’s Engineering team evaluated and planned alternative solutions if access was not restored in a timely fashion. Due to the fact that access was cut on an account level, the geographic redundancy previously established by Shogun did not allow for failover.
Shogun’s Engineering team then began re-creating the Frontend builder infrastructure in a separate account at our provider to restore service.
At 11:52 AM ET Shogun’s Engineering team completed the installation, configuration and testing of the new builder infrastructure.
Shogun Engineering continues to work with our partners to restore access to the underlying service on our original account.
Preventative Measures:
Shogun will be investigating other technology partnerships for the Frontend site builder and will maintain backup builder infrastructure in a separate account which will allow for an instantaneous failover to a backup builder in case of similar issues in the future.
Shogun is continuing to work with our provider on their own RCA for this issue and we will provide relevant updates as more information becomes available.