Site Outage – Well, That Wasn’t Fun

For the better part of yesterday, Murfie experienced a site-wide outage. As of the wee hours of this morning, at 12:11 AM CT, everything on murfie.com thankfully returned to normal.

Our outage was due to the Internet-wide problems caused by the Amazon Web Services (AWS) failures in their Northern Virginia facility.

Murfie.com lives on Heroku. For those unfamiliar with Heroku, they are a web application hosting provider which powers thousands of websites, including many large and familiar brands. Heroku is in turn built on top of AWS. Yesterday, a failure at this Amazon datacenter caused a domino effect which took down Heroku and in turn, Murfie. Once the AWS failure occurred, Heroku’s engineers scrambled to bring all of their customers’ sites back on-line throughout the day. By about 6:00 PM CT, the Murfie website was breathing on its own again. Right around midnight, Heroku was considering the problem resolved and Murfie.com was back on its feet at full strength.

But the fact remains, it is ultimately our responsibility, and nobody else’s, to make sure your Murfie services are operational 24/7. We are actively making efforts to see to it that what happened yesterday does not happen again. Your music collections are too valuable for you to lose access for such an extended span of time. To be sure, we are not happy with how things went down on Monday. However, we very much appreciate everyone’s patience, and we’ll do our darnedest to make sure yesterday’s outage doesn’t repeat.

As such, our first priority is to modify our technical infrastructure so that we can quickly serve up your CD collections from our existing mirrors of your music data. While Murfie was down, we were frustratedly staring at the hundreds of terabytes of music we had available to serve you, but could not. Our second priority is that you’ll know what’s going on during any sort of outage. You deserve to be notified and kept abreast, and we will soon be implementing a system to ensure you know the status of the Murfie servers and services.

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s