Cosmic Conservative

Rational Conservatism from the heart of America

Qwest truly sucks

By CosmicConservative
So my internet service provider is Qwest. Qwest is what is left of US West, one of the original seven Regional Bell Operating Companies, which were spawned in the early 80s when the federal government split up the monotlithic Bell Telephone Company. Disclaimer: I used to work for Qwest. Although I worked in the yellow pages printing division, not the "phone company" part of Qwest. Yesterday, sometime between noon and about 4:00 pm our internet service died. By the time I got home it had been out a couple of hours. Now, as a short aside, we used to have internet service through a local ISP called "wispertel". Wispertel was a small entrepeneurial company that provided microwave internet connectivity in remote locations that had not yet been "connected" through the much more common internet connectivity such as DSL lines. Wispertel probably had a couple dozen employees tops, if even that many. During our time with Wispertel we had some outages, but the longest outage we had with Wispertel was probably a few hours. And that was from a small startup company that was more or less run on a shoestring. As of this moment I have not had internet connectivity at my home for 18 hours at least, and Qwest "technical support" (which is a whole different sort of joke) hasn't got a clue about when they will have service restored. They say it should be "within 48 hours" but that their "trained and professional engineering team" is working diligently on the problem. Now here's the deal. I work for a company that provides internet and direct network connectivity for millions of businesses and customers worldwide. We process financial transactions at the rate of millions per day. In fact our infrastructure is quite comparable to that of a Qwest. I know quite a bit about the internal data center details of Qwest having spent some time working with the Qwest data center people. In fact I have been inside the main Qwest data centers in the Denver area on more than one occasion. I haven't been inside the data centers of my current company, but my job is to help manage the consolidation of data centers so I have seen detailed diagrams of the data centers and have a good idea of how they compare to Qwest. The reason that is relevant is because if we had any one of our key customer-facing network services down for more than an hour or two, it would be seriously bad news. We require redundancy and disaster recovery for all critical applications. Something like "residential internet connectivity" would certainly qualify as a "critical application" for us, and presumably, for Qwest. The "technical support" I talked to (which was obviously an offshore call center manned by people whose technical expertise is limited to knowing which buttons to push on their computer screen) told me that their "system" had an entry from the "engineer" working on the outage which said simply that the outage was due to the need to replace a server, and that the server replacement would take no more than 48 hours. Now what this reveals is rather stunning considering that Qwest likes to style itself as a "world-class communications company." What it means is that they have internet connectivity for probably tens of thousands of their customers (entire towns are affected by this) apparently supported by a SINGLE server which has NO REDUNDANCY and NO BACKUP and whose replacement appears to require PURCHASING AND SHIPPING A NEW SERVER TO THEIR DATA CENTER. As I said, I work in this business and I know how long it takes to stand up equipment of this sort, and that is usually measured in minutes or hours, not DAYS. When you get into DAYS for this sort of thing, that is virtually guaranteed to mean that the company is having to PURCHASE equipment to fix the problem. This is frankly a STUNNING display of HIGH LEVEL corporate incompetence. This isn't a problem with some engineer, this is a problem with a complete lack of planning for hardware problems at the MOST BASIC data center level. When the "tech support" flunkie said "our trained and professional engineers are working on this" I laughed in his face. I told him that if Qwest HAD any "trained and professional engineers" that this sort of thing would never happen. This is baby data center stuff. This is the sort of thing that small startup companies deal with. This is simple basic redundancy. For Qwest to have ONE SERVER fail and have this sort of impact is gross incompetence. But the bottom line is that I can't do anything about it because there is NO OTHER high speed internet connectivity in my area. When Qwest finally got DSL in our area, that pretty much killed all the smaller internet companies such as the microwave internet company. So I can't go to Comcast, for example. There IS a satellite internet that I could go to, but that is expensive and requires installing satellite dishes on my roof. So I will probably stick with Qwest in spite of this stunning display of root level incompetence in their data center management. Sheesh.... Hopefully I will have connectivity back before Thursday morning when I have to be working from home....

Comments (9)

JSullins
Wondered why you made no nightly appearance yesterday ;)
CosmicConservative
Heh, was probably good to take a break from WoW, and I did spend the time reading a good book, so I didn't "suffer" from the outage, although it is a major inconvenience and is driving my son and wife insane as well. But if it isn't restored quickly, it will actually start to impact my job responsibilities. And I am quite sure that there are hundreds if not thousands of people who rely on their internet connection for their livelihood, and for Qwest to take 48 hours to respond to this is absolutely un-freaking-believable. Just gives you a chance to catch up with my rogue anyway. ;)
CosmicConservative
Woohoo!! According to the Cosmic Wife we have internet service again! Yahoo!!! Not quite 24 hours.... but more than 20.
JSullins
Fluffy is 42 going on 43, that's caught up (for now) isn't it?
CosmicConservative
Well, Rhlyssa is 43 going on 44, so that should be close enough. I will have to attend a firewall implementation change call tonight, so I may be a bit distracted, but I am only monitoring, not participating, so I should be able to multi-task... Unless something comes up I need to track as a risk or issue, that is.
Agreed
I use Qwest, too. The connection keeps dropping every few minutes. Tonight the connection will work for a few minutes, then drop continuously for the next 15 minutes. I called support a few days ago. They--some guy in India--tried to tell me that it was a problem with the wiring inside my home. No. That is what they always say to people so they don't have to face any responsibility for their shortcomings. I wish there was more broadband competition in the local market. If there was, I certainly wouldn't be using Qwest.
Hate Them
I hate AMERICANS!!! ur customer service suckssss... thats why ..there are off shore agents assiting u..morons..
Andy
Hate Them: Sometimes I want to respond to a comment, but then I realize that it's way too much trouble for "u..morons.." Take an English class. Dang Cosmic! You take CosmicDaughter to college, and all heck breaks loose. I've never seen most of these folks in the "comments." I think you are being targeted by India (we all know their "customer service" is superior 'cough cough'." You must have bad Karma. ;)
CosmicConservative
Andy: And I just went through another 20+ hour outage in my qwest service.... karma indeed.