travis' brain dump

This one falls into the tech posts kiddos so if you’re looking for humor, personal info, a rant or if your head hurts when you read my tech posts…. skip it. 🙂

As you could tell from my relief post yesterday we’re in the middle of deploying CommVault (site) software across our entire company as well as ExaGrid (site) disk-based systems in two of our seven primary backup locations. As I’m sure this post will be picked up by search engines I hope it’s a good base for anyone who may be looking at either of these products to get some first hand info from someone who just switched over and is in the middle of deployment. Also it’s good to get the feel of how things are in the infant stage of a deployment since this is when most of the headaches are.

I’ll start with ExaGrid since it’s going to be the shortest description since it’s that cut and dried, they don’t mess around. The deployment of the Denver site took all of 30 minutes. That includes rack time kiddos. My only complaint was the rails for the devices. They were not the best but hey, when you’re comparing them to the ease of HP rails…. yeah… Soo…. on to the deployment. If you can read you can roll this thing out. I’m dead serious. It’s that easy. It only requires the following skills:

1) Can you read?
2) Can you count?
3) Do you have a basic understanding of what an IP address is?
4) Do you have a basic understanding of what a file share is?

That’s it. Seriously. A monkey could put this thing in and get it ready to talk to any number of backup solutions. This product gets mad props for deployment and setup ease. In the support department, I did actually have to call support because of a dumb oversight on my end. Read: USER ERROR. 🙂 Somehow when we were setting up the storage policies for CommVault Galaxy we selected their “Single Instance Storage” component on stuff going to the ExaGrid. Don’t. Not only are you going to make your ExaGrid extremely unhappy but you’re trying to defeat the purpose of the wonderful deduplication process of the device. Big note: MAKE SURE YOU NEVER ENABLE SIS IN GALAXY WHEN CONFIGURING WITH EXAGRID. 🙂 Back to the subject of support, AWESOME. I love being able to call support and not get some guy in another country who I can’t understand. I’m sorry folks, but the last thing I want to do when I am having problems with a computer is make a call into support and need an freakin translator just to solve a simple problem. ExaGrid gets huge applause for their support.

The Report Card says:

Deployment:  A+
Ease of Use:   A+
Support:         A+

I also have to say the first run results of our ExaGrid unit are quite awesome. The deduplication statistics are well above what I expected as we’re showing almost a 2:1 depulication ratio across our site. For more detailed terms, we have 5,585.04 GB of data we’re backing up first run and it’s only consuming 3,048.30 GB. As it sits, it appears we’ll be able to backup even more data than we previously thought possible just because of the way this system works. So… in addition to the scores above:

Functionaility: A+

This moves us on to CommVault Galaxy. Wow. That’s the first words out of my mouth. You know all that stuff you know about ArcServ, BackupExec, etc etc?  Throw it out the window because it’s useless and honestly was just wasting your brain cells. Once you’ve cleared your mind and centered your Chi prepare to enter the realm of Galaxy. Ok, so that was a little over the top but you get what I’m saying. I had to rethink how I saw backups once I walked into this product and not in the bad way. I have to say for our setup of 1 local site and 6 remote sites for backups, this was the ABSOLUTE BEST CHOICE. To sit here in Denver and manage backups across the company without having to open up multiple desktops and play switch the screen… Priceless.

We had some bumps getting things started up but I have to say considering all things, it went really smooth. A little back story here… Our previous product, ArcServ (*cough*crap*cough*) kept blowing up during backup jobs over the last few weeks here and left us pretty much holding an empty bag right about the time our equipment and software showed up. Not to mention the fact that it completely crapped out on us because our tape library decided to have a fit about one of it’s drives. It flat refused to run any backups with a drive out because the library kept throwing errors. Mind you it never crashed, it just constantly displayed an error. Despite the fact we didn’t have professional services coming in to help roll out the product until Tuesday and Quantum wouldn’t be in until Monday to fix the library, I couldn’t wait. I pulled everything out, hooked it all up and went to setting it up. Having never setup this product before I actually had a crippled tape library *RUNNING* backups within a day. Granted, Galaxy flat told me ‘ you’ve got issues ‘ but hey, it was way more informative than our previous software and actually offered way better diagnostics and ways around those issues. So we ran a backup through the weekend here and had at least some data to rely on thanks to Galaxy.

Once professional services came in on Tuesday I was schooled in the error of my thinking in all the right ways. Boy did I over complicate things when I setup that job to run over the weekend. Why did I do this? I kept thinking like I was using one of the other products. So once I got centered and cleared my mind I was shown the path to righteousness and mended my ways. All is now well in client policy land. I wish I could sit here for an hour or two and list off all the features I find cool about this product but I really don’t have the time right now. I just can’t say enough how much this product makes sense to use. It just works.

The Report Card Says:

Deployment:  A+
Ease of Use:   A+
Support:         A+
Functionality: A+

With all that behind me I’m sitting here today just breathing a sigh of relief. Our backups here in Denver and out in Tulsa are working properly and I’m looking forward to moving the deployment forward over the next couple of weeks to get the rest of our sites online and operational. It’s going to be a lot of hard work but I think at this point the hardest part is going to be just getting the hardware online, lol.

Again, a huge amount of kudos goes out to the support personal from ExaGrid (Stefan and Dave), CommVault (Don) and Lewan (Doug and Terrance) and a huge thank you to Greg as well who was on the spot any time I needed him this week to rally the troops while we worked through some little quirks and such here and there. You rock man! (and Happy Birthday)

Ok, this week is by far the busiest week I think I have ever gone through, PERIOD! 

For the record… It’s never good when you’re doing a MAJOR deployment of a company DR/Backup solution to have your counterpart on vacation… This entire week has been one thing after another, lol. If it hasn’t been the rush of things breaking it has been the constant workload trying to get this solution deployed. Nothing like juggling the usual issues rolling in with the deployment while having two people in your office for half the week. 🙂

And in case you ever wonder…. CommVault Galaxy + Exagrid = abso-friggin-lutely awesome! Nothing like doing a backup that took you 50+ hours to tape in less than 24 for the first backup then using synthetic full’s being able to complete all backup jobs to disk in less than 5 hours from there on out. We’ll still have to see how the tape portion moves this weekend but it should be a lot better considering we have dedicated connections between the storage point and the tape devices now. So far I’ve gotten our Denver and Tulsa offices online. That only leave 5 more to go, LOL. All in all it should be pretty quick now that we have the system all figured out.

I’ll be so glad when this is all deployed… Then we can sleep at night. 🙂

Ok, so today I have confirmed it.

On occasion I use the Azureus bit torrent client in order to share out or download different files with colleagues and friends of mine, depending on the situation or the type of file. Whenever I would fire up the application, I would *ALWAYS* get a notification that the client couldn’t map traffic to ports XXXX-XXXX no matter which ones I tried. The firewall was setup to allow it through to this computer always and the ports were confirmed open on remote connects however whenever the client would talk out and have it try to talk back it would ALWAYS give me the announcement that it recieved no notification back and there was a problem.

So today, being the first day I fired it up while connected to Qwest. I get no such errors any longer.

Coincidence? I think not.  *sigh*

So… I wander away from the blog for a few hours to go get some things done (meetings and such) and come back to discover a comcast related comment and additional login on my blog… wow?

The comment, which you can see in the previous post, was:

Travis if you are still having trouble with your speeds or bandwidth please contact us at We_Can_Help@cable.comcast.com and we will be glad to help you out.cmcstcaresbill

So you’re telling me that there is someone out there watching the web for posts with comcast bashing? Un-freakin-believable. LOL I went out on the web to do a little research just to see if this was for real. I’m not one to be readily open to the idea that a company which gives you a laundry list of excuses of why their speed is not the issue but it is in turn an issue with the customer’s computer, switch, firewall, etc… is ready to be of assistance. Well, it turns out this is completely real.  o.O

cited from: http://www.thenewstribune.com/business/story/343270.html

Frank Eliason, a Comcast Corp. manager in Center City with the daunting assignment of monitoring the nation’s blogosphere for venomous posts aimed at the company, answered right away: “That should not be. We should have that looked at. Send an e-mail with account info to We_Can_Help@cable.comcast.com“>We_Can_Help@cable.comcast.com.”

Under siege for customer-service woes detailed on Comcastmustdie.com and other blogs, the Philadelphia cable giant has gone on the offensive, trawling the Internet for Comcast chatter. Eliason’s assignment is very specific: If someone has a Comcast problem and is talking about it online, he contacts that person and offers help.

If Eliason thinks it’s an emergency that could spiral into unpleasantness, like an expletive-loaded blog bomb, he gets on the phone and cuts through the corporate red tape.

But a cautionary note: Eliason’s quick action and kind words don’t necessarily lead to a quick fix, as Chambers discovered.

Eliason’s blog spotting is part public relations and part acknowledgment that the Internet is playing a broader role in defining company brands. Technology companies woke up to this fact after “Dell Hell” postings by blogger Jeff Jarvis in 2005.

Comcast executives say the company’s customer-service problems deepened as it expanded through acquisitions and added millions of high-speed Internet and phone customers. The company, with $31 billion in annual revenue, has leaned too heavily on outsourcers for phone help and repairs, they say.

On Twitter, where users write blurbs on what they’re doing or thinking at the moment, a passing complaint can be an early warning signal to Comcast. The site, said Biz Stone, a Twitter cofounder, is the sort of forum that Comcast should monitor.

“If Comcast can get to those influencers, the complaints will not grow to a full blog post,” he said. Eliason has posted about 600 messages, or “tweets,” on Twitter.

Comcast is “waking up to the fact that a bad rap in the social-networking space could spread like crazy,” said Shel Holtz, a public-relations consultant in the San Francisco area. “If consumers are talking to each other about your brand, you should participate in that conversation and have a good story to tell.”

The story in this case is more like a horror flick on Comcast’s FEARnet channel. Comcast has had a public-relations nightmare with Comcastmustdie.com, launched in October by Advertising Age columnist Bob Garfield.

If you go out to google and you search out the email address from the comment, it’s all over the place. I’m not sure how I feel about this. It seems like a damage control effort which, in my own opinion, is a bit too little and more than just a bit too late. I’m not even going to bother writing an email to the address however it does merit a blog response.

Instead of spending so much effort on trying to do post-loss damage control, why aren’t more resources being put into training customer service reps to be more technical and provide logical thought out responses which are helpful as to opposed to canned messages which deny all responsibility by comcast for the issues being experienced. Today’s world, while it still contains many users who are absolutely clueless, there are many individuals who have some or even a lot of technical expertise and find it offensive that you’re having a tier one support phone monkey question even the most basic of troubleshooting with us? If you had someone trained to assess the situation and quite possibly offer some level of support worth a darn you might save more customers from leaving disgruntled. At the very least the person may come away from it, despite going to another provider, with a feeling of ‘ we tried everything we could and this setup just didn’t work out ‘ leaving you with less work to do in damage control.

Beyond that… don’t patronize people who are obviously way smarter than your CYA department by feeding them lines of complete bull to try to quell their rants and raves when they’ve discovered that they have been cheated out of bandwidth they’ve paid for and had it reduced in quality when they utilize it to it’s fullest. There is a REASON we want a 6mbit pipe and pay for it to be that speed and when you let some tier one phone jockey who is either a) a complete idiot or b) someone with intelligence that has been bound and gagged by internal policy that they are at fault you’re going to lose customers and you’re going to get blogsite comments and websites dedicated to the hatred of your system.

My advice… take a good long look at yourselves before continuing to point the finger at the end user and or standing by policy which is completely wrong. Evaluate your customer service expectations and deliverables and make a change to better yourselves. This also includes informing your installers and service technicians of truthful information and not filling their heads full of propaganda to spread as well. Those poor guys are so brainwashed it’s almost sad.

*shrug*

I just don’t know. I’m done writing.