The Test System That Wasn't

About Us

Advertise

Newest First | Oldest First | Threaded View

[close this box]

Page 1 / 2 > >>

50%

dr2127dr,
User Rank: Strategist
9/17/2014 | 12:45:08 AM

Re: It's Never the Vendor's Fault...

Susan,

Not generally - the one I mentioned seems to be the exception. On the other hand we have cloud with the #1 provider of data warehouses and they are the opposite - very zealous on the dev, test and production process. They are new to cloud but have been around forever. If you know data warehouseing systems you know who I'm talking about.

Reply | Post Message | Messages List | Start a Board

50%

Susan_Nunziata,
User Rank: Strategist
9/16/2014 | 6:51:38 PM

Re: It's Never the Vendor's Fault...

@dr2127dr: Are you seeing a more lax attitude about test systems from SaaS/Cloud vendors than you did from vendors selling on-premise systems? Or is this an across-the-board issue in IT?

And, to answer your question: when all of us old dogs bag it who is going to manage this kind of crazy behavior?

I'd say we are all doomed.

Reply | Post Message | Messages List | Start a Board

50%

Susan_Nunziata,
User Rank: Strategist
9/16/2014 | 6:48:15 PM

Re: Here's one guess

@moarsauce123: I wish you were makiing up the part about the Sys Admin not knowing what Firefox and Chrome are....that is truly frightening. Excellent story, though. I did laugh out loud reading that, even though it must have entirely frustrated you when it happened.

But really, why can't we just make all our customers visit our website only using IE?

<grin>

Reply | Post Message | Messages List | Start a Board

100%

moarsauce123,
User Rank: Ninja
9/13/2014 | 10:30:08 AM

Here's one guess

OK, my guess does not really fit because no vendor is involved, but here it goes because it just happened to me yesterday:

Me: Hosting sys admin, customers complain that they see a certification warning when they hit our web sites. I can reproduce this and it appears as that the certificate is expired.

Sys admin: Ah yea, that is possible, I got a bunch of emails throughout the past weeks from the issuer.

Me: And you ignored them?

Sys admin: Well...the cert was still good.

Me: Until today when the customers start complaining.

Sys admin: Put in a helpdesk ticket so that we can schedule the work

Me: What???? You go and fix this right now, we are dead in the water! You've gotten enough emails to have you schedule that work!

Sys admin: Well...I talk to my manager and see what we can do.

....some hours later....

Sys admin: OK, we managed to squeeze this work order in. It is all set.

Me: Let me try it right now.....works fine in IE, but not with Firefox.

Sys admin: Forefox? What is that?

Me: It is a web browser, it is the second most used browser of our customers after Chrome.

Sys admin: Like chrome on a car?

Me: Exactly! *groans* No, they are both web browsers. Now go back to your manager and tell him to tell you to fix it!

Sys admin: Can you submit a helpdes...

Me: NO!!!

Sys admin: Alright...I'll ask.

...some hours later....

Sys admin: You are all set. Someone misconfigured IIS and the new cert now works for other browsers as well. Would have been easier to tell all customers to stop using these wonky browsers and go with what is the best on the market: Internet Explorer.

Me: Won't fly...but thanks for fixing it. I can confirm that it works now.

...a few minutes later...

Me: Hey, sys admin, customers are still complaining, they again get the expired cert warning. What the heck?

Sys admin: Uhmmmm.....helpdesk ticket??

Me: No, manager, pronto, fix it!!

...a few minutes later...

Sys admin: Oh, this is now all fixed. We spun up a test server to check out different certs and it happened to be an exact clone of your web server so we got an IP conflict.

Me: *cries* Thanks!

Reply | Post Message | Messages List | Start a Board

50%

Joe Stanganelli,
User Rank: Author
9/13/2014 | 7:44:38 AM

Testing

The sad part is that good test processes do not begin and end with a testbed; vendors need to be doing more to test their products and patches.

As an IT old-timer told me in a recent interview: "One QA lady I knew rolled a Coke can across the keyboard as her first test—if your application failed to recover, you got it back right then."

Reply | Post Message | Messages List | Start a Board

50%

Joe Stanganelli,
User Rank: Author
9/13/2014 | 7:41:19 AM

Re: It's Never the Vendor's Fault...

@dr2127dr: I wouldn't worry. That's how new dogs become old dogs -- by dealing with this kind of garbage! ;)

Reply | Post Message | Messages List | Start a Board

50%

Technocrati,
User Rank: Ninja
9/12/2014 | 10:57:38 PM

Misunderstood Pencil Pushers and Web Surfers ?

Wow ! I really enjoyed this new section of Grumpy IT Guy. This could have been called "Tech -Guy Uncut" as well but I think grumpy is a good description. Because let's face it, even though the IT Guy's or Girl's main purpose in life is to "protect and serve" . Everyone is entitled to a bad day, week or month even. I know many would say a month is pushing it, well as a result of this view - the Grump IT Guy really starts to fester. Because at this point, you are surly talking to someone outside of tech, and most season tech guys know that is a losing battle.

To hear this type of concern from a "pencil pusher" or the proverbial "professional "web surfer " ( Is it time to go home yet ? ) only servers to raise the blood pressure. And at this point, even the effort to defend one's self only serves to produce more stress. There used to be only two certainites - death and taxes. Well I think we can add this situation to it as well.

The situation depicted in the first episode of Grumpy IT Guy, happens to IT people at least once a year. And If near disaster does not almost strike year after year - there would be no grump IT Guy.

It is refreshing to consider that No. We ( IT professionals ) don't always have to come running with a smile because we are probably dealing with something that doesn't require one. There is probably nothing to smile about and I wish - the web-surfer and the pencil pusher would be aware enough to understand that.

But apparently I am asking for too much because I have never seen this understanding extended for an extended period of time. Because "bad stuff" just doesn't happen in their worlds. Even after the pencil pusher lays you off - they don't consider that "bad stuff". And if you are lucky enough you might even catch them smiling after they do it. Ah the irony, pencil pushers and Web Surfers are good for that - always.

So here's to the Grumpy IT Guy - I am sure he will be telling it like it is and that is good because I lost my rose covered glasses years ago.

Reply | Post Message | Messages List | Start a Board

100%

dr2127dr,
User Rank: Strategist
9/12/2014 | 7:18:12 PM

Re: It's Never the Vendor's Fault...

This is prefect. We have a supply chain platform that we moved from on premise to SaaS / cloud. Same vendor - this was an upgrade.

So when I asked where is the test system - they said what test system? - I said, you know the system where we test things before cut over and or move changes into production for one of the most critical systems in a retail enterprise. They said - we don't have a test system - we test the code and then move it to to your production system - I said - really and if it goes bad then what? They said - oh we fix it ...... and non of our other custoemrs have asked for this. Really - not even the largest jean manufacture on the planet?

you get the picture - in the end we provided the test system..... This stuff happens a lot more than everyone would admit. My question - when all of us old dogs bag it who is going to manage this kind of crazy behavior?

Reply | Post Message | Messages List | Start a Board

100%

Somedude8,
User Rank: Ninja
9/12/2014 | 2:27:18 PM

My guess

I am going to put my money on a screwed up config that points some little deep dark hidden piece of both environments to a production database. The config mistake happened a long time ago. The recent patch made some schema changes (that were probably pretty questionable to begin with, but vendor ideas of good database design is another train wreck story for another day...).

Someone is going to work a 70-80 hour week without overtime to figure this out and fix it, and it won't be the person who caused it. At least 3 C-level people will get involved, certain that if they are assertive enough that the problem will get fixed. One mid level manager is refreshing his resume right now and checking LinkedIn for the first time in ages, certain that at least one of his 11 contacts must know of an opening somewhere, and will be on Dice.com by the end of the day, which will trigger some alert based on them accessing known job hunting URLs from the office. That alert will never be read by anyone.

Yes, I have seen this movie.

Reply | Post Message | Messages List | Start a Board

100%

TerryB,
User Rank: Ninja
9/12/2014 | 1:24:07 PM

Re: No consequences

@Kevin, I think you need to work with better class of vendor and sys admin. That is an exception experience, not a "day in the life".

I think the real msg here is just how complicated things have gotten in systems. It's a wonder to me anyone gets anything to work anymore in these multi server, multi role implementations. And it is huge factor in implementing security properly.

But doesn't appear to be going away. Now everyone wants to go to "cloud", that magical place where everything is secure and just works. But nobody is exactly sure how or why.....

Reply | Post Message | Messages List | Start a Board

Page 1 / 2 > >>

Reports

The State of Cloud Computing - Fall 2020

Download this report to compare how cloud usage and spending patterns have changed in 2020, and how respondents think they'll evolve over the next two years.

Download Now!

More Reports