THE COMPANY
A large firm thought it had a straightforward plan.
The company was going to shut down its pre-owned IBM® zBC12 mainframe and storage on a Saturday morning, move the hardware a couple of miles to a new datacenter, start the hardware back up, and have everything running by Sunday night at the latest. Monday morning they would be back in business.
The company’s data analysis software, including SAS and homegrown applications, were all running on that mainframe. It was absolutely their line-of-business system.
So, they shut it down and moved the hardware to the new location. Everything went well until the company tried to restart the system.
They couldn’t.
FAILOVER WAS NOT AN OPTION.
Pete Falco, Senior Account Executive for LRS® IT Solutions, described what happened.
“When the move occurred and they tried to come back up after the move on Sunday, the machines were not able to be brought up to do any functional work, so their business was dead,” he said.
The usual remedy for this type of situation would be to failover to the disaster recover, or DR, site to keep the business functioning. In this case though, failover was not an option.
As Pete explained, “The key issue was that, unlike most companies who would have failed over to their DR site, this company had no mainframe DR site whatsoever. They were not capable of failing over and their business would be totally down unless the primary location got brought up.”
LRS had urged the company to develop a DR plan or to consider buying a newer mainframe for the new datacenter instead of moving the older hardware.
The company had rejected those recommendations because of the cost involved.
THE COST OF BEING ‘PENNY WISE’
Michael Duncan, Senior Manager, Service Delivery, noted that companies choosing to be “penny wise” by opting against having a DR plan can obviously cripple their business when a disaster does strike.
“We’re here to advise them on what is the least risk way of operating, but sometimes they can’t help themselves,” Mike said. “In cases like this client, we’re here through thick and thin to keep them going.”
Another penny-wise action was the company’s decision to forego IBM maintenance coverage and instead opt for less-expensive third-party maintenance coverage.
Realizing the scope of its disaster, the company asked LRS IT Solutions to join a Sunday conference call with their IT team and the third-party support firm.
LRS gathered its staff of mainframe experts, including David Hale, IBM z Systems Solution Advisor, and Sam Cohen, System Z Consultant/Technical Team Lead, who interrupted his vacation to help work out a solution.
The Sunday conference call continued on Monday as the group hashed out possible solutions. By 3 am Tuesday, everyone took a break and met again three hours later.
“Because of the people we have, we’re willing to roll up our sleeves and do whatever it takes in order for a customer’s business to keep going.” — Pete Falco LRS IT Solutions Senior Account Executive
TIME TO REBUILD THE CONFIGURATION
Because the company didn’t have IBM maintenance, they were restricted from obtaining the necessary microcode level to restart their system.
That meant the only available solution involved David sitting down with the mainframe in its new datacenter and getting it restarted. He was armed with CDs containing operating system code and used them to build a small system on the mainframe. Once that step was completed, he was able to correct the mainframe’s configuration file.
By 3:30 that afternoon, David had the mainframe up and operating. The client was able to get into production by 8 pm Tuesday and begin to catch up on workloads.
TRUE VALUE OF LRS IT SOLUTIONS
On Wednesday morning, the CIO called the LRS team to thank them for going above and beyond to save the client’s business. He mentioned that, had the delay in business continued beyond Tuesday, the company would have incurred $25 million in costs for every additional hour of downtime.
Going above and beyond meant that David Hale, for one, managed to get a total of four hours of sleep from Saturday through late Tuesday.
Pete Falco lauded David and Sam Cohen, among others. “Because of the people we have, we’re willing to roll up our sleeves and do whatever it takes in order for a customer’s business to keep going,” he said.
Living through two business days of downtime taught the client how valuable LRS IT Solutions can be. The client also learned the value of having a DR plan in place, as well as the value of IBM maintenance, in spite of the cost. They will be meeting with LRS to move forward on both.
SOLUTIONS
- Round-the-clock consultation with client, IBM, and third-party maintenance firm to determine solution
- Hands-on restart of main-frame, including hours spent correcting system configuration file
- Gained client commitment to develope DR plan
- Over 30 years of experience in implementing IT solutions
- IBM Premier Business Partner
DOWNLOAD PDF