Disaster Averted for Most IT Systems

Irene's Floods Cause IT Havoc in Vermont
Disaster Averted for Most IT Systems
IT systems operated by governments, hospitals, financial institutions and other businesses averted catastrophe, for the most part, as Hurricane and then Tropical Storm Irene stormed through the Eastern seaboard over the weekend.

With the exception of the state of Vermont, where flooding knocked out one data center, most other state governments from North Carolina to Maine reported scattered power outages that disrupted IT services and websites. Officials mostly deemed the disruptions more of an inconvenience than a disaster to recover from.

"We were at the emergency operations center in Smyrna during the entire storm and were ready to pull the trigger on our COOP (continuity of operations plan) and DR (disaster recovery) plans if need be," Elayne Starkey, Delaware state chief information security officer says. "We dodged a bullet, for sure."

In New York City, several hospitals in flood-prone areas that evacuated all patients on Friday, reopened, at least partially, on Monday after restoring access to information systems. And, another hospital supplemented its business continuity plan with the approach of Irene. Meanwhile, the FBI issued a warning of phishing scams tied to Irene. The National Credit Union Administration also is taking precautions to ensure the safety of its institutions and staff. Three priorities laid out in a are ensuring the safety of credit union staff, keeping facilities and operations available to members and providing material and technical assistance, as needed, to affected credit unions.

And the Federal Deposit Insurance Corp. has issued guidance to financial institutions affected by Hurricane Irene. "The FDIC recognizes the serious impact of Hurricane Irene on customers and operations of financial institutions and will provide regulatory assistance to institutions subject to its supervision," the guidance states.

Institutions along the East Coast of the United States were significantly damaged due to the hurricane. In the wake of the disaster, the FDIC is encouraging institutions to work constructively with borrowers experiencing difficulties beyond their control because of hurricane damage.

After a week causing mayhem in the Caribbean, Irene struck eastern North Carolina's Outer Banks on Saturday morning, moving up the Atlantic coast to make a second landfall the following morning in southern New Jersey. Later Saturday, Irene made a third landfall in Brooklyn's Coney Island. Moving up through upstate New York and New England, Irene - downgraded to a tropical storm - caused massive flooding, especially in New York State and Vermont. As of Monday morning, the Federal Emergency Management Agency reports, some 5.4 million people along the East Coast were without power. Irene also is blamed for at least 44 deaths.

Most states reported few if any disruptions of IT services. New Hampshire Chief Information Officer Bill Rogers says the storm caused power fluctuations at its data centers, but backup generators and uninterruptible power supply systems kept servers functioning. Connecticut reported only 35 minor outages of different state network links, most related to power loss. "Those networks are coming back up when power does," state Chief Information Officer Mark Reynolds says. "Alternative network and communication has been established for all the off-line links. We have no systems offline."

Vermont Hit Hard

In Vermont, before Irene hit the state Sunday night, the government backed up data, so when floods swamped one data center that lost power as well as some hardware including servers and arrays, critical information remained secure at a remote site. Vermont officials knew the data center was vulnerable to flooding. "But we were not expecting six feet of water," Vermont state CISO Kris Rowley says. "The rainfall amounts looked more like snowfall amounts."

Flooding also presented a challenge to coordinate recovery efforts as state officials had to evacuate its emergency management department Sunday night. Still, Rowley says, communications remained opened. "Our IT departments have been in constant touch for the same period of time," she says. "The governor closed all state offices today (Monday) but our IT department has been actively working since early morning trying to reestablish systems, e-mail and Internet to agencies that are down."

Yet, it could take days if not weeks for the state to recover from the massive flooding. "There is still flood waters in areas where work needs to be done, including inside buildings," Rowley says. "Until that clears, work will progress slowly."

NYC Hospitals Revived

As the two campuses of Staten Island University Hospital in New York City were evacuated, the IT department shut down all phone systems, the computer network and all applications. The hospital sustained only minor damage, including some water damage in "peripheral" portions of the IT department, CIO Kathy Kania says. "We were really lucky."

All information systems were restored to full operations in an eight-hour period Sunday, thanks, in large part, to a business continuity plan that worked well, Kania says. The hospital was back open for business on Monday.

Several IT vendors helped the staff resolve some minor issues that came up during the restart of systems, the CIO says. "We began the startup of systems at 1:30 p.m. Sunday, and all systems were fully functional, with all applications tested, by 9:30 p.m."

Based on the disaster recovery experience, the hospitals will beef up their business continuity plan so it offers even more details, Kania says. For example, it would have been helpful, she says, to have phone numbers of more staff members handy to use when certain staff responsible for specific functions were unreachable.

"Other than that," she says, "our team was ... well-prepared to bring down our systems and then bring them back up in the right order ... and under tremendous stress."

Business Continuity Plan Supplemented

Memorial Sloan Kettering Cancer Center was one of many New York hospitals that were designated to receive patients from those facilities that were evacuated. It received about 30 transferred patients before the storm hit, says Linda Reissman, the hospital's director of emergency management.

Although the cancer center had a business continuity plan, it drafted a detailed supplement when it knew the hurricane was approaching and extra patients were on the way. "We mapped out strategies specific to this storm," she says.

For example, when city officials made it clear all public transportation would shut down for much of the weekend, along with the bridges to the island of Manhattan, the hospital made arrangements to ensure that it had enough staff available. More than 350 staff members spent the night at the hospital Saturday and Sunday, including about 250 who were not scheduled to work and volunteered for the effort. "We had administrators cleaning tables in the cafeteria," Reissman says.

The hospital acquired extra food and supplies in advance of the storm. Other preparation steps included topping off power generators, placing emergency lighting in critical areas, including kitchens, and placing extra emergency power outlets in the core of the building.

Fortunately, the cancer center never lost power, and all its information systems remained fully functional throughout the storm. Plus, it sustained only minor water damage.

Phishing Scams, in Wake of Irene

The Federal Bureau of Investigation on Monday issued a statement on fraudulent e-mails in light of Hurricane Irene. "Disasters prompt individuals with criminal intent to solicit contributions purportedly for a charitable organization or a good cause," the release states.

The FBI release links to a note prepared by the Internet Crime Complaint Center on fraudulent charitable contribution schemes. The note warns against spam e-mail, clicking on unsolicited links and donating through any venue other than a reputable organization's website.

Scams after a disaster aren't new. Just last year, phishing schemes targeted financial institutions and customers after the BP oil spill.

Criminals will typically use humanitarian-type scams to trick people into giving money to fake charities. The fraudsters will use e-mail, or links from social media sites like Facebook or Twitter, to harvest credit card details and cash, says Ben Knieff, director of product marketing at NICE Actimize, a provider of financial crime, risk and compliance solutions.

"I've been seeing phishing types of e-mails and links starting really about mid-last week," Knieff said. "Once the hurricane hit the Bahamas, you started to see these links for donation sites that were purportedly going to the Bahamas."

A lot of the attacks are moving to social networks where fraudulent links appear to be coming from legitimate friends or someone consumers trust.

Fraudsters see disasters as good marketing opportunities and will use phishing schemes to get people to provide log-in details. These schemes include such e-mails as saying a user's account has been affected because of the disaster and that they need to log-in to confirm their account.

Business disruption

When disasters hit, there's a real disruption between customers and financial institutions. Small business owners may not rely on typical means to conduct business. "Fraudsters know this so they can use this to their advantage in terms of social engineering," Knieff explains.

Tips Financial Institutions to Follow

  • Encourage customers and members to contact their banking institutions when they believe they've been scammed.
  • Users who click an illegitimate link should be told to run up-to-date anti-virus/anti-malware software as soon as possible.
  • Advise customers and members to change account passwords for online banking. People often use the same password for different sites, including banking sites. In the Sony breach, for example, jeopardized user names and passwords for some customers were also the same for banking sites. A unique password should be assigned for online banking.
  • Institutions and customers need to pay close attention to accounts in the weeks following a disaster. At the very least, people should review their accounts every few days.
  • The Federal Trade Commission, in a statement warning of possible scams in the aftermath of a disaster, says individuals should be wary of urgent appeals for charitable donations. To find out more about charity fraud, the FTC has prepared a fact sheet.


About the Author

Information Security Media Group

Information Security Media Group (ISMG) is the world's largest media company devoted to information security and risk management. Each of its 37 media sites provides relevant education, research and news that is specifically tailored to key vertical sectors including banking, healthcare and the public sector; geographies from North America to Southeast Asia; and topics such as data breach prevention, cyber risk assessment and fraud. Its yearly global summit series connects senior security professionals with industry thought leaders to find actionable solutions for pressing cybersecurity challenges.




Around the Network

Our website uses cookies. Cookies enable us to provide the best experience possible and help us understand how visitors use our website. By browsing healthcareinfosecurity.com, you agree to our use of cookies.