Monday, May 2, 2011

What Keeps Me Up at Night in the Data Center

Last week, I keynoted the Markley Group annual meeting and spoke about the data center issues that keep me up at night.

1. At Harvard Medical School, increasing amounts of research is done in "silicon" instead of wet labs. The growth in demand is unpredictable and bursty. When grants are funded, demand for new equipment is instantaneous. Data centers often have fixed real estate, limited power, and constrained capital budgets for expansion, making unplanned expansion problematic.

2. There is zero tolerance for downtime in the face of constantly changing technologies. We need to continuously innovate, providing the latest technology while maintaining existing systems at high levels of reliability.

3. Power and cooling needs are increasing exponentially. We've already virtualized all our application servers and we're beginning to virtualize database servers. Virtualizing high performance computing nodes does not really help since those nodes require maximal raw processing power. Harvard Medical School's compute cluster has 6000 cores. Our data center infrastructure needs optimal power usage efficiency to minimize energy costs.

4. Storage demand is now multi-petabyte. Drive density is increasing and costs are falling, but backing up and archiving petabytes is still a challenge.

5. Regulatory and compliance requirements now require searching and e-discovery of increasingly complex data stores. Although most healthcare organizations typically do not face Sarbanes-Oxley  reporting requirements, other requirements such as HIPAA, ARRA/HITECH, and the Affordable Care Act have their own data retention and analysis implications.

My solution to many of these issues has been to create "elastic data centers" using external hosting facilities such as those provided by the Markley Group. Harvard Medical School has two such floors - a "low density" 5kw/rack 1000 square foot floor with an option to expand to 5000 share feet and a "high density" 30kw/rack floor with unlimited expansion capabilities. This flexibility enables me to shift the burden of power and cooling planning to someone else, while enabling me to serve my customers on demand.

BIDMC's EHR hosting center is another example of an elastic data center. We provide a private cloud with eClinical Works EHR offered via a Software as a Service model. The problem is that we do not know how many clinicians we'll support over time, so we contracted for an outsourced hosting center with easy expandability.

What will the next few years bring in data centers? My prediction is that

• On demand storage and compute cycles from private cloud facilities will become commonplace
• Clusters and Grids will enable communities of collaborators to flexibly share processing power
• Green Data Centers with Power Usage Effectiveness less than 1.50 will reduce the rate of growth of data center energy costs
• HIPAA compliant private clouds will evolve to enables EHRs and other person identified data to be hosted in the cloud
• The amount of storage and compute cycles needed to meet increasing demands will strain existing hospital-owned data centers, resulting in more elastic data centers hosted externally.

It's an exciting time to be in IT!

1 comment:

Medical Quack said...

Thanks much for reinforcing the cloud technology. With the recent Amazon issue I considered that a wake up call, but it's not one to put fear or resistance to use cloud services by any means. We all move along and learn everyday with all these variables out there today.

By the way during the Amazon outage your name was mentioned on a perhaps questionable AWS forum. It was an SOS put out there for help and normally I look at some of these discussion as 50% truth and 50% opinion to a degree, but none the less if this was in fact a valid situation with someone not having a backup, it makes me begin to wonder about what is getting funded and if enough due diligence is done from the onset.

Over at the AWS forum one commenter suggested that the device company should ask Dr. Halamka for help and you may have already seen this by now:)

https://forums.aws.amazon.com/thread.jspa?threadID=65649&start=0&tstart=0

It was kind of a mixed bag as one thought this was a hoax and some not as they listed their account number and so forth before the final commenter with his opinion. At any rate the whole scenario was a wake up call and we all learn as we go along and the cloud is not going away anytime soon even though with the press consumers might feel a bit uneasy at present it will move forward. Good comments such as what wrote here is very helpful to re-establish the reality of the clouds:)

I liked the "silicon" research mention too as many don't realize how much research goes on before an actual "wet" lab is created today.