How To: Replacing A Failed PERC Card in a Dell Poweredge System

This blog will bring you through the process of replacing a failed PERC card in a Dell Poweredge system. This process is identical for the PERC5 and older, however, PERC4 and earlier controllers will use a different configuration utility.

  • First, remove the failed card from the system and put the replacement card in.
  • Power on the system. There are several different ways that a new card can behave when it’s first powered on.
    • It may automatically import the array off of your hard drives, if this is the case you will see a message while it’s going through post that references the number of virtual drives handled by the controller. If you see this (image below) you will not need to do anything further and the server should boot through the OS:

image 1

  • The other potential behavior is a message saying, “foreign configuration(s) found on adapter” as shown here:

image 2

  • If you get the message about foreign configurations on the adapter, press “c” or “Ctrl+R” to enter the raid controller utility menu. On the upper right hand corner a message about foreign configurations will be present as shown here:

image 3

  • Highlight the top option on the main screen and press “f2”. This action will bring up a sub-menu and “foreign config” will be one of the options on the sub-menu. Press “enter” or the right arrow on “foreign config” and it will bring up another sub-menu with “import” and “clear” being the options, click on “import”, as shown.

image 4

  • Then, it will take a few minutes to process and, assuming everything else is fully functional. the configuration utility should update and show a list of all of the virtual drives that were configured on the system. This will look like this:

Image 5

This concludes the process for replacing a failed PERC card in a Dell Poweredge system. We hope you were successful in your process. Questions and comments are welcome below if you appreciate our technical support blogs or if you have additional questions!

 

 

 

 

How To: Run Onboard Diagnostics for the Dell Generation 11 and Up Servers

USED 5 21 18

Built into the lifecycle controller; the Dell, generation 11 and up, servers come with onboard diagnostics that can be used to test almost all of the hardware within the system.

  1. The first thing you need to do in order to reach the onboard diagnostics is to boot into the lifecycle controller; to do this press f10 while the system is going through its POST process. It will start by saying, “press f10 to enter system services” on the top right hand corner of the screen. When pressed it will change the message to inform you that it will attempt to boot into the lifecycle controller at the end of POST.
  2. After it’s booted into the lifecycle controller, which can take several minutes depending on the model, you will be brought to a UEFI menu with full mouse and keyboard support. Look for “hardware diagnostics” on the left hand column of options.
  3. Once the “hardware diagnostics” option is selected there will be a link on the right that you can click into.
  4. On Generation 11 servers you will be brought into a two-option menu after entering the hardware diagnostics; one of the options is “MPmemory” and the other option is “diagnostics”. The “MPmemory” option will allow you to run tests on all of the system memory within the system. There is also an express option that will run a set number of tests and a custom option that will allow you to select a number of more time consuming options. The other menu, “diagnostics,” will have a number of different types of tests you can run. You can either select “express” or “full tests,” which will run the system through a set number of tests or you can use custom tests which will bring you to a menu were you can manually select and deselect different devices to test.
  5. On Generation 12 servers you will be brought into one single diagnostic menu which will test everything from within one test area.
  6. After all tests are completed there should be an error readout area that will list any devices that failed. It should also give some information on what failed on each device, if any.

This finalizes the steps to take in order to run onboard diagnostics for Dell, Generation 11 and up, servers. We hope these steps aided in your success and we thank  you for visiting our blog and our bi-weekly How To!

How To: Upgrade Firmware on Generation 11 and Up Servers Using the Lifecycle Controller

rawpixel-552393-unsplash

Listed below are some simple steps to follow in order to upgrade the firmware on servers that are classified as generation 11 and up. This method of upgrading uses the lifecycle controller.

  1. Boot the server and press F10 during post. You will notice it will reference “entering system services” or “lifecycle controller,” this  will depend on the model of the server.
  2. Once it has booted into the lifecycle controller you will see, on the left hand column, a “firmware update” or a “platform update,” depending on what generation server you’re on. Click this option and it will give you several links within the right side of the screen. Then click “Launch firmware update” as the top option.
  3. Once there, you will have three options to select, “FTP server” being the top option, select this option.
  4. If you have never configured a NIC within the lifecycle controller it will give you a message saying “Network is not configured” and ask if you would like to configure it now. Accept to go into the NIC configuration.
  5. Once in there, it will give you a list of available NIC ports to configure. Select one that has a cable plugged into it and it will give you an option of “static” or “DHCP” configuration. Select whichever will be applicable for your network. Once selected, click “finish” on the bottom right corner.
  6. Once you have the NIC port set up and it successfully configures, it will send you back to the initial selection of “FTP server”, Local Drive, and Network Share. Select “FTP Server” again.
  7. Now that the NIC is configured it should send you to a page with several fields. The top field should be filled in already with “ftp.dell.com.” You may leave the rest of the options blank. Now, click “next” on the bottom right corner.
  8. It will now start connecting to the FTP server and will likely take several minutes to go through and detect all of the potential updates for the system.
  9. After it has found all the potential updates it will bring you to a list where you may select which devices to update. Once you’ve selected your devices and clicked “accept” it will start downloading all of the updates that were selected and automatically run the updates. The system will likely reboot several times during the update process. The number of reboots will depend on how many updates are selected.
  10. You will know it’s complete when it sends you back into the lifecycle controller a final time and does not go back into the “firmware” updater window.

This finalizes the steps to take in order to upgrade the firmware on your generation 11 and up server. We hope these steps were easy to follow and guided you to success on your firmware upgrade. Thank you for visiting our blog and this How To!

How Velocity Tech Solutions Saved Spring Break

Spring Break is upon us. Each year around this time, I think about one of my favorite Velocity Tech Solutions “saving asses among the masses” stories.

This is Minnesota and spring break can feel more like winter solstice, so when families can actually get a room at the Water Park of America they feel like a Florida vacation without being in Florida. That is a BIG Deal!

yes

I was the lucky one to carry the emergency phone that night. I happened to be at a social event with our President, Kay Winchell, which was a bonus for the customer since she is a network engineer in disguise.

I answered the phone at 10pm and I heard panic. A server was down and so was the Water Park of America! Kids were crying, parents were screaming, parents were weeping. It was pandemonium! The tech from Water Park of America was beside himself. He had spent the entire day dealing with another vendor and paid for parts to be couriered across the Twin Cities multiple times and the server was still not repaired. That vendor had not asked questions and just kept throwing parts at him without helping troubleshoot the issue.

shutterstock_497021242

We met him at the office and he was so exhausted he couldn’t remember the model of the server . We walked around our warehouse for an hour (It’s a big warehouse!) and he finally saw the server that was the same model.

Kay grabbed the server, I grabbed the parts and we went through creating his error. Within minutes we discovered the issue.

The tech rushed back with the cache module that was needed and within an hour the Water Park of America  was up and running and our customer was ecstatic! (And so were the parents that promised the kids the water slide that was as dry as a bone all day long!) We helped the tech become the Hero instead of the Villain. The only thing that could have gone better was if he had found us much earlier!

That night, it was decided, we don’t just sell servers, we provide the “WOW” experience and it became our #1 core value.

www.dellserverpros.com

#velocitytechsolutions.com

 

 

 

Cyber Security- The Ostrich Approach is hurting the world

 

Image result for ostrich with head in sand

 

 

How important is cyber security to you?  You folks in education, and healthcare how about you? IT professionals, I know you have your pretty little firewalls, but seriously does your software update every minute so it catches every threat out there? Perhaps its “good enough”  It’s 2018, in case anyone hasn’t noticed and Cyber Security continues to be a huge issue for every business that has a network. That means EVERY business out there is subject to these threats.

In 2017, hackers stole $172 BILLION dollars from people all over the world according to an article from MIT Technology Review. $19.4 Billion happened in the United States. I am willing to bet that in each of these threats, those pretty little firewalls were up and running and all the lights were flashing that all was good in the world.
The forecast for 2018 isn’t any better. According to Forbes, education and healthcare are going to be big targets this year. Educational institutions lack resources to defend their endpoints. school systems are a lightly secure network of endpoints that contain personally identifiable information on students, parents and staff.
 
The increased use of IoT in the healthcare industry will also create data security concerns in 2018. In the era of connected devices, the healthcare industry needs to make patient security a top priority by increasing security protocols. To combat this, businesses should look to third-party security providers to encrypt these devices and monitor with live ethical hackers that can actually see an intrusion and catch it BEFORE it infects  your network or brings it down.
So stopping it BEFORE devistation?? What a novel idea.
So why are we taking the Ostrich approach again? The pretty little firewalls we have so much faith in are obviously not getting the job done. Oh, oh wait it’s too expensive, right? Once a school is hacked and the child’s data is in the wrong hands, have you figure out what that childs value is? How much money does it cost you to be down?
Sometimes its expensive to be cheap.  For a few hundred, maybe a few thousand dollars,  you can stop adding to the billions lost. Get a pen test  or have your network monitored.
Anne Tarantino
www.velocitytechsolutions.com

 
 

How critical is your companies data and what is it’s value to you?

This article was posted in a local Minneapolis paper last week. Sit back and think about your own company’s data and how much that is worth. Think about liability and what that might cost. What happened at Fairview Health Services isn’t uncommon.

There isn’t a one size fits all solution, but research what fits best for your company. Read this article and think about what a disaster like this could cost you.

 

Whistleblower: Fairview Health Services’ IT system keeps crashing

itemprop

 

Fairview Health Services started small.

Founded in 1906 by a group of Minneapolis Lutherans, the hospital provided care to the city’s Norwegian immigrant community.

Over the past century, Fairview’s grown into a behemoth. Still headquartered in Minneapolis, the nonprofit healthcare organization today employs almost 25,000 staffers at various hospitals, dozens of clinics, 50-plus senior housing locations, and nearly 30 retail pharmacies.

That kind of expansion doesn’t come without growing pains. And not just when it comes to its clunky impending merger with University of Minnesota Physicians, or the game of musical chairs playing out at its president and CEO position.

The hospital system’s IT department is regularly straining to keep its systems online — and sometimes scrambling to get them working again at all.

For many years the nonprofit used global conglomerate Hitachi’s computer storage system. Hitachi served as an electronic warehouse for the volumes of medical records generated by 70,000 inpatients and 6.5 million outpatient visits each year.

 

The importance of a health care provider’s computer storage cannot be overstated.

It’s the foundation of the inverted IT triangle, with streams of data funneling downward through applications, to the server, to the network. The storage system is assigned with receiving, compressing, and saving all that information, like a patient’s medications history or the latest lab test results.

“Storage is critical,” says an IT professional familiar with Fairview’s system who spoke to City Pages on the condition of anonymity because he’s still employed in the field. “In compressing all that information up front, it’s working super hard and must be 100 percent active and performing functionally. Otherwise, you can have problems.”

In the fall of 2015 Fairview installed a new storage system. Hitachi’s successor, EMC, a company owned by the multinational corporation Dell, supposedly would be a state-of-the-art replacement. But the Dell EMC system is having stubborn problems that are affecting other crucial IT components.

According to internal Fairview documents, glitches related to the EMC storage system are limiting care givers’ access to Epic, a data system in use at Fairview and many other American hospitals. Epic’s applications are responsible for everything from registering a patient and scheduling blood work to fulfilling pharmacy orders.

In some instances, Fairview staff have intermittent access to the software. In others, chronic issues cause the entire system to be shut down.

And that, in turn, is creating issues for Fairview and its patient caretakers, according to documents obtained by City Pages, and interviews with current employees at the hospital system, who all agreed to speak only on the condition of anonymity for fear of professional repercussions.

One employee has reached out to former Minnesota Attorney General Mike Hatch.

 

“I can confirm I have met with one of the employees, the whistleblower, if you will, who is pursuing the whistleblower matter,” Hatch says.

Hatch added that he was not personally handling the employee’s case, and said he had “forwarded on the [employee’s] message to people in the state government.”

That employee, a veteran Fairview IT worker, says under its old storage system with Hitachi, the hospital chain had one across-the-board IT system outage in 12 years. Since switching to EMC in fall of 2015, it’s had three crashes in one year.

The staffer gives an example of a random hospital patient who checks in at a Fairview hospital. The patient’s name is introduced to the system, where health care professionals can access or add to his or her medical records. If there’s a hiccup somewhere within the larger IT system, Epic can often take the brunt of it. If Epic’s not available, Fairview staff are back to pen and paper.

“So in other words,” the staffer says, “you can’t pull people’s information who are at the hospitals or clinics. So there’s the potential for an impact for whatever they’re going to have done.”

The story of Fairview’s IT problems begins almost two years ago. With its existing Hitachi system needing an upgrade, Fairview was in the market for a new deal. Tasked with finding it was the nonprofit’s newly hired vice president of infrastructure Don Tierney.

This was no small decision. Nor would it come cheap. The system, for instance, would have to fluently interface with Fairview’s more than 1,500 computer applications. The storage hardware and accompanying software had a total price tag of roughly $3 million.

Various companies courted Fairview and Tierney: Hitachi, IBM, Computex and Pure Storage, and a company called EMC. The nonprofit’s IT staff favored sticking with Hitachi and springing for an upgrade.

Tierney awarded the contract to EMC.

“[Tierney] basically said to us, ‘This is what we’re going to get, and you guys don’t have a choice,'” says a Fairview employee. “I have to think they now, at least somewhat, regret that decision. Because the product that they bought wasn’t ready, wasn’t fully baked to handle what it was purchased to do.”

Among the incidents seen since the storage switch was a mid-April ordeal lasting parts of two days, in which “several of our technology systems, including Epic… were behaving inconsistently and a major outage was declared,” according to an April 22 email from Fairview Chief Information Officer Jacques Alistair, Tierney, and another Fairview vice president, Julie Flaschenriem.

The group email, addressed to the Physician and Ambulatory Informatics committee and Nursing leadership, among others, says “intermittent access problems” began “around 2:30 p.m.” It goes on to say that “[a]t 4:50 p.m. access to Epic was disabled for all users; for patient care, it was riskier to have inconsistent access versus no access to Epic.”

The problems began to get reconciled “at 6:30 p.m. and the last hospital finished their reconciliation processes around 11 p.m.,” the email continues.

In this episode, the “major outage” resulted in “access to Epic to freeze” — meaning doctors and nurses couldn’t open the software program they use almost constantly — according to an internal email, which also cites problems with “users’ access, inability to log in and system slowness.”

Tierney would admit as much months later in a Fairview document, which begins, “When systems — Epic or otherwise — are down, taking care of patients becomes more difficult.”

He continues: “IT fully recognizes just how disruptive outages are for everyone, especially to those providing patient care.”

An unreliable IT system raises the potential of compromised care, according to a former Fairview nurse, who worked for the hospital system for eight years starting in the mid-2000’s.

“In the [Epic] system, it does everything for us,” she says. “For the office visit, we enter in all the patient’s vitals, the history, the lab orders are ordered that way, any types of scans are ordered. It’s all electronic surgery scheduling.”

She gives the example of waiting for blood readings for enzymes on a patient who might have experienced a “cardiac event.” In that situation, there’s not a moment to spare.

“If you can’t get that reported to you right away,” the nurse says, “that the patient had a cardiac event — and the Epic is down, and you can’t see it and the person in the lab can’t see it — it wastes time. And it puts the life of the patient in danger.”

Adds a current Fairview employee, “However many years ago, everything was put down on paper. When there’s outages, every clinic, every hospital has downtime procedures when everything is written down by hand. So basically after the computer systems come back up, [staff has] has to go back and key in all that information. But if something gets missed, something gets thrown away, a paper gets lost, it’s kind of a bad deal.”

Hatch, who has reviewed some of the same internal documents obtained by City Pages, agrees.

“You’ve got a major hospital with 20,000 people working there,” he says. “You want to make sure everything is operating in the patients’ best interests. These communications and failures should raise concern.”

In recent months Fairview’s IT issues haven’t improved.

On September 1, “a major outage was declared” just after 9 a.m., an email written later that same day by Tierney acknowledges.

“I’d like to begin by recognizing and apologizing for the difficulties this — and all — system outages cause,” it says. “We know outages cause tremendous complications related to patient care and satisfaction, and for many of you, they make your jobs more difficult.

“Today’s event was a result of too much activity occurring on recently implemented storage system.”

Internal documents show the “event” started “around 8:30 a.m.”

Just after 9:00 a.m. that morning, Fairview IT cardiology manager Patty Vondlerstine wrote, “Users can’t access Epic,” tagging her email “High” importance.

The email chain in the ensuing hours instructs staff to contact Fairview “Operations” for any closing of departments such as “Clinics, OR’s, etc.” It also instructs Fairview’s pharmacies, “for patient safety, [that they] do not update medication records for patients who have moved location” since the outage began.

The issues lasted for hours. All Epic users weren’t granted full access to the system until 7:05 p.m. — more than 10 hours after issues were first reported — according to one of Tierney’s September 1 emails.

“Having to write everything down then input it into the system once it’s back up, I think, really opens you up for human error,” says the former Fairview nurse, who’s worked in the field for three decades. “You can’t order labs electronically so you have to pull a paper lab order sheet, write it down, send somebody to the lab to get this done. Then they’re writing this down. And you just hope everything will get re-entered the way it should be when it goes back up.”

The nurse calls her former employer’s IT problems “a huge deal” for those tasked with on-the-floor patient care.

“I can’t come up with a specific life-threatening situation off the top of my head,” she says, “but if you can’t verify who somebody is, their vitals, what medications they need, what the labs say, if somebody doesn’t get something they were supposed to or if they get something they weren’t supposed to, it sets you up for a huge liability and the possibility of a lawsuit.”

Fairview declined to get into specifics about its IT system and its outages during the past year. Camie Melton Hanily, director of communications and public affairs for the hospital system, sent the following statement in response to City Pages’ questions:

“Patient safety is always our top priority. Like other health care organizations, we have well established plans and processes for care continuity in instances when a particular tool or system is unavailable. It is not our policy to comment on specific patient or employee circumstances.”

Hey You Get Outta that Cloud- or at least ask some questions before you get hung up there

For those of you that went to the cloud, or are thinking of moving to the cloud, can you answer with certainty who owns your data? This isn’t a new question or new controversy, as big data gets bigger, as more data is stored in the cloud, as  more devices hit the market and as more hackers are getting into our banks and government servers  do you own your data and if not who does and where is it?

Over the past few years there has been controversy over the “Cloud” and who owns what data where.   For those willing to play “who’s data is it anyway”, the legal issues aren’t getting any clearer.

As a consumer user of the cloud -posting my so important pictures of my yellow lab Riley on Facebook

rileyhead Riley, the best dog ever!

or using my Gmail account or the obsessive habit of using my Amazon prime account so I can feel like a kid at Christmas every day seeing a box on my doorstep, I fail to realize the pain of this issue: probably because it’s so convenient. I then suggested the cloud as a solution for one of my customers.  That’s when it hit me.

As someone working in technology (ok I’m a sales geek) I need to really think about how real and complicated this issue is to better serve my customers by educating them in the pros and cons of using the cloud

For those thinking of going to the cloud, it seems like such and easy thing. So you call Mr. Cloud company and say “Mr. Cloud company, I want to put my data way up in the cloud so no one can get it.” Mr. Cloud says ok “we’ll store your data and all will be safe in the world forever and ever amen.” You sign the contract there, you’re in the cloud. You’re happy your data is safe, no one will ever get your data, you will have access to it at all times and you don’t have to hire and pay someone to support it.

What you don’t ask Mr.  Cloud company is “what is the trail of your cloud”? Why would you ask that? What is the “trail of the cloud” Clouds don’t trail! Have you ever looked up and seen those long skinny clouds? Yeah, they trail.

trailing-cloud

In some cloud companies you give your data to a cloud provider who then outsources its work to another storage or process provider, who’s responsible if your information is lost or damaged? What if that outsourcing happens in another country? So if data is created in one country, but then stored in another the legal rules that apply become blurred. YIKES!!

Now you worry about your data. You call an attorney. What area of law is this? Cloud law isn’t a thing……yet. There are 3 main areas of law (and maybe more) that cover this: Copyright, Confidentiality and Contract. So do you need 3 attorneys? Also if your data is stored or outsourced in another country do their rules apply?

lawpic

 

In speaking with a customer of mine from a University, he mentioned real concern about security in regard to the cloud. He mentioned his concern regarding student personal information as well as student loan information. Once student loan information is breached now we have a tax payer issue, and as he put it, “now we have a federal issue”.

There are guidelines from PTAC – the US Department of Education’s Privacy technical assistance center. But it gets a little “cloudy” regarding the cloud.

What it boils down to is Data Mining or Big Data.  In the education world to use as an example; this is a huge no no as it violates the “no commercial use of student data” policy.  According to Education Weekly when the litigation started in 2014, consent was not given to scan or index emails under the Google for education platform.

This issue isn’t any clearer in 2016 as UC Berkeley has this lawsuit pending for the same thing, called. “UC Berkeley students sue Google Alleging their emails were illegally scanned”.

I have only discussed a bit of the issue, but how about your industry? How about your data? Think about what you personally put out there? Your buying habits, your search habits. What about when you are in crisis? Is that something we want out in the “cloud”?

Should you decide to go to the cloud, read your contracts, ask some questions. Make sure the provider can specify who will be responsible for the data should it be lost or stolen. There should also be a provision in that contract as to who is responsible if the cloud company goes bankrupt, or is purchased by another cloud company.

If you need some help to get started in this process, help is here just give us a call.

www.velocitytechsolutions.com