FlexPod XCS Reviews

Our FlexPod solution is designed to isolate a sensitive data environment. We're able to use the technology to silo it away from the rest of our hosting environment.

1. Being able to have completely compatible hardware top to bottom and storage compute networking. This way, we're not spending time researching what works with what. 
2. The single call to support for any issue. We like to use the phrase, "One throat to choke."

Make it easier to refresh hardware. We got to the point where we couldn't fix vulnerabilities without refreshing the hardware, then that became a little too expensive for us to do.

We would like FlexPod to have in its roadmap: Keeping the hardware refreshed. It should be a little less expensive, not having all of the pieces go end of life at the same time.

The stability is great. We've had it for six or seven years, and it's still working.

It is absolutely resilient. It has built-in redundancy and multipathing. If there is an issue with one part, it doesn't fail automatically.

It's flexible. You can scale up or out. Our environment has never needed it, but the option has always been there.

The tech support has been great. Being able to have one call, whether it is to Cisco or NetApp. If there was ever an issue, they can hand the ticket over without us having to do it ourselves.

When it was bought, it was through a reseller (CDW). We go through them for a lot of stuff, and they are great. They are in touch with all of the vendors. We can go to them, then they can reach out and put us in touch with the vendors. They're a fantastic middleman.

The investment has been great. We put our money into it and it has given us something that we have been able to rely on for years.

We have been able to save time on new service deployments. When it was originally setup, it probably saved us several hours.

FlexPod is worth consideration. It's not necessarily something that you have to buy as a pod. You can buy the pieces individually, then get it classified. Anybody who is looking to consolidate physical into a virtual environment, it's great for that or any type of private hosting environment. It works really well.

The validate designs and overall versatility are some of the reasons that we decided to go with FlexPod. It's all been prevalidated, and we know it will work, which is valuable for us.

This solution is innovative when it comes to compute storage and networking. It comes back to the compatibility. Everything working from top to bottom has been great. Also, knowing the technology has been validated makes everything more streamlined.

I'm part of the managed services team, and our current FlexPod is a private cloud. However, FlexPod gives you the opportunity to keep it private, but at the same time, you have the ability to go hybrid, making it public. So, it's very versatile.

Despite our FlexPod being six to seven years old, it still works to this day. We do face some vulnerability issues that can only be fixed with a hardware refresh. Unfortunately, we went a different direction away from FlexPod. Everything had been great up until we had to do the tech refresh.

We use it for its resilience and redundancy. Storage-wise, we use it for its deduplication. The primary use is to keep storage for 24-hours and have no issues.

Our footprint is lower than it used to be.

• Replication
• Deduplication
• Inline dedupe
• Scalability
• Compression, which saves us a lot of data.

We would like to have more monitoring and reporting, because today some of the reporting, and if you purchase it separately is expensive. We use OnCommand Unified Manager today, which is great, but we are looking for more of that.

It is very stable and resilient with no downtime. If I needed any storage or shelf tomorrow, I would just add it to our cluster and there would be no downtime. This is one of the best things about FlexPod.

For example, if we have to add more storage, there is no downtime. If we upgrade any firmware, we do it without any downtime. Also, with a test environment, we can be up and running in a couple of minutes.

We can scale in a matter of hours.

I am happy with FlexPod's tech support. If we need support on it, we go to one place and get everything that we need in one shot.

The overall versatility and validate designs are great. We previously used a different platform, but we gained a lot of utilization with FlexPod.

The initial setup is straightforward. You just follow the steps. As long as you're not missing steps, especially as it is integrated with OnCommand System Manager or command line, the process is straightforward.

We use a NetApp reseller, who has great technical experts if we have any issues.

We have saved time and money for new service deployments. Without FlexPod, it probably takes about ten hours. With FlexPod, within two hours, we are up and running. So, we have seen about an 80 percent time decrease.

We are mostly a NetApp environment, so we did not consider another vendor. If there was an issue with NetApp, we would have left a long time ago.

Try it. Nowadays, they will give you access online to check it out and see how it works.

It is innovative because it integrates with different platforms.

We have seen an 80 percent increase in application performance.

FlexPod for Managed Private Cloud gives us what we need. We don't have any issues with it.

We are planning to eventually go to the cloud. So, the multi-cloud capability being there in the future is exciting.

There are very many features, but from experience, the single point of contact for support has been valuable to our customers. They do not have the headache of seeking support with multiple vendors. The single point of support supports hardware, hypervisor, and guest systems running in the FlexPod environment. (NOTE: Your level of support with the guest OS vendor such as RedHat MUST BE premium support, which means spending a little bit more on support, but total peace of mind.)

We don’t use it within our organization. We deploy this for other organizations. We can share the white papers once we are done with customers that we have successfully done this with.

It’s not the easiest solution to deploy and it doesn’t come integrated like VCE Vblock and HPE ConvergedSystem is (not really plug and play). To simplify what I mean, it’s like seeing a bed in the store; you buy it; and it’s delivered to your house with an easy-to-setup manual and the bed in different pieces with different sets of screws and you need to know what fits where.

If you have expertise setting up such environments, then you are good, but for customers or novices, it becomes a nightmare and stuff may actually be left out.

I have used this for the last four years.

I have not encountered any stability issues.

Unlike most converged and “hypercoverged” solutions, you can scale up or scale out on either compute, storage, networking, or ALL. Sometimes, the client just needs scale on one aspect; for example, add in storage. The design doesn’t change. With hypercoverged, you find you need extra compute, you need to add a whole node (compute, storage, and networking), which means you don’t have a cost-effective solution that offers a true ROI or, with VCE Vblock, you need another block.

Technical support is outstanding (10/10). Remember, you will pay a little more for additional support for your guest operating system and app vendors such as Microsoft Exchange, but in turn, total piece of mind.

We previously used VCE Vblock, VSPEX, Nutanix, SimpliVity, VxRail, HPE ConvergedSystem, and many others.

We still use some of these products, such as Nutanix, because of many advantages it has especially with new private cloud clients (VDI, virtualization, etc.) and service provider architectures, but the bottom line for a customer is:

• Price in terms of ROI and value for money.
• Flexibility: Does the solution scale both up and out considering the previous point?
• Ease of management, reporting, provisioning (relieving the headache for day-to-day management of infrastructure).
• Support that goes beyond their own boundaries.

I must say, initial setup is the only challenge for a new FlexPod deployment team but once you have set it up, it is very easy to manage and scale.

Pricing and licensing is very affordable from a FlexPod perspective. Bear in mind, for total peace-of-mind support, make sure the guest operating system and off-the-shelf application vendors have premium support so that you can integrate all their supports together into one.

We deploy cloud solutions. We constantly evaluate products.

Definitely ensure you size the environment correctly.

We use on-prem for our unified network communications solution, which is basically a Call Center. We have a Cisco call center running Cisco software on FlexPod. That is our primary and only use case right now for it. We use HP as a computer hardware solution for the enterprise. We'll probably expand our use of FlexPod and we use it for everything else including the entire production environment.

The components we are currently using are Cisco Blade Server Chassis. The blades are M4, Fabric Interconnect 6248, I believe. The very first version. They are very old. We'll probably be upgrading in a few years, or maybe next year. We have MDS switches, 9148 — they are pretty old too. We will have to upgrade them. And then we have NetApp FAS8040 disks.

This is a solution that's designed by Cisco and NetApp together. We have our reference designs, so we pretty much can skip the part where we need the implementation delivery. We can work directly with the vendor like NetApp, and tell them "We need this, and here is your validated design." But that's what helps right there.

The most valuable feature of the solution is the flexibility in configuration and the setup. We have not had any major issues so far.

I would like to see the FlexPod infrastructure get updated more often. Things like the firmware, the software packages, and the compatibility matrix have to update more often and seem to lag in development. We are kind of dragging on this. Because we were not performing all those updates more often, we are kind of delayed a few years.

For example, we are using the FlexPod for the Call Center and Call Center has specific versions of their software. Cisco has recommended that we use NetApp version 91 as the final version for the Call Center software. We can not go beyond 91 and 91 is a two-year-old OS. There have 96 out already and will probably be presenting 97 in a few days. So we are literally lagging behind by years.

We had a problem with the stability at one point. We found it through Cisco tech support while they were helping us to troubleshoot this solution. We found a drop in the Fibre Channel frames. This drop occurred on the HBAs (Hot Bus Adapter) of the NetApp controller. But we had to troubleshoot every step of the way to figure it out. Because basically what was happening was that the Call Center virtual machines were crashing.

However, the hardware stability is very good. We have no hardware issues on the server site.

The scalability of the solution is brilliant and works great. With the fabric interconnects, you can scale it horizontally. I don't know the actual stats, but I believe with the newer fabric interconnects, you can scale indefinitely pretty much.

We did not purchase this product with the support contract, but I would like to try it. So our next purchase would be for sure including that part number for the FlexPod support.

When we went to invest in a new solution we were looking for ease of implementation, peace of mind, future reference, and stateless design. That is the ability to take out any hardware piece, replace it with a new one, reboot and — boom — ready to go.

The initial setup was very straightforward, but you have to know the integration document for the Cisco validated design. It is a must to know. Basically, the guy who performs the implementation should know it like the bible. It is actually the bible for those guys, and for the people who are using it. Those validated designs make our engineer lives, much easier.

We did involve a reseller. I can't mention them by name but they are a very well-known company. The process was great. They set up everything, and they are still helping us with the architecture and the new features, and so on so forth. As a matter of fact, they would be helping us updating firmware on the fabric interconnects in the next few months.

Our return on investment is that the solution has made us more efficient. It is a ready-to-go solution. It's like you can start using it almost within a week of the delivery.

Our company makes purchases like CapEx (capital expenditure). So we do not have to maintain the license.

Because I don't have experience with the Cloud integrations, that did end up affecting choices initially. For now, we are going to be staying with NetApp but we are also looking at other products like Pure Storage and Nimble.

The solutions that validate the designs for major enterprise apps is a nice thing to have because there are many components. For a single person or even for an entire IT department, it will be impossible to correlate software versions, hardware versions, firmware versions, and everything else. It's a huge matrix.

The vendor has to provide the compatibility matrix, obviously and has to provide the complete vertical to give those numbers, per each component, for all software, and for the firmware. The customer can't figure it out by themselves. So, and that's the reason for the FlexPod, so they can buy and integrate everything together.

We are not on the Cloud yet at all. I would say we'll be looking into it when it's time because I understand this is inevitable. So we understand the push on us into this territory, and I know it is all about the Cloud now. A few years ago it was completely different. Now, it is all Data Fabric Cloud, Azure, and Usenet.

The product has decreased the unplanned downtime incidents in our organization. Even the incident I mentioned about the crashing virtual machines was identified and solved in one day.

On a scale of one to ten where ten is the best, I would rate FlexPod as a ten. I love it all. I could give it an eleven. 

My advice to people considering the solution is simple: read the Cisco validated design, remember it, and use it. It is a must to have and must to know, and must to use.

For both data centers, everything that we use IT-wise is run on both of them.

We currently use versions 4.1 and 3.2.

The solution simplifies infrastructure from edge to core to cloud. It makes supporting it, troubleshooting it, and documentation a lot easier. Time to resolving a problem goes down quite a bit as well.

The most valuable feature is the one call number for support and the fact that all the documentation comes with it. They have all of the preprepared plans for the deployment model and we can just choose which one we want for VMware, etc. The hardware is all listed. We buy that and away we go. It's called validated design.

The validated design is nice if we have issues with anything. We can call the vendor, or if anyone says anything, we can say, "Well, we're already running by the certified design to the verify design. We're not doing anything out of the ordinary." It makes support a lot easier.

The solution’s validated designs for major enterprise apps in our organization is very important because of the whole troubleshooting problem, or if we run into any supportability problems. We say, "We've done what was asked of the company. It is all verified. We shouldn't have any funny things happening." As for management, if they come down and ask questions, we can say, "We're following best practices."

Mainly, the interface needs improvement. I'm not a big fan of UCS Manager, sometimes. I believe they released the new one, and it seems like in every version something changes and something else doesn't work. When they switched to HTML5, I believe we had issues in version 3.2. They fixed it in the next version. The amount of work to upgrade a system for change control is tedious to have issues every time. I would recommend more regression testing, then testing the different browsers in that.

The solution decreased the unplanned downtime incidents in our company because it's newer, more supported. HPE had a lot of bugs in the system. Our guys would go to make a change, then all of a sudden, they would run into a bug. Next thing, we are down. There is a lot better documentation and support behind the FlexPods.

We haven't had any issues yet. Anytime that we've ever had to scale, we just add another blade chassis, and away we go. We throw in more blades. It is very easy. We reuse all of our templates for that. So, it is very quick to deploy new hardware.

The technical support is the best out of all the ones that I'm responsible for calling, e.g., compared to Dell EMC. We've had issues with Dell EMC in the past, HPE as well. 

Anytime that I have called NetApp, they have an answer right away. Before with Dell EMC and HPE, we've been bounced around in their Tier 1 and 2 before you get to talk to someone who knows what is going on. That doesn't seem to happen with NetApp, or if it happens behind the scenes, we don't see it.

The solution’s unified support for the entire stack is very important to us. When we have trouble with Fibre Channel or networking, it's just one number to call. You get someone who knows the whole stack versus having to chase down Brocade, Cisco, or NetApp.

Before, we used to run on HPE Blade Centers, so we had a networking guy, an HPE Blade Center guy, and a VMware guy. Using UCS and FlexPod, we now have two people at the company who run that whole stack, so there is no finger-pointing. It eases a lot of troubleshooting, because it's just two people versus multiple teams.

It has improved the application performance in our company. For us, it was about replacing old hardware with new hardware. The application performance was slow before, and it is better now.

It was straightforward because I have done it multiple times before. I've had to do it probably four times now. Now, I just know what I need to do versus the first time I had to it. We worked with a reseller and basically read all the documentation the first time.

The process for deployment is rack and stack, then upgrade to the latest firmware. We go through all our templates and gather what we're currently using compared to what the latest version of UCS offers. We make any updates, as necessary,  then reconfigure, redeploy, and away we go.

We did it ourselves.

We went from two racks down to one at one location. We stayed the same at another location. Power-wise, we never really paid attention to it. With cooling, there is less hardware.

The solution has saved our company time.

We decided on NetApp mainly cost because of cost and the fact that we already have the in-house knowledge and expertise. Therefore, it just made sense to stay within the ecosystem we were in.

Usually, we have a look at other vendors, like Dell EMC and HPE. However, currently, it was based on the time cycle of the hardware refresh. It made sense to just go with what we already had.

We are looking at going down the next refresh with NVMe, and NetApp is the only one who offers that end-to-end solution.

I would rate the solution as an eight (out of 10). There is always room for improvement, but it's the best technology that I have used so far.

Genuinely have an understanding of where you want to go. We've had issues before at other companies where people like a hardware. Don't look at the hardware. Instead, look at what you want to do, then work backwards.

Right now, all of our needs are currently being met. I know we're going to move towards NVMe with the one data center once we update. However, that is pretty much the newest thing on the radar for me.

We use this solution for the VMware virtualization of all of our servers. We use the Cisco UCS for the blade servers.

From a server storage side, we were previously using the HP BladeSystem c7000 chassis for our blade servers. It was much harder to update the firmware when compared to the Cisco UCS.

From a UCS side, it is very simple to go from an ESXi host that is on an M4 blade and switch it out to an M5 blade by changing the service profiles on the blades. It is very easy and quick. 

The procedure for contacting technical support could be simplified.

UCS has been around for, I'm assuming, about ten years, and it has only gotten better with time. I like it.

Scalability is very easy. You just scale up or scale down, whenever you want.

When dealing with technical support, which was not often, it was tied to our account. That was difficult because I had to go through a partner to find out what our accounts were before I could get support. I wish that part was a little easier.

Our previous solution was an HP c7000 BladeSystem with 1-gigabit passthrough modules, and we were going to a 10-gigabit solution. We wanted something that was easier, better, and would support 10-gigabit. We actually ended up going to a 40-gigabit solution.

The HP solution, HP Virtual Connect Flex-10, only supported 10-gigabit modules.

The initial setup was pretty much straightforward.

We have deployed two different systems. The first one was by CDW, which went perfectly well. The second one was by Precidia, which also went perfectly well. Both of these resellers knew what they were doing and everything went smoothly.

We only looked at Cisco at the time.

My advice for anybody who is considering this solution depends on what they're going with. If it is the converged infrastructure then the UCS is probably the way to go. If instead, they are going with the hyperconverged infrastructure, then I would suggest going with the HyperFlex solution.

The biggest lesson that I have learned from this solution is the ease of actually setting it up and learning it.

I would rate this solution a nine out of ten.

We have been using FlexPod for five years. We use it for our Epic environment.

We get a unified, collaborative support model. It conforms with the CVD and it helps us with maintaining supportability. All of our vendors give us the support that we need in a timely and effective manner.

It has also been very versatile. We have others that do not exactly conform with it and yet we still benefit from the collaborative support model. And we're not required to go to a certain thing if it doesn't work well or isn't the best case for our situation. That's been wonderful.

We're using the Epic environment on-demand workflow, and that has saved us quite literally thousands of man-hours by helping us refresh, back up, and create new instances. We wouldn't have been able to do so if it wasn't for all of that time-saving. Being able to have SUP, REL, and REL VAL DR instances, we would need to double our staff, at least, to be able to do that.

• The design has already been validated.
• The support element, the lack of finger-pointing, where all of the different vendors are working together collaboratively, sharing data, opening tickets with one another.
• We already use UCS and it goes well with the vendors that we have picked.

At the beginning, there was a little bit of confusion among the support folks on how to open up tickets with the others. There needs to be a little more helping of the partners to make sure that they are able to handle opening tickets with the other vendors.

For our Epic environment, aside from an environmental data center problem, a heating issue, we have not had any issues whatsoever with the infrastructure. From a resilience perspective, we've set it and we've been able to forget it for the most part.

We planned a five-year model for our Epic environment, so that we haven't really needed to scale. For other areas of our environment, it has scaled fairly well. The data mobility helps a lot with that, if we have to do a refresh. It's just simply vol moves, etc.

One thing to note would be that we're now looking to go into a MetroCluster IP with our FlexPod. Going from Fabric to IP, we're not able to do that with vol move. Still, we can do SnapMirror relationships and get all that data moved over.

The one other thing would be that in the transition from 7-Mode to CDOT there was no unplanned downtime, and it went very well with all the tools that NetApp has provided us.

Just fantastic tech support. The chat functionality gets us in touch with top-level engineers when we need it. As a hospital, that level of support is priceless.

We had been using Dell EMC storage before, VPLEX, etc. We had a good bit of experience with that. 

Because our VAR helped us with a lot of it, from our perspective it was very simple.

Initially, for our Epic environment, we used OST. Our VAR, PEAK Resources, also specializes in converged infrastructure. It went fairly well. It was a little bit rocky at the start, but that's why we picked our VAR. Their experience and the level of investment that they make in NetApp are fantastic. They helped us a lot.

We looked at Dell EMC VCE very seriously, as it's a converged product. NetApp was a lot more flexible, it didn't require a forklift approach. We had a really great experience with NetApp specifically. We were already using Cisco, for both network as well as compute, and it just seemed like just a great play, to have that flexibility and to have the support model to help us. And it has proven to be great.

If a colleague was looking at this or similar solutions, I would help them to understand what we've done with it for Epic and the success that we've had. I would share with them the examples of converged support as well as the stability that we've had. They are what has really made this a success story.

Regarding private, hybrid, and multi-cloud environments, I love it. The idea of the state of Fabric. We haven't been able to leverage the public cloud portion of it yet, but the whole vision of the data movement is where we want to stay, so that we're ready for the cloud where we can do that. As for private, we're looking to bring up StorageGRID to be able to offload cold blocks on our AFF. That kind of a feature set is wonderful. We don't use FlexPod for managed private cloud.

In terms of FlexPod being innovative when it comes to compute, storage and networking, it stays current. We're not five versions back because we're having to be conformed with other solutions. It seems like NetApp is doing a great job of making sure all their vendors are keeping things up to date. There have been some other than Day One-types of events that it's impossible to really get to. We're not waiting long for things to come up.

As for improvement in application performance, we started with an All Flash Epic so we've had really wonderful sub-millisecond latencies from the get-go. We haven't experienced degraded performance.

We are using it in two cases. 

1. We are using it for a database solution, so we're moving all of our 12G database systems onto Oracle UCS with flash as the hot store. We are still using spinning disk as the cold store. Initial tests are going really well. We're actually moving our first big load on in a couple of weeks. 
2. We are using Oracle VM Citrix-based hypervisor, full solution, with FlexPod as the repo on the back-end. All the guest nodes are running on UCS B200 M4s. We have A700, A300, and A200 on the back-end for various slavers and pulls, and they are all working great.
   

We just put our first full workloads on it about a month ago. Since then, everyone has been saying after booting it, this was their fastest startup ever in that environment. Thus, it is working well so far.

For the management side of our UCS, it is a single pane of glass for multiple people, whether it is data center, sysadmin, or server deployment. 

On the UCS side for server profile, there is a type of layer of abstraction from the actual hardware. It is a lot easier to do hardware replacement, as long as you are Fibre Channel booting, you can just replace hardware which breaks and have things come right back online.

Flash obviously just adds speed and bandwidth to everything.

The most valuable features in a data center, or parts of it, are footprint and power consumption. The flash side is the first time that Moore's Law breaks. It gets smaller, and also takes less power.

We have had a bit of struggle on the support side. 

I am not looking into the next iterations of it yet, because we are still standing up some parts of what we have now. 

I would like to see the partnership with NetApp and Cisco continue. We have been a NetApp shop for a long time. We have seen partner agreements between NetApp and tech companies fall apart over time. They were with Hitachi for a while, then 3Par for a while, and so on. However, we have a lot vested in Cisco and NetApp now. We would like to see the Flexpod service agreement strengthen as we continue to benefit on the customer side.

We like NetApp and Cisco. I do not want to have to figure out how to make either of them work once they have decided to part ways. Therefore, it is important to us that they hold together.

We are pretty new, but so far, we have not had issues on the flash side. We just moved our first production workloads onto it about six to eight weeks ago. We did not find MTBF early.

We purchased what we needed, so there is some room for scalability. We went big with the A700, thus we are using the 15TB SSDs. So far, so good.

From a customer perspective, it has been up to us to make sure that we get both NetApp and Cisco involved when we have had an issue. Unfortunately, we cannot just contact one side and they contact the other vendor to sort of work it out in the background, leaving you in the loop. We are interested in what is going on, but we have felt that we have had to pull the two sides together to make the FlexPod service side of it work to our benefit.

They have worked with us to sort things out. Admittedly, we are running the bleeding edge of things. We are using Oracle UEK, which was not necessarily on either side's support matrix right upfront. Therefore, we have had some issues getting both sides' corporate to play nice. Eventually, it did get sorted out, and we are getting these things resolved.

We previously worked with physical servers. We had a lot of HPE c7000 class. We started with RLX, which was pre-HPE. HPE bought them. 

We have played with the P class and C class, doing a couple different proven concepts along the way. We had Dell and Cisco, and some other people all come in, and they taught their stuff.

This time around, from the managed solution side of it, Cisco is what sold us. Hardware is hardware, but how you put the solution together was the selling point for us. To be able to get something saying, "You build it like this." Not, you have a bunch of parts, what do you want to do with it? This is what a lot of the other vendors are still doing. They are tailoring hardware to your workload after you have bought hardware. 

As opposed to finding the solution you need, helping you build it upfront, presenting the hardware and dock, then showing you how to build it. This is what is nice for us. While a little rough, once you have built it, the support matrix says, "Here is newest version of firmware. Here is newest version of something else." Then, off you go. They do not necessarily take all iterations of change back to the lab to recertify them. 

You work with the vendor partnership to keep you in compliance going forward. This is one of the reasons that I want to see the NetApp and Cisco partnership hold together. Otherwise, we are right back where we were before. NetApp has an update and Cisco has an update, and neither one of them have tested it with each other. 

Setup was pretty easy. This was our first venture into UCS at all. It was a steep learning curve figuring it out. We are using Central to manage six different domains, so getting that hierarchy put together upfront so we could do global templating across all those domains was a little rough, mostly from a conceptual standpoint. It deploys easily now that we have got it out.

We have been a NetApp shop forever, so that part was a piece of cake.

Make sure that you engage as much with whom you are buying from as a partnership, not just as a purchase. 

We did a PoC with four different vendors to test out combinations of a hardware build. Storage was kind of a given as we have been a NetApp shop forever. We have gone through a couple other ones. We really like 3PAR, but that's a different story.

The hardware has been rock solid so far. It has gone up easy. It runs well. We have not had issues with it.

Pay attention to what you need upfront as you are building it. Know the workload that you are trying to solve with it. Make sure you are buying for performance, not just capacity. 

Most important criteria when selecting a vendor: It is important that they care about the business that we do after sale. It is one thing to get a quote, obtain the parts, and make sure you have all the right things upfront. Your business is going to change the next day, especially for a business like us. We are in a multi-customer type of environment where somebody will have a new bright idea tomorrow. Therefore, we need to be adaptable. It is important to have a partnership with the people that we purchase from. Thus, ongoing modifications can continue to be part of the conversation, not just, "I sold you something. Let me know when it is time to renew your contract."