Posts Tagged ‘storage system performance’

How to diagnose hoop array controller opening problems

Friday, October 14th, 2011

Most of a importance around improving a customer’s storage complement opening is placed on the
disk drives themselves. The standard solutions to a opening problem are to possibly supplement drives to
the RAID organisation or supplement solid-state hoop (SSD) to a system. The plea with those approaches is
that during some indicate in a expansion of a storage system, a controllers turn a bottleneck.
In that case, it doesn’t matter how quick a drives are or how quick a storage network is, a disk
array controller
can’t keep up. So when creation improvements to a storage complement and storage
network, to make certain your business get a opening formula they expect, it’s vicious that
the storage controller be evaluated to make certain it can hoop all of a surrounding upgrades.

The hoop array controller typically has 3 primary components. This initial is a CPU that
processes all a information that is sent to it and that it stored on a storage system. Twenty years
ago, this estimate was comparatively simple. Today, it’s not so simple, with some-more sophisticated
storage systems and loads of storage-centric services.


To continue reading for free, register next or login

second member of a storage controller is a I/O ports. These are a connectors that
are receiving and promulgation information to a connected hosts and a connectors that are promulgation and
reading information from a storage subsystem. The accessible bandwidth that these connectors support
also plays a means in how quick a controller can expostulate a underlying hoop media.

The final member is a program executed by thecontroller’s
processor. Thanks to advancements in technology, a storage controller is obliged for an
increasingly formidable series of functions. Each use supposing by a controller’s software
consumes some-more and some-more of a processor’s resources.

Disk array controller bottlenecks

At a simplest level, a hoop array controller handles all a volume government functions,
removing that shortcoming from a hosts that bond to it. This includes creation multiple
independent hoop drives demeanour like one, as good as a now-expected
protection such as RAID 1 (mirroring,
RAID 4, RAID 5 and RAID 6). The idea of these services was to make a government of storage easier
on a connected horde so that it can spend a estimate energy elsewhere.

In a past dual decades, storage systems have turn some-more intelligent, holding on additional
storage chores—such as snapshots,
and automated
—each adding to a weight on a controller.

For business experiencing storage opening problems, it’s not protected to assume that we can
just sell SSD to them and a problem will go away. The storage controller could be a source of
the I/O problem. To establish either a controller is to blame, there are dual areas to consider.
The initial is a CPU function of a storage controller. Most storage complement program will
report this information, though customarily not with a fact we need. Rather, we need a average
utilization metrics over a given time duration and within a specific time window. A news of the
average CPU function of a controller that factors in 9 p.m. to 6 a.m., when no one is in the
office, will askance a results. To get a information we need, demeanour to modernized storage management
software accessible from third parties—providing we another event to supplement value.

The second area to inspect is I/O function to a controller from a hosts and from the
controller to a storage shelves. This information competence be harder to get. Some manufacturers make
this info available, though many do not, in that case, we competence need a third-party software

In both cases, a program apparatus should be means to indicate out when a hoop array controller
either has extreme CPU function or if one of a I/O bandwidths is being exceeding. At this
point, problem showing becomes a math issue, subtracting a intensity CPU ability or I/O
bandwidth that is accessible from what is being demanded. If a outcome is reduction than 10% headroom at
peak periods, afterwards we should demeanour during a intensity causes.  

If it turns out that your patron does have a storage controller bottleneck, there are three
possible causes. The initial means of a storage controller bottleneck is too most ability behind the
controller. Most storage systems explain a support expostulate count that is about half of what the
controller can support while underneath full load. They are banking on a storage being idle often
enough that a storage complement services (snapshots, skinny provisioning, etc.) can fast locate up
even if there are too many drives in a system. If a storage complement is going to be heavily
utilized and all a drives in a complement need to perform during a highest-possible rate, a system
might not scale to anything tighten to what a leaflet indicates.

Another probable means of a hoop array controller bottleneck is too many services being
performed. The list of functions that a storage controller performs in a complicated storage complement is
pretty endless compared with 20 years ago, with some-more on a way, such as deduplication and
compression. In an active system, this competence only be too most shortcoming for a controller.

The final means of a bottleneck is increasing pointless I/O interjection to server virtualization. Before
server virtualization, any joining horde had one application, and oftentimes we would run out
of storage network ports before we could surpass a opening of a storage controller.
Nowadays, any joining horde competence support dozens of workloads, any with a possess I/O pattern.
This means that any horde now has a solid tide of pointless I/O that creates a storage controller
work harder to find a information that any practical appurtenance is requesting.

How vendors are addressing a problem

Storage vendors have a far-reaching accumulation of solutions to this controller bottleneck problem—from
increasing tender estimate power, building specific CPUs to hoop specific services, scaling out
storage or offloading storage services to a practical infrastructure or hypervisor. In a next
article, we will fact these intensity solutions so we can improved position them with your

George Crump is boss of Storage
, an IT researcher organisation focused on a storage and virtualization

This was initial published in Oct 2011

Article source:

Storage opening management: 10 storage complement opening tips

Thursday, August 18th, 2011

What you’ll learn in this tip: When it comes to storage opening management, there’s no tip recipe for present success.
But there are intelligent ways to proceed a problem. Read these 10 tips to find out how things like
continuous information insurance (CDP), active multipathing and infrequently simply doubt a number
of queries users contention can make a disproportion in your shop.

Given a choice between fine-tuning information storage for ability or for performance, many data
storage managers would select a latter. Tips and tricks to boost storage speed are common, but
they’re not all equally effective in any environment. A accumulation of products and technologies do
have good intensity for many shops, from optimizing server-side entrance to improving the
storage-area network (SAN). We’ll demeanour during some effective, nonetheless mostly overlooked, methods to speed up
storage complement performance.

Networked storage is impossibly complex, requiring a diverse

set of hardware and software
elements to interoperate smoothly. Not surprisingly, one of a many common causes of delayed storage performance is a misconfiguration or tangible disaster of one or more
of these components. Therefore, a initial place to demeanour for softened opening is in a existing
storage I/O stack.

Check server and storage array logs for signs of earthy faults; I/O retries, trail transformation and
timeouts along a organic couple are certain signs. Try to besiege a unwell element, nonetheless start with
cable-related components. Flaky transceivers and cables are common, and can severely impact
performance while still vouchsafing things run good adequate to go unnoticed. These apparatus mostly fail
after being physically disturbed, so be generally observant after installation, emigration or
removal of information core equipment.

1. Freshen firmware and drivers

Manufacturers are constantly regulating bugs, and new capabilities can hide in with software
updates. It’s scold to stay on tip of driver and firmware
updates for all components in a storage network, with scheduled and active testing, tuning and
upgrading. Microsoft Corp. and VMware Inc. have been actively adding new opening facilities to
the storage stacks in Windows and vSphere, mostly nonetheless many fanfare. SMB 2.0 and 2.1, for
example, dramatically accelerated Windows record sharing, generally over slower networks. Updates to
NTFS and VMFS have also
routinely softened opening and scalability. Stay tuned to storage blogs and publications to
keep on tip of these developments.

But we should note that not all updates are value a time and effort, and some can be
downright perilous. Make certain your pattern is upheld by all vendors concerned and has been
thoroughly tested, and never use beta formula in production. As a systems administrator, we tend to be
conservative about what we hurl out, watchful for reports from others before holding a plunge

2. Question a queries

Most of a tips concentration on locating and expelling bottlenecks in a storage stack, nonetheless one should also cruise shortening the
I/O bucket before it’s created. Working with database administrators (DBAs) to balance their queries for
efficiency and opening can compensate off vast time, given a reduced I/O effort advantages everybody and
every application.

3. Break down backup bottlenecks

Traditional backup applications are intensely fatiguing on storage resources, transfer massive
volumes of information according to a daily and weekly schedule. Improving a performance of backups so they can fit within their reserved “window” has
become a priority for information insurance pros, nonetheless a techniques employed can assistance urge altogether data storage performance as well.

One effective routine to revoke a backup break is to widespread it out regulating continuous information insurance (CDP) technology. Built into many products intended
for practical servers, CDP ceaselessly copies information from a server rather than collecting it in a
single, strong operation. This is generally profitable in virtual appurtenance (VM) environments given a nightly backup “kick off” across
multiple guest can vanquish storage responsiveness, from a train to a horde train adapter (HBA) or
network interface label (NIC) to a array. Microsoft and VMware also have technologies to offload
backup-related snapshots to storage arrays that are softened means to hoop information movement.

4. Offload practical appurtenance I/O with VAAI

The recover of VMware vSphere 4.1 enclosed many new features, nonetheless one of a many vicious was
the vStorage API for Array Integration (VAAI). This new interface allows VMware ESX to coordinate certain I/O tasks with upheld Fibre Channel (FC)
or iSCSI storage systems, integrating a hypervisor and array to work some-more closely and effectively

VAAI includes 3 “primitives,” or formation points:

  1. Unused storage can be expelled for thin provisioning regulating a fit “write_same” SCSI command, increasing
    capacity function and shortening I/O overhead.
  2. Snapshot and mirroring operations can be offloaded to a storage array,
    greatly shortening a network, hypervisor and handling complement I/O workload.
  3. Access locking can take place during a turn some-more granular than a whole LUN, shortening contention
    and wait time for practical machines.

Although nothing of these screams “storage opening tuning,” a net outcome can be a dramatic
reduction in a I/O effort of a hypervisor as good as reduction trade over a SAN. Analysts
expect serve improvements (including NFS support) in destiny versions of VMware vSphere, and one
imagines that Microsoft is operative on identical formation facilities for Hyper-V.

5. Balance practical appurtenance I/O with SIOC

While not a opening acceleration record per se, VMware vSphere Storage I/O Control (SIOC) is a “quality of service” mechanism
that creates I/O performance some-more predictable. SIOC monitors a response latency of VMFS
datastores and acts to stifle behind a I/O of lower-priority machines to say a performance
of others. In practice, SIOC reduces a impact of “noisy neighbors” on prolongation virtual
machines, improving their responsiveness. This helps keep concentration developers and managers
happy, bringing a coming of softened opening even nonetheless sum throughput stays the

What doesn’t work

Besides looking during how to ratchet adult storage network performance, we also need to cruise some
not-so-effective approaches to improving performance. Testing can exhibit engaging outcomes:
Enabling jumbo frames on Ethernet networks, for example, typically hasn’t yielded many of a
performance benefit.

One common doubt relates to a merits of several storage protocols and a common belief
that Fibre Channel is inherently faster than iSCSI, NFS or SMB. This isn’t a box generally,
although implementations and configurations vary. Similar architectures furnish identical levels of
performance regardless of protocol.

One should also be discreet about contracting “bare-metal” technologies in practical environments,
including paravirtualized drivers, proceed I/O like VMDirectPath and tender device mapping (RDM). None
delivers many opening alleviation and all meddle with fascinating facilities like VMotion.

6. Streamline a server side

Today’s multicore servers have CPU energy to spare, nonetheless network interface cards (NICs) and HBAs have traditionally been sealed to a
single processor core. Receive-side scaling (RSS) allows these interface cards to distribute
processing opposite mixed cores, accelerating performance.

Hypervisors face another charge when it comes to classification I/O and directing it to
the scold practical appurtenance guest, and this is where Intel Corp.’s practical appurtenance device queues
(VMDq) record stairs in. VMDq allows a Ethernet adapter to promulgate with hypervisors like
Microsoft Hyper-V and VMware ESX, organisation packets according to a guest practical appurtenance they’re
destined for.

Technologies like RSS and VMDq assistance accelerate I/O trade in perfectionist server virtualization applications, delivering extraordinary levels of performance.
By leveraging these technologies, Microsoft and VMware have demonstrated a correspondence of
placing perfectionist prolongation workloads on practical machines.

7. Get active multipathing

Setting adult mixed paths between servers and storage systems is a normal proceed for high
availability, nonetheless modernized active implementations can urge storage opening as well.

Basic multipathing program merely provides for failover, bringing adult an alternative
path in a eventuality of a detriment of connectivity. So-called “dual-active” configurations assign
different workloads to any link, improving function nonetheless restricting any tie to a single
path. Some storage arrays support trunking mixed connectors together or a full active-active
configuration, where links are many-sided and a full intensity can be realized.

Modern multipathing frameworks like Microsoft MPIO, Symantec Dynamic Multi Path (DMP) and VMware
PSA use storage array-specific plug-ins to capacitate this arrange of active multipathing. Ask your
storage businessman if a plug-in is available, nonetheless don’t be astounded if it costs additional or requires a
special craving license.

8. Deploy 8 Gbps Fibre Channel

Fibre Channel throughput has ceaselessly doubled given a initial 1 Gbps FC products appeared,
yet retrograde harmony and interoperability have been confirmed along a way. Upgrading to
8 Gbps FC is a elementary approach to accelerate storage I/O, and can be remarkably
affordable: today, 8 Gbps FC switches and HBAs are widely accessible and labelled approximately the
same as common 4 Gbps parts. As SANs are stretched and new servers and storage arrays are purchased,
buying 8 Gbps FC rigging instead of 4 Gbps is a no-brainer; and 16 Gbps FC apparatus is on the

Remember that throughput (usually voiced as megabytes per second) isn’t a usually metric of
data storage performance; latency is usually as critical. Often gifted in terms of I/O operations
per second (IOPS) or response time (measured in milliseconds or nanoseconds), latency is a speed
at that particular I/O requests are processed and has turn vicious in virtualized server
environments. Stacking mixed practical servers together behind a singular I/O interface requires
quick estimate of packets, not usually a ability to tide vast amounts of consecutive data.

Each doubling of Fibre Channel throughput also halves a volume of time it takes to routine an
I/O operation. Therefore, 8 Gbps FC isn’t usually twice as discerning in terms of megabytes per second, it
can also hoop twice as many I/O requests as 4 Gbps, that is a genuine bonus for server

9. Employ 10 Gbps Ethernet

Fibre Channel isn’t alone in cranking adult a speed. Ethernet opening has recently jumped by
a cause of 10, with 10 Gbps Ethernet (10 GbE) apropos increasingly common and affordable, nonetheless 10
GbE storage array accessibility lags rather behind NICs and switches. Environments regulating iSCSI or
NAS protocols like SMB and NFS can knowledge large opening improvements by relocating to 10 Gbps
Ethernet, supposing such a network can be deployed.

An choice to end-to-end 10 Gb Ethernet is trunking or fastening 1 Gbps Ethernet links using
the couple assembly control custom (LACP). In this way, one can emanate multigigabyte Ethernet
connections to a host, between switches or to arrays that haven’t nonetheless been upgraded to 10 GbE.
This helps residence a “Goldilocks problem” where Gigabit Ethernet is too delayed nonetheless 10 Gbps Ethernet
isn’t nonetheless attainable.

Fibre Channel over Ethernet (FCoE) brings together a Fibre Channel and
Ethernet worlds and promises softened opening and larger flexibility. Although one would assume
that a 10 Gbps Ethernet links used by FCoE would be 20% faster than 8 Gbps FC, a disproportion in
throughput is an considerable 50%, interjection to a some-more fit encoding method. FCoE also promises
reduced I/O latency, nonetheless this is mitigated when a overpass is used to a normal Fibre Channel
SAN or storage array. In a prolonged term, FCoE will urge performance, and some environments are
ready for it today.

10. Add cache

Although a quickest I/O ask is one that’s never issued, as a means of speeding things up,
caching is a
close second. Caches are appearing via a I/O chain, earnest softened responsiveness by
storing frequently requested information for after use. This is frequency a new technique, but
interest has strong with a appearance of affordable NAND flash

There are radically 3 forms of cache offering today:

  1. Host-side caches place NVRAM or NAND peep in a server, mostly on a high-performance PCI Express card. These keep I/O off a network nonetheless are usually useful on a
    server-by-server basis.
  2. Caching appliances lay in a network, shortening a bucket on a storage array. These serve
    multiple hosts nonetheless deliver concerns about accessibility and information coherence in a eventuality of an
  3. Storage array-based caches and tiered storage solutions are also common, including NetApp’s
    Flash Cache cards (formerly called Performance Acceleration Module or PAM), EMC’s Fully Automated
    Storage Tiering (FAST) and Hitachi Data Systems’ Dynamic Tiering (DT).

Still no china bullet

There are many options for improving storage performance, nonetheless there’s still no singular silver
bullet. Although storage vendors are discerning to explain that their latest innovations (from tiered storage to FCoE) will solve information storage opening issues, nothing is
foolish adequate to concentration on usually one area. The many effective opening alleviation strategy
starts with an research of a bottlenecks found in existent systems and ends with a devise to
address them.

BIO: Stephen Foskett is an eccentric consultant and author specializing in enterprise
storage and cloud computing. He is obliged for Gestalt IT, a village of eccentric IT
thought leaders, and organizes their Tech Field Day events. He can be found online at, and on Twitter during @SFoskett.

This essay was creatively published in Storage magazine.

This was initial published in Aug 2011

Article source:

VDI benchmark launched to magnitude storage performance, costs

Wednesday, August 3rd, 2011

Analyst organisation Evaluator Group currently launched a VDI-IOmark, a practical desktop infrastructure (VDI)
for storage. The benchmark measures I/O
and costs per desktop for VDI

Evaluator Group comparison researcher Russ Fellows pronounced a thought for a benchmark was innate about a
year ago when vendors began edition costs per desktop for their storage when used with VDI. He
said those total were not formed on customary testing, and there was no approach to determine a claims or
compare opposite vendors’ offerings.

“VDI deployments are removing derailed since of costs and I/O
,” Fellows said. “There’s probably no picturesque sizing for performance. “We started
getting inquires from finish users on VDI seeking what they need to know about storage. We famous a
need for a benchmark, nonetheless that s not something an researcher customarily does.”

The researcher organisation put together a VDI-IOmark organization,
which it describes as a bloc of attention groups, vendors and IT users. No members of the
organization have been identified, nonetheless a press recover announcing a VDI benchmark included
a quote from BlueArc’s clamp boss of product marketing, Ravi Chalaka.

The benchmark is storage-specific and usually measures VMware
implementations now, nonetheless Fellows pronounced Evaluator Group skeleton to supplement support for Citrix
and Microsoft Hyper-V.

The VDI benchmark can run on servers doing 1,000 VDI users, 12 CPU, 96 GB of RAM and multiple
I/O ports.

There are already VDI benchmarks available, though Fellows pronounced VDI-IOmark costs reduction and is easier
to use and set adult than a Login VSI and RAWC
benchmarks, that demeanour during a whole system.

VDI-IOmark tests storage complement opening only. It does not exam a potency of the
hypervisor, server capacity, CPU or memory. It looks during a I/O compulsory for clone, boot, login,
user workload, pathogen indicate and reclone operations. VDI-IOmark calculates a cost for storage
licenses for tested storage ability and three-year upkeep and divides that by a series of
VDI instances to get a cost per user.

The benchmark criteria embody limit response times; all storage contingency reside on a systems
being tested. VDI-IOmark will tell price/performance benchmarks when vendors make them

VDI-IOmark will assign an annual permit price for commencement during $2,500 for vendors. End users can
get a permit giveaway for 30 days for evaluation.

Like any storage
, VDI-IOmark will need attention buy-in to be valuable.

Article source: