HomeCloud ComputingCisco MDS 64G SAN Analytics: Structure evolution

Cisco MDS 64G SAN Analytics: Structure evolution

Cisco just lately introduced software program availability of NX-OS 9.2(2) with help for SAN Analytics on the Cisco MDS 9700 Collection switches with 64G Modules. This software program launch begins the subsequent section within the structure evolution of SAN Analytics.

On this weblog we are going to do a high-level comparability of SAN Analytics Structure between the Cisco MDS 32G and 64G platforms and have a look at a few of the new improvements of Cisco MDS 64G SAN Analytics.

However first, let’s cowl methodologies used for efficiency monitoring. Utilization, Saturation and Errors (USE) is a generic methodology for efficient efficiency monitoring of any system. The USE metrics establish efficiency bottlenecks of a system. Within the context of a storage system, we are able to add Latency as a further aspect into the USE methodology to create LUSE. A full visibility into LUSE metrics of a storage infrastructure is vital for efficiency monitoring and troubleshooting.

SAN Analytics and SAN Insights are advance options of the Cisco MDS 32G switches since NX-OS 8.3(2):

  • SAN Analytics is an advance function of Cisco MDS switches that collects storage I/O metrics from switches unbiased of host and storage techniques. Over 70 metrics are collected per-port, per-flow (ITL/ITN) and streamed out. These metrics will be labeled into one of many ‘LUSE’ classes.
  • SAN Insights is a functionality of Cisco Nexus Dashboard Material Controller (Previously DCNM) SAN that receives the metrics stream from SAN Analytics. It gives the visualization and evaluation of material vast I/O metrics utilizing the ‘LUSE’ framework.

Cisco MDS 32G SAN Analytics

Entry Management Lists (ACL) implement entry management on each body switched by the ASIC. The ACLs are matched extracting sure fields from the body header and on a match the motion similar to the entry is taken. On an F-port, FC Exhausting Zoning entries are programmed as ACLs within the ingress course primarily based on Zoning configuration to match on the body SID and DID with an motion to “ahead” the body to the vacation spot.

On Cisco MDS 32G switches, the I/O metrics are computed by capturing FC body headers within the knowledge path utilizing an ACL primarily based ‘Faucet’ programmed within the ASIC on ingress and egress course of the analytics enabled ports. These Faucet ACLs match on frames of curiosity for Analytics viz. CMD_IU, 1st DATA_IU, XRDY_IU, RSP_IU and ABTS. A replica of the body matching the Faucet ACL is forwarded to an on-board NPU linked to the 32G ASIC.

When SAN analytics is enabled on a port, the ACLs are programmed relying on the port kind and course as proven in Determine 1 beneath:

  • F_Port Ingress: Analytics Faucet ACLs + Zoning ACLs
  • F_Port Egress, E_Port Ingress, E_Port Egress: Analytics Faucet ACLs solely
Determine 1: Port Analytics Faucet and Zoning


The Cisco MDS 32G NPU software program Analytics Engine will be modified to accommodate customized metrics (Eg: NVMe Flush command metrics) or futuristic storage command units (Eg: NVMe-KV) with the required ACL Faucets in place.

Cisco MDS 64G SAN Analytics

The Analytics Engine strikes into the ASIC on Cisco MDS 64G switches, giving it a {hardware} acceleration. The Cisco MDS 64G Module has two 64G ASICs and every ASIC has six {hardware} Analytics Engines (one for each 4 ports). These Analytic Engines can compute I/O metrics at line price on all ports concurrently with capability to research upwards of 1 billion IOPS per Module. The {hardware} Analytics Engines have built-in Faucets and don’t want the ACL primarily based Faucets to be programmed.

The metrics computed by {hardware} Analytics Engines are saved in a database contained in the ASIC and periodically flushed to the NPU. The NPU runs a light-weight software program course of on high of DPDK (an open supply extremely environment friendly and quick packet processing framework) that collects and accumulates the metrics pushed periodically from the {hardware} Analytics Engine. Despite the fact that the NPU doesn’t run an Analytics Engine, it maintains the persistent metrics database per-flow and stays the vital aspect of the answer. The delivery of metrics from the NPU database to the Supervisor is an identical to the Cisco MDS 32G Structure. The Cisco MDS 64G {hardware} Analytics Engine doesn’t preclude a NPU software program Analytics Engine to be enabled in a future software program launch for flexibility and programmability advantages.

A comparability of the Cisco MDS 32G and MDS 64G architectures are proven in Determine 2 beneath:

Determine 2: Cisco MDS 32G and MDS 64G SAN Architectures


The Cisco MDS 64G {hardware} Analytics Engine computes some extra metrics for deeper I/O visibility:

  • Multi-sequence write I/Os are giant writes involving a number of XRDY sequences. The write alternate completion time for these writes embody delays launched by the Host (Rx XRDYn to Tx first DATAn+1) and the Storage (Rx Final DATAn-1 to Tx XRDYn). These metrics present higher evaluation and correct pinpointing of huge write efficiency points. The Analytics Engine individually tracks:

    • Avg/Min/Max host write delay
    • Avg/Min/Max storage write delay

  • The full busy time metric tracks the full time there was not less than one excellent I/O per-flow. This metric helps to characterize the ‘busyness’ of a move relative to different flows.

The {hardware} Analytics Engine by default tracks SCSI and NVMe I/O metrics at ITL/ITN granularity. Nevertheless, it may also be programmed to trace metrics for numerous move granularity of IT, ITL-VMID, ITN-NVMeConnectionID or ITN-NVMeConnectionID-VMID. This provides flexibility in selecting the granularity of metrics and I/O visibility.

The 1GbE analytics port on the Cisco MDS 64G Module can stream the per-flow metrics instantly (with out involvement of Supervisor) in an ASIC native or normal gPB/gRPC format. This will serve future use-cases that require visibility into micro telemetry occasions, which might require excessive frequency telemetry streaming.


The Cisco MDS SAN Analytics and SAN Insights is a key answer to observe and troubleshoot efficiency issues within the MDS FC SAN utilizing a ‘LUSE’ or any equal methodology. The Cisco MDS 64G platforms (working at any pace) now comes with a {hardware} Analytics Engine that may compute I/O metrics at line price on all ports. The Cisco MDS structure is the trade’s most versatile, programmable, scalable, and future proof SAN options with no forklift improve of chassis or rip and change to undertake the most recent SAN improvements.

To study extra, go to Cisco SAN Analytics and SAN Telemetry Streaming Resolution Overview

Cisco MDS 9000 is constructed to satisfy at present’s calls for whereas accommodating future innovation. The Cisco MDS structure is the trade’s most versatile, programmable, scalable, and future proof SAN options that help Multi-Era and Multi-Pace interoperability of current 16G, 32G, and new 64G line playing cards in current chassis for sleek migration and adoption of the most recent SAN improvements.



Cisco SAN Analytics Diaries Half-1: Efficiency metrics

Unlock SAN Innovation with Cisco Nexus Dashboard Material Controller Resolution Overview

Cisco SAN Insights Discovery (SID) Instrument

Forestall SAN Congestion with Cisco MDS DIRL

Optimize, Speed up, and Simplify SANs Non-disruptively

Accelerated SAN efficiency




Please enter your comment!
Please enter your name here

Most Popular

Recent Comments