Brocade Diagnostic Port Testing

Recently I deployed a couple of new switches into a Brocade fabric (6510 and 6505 attached to an existing 6510).

The ISLs were very short, but these were 16Gb ISLs over OM2 fiber…re-used OM2.  There’s always that question mark when connecting up as to whether the physical cabling is going to be a problem.  As speeds climb higher and higher, the supported distances get shorter (and the greater the likelihood that a marginal cable is going to cause a serious issue).

To assist in heading off issues like this before they become angry fabric monsters, Brocade built in diagnostic port tests (“D_Port”).

I’m not going to delve into great detail about the tests themselves, just the high points. Here is a link to the manual on it, feel free to check it out.

martin2341 posted a couple of fantastic videos from Brocade about this feature.  Check them out as well!

The switch model and FOS version you are running dictate whether you can use the dport testing.

dportsupport

Second caveat – two of the tests are only supported with 10Gb, 16Gb, and (sometimes?) 8Gb LR or LW Fibre Channel SFPs. So if you have 8Gb SR (what I consider most common at least in recent times) you appear to be out of luck until you move to 16Gb.

Finally, you have to take down the link in order to test it, so it is probably not something you are going to want to run on active production just for kicks…but then that isn’t what it was designed for.  In my case I was adding new switches and wanted to validate 16Gb ISLs, so this was a textbook use case for dport diagnostics.

I ran the following commands (note there are two switches: Sw1 and Sw2…after all these are ISLs).  First we bring down and configure the dport on switch 1:

Sw1> portdisable PORTNUMBER
Sw1> portcfgdport --enable PORTNUMBER

At this point, from switchshow output the port will look like:

23  23   0a1700   id    N16   In_Sync     FC  D-Port Disabled

Next, repeat for switch 2 and bring both ports up:

Sw2> portdisable PORTNUMBER
Sw2> portcfgdport --enable PORTNUMBER
Sw1> portenable PORTNUMBER
Sw2> portenable PORTNUMBER

At this point dport testing begins automatically.  Whatever switch you brought the last port up on will be the “master” of sorts, and the other switch will be the responder.  On my switch, this is what it looked like (switch message output to console in bold/italic, I didn’t type anything after the portenable command):

Sw2:admin> portenable 23
Sw2:admin> D-Port electrical loopback test started on slot 0 port 23.
D-Port test failed on slot 0 port 23 due to offline.
D-Port electrical loopback test started on slot 0 port 23.
D-Port electrical loopback test completed on slot 0 port 23.
D-Port optical loopback test started on slot 0 port 23.
D-Port optical loopback test completed on slot 0 port 23.
D-Port frame traffic test started on slot 0 port 23.
D-Port frame traffic test completed on slot 0 port 23.

Note that the first electrical loopback test failed “due to offline.”  This actually happened every time – this is probably due to a quasi-race condition with the port coming online.  So nothing to be alarmed about.

Now that the tests are finished, we want to see results!  Use the following command to get a summary of all port tests on the switch:

Sw2:admin> portdporttest --show all
Port  State     SFP Capabilities  Test Result
=============================================
23    ONLINE    E,O               PASSED

And use this command to get a lot of nice detailed information about a specific port:

Sw2:admin> portdporttest --show 23
D-Port Information:
===================
Port: 23
Remote WWNN: 10:00:00:05:33:a9:f4:2e
Remote port: 46
Mode: Automatic
Start time: Wed Feb 26 14:58:13 2014
End time: Wed Feb 26 14:58:33 2014
Status: PASSED
================================================================================
Test Start time Result EST(HH:MM:SS) Comments
================================================================================
Electrical loopback 14:58:16 PASSED -------- ----------
Optical loopback 14:58:22 PASSED -------- ----------
Link traffic test 14:58:28 PASSED -------- ----------
================================================================================
Roundtrip link latency: 265 nano-seconds
Estimated cable distance: 0 meters

Pretty sweet – the test only took 20 seconds, and even gives you latency numbers and cable length estimation (this cable is actually 1m long lest anyone thing that I’ve managed to ISL switches without a cable).  I’m ran this from the “master” side switch. If you run the stats command from the responder side, you’ll know because…well…..

Sw1:admin> portdporttest --show 46
D-Port Information:
===================
Port:                           46
Remote WWNN:                    00:00:00:00:00:00:00:00
Remote port:                    -1
Mode:                           Manual
No. of test frames:             1 Million
Test frame size:                1024 Bytes
FEC (enabled/option/active):    Yes/No/No
CR (enabled/option/active):     Yes/No/No
Start time:                     Wed Feb 26 11:14:01 2014
End time:                       Wed Feb 26 11:14:20 2014
Status:                         RESPONDER
================================================================================
Test                    Start time      Result          EST(HH:MM:SS)   Comments
================================================================================
Electrical loopback     11:14:01        PASSED          --------        ----------
Optical loopback        11:14:08        RESPONDER       --------        See remote port results
Link traffic test       11:14:14        RESPONDER       --------        See remote port results
================================================================================
Roundtrip link latency:         258 nano-seconds
Estimated cable distance:       0 meters

Finally to pack everything up and head home:

Sw1> portdisable PORTNUMBER
Sw1> portcfgdport --disable PORTNUMBER
Sw2> portdisable PORTNUMBER
Sw2> portcfgdport --disable PORTNUMBER
Sw1> portenable PORTNUMBER
Sw2> portenable PORTNUMBER

So all in all a good, quick, and easy way to have peace of mind, especially in situations where you might not be working with brand new cable fresh from the ol’ cable tree.