Optical SFP thresholds

[killabee]

I'm running into an issue where two of my SFPs are reporting low receive power warnings (for brevity, I'll just show one):

Code Select Expand


Apr 15 10:26:24: %SFF8472-5-THRESHOLD_VIOLATION: Te1/1/2: Rx power low warning; Operating value: -17.0 dBm, Threshold value: -15.8 dBm.


Sure enough when I do "show interface X transceiver detail" it shows me the Rx power is below the warning threshold:

Code Select Expand


4500X#show interfaces t1/1/2 transceiver detail
           Optical            High Alarm  High Warn  Low Warn   Low Alarm
           Receive Power      Threshold   Threshold  Threshold  Threshold
Port       (dBm)              (dBm)       (dBm)      (dBm)      (dBm)
-------    -----------------  ----------  ---------  ---------  ---------
Te1/1/2    -16.9                 1.9        -1.0       -15.8      -19.8
4500X#


What I don't get is that when these SFPs were previously installed on a N5K, the N5K had MUCH higher warning thresholds and these SFPs were NOT alarming/warning then.  Recently I moved these SFPs to a 4500X and this issue started.  What's stranger is if I move one of the SFPs to some other ports on the 4500X then I'll get different threshold values. 

How is the threshold determined? Is it possible to change it? (I couldn't find any commands using the '?' or on Google).

Unfortunately, this isn't a case where I can do too much with the actual physical fiber cable because it's a 16 mile dark fiber run.  We are using using ER optics and the Tx power on the far-end is still the same as it was before I moved the connection.  We've already optimized the fiber run as much as we could (e.g. clean jumpers, repatch bad ones, etc).  And once again, this was NOT an issue when the fiber and SFPs terminated on a N5K.  As you can see below, the N5K thresholds were much higher:

Code Select Expand


Nexus# sh int e1/3 transceiver details
Ethernet1/3
           SFP Detail Diagnostics Information (internal calibration)
  ----------------------------------------------------------------------------
                Current              Alarms                  Warnings
                Measurement     High        Low         High          Low
  ----------------------------------------------------------------------------
  Rx Power      -20.00 dBm      -6.00 dBm  -25.22 dBm   -7.01 dBm    -23.97 dBm
  ----------------------------------------------------------------------------
  Note: ++  high-alarm; +  high-warning; --  low-alarm; -  low-warning
  *** SFP diagnostics data may be invalid!  ***
Nexus#


[routerdork]

When you move the optic to a different port on the 4500X is it on the built-in ports? Or a difference in the built-in vs. module maybe? I can't say I remember seeing different readings on different platforms but it sure does sound like Nexus has better hardware. Possibly even features normally seen in DWDM equipment that allow it to accept a wider tolerance range.

[routerdork]

Also I don't know what you are transmitting at but I'd expect that receive to be a little better. A distance of 16 miles is barely over half the recommended range for an ER optic under optimal conditions. Have you ever seen an OTDR or Power Meter readings for the fiber pair your are using? Might be beneficial. I have cleaned over and over many times only to find it just needed to be cleaned one more time.

[Otanx]

The "SFP diagnostics data may be invalid!" line on the Nexus output makes me think that the Nexus is reporting faulty data, and the 4500 is giving you good information.

-Otanx

[routerdork]

The "SFP diagnostics data may be invalid!" line on the Nexus output makes me think that the Nexus is reporting faulty data, and the 4500 is giving you good information.

-Otanx

I didn't even see that. Also thinking about this I did have one vendor tell me that their optics could be +/- 3dBm of the reported value. Wasn't Cisco that said this but food for thought.

[killabee]

Thanks guys.  I was connectibg to built in ports, but when I tried it again a few days ago I saw  different behavior.  I'll keep pounding at it.

[killabee]

So I think I know part of what's happening...

The optical thresholds are read from the thresholds table ("show interface transceiver threshold-table").  The switch detects what kind of optic is installed (e.g. SFP, GBIC, etc) and reads what its optical Rx power is.  It compares that value to what it thinks the optic is in the threshold table and reads the warning and alarm values from there.

So in the case of where my Rx warning is -15.8 and alarm is -19.8, that's coming from the "10GE ER SFP+" section of the threshold table:

Code Select Expand


4500X#show interfaces transceiver threshold-table | begin 10GE ER SFP+
              Optical Tx     Optical Rx    Temp     Laser Bias    Voltage
                                                    current
             -------------  -------------  ------   ------------  ---------
10GE ER SFP+
Min1             -8.70        -19.80        -5            N/A          3.00
Min2             -4.70        -15.80        0             N/A          3.10
Max2              4.00         -1.00        70            N/A          3.50
Max1              7.00          2.00        75            N/A          3.60


Interestingly enough, there's a threshold table section for DWDM optics (that's what I'm using), and its values are much higher:

Code Select Expand


4500X#show interfaces transceiver threshold-table | begin 10GE DWDM SFP+
              Optical Tx     Optical Rx    Temp     Laser Bias    Voltage
                                                    current
             -------------  -------------  ------   ------------  ---------
10GE DWDM SFP+
Min1             -5.00        -31.00        -5            N/A          3.00
Min2             -1.00        -27.00        0             N/A          3.10
Max2              3.00         -7.00        70            N/A          3.50
Max1              6.00         -3.00        75            N/A          3.60


So why does the switch think I have an ER optic instead of a DWDM optic?

Code Select Expand


4500X#show int t1/1/2 capabilities
TenGigabitEthernet1/1/2
  Model:                 WS-C4500X-16-Gbic
  Type:                  10GBase-ER


Code Select Expand


4500X#show idprom interface t1/1/3
SFP+ Serial EEPROM Contents:
Common Block:
Identifier        = SFP+ [0x3]
Connector         = LC Connector [0x7]
Transceiver
  Type             = SFP+ 10G-ER
Vendor name       = US Critical
Vendor OUI        = 0
Vendor Part No.   = DWDM-SFP10G30.33
Vendor Part Rev.  = 1.0
Vendor Serial No. = XXXXXXXXXXXXXX
Diag monitoring   = Implemented
Calibration type  = Internal

[hex output sniped]



I'm assuming it reads it from the "Type" or hex values...and no, it's not a Cisco-stampled SFP :-/

I can't explain why moving the port around sometimes gives me higher readings, though. 

[routerdork]

Are you running the right code? The link below states 3.5.0E or newer for DWDM.
http://www.cisco.com/c/en/us/td/docs/interfaces_modules/transceiver_modules/compatibility/matrix/10GE_Tx_Matrix.html

Also since it's not Cisco have you tried using the

Code Select Expand

service unsupported-transceiver command?

Weird that you get DOM though.

[killabee]

You scared me for a little with the "right code" comment....good call and thanks for that link!

Thankfully I'm running 3.5.3. So my coworker and I looked at the optics order and saw that they're actually "DWDM ZR" optics (although they're coded as ER).  We spoke to the 3rd party vendor and they confirmed that they code the optics that way for a reason.  We explained the issue and they agreed to recode them properly. 

That should hopefully get the switch to properly see them as DWDM or ZR optics and get the thresholds right.

Cool stuff!