Macro 32 Ramblings

Mind Archive

UCS Setting Fabric Ports to 1G

From:http://jeffsaidso.com/2010/12/ucs-only-runs-at-10-ge-and-requires-nexus/

UCS Only Runs at 10 GE and Requires Nexus – NOT!


So, every time I hear someone say “UCS only runs at 10 GE speeds” I look around to see if one of my competitors is talking. And they have been somewhat effective in continuing to spread this non-truth. To be fair, Cisco didn’t help the situation by not enabling this feature when UCS first shipped. However, starting with UCS firmware release 1.3.1, you can indeed run at 1 GbE speeds. As far as requiring Nexus, that’s another bit of FUD that has never been true. Are there advantages to running at 10 GE? Absolutely. Is it required for a supported UCS configuration? Absolutely not.

Most times when we hear the request for 1 GbE, it comes from a customer with a lab environment where 10 GE is not available. However, it also comes up from a few customers who just don’t have enough 10 GE ports in production, but they really want to deploy UCS. The steps required to enable 1 GbE are pretty simple and I’ll cover them here. There are a few things you should know before you just dive in and enable it though. Here are some guidelines:

  1. 1 GbE support is only available on the first 8 ports of a Cisco UCS 6120 Fabric Interconnect (FI) and the first 16 ports of a Cisco UCS 6140 FI.
  2. When you insert a 1 GbE transceiver into an FI, you need to select the speed in UCS Manager.
  3. LACP-based port channels are allowed so long as all port members run at the same speed (and meet the normal port channel requirements).
  4. 1 GbE support is only available in ports defined for Uplinks, not Server Ports (not sure why one might want to try, but I see a lot of whacky things).

So if you wanted to do this, you’d have to know which transceivers Cisco supports. Both copper and fiber are supported and the following table provides more details and part numbers:

Source: http://www.cisco.com/en/US/prod/collateral/ps10265/ps10276/data_sheet_c78-524724.html

Enabling 1 GbE Support in Cisco UCS

There’s really not much to enabling the feature. On the Fabric Interconnect in question, expand Fixed Module->Uplink Ethernet Ports and then select Port X (where X is the port you wish to set to 1 GbE).

On the right side of the screen, Click “Show Interface”. This will produce a screen similar to the following where you simply choose the 1GE radio button and you’re all set as long as you have inserted a 1 GbE transceiver into Port X.

Pretty easy stuff, but extremely important when you need it. One last thing I wanted to point out is that when you don’t set the right port speed, you will see an error of “SFP Validation Failed”. Not the error I would have chosen, but at least you now know what it means if you see it.

Let me know if you have any questions.

-Jeff