Skip to main content

Cisco FI Recovery

 

Pre-troubleshooting checks:

1. Check Cluster State

Central-BLRQA-Mini-10-B# show cluster state

Cluster Id: 0xb1125e5643ad11e0-0x921574a02f402f50

B: UP, SUBORDINATE

A: UNRESPONSIVE, PRIMARY

HA NOT READY

2. Check f/w versions on active node(if any)

Central-BLRQA-Mini-10-B# sh version

System version: 3.1(0.297c)

Central-BLRQA-Mini-10-B# show fabric-interconnect a detail

Central-BLRQA-Mini-10-B# show fabric-interconnect b detail

3. Bootability of node which is not active:

UCSM prompt or Loader prompt

Troubleshooting:

1. Go to the linux prompt of both the FIs

a. For the FI which is bootable, connect console port and login using  adminbackup

b. For the FI which is not in proper state, connect console port and power cycle the switch.

While the switch is rebooting , press ctlr+L to come to Loader prompt.

loader> dir

(displays the list of images including the kickstart image)

loader> boot  <kickstart image name>

switch(boot)# start

bash-3.2#

c. If kickstart image is not available, goto https://wiki.cisco.com/display/391974/FI+bringup+procedure+from+Loader+prompt

2. At the linux/bash prompt on both the FIs, issue the following command to erase the DB. This is    required to stop the auto-install process(if running)

bash-3.2# rm /opt/db/nvram/dme.db /opt/db/flash/dme.db

3. At the linux/bash prompt on both the FIs, perform the following:

a.  Verify the existing packages in the system to revert back to

bash-3.2# cd /bootflash/installables/switch/

bash-3.2# ls –l

total 5614168

-rwxr-xr-x 1 root root  38409728 Mar 19 06:19 ucs-6100-k9-kickstart.5.2.3.N2.2.21.179.gbin

-rwxr-xr-x 1 root root  38422016 Jul 22 06:23 ucs-6100-k9-kickstart.5.2.3.N2.2.21.188.gbin

-rwxr-xr-x 1 root root  38370304 Aug  7 12:29 ucs-6100-k9-kickstart.5.2.3.N2.2.21.198.bin

-rwxr-xr-x 1 root root  38376960 Aug  7 09:59 ucs-6100-k9-kickstart.5.2.3.N2.2.21.205.gbin

-rwxr-xr-x 1 root root 272711285 Mar 19 06:40 ucs-6100-k9-system.5.2.3.N2.2.21.179.gbin

-rwxr-xr-x 1 root root 273244129 Jul 22 06:23 ucs-6100-k9-system.5.2.3.N2.2.21.188.gbin

-rwxr-xr-x 1 root root 271342749 Aug  7 12:29 ucs-6100-k9-system.5.2.3.N2.2.21.198.bin

-rwxr-xr-x 1 root root 273317725 Aug  7 09:59 ucs-6100-k9-system.5.2.3.N2.2.21.205.gbin

-rwxr-xr-x 1 root root 178303518 Jun 10 12:50 ucs-manager-k9.2.2.2.135.gbin

-rwxr-xr-x 1 root root 181035134 Jul 22 06:23 ucs-manager-k9.2.2.2.188.gbin

-rwxr-xr-x 1 root root 183592165 Aug  7 12:29 ucs-manager-k9.2.2.2.198.bin

--rwxr-xr-x 1 root root  93776495 Aug  7 09:57 ucs-mgmtext.2.2.2.205.gbin

b. Edit the following file with proper images which were working fine before upgrade

bash-3.2# cd /mnt/cfg/1/boot/grub/

bash-3.2#  vi menu.lst.local

bash-3.2#  cat  menu.lst.local

# 697741597

#

# General configuration

#

disable certificate

# Menu entry for the available image

title bootflash:/installables/switch/ucs-6100-k9-kickstart.5.2.3.N2.2.21.188.gbin

boot bootflash:/installables/switch/ucs-6100-k9-kickstart.5.2.3.N2.2.21.188.gbin bootflash:/installables/switch/ucs-6100-k9-system.5.2.3.N2.2.21.188.gbin

c. update the UCSM image too with the same version

bash-3.2# cd /bootflash/

bash-3.2# ln  –sf  /bootflash/installables/switch/ucs-manager-k9.2.2.2.188.gbin   nuova-sim-mgmt-nsg.0.1.0.001.bin

d. Reboot the system

4. The system should come back to the older version and cluster should be up and running.

SAM-QA-20-B# show cluster state

Cluster Id: 0xf76a633e5f9511e2-0x826d547fee5682a4

 

B: UP, PRIMARY

A: UP, SUBORDINATE

 

HA NOT READY

SAM-QA-20-B#

Comments

Post a Comment

Popular posts from this blog

Recovering Fabric Interconnects When You Do Not Have Working Images on The Fabric Interconnect or The Bootflash

  Recovering Fabric Interconnects When You Do Not Have Working Images on The Fabric Interconnect or The Bootflash You can perform these steps when both or any fabric interconnect goes down during firmware upgrade, gets rebooted, and is stuck at the loader prompt, and you do not have working images on the fabric interconnect. Procedure Step 1 Reboot the switch, and in the console, press  Ctrl+L  as it boots to get the loader prompt. Note   You may need to press the selected key combination multiple times before your screen displays the loader prompt. Example: loader> Step 2 Configure the interface to receive the kickstart image through TFTP. Enter the local IP address and subnet mask for the system at the loader> prompt, and press  Enter . Example: loader> set ip 10.104.105.136 255.255.255.0 Specify the IP address of the default gateway. Example: loader> set gw 10.104.105.1 Boot the kickstart image file from the required server. Example...
Post a Comment
Read more
  Upgrading an Old Nexus 3048 to Current NX-OS Posted on 2018/11/16 Yeah, I know. I’m supposed to be a Wi-Fi guy. However, I sometimes still have to touch routers and switches. I have some help for you if you need to upgrade some old Nexus switches. The Nexus 3048s are great DC caliber gig switches with proper front-to-back airflow. Used ones are really cheap and I think they make great management switches, which is what I’m using them for. If you have a Cisco Nexus 3048 with an older release of NX-OS on it, you need to go through several upgrades to get it to NX-OS v7. Here’s a walkthrough on the procedures. If you find you already on v6, for example, you can skip the 5.x step. Upon initial boot, you will need to abort Power-On Auto Provisioning. You will then be prompted for the admin password. Skip the basic configuration dialog. After a few moments, you will be able to login with the admin credentials you set during boot. I suggest downloading all the requ...
Post a Comment
Read more

Cisco ISR4300, user credential is not working and password recovery disabled, How to recover the access to the router

Cisco 4331, user credential is not working and password recovery disabled Step by step solution to recover the access to the router  First, make sure you have a configuration backup because this following step will erase all the config Console to the router, using Terra Term Power cycle System Bootstrap, Version 11.1(4675) [kluk 143], INTERIM SOFTWARE Copyright (c) 1994-1996 by cisco Systems, Inc. C3600 processor with 32768 Kbytes of main memory Main memory is configured to 64 bit mode with parity disabled PASSWORD RECOVERY FUNCTIONALITY IS DISABLED program load complete, entry point: 0x80008000, size: 0x2733f4 When I see the word "Password Recovery Functionality is Disabled" Send CTRL Break  It will prompt a question to Reset to factory default PASSWORD RECOVERY IS DISABLED. Do you want to reset the router to factory default configuration and proceed [y/n] ? y Reset router configuration to factory default. It should boot to router prompt Now, you can re-configure it again...
Post a Comment
Read more