Sparc 5440 hangs during boot from cdrom

Question

We have a Sun Sparc 5440 server that originally had 64 GB of RAM that we were trying to upgrade to 128 GB. During the process, the SCSI root disk seems to have gotten corrupted, so we are trying to rebuild the system.

We have a DVD of Solaris 10 (a requirement because this server mirrors a production server). It's the most current version as downloaded from Oracle (sol-10-u11-ga-sparc-dvd.iso). When we put the DVD into the drive and do a boot cdrom it chugs along for a while and then just freezes.

We've verified that the DVD works on another Sparc system, so we've ruled that out.

We've also ruled out that it's the DVD drive (by connecting a USB-DVD drive up and booting from that).

After searching, we tried booting with the -v flag and this is the output that we see:

{0} ok boot /pci@400/pci@0/pci@9/pci@0/usb@0,2/storage@1/disk -rsv
Boot device: /pci@400/pci@0/pci@9/pci@0/usb@0,2/storage@1/disk  File and args: -rsv
hsfs-file-system
Loading: /platform/sun4v/boot_archive
ramdisk-root ufs-file-system
Loading: /platform/SUNW,T5440/kernel/sparcv9/unix
module /platform/SUNW,T5440/kernel/sparcv9/unix: text at [0x1000000, 0x10cbab5] data at 0x1800000
module /platform/SUNW,T5440/kernel/sparcv9/genunix: text at [0x10cbab8, 0x12b4e0f] data at 0x1a16800
module /platform/SUNW,T5440/kernel/misc/sparcv9/platmod: text at [0x12b4e10, 0x12b4e27] data at 0x1a6e4b0
module /platform/SUNW,T5440/kernel/cpu/sparcv9/SUNW,UltraSPARC-T2+: text at [0x12b4e30, 0x12bad2f] data at 0x1a6ec00
SunOS Release 5.10 Version Generic_147147-26 64-bit
Copyright (c) 1983, 2013, Oracle and/or its affiliates. All rights reserved.
os-io Ethernet address = 0:14:4f:ec:e9:b2
Using default device instance data
mem = 133857280K (0x1fea000000)
avail mem = 133886173184
root nexus = T5440
pseudo0 at root
pseudo0 is /pseudo
scsi_vhci0 at root
scsi_vhci0 is /scsi_vhci
ramdisk0 at root
ramdisk0 is /ramdisk-root
root on /ramdisk-root:a fstype ufs
virtual-device: cnex0
cnex0 is /virtual-devices@100/channel-devices@200
pseudo-device: dld0
dld0 is /pseudo/dld@0
px2 at root: 0x600 0x0
px2 is /pci@600
PCI Express-device: pci@0, pcieb10
pcieb10 is /pci@600/pci@0
PCIE-device: pci@c, pcieb12
pcieb12 is /pci@600/pci@0/pci@c
PCIE-device: SUNW,XVR-300@0, nfb0
nfb0 is /pci@600/pci@0/pci@c/SUNW,XVR-300@0
px0 at root: 0x400 0x0
px0 is /pci@400
PCI Express-device: pci@0, pcieb0
pcieb0 is /pci@400/pci@0
PCIE-device: pci@9, pcieb2
pcieb2 is /pci@400/pci@0/pci@9
PCIE-device: pci@0, pcieb4
pcieb4 is /pci@400/pci@0/pci@9/pci@0
PCIE-device: usb@0,2, ehci0
ehci0 is /pci@400/pci@0/pci@9/pci@0/usb@0,2
PCIE-device: usb@0, ohci0
ohci0 is /pci@400/pci@0/pci@9/pci@0/usb@0
PCIE-device: usb@0,1, ohci1
ohci1 is /pci@400/pci@0/pci@9/pci@0/usb@0,1
cpu0: UltraSPARC-T2+ (chipid 0, clock 1164 MHz)
cpu1: UltraSPARC-T2+ (chipid 0, clock 1164 MHz)
cpu2: UltraSPARC-T2+ (chipid 0, clock 1164 MHz)
...
cpu80: UltraSPARC-T2+ (chipid 1, clock 1164 MHz)
cpu81: UltraSPARC-T2+ (chipid 1, clock 1164 MHz)
cpu82: UltraSPARC-T2+ (chipid 1, clock 1164 MHz)
PCIE-device: pci@d, pcieb5
pcieb5 is /pci@400/pci@0/pci@d
cpu83: UltraSPARC-T2+ (chipid 1, clock 1164 MHz)
cpu84: UltraSPARC-T2+ (chipid 1, clock 1164 MHz)
...
cpu104: UltraSPARC-T2+ (chipid 1, clock 1164 MHz)
cpu105: UltraSPARC-T2+ (chipid 1, clock 1164 MHz)
cpu106: UltraSPARC-T2+ (chipid 1, clock 1164 MHz)
USB 2.0 device (usb644,1008) operating at hi speed (USB 2.x) on USB 2.0 root hub: storage@1, scsa2usb2 at bus address 2
        TEAC USB DVD Reader DEF10BED1014
scsa2usb2 is /pci@400/pci@0/pci@9/pci@0/usb@0,2/storage@1
/pci@400/pci@0/pci@9/pci@0/usb@0,2/storage@1 (scsa2usb2) online
cpu107: UltraSPARC-T2+ (chipid 1, clock 1164 MHz)
cpu108: UltraSPARC-T2+ (chipid 1, clock 1164 MHz)
cpu109: UltraSPARC-T2+ (chipid 1, clock 1164 MHz)
...
cpu157: UltraSPARC-T2+ (chipid 2, clock 1164 MHz)
cpu158: UltraSPARC-T2+ (chipid 2, clock 1164 MHz)
cpu159: UltraSPARC-T2+ (chipid 2, clock 1164 MHz)
USB 2.0 device (usb4b4,6560) operating at hi speed (USB 2.x) on USB 2.0 root hub: hub@2, hubd0 at bus address 3
hubd0 is /pci@400/pci@0/pci@9/pci@0/usb@0,2/hub@2
/pci@400/pci@0/pci@9/pci@0/usb@0,2/hub@2 (hubd0) online
cpu160: UltraSPARC-T2+ (chipid 2, clock 1164 MHz)
cpu161: UltraSPARC-T2+ (chipid 2, clock 1164 MHz)
cpu162: UltraSPARC-T2+ (chipid 2, clock 1164 MHz)
...
cpu216: UltraSPARC-T2+ (chipid 3, clock 1164 MHz)
sd3 at scsa2usb2: target 0 lun 0
sd3 is /pci@400/pci@0/pci@9/pci@0/usb@0,2/storage@1/disk@0,0
cpu217: UltraSPARC-T2+ (chipid 3, clock 1164 MHz)
cpu218: UltraSPARC-T2+ (chipid 3, clock 1164 MHz)
cpu219: UltraSPARC-T2+ (chipid 3, clock 1164 MHz)
cpu220: UltraSPARC-T2+ (chipid 3, clock 1164 MHz)
cpu221: UltraSPARC-T2+ (chipid 3, clock 1164 MHz)
cpu222: UltraSPARC-T2+ (chipid 3, clock 1164 MHz)
USB 2.0 device (usb451,625f) operating at hi speed (USB 2.x) on USB 2.0 external hub: storage@3, scsa2usb3 at bus address 4
        Texas Instruments Incorporated 6250 FW: 04.01.02.22.562AD423 ACD6CE6E2E1A
scsa2usb3 is /pci@400/pci@0/pci@9/pci@0/usb@0,2/hub@2/storage@3
/pci@400/pci@0/pci@9/pci@0/usb@0,2/hub@2/storage@3 (scsa2usb3) online
cpu223: UltraSPARC-T2+ (chipid 3, clock 1164 MHz)
cpu224: UltraSPARC-T2+ (chipid 3, clock 1164 MHz)
cpu225: UltraSPARC-T2+ (chipid 3, clock 1164 MHz)
cpu226: UltraSPARC-T2+ (chipid 3, clock 1164 MHz)
cpu227: UltraSPARC-T2+ (chipid 3, clock 1164 MHz)
cpu228: UltraSPARC-T2+ (chipid 3, clock 1164 MHz)
cpu229: UltraSPARC-T2+ (chipid 3, clock 1164 MHz)
cpu230: UltraSPARC-T2+ (chipid 3, clock 1164 MHz)
cpu231: UltraSPARC-T2+ (chipid 3, clock 1164 MHz)
sd4 at scsa2usb3: target 0 lun 0
sd4 is /pci@400/pci@0/pci@9/pci@0/usb@0,2/hub@2/storage@3/disk@0,0
cpu232: UltraSPARC-T2+ (chipid 3, clock 1164 MHz)
cpu233: UltraSPARC-T2+ (chipid 3, clock 1164 MHz)
cpu234: UltraSPARC-T2+ (chipid 3, clock 1164 MHz)
...
cpu254: UltraSPARC-T2+ (chipid 3, clock 1164 MHz)
cpu255: UltraSPARC-T2+ (chipid 3, clock 1164 MHz)
/pci@400/pci@0/pci@9/pci@0/usb@0,2/storage@1/disk@0,0 (sd3) online
/pci@400/pci@0/pci@9/pci@0/usb@0,2/hub@2/storage@3/disk@0,0 (sd4) online
PCIE-device: SUNW,qlc@0, qlc0
qlc0 is /pci@400/pci@0/pci@d/SUNW,qlc@0
PCIE-device: SUNW,qlc@0,1, qlc1
qlc1 is /pci@400/pci@0/pci@d/SUNW,qlc@0,1
px1 at root: 0x500 0x0
px1 is /pci@500
PCI Express-device: pci@0, pcieb6
pcieb6 is /pci@500/pci@0
PCIE-device: pci@9, pcieb7
pcieb7 is /pci@500/pci@0/pci@9
PCIE-device: SUNW,qlc@0, qlc2
qlc2 is /pci@500/pci@0/pci@9/SUNW,qlc@0
PCIE-device: SUNW,qlc@0,1, qlc3
qlc3 is /pci@500/pci@0/pci@9/SUNW,qlc@0,1
fp5 is /pci@400/pci@0/pci@d/SUNW,qlc@0,1/fp@0,0
fp4 is /pci@400/pci@0/pci@d/SUNW,qlc@0/fp@0,0
fp6 is /pci@500/pci@0/pci@9/SUNW,qlc@0/fp@0,0
fp7 is /pci@500/pci@0/pci@9/SUNW,qlc@0,1/fp@0,0
PCIE-device: pci@1, pcieb1
pcieb1 is /pci@400/pci@0/pci@1
PCIE-device: pci@c, pcieb3
pcieb3 is /pci@400/pci@0/pci@c
PCIE-device: pci@c, pcieb8
pcieb8 is /pci@500/pci@0/pci@c
PCIE-device: pci@d, pcieb9
pcieb9 is /pci@500/pci@0/pci@d
PCIE-device: pci@9, pcieb11
pcieb11 is /pci@600/pci@0/pci@9
PCIE-device: pci@d, pcieb13
pcieb13 is /pci@600/pci@0/pci@d
px3 at root: 0x700 0x0
px3 is /pci@700
PCI Express-device: pci@0, pcieb14
pcieb14 is /pci@700/pci@0
PCIE-device: pci@9, pcieb15
pcieb15 is /pci@700/pci@0/pci@9
PCIE-device: pci@c, pcieb16
pcieb16 is /pci@700/pci@0/pci@c
iscsi0 at root
iscsi0 is /iscsi

We have left it sitting overnight and it never gets further.

Any idea what could be going wrong or suggestions to debug this further?

Still not able to get this to boot of DVD, but we believe the root issue is that the ILOM version on the server doesn't support the newest Solaris 10 DVD. In any event, we got the system functional again with help from a vendor by booting off of a raid mirror of the root disk. — Alcanzar, Oct 06 '16 at 20:43

score 2 · Answer 1 · answered May 16 '18 at 07:39

Seems that you replaced disks from other system with hardware RAID on it. You need to remove this RAID OR activate it.

The first way.

The longer way is to remove it via OS, but you need to know minimal about OBP: Solaris hangs if it see alien RAID during boot, thats why you need to remove all disks and boot from DVD or net and then insert disks when solaris boots. Then scan disks via devfsadm and remove RAID via raidctl. I do not like this way due to need to be present near the server.

The second way.

To activate alien RAID on internal disks via OBP.

ok setenv fcode-debug? true
ok setenv auto-boot? false
ok reset-all

Run select command, this is platform-dependent. You may watch show-disks command to remind the correct path for your platform.

Path select instructions for T5440:

ok select /pci@400/pci@0/pci@1/scsi@0/

Also I provide selections for other platforms for other people.

Path select instructions for T1000

ok select /pci@7c0/pci@0/pci@8/scsi@2

Path select instructions for T2000 (pn 501-6843)

ok select /pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@2

Path select instructions for T2000 (pn 501-7501):

ok select /pci@780/pci@0/pci@9/scsi@0

Path select instructions for V215/V245:

ok select /pci@1e,600000/pci@0/pci@a/pci@0/pci@8/scsi@1

Path select instructions for V445/Boston:

ok select /pci@1f,700000/pci@0/pci@2/pci@0/pci@8/LSILogic,sas@1

Path select instructions for V445 with 501-7244 SAS controller

ok select /pci@1e,600000/pci@0/pci@2/scsi@0

Path select instructions for T5120/T5220/Huron:

ok select /pci@0/pci@0/pci@2/scsi@0

Path select instructions for T5140/T5240:

ok select /pci@400/pci@0/pci@8/scsi@0/

After this you may administrate RAID volumes from OBP:

ok show-volumes ( Look for the inactive volume)
ok X activate-volume (X is the volume number)

Also you may delete volume via

ok X delete-volume (X is the volume number)

To finish:

ok unselect-dev
ok probe-scsi-all (to check for disks)

When dealing with multiple volumes make sure they are activated in descending order:

ok show-volumes
ok 1 activate-volume
ok 0 activate-volume
ok unselect-dev
ok probe-scsi-all (to check for disks)

Failure to do so will result in the following error and will require a reset-all prior to activating the second volume:

ok 0 activate-volume
Volume 0 is now activated
ok 1 activate-volume
RAID volume request failed, ioc-status = 0007
Volume 1 could not be activated

Following activation the first volume will remain 'Degraded' until both volumes have completed syncing.

Do not forget to return values:

ok setenv fcode-debug? true
ok setenv auto-boot? true
ok reset-all

Sparc 5440 hangs during boot from cdrom

1 Answers1

The first way.

The second way.