Search This Blog

Saturday, 26 March 2011

Sunfire 20K/25K Dynamic Reconfiguration

 

Here is the procedure for detaching one system board (SB13 from one domain (say Domain A) and adding the the same with another Domain (Domain F).

1. Make sure the board is available in Domain A

root@Domain-A # cfgadm -al|grep SB13
Ap_Id                          Type         Receptacle   Occupant     Condition
SB13                           V3CPU        connected    configured   ok
SB13::cpu0                     cpu          connected    configured   ok
SB13::cpu1                     cpu          connected    configured   ok
SB13::cpu2                     cpu          connected    configured   ok
SB13::cpu3                     cpu          connected    configured   ok
SB13::memory                   memory       connected    configured   ok

2. Disconnect the board from Domain A using below command.

root@Domain-A # cfgadm -c disconnect SB13

Below messages may be visible in system log while doing above activity.

root@Domain-A # tail -15 /var/adm/messages
Mar 26 15:13:47 Domain-A unix: [ID 177789 kern.info] kphysm_delete: mem = 67108864K (0x1000000000)
Mar 26 15:13:47 Domain-A unix: [ID 585997 kern.info] kphysm_delete: avail mem = 57300893696
Mar 26 15:13:47 Domain-A dr: [ID 427603 kern.notice] OS unconfigure dr@0:SB13::cpu0
Mar 26 15:13:58 Domain-A dr: [ID 427603 kern.notice] OS unconfigure dr@0:SB13::cpu1
Mar 26 15:14:19 Domain-A dr: [ID 427603 kern.notice] OS unconfigure dr@0:SB13::cpu2
Mar 26 15:14:31 Domain-A dr: [ID 427603 kern.notice] OS unconfigure dr@0:SB13::cpu3
Mar 26 15:14:52 Domain-A dr: [ID 427603 kern.notice] OS unconfigure dr@0:SB13::memory
Mar 26 15:19:00 Domain-A genunix: [ID 408114 kern.info] /memory-controller@1a0,400000 (mc-us38) offline
Mar 26 15:19:00 Domain-A genunix: [ID 408114 kern.info] /memory-controller@1a1,400000 (mc-us39) offline
Mar 26 15:19:00 Domain-A genunix: [ID 408114 kern.info] /memory-controller@1a2,400000 (mc-us310) offline
Mar 26 15:19:00 Domain-A genunix: [ID 408114 kern.info] /memory-controller@1a3,400000 (mc-us311) offline
Mar 26 15:19:00 Domain-A genunix: [ID 408114 kern.info] /address-extender-queue@1be,0 (axq4) offline

3. Once the disconnect command is finished, check the SB status is showing as disconnected.

root@Domain-A # cfgadm -al|grep SB13
SB13                           V3CPU        disconnected unconfigured unknown

4. Now SB13 can be deleted from domain A and attached to domain F (from system controller)
sms-svc@25k-SC1$ showboards
Retrieving board information. Please wait.
..................
Location    Pwr    Type of Board   Board Status  Test Status   Domain
--------    ---    -------------   ------------  -----------   ------
SB13        Off    V3CPU           Assigned      Unknown       Domain-F

 Deleting SB13  from Domain A
sms-svc@25k-SC1$ deleteboard SB13
SB13 successfully unassigned.
sms-svc@25k-SC1$ showboards |grep SB13
SB13        Off    V3CPU           Available     Unknown       Isolated

Attaching SB13 with Domain F

sms-svc@25k-SC1$ addboard -d F SB13
assign SB13
.
assign SB13 done
poweron SB13
..........
poweron SB13 done
test SB13                                                                                                                                         .......    test SB13 done
connect SB13                                                                                                                                          ....   connect SB13 done
configure SB13
.......
configure SB13 done
..
notify online SUNW_cpu/cpu416
notify online SUNW_cpu/cpu420
notify online SUNW_cpu/cpu417
notify online SUNW_cpu/cpu421
notify online SUNW_cpu/cpu418
notify online SUNW_cpu/cpu422
notify online SUNW_cpu/cpu419
notify online SUNW_cpu/cpu423
.
notify add capacity (8 cpus)
notify add capacity (4194304 pages)
notify add capacity SB13 done
.

Once the above commands finished successfully, the SB13 can be visible in Domain F

sms-svc@25k-SC1$ showboards |egrep 'Domain-A|Domain-F'

SB0         On     V3CPU           Active        Passed        Domain-A
SB1         On     V3CPU           Active        Passed        Domain-A
SB10        On     V3CPU           Active        Passed        Domain-F
SB11        On     V3CPU           Active        Passed        Domain-F
SB13        On     V3CPU           Active        Passed        Domain-F
SB17        On     V3CPU           Active        Passed        Domain-F
IO0         On     V2HPCIX         Active        Passed        Domain-A
IO1         On     V2HPCIX         Active        Passed        Domain-A
IO10        On     V2HPCIX         Active        Passed        Domain-F
IO11        On     V2HPCIX         Active        Passed        Domain-F

Below messages can be seen while attaching the SB with domain F

root@Domain-F #tail -f /var/adm/messages

Mar 26 15:38:51 Domain-F rootnex: [ID 349649 kern.info] axq5 at root: SAFARI 0x1be 0x0
Mar 26 15:38:51 Domain-F genunix: [ID 936769 kern.info] axq5 is /address-extender-queue@1be,0
Mar 26 15:38:51 Domain-F genunix: [ID 408114 kern.info] /address-extender-queue@1be,0 (axq5) online
Mar 26 15:38:51 Domain-F dr: [ID 606818 kern.notice] OS configure dr@0:SB13::cpu0
Mar 26 15:38:51 Domain-F dr: [ID 606818 kern.notice] OS configure dr@0:SB13::cpu1
Mar 26 15:38:51 Domain-F dr: [ID 606818 kern.notice] OS configure dr@0:SB13::cpu2
Mar 26 15:38:51 Domain-F dr: [ID 606818 kern.notice] OS configure dr@0:SB13::cpu3
Mar 26 15:38:51 Domain-F unix: [ID 270833 kern.info] cpu416: UltraSPARC-IV+ (portid 416 impl 0x19 ver 0x22 clock 1800 MHz)
Mar 26 15:38:51 Domain-F unix: [ID 270833 kern.info] cpu420: UltraSPARC-IV+ (portid 416 impl 0x19 ver 0x22 clock 1800 MHz)
Mar 26 15:38:51 Domain-F unix: [ID 270833 kern.info] cpu417: UltraSPARC-IV+ (portid 417 impl 0x19 ver 0x22 clock 1800 MHz)
Mar 26 15:38:51 Domain-F unix: [ID 270833 kern.info] cpu421: UltraSPARC-IV+ (portid 417 impl 0x19 ver 0x22 clock 1800 MHz)
Mar 26 15:38:51 Domain-F unix: [ID 270833 kern.info] cpu418: UltraSPARC-IV+ (portid 418 impl 0x19 ver 0x22 clock 1800 MHz)
Mar 26 15:38:51 Domain-F unix: [ID 270833 kern.info] cpu422: UltraSPARC-IV+ (portid 418 impl 0x19 ver 0x22 clock 1800 MHz)
Mar 26 15:38:51 Domain-F unix: [ID 270833 kern.info] cpu419: UltraSPARC-IV+ (portid 419 impl 0x19 ver 0x22 clock 1800 MHz)
Mar 26 15:38:51 Domain-F unix: [ID 270833 kern.info] cpu423: UltraSPARC-IV+ (portid 419 impl 0x19 ver 0x22 clock 1800 MHz)
Mar 26 15:38:51 Domain-F unix: [ID 519443 kern.info] cpu 416 initialization complete - restarted
Mar 26 15:38:51 Domain-F unix: [ID 519443 kern.info] cpu 420 initialization complete - restarted
Mar 26 15:38:51 Domain-F unix: [ID 519443 kern.info] cpu 417 initialization complete - restarted
Mar 26 15:38:51 Domain-F unix: [ID 519443 kern.info] cpu 421 initialization complete - restarted
Mar 26 15:38:51 Domain-F unix: [ID 519443 kern.info] cpu 418 initialization complete - restarted
Mar 26 15:38:51 Domain-F unix: [ID 519443 kern.info] cpu 422 initialization complete - restarted
Mar 26 15:38:51 Domain-F unix: [ID 519443 kern.info] cpu 419 initialization complete - restarted
Mar 26 15:38:51 Domain-F unix: [ID 519443 kern.info] cpu 423 initialization complete - restarted
Mar 26 15:38:51 Domain-F dr: [ID 606818 kern.notice] OS configure dr@0:SB13::memory
Mar 26 15:38:51 Domain-F rootnex: [ID 349649 kern.info] mc-us312 at root: SAFARI 0x1a0 0x400000 ...
Mar 26 15:38:51 Domain-F genunix: [ID 936769 kern.info] mc-us312 is /memory-controller@1a0,400000
Mar 26 15:38:51 Domain-F genunix: [ID 408114 kern.info] /memory-controller@1a0,400000 (mc-us312) online
Mar 26 15:38:51 Domain-F rootnex: [ID 349649 kern.info] mc-us313 at root: SAFARI 0x1a1 0x400000 ...
Mar 26 15:38:51 Domain-F genunix: [ID 936769 kern.info] mc-us313 is /memory-controller@1a1,400000
Mar 26 15:38:51 Domain-F genunix: [ID 408114 kern.info] /memory-controller@1a1,400000 (mc-us313) online
Mar 26 15:38:51 Domain-F rootnex: [ID 349649 kern.info] mc-us314 at root: SAFARI 0x1a2 0x400000 ...
Mar 26 15:38:51 Domain-F genunix: [ID 936769 kern.info] mc-us314 is /memory-controller@1a2,400000
Mar 26 15:38:51 Domain-F genunix: [ID 408114 kern.info] /memory-controller@1a2,400000 (mc-us314) online
Mar 26 15:38:51 Domain-F rootnex: [ID 349649 kern.info] mc-us315 at root: SAFARI 0x1a3 0x400000 ...
Mar 26 15:38:51 Domain-F genunix: [ID 936769 kern.info] mc-us315 is /memory-controller@1a3,400000
Mar 26 15:38:51 Domain-F genunix: [ID 408114 kern.info] /memory-controller@1a3,400000 (mc-us315) online
Mar 26 15:38:51 Domain-F unix: [ID 700753 kern.info] kphysm_add_memory_dynamic: adding 33554432K at 0x1e000000000
Mar 26 15:39:13 Domain-F unix: [ID 323408 kern.info] kphysm_add_memory_dynamic: mem = 125829120K (0x1e00000000)
Mar 26 15:39:13 Domain-F unix: [ID 401001 kern.info] kphysm_add_memory_dynamic: avail mem = 35283116032

New the new board is available in Domain F.

root@Domain-F # cfgadm -al|grep SB13
SB13                           V3CPU        connected    configured   ok
SB13::cpu0                     cpu          connected    configured   ok
SB13::cpu1                     cpu          connected    configured   ok
SB13::cpu2                     cpu          connected    configured   ok
SB13::cpu3                     cpu          connected    configured   ok
SB13::memory                   memory       connected    configured   ok

No comments:

Post a Comment