Search In The Blog

Saturday 11 February 2017

How to stop and start one node in RAC cluster (11g)

How to stop and start one node in RAC cluster (11g)



We are having RAC two node cluster and we would like to restart only one node (here first node)

Many times even experienced DBA fall in question that how to stop and start only one node out of complete RAC. This is obvious that as even you know the command but you want to avoid a mistake as any single wrong command has its significant impact on your cluster.

In this exercise we will see how to stop and start one node.

To stop and start the cluster you must login as root user.

Version - 11gr2
DB Name – mydb (mydb1, mydb2)
RAC node 1 – mymypcpepsd1
RAC node 2 -  mymypcpepsd1

The sequence of stopping node is as follows – DB instance -> node apps -> ASM -> cluster
The sequence of starting node is as follows – Cluster -> ASM -> node apps -> DB instance

You can perform crsctl and srvctl commands any of the nodes.

Step 1:
Take the status of cluster resources running on the RAC cluster this will help you to crosscheck everything is well after restart.

I logged in as root user here; you can also execute this command by grid owner.

# cd /u01/app/11.2.0/grid/
# cd bin
#
$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DBDATA.dg
               ONLINE  ONLINE       mypcpepsd1
               ONLINE  ONLINE       mypcpepsd2
ora.DBFRA.dg
               ONLINE  ONLINE       mypcpepsd1
               ONLINE  ONLINE       mypcpepsd2
ora.LISTENER.lsnr
               ONLINE  ONLINE       mypcpepsd1
               ONLINE  ONLINE       mypcpepsd2
ora.OCRVOTE.dg
               ONLINE  ONLINE       mypcpepsd1
               ONLINE  ONLINE       mypcpepsd2
ora.asm
               ONLINE  ONLINE       mypcpepsd1                 Started
               ONLINE  ONLINE       mypcpepsd2                 Started
ora.gsd
               OFFLINE OFFLINE      mypcpepsd1
               OFFLINE OFFLINE      mypcpepsd2
ora.net1.network
               ONLINE  ONLINE       mypcpepsd1
               ONLINE  ONLINE       mypcpepsd2
ora.ons
               ONLINE  ONLINE       mypcpepsd1
               ONLINE  ONLINE       mypcpepsd2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       mypcpepsd1
ora.cvu
      1        ONLINE  ONLINE       mypcpepsd1
ora.mydb.db
      1        ONLINE  ONLINE       mypcpepsd1                 Open
      2        ONLINE  ONLINE       mypcpepsd2                 Open
ora.oc4j
      1        OFFLINE OFFLINE
ora.mypcpepsd1.vip
      1        ONLINE  ONLINE       mypcpepsd1
ora.mypcpepsd2.vip
      1        ONLINE  ONLINE       mypcpepsd2
ora.scan1.vip
      1        ONLINE  ONLINE       mypcpepsd1
$
$


Step 2:
 Now we are proceeding with stopping the database instance of node –

You need to perform this by ORACLE owner.

$ORACLE_HOME/bin/bin/srvctl stop instance –d mydb –I mydb1


 Step 3:
 Shutdown node apps on forst node.
$ORACLE_HOME/bin/bin/srvctl stop nodeapps -n mypcpepsd1


Step 4:
Now stop the cluster by below command as root user.

# $GRID_HOME/bin/crsctl stop cluster
CRS-2673: Attempting to stop 'ora.crsd' on 'mypcpepsd1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'mypcpepsd1'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'mypcpepsd1'
CRS-2673: Attempting to stop 'ora.DBDATA.dg' on 'mypcpepsd1'
CRS-2673: Attempting to stop 'ora.DBFRA.dg' on 'mypcpepsd1'
CRS-2673: Attempting to stop 'ora.OCRVOTE.dg' on 'mypcpepsd1'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'mypcpepsd1' succeeded
CRS-2673: Attempting to stop 'ora.mypcpepsd1.vip' on 'mypcpepsd1'
CRS-2677: Stop of 'ora.DBDATA.dg' on 'mypcpepsd1' succeeded
CRS-2677: Stop of 'ora.DBFRA.dg' on 'mypcpepsd1' succeeded
CRS-2677: Stop of 'ora.mypcpepsd1.vip' on 'mypcpepsd1' succeeded
CRS-2672: Attempting to start 'ora.mypcpepsd1.vip' on 'mypcpepsd2'
CRS-2676: Start of 'ora.mypcpepsd1.vip' on 'mypcpepsd2' succeeded
CRS-2677: Stop of 'ora.OCRVOTE.dg' on 'mypcpepsd1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'mypcpepsd1'
CRS-2677: Stop of 'ora.asm' on 'mypcpepsd1' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'mypcpepsd1'
CRS-2677: Stop of 'ora.ons' on 'mypcpepsd1' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'mypcpepsd1'
CRS-2677: Stop of 'ora.net1.network' on 'mypcpepsd1' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'mypcpepsd1' has completed
CRS-2677: Stop of 'ora.crsd' on 'mypcpepsd1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'mypcpepsd1'
CRS-2673: Attempting to stop 'ora.evmd' on 'mypcpepsd1'
CRS-2673: Attempting to stop 'ora.asm' on 'mypcpepsd1'
CRS-2677: Stop of 'ora.evmd' on 'mypcpepsd1' succeeded
CRS-2677: Stop of 'ora.asm' on 'mypcpepsd1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'mypcpepsd1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'mypcpepsd1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'mypcpepsd1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'mypcpepsd1'
CRS-2677: Stop of 'ora.cssd' on 'mypcpepsd1' succeeded
#

If you want to stop complete cluster then yu can user below commmnd.
$GRID_HOME/bin/crsctl stop cluster -all


Step 5:
 Complete the activity you want to perform. For which you have taken this node restart.  (optional)


Step 6:
 Start the cluster, again to start the cluster you must login as root user.

#
# $GRID_HOME/bin/crsctl start cluster
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'mypcpepsd1'
CRS-2676: Start of 'ora.cssdmonitor' on 'mypcpepsd1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'mypcpepsd1'
CRS-2672: Attempting to start 'ora.diskmon' on 'mypcpepsd1'
CRS-2676: Start of 'ora.diskmon' on 'mypcpepsd1' succeeded
CRS-2676: Start of 'ora.cssd' on 'mypcpepsd1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'mypcpepsd1'
CRS-2676: Start of 'ora.ctssd' on 'mypcpepsd1' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'mypcpepsd1'
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'mypcpepsd1'
CRS-2676: Start of 'ora.evmd' on 'mypcpepsd1' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'mypcpepsd1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'mypcpepsd1'
CRS-2676: Start of 'ora.asm' on 'mypcpepsd1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'mypcpepsd1'
CRS-2676: Start of 'ora.crsd' on 'mypcpepsd1' succeeded
#


Step 6:
 Start the database instance (by oracle user)

$ORACLE_HOME/bin/srvctl stat instance –d mydb –I mydb1


Step 7:
 Verify that all cluster resources are properly started. (Here step one where we have taken the status of resources will help you)


I hope this exercise will help you. Thank you!!!