During a new setup of our 11g R1 (11.1.0.7) on Solaris platform, initially the system is provided with only single path to disks (LUNs) on SAN. Later the sys admins would like to enable the multipathing. 🙁 Obviously, it is expected that the disk paths will be changed. This will not only affect the ASM disk groups, but also the OCR and Vote disks.
The path that is, for example, c4t50060E80045C5930d17s6, after multipathing, will become something like this c4t60060E80056FB30000006FB300001019d17s6.
The disk path change for the ASM disk groups is not a major concern due to the fact that all needed information is already stored in the disk header (so the disk path name is irrelevant here). Merely, update the ASM’s disk discovery path parameter (asm_diskstring) to the new path, everything should be fine. (Reference: Metalink Doc ID 466231.1 – ASM Survive Change Of Disc Path?)
The main concern is with cluster registry information stored in the OCR and Voting disks which obviously their paths will not be the same after multipathing. It is expected that CRS won’t start up because the ocr.loc still contains the old paths which will no longer exist after reboot.
We could not run ocrconfig -replace ocr and ocrconfig -replace ocrmirror before multipathing as the new disk paths are not there yet, or after that as CRS will not be available. Note that to move the OCR location for Oracle RAC 10gR2 or later, use “ocrconfig -replace” command. Manual edit of the ocr.loc at /var/opt/oracle is not recommended.
Fortunately there is a way to perform disk path change while CRS is off-line.
Before multipating
Backup all OCR and Voting disks.
For OCR, use:
# ocrconfig -manualbackup
For Voting disks, use:
# dd if={VOTING_PATH} of={VOTING_BACKUP_PATH}
After multipathing (and server reboot), the CRS is not available.
1. Even with CRS down, ensure that there is no other CRS-related processes (evmd or cssd) running.
Some processes will be restarted automatically even they’re terminated manually with using kill -9.
To ensure that no process running, backup and comment the last three line of the /etc/inittab.
#h1:3:respawn:/etc/init.d/init.evmd run >/dev/null 2>&1 </dev/null #h2:3:respawn:/etc/init.d/init.cssd fatal >/dev/null 2>&1 </dev/null #h3:3:respawn:/etc/init.d/init.crsd run >/dev/null 2>&1 </dev/null
Run init q to take change into effect.
Don’t forget to un-comment them when all tasks are completed.
2. On all nodes, run the following commands as root to point to the new disk paths for the master and the mirror:
# ocrconfig -repair ocr /dev/rdsk/c5t60060E80045C590000005C5900000C03d0s6 # ocrconfig -repair ocrmirror /dev/rdsk/c5t60060E80045C590000005C5900000C04d0s6
3. As root, on one node, run:
# ocrconfig -overwrite
4. Run ocrcheck on all nodes to verify new paths:
# ocrcheck Status of Oracle Cluster Registry is as follows : Version : 2 Total space (kbytes) : 1036532 Used space (kbytes) : 5408 Available space (kbytes) : 1031124 ID : 582752590 Device/File Name : /dev/rdsk/c5t60060E80045C590000005C5900000C03d0s6 Device/File integrity check succeeded Device/File Name : /dev/rdsk/c5t60060E80045C590000005C5900000C04d0s6 Device/File integrity check succeeded Cluster registry integrity check succeeded
5. Next, we’ll also change the paths for voting disk.
Delete the old paths and add new ones in the order to ensure that at least one disk is available. The option -force will be required in this case.
# crsctl delete css votedisk /dev/rdsk/c3t50060E80045C5920d15s6 -force # crsctl add css votedisk /dev/rdsk/c5t60060E80045C590000005C5900000C05d0s6 -force # crsctl delete css votedisk /dev/rdsk/c3t50060E80045C5920d16s6 -force # crsctl delete css votedisk /dev/rdsk/c3t50060E80045C5920d17s6 -force # crsctl add css votedisk /dev/rdsk/c5t60060E80045C590000005C5900000C06d0s6 -force # crsctl add css votedisk /dev/rdsk/c5t60060E80045C590000005C5900000C07d0s6 –force
Now we should be able to start CRS.
Reference: Method 2 (Offline) in Note 428681.1 :OCR / Vote disk Maintenance Operations: (ADD/REMOVE/REPLACE/MOVE)
Pingback: Tweets that mention OraExplorer » Migration of the 11g R1 OCR and Voting disks to different paths when Cluster Ready Service (CRS) is off-line -- Topsy.com