Votedisk can’t be add & remove while CRS is running, or else it would corrupt something.
OCR can be added & replaced while CRS is running.
ENV: Oracle Clusterware 10.2.0.4
Detailed logfile:
While CRS is still running…
# ./crsctl add css votedisk /dev/rdsk/ c4t60060E80056F160000006F1600000669d0s1
Cluster is not in a ready state for online disk addition
# ./crsctl add css votedisk /dev/rdsk/ c4t60060E80056F160000006F1600000669d0s1 -force
Now formatting voting disk: /dev/rdsk/ c4t60060E80056F160000006F1600000669d0s1
successful addition of votedisk /dev/rdsk/c4t60060E80056F160000006F1600000669d0s1.
# ./crsctl query css votedisk
0. 0 /dev/rdsk/c4t60060E80056F160000006F1600000469d0s1
1. 0 /dev/rdsk/c4t60060E80056F160000006F1600000669d0s1
located 2 votedisk(s).
Looks good, continue add another one.
# ./crsctl add css votedisk /dev/rdsk/c4t60060E80056F160000006F1600000869d0s1
Cluster is not in a ready state for online disk addition
# ./crsctl add css votedisk /dev/rdsk/c4t60060E80056F160000006F1600000869d0s1 -force
Now formatting voting disk: /dev/rdsk/c4t60060E80056F160000006F1600000869d0s1
successful addition of votedisk /dev/rdsk/c4t60060E80056F160000006F1600000869d0s1.
It’s corrupted. Entry 1 and entry 2 both point to “*869d0s1”.
# ./crsctl query css votedisk
0. 0 /dev/rdsk/c4t60060E80056F160000006F1600000469d0s1
1. 0 /dev/rdsk/c4t60060E80056F160000006F1600000869d0s1
2. 0 /dev/rdsk/c4t60060E80056F160000006F1600000869d0s1
located 3 votedisk(s).
Has to stop CRS on all nodes.
# ./crsctl stop crs
Stopping resources. This could take several minutes.
Successfully stopped CRS resources.
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.
Then I want to delete corrupted entry 1 and entry 2.
# ./crsctl delete css votedisk /dev/rdsk/c4t60060E80056F160000006F1600000869d0s1
Cluster is not in a ready state for online disk removal
# ./crsctl delete css votedisk /dev/rdsk/c4t60060E80056F160000006F1600000869d0s1 -force
no votedisk found matching path specified /dev/rdsk/c4t60060E80056F160000006F1600000869d0s1.
It shows “0” for entry 1 and entry 2.
# ./crsctl query css votedisk
0. 0 /dev/rdsk/c4t60060E80056F160000006F1600000469d0s1
1. 0 0
2. 0 0
Add another 2 votedisk while CRS is down.
# ./crsctl add css votedisk /dev/rdsk/c4t60060E80056F160000006F1600000669d0s1 -force
Now formatting voting disk: /dev/rdsk/c4t60060E80056F160000006F1600000669d0s1
successful addition of votedisk /dev/rdsk/c4t60060E80056F160000006F1600000669d0s1.
# ./crsctl add css votedisk /dev/rdsk/c4t60060E80056F160000006F1600000869d0s1 -force
Now formatting voting disk: /dev/rdsk/c4t60060E80056F160000006F1600000869d0s1
successful addition of votedisk /dev/rdsk/c4t60060E80056F160000006F1600000869d0s1.
# ./crsctl query css votedisk
0. 0 /dev/rdsk/c4t60060E80056F160000006F1600000469d0s1
1. 0 0
2. 0 0
3. 0 /dev/rdsk/c4t60060E80056F160000006F1600000669d0s1
4. 0 /dev/rdsk/c4t60060E80056F160000006F1600000869d0s1
located 5 votedisk(s).
Delete “0” votedisk. Looks good.
# ./crsctl delete css votedisk 0 -force
successful deletion of votedisk 0.
# ./crsctl query css votedisk
0. 0 /dev/rdsk/c4t60060E80056F160000006F1600000469d0s1
1. 0 0
2. 0 /dev/rdsk/c4t60060E80056F160000006F1600000669d0s1
3. 0 /dev/rdsk/c4t60060E80056F160000006F1600000869d0s1
located 4 votedisk(s).
# ./crsctl delete css votedisk 0 -force
successful deletion of votedisk 0.
# ./crsctl query css votedisk
0. 0 /dev/rdsk/c4t60060E80056F160000006F1600000469d0s1
1. 0 /dev/rdsk/c4t60060E80056F160000006F1600000669d0s1
2. 0 /dev/rdsk/c4t60060E80056F160000006F1600000869d0s1
located 3 votedisk(s).
Want to add mirrored OCR while CRS is down. It reports error “PROT-1”.
# ./ocrconfig -replace ocrmirror /dev/rdsk/c4t60060E80056F160000006F1600000669d0s5
PROT-1: Failed to initialize ocrconfig
After start CRS on all nodes. Mirrored OCR can be added.
# ./ocrconfig -replace ocrmirror /dev/rdsk/c4t60060E80056F160000006F1600000669d0s5
# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) : 1151724
Used space (kbytes) : 4608
Available space (kbytes) : 1147116
ID : 1986342521
Device/File Name : /dev/rdsk/c4t60060E80056F160000006F1600000469d0s5
Device/File integrity check succeeded
Device/File Name : /dev/rdsk/c4t60060E80056F160000006F1600000669d0s5
Device/File integrity check succeeded
Cluster registry integrity check succeeded
# cat /var/opt/oracle/ocr.loc
#Device/file getting replaced by device /dev/rdsk/c4t60060E80056F160000006F1600000669d0s5
ocrconfig_loc=/dev/rdsk/c4t60060E80056F160000006F1600000469d0s5
ocrmirrorconfig_loc=/dev/rdsk/c4t60060E80056F160000006F1600000669d0s5
local_only=false#
[detached]