Louis.Kim 의 제 3막

Hotbackup 도중 베리타스 에서 DB를 내리게 될 경우..

SQL> alter tablespace users begin backup;

Tablespace altered.

-----------------

Alert Log

SQL> alter database datafile '/oradata/EAIDB/users01.dbf' end backup;

Database altered.

SQL> alter database open;

Database altered.

SQL> recover database;

Media recovery complete.

SQL> alter database open;

Database altered.

$instfix -i -k "IY68975"

    All filesets for IY68975 were found.

<-- 현재 설치 되어 있는 패치

$instfix -i -k "IZ41855"

    There was no data for IZ41855 in the fix database.

<--현재 설치 되어 있지 않는 패치

Usage: instfix [-R Path] [-T [-M platform]] [-s string] 

          [ -k keyword | -f file ] [-d device] [-S] 

          [-p | [-i [-c] [-q] [-t type] [-v] [-F]]] [-a] 

Function: Installs or queries filesets associated with keywords or fixes.

        -a Display the symptom text (can be combined with -i, -k, or -f).

        -c Colon-separated output for use with -i. Output includes keyword

           name, fileset name, required level, installed level, status, and

           abstract.  Status values are < (down level), = (correct level),

           + (superseded), and ! (not installed).

        -d Input device (not valid with flags -i or -a).

        -F Returns failure unless all filesets associated with the fix

           are installed.

        -f Input file containing keywords or fixes. Use '-' for standard input.

           The -T option produces a suitable input file format for -f.

        -i Use with -k or -f option to display whether specified fixes or 

           keywords are installed.  Installation is not attempted.

           If neither -k nor -f is specified, all known fixes are displayed.

        -k Install filesets for a keyword or fix.

        -M Use with -T option to display information for fixes present

           on the media that have to do with the platform specified.

        -p Use with -k or -f to print filesets associated with keywords.

           Installation is not attempted when -p is used.

        -q Quiet option for use with -i.  If -c is specified, no heading is 

           displayed.  Otherwise, no output is displayed.

        -R User Specified Install Location

        -S Suppress multi-volume processing.

        -s Search for and display fixes on media containing a specified string.

        -T Display fix information for complete fixes present on the media.

        -t Use with -i option to limit search to a given type.  Currently

           valid types are 'f' (fix) and 'p' (preventive maintenance).

        -v Verbose option for use with -i.  Gives information about each

           fileset associated with a fix or keyword.

           to the environment provided.

# service o2cb configure

Configuring the O2CB driver.

This will configure the on-boot properties of the O2CB driver.

The following questions will determine whether the driver is loaded on

boot.  The current values will be shown in brackets ('[]').  Hitting

ENTER without typing an answer will keep that current value.  Ctrl-C

will abort.

Load O2CB driver on boot (y/n) [n]: y

Cluster stack backing O2CB [o2cb]: 

Cluster to start on boot (Enter "none" to clear) [ocfs2]: ocfscluster1

Specify heartbeat dead threshold (>=7) [31]: 

Specify network idle timeout in ms (>=5000) [30000]: 

Specify network keepalive delay in ms (>=1000) [2000]: 

Specify network reconnect delay in ms (>=2000) [2000]: 

Writing O2CB configuration: OK

Loading filesystem "configfs": OK

Mounting configfs filesystem at /sys/kernel/config: OK

Loading stack plugin "o2cb": OK

Loading filesystem "ocfs2_dlmfs": OK

Creating directory '/dlm': OK

Mounting ocfs2_dlmfs filesystem at /dlm: OK

Setting cluster stack "o2cb": OK

Checking O2CB cluster configuration : Failed

# o2cb_ctl -C -n ocfscluster1 -t cluster -a name=ocfscluster1

# o2cb_ctl -C -n ocfsrac1 -t node -a number=0 -a ip_address=192.168.131.100 -a ip_port=7777 -a cluster=ocfscluster1

# o2cb_ctl -C -n ocfsrac2 -t node -a number=1 -a ip_address=192.168.131.110 -a ip_port=7777 -a cluster=ocfscluster1

rac1:/root>service o2cb status

Driver for "configfs": Loaded

Filesystem "configfs": Mounted

Stack glue driver: Loaded

Stack plugin "o2cb": Loaded

Driver for "ocfs2_dlmfs": Loaded

Filesystem "ocfs2_dlmfs": Mounted

Checking O2CB cluster "ocfscluster1": Offline

rac1:/root>service o2cb start ocfscluster1

Setting cluster stack "o2cb": OK

Registering O2CB cluster "ocfscluster1": OK

Setting O2CB cluster timeouts : OK

2015-10-10 12:49:30.874: [ COMMCRS][9526]clsc_receive: (114771870) error 2

2015-10-10 21:54:31.062: [ COMMCRS][9526]clsc_receive: (114771870) error 2

2015-10-13 10:41:30.444: [  CRSEVT][11008]32CAAMonitorHandler :: 0:Could not join /oracle/crs/bin/racgwrap(check)

category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child

2015-10-13 10:41:30.449: [  CRSEVT][11008]32CAAMonitorHandler :: 0:Action Script /oracle/crs/bin/racgwrap(check) timed out for ora.glvndbp05.vip! (timeout=60)

2015-10-13 10:41:30.449: [  CRSAPP][11008]32CheckResource error for ora.glvndbp05.vip error code = -2

2015-10-13 10:43:03.482: [  CRSEVT][11011]32CAAMonitorHandler :: 0:Could not join /oracle/crs/bin/racgwrap(check)

category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child

2015-10-13 10:43:03.482: [  CRSEVT][11011]32CAAMonitorHandler :: 0:Action Script /oracle/crs/bin/racgwrap(check) timed out for ora.glvndbp05.vip! (timeout=60)

2015-10-13 10:43:03.482: [  CRSAPP][11011]32CheckResource error for ora.glvndbp05.vip error code = -2

2015-10-13 10:44:36.509: [  CRSEVT][11017]32CAAMonitorHandler :: 0:Could not join /oracle/crs/bin/racgwrap(check)

category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child

[CRSEVT] CAAMonitorHandler :: 0:Could not join .../crs/bin/racgwrap(check)

category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child

[CRSEVT] CAAMonitorHandler :: 0:Action Script ../crs/bin/racgwrap(check) timed out for 오라클.vip! (timeout=60)

10g RAC 환경에서 있는 bug 인데 racgmain check 데몬이 비정상적으로 fork 되면서 메모리 사용율이 올라가게 되어 결국 나중엔 시스템을 사용할수 없는 지경까지 이르게 됨.

 oracle 26024     1  0  Dec  6  ?         0:00 /oracle/crs/bin/racgmain check

 oracle 23218     1  0  Dec  6  ?         0:00 /oracle/crs/bin/racgmain check

 oracle 23179     1  0  Dec  4  ?         0:00 /oracle/ora10/bin/racgmain check

 oracle 27277     1  0  Dec  6  ?         0:00 /oracle/ora10/bin/racgmain check

 oracle  1028     1  0  Dec  5  ?         0:00 /oracle/ora10/bin/racgmain check

 oracle  7991     1  0  Dec  4  ?         0:00 /oracle/ora10/bin/racgmain check

 oracle 15324     1  0  Dec  3  ?         0:00 /oracle/ora10/bin/racgmain check

 oracle 14314     1  0  Dec  4  ?         0:00 /oracle/ora10/bin/racgmain check

 oracle 10895     1  0  Dec  4  ?         0:00 /oracle/ora10/bin/racgmain check

 oracle   404     1  0  Dec  3  ?         0:00 /oracle/ora10/bin/racgmain check

해결책은 아래와 같이 CRS bundle #2 patchset을 적용시키거나 workaround 방법을 써서 조치해 주어야 함.

=====================================================================================

Applies to:

Oracle Server - Enterprise Edition - Version: 10.2.0.1 to 11.1.0.6

Information in this document applies to any platform.

Oracle Server Enterprise Edition - Version: 10.1.0.2 to 10.2.0.4

Symptoms

System slows down and many "racgmain check" processes may appear in ps output.  CRS log would show the following messages.

oracle@HA5-ZW05:[/home/oracle] ps -ef|grep "racgmain check"|wc -l

1290

~~~~

CAAMonitorHandler :: 0:Action Script /opt/oracle/product/crs/bin/racgwrap(check) timed out for ora.harac1.vip! (timeout=60)

CheckResource error for ora.harac1.vip error code = -2

CAAMonitorHandler :: 0:Could not join /opt/oracle/product/crs/bin/racgwrap(check)

category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, 

other: Abnormal termination of the child

~~~~

Cause

crsd.bin invokes the racgmain to check the status of the resources that are managed by CRS. The racgmain is invoked through the wrapper script racgwrap. 

If the resource action timed out, crsd kills the action script, which is racgwrap, while racgmain process will not be killed. Over time, this might create lot of orphan racgmain processes in the system. This would eventually slow down the due to the resource contention at the OS level.

Internal bug:6196746  addresses this issue.

Solution

This is fixed in 11.1.0.7 patchset.. If you are running into this issue in 10gR2, please go ahead and apply 10.2.0.4 patchset and the latest CRS bundle patch. This fix is included  in CRS bundle patch from bundle #2 onwards.

Following option could be used as a temporary workaround until the patch is applied.

1.  Make a copy of racgwrap located under $ORACLE_HOME/bin and $CRS_HOME/bin on ALL Nodes

2.  Edit the file racgwrap and modify the last 3 lines from:

~~~

$ORACLE_HOME/bin/racgmain "$@"

status=$?

exit $status

to:

# Line added to fix for Bug 6196746

exec $ORACLE_HOME/bin/racgmain "$@"

~~~

3.  Kill all the orphan racgmain processes running.

$ ps -ef|grep "racgmain check"

oracle 18701 1 0 Aug 1 ? 0:00 /oracle/product/10.2.0/database/bin/racgmain check

oracle 14653 1 0 Aug 1 ? 0:00 /oracle/product/10.2.0/database/bin/racgmain check

oracle 24517 1 0 Aug 1 ? 0:00 /oracle/product/10.2.0/database/bin/racgmain check

$ kill -9 <PID of racgmain>

Symptoms

Symptom 1:

In 10g RAC on unix platform, VIP on one nodes always shows "UNKNOWN". 

Symptom 2:

When try to start it up, it report following error:

CRS-1028: Dependency analysis failed because of:

CRS-0223: Resource 'ora.rac-test1.vip' has placement error.  

Symptom 3:

In CRSD log, find following error:

2015-03-19 10:51:09.772: [  CRSRES][3737213248][ALERT]0`ora.rac-test1.vip` on member `rac-test1` has experienced an unrecoverable failure.

2015-03-19 10:51:09.772: [  CRSRES][3737213248]0Human intervention required to resume its availability.

2015-03-19 10:51:09.772: [  CRSEVT][3741415744]0CAAMonitorHandler :: 0:Could not execute /u01/app/oracle/product/10.2.0/crs_1/bin/racgwrap(stop) for ora.rac-test2.vip

category: 1234, operation: scls_canexec, loc: , OS error: 0, other: no exe permission, file /u01/app/oracle/product/10.2.0/crs_1/bin/racgwrap    ===> No execute permission for this file.

Solution

1. Shutdown all the resource of this node: instance/asm/nodeapps/crs

2. Change permission of those 2 files to 751 on the node with issue

chmod 751 /u01/app/oracle/product/10.2.0/crs_1/racg/admin/racgwrap

chmod 751 /u01/app/oracle/product/10.2.0/crs_1/bin/racgeut

3. Then you can startup all the resource and check whether VIP is online.

rac1:/root>mkfs.ocfs2 -b 4K -C 32K -N 2 -L "OCFS2Filesystem" /dev/sdb1

mkfs.ocfs2 1.8.0

Cluster stack: classic o2cb

/dev/sdb1 is apparently in use by the system; will not make a ocfs2 volume here!

<--이렇게 에러가 - _ -;;;

아래와 같이 상태 확인

rac1:/root>dmsetup status

rac-vote01: 0 614400 linear 

rac-control02: 0 204800 linear 

rac-undotbs2: 0 1638400 linear 

rac-control01: 0 204800 linear 

rac-undotbs1: 0 1638400 linear 

rac-system: 0 1638400 linear 

rac-users: 0 1638400 linear 

rac-redo06: 0 409600 linear 

rac-spfile: 0 204800 linear 

rac-temp: 0 1638400 linear 

rac-redo05: 0 409600 linear 

rac-redo04: 0 409600 linear 

rac-sysaux: 0 1638400 linear 

rac-redo03: 0 409600 linear 

rac-vote03: 0 614400 linear 

rac-redo02: 0 409600 linear 

rac-ocr02: 0 614400 linear 

rac-vote02: 0 614400 linear 

rac-redo01: 0 409600 linear 

rac-control03: 0 204800 linear 

rac-ocr01: 0 614400 linear 

젠장 내 잘못이다..기존에 raw device 구성 중에 이 짓을 했으니...ㅠㅠ

아래와 같이 지워주자..

rac1:/root>dmsetup remove_all

다시 진행해 보자..!!

rac1:/root>mkfs.ocfs2 -b 4K -C 32K -N 2 -L "OCFS2Filesystem" /dev/sdb1

mkfs.ocfs2 1.8.0

Cluster stack: classic o2cb

Label: OCFS2Filesystem

Features: sparse extended-slotmap backup-super unwritten inline-data strict-journal-super xattr indexed-dirs refcount discontig-bg

Block size: 4096 (12 bits)

Cluster size: 32768 (15 bits)

Volume size: 21467922432 (655149 clusters) (5241192 blocks)

Cluster groups: 21 (tail covers 10029 clusters, rest cover 32256 clusters)

Extent allocator size: 12582912 (3 groups)

Journal size: 134184960

Node slots: 2

Creating bitmaps: done

Initializing superblock: done

Writing system files: done

Writing superblock: done

Writing backup superblock: 3 block(s)

Formatting Journals: done

Growing extent allocator: done

Formatting slot map: done

Formatting quota files: done

Writing lost+found: done

mkfs.ocfs2 successful

  )

10.10.100.101 vip-linux1

10.10.100.102 vip-linux2

C:\> sqlplus system/manager@orcltest

COLUMN instance_name    FORMAT a13

COLUMN host_name        FORMAT a9

COLUMN failover_method  FORMAT a15

COLUMN failed_over      FORMAT a11

SELECT

    instance_name

  , host_name

  , NULL AS failover_type

  , NULL AS failover_method

  , NULL AS failed_over

FROM v$instance

UNION

SELECT

    NULL

  , NULL

  , failover_type

  , failover_method

  , failed_over

FROM v$session

WHERE username = 'SYSTEM';

INSTANCE_NAME HOST_NAME FAILOVER_TYPE FAILOVER_METHOD FAILED_OVER

------------- --------- ------------- --------------- -----------

orcl1         linux1

                        SELECT        BASIC           NO

위에서 설정한 SQL*Plus 세션에서 로그아웃 하지 않습니다!

위 쿼리를 수행한 다음, abort 옵션을 사용하여 linux1 노드의 orcl1 인스턴스를 셧다운 합니다. 이 작업을 수행하기 위해 아래와 같이 srvctl 커맨드라인 유틸리티를 사용합니다:

# su - oracle

$ srvctl status database -d orcl

Instance orcl1 is running on node linux1

Instance orcl2 is running on node linux2

$ srvctl stop instance -d orcl -i orcl1 -o abort

$ srvctl status database -d orcl

Instance orcl1 is not running on node linux1

Instance orcl2 is running on node linux2

다시 앞의 SQL 세션으로 돌아가, 버퍼에 저장된 SQL 구문을 재실행합니다: 

COLUMN instance_name    FORMAT a13

COLUMN host_name        FORMAT a9

COLUMN failover_method  FORMAT a15

COLUMN failed_over      FORMAT a11

SELECT

    instance_name

  , host_name

  , NULL AS failover_type

  , NULL AS failover_method

  , NULL AS failed_over

FROM v$instance

UNION

SELECT

    NULL

  , NULL

  , failover_type

  , failover_method

  , failed_over

FROM v$session

WHERE username = 'SYSTEM';

INSTANCE_NAME HOST_NAME FAILOVER_TYPE FAILOVER_METHOD FAILED_OVER

------------- --------- ------------- --------------- -----------

orcl2         linux2

                        SELECT        BASIC           YES

SQL> exit

위 실행 결과에서, 세션이 linux2 노드의 orcl2 인스턴스로 페일오버 되었음을 확인

CTF  : 신규접속자

- 한쪽 Instance 가 장애가 나더라도  자동으로 다른 Instance로 접속할  수 있게 함

- RAC 설치시 기본으로 됨

TAF  : 기존접속자

- 기존 접속자를 넘겨주는 기술

- 별도로 설정해야 사용가능 

* 이미지가 작게 보일 것 같아서 아래와 같이 유사 내용을 가져왔습니다.

INFO: Problem in configuration: PRCN-2061 : Failed to add listener ora.LISTENER.lsnr
INFO: PRCN-2065 : Port(s) 1521 are not available on the nodes given
INFO: PRCN-2067 : Port 1521 is not available across node(s) "hww-poc1-VIP,hww-poc2-VIP"
INFO: Oracle Net Listener Startup:
INFO:     Listener does not exists.
INFO: Check the trace file for details: /home/grid/app/grid/cfgtoollogs/netca/trace_Ora11g_gridinfrahome1-1410287PM2700.log
INFO: Oracle Net Services configuration failed.  The exit code is 1

[root@rac1 install]# pwd

/oracle/grid/crs/install

[root@rac1 install]# ./rootcrs.pl -deconfig -force -verbose

Using configuration parameter file: ./crsconfig_params

CRS-4535: Cannot communicate with Cluster Ready Services

CRS-4000: Command Stop failed, or completed with errors.

CRS-4535: Cannot communicate with Cluster Ready Services

CRS-4000: Command Delete failed, or completed with errors.

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac1'

CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1'

CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rac1'

CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1'

CRS-2673: Attempting to stop 'ora.evmd' on 'rac1'

CRS-2673: Attempting to stop 'ora.asm' on 'rac1'

CRS-2677: Stop of 'ora.evmd' on 'rac1' succeeded

CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded

CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac1'

CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac1' succeeded

CRS-2677: Stop of 'ora.drivers.acfs' on 'rac1' succeeded

CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.cssd' on 'rac1'

CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.crf' on 'rac1'

CRS-2677: Stop of 'ora.crf' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1'

CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1'

CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac1' has completed

CRS-4133: Oracle High Availability Services has been stopped.

Successfully deconfigured Oracle Restart stack

[root@rac2 install]# ./rootcrs.pl -deconfig -force -verbose

Using configuration parameter file: ./crsconfig_params

CRS-4047: No Oracle Clusterware components configured.

CRS-4000: Command Stop failed, or completed with errors.

CRS-4047: No Oracle Clusterware components configured.

CRS-4000: Command Delete failed, or completed with errors.

CRS-4047: No Oracle Clusterware components configured.

CRS-4000: Command Stop failed, or completed with errors.

You must kill ohasd processes or reboot the system to properly 

cleanup the processes started by Oracle clusterware

ACFS-9313: No ADVM/ACFS installation detected.

Either /etc/oracle/olr.loc does not exist or is not readable

Make sure the file exists and it has read and execute access

Failure in execution (rc=-1, 256, No such file or directory) for command /etc/init.d/ohasd deinstall

Successfully deconfigured Oracle Restart stack

[root@rac1 oracle]# $GRID_HOME/root.sh

[root@rac2 oracle]# $GRID_HOME/root.sh

[설명] 양쪽 노드에서 아래와 같이 Grid가 설치되어있는 홈 디렉토리에서 작업을 진행 하도록 하겠습니다. 우선 현재 crs, asm 등등 리소스를 확인 합니다. 어차피 띄워져 있어도 자동으로 모두 제거 하도록 하겠습니다. 아래 작업은 노드 1, 노드 2에서 모두 해주셔야 합니다.

[설명] 로컬 인벤토리에 있는 데이터 파일도 모두 제거 합니다.( 제거하지 않을 경우 그리드 설치시 에러가 발생 됩니다.)
[root@rac2 oraInventory]# $ORACLE_HOME/oraInventory/rm -rf *

[설명] 환경설정이 되어있으므로 아래와 같이 모두 제거 합니다. 
[root@rac2 u01]# rm -rf /etc/ora*

[설명] 데몬이 설정되어 있다면 rootdeinstall.sh를 반드시 수행 하셔야 합니다.  이후 아래 파일을 제거 하시기 바랍니다.
rm -f /etc/init.d/init.cssd 
rm -f /etc/init.d/init.crs 
rm -f /etc/init.d/init.crsd 
rm -f /etc/init.d/init.evmd 
rm -f /etc/rc2.d/K96init.crs
rm -f /etc/rc2.d/S96init.crs
rm -f /etc/rc3.d/K96init.crs
rm -f /etc/rc3.d/S96init.crs
rm -f /etc/rc5.d/K96init.crs
rm -f /etc/rc5.d/S96init.crs
rm -Rf /etc/oracle/scls_scr
rm -f /etc/inittab.crs 
cp /etc/inittab.orig /etc/inittab

[oracle@rac1:/oracle/grid]$ cd deinstall/

[oracle@rac1:/oracle/grid/deinstall]$ ./deinstall

Checking for required files and bootstrapping ...

Please wait ...

Location of logs /tmp/deinstall2015-09-11_09-37-29AM/logs/

############ ORACLE DEINSTALL & DECONFIG TOOL START ############

######################### CHECK OPERATION START #########################

## [START] Install check configuration ##

Checking for existence of the Oracle home location /oracle/grid

Oracle Home type selected for deinstall is: Oracle Grid Infrastructure for a Cluster

Oracle Base selected for deinstall is: /oracle/app/oracle

Checking for existence of central inventory location /oracle/app/oraInventory

Checking for existence of the Oracle Grid Infrastructure home /oracle/grid

The following nodes are part of this cluster: rac1,rac2

Checking for sufficient temp space availability on node(s) : 'rac1,rac2'

## [END] Install check configuration ##

Traces log file: /tmp/deinstall2015-09-11_09-37-29AM/logs//crsdc.log

Enter an address or the name of the virtual IP used on node "rac1"[rac1-vip]

 > 

The following information can be collected by running "/sbin/ifconfig -a" on node "rac1"

Enter the IP netmask of Virtual IP "192.168.131.120" on node "rac1"[255.255.255.0]

 > 

Enter the network interface name on which the virtual IP address "192.168.131.120" is active

 > 

Enter an address or the name of the virtual IP used on node "rac2"[rac2-vip]

 > 

The following information can be collected by running "/sbin/ifconfig -a" on node "rac2"

Enter the IP netmask of Virtual IP "192.168.131.130" on node "rac2"[255.255.255.0]

 > 

Enter the network interface name on which the virtual IP address "192.168.131.130" is active

 > 

Enter an address or the name of the virtual IP[]

 > 

Network Configuration check config START

Network de-configuration trace file location: /tmp/deinstall2015-09-11_09-37-29AM/logs/netdc_check2015-09-11_10-08-47-AM.log

Specify all RAC listeners (do not include SCAN listener) that are to be de-configured [LISTENER,LISTENER_SCAN1]:

Network Configuration check config END

Asm Check Configuration START

ASM de-configuration trace file location: /tmp/deinstall2015-09-11_09-37-29AM/logs/asmcadc_check2015-09-11_10-08-48-AM.log

ASM configuration was not detected in this Oracle home. Was ASM configured in this Oracle home (y|n) [n]: 

ASM was not detected in the Oracle Home

######################### CHECK OPERATION END #########################

####################### CHECK OPERATION SUMMARY #######################

Oracle Grid Infrastructure Home is: /oracle/grid

The cluster node(s) on which the Oracle home deinstallation will be performed are:rac1,rac2

Oracle Home selected for deinstall is: /oracle/grid

Inventory Location where the Oracle home registered is: /oracle/app/oraInventory

Following RAC listener(s) will be de-configured: LISTENER,LISTENER_SCAN1

ASM was not detected in the Oracle Home

Do you want to continue (y - yes, n - no)? [n]: y

A log of this session will be written to: '/tmp/deinstall2015-09-11_09-37-29AM/logs/deinstall_deconfig2015-09-11_09-37-37-AM.out'

Any error messages from this session will be written to: '/tmp/deinstall2015-09-11_09-37-29AM/logs/deinstall_deconfig2015-09-11_09-37-37-AM.err'

######################## CLEAN OPERATION START ########################

ASM de-configuration trace file location: /tmp/deinstall2015-09-11_09-37-29AM/logs/asmcadc_clean2015-09-11_10-08-51-AM.log

ASM Clean Configuration END

Network Configuration clean config START

Network de-configuration trace file location: /tmp/deinstall2015-09-11_09-37-29AM/logs/netdc_clean2015-09-11_10-08-51-AM.log

De-configuring RAC listener(s): LISTENER,LISTENER_SCAN1

De-configuring listener: LISTENER

    Stopping listener: LISTENER

    Warning: Failed to stop listener. Listener may not be running.

Listener de-configured successfully.

De-configuring listener: LISTENER_SCAN1

    Stopping listener: LISTENER_SCAN1

    Warning: Failed to stop listener. Listener may not be running.

Listener de-configured successfully.

De-configuring Naming Methods configuration file on all nodes...

Naming Methods configuration file de-configured successfully.

De-configuring Local Net Service Names configuration file on all nodes...

Local Net Service Names configuration file de-configured successfully.

De-configuring Directory Usage configuration file on all nodes...

Directory Usage configuration file de-configured successfully.

De-configuring backup files on all nodes...

Backup files de-configured successfully.

The network configuration has been cleaned up successfully.

Network Configuration clean config END

---------------------------------------->

The deconfig command below can be executed in parallel on all the remote nodes. Execute the command on  the local node after the execution completes on all the remote nodes.

Run the following command as the root user or the administrator on node "rac2".

/tmp/deinstall2015-09-11_09-37-29AM/perl/bin/perl -I/tmp/deinstall2015-09-11_09-37-29AM/perl/lib -I/tmp/deinstall2015-09-11_09-37-29AM/crs/install /tmp/deinstall2015-09-11_09-37-29AM/crs/install/rootcrs.pl -force  -deconfig -paramfile "/tmp/deinstall2015-09-11_09-37-29AM/response/deinstall_Ora11g_gridinfrahome1.rsp"

Run the following command as the root user or the administrator on node "rac1".

/tmp/deinstall2015-09-11_09-37-29AM/perl/bin/perl -I/tmp/deinstall2015-09-11_09-37-29AM/perl/lib -I/tmp/deinstall2015-09-11_09-37-29AM/crs/install /tmp/deinstall2015-09-11_09-37-29AM/crs/install/rootcrs.pl -force  -deconfig -paramfile "/tmp/deinstall2015-09-11_09-37-29AM/response/deinstall_Ora11g_gridinfrahome1.rsp" -lastnode

Press Enter after you finish running the above commands

<----------------------------------------

**여기서 Enter 치지 말고, console 를 하나 더 열어 root 로 위에 빨간색 명령어 진행

[root@rac2 ~]# /tmp/deinstall2015-09-11_09-37-29AM/perl/bin/perl -I/tmp/deinstall2015-09-11_09-37-29AM/perl/lib -I/tmp/deinstall2015-09-11_09-37-29AM/crs/install /tmp/deinstall2015-09-11_09-37-29AM/crs/install/rootcrs.pl -force  -deconfig -paramfile "/tmp/deinstall2015-09-11_09-37-29AM/response/deinstall_Ora11g_gridinfrahome1.rsp"

Using configuration parameter file: /tmp/deinstall2015-09-11_09-37-29AM/response/deinstall_Ora11g_gridinfrahome1.rsp

...

**기다리면 succeeded 등이 뜨면서 끝날때까지 기다린다.(난.. failed..Y Y)

Successfully deconfigured Oracle clusterware stack on this node <--이런식으로..

[root@rac1 ~]# 

[root@rac1 ~]# root

-bash: root: command not found

[root@rac1 ~]# /tmp/deinstall2015-09-11_09-37-29AM/perl/bin/perl -I/tmp/deinstall2015-09-11_09-37-29AM/perl/lib -I/tmp/deinstall2015-09-11_09-37-29AM/crs/install /tmp/deinstall2015-09-11_09-37-29AM/crs/install/rootcrs.pl -force  -deconfig -paramfile "/tmp/deinstall2015-09-11_09-37-29AM/response/deinstall_Ora11g_gridinfrahome1.rsp" -lastnode

...

Successfully deconfigured Oracle clusterware stack on this node

다시 deinstall 하는 곳으로 넘어와서 Enter를 친다.

<----------------------------------------

Setting the force flag to false

Setting the force flag to cleanup the Oracle Base

Oracle Universal Installer clean START

Detach Oracle home '/oracle/grid' from the central inventory on the local node : Done

Delete directory '/oracle/grid' on the local node : Done

Delete directory '/oracle/app/oraInventory' on the local node : Done

Delete directory '/oracle/app/oracle' on the local node : Done

Detach Oracle home '/oracle/grid' from the central inventory on the remote nodes 'rac2' : Done

Delete directory '/oracle/grid' on the remote nodes 'rac2' : Done

Delete directory '/oracle/app/oraInventory' on the remote nodes 'rac2' : Done

Delete directory '/oracle/app/oracle' on the remote nodes 'rac2' : Done

Oracle Universal Installer cleanup was successful.

Oracle Universal Installer clean END

## [START] Oracle install clean ##

Clean install operation removing temporary directory '/tmp/deinstall2015-09-11_09-37-29AM' on node 'rac1'

Clean install operation removing temporary directory '/tmp/deinstall2015-09-11_09-37-29AM' on node 'rac2'

## [END] Oracle install clean ##

######################### CLEAN OPERATION END #########################

####################### CLEAN OPERATION SUMMARY #######################

Following RAC listener(s) were de-configured successfully: LISTENER,LISTENER_SCAN1

Oracle Clusterware is stopped and successfully de-configured on node "rac1"

Oracle Clusterware is stopped and successfully de-configured on node "rac2"

Oracle Clusterware is stopped and de-configured successfully.

Successfully detached Oracle home '/oracle/grid' from the central inventory on the local node.

Successfully deleted directory '/oracle/grid' on the local node.

Successfully deleted directory '/oracle/app/oraInventory' on the local node.

Successfully deleted directory '/oracle/app/oracle' on the local node.

Successfully detached Oracle home '/oracle/grid' from the central inventory on the remote nodes 'rac2'.

Successfully deleted directory '/oracle/grid' on the remote nodes 'rac2'.

Successfully deleted directory '/oracle/app/oraInventory' on the remote nodes 'rac2'.

Successfully deleted directory '/oracle/app/oracle' on the remote nodes 'rac2'.

Oracle Universal Installer cleanup was successful.

Run 'rm -rf /etc/oraInst.loc' as root on node(s) 'rac1,rac2' at the end of the session.

Run 'rm -rf /opt/ORCLfmap' as root on node(s) 'rac1,rac2' at the end of the session.

Oracle deinstall tool successfully cleaned up temporary directories.

#######################################################################

############# ORACLE DEINSTALL & DECONFIG TOOL END #############

[oracle@rac1:/oracle/grid/deinstall]$ 

[root@rac1 ~]# rm -rf /etc/oraInst.loc

[root@rac1 ~]# rm -rf /opt/ORCLfmap

----------------------------------------------------------

[root@rac2 ~]# rm -rf /opt/ORCLfmap

[root@rac2 ~]# rm -rf /etc/oraInst.loc