Submit Search
Upload
Rac 12c rel2_operational_best_practices_sangam_2017_as_pdf
•
5 likes
•
811 views
Anil Nair
Follow
Oracle RAC Operational Best Practices as presented at Sangam 2017
Read less
Read more
Technology
Report
Share
Report
Share
1 of 59
Download Now
Download to read offline
Recommended
Oracle RAC features on Exadata
Oracle RAC features on Exadata
Anil Nair
Oracle RAC - New Generation
Oracle RAC - New Generation
Anil Nair
New availability features in oracle rac 12c release 2 anair ss
New availability features in oracle rac 12c release 2 anair ss
Anil Nair
Rac 12c rel2_operational_best_practices_sangam_2017
Rac 12c rel2_operational_best_practices_sangam_2017
Anil Nair
Smart monitoring how does oracle rac manage resource, state ukoug19
Smart monitoring how does oracle rac manage resource, state ukoug19
Anil Nair
Using Machine Learning to Debug Oracle RAC Issues
Using Machine Learning to Debug Oracle RAC Issues
Anil Nair
Con8780 nair rac_best_practices_final_without_12_2content
Con8780 nair rac_best_practices_final_without_12_2content
Anil Nair
Step by Step instructions to install Cluster Domain deployment model
Step by Step instructions to install Cluster Domain deployment model
Anil Nair
More Related Content
What's hot
Anil nair rac_internals_sangam_2016
Anil nair rac_internals_sangam_2016
Anil Nair
New Generation Oracle RAC Performance
New Generation Oracle RAC Performance
Anil Nair
Using Machine Learning to Debug complex Oracle RAC Issues
Using Machine Learning to Debug complex Oracle RAC Issues
Anil Nair
Exadata master series_asm_2020
Exadata master series_asm_2020
Anil Nair
Collaborate 17 Oracle RAC 12cRel 2 Best Practices
Collaborate 17 Oracle RAC 12cRel 2 Best Practices
Anil Nair
Oracle RAC BP for Upgrade & More by Anil Nair and Markus Michalewicz
Oracle RAC BP for Upgrade & More by Anil Nair and Markus Michalewicz
Markus Michalewicz
Understanding oracle rac internals part 1 - slides
Understanding oracle rac internals part 1 - slides
Mohamed Farouk
Oracle RAC 12c Practical Performance Management and Tuning OOW13 [CON8825]
Oracle RAC 12c Practical Performance Management and Tuning OOW13 [CON8825]
Markus Michalewicz
Understanding oracle rac internals part 2 - slides
Understanding oracle rac internals part 2 - slides
Mohamed Farouk
Oracle RAC 12c Release 2 - Overview
Oracle RAC 12c Release 2 - Overview
Markus Michalewicz
Oracle RAC 12c Rel. 2 & Cluster Architecture Internals OOW17 by Anil Nair
Oracle RAC 12c Rel. 2 & Cluster Architecture Internals OOW17 by Anil Nair
Markus Michalewicz
Oracle Multitenant meets Oracle RAC - IOUG 2014 Version
Oracle Multitenant meets Oracle RAC - IOUG 2014 Version
Markus Michalewicz
Oracle RAC 12c Overview
Oracle RAC 12c Overview
Markus Michalewicz
Oracle RAC 12c Rel. 2 for Continuous Availability
Oracle RAC 12c Rel. 2 for Continuous Availability
Markus Michalewicz
What's New and Coming in Oracle ASM 12c Rel. 2 - by Jim Williams
What's New and Coming in Oracle ASM 12c Rel. 2 - by Jim Williams
Markus Michalewicz
Oracle RAC 12c Best Practices with Appendices DOAG2013
Oracle RAC 12c Best Practices with Appendices DOAG2013
Markus Michalewicz
Expert performance tuning tips for Oracle RAC
Expert performance tuning tips for Oracle RAC
SolarWinds
TFA Collector - what can one do with it
TFA Collector - what can one do with it
Sandesh Rao
Oracle RAC Internals - The Cache Fusion Edition
Oracle RAC Internals - The Cache Fusion Edition
Markus Michalewicz
Oracle RAC and Your Way to the Cloud by Angelo Pruscino
Oracle RAC and Your Way to the Cloud by Angelo Pruscino
Markus Michalewicz
What's hot
(20)
Anil nair rac_internals_sangam_2016
Anil nair rac_internals_sangam_2016
New Generation Oracle RAC Performance
New Generation Oracle RAC Performance
Using Machine Learning to Debug complex Oracle RAC Issues
Using Machine Learning to Debug complex Oracle RAC Issues
Exadata master series_asm_2020
Exadata master series_asm_2020
Collaborate 17 Oracle RAC 12cRel 2 Best Practices
Collaborate 17 Oracle RAC 12cRel 2 Best Practices
Oracle RAC BP for Upgrade & More by Anil Nair and Markus Michalewicz
Oracle RAC BP for Upgrade & More by Anil Nair and Markus Michalewicz
Understanding oracle rac internals part 1 - slides
Understanding oracle rac internals part 1 - slides
Oracle RAC 12c Practical Performance Management and Tuning OOW13 [CON8825]
Oracle RAC 12c Practical Performance Management and Tuning OOW13 [CON8825]
Understanding oracle rac internals part 2 - slides
Understanding oracle rac internals part 2 - slides
Oracle RAC 12c Release 2 - Overview
Oracle RAC 12c Release 2 - Overview
Oracle RAC 12c Rel. 2 & Cluster Architecture Internals OOW17 by Anil Nair
Oracle RAC 12c Rel. 2 & Cluster Architecture Internals OOW17 by Anil Nair
Oracle Multitenant meets Oracle RAC - IOUG 2014 Version
Oracle Multitenant meets Oracle RAC - IOUG 2014 Version
Oracle RAC 12c Overview
Oracle RAC 12c Overview
Oracle RAC 12c Rel. 2 for Continuous Availability
Oracle RAC 12c Rel. 2 for Continuous Availability
What's New and Coming in Oracle ASM 12c Rel. 2 - by Jim Williams
What's New and Coming in Oracle ASM 12c Rel. 2 - by Jim Williams
Oracle RAC 12c Best Practices with Appendices DOAG2013
Oracle RAC 12c Best Practices with Appendices DOAG2013
Expert performance tuning tips for Oracle RAC
Expert performance tuning tips for Oracle RAC
TFA Collector - what can one do with it
TFA Collector - what can one do with it
Oracle RAC Internals - The Cache Fusion Edition
Oracle RAC Internals - The Cache Fusion Edition
Oracle RAC and Your Way to the Cloud by Angelo Pruscino
Oracle RAC and Your Way to the Cloud by Angelo Pruscino
Similar to Rac 12c rel2_operational_best_practices_sangam_2017_as_pdf
AUSOUG - NZOUG-GroundBreakers-Jun 2019 - 19c RAC
AUSOUG - NZOUG-GroundBreakers-Jun 2019 - 19c RAC
Sandesh Rao
AIOUG-GroundBreakers-Jul 2019 - 19c RAC
AIOUG-GroundBreakers-Jul 2019 - 19c RAC
Sandesh Rao
The Top 5 Reasons to Deploy Your Applications on Oracle RAC
The Top 5 Reasons to Deploy Your Applications on Oracle RAC
Markus Michalewicz
C6 oracles storage_strategy_from_databases_to_engineered_systems_to_cloud
C6 oracles storage_strategy_from_databases_to_engineered_systems_to_cloud
Dr. Wilfred Lin (Ph.D.)
Using Snap Clone with Enterprise Manager 12c
Using Snap Clone with Enterprise Manager 12c
Pete Sharman
The Oracle RAC Family of Solutions - Presentation
The Oracle RAC Family of Solutions - Presentation
Markus Michalewicz
TechEvent 2019: Create a Private Database Cloud in the Public Cloud using the...
TechEvent 2019: Create a Private Database Cloud in the Public Cloud using the...
Trivadis
20191001 bkk-secret-of inno-db_clusterv1
20191001 bkk-secret-of inno-db_clusterv1
Ivan Ma
Mysql NDB Cluster's Asynchronous Parallel Design for High Performance
Mysql NDB Cluster's Asynchronous Parallel Design for High Performance
Bernd Ocklin
Přehled portfolia ODA a praktických případů v regionu EMEA
Přehled portfolia ODA a praktických případů v regionu EMEA
MarketingArrowECS_CZ
What's New in MySQL 8.0 @ HKOSC 2017
What's New in MySQL 8.0 @ HKOSC 2017
Ivan Ma
Oracle Storage a ochrana dat
Oracle Storage a ochrana dat
MarketingArrowECS_CZ
Meetup Oracle Database MAD_BCN: 1.1 Servicios de Oracle Database en la nube
Meetup Oracle Database MAD_BCN: 1.1 Servicios de Oracle Database en la nube
avanttic Consultoría Tecnológica
Why MySQL High Availability Matters
Why MySQL High Availability Matters
Matt Lord
MySQL 8.0 in a nutshell
MySQL 8.0 in a nutshell
OracleMySQL
State ofdolphin short
State ofdolphin short
Mandy Ang
OOW16 - Oracle Enterprise Manager 13c Cloud Control for Managing Oracle E-Bus...
OOW16 - Oracle Enterprise Manager 13c Cloud Control for Managing Oracle E-Bus...
vasuballa
#PCMVision: Oracle Hybrid Cloud Solutions
#PCMVision: Oracle Hybrid Cloud Solutions
PCM
6° Sessione Oracle - CRUI: Oracle Database Appliance:Il potere dell’ingegner...
6° Sessione Oracle - CRUI: Oracle Database Appliance:Il potere dell’ingegner...
Jürgen Ambrosi
Oracle Extended Clusters for Oracle RAC
Oracle Extended Clusters for Oracle RAC
Markus Michalewicz
Similar to Rac 12c rel2_operational_best_practices_sangam_2017_as_pdf
(20)
AUSOUG - NZOUG-GroundBreakers-Jun 2019 - 19c RAC
AUSOUG - NZOUG-GroundBreakers-Jun 2019 - 19c RAC
AIOUG-GroundBreakers-Jul 2019 - 19c RAC
AIOUG-GroundBreakers-Jul 2019 - 19c RAC
The Top 5 Reasons to Deploy Your Applications on Oracle RAC
The Top 5 Reasons to Deploy Your Applications on Oracle RAC
C6 oracles storage_strategy_from_databases_to_engineered_systems_to_cloud
C6 oracles storage_strategy_from_databases_to_engineered_systems_to_cloud
Using Snap Clone with Enterprise Manager 12c
Using Snap Clone with Enterprise Manager 12c
The Oracle RAC Family of Solutions - Presentation
The Oracle RAC Family of Solutions - Presentation
TechEvent 2019: Create a Private Database Cloud in the Public Cloud using the...
TechEvent 2019: Create a Private Database Cloud in the Public Cloud using the...
20191001 bkk-secret-of inno-db_clusterv1
20191001 bkk-secret-of inno-db_clusterv1
Mysql NDB Cluster's Asynchronous Parallel Design for High Performance
Mysql NDB Cluster's Asynchronous Parallel Design for High Performance
Přehled portfolia ODA a praktických případů v regionu EMEA
Přehled portfolia ODA a praktických případů v regionu EMEA
What's New in MySQL 8.0 @ HKOSC 2017
What's New in MySQL 8.0 @ HKOSC 2017
Oracle Storage a ochrana dat
Oracle Storage a ochrana dat
Meetup Oracle Database MAD_BCN: 1.1 Servicios de Oracle Database en la nube
Meetup Oracle Database MAD_BCN: 1.1 Servicios de Oracle Database en la nube
Why MySQL High Availability Matters
Why MySQL High Availability Matters
MySQL 8.0 in a nutshell
MySQL 8.0 in a nutshell
State ofdolphin short
State ofdolphin short
OOW16 - Oracle Enterprise Manager 13c Cloud Control for Managing Oracle E-Bus...
OOW16 - Oracle Enterprise Manager 13c Cloud Control for Managing Oracle E-Bus...
#PCMVision: Oracle Hybrid Cloud Solutions
#PCMVision: Oracle Hybrid Cloud Solutions
6° Sessione Oracle - CRUI: Oracle Database Appliance:Il potere dell’ingegner...
6° Sessione Oracle - CRUI: Oracle Database Appliance:Il potere dell’ingegner...
Oracle Extended Clusters for Oracle RAC
Oracle Extended Clusters for Oracle RAC
Recently uploaded
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IES VE
20150722 - AGV
20150722 - AGV
Jamie (Taka) Wang
20230104 - machine vision
20230104 - machine vision
Jamie (Taka) Wang
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
Eric D. Schabell
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
SkyPlanner
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
Md Hossain Ali
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
Adtran
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
Daniel Santiago Silva Capera
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
IES VE
201610817 - edge part1
201610817 - edge part1
Jamie (Taka) Wang
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
Jamie (Taka) Wang
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
David Newbury
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Aijun Zhang
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
Adam Moalla
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
YounusS2
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Will Schroeder
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
IES VE
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
Christian Posta
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Safe Software
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
shyamraj55
Recently uploaded
(20)
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
20150722 - AGV
20150722 - AGV
20230104 - machine vision
20230104 - machine vision
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
201610817 - edge part1
201610817 - edge part1
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Rac 12c rel2_operational_best_practices_sangam_2017_as_pdf
1.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Oracle RAC 12c Release 2 and Clusterware Internals Anil Nair Sr. Principal Product Manager, Oracle Real Application Clusters (RAC) Dec 25th, 2018 @RACMasterPM, http://www.linkedin.com/in/anil-nair-01960b6 http://www.slideshare.net/AnilNair27/ 20 Years of Continuous Innovation
2.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 2
3.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | • Please remember to provide feedback – Session ID Session ID: 1369 – Oracle RAC 12 Release 2 and Clusterware Internals • Reach out to us with follow up questions, suggestions etc. via Twitter @RACMasterPM Linkedin http://www.linkedin.com/in/anil-nair- 01960b6 Feedback is important for successful collaboration
4.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Program Agenda Cluster Domain Architecture RAC New Features Sample Problems and Resolution FAQ 1 2 4 3 4
5.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Program Agenda Cluster Domain Architecture1 2 5 3 4
6.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Oracle RAC 12c Rel. 2 Cluster Domain • Simplified Management – Fleet Management for installation, update, patching and maintenance • Reduced Local Overhead – Member Clusters benefit from the consolidation of common services on the Domain Services Cluster • Improved IO Performance – Utilizing consolidated shared storage 6 Centralized Management for Cluster Estates “too big to manage” otherwise
7.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | 7 Cluster Domain Application Member Cluster Uses ASM Database Member Cluster Uses local ASM Database Member Cluster Uses ASM Service Database Member Cluster Uses IO Service Domain Services Cluster (DSC) Shared ASM IO Service ACFS Service ASM Service TFA Service Management Service RHP Service
8.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | 8 The DSC – The Heart of the Cluster Domain Domain Services Cluster (DSC) IO Service ASM Service TFA Service Management Service Shared ASM ACFS Service RHP Service • The DSC hosts services that are consumed by Member Clusters, including: – Management Service for centralized and simplified management – Trace File Analyzer (TFA) for centralized diagnostics – Rapid Home Provisioning (RHP) for software fleet management – Storage Services (ACFS, ASM direct or indirect over IO Service)
9.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Domain Services Cluster (DSC) IO Service ASM Service Shared ASM ACFS Service RHP Service 9 The DSC Management Service TFA Service Management Service Applied Machine Learning for Database Diagnostics • Efficient diagnosis using Machine Learning • Automatically performs corrective actions to prevent possible issues • Provides simple alerts & recommendations for issues that require manual intervention Subject Matter ExpertLog ASH Metrics ML Knowledge Extraction Model Generation Human Supervision Application Optimized Models Feedback
10.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | 10 Rapid Home Provisioning Service Domain Services Cluster (DSC) IO Service ASM Service Shared ASM ACFS Service RHP Service TFA Service Management Service • Provision new pools onto base machines • DB and GI: provision, scale, patch, upgrade • Custom workflow framework • Notification model • Audit capabilities RHP Service Member Cluster Member Cluster Member Cluster Fleet Management installation, update, patching and maintenance
11.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | 11 Domain Services Cluster (DSC) Availability • Services provided by the DSC are unaffected by nodes joining/ leaving the DSC cluster • DSC can be patched and upgraded independently without affecting the services provided by DSC • Use Member Clusters for user databases (not the DSC)
12.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | 12 Cluster Domain Application Member Cluster Uses ASM Database Member Cluster Uses local ASM Database Member Cluster Uses ASM Service Database Member Cluster Uses IO Service Domain Services Cluster (DSC) Shared ASM IO Service ACFS Service ASM Service TFA Service Management Service RHP Service
13.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | • A Member Cluster is a Standalone Cluster utilizing shared services on the Domain Services Cluster • It automatically benefits from the management, TFA & RHP service. – ASM services are optional – and can be utilized as needed 13 Member Clusters = Standalone Cluster + Benefits Database Member Cluster Uses local ASM Database Member Cluster Uses ASM Service of DSC Database Member Cluster Uses ASM IO Service
14.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | 14 Same Tools and Commands for all Types of Deployments [GRID]> crsctl get cluster name CRS-6724: Current cluster name is 'SolarCluster' [GRID]> crsctl get cluster class CRS-41008: Cluster class is 'Standalone Cluster' [GRID]> crsctl get cluster type CRS-6539: The cluster type is 'flex'. [GRID]> crsctl get cluster name CRS-6724: Current cluster name is 'SalesCluster' [GRID]> crsctl get cluster class CRS-41008: Cluster class is ‘Database Member Cluster' [GRID]> crsctl get cluster type CRS-6539: The cluster type is 'flex'.
15.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | • Easy conversion from Standalone to Member Clusters • Member Cluster GI version can be higher or equal to the GI version on the Domain Services Cluster 15 Convert a Standalone Cluster to Member Cluster Convert Database Member Cluster Uses local ASM
16.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Program Agenda RAC New Features 1 2 16 3 4
17.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | 17 All Deployments Benefit from Oracle RAC Features PDB Isolation Hang Manager Choose the “right path” Zero Impact Patching Scalable Sequences
18.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | 1010101000010011100101001 1011100101001110010100101 0010010100001111010100101 1010101000010011100101001010010100100111001010010100100 1011100101001110010100101111110100111001010010111101101 0010010100001111010100101001010100100110101001010101000 18 • Using Oracle Multitenant, PDBs can be opened as singletons (in one database instance only), in a subset of instances or all in instances at once. • If certain PDBs are only opened on some instances, Pluggable Database Isolation – improves performance by • Reducing DLM operations for PDBs not open in all instances. • Optimizing block operations based on in-memory block separation. – improves availability by • Ensuring that instance failures of instances only hosting singleton PDBs will not impact other instances of the same RAC-based CDB. Optimized Singleton Workload Scaling Pluggable Database Isolation NEW IN 12.2PRIOR TO 12.2 Inter Cluster Messages
19.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Database Member Cluster Uses ASM Service 19 Pluggable Database and Service Isolation Provides higher Isolation in Member Cluster Consolidated Environments
20.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Hang Manager • Hang Manager “manages” database process(es) that are not progressing – Manage = detection and resolution – Considers cross-layer hangs • I.E: Hangs caused by a blocked ASM resource. • In addition, deadlocks are now resolved by Hang Manager • User defined control via PL/SQL • Early Warning exposed via (V$ view) 20 Resolving cross-layer hangs in the Cluster Domain Database Member Cluster Uses ASM IO Service IO Service ASM Service
21.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | 21 Resolving cross-layer hangs in the Cluster Domain Provides higher Availability in Member Cluster Consolidated Environments Database Member Cluster Uses ASM Service W W W B
22.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Oracle RAC Always Chooses the “Right Path” 22 • RAC determines the optimal path to serve blocks – network or disk • SSDs and NVMe storage technology continue to drive down latency • e.g. flash storage may provide better access times to data than the private network under high load • RAC takes those statistics into account Query BlockBlock Network congestion
23.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Database Member Cluster Uses ASM Service 23 The “Right Path” Ensures Performance on Member Clusters Effectively reducing the impact of competing workloads overwhelming the I/O on the DSC B u s y
24.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Zero Impact Patching • Zero Impact Patching enables patching of the Oracle Grid Infrastructure without interrupting database operations. • Patches are applied out-of-place and in a rolling fashion with one node being patched at a time while the database instance(s) on that node remain up and running. • Zero Impact Patching supports Oracle Real Application Clusters (RAC) databases on clusters with two or more nodes. 24 Never take down a database instance to patch Grid Infrastructure
25.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Operational Details of Zero Impact Patching • During patch application, the stack is monitored by a special CSSD running out of place • This CSSD performs Heartbeat operations • Once the patch is done, the special CSSD concedes control to the CSSD running out of the patched home • The steps are performed on all the nodes until the entire cluster is patched. • Any interruptions in normal activity during patching will trigger the CSSD to evict the node. 25
26.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | • Sequences cause Index block contention during heavy OLTP especially on tables which use sequence values as keys • This is not RAC specific but the problem gets amplified in RAC environments • The effects of this issue is mitigated via Scalable Sequences 26 Scalable Sequences
27.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | • New option in Create/Alter Sequence SCALE {Extend |NoExtend | NoScale } • When Scale is specified an internal algorithm uses the instance id and other session specific information to generate an offset • DBAs need to alter existing sequences • DBA/User/ALL_sequences will reflect the additional attributes 27 Scalable Sequences Operational details
28.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Program Agenda Sample Problems and Resolution 1 2 28 3 4
29.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Sample Problems and Resolution 29 Problem 1 – Node Eviction
30.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Node Eviction – Node 1 ocssd.trc [ Part 1 ] • 2010-08-13 17:00:22.818: [ CSSD][4106599328] clssnmPollingThread: node anair2 (2) at 50% heartbeat fatal, removal in 14.520 seconds • 2010-08-13 17:00:29.833: [ CSSD][4106599328] clssnmPollingThread: node anair2 (2) at 75% heartbeat fatal, removal in 7.500 seconds • 2010-08-13 17:00:37.337: [ CSSD][4106599328] clssnmPollingThread: Removal started for node anair2 (2), flags • 2010-08-13 17:00:37.340: [ CSSD][4085619616]clssnmCheckSplit: Node 2, anair2, is alive, DHB (1281744040, 1396854) more than disk timeout of 27000 after the last NHB (1281744011, 1367154) 30 Network heartbeat missing from node 2 for 15 consecutive seconds Network heartbeat is still missing Finally eviction starts Node 2 is still updating the Voting disks
31.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Node Eviction – Node 1 ocssd.trc [ Part 2 ] • 2010-08-13 17:00:37.340: [ CSSD][4085619616](:CSSNM00007:) clssnmrEvict: Evicting node 2, anair2, from the cluster in incarnation 169934272, node birth incarnation 169934271, death incarnation 169934272, stateflags 0x24000 • 2010-08-13 17:01:07.705: [ CSSD][4043389856]clssgmCMReconfig: reconfiguration successful, incarnation 169934272 with 1 nodes, local node number 1, master node number 1 31 Eventually Node 2 eviction process starts Concluding with a reconfiguration
32.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Node Eviction – Node 2 ocssd.trc [ Part 1 ] • 2010-08-13 17:00:26.213: [ CSSD][4073040800] clssnmPollingThread: node anair1 (1) at 50% heartbeat fatal, removal in 14.540 seconds • 2010-08-13 17:00:40.702: [ CSSD][4073040800] clssnmPollingThread: Removal started for node anair1 (1), flags • 2010-08-13 17:00:40.706: [ CSSD][4052061088]clssnmCheckSplit: Node 1, anair1, is alive, DHB (1281744036, 1243744) more than disk timeout of 27000 after the last NHB (1281744007, 1214144) 32 Huh! So who is right? It also detects that Node 1 is still updating the Voting disks
33.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Node Eviction – Node 2 ocssd.trc [ Part 2 ] • 2010-08-13 17:00:40.707: [ CSSD][4052061088](:CSSNM00008:)clssnmCheckDskInfo: Aborting local node to avoid splitbrain. Cohort of 1 nodes with leader 2, anair2, is smaller than cohort of 1 nodes led by node 1, anair1, based on map type 2 • 2010-08-13 17:00:40.707: [ CSSD] [4052061088]################################### 2010-08-13 17:00:40.707: [ CSSD][4052061088]clssscExit: CSSD aborting from thread clssnmRcfgMgrThread 2010-08-13 17:00:40.707: [ CSSD] [4052061088]################################### 33 it correctly detects and aborts local node to prevent split brain And does the right thing
34.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | So what is the conclusion ? 34 Hmmm….I think it is the network.
35.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | What does OS Watcher say? • netstat does not show any issues # grep "zzz|udpInOverflows|ipReasmFails" • OSW data itself is missing possibly due to scheduling issues • Just prior to the issue Top reports top - 13:23:52 up 25 days, 21:08, 1 user, load average: 3.43, 3.01, 3.02 Cpu(s): 16.8%us, 23.2%sy, 0.0%ni, 56.5%id, 3.1%wa, 0.1%hi, 0.3%si, Mem: 74027752k total, 73689744k used, 338008k free, 1516k Swap: 16771852k total, 9069988k used, 7701864k free, 25836528k PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1049 root 11 -5 0 0 0 R 55.0 0.0 0:57.42 [kswapd0] 35 No issues seen in netstat & traceroute Oh no! It is swapping Why is the OSW data missing?
36.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Sample Problems and Resolution 36 Problem 2 – Slow I/O
37.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | • Start the Database on all instances to simulate physical reads • Simulate I/O on shared storage – Loading data (sqlldr,expdp) OR – Swingbench • **behavior may depend on your test setup, HCA, HBA etc 37 Simulate I/O performance issue swingbench swingbench
38.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | chactl query diagnosis-db 38 $ chactl query diagnosis -db sales -start "2017-03-31 10:00:50" -end "2017-03-31 10:25:50“ 2017-03-31 10:01:10.0 Database sales DB Control File IO Performance (sales_1) [detected] 2017-03-31 10:01:10.0 Database sales DB Control File IO Performance (sales_2) [detected] 2017-03-31 10:01:13.0 Database sales DB CPU Utilization (sales_2) [detected] 2017-03-31 10:01:33.0 Database salesDB Log File Switch (sales_1) [detected] Consolidates and displays information from all instances
39.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | What does OS Watcher say? • iostat confirms I/O performance issue archive/oswiostat/xxxxxxxx_iostat_17.03.31.1000.dat • Increase in Reads/writes Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util xvda 0.00 0.00 0.00 4.00 0.00 32.00 8.00 0.00 0.25 0.25 0.10 xvdb 0.00 0.00 6.50 2.00 146.00 21.00 19.65 0.00 0.24 0.24 0.20 • --- • --- Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util xvda 0.00 0.00 0.00 144.00 434.00 53.00 12.00 0.00 0.45 1.13 0.10 xvdb 0.00 0.00 6.50 219.00 414.00 67.00 13.65 0.00 0.33 9.44 0.20 39 There is more overall I/O as seen by the increase in the number of writes, reads * values may change depending on test env. **Output has been formatted for presentation
40.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Sample Problems and Resolution 40 Problem 3 – CPU bound
41.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | • Ensure Grid Infrastructure is running for at least an hour • Run some normal load • Simulate excessive CPU using a CPU stressing program on 3 nodes – stress -- C program 41 Simulate CPU load stress stress stress
42.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | chactl query diagnosis -cluster 42 $chactl query diagnosis –start "2017-03-31 10:00:50" -end "2017-03-31 10:25:50“ 2017-03-31 10:01:25.0 Host rwsxxxxx9 Host Memory Consumption [detected] 2017-03-31 10:01:29.0 Host rwsxxxxx0 Host Memory Consumption [detected] 2017-03-31 10:01:23.0 Host rwsxxxxx1 Host Memory Consumption [detected] Problem: Host Memory Consumption Description: CHA detected that more memory than expected is consumed on this server. The memory is not allocated by sessions of this database. Cause: The Cluster Health Advisor (CHA) detected an increase in memory consumption by other databases or by applications not connected to a database on this node. Action: Identify the top memory consumers by using the Cluster Health Monitor (CHM). Note: This is a 4 node cluster but the memory consumption issue is reported on 3 of the 4 nodes
43.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | chatcl actions & resolutions in text or html format 43
44.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | What does OS Watcher say? • OSWatcher confirms chactl diagnosis • mpstat (platform dependent) zzz ***Fri Mar 31 10:10:29 PST 2017 10:10:29 CPU %usr %nice %sys %iowait %irq %soft %steal %guest %idle 10:10:29 all 74.01 0.00 7.95 12.20 0.00 0.13 0.13 0.00 5.10 10:10:29 0 44.89 0.00 8.79 12.09 0.00 0.00 0.00 0.00 31.23 10:10:29 1 56.00 0.00 4.00 18.00 0.00 0.00 0.00 0.00 23.00 • Top also reports the CPU stress program Tasks: 454 total, 4 running, 450 sleeping, 0 stopped, 0 zombie Cpu(s): 94.9%us, 4.8%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.3%si, 0.0%st PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 20752 racusr 20 0 2498m 45m 39m R 85.9 0.3 752:32.40 stress 44 System is indeed CPU starved * values may change depending on test env. **Output has been formatted for presentation
45.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Sample Problems and Resolution 45 Problem 4 – Why did my Database Instance move to different node?
46.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | • Provides common view of cluster- wide activities in a co-ordinated fashion • Customer readable summary of all actions in a cluster • Each activity is a summary of the issue – Further details are in the trace files 46 Clusterware Activity Log On Friday, my instances were running on Nodes 1 & 2, but today it is only running on Node 1
47.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | 47 crsctl query calog -- Syntax $ crsctl query calog -h Usage: crsctl query calog [-aftertime <after_timestamp>] [-beforetime <before_timestamp>] [-duration <time_interval> | - follow] [-filter <filter_expression>] [-fullfmt | -xmlfmt] Lists the cluster activity log activities matching the specified criteria Where after_timestamp Time in format "YYYY-MM-DD HH24:MI:SS[.FF][[+-]HH:MM]" OR "YYYY-MM-DD" OR "HH24:MI:SS[.FF][[+-]HH:MM]" before_timestamp Time in format "YYYY-MM-DD HH24:MI:SS[.FF][[+-]HH:MM]" OR "YYYY-MM-DD" OR "HH24:MI:SS[.FF][[+-]HH:MM]" time_interval Time interval in format "DD HH:MM:SS" filter_expression Refer to product documentation. -follow Gives continuous output -fullfmt Detail output format -xmlfmt XML output format
48.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | 48 crsctl query calog output $ crsctl query calog -aftertime "2017-03-08 15:09:46.522-07:00" 2017-04-12 20:05:04.668000 : Attempting to start 'ora.anair1.vip' on ‘anair1' : 14920191617156230/1194/11 : ….. 2017-04-12 20:05:06.559000 : Attempting to start 'ora.LISTENER.lsnr' on ‘anair1' : 14920191617156230/1194/16 … 2017-04-12 20:05:32.038000 : Start of 'ora.FRA.dg' on ‘anair1' succeeded : 14920191617156230/1194/27 : 2017-04-12 20:05:32.040000 : Attempting to start 'ora.sales.db' on ‘anair1' : 14920191617156230/1194/28 : 2017-04-12 20:05:59.415000 : Start of 'ora.sales.db' on ‘anair1' succeeded : 14920191617156230/1194/30 : -- Format of output records is: DATE & TIME (YYYY-MM-DD HH24:MI:SS[.FF][[+-]HH:MM]): Event text: ACTID Possibly network issues caused VIP relocation
49.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | 49 crsctl query calog filter output using actid $ crsctl query calog -filter "actid == 14920191617156230/2449732/1" 2017-08-03 16:30:24.678000 : Attempting to start 'ora.sscdb.db' on ‘anair1' : 14920191617156230/2449732/1 : 2017-08-03 16:30:24.698000 : Start of 'ora.sscdb.db' on ‘anair1' succeeded : 14920191617156230/2449732/1 : $ crsctl query calog -filter "actid ~= “14920191617156230” 2017-08-03 16:25:20.658000 : Stop of 'ora.sscdb.test.svc' on anair1' succeeded : 14920191617156230/2449007/2 : -filter: Use ~= or == on actid to find related actions
50.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | crsctl query calog filtered output using name $crsctl query calog -filter "entity_name == ora.sscdb.db" 2017-04-12 18:59:28.330000 : Attempting to start 'ora.sscdb.db' on ‘anair2' : 14919830844562257/9074/1 : 2017-04-12 18:59:28.334000 : Attempting to start 'ora.sscdb.db' on ‘anair1' : 14919830844562257/9074/2 : 2017-04-12 18:59:28.340000 : Attempting to start 'ora.sscdb.db' on ‘anair1' : 14919830844562257/9074/3 : 2017-04-12 18:59:28.694000 : Resource 'ora.sscdb.db' has been modified. : 14919830844562257/9074/4 : 2017-04-12 18:59:28.800000 : Resource 'ora.sscdb.db' has been modified. : 14919830844562257/9074/5 : … 2017-04-12 19:32:39.631000 : Attempting to stop 'ora.sscdb.db' on ‘anair2' : 14919830844562257/9853/2 : 2017-04-12 19:32:42.572000 : Stop of 'ora.sscdb.db' on ‘anair2' succeeded : 14919830844562257/9853/4 : 2017-04-12 19:45:01.175000 : Attempting to stop 'ora.sscdb.db' on ‘anair1' : 14919830844562257/10151/4 : 2017-04-12 19:45:04.176000 : Stop of 'ora.sscdb.db' on ‘anair1' succeeded : 14919830844562257/10151/7 : 50
51.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Additional examples for querying calog • crsctl query calog -duration "00 00:02:00“ << last two minutes >> • crsctl query calog -duration "00 02:00:00“ << last two hours >> • crsctl query calog –follow << like tail… >> • crsctl query calog -aftertime "2017-03-08 17:49:12.397" –follow • crsctl query calog -filter "entity_name == foo“ << activity on foo >> • crsctl query calog -filter "actid == 145567015239630019/49543/1" • crsctl query calog –xmlfmt << in xml format >> 51
52.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Program Agenda FAQ 1 2 52 3 4
53.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | 53 Frequently asked Question # 1 Why does MGMT DB need so much space?
54.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Cluster Type Redundancy MGMT DG (GB) Domain Services Cluster (2 Node DSC with 4 Member Clusters of 2 Nodes each) External 188 Normal 376 High 564 Flex 376 Standalone Cluster (4 Node Cluster) External 38 Normal 76 High 114 Flex 76 • Oracle GI 12c Release 2 feature AHF (Autonomous Health Framework) collects, Co-relates & stores diagnostics data from OS and DB in MGMT • In DSC, one PDB per member cluster is provisioned to store member cluster diagnostics data • The data is used by AHF components like cluster health advisor to both prevent and help diagnose issues 54 GIMR space requirements
55.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | 55 Frequently asked Question # 2 Should I continue to use Oswatcher?
56.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | Should I continue to use OSWatcher? • Hopefully by now, the value benefits of AHF is clear • AHF continues to be enhanced to improve – Diagnosing large number of deployments – Correlating data to speed diagnosis – Preventing issues in the first place – Utilizing latest technologies like Machine learning • AHF is mandatory (12.2+) but customers can choose to use OSW along with AHF 56 Learn more about Autonomous Health Framework https://www.slideshare.net/MarkusMichalewicz/cl uster-health-advisor-cha-deep-dive-by-mark- scardina
57.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | • opatch automatically patches MGMT database if required • clients of MGMT connect using encrypted credentials • MGMT listener automatically maintained by the clusterware agent 57 Frequently asked Question # 3 But it is still one more database for me to manage
58.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | 58
59.
Copyright © 2017,
Oracle and/or its affiliates. All rights reserved. | 59
Download Now