By: Anju Garg
Oracle Clusterware 11g release 2 laid the foundation of policy-based cluster management by introducing server pools, which are logical groups of servers hosting the resources managed by Oracle Clusterware. Examples of resources that Oracle Clusterware manages are database instances, database services, application VIPs, and application components. Originally, member servers of server pools were restricted by a set of basic attributes characterizing servers. There was no way to distinguish between types of servers; all servers were considered to be equal in relation to their processors, physical memory, and other characteristics. This could lead to sub-optimal performance of the applications assigned to the server pools whose member servers did not meet the applications' requirements.
Oracle Grid Infrastructure 12c enhances the use of server pools by introducing server attributes e.g. MEMORY, CPU_COUNT etc. which are associated with each server. Server pools can be configured to have members belonging to a category of servers which share a particular set of attributes. Moreover, to reallocate servers to various server pools based on workload, the administrators can maintain a library of policies and switch between them as required rather than doing it manually.
This article discusses in detail the new features of policy based cluster management in 12c.
There are two styles of configuring resource management in a cluster:
– A particular server is always part of one server pool at any point in time
– A particular resource can run on more than one server (pool)
– More than one resource can run on one particular server (pool)
– A cluster can host more than one databases
– There can be only one instance of a particular RAC database on a specific server at any point in time
– Database services come as “singletons” or “uniform”. They run either on one server or on all servers in a specific server pool
– A particular service is bound to the database it runs on
– One Service can be assigned to only one server pool.
Thus, the server pool definitions effectively define a policy which dictates placement and prioritization of servers. Since the control is based on policies, this style is workload-management compliant and best for a large number of nodes. Policy-managed cluster management enables:
Limitations of policy-based management in Oracle Clusterware 11g R2:
In versions earlier than Oracle Clusterware 12c, the assignment of servers to server pools was based on the relative IMPORTANCE and the minimum and maximum number of servers associated with the server pool. Because there was no way to distinguish between types of servers, all servers were assumed to be homogeneous with regard to their CPU count, physical memory, and other characteristics. Hence, placement of servers in server pools could not be governed by server attributes. To prevent sub-optimal performance of applications resulting from their execution on servers with inadequate attributes, administrator needed to manually map workload to servers of differing compute power or other specific attributes accordingly. Besides being a burden to the administrator, such manual resource management is error prone when considering capacity requirements in the event of outages.
Moreover, although the server pools can be configured such that each of the applications is assigned to run in its own server pool, this configuration does not consider the fact that server requirements of applications may be different at different times of the day, week, or month. Reporting applications, for example, typically use more resources on weekends and month ends. To meet such requirements is cumbersome because changes must be made to the server pool configuration manually.
Policy-Based Cluster Management In 12c
With Oracle Clusterware 12c, policy based cluster management is enhanced in three important ways:
This article discusses the first two new features.
With Oracle Clusterware 12c, the notion of server categorization is introduced to support clusters with heterogeneous servers. Categorization allows servers to be differentiated based on their attributes which can further govern placement of servers in the server pools.
Server categorization works as follows:
These attributes are:
Using these server attributes, Oracle Grid Infrastructure 12c understands heterogeneous servers in a cluster by storing server attributes on a per server basis.
Example: Consider a cluster having 3 hub nodes and 2 leaf nodes.
- View extended attributes of servers in the cluster. Notice that:
- List categories created by default
- List servers in the two default categories
New user-defined categories can be created. Various server category attributes are:
Note that a server can belong to multiple categories at the same time. If you change the value of any of the server attributes listed in the EXPRESSION attribute of a server category, then you must restart the Oracle Clusterware technology stack on the affected servers before the new values take effect.
- Create new categories
Small: Leaf Node and Memory < 2000
Big: Hub Node and Memory >= 2000
- View new categories
- List servers in new categories
Server categories ensure that server assignments to server pools will meet the required minimums. If a server outage arises, criticality of the workload will be determined using IMPORTANCE of the associated server pool and predictable policies will be applied to re-allocate the resources in the cluster accordingly. If the remaining cluster size is insufficient to host the required workload, the same policies will be used to potentially move compute resources of a server pool of lesser IMPORTANCE to the server pool of higher IMPORTANCE in order to satisfy the compute resource requirements of business-critical workloads.
- Create server pools and associate them with big / small categories as follows:
- View server pools and associated categories:
Extended Policy Framework
Policy-managed environment in Oracle Clusterware 12c provides the infrastructure to automatically handle fluctuating server time demands of various applications in the cluster. It offers the capability to create various policies (server pool configurations) in accordance with business needs or application demand, so that pools provide the right service at the right time.
Typically, administrators create multiple policies to reflect different server time requirements based on business needs or demand, or based on calendar dates or times of the day. A collection of zero or more such user-defined policies is called a policy set. Each policy contains exactly one definition for each server pool controlled by the policy set. There is only one policy set defined for the cluster. These policies can be activated at will such that at any point in time only one policy is in effect for the cluster. Oracle Clusterware manages the server pools according to the active policy in the policy set. With a cluster configuration policy set, for example, more servers can be allocated to an OLTP workload during weekly business, whereas on the weekends and evenings, batch workloads can be assigned more servers, while performing transitions of server pool configuration automatically. The ability to change server pool properties as a group permits Clusterware to optimize the transition to be minimally disruptive, which would be very hard to do by issuing one command at a time.
To understand how to create and use a policy set and its included policies, consider the following case where we have four server pools: bigpool, smallpool, testpool and backuppool. Appropriate services can be registered to the server pools at any time. As soon as a server is assigned to a pool, the corresponding services will automatically start. Our requirement is as follows:
- Small application(s) will run on 2 small servers (smallpool)
- Big application(s) will run on 3 big servers (bigpool)
- Small application(s) will run on 1 small server (smallpool)
- Big application(s) will run on 2 big servers (bigpool)
- Test applications will run on 1 small server (testpool)
- Backup service will run on 1 big server (backuppool)
- Add Day and Night Policies
- Set the SERVER_POOL_NAMES policy set attribute to define the scope of the server pools that are controlled by the policy set.
- Set the attributes for the server pools in Day policy.
The Day policy
- Set the attributes for the server pools in Night policy.
The night policy enables
- View Day and Night Policies
- Policy Activation: The policy set is now configured and controlling the four server pools with two different policies. You can activate policies when necessary, prompting Oracle Clusterware to reconfigure a server pool according to each policy's configuration. Let us activate the Day policy
- Verify that as per the Day policy,
- Activate Night policy.
- Verify that as per the Night policy
Note that, because the server pools have different sizes, as servers move between server pools, some applications will be stopped and others will be started
- Modify the configuration of night policy so that backup runs on 2 big servers and big applications run on 1 big server. Reactivate the night policy
- Verify that as per the modified night policy, one big server (host02) has been taken away from bigpool and has been assigned to backuppool so that bigpool is left with only one server and backuppool has 2 servers as desired.
Thus, using the crsctl modify policyset command, Oracle Clusterware changed server pool configuration, moved servers according to the requirements of the policy, and stopped and started the applications.
crsctl modify policyset
- The assignment of servers to server pools was based on the relative IMPORTANCE and the minimum and maximum number of servers associated with the server pool. As there was no way to distinguish between types of servers, placement of servers in server pools could not be governed by server attributes.
- Although the server pools could be configured such that each of the applications was assigned to run in its own server pool, this configuration, however, did not consider the fact that server requirements of applications may be different at different times of the day, week, or month. Though server pool properties could be changed to meet such requirements, it was very cumbersome to do by issuing one command at a time.
- Server Categorization – Servers can be differentiated based on their attributes, which can further govern placement of servers in the server pools.
- Extended Policy framework – Policies can be configured to change server pool attributes in accordance with business needs or application demand, so that pools provide the right service at the right time. The administrators can maintain a library of policies and switch between them as required.
https://docs.oracle.com/database/121/CWADD/pbmgmt.htm#CWADD92894 (table, note in doc)