In Pacemaker/Corosync cluster, there are may aspects/key elements that we need to understand before playing with cluster operations. Otherwise , it might cause unnecessary outage/downtime for the services. The most important elements are setting the preferred resource location , ordering (defining dependencies) , resource fail counts, resource-stickiness, colocation, clone, master/slave, promote/demote etc. Let’s go through the article and understand how these elements contributes in cluster operation.
Cluster Status:
Preferred Resource Location:
Pacaemaker/corosync allows resource to choose the preferred location. You can defined the preferred location using “pcs constraint” command. Here we are just intimating “UAKVM2” resource’s preferred node as UA-HA with a score of 50. The score here indicates how badly we would like the resource to run somewhere.
If you didn’t specify any score, then resource prefer to run all the time on UA-HA. The default score is INFINITY.
From PCS Man Page,
Using location constraint, you can also avoid specific node to run a particular resource.
At any time , you can remove the location constraint using the constraint ID.
To get the constraint id, use “–full” option.
Remove the constraint which we have created.
When defining constraints, you also need to deal with scores. Scores of all kinds are integral to how the cluster works. Practically everything from migrating a resource to deciding which resource to stop in a degraded cluster is achieved by manipulating scores in some way. Scores are calculated on a per-resource basis and any node with a negative score for a resource cannot run that resource. After calculating the scores for a resource, the cluster then chooses the node with the highest score. INFINITY is currently defined as 1,000,000. Additions or subtractions with it stick to the following three basic rules:
Any value + INFINITY = INFINITY
Any value – INFINITY = -INFINITY
INFINITY – INFINITY = -INFINITY
When defining resource constraints, you specify a score for each constraint. The score indicates the value you are assigning to this resource constraint. Constraints with higher scores are applied before those with lower scores. By creating additional location constraints with different scores for a given resource, you can specify an order for the nodes that a resource will fail over to.
Resource Ordering: (Defining Resource dependencies):
You need to define the resource ordering if you are not using the resource groups. Most of the cases , resources are need to start in sequential. For an example, File-system resource can’t be started prior to volume group resource. Similar to that IP resource should be on-line before starting the Apache resource.
Let’s assume that we do not have the resource group and following resources are configured in cluster
At this point , no constraint has been configured.
Plan for the Resource order:
Volume Group (LVM) – vgres
Filesystem – webvolfs (To store the website data)
IP address – Cluster IP (To access the website )
Apache – webres (To provide the web services)
To achieve the above resource order, use the following set of commands.
We have successfully configured the resource dependencies.
To remove the resource dependencies, use the following set of commands.
1. List the constraints with id.
2. Remove the order constraint using following command.
You need to configure the resource order constraint when you do not have the resource group. If you have resource group, it does the resource ordering and reduces the manual work.
Resource fail counts & Migration Threshold:
Migration thresholds defines that how many times failed resource should try to start on the running node. For an example, if you define migration-threshold=2 for a resource , it will automatically migrate to a new node after 2 failures.
To set the migration threshold, use the following command.
Resource fail count will come in to play, when it reaches the configured migration-threshold value. If this resource failed on running node , it will tried to start the resource on same node for 4 times. If it still fails, then it will move the resource to the next available node based configured constraint.
To see the fail counts , use the one of the following command.
Do you would like to reset the fail-counts manually ?
OR
Check the resource fail-count again.
Resource-Stickiness:
In some circumstances, it is highly desirable to prevent healthy resources from being moved around the cluster. Moving resources almost always requires a period of downtime. For complex services like Oracle databases, this period can be quite long. To address this, Pacemaker has the concept of resource stickiness which controls how much a service prefers to stay running where it is.
1. Check the Resource status:
2. Let’s stop the cluster services on UA-HA.
3. UAKVM2 resource group should be moved to UA-HA2 automatically.
4. Start the cluster service on UA-HA and see what happens to UAKVM2.
UAKVM2 is moved back to UA-HA node but this will create some sort downtime. If you have configured the resource-stickness, you could prevent the resource moving from one node to another.
Perform Step 2 to Step 4 & see the difference . UAKVM2 should be running on UA-HA2.
colocation:
When the location of one resource depends on the location of another one, we call this colocation. I would say colocaiton and resource order required when you are not using resource group.
Assuming that you have configured the volume group resource and File-system resource. You have also configured the resource order that which one to start first. But cluster might try to start volume group resource on one node and filesystem resource on other node. In such a cases, we need to tell to the cluster that run the filesystem resource where you are running the volume group resource.
Let’s see that how we can configure the colocation between vgres (LVM VG) & webvolfs (filesystem).
We have successfully configured the colocation parameter between vgres & webvolfs. In this case, webvolfs resource follows the vgres.
We will see clone,Master/slave and Promote/Demote stuffs on up coming articles.
Hope this article is informative to you. Share it ! Comment it !! Be Sociable !!!
The post RHEL 7 – Pacemaker – Define the Resource Behaviour – Part 9 appeared first on UnixArena.