diff options
Diffstat (limited to '__root__/doc/rgmanager-pacemaker/02.resources.txt')
| -rw-r--r-- | __root__/doc/rgmanager-pacemaker/02.resources.txt | 530 |
1 files changed, 530 insertions, 0 deletions
diff --git a/__root__/doc/rgmanager-pacemaker/02.resources.txt b/__root__/doc/rgmanager-pacemaker/02.resources.txt new file mode 100644 index 0000000..1a224fe --- /dev/null +++ b/__root__/doc/rgmanager-pacemaker/02.resources.txt @@ -0,0 +1,530 @@ +IN THE LIGHT OF RGMANAGER-PACEMAKER CONVERSION: 02/CLUSTERED RESOURCE PROPERTIES + +Copyright 2016 Red Hat, Inc., Jan Pokorný <jpokorny @at@ Red Hat .dot. com> +Permission is granted to copy, distribute and/or modify this document +under the terms of the GNU Free Documentation License, Version 1.3 +or any later version published by the Free Software Foundation; +with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. +A copy of the license is included in the section entitled "GNU +Free Documentation License". + + +Preface +======= + +This document elaborates on how selected resource relationship properties +(denoting the run-time behavior) formalized by the means of LTL logic maps +to particular RGManager (R) and Pacemaker (P) configuration arrangements. +Due to the purpose of this document, "selected" here means set of +properties one commonly uses in case of the former cluster resource +manager (R). + +Properties are categorised, each is further dissected based on +the property variants (basically holds or doesn't, but can be more +convoluted), and for each variants, the LTL model and R+P specifics +are provided. + + +Outline +------- + +Resource-resource interaction properites, PROPERTY(RESOURCE1, RESOURCE2) +. ORDERING +. COOCCURRENCE +Relative resource-node assignment properties, PROPERTY(RESOURCE) +. STICKY +. EXCLUSIVE +Explicit resource-node assignment properties, PROPERTY(RESOURCE, NODE) +. AFFINITY +Other resource properties, PROPERTY(RESOURCE) +. RECOVERY --> see RECOVERY(GROUP) and FAILURE-ISOLATION(GROUP, RESOURCE) +. ENABLED --> Pacemaker only, same as ENABLED(GROUP) + +# XXX: service ref=... + single node failover domains vs. clone + + + +Resource-resource interaction properites +======================================== + +Generally a relation expressed by a predicate PROPERTY(RESOURCE1, RESOURCE2), +implying modification of the behavior of cluster wrt. resource-resource +pair: + +PROPERTY(RESOURCE1, RESOURCE2) -> ALTER((RESOURCE1, RESOURCE2)) + + +Ordering (time-based interaction dependence of the resources/their states) +-------------------------------------------------------------------------- + +ORDERING ::= ORDERING(RESOURCE1, RESOURCE2, NONE) + | ORDERING(RESOURCE1, RESOURCE2, WEAK) + | ORDERING(RESOURCE1, RESOURCE2, STRONG) + | ORDERING(RESOURCE1, RESOURCE2, ASYMMETRIC) +. ORDERING(RESOURCE1, RESOURCE2, NONE) ... no time-based dependency + of the states +. ORDERING(RESOURCE1, RESOURCE2, WEAK) ... start of 2nd preceeded by start + of 1st (stop viceversa/LIFO), + but only when both events happen + at the same time +. ORDERING(RESOURCE1, RESOURCE2, STRONG) ... runtime of 2nd won't exceed + runtime of 1st +. ORDERING(RESOURCE1, RESOURCE2, ASYMMETRIC) + ... runtime of 2nd won't exceed + runtime of 1st (as STRONG) only + at the start phase (XXX then + the lifetimes are not + correlated???) + +assumed variables (+ constraints) to be combined with proper preconditions: +. A1, A2 in NODES +. B in RUNNABLE(A1), C in RUNNABLE(A2) + +propositional variables: +. a ... RUNNING(A1, B) +. b ... RUNNING(A2, C) +. c ... intention RUNNING(A1, B) +. d ... intention RUNNING(A2, C) + +R: driven by + XXX implicit and explicit ordering + +P: driven by `order` constraint (or `group` arrangement) + +ORDERING(B, C, WEAK) [2. weak ordering] +~~~~~~~~~~~~~~~~~~~~ +~a AND ~b AND c AND d -> (X a OR TRUE) AND X X b [start: B, then C] +a AND b AND ~c AND ~d -> X ~b AND X X ~a [stop: C, then B] + +R: TBD + +P: driven by specifying `kind` as `Optional` (or `score` as `0`) + # pcs constraint order B then C kind=Optional + or + # pcs constraint order B then C score=0 + +ORDERING(B, C, STRONG) [3. strong ordering] +~~~~~~~~~~~~~~~~~~~~~~ +~a AND ~b AND c AND d -> X a AND X X b [see weak ordering] +~a AND ~b AND d -> ~a AND ~b AND c AND d [stronger, follows as per previous???] +a AND b AND ~c AND ~d -> X ~b AND X X ~a [see weak ordering] +a AND b AND ~c -> a AND b AND ~c AND ~d [stronger, follows as per previous] + +R: TBD + +P: driven by omitting both `kind` and `score` (or specifying `kind` as + `Mandatory` or `score` as non-zero) + or (b) using `group` (which implies also `colocation` constraint) + - (a) + # pcs constraint order B then C + # pcs constraint order B then C kind=Mandatory + or + # pcs constraint order B then C score=1 + # XXX pcs constraint order set B C + - (b) + # XXX pcs resource group add SOMENAME C/RESOURCE2 B/RESOURCE1 + +ORDERING(B, C, ASYMMETRIC) [4. asymmetric ordering] +~~~~~~~~~~~~~~~~~~~~~~~~~~ +~a AND ~b AND c AND d -> X a AND X X b [see weak ordering] +~a AND ~b AND d -> ~a AND ~b AND c AND d [stronger, follows as per previous???] + +R: TBD + +P: driven by specifying `symmetrical` as `false` (default is `true`) + # pcs constraint order B then C symmetrical=false + + +Cooccurrence (location-based interaction dependence of the resources) +--------------------------------------------------------------------- + +COOCCURRENCE ::= COOCURRENCE(RESOURCE1, RESOURCE2, NONE) + | COOCURRENCE(RESOURCE1, RESOURCE2, POSITIVE) + | COOCURRENCE(RESOURCE1, RESOURCE2, NEGATIVE) + | COOCURRENCE(RESOURCE1, RESOURCE2, SCORE), + SCORE in {..., -1, 0, 1, ...}, 0~NONE, -INF~NEGATIVE, + +INF~POSITIVE +. COOCCURRENCE(RESOURCE1, RESOURCE2, NONE) ... not any occurrence + relationship (default) +. COOCCURRENCE(RESOURCE1, RESOURCE2, POSITIVE) ... positive occurrence + relationship (flat model) +. COOCCURRENCE(RESOURCE1, RESOURCE2, NEGATIVE) ... negative occurrence + relationship (flat model) +. COOCCURRENCE(RESOURCE1, RESOURCE2, SCORE) ... score-based/advisory + occurrence relationship + +note: COOCCURRENCE relation between RESOURCE1 and RESOURCE2 is not symmetric, + RESOURCE1 is "dependent", RESOURCE2 is "leader" + +assumed variables (+ constraints) to be combined with proper preconditions: +. A1, A2 in NODES +. B, C in RUNNABLE(A1) intersection RUNNABLE(A2) + +propositional variables: +. a ... RUNNING(A1, B) +. b ... RUNNING(A2, C) +. c ... A1 == A2 +. d ... RUNNING(A2, B) + +R: driven by various arrangements + +P: driven by `colocation` constraint (or `group` arrangement) + +COOCCURRENCE(B/RESOURCE1, C/RESOURCE2, NONE) [1. no cooccurrence] +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +R: default + +P: default, no need for that, otherwise specifying `score` as `0` + # pcs constraint colocation add B/RESOURCE1 with C/RESOURCE2 0 + +COOCCURRENCE(B/RESOURCE1, C/RESOURCE2, POSITIVE) [2. positive cooccurrence] +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +a AND b -> c [basic positive cooccurrence condition] +X b -> X d ["dependent" follows its "leader"] + +R: driven by grouping set of sequentially dependent resources + either using subsequent nesting so as to preserve the predestined + order unconditionally (referred to as **explicit ordering**) + or just enumerating them on the same level within the hierarchy + to allow for (reasonably preselected) reordering as designated + (via service.sh metadata) rules (referred to as **implicit ordering**) + hierarchically (in general, but using just the latter provision + it can be plain flat) into service/vm stanza + - `__independent_subtree` must not be used + +P: driven by (a) `colocation` constraint, specifying `INFINITY`, + or (b) using `group` (which implies also `order` constraint) + - (a) + # pcs constraint colocation add B/RESOURCE1 with C/RESOURCE2 + # pcs constraint colocation add B/RESOURCE1 with C/RESOURCE2 INFINITY + # pcs constraint colocation set C/RESOURCE2 B/RESOURCE1 + - (b) + # pcs resource group add SOMENAME C/RESOURCE2 B/RESOURCE1 + +COOCCURRENCE(B/RESOURCE1, C/RESOURCE2, NEGATIVE) [3. negative cooccurrence] +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +a AND b -> ~c [basic negative cooccurrence condition] +X b -> X ~d ["dependent" escapes its "leader"] + +R: driven/emulated solely by disjunct failover domains + - XXX and possibly with `follow_service.sl` in `central_processing` mode + +P: driven by `colocation` constraint, specifying `-INFINITY` + - using -INFINITY as value + # pcs constraint colocation add B/RESOURCE1 with C/RESOURCE2 -INFINITY + # pcs constraint colocation set C/RESOURCE2 B/RESOURCE1 setoptions score=-INFINITY + +COOCURENCE(B/RESOURCE1, C/RESOURCE2, SCORE) [4. score-based/advisory occurrence] +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +TBD + +R: XXX not supported unless a way to emulate this using `central_processing` + (?) or sharing the same `ordered` failover domain (?) + +P: driven by `colocation` constraint, specifying `SCORE` (value) + # pcs constraint colocation add B/RESOURCE1 with c/RESOURCE2 SCORE + # pcs constraint colocation set C/RESOURCE2 B/RESOURCE1 setoptions score=SCORE + + + +Relative resource-node assignment properties +============================================ + +Generally a relation expressed by a predicate PROPERTY(RESOURCE), +implying modification of the behavior of cluster wrt. the node +running RESOURCE: + +PROPERTY(RESOURCE) AND RUNNING(NODE, RESOURCE) -> ALTER(NODE) + + +Resource stickiness property (not moving back to preferred location) +-------------------------------------------------------------------- + +STICKY ::= STICKY(RESOURCE, FALSE) + | STICKY(RESOURCE, TRUE) + | STICKY(RESOURCE, STICKINESS), STICKINESS in {0, 1, 2, ...}, + 0~FALSE +. STICKY(RESOURCE, FALSE) ... unsticky/cruising resource (default) +. STICKY(RESOURCE, TRUE) ... sticky resource (flat model) +. STICKY(RESOURCE, STICKINESS) ... sticky resource (prioritized model) + +assumed variables (+ constraints) to be combined with proper preconditions: +. A1, A2 in NODES +. B in RUNNABLE(A1) AND B in RUNNABLE(A2) +. SCORE(B, A1) < SCORE(B, A2) + +propositional variables: +. a ... RUNNING(A1, B) +. b ... ACTIVE(A2) +. c ... RUNNING(A2, B) resource X running (relocation if A1 != A2) + +R: driven by `/cluster/rm/failoverdomains/failoverdomain/@nofailback` + - note: only applies to service/vm (not primitive resources) +P: driven by `stickiness` parameter + - group is a sum of stickiness values of underlying resources + + +STICKY(B/RESOURCE, FALSE) [1. model of unsticky/cruising resource] +~~~~~~~~~~~~~~~~~~~~~~~~~ +a AND b -> X c + +R: default, no need for that, otherwise specifying `@nofailback` as `0` + +P: default, no need for that, otherwise specifying `stickiness` as `0` + # pcs resource meta B/RESOURCE stickiness= + # pcs resource meta B/RESOURCE stickiness=0 + +STICKY(B/RESOURCE, TRUE) [2. model of sticky resource] +~~~~~~~~~~~~~~~~~~~~~~~~ +a AND b -> X a + +R: driven by specifying `@nofailback` as positive number + +P: driven by specifying `stickiness` as `INFINITY` + # pcs resource meta B/RESOURCE stickiness=INFINITY + +STICKY(B/RESOURCE, STICKINESS) [3. model of sticky resource with priorities] +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +TBD + +R: XXX not supported unless a way to emulate this using ordered failover + domains(?) + +P: driven by specifying `stickiness` as `STICKINESS` (value) + # pcs resource meta B/RESOURCE stickiness=STICKINESS + + +Node-exclusiveness resource property (optionally with priority-based preemption) +-------------------------------------------------------------------------------- + +EXCLUSIVE ::= EXCLUSIVE(RESOURCE, FALSE) + | EXCLUSIVE(RESOURCE, TRUE) + | EXCLUSIVE(RESOURCE, PRIORITY), PRIORITY in {0, 1, ...}, 0~FALSE +. EXCLUSIVE(RESOURCE, FALSE) ... non-exclusive resource (default) +. EXCLUSIVE(RESOURCE, TRUE) ... exclusive (flat model) +. EXCLUSIVE(RESOURCE, PRIORITY) ... exclusive (prioritized pre-emptive model) + +assumed variables (+ constraints) to be combined with proper preconditions: +. A in NODES +. B, C in RUNNABLE(A) +. I, J in {0, 1, ...}: I > J + +R: driven by `/cluster/rm/<service>/@exclusive` + - note: only applies to service/vm (not primitive resources) +P: driven by `utilization` constraint + (or possibly by a set of `colocation` constraints: + for all r in RESOURCES\RESOURCE: COOCCURRENCE(RESOURCE, r, -INF) [1] + (see [3. negative cooccurrence]) + +[1. model of non-node-exclusive/co-occurrence-positive resource, based on 2.] +for all B' in RUNNABLE(A): EXCLUSIVE(B', FALSE) [implies EXCLUSIVE(B, FALSE)] +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +~(a OR b) OR ~(~a -> ~b R ~a AND ~b -> ~a R ~b) += ~(a OR b) OR ~(~a -> ~b R ~a) OR ~(~b -> ~a R ~b) += ~(a OR b) OR ~(a OR ~b R ~a) OR ~(b OR ~a R ~b) += ~(a OR b) OR ~a AND ~(~b R ~a) OR ~b AND ~(~a R ~b) += ~(a OR b) OR ~a AND ~(~(a U b)) OR ~b AND ~(~(b U a)) += ~(a OR b) OR ~a AND a U b OR ~b AND b U a += ~(a OR b) OR b OR a += ~a AND ~b OR b OR a [= true, i.e. no restriction wrt. modelled property, QED] +. a ... RUNNING(A, B) +. b ... exists B' in 2^RUNNABLE(A)\{}: RUNNING(A, B') + +R: default (`@exclusive` = 0) + +P: default (no such purposefully full utilization of node resources + specified) + - pcs: support is arriving + (https://bugzilla.redhat.com/show_bug.cgi?id=1158500) + +[2. flat model of node-exclusive/co-occurrence-less resource, no preemption] +EXCLUSIVE(B, TRUE) +~~~~~~~~~~~~~~~~~~ +~(a OR b) OR (~a -> ~b R ~a) AND (~b -> ~a R ~b) [mutual exclusion] +. a ... RUNNING(A, B) +. b ... exists YS in 2^(RUNNABLE(A)\{B})\{}: for all Y in YS: RUNNING(A, Y) + (assuming valid state, i.e., exclusiveness property + would be recursively satisfied also within YS) + +R: driven by the arrangement: + - not `central_processing mode` and `@exclusive` != 0 + +P: driven by the arrangement: + - resources (presumably uniformly, but broken down to utilization + of comprising primitives) require full utilization of what + resources (presumably uniformly) each node provides + +[3. model of prioritized pre-emptive node-exclusive resource] +EXCLUSIVE(B, I) +EXCLUSIVE(C, J) +~~~~~~~~~~~~~~~ +~(a OR c) OR (~a -> ~c R ~a) AND (~c -> ~a R ~c), [mutual exclusion] +(b AND X a) -> (X ~b) [exlusivness priority wins] +. a ... RUNNING(A, B) +. b ... RUNNING(C, B) +. c ... exists YS in 2^({Y | Y in RUNNABLE(A) + AND (EXCLUSIVE(Y, FALSE) + OR exists K > I: EXCLUSIVE(Y, K))}\{B} + )\{}: for all Y' in YS: RUNNING(A, Y') + (assuming valid state, i.e., exclusiveness property + would be recursively satisfied also within YS) + +R: driven by the arrangement: + - only in `central_processing mode` + - https://access.redhat.com/site/node/47037 + - also see https://bugzilla.redhat.com/show_bug.cgi?id=744052#c4 + (also relevant to Pacemaker) + - exclusive resource with value of the respective parameter specifying + priority in the inverse sense (1 is highest, XXX or 0?) + +P: driven by the arrangement: + - see 1. + prioritization defined by the means of priority per primitive, + - https://access.redhat.com/site/solutions/65542 + - for 2., 3.: + - however modelling "exclusive resource cannot be started on node, + with non-exclusive resources already running" seems to be close + to impossible (XXX or in a intrusive way, like setting default + utilization + priority for those resources not overriding + these defaults)? + + + +Explicit resource-node(s) assignment property +============================================= + +Generally a relation expressed by a predicate PROPERTY(RESOURCE, NODE), +implying modification of the behavior of cluster wrt. resource-node +pair: + +PROPERTY(RESOURCE, NODE) -> ALTER((RESOURCE, NODE)) + + +Resource-node affinity +---------------------- + +AFFINITY ::= AFFINITY(RESOURCE, NODE, NONE) + | AFFINITY(RESOURCE, NODE, FALSE) + | AFFINITY(RESOURCE, NODE, TRUE) + | AFFINITY(RESOURCE, NODE, WEIGHT), WEIGHT in {..., -1, 0, 1, ...}, + 0~NONE, -INF~FALSE, +INF~TRUE +. AFFINITY(RESOURCE, NODE, NONE) ... no special affinity (default) +. AFFINITY(RESOURCE, NODE, FALSE) ... anti-affinity (the node cannot run) +. AFFINITY(RESOURCE, NODE, TRUE) ... node forms a set of executive nodes +. AFFINITY(RESOURCE, NODE, WEIGHT) ... prioritized model + +assumed variables (+ constraints) to be combined with proper preconditions: +. AN in NODES^2\{} +. B in intersection( { RUNNABLE(A) | A in AN } ) + +propositional variables: +. a ... RUNNING(A', B) +. b ... A' in AN +. c ... STICKY(B, FALSE) <-- see above +. d ... A' in {A'' | A'' in AN: WEIGHT(B, A'') == max({WEIGHT(B, A''') + | A''' in AN})} + +R: driven by the failoved domains arrangement for given service/vm + - XXX check that RGManager indeed behaves like Pacemaker's + `symmetric-cluster` +P: driven by `location` constraint + - only default `symmetric-cluster` is considered + +exists A in AN: AFFINITY(B, A, NONE) [1. no resource-node affinity] +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +R: driven by general arrangement patterns (possibilities with "NONE"): + - if for all A in AN: AFFINITY(B, A, NONE): + - nodes AN form attached failover domain, and this is either not `ordered` + or with the same `priority` for ordering + - alternatively, service/vm B has no failoverdomain explicitly attached + - else if for all A in AN: AFFINITY(B, A, NONE) OR AFFINITY(B, A, TRUE): + - nodes AN for which A in AN: AFFINITY(B, A, TRUE) form attached failover + domain, and this is not `restricted` (to allow those "NONE" nodes to + be ever used) + - or nodes AN form attached failover domain, and it is `ordered` with + two levels of priorities (for TRUE higher, for NONE lower) + - else if for all A in AN: AFFINITY(B, A, NONE) OR AFFINITY(B, A, FALSE): + - nodes AN for which A in AN: AFFINITY(B, A, NONE) form attached failover + domain, and this is either not `ordered` or with the same priority for + ordering, and has to be `restricted` to avoid slipping nodes + A in AN: AFFINITY(B, A, FALSE) + - else if for all A in AN: AFFINITY(B, A, NONE) OR AFFINITY(B, A, FALSE) + OR AFFINITY(B, A, TRUE): + - intersection of the previous two cases + - else if for all A in AN: AFFINITY(B, A, NONE) OR AFFINITY(B, A, WEIGHT), + -INFINITY < WEIGHT < INFINITY: + - either any WEIGHT < 0 becomes truncated to 0 (hence dropping losing + part of the configuration), or if such one present, the whole range + is rescaled to 0 < WEIGHT < INFINITY/numeric equivalent + and those member nodes having no affinity in this case have + lowest priority (hence highest `priority` value) + - either way, leads to AN forming attached failover domain, and it has + to be `ordered` with priorities as per previous point + - else? + +P: default, no need for that, otherwise specifying `score` as `0` + # pcs constraint location B prefers A= (??? to remove it as such) + # pcs constraint location B prefers A=0 + # pcs contraint location add SOMEID B A 0 + +exists A in AN: AFFINITY(B, A, FALSE) [2. antagonist resource-node affinity] +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +X a -> X ~b + +R: driven by this arrangement: + - if for all A in AN: AFFINITY(B, A, FALSE): + - nodes NODES\AN form attached failover domain, which has to + be `restricted` to prevent slipping onto any node in AN + - else: + - combine using patterns as in 1. + +P: by specifying (directly or indirectly) `score` as `-INFINITY` + # pcs constraint location B avoids A + # pcs constraint location B avoids A=INFINITY + # pcs constraint location B prefers A=-INFINITY (???) + # pcs contraint location add SOMEID B A -INFINITY + # crm_resource --ban --resource B --host A ... + +exists A in AN: AFFINITY(B, A, TRUE) [3. (positive) resource-node affinity] +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +X a -> X b + +R: driven by this arrangement: + - if for all A in AN: AFFINITY(B, A, TRUE): + - nodes AN form selected failover domain + - else: + - combine using patterns as in 1. + +P: by specifying (directly or indirectly) `score` as `INFINITY` + # pcs constraint location B prefers A + # pcs constraint location B prefers A=INFINITY + # pcs constraint location B avoids A=-INFINITY (???) + # pcs contraint location add SOMEID B A INFINITY + +[4. model of resource-node affinity with priorities] +for all A in AN: AFFINITY(B, A, -INFINITY < WEIGHT(B,A) < INFINITY) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +X a AND c -> X d [when not sticky, resource will run on the node + this resource has greatest affinity to] + +R: driven by this arrangement: + - see the respective pattern at 1. + +P: by specifying `score` as WEIGHT + # pcs constraint location B {prefers|avoids} A=WEIGHT(B,A) + # pcs constraint location add SOMEID B A WEIGHT(B,A) + + +Initial exclusive resource-node affinity +---------------------------------------- + +TBD: +https://bugzilla.redhat.com/994215#RFE--Control-which-node-a-service-autostarts-on + + + +References +========== + +[1] http://oss.clusterlabs.org/pipermail/users/2016-January/002197.html + +: vim: set ft=rst: <-- not exactly, but better than nothing |