Replica

Usage example

Basically, when you want a simple replica configuration without any hubs, you can use create_topology function. In more complex cases you have to use our Replica API to build your own topology exactly the way you want it. Still, it is better if you’ll use the ‘create_topology’ method for basic initial setup and then you can continue to expand it.

from lib389.topologies import create_topology

topology = create_topology({ReplicaRole.MASTER: 2,
                            ReplicaRole.CONSUMER: 2})

For basic Replica operations (the rest in the docs bellow):

from lib389.replica import Replicas

replicas = Replicas(standalone)
# Enable replication
# - changelog will be created
# - replica manager will be with the defaults
# - replica.create() will be executed
replica = replicas.enable(suffix=DEFAULT_SUFFIX,
                          role=ReplicaRole.MASTER,
                          replicaID=REPLICAID_MASTER_1)

# Or you can get it as usual DSLdapObject
replica = replicas.list()[0]

# Roles - ReplicaRole.MASTER, ReplicaRole.HUB, and ReplicaRole.CONSUMER
# For masters and hubs you can use the constants REPLICAID_MASTER_X and REPLICAID_HUB_X
# Change X for a number from 1 to 100 - for role ReplicaRole.MASTER only

# Disable replication
# - agreements and replica entry will be deleted
# - changelog is not deleted (but should?)
replicas.disable(suffix=DEFAULT_SUFFIX)

# Get RUV entry
replicas.get_ruv_entry()

# Get DN
replicas.get_dn(suffix)

# Promote
replicas.promote(suffix=DEFAULT_SUFFIX,
                 newrole=ReplicaRole.MASTER,
                 binddn=REPL_BINDDN,
                 rid=REPLICAID_MASTER_1)
# Demote
replicas.demote(suffix=DEFAULT_SUFFIX,
                newrole=ReplicaRole.CONSUMER)
# Test, that replication works
replicas.test(master2)

# Additional replica object methods
# Get role
replica.get_role()

replica.deleteAgreements()

Module documentation

class lib389.replica.ReplicationManager(suffix, logger=None)[source]

The lib389 replication manager. This is used to coordinate replicas and agreements between servers.

Unlike the raw replicas / agreement types that manipulate the servers configuration, this is a “high level” coordination type. It’s capable of taking multiple instances and joining them. It consumes many lib389 types like Replicas, Agreements and more.

It is capable of creating the first master in a topoolgy, joining masters and consumers to that topology, populating per-server replication credentials, dynamic rid allocation, and more.

Unlike hand management of agreements, this is able to take simpler steps to agreement creation. For example:

repl = ReplicationManager(<suffix>) repl.create_first_master(master1) repl.join_master(master1, master2)

Contrast to previous implementations of replication which required much more knowledge and parameters, this is able to securely add masters.

Parameters:
  • suffix (str) – The suffix to replicate.
  • logger (python logging) – A logging interface
create_first_master(instance)[source]

In a topology, this creates the “first” master that has the database and content. A number of bootstrap tasks are performed on this master, as well as creating it’s replica type.

Once the first master is created, all other masters can be joined to it via “join_master”.

Parameters:instance (lib389.DirSrv) – An instance
disable_to_master(to_instance, from_instances=[])[source]

For all masters “from” disable all agreements “to” instance.

Parameters:
  • to_instance (lib389.DirSrv) – The instance to stop recieving data.
  • from_instances (list[lib389.DirSrv]) – The instances to stop sending data.
enable_to_master(to_instance, from_instances=[])[source]

For all masters “from” enable all agreements “to” instance.

Parameters:
  • to_instance (lib389.DirSrv) – The instance to start recieving data.
  • from_instances (list[lib389.DirSrv]) – The instances to start sending data.
ensure_agreement(from_instance, to_instance, init=False)[source]

Guarantee that a replication agreement exists ‘from_instance’ send data ‘to_instance’. This can be for any instance, master, hub, or consumer.

Both instances must have been added to the topology with create first master, join_master, join_consumer or join_hub.

Parameters:
  • from_instance (lib389.DirSrv) – An instance already in the topology.
  • to_instance (lib389.DirSrv) – An instance to replicate to.
get_rid(instance)[source]

For a given master, retrieve it’s RID for this suffix.

Parameters:instance (lib389.DirSrv) – The instance
Returns:str
join_consumer(from_instance, to_instance)[source]

Join a new consumer to this instance. This will complete a total init of the data “from instance” to “to instance”.

This can be conducted from any master or hub in the topology as “from” master.

Parameters:
  • from_instance (lib389.DirSrv) – An instance already in the topology.
  • to_instance (lib389.DirSrv) – An instance to join to the topology.
join_hub(from_instance, to_instance)[source]

Join a new hub to this instance. This will complete a total init of the data “from instance” to “to instance”.

This can be conducted from any master or hub in the topology as “from” master.

Not implement yet.

Parameters:
  • from_instance (lib389.DirSrv) – An instance already in the topology.
  • to_instance (lib389.DirSrv) – An instance to join to the topology.
join_master(from_instance, to_instance)[source]

Join a new master in MMR to this instance. This will complete a total init of the data “from instance” to “to instance”.

This can be conducted from any master in the topology as “from” master.

Parameters:
  • from_instance (lib389.DirSrv) – An instance already in the topology.
  • to_instance (lib389.DirSrv) – An instance to join to the topology.
remove_master(instance, remaining_instances=[], purge_sa=True)[source]

Remove an instance from the replication topology.

If purge service accounts is true, remove the instances service account.

The purge_sa must be conducted on a remaining master to guarantee the result.

We recommend remaining instances contains all masters that have an agreement to instance, to ensure no dangling agreements exist. Masters with no agreement are skipped.

Parameters:
  • instance – An instance to remove from the topology.
  • remaining_instances (list[lib389.DirSrv]) – The remaining masters of the topology.
  • purge_sa (bool) – Purge the service account for instance
test_replication(from_instance, to_instance, timeout=20)[source]

Wait for a replication event to occur from instance to instance. This shows some point of synchronisation has occured.

Parameters:
  • from_instance (lib389.DirSrv) – The instance whos state we we want to check from
  • to_instance (lib389.DirSrv) – The instance whos state we want to check matches from.
  • timeout (int) – Fail after timeout seconds.
test_replication_topology(instances, timeout=20)[source]

Confirm replication works between all permutations of masters in the topology.

Parameters:
  • instances (list[lib389.DirSrv]) – The masters.
  • timeout (int) – Fail after timeout seconds.
wait_for_replication(from_instance, to_instance, timeout=20)[source]

Wait for a replication event to occur from instance to instance. This shows some point of synchronisation has occured.

Parameters:
  • from_instance (lib389.DirSrv) – The instance whos state we we want to check from
  • to_instance (lib389.DirSrv) – The instance whos state we want to check matches from.
  • timeout (int) – Fail after timeout seconds.
wait_for_ruv(from_instance, to_instance, timeout=20)[source]

Wait for the in-memory ruv ‘from_instance’ to be advanced past on ‘to_instance’. Note this does not mean the ruvs are “exact matches” only that some set of CSN states has been advanced past. Topics like fractional replication may or may not interfer in this process.

In essence this is a rough check that to_instance is at least at the replication state of from_instance. You should consider using wait_for_replication instead for a guarantee.

Parameters:
  • from_instance (lib389.DirSrv) – The instance whos state we we want to check from
  • to_instance (lib389.DirSrv) – The instance whos state we want to check matches from.
class lib389.replica.RUV(ruvs, logger=None)[source]

Represents the server in memory RUV object. The RUV contains each update vector the server knows of, along with knowledge of CSN state of the replica we have sent data to.

Parameters:
  • ruvs (list[str]) – A list of nsds50ruv values.
  • logger (logging object) – A logging interface.
alloc_rid()[source]

Based on the RUV, determine an available RID for the replication topology that is unique.

Returns:str
is_synced(other_ruv)[source]

Compare two server ruvs to determine if they are synced. This does not mean that replication is in sync (due to things like fractional repl), but in some cases can show that “at least some known point” has been achieved in the replication process.

Parameters:other_ruv (RUV object) – The other ruv object
Returns:bool
class lib389.replica.Replicas(instance)[source]

Replica DSLdapObjects for all replicas

Parameters:instance (lib389.DirSrv) – A instance
create(rdn=None, properties=None)[source]

Create an object under base DN of our entry

Parameters:
  • rdn (str) – RDN of the new entry
  • properties (dict) – Attributes for the new entry
Returns:

DSLdapObject of the created entry

ensure_state(rdn=None, properties=None)[source]

Create an object under base DN of our entry, or assert it exists and update it’s properties.

Parameters:
  • rdn (str) – RDN of the new entry
  • properties (dict) – Attributes for the new entry
Returns:

DSLdapObject of the created entry

get(selector=[], dn=None)[source]

Get a child entry (DSLdapObject, Replica, etc.) with dn or selector using a base DN and objectClasses of our object (DSLdapObjects, Replicas, etc.) After getting the replica, update the replica._suffix parameter.

Parameters:
  • dn (str) – DN of wanted entry
  • selector – An additional filter to objectClasses, i.e. ‘backend_name’
Returns:

A child entry

list()[source]

Get a list of children entries (DSLdapObject, Replica, etc.) using a base DN and objectClasses of our object (DSLdapObjects, Replicas, etc.)

Returns:A list of children entries
class lib389.replica.Replica(instance, dn=None)[source]

Replica DSLdapObject with: - must attributes = [‘cn’, ‘nsDS5ReplicaType’, ‘nsDS5ReplicaRoot’,

‘nsDS5ReplicaBindDN’, ‘nsDS5ReplicaId’]
  • RDN attribute is ‘cn’
  • There is one “replica” per backend
Parameters:
  • instance (lib389.DirSrv) – A instance
  • dn (str) – Entry DN
add(key, value)[source]

Add an attribute with a value

Parameters:
  • key (str) – an attribute name
  • value (str) – an attribute value
apply_mods(mods)[source]

Perform modification operation using several mods at once

Parameters:mods (list of tuples) – [(action, key, value),]
Raises:ValueError - if a provided mod op is invalid
begin_task_cl2ldif()[source]

Begin the changelog to ldif task

classmethod compare(obj1, obj2)[source]

Compare if two RDN objects have same attributes and values.

This comparison is a loose comparison, not a strict one i.e. “this object is this other object” It will just check if the attributes are same. ‘nsUniqueId’ attribute is not checked intentionally because we want to compare arbitrary objects i.e they may have different ‘nsUniqueId’ but same attributes.

Example:

cn=user1,ou=a
cn=user1,ou=b

Comparision of these two objects should result in same, even though their ‘nsUniqueId’ attribute differs.

Parameters:
  • obj1 (lib389._mapped_object.DSLdapObject) – An entry to check
  • obj2 (lib389._mapped_object.DSLdapObject) – An entry to check
Returns:

True if objects have same attributes else returns False

Raises:

ValueError - if obj1 or obj2 don’t inherit DSLdapObject

create(rdn=None, properties=None, basedn=None)[source]

Add a new entry

Parameters:
  • rdn (str) – RDN of the new entry
  • properties (dict) – Attributes for the new entry
  • basedn – Base DN of the new entry
Returns:

DSLdapObject of the created entry

delete()[source]

Delete a replica related to the provided suffix.

If this replica role was ReplicaRole.HUB or ReplicaRole.MASTER, it also deletes the changelog associated to that replica. If it exists some replication agreement below that replica, they are deleted.

Returns:

None

Raises:
  • InvalidArgumentError - if suffix is missing
  • ldap.LDAPError - for all other update failures
demote(newrole)[source]

Demote a replica to a hub or consumer

Parameters:newrole (ReplicaRole) – The new replication role for the replica: CONSUMER and HUB
Returns:None
Raises:ValueError - If replica is not demoted
display()[source]

Get an entry but represent it as a string LDIF

Returns:LDIF formatted string
display_attr(attr)[source]

Get all values of given attribute - ‘attr: value’

Returns:Formatted string
dn

Get an object DN

Returns:DN
ensure_present(attr, value)[source]

Ensure that an attribute and value are present in a state, or add it.

Parameters:
  • key (str) – an attribute name
  • value (str) – an attribute value
ensure_state(rdn=None, properties=None, basedn=None)[source]

Ensure an entry exists with the following state, created if necessary.

Parameters:
  • rdn (str) – RDN of the new entry
  • properties (dict) – Attributes for the new entry
  • basedn – Base DN of the new entry
Returns:

DSLdapObject of the created entry

exists()[source]

Check if the entry exists

Returns:True if it exists
get_agreements()[source]

Return the set of agreements related to this suffix replica

Returns:Agreements object
get_all_attrs(use_json=False)[source]

Get a dictionary having all the attributes of the entry

Returns:Dict with real attributes and operational attributes
get_attr_val_bytes(key, use_json=False)[source]

Get a single attribute value from the entry in bytes type

Parameters:key (str) – An attribute name
Returns:A single bytes value
Raises:ValueError - if instance is offline
get_attr_val_int(key, use_json=False)[source]

Get a single attribute value from the entry in int type

Parameters:key (str) – An attribute name
Returns:A single bytes value
Raises:ValueError - if instance is offline
get_attr_val_utf8(key, use_json=False)[source]

Get a single attribute value from the entry in utf8 type

Parameters:key (str) – An attribute name
Returns:A single bytes value
Raises:ValueError - if instance is offline
get_attr_val_utf8_l(key, use_json=False)[source]

Get a single attribute value from the entry in utf8 type

Parameters:key (str) – An attribute name
Returns:A single bytes value
Raises:ValueError - if instance is offline
get_attr_vals_bytes(key, use_json=False)[source]

Get attribute values from the entry in bytes type

Parameters:key (str) – An attribute name
Returns:A single bytes value
Raises:ValueError - if instance is offline
get_attr_vals_int(key, use_json=False)[source]

Get attribute values from the entry in int type

Parameters:key (str) – An attribute name
Returns:A single bytes value
Raises:ValueError - if instance is offline
get_attr_vals_utf8(key, use_json=False)[source]

Get attribute values from the entry in utf8 type

Parameters:key (str) – An attribute name
Returns:A single bytes value
Raises:ValueError - if instance is offline
get_attr_vals_utf8_l(key, use_json=False)[source]

Get attribute values from the entry in utf8 type and lowercase

Parameters:key (str) – An attribute name
Returns:A single bytes value
Raises:ValueError - if instance is offline
get_compare_attrs(use_json=False)[source]

Get a dictionary having attributes to be compared i.e. excluding self._compare_exclude

get_rid()[source]

Return the current replicas RID for this suffix

Returns:str
get_role()[source]

Return the replica role

Returns:ReplicaRole.MASTER, ReplicaRole.HUB, ReplicaRole.CONSUMER
get_ruv()[source]

Return the in memory ruv of this replica suffix.

Returns:RUV object
lint()[source]

Override this to create a linter for a type. This means that we can detect and report common administrative errors in the server from our cli and rest tools.

The structure of a result is:

{
  dsle: '<identifier>'. dsle == ds lint error. Will be a code unique to
                      this module for the error, IE DSBLE0001.
  severity: '[HIGH:MEDIUM:LOW]'. severity of the error.
  items: '(dn,dn,dn)'. List of affected DNs or names.
  detail: 'msg ...'. An explination of the error.
  fix: 'msg ...'. Steps to resolve the error.
}
Returns:An array of these dicts, on None if there are no errors.
present(attr, value=None)[source]

Assert that some attr, or some attr / value exist on the entry.

Parameters:
  • attr (str) – an attribute name
  • value (str) – an attribute value
Returns:

True if attr is present

promote(newrole, binddn=None, rid=None)[source]

Promote the replica to hub or master

Parameters:
  • newrole (ReplicaRole) – The new replication role for the replica: MASTER and HUB
  • binddn (str) – The replication bind dn - only applied to master
  • rid (int) – The replication ID, applies only to promotions to “master”
Returns:

None

Raises:

ValueError - If replica is not promoted

raw_entry()[source]

Get an Entry object

Returns:Entry object
rdn

Get an object RDN

Returns:RDN
remove(key, value)[source]

Remove a value defined by key

Parameters:
  • key (str) – an attribute name
  • value (str) – an attribute value
remove_all(key)[source]

Remove all values defined by key (if possible).

If an attribute is multi-valued AND required all values except one will be deleted.

Parameters:key (str) – an attribute name
rename(new_rdn, newsuperior=None)[source]

Renames the object within the tree.

If you provide a newsuperior, this will move the object in the tree. If you only provide a new_rdn, it stays in the same branch, but just changes the rdn.

Note, if you use newsuperior, you may move this object outside of the scope of the related DSLdapObjects manager, which may cause it not to appear in .get() requests.

Parameters:
  • new_rdn (str) – RDN of the new entry
  • newsuperior (str) – New parent DN
replace(key, value)[source]

Replace an attribute with a value

Parameters:
  • key (str) – an attribute name
  • value (str) – an attribute value
replace_many(*args)[source]

Replace many key, value pairs in a single operation. This is useful for configuration changes that require atomic operation, and ease of use.

An example of usage is replace_many((key, value), (key, value))

No wrapping list is needed for the arguments.

Parameters:*args

tuples of key,value to replace.

set(key, value, action=2)[source]

Perform a specified action on a key with value

Parameters:
  • key (str) – an attribute name
  • value (str) – an attribute value
  • action (int) –
    • ldap.MOD_REPLACE - by default
    • ldap.MOD_ADD
    • ldap.MOD_DELETE
Returns:

result of modify_s operation

Raises:

ValueError - if instance is not online

test_replication(replica_dirsrvs)[source]

Make a “dummy” update on the the replicated suffix, and check all the provided replicas to see if they received the update.

Parameters:*replica_dirsrvs

DirSrv instance, DirSrv instance, …

Returns:True - if all servers have recevioed the update by this replica, otherwise return False
Raises:LDAPError - when failing to update/search database