summaryrefslogtreecommitdiffstats
path: root/ipalib/plugins/migration.py
Commit message (Collapse)AuthorAgeFilesLines
* Remove support for DN normalization from LDAPClient.Jan Cholasta2013-03-011-3/+2
|
* Use full DNs in plugin code.Jan Cholasta2013-03-011-3/+6
|
* Update argument docs to reflect dropped CSV supportPetr Viktorin2013-02-221-7/+7
| | | | https://fedorahosted.org/freeipa/ticket/3352
* Prevent a crash when no entries are successfully migrated.Rob Crittenden2013-02-081-0/+1
| | | | | | | It would fail in _update_default_group() because migrate_cnt wasn't defined in context. https://fedorahosted.org/freeipa/ticket/3386
* Improve migration performanceRob Crittenden2013-02-051-8/+88
| | | | | | | | | | | | | | | | | | | Add new users to the default users group in batches of 100. The biggest overhead of migration is in calculating the modlist when managing the default user's group and applying the changes. A significant amount of time can be saved by not doing this on every add operation. Some other minor improvements include: Add a negative cache for groups not found in the remote LDAP server. Replace call to user_mod with a direct LDAP update. Catch some occurances of LimitError and handle more gracefully. I also added some debug logging to report on migration status and performance. https://fedorahosted.org/freeipa/ticket/3386
* Fix migration for openldap DSMartin Kosek2013-02-011-1/+13
| | | | | | | | | | | | | | | | | openldap server does not store its schema in cn=schema entry, but rather in cn=subschema. Add a fallback to ldap2 plugin to read from this entry when cn=schema is not found. ldap2 plugin uses the schema when doing some of the automatic encoding, like an automatic encoding of DN object. IPA migration plugin DN attribute processing is now also more tolerant when it finds that some DN attribute was not autoencoded. It tries to convert it to DN on its own and report a warning and continue with user processing when the conversion fails instead of crashing with AssertionError and thus abandoning the whole migration run. https://fedorahosted.org/freeipa/ticket/3372
* Convert uniqueMember members into DN objects.Rob Crittenden2013-01-111-3/+9
| | | | | | | We were asserting that they should be DN objects but weren't converting them anywhere. https://fedorahosted.org/freeipa/ticket/3339
* Use DN objects instead of stringsJohn Dennis2012-08-121-47/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Convert every string specifying a DN into a DN object * Every place a dn was manipulated in some fashion it was replaced by the use of DN operators * Add new DNParam parameter type for parameters which are DN's * DN objects are used 100% of the time throughout the entire data pipeline whenever something is logically a dn. * Many classes now enforce DN usage for their attributes which are dn's. This is implmented via ipautil.dn_attribute_property(). The only permitted types for a class attribute specified to be a DN are either None or a DN object. * Require that every place a dn is used it must be a DN object. This translates into lot of:: assert isinstance(dn, DN) sprinkled through out the code. Maintaining these asserts is valuable to preserve DN type enforcement. The asserts can be disabled in production. The goal of 100% DN usage 100% of the time has been realized, these asserts are meant to preserve that. The asserts also proved valuable in detecting functions which did not obey their function signatures, such as the baseldap pre and post callbacks. * Moved ipalib.dn to ipapython.dn because DN class is shared with all components, not just the server which uses ipalib. * All API's now accept DN's natively, no need to convert to str (or unicode). * Removed ipalib.encoder and encode/decode decorators. Type conversion is now explicitly performed in each IPASimpleLDAPObject method which emulates a ldap.SimpleLDAPObject method. * Entity & Entry classes now utilize DN's * Removed __getattr__ in Entity & Entity clases. There were two problems with it. It presented synthetic Python object attributes based on the current LDAP data it contained. There is no way to validate synthetic attributes using code checkers, you can't search the code to find LDAP attribute accesses (because synthetic attriutes look like Python attributes instead of LDAP data) and error handling is circumscribed. Secondly __getattr__ was hiding Python internal methods which broke class semantics. * Replace use of methods inherited from ldap.SimpleLDAPObject via IPAdmin class with IPAdmin methods. Directly using inherited methods was causing us to bypass IPA logic. Mostly this meant replacing the use of search_s() with getEntry() or getList(). Similarly direct access of the LDAP data in classes using IPAdmin were replaced with calls to getValue() or getValues(). * Objects returned by ldap2.find_entries() are now compatible with either the python-ldap access methodology or the Entity/Entry access methodology. * All ldap operations now funnel through the common IPASimpleLDAPObject giving us a single location where we interface to python-ldap and perform conversions. * The above 4 modifications means we've greatly reduced the proliferation of multiple inconsistent ways to perform LDAP operations. We are well on the way to having a single API in IPA for doing LDAP (a long range goal). * All certificate subject bases are now DN's * DN objects were enhanced thusly: - find, rfind, index, rindex, replace and insert methods were added - AVA, RDN and DN classes were refactored in immutable and mutable variants, the mutable variants are EditableAVA, EditableRDN and EditableDN. By default we use the immutable variants preserving important semantics. To edit a DN cast it to an EditableDN and cast it back to DN when done editing. These issues are fully described in other documentation. - first_key_match was removed - DN equalty comparison permits comparison to a basestring * Fixed ldapupdate to work with DN's. This work included: - Enhance test_updates.py to do more checking after applying update. Add test for update_from_dict(). Convert code to use unittest classes. - Consolidated duplicate code. - Moved code which should have been in the class into the class. - Fix the handling of the 'deleteentry' update action. It's no longer necessary to supply fake attributes to make it work. Detect case where subsequent update applies a change to entry previously marked for deletetion. General clean-up and simplification of the 'deleteentry' logic. - Rewrote a couple of functions to be clearer and more Pythonic. - Added documentation on the data structure being used. - Simplfy the use of update_from_dict() * Removed all usage of get_schema() which was being called prior to accessing the .schema attribute of an object. If a class is using internal lazy loading as an optimization it's not right to require users of the interface to be aware of internal optimization's. schema is now a property and when the schema property is accessed it calls a private internal method to perform the lazy loading. * Added SchemaCache class to cache the schema's from individual servers. This was done because of the observation we talk to different LDAP servers, each of which may have it's own schema. Previously we globally cached the schema from the first server we connected to and returned that schema in all contexts. The cache includes controls to invalidate it thus forcing a schema refresh. * Schema caching is now senstive to the run time context. During install and upgrade the schema can change leading to errors due to out-of-date cached schema. The schema cache is refreshed in these contexts. * We are aware of the LDAP syntax of all LDAP attributes. Every attribute returned from an LDAP operation is passed through a central table look-up based on it's LDAP syntax. The table key is the LDAP syntax it's value is a Python callable that returns a Python object matching the LDAP syntax. There are a handful of LDAP attributes whose syntax is historically incorrect (e.g. DistguishedNames that are defined as DirectoryStrings). The table driven conversion mechanism is augmented with a table of hard coded exceptions. Currently only the following conversions occur via the table: - dn's are converted to DN objects - binary objects are converted to Python str objects (IPA convention). - everything else is converted to unicode using UTF-8 decoding (IPA convention). However, now that the table driven conversion mechanism is in place it would be trivial to do things such as converting attributes which have LDAP integer syntax into a Python integer, etc. * Expected values in the unit tests which are a DN no longer need to use lambda expressions to promote the returned value to a DN for equality comparison. The return value is automatically promoted to a DN. The lambda expressions have been removed making the code much simpler and easier to read. * Add class level logging to a number of classes which did not support logging, less need for use of root_logger. * Remove ipaserver/conn.py, it was unused. * Consolidated duplicate code wherever it was found. * Fixed many places that used string concatenation to form a new string rather than string formatting operators. This is necessary because string formatting converts it's arguments to a string prior to building the result string. You can't concatenate a string and a non-string. * Simplify logic in rename_managed plugin. Use DN operators to edit dn's. * The live version of ipa-ldap-updater did not generate a log file. The offline version did, now both do. https://fedorahosted.org/freeipa/ticket/1670 https://fedorahosted.org/freeipa/ticket/1671 https://fedorahosted.org/freeipa/ticket/1672 https://fedorahosted.org/freeipa/ticket/1673 https://fedorahosted.org/freeipa/ticket/1674 https://fedorahosted.org/freeipa/ticket/1392 https://fedorahosted.org/freeipa/ticket/2872
* Improve migration NotFound errorMartin Kosek2012-06-051-5/+12
| | | | | | | | | | | | | When no user/group was found, migration plugin reported an ambiguous error about invalid container. But the root cause may be for example in a wrong list of user/group objectclasses. Report both in the error message to avoid user confusion. User/group objectclass attribute is now also marked as required. Without the list of objectclasses, an invalid LDAP search is produced. https://fedorahosted.org/freeipa/ticket/2206
* Do not fail migration because of duplicate groupsMartin Kosek2012-04-171-1/+6
| | | | | | | | | | | When 2 groups in a remote LDAP server share the same GID number, the migration may fail entirely with incomprehensible message. This should not be taken as unrecoverable error - GID number check is just a sanity check, a warning is enough. This patch also makes sure that GID check warnings include a user name to make an investigation easier. https://fedorahosted.org/freeipa/ticket/2644
* don't append basedn to container if it is includedJohn Dennis2012-04-161-2/+8
| | | | | | | | | | | ticket #2566 When specifying a container to ds-migrate we should not automatically append the basedn if it is provided by the end-user. This is easy to detect using DN objects because DN objects have a endswith() method which can easily and correctly ascertain if a base already exists.
* Validate DN & RDN parameters for migrate commandJohn Dennis2012-04-161-8/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Ticket #2555 We were generating a traceback (server error) if a malformed RDN was passed as a parameter to the migrate command. * add parameter validation functions validate_dn_param() and validate_rdn_param() to ipalib.util. Those functions simply invoke the DN or RDN constructor from our dn module passing it the string representation. If the constructor does not throw an error it's valid. * Add the parameter validation function pointers to the Param objects in the migrate command. * Make the usercontainer and groupcontainer parameters required. passing --usercontainer= on the command line will produce ipa: ERROR: 'user_container' is required * Fix _get_search_bases() so if a container dn is empty it it just uses the base dn alone instead of faulting (currently bullet-proofing because now the containers are required). * Update the doc for usercontainer and groupcontainer to reflect the fact they are DN's not RDN's. A RDN can only be one level and it should be possible to have a container more than one RDN removed from the base.
* Don't create private groups for migrated users, check for valid gidnumberRob Crittenden2012-04-031-31/+67
| | | | | | | | | | | | | | | | | Migrated users don't get a private group, there is no safe way to verify that the namespace is correct without redoing the uidnumber as well. Verify that the GID at least points to a valid group on the remote server and warn if it doesn't (this doesn't guarantee that the group gets migrated but at least we try). If the remote entry has no gidNumber then don't migrate that user. We don't know why that user is non-POSIX, it could be a special user used for auth, for example. Add a loginshell if the remote user doesn't have one. https://fedorahosted.org/freeipa/ticket/2562
* Normalize the primary key value to lowercase during migration.Rob Crittenden2012-03-221-0/+1
| | | | https://bugzilla.redhat.com/show_bug.cgi?id=804609
* Fix attributes that contain DNs when migrating.Rob Crittenden2012-03-221-1/+43
| | | | | | | Some attributes, like secretary and manager, may point to other LDAP entries. We need to fix these during migration. https://fedorahosted.org/freeipa/ticket/2562
* Fix migration plugin compat checkMartin Kosek2012-03-111-3/+7
| | | | | | | | | | | | Ticket #2274 implements a check for compat plugin and warns user if it is enabled. However, there are 2 issues connected with the plugin: 1) The check is performed against the remote (migrated) LDAP server and not the local LDAP server, which does not make much sense 2) When the compat plugin is missing in cn=plugins,cn=config, it raises an error and thus breaks the migration This patch fixes both issues. https://fedorahosted.org/freeipa/ticket/2508
* Migration warning when compat enabledOndrej Hamada2012-02-291-2/+28
| | | | | | | | | | | Added check into migration plugin to warn user when compat is enabled. If compat is enabled, the migration fails and user is warned that he must turn the compat off or run the script with (the newly introduced) option '--with-compat'. '--with-compat' is new flag. If it is set, the compat status is ignored. https://fedorahosted.org/freeipa/ticket/2274
* Don't set migrated user's GID to that of default users group.Rob Crittenden2012-02-291-3/+8
| | | | | | The GID should be the UID unless UPG is disabled. https://fedorahosted.org/freeipa/ticket/2430
* Add support defaultNamingContext and add --basedn to migrate-dsRob Crittenden2012-02-291-13/+28
| | | | | | | | | | | | | | | | | | | | There are two sides to this, the server and client side. On the server side we attempt to add a defaultNamingContext on already installed servers. This will fail on older 389-ds instances but the failure is not fatal. New installations on versions of 389-ds that support this attribute will have it already defined. On the client side we need to look for both defaultNamingContext and namingContexts. We still need to check that the defaultNamingContext is an IPA server (info=IPAV2). The migration change also takes advantage of this and adds a new option which allows one to provide a basedn to use instead of trying to detect it. https://fedorahosted.org/freeipa/ticket/1919 https://fedorahosted.org/freeipa/ticket/2314
* Improve migration helpMartin Kosek2012-02-031-8/+21
| | | | | | | | | Improve migration help topic so that it easier understandable: - Add missing list of Topic commands - Add one more example to demonstrate migration abilities - Add breaks to too long lines to improve readibility https://fedorahosted.org/freeipa/ticket/2174
* Parse comma-separated lists of values in all parameter types. This can be ↵Jan Cholasta2011-11-301-11/+17
| | | | | | | | | | | | | enabled for a specific parameter by setting the "csv" option to True. Remove "List" parameter type and replace all occurences of it with appropriate multi-valued parameter ("Str" in most cases) with csv enabled. Add new parameter type "Any", capable of holding values of any type. This is needed by the "batch" command, as "Str" is not suitable type for the "methods" parameter. ticket 2007
* ticket 2022 - modify codebase to utilize IPALogManager, obsoletes loggingJohn Dennis2011-11-231-1/+0
| | | | | | | | | | | | change default_logger_level to debug in configure_standard_logging add new ipa_log_manager module, move log_mgr there, also export root_logger from log_mgr. change all log_manager imports to ipa_log_manager and change log_manager.root_logger to root_logger. add missing import for parse_log_level()
* Improve handling of GIDs when migrating groupsMartin Kosek2011-10-111-11/+62
| | | | | | | | | | | | Since IPA v2 server already contain predefined groups that may collide with groups in migrated (IPA v1) server (for example admins, ipausers), users having colliding group as their primary group may happen to belong to an unknown group on new IPA v2 server. Implement --group-overwrite-gid option to overwrite GID of already existing groups to prevent this issue. https://fedorahosted.org/freeipa/ticket/1866
* migrate process cannot handle multivalued pkey attributeMartin Kosek2011-10-031-1/+17
| | | | | | | | When group/user is migrated, the attribute used for RDN may be multivalued. Make sure that we pick the value used in the RDN which should be the unique one and not just the first one. https://fedorahosted.org/freeipa/ticket/1892
* ticket 1669 - improve i18n docstring extractionJohn Dennis2011-08-241-19/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch reverts the use of pygettext for i18n string extraction. It was originally introduced because the help documentation for commands are in the class docstring and module docstring. Docstrings are a Python construct whereby any string which immediately follows a class declaration, function/method declaration or appears first in a module is taken to be the documentation for that object. Python automatically assigns that string to the __doc__ variable associated with the object. Explicitly assigning to the __doc__ variable is equivalent and permitted. We mark strings in the source for i18n translation by embedding them in _() or ngettext(). Specialized extraction tools (e.g. xgettext) scan the source code looking for strings with those markers and extracts the string for inclusion in a translation catalog. It was mistakingly assumed one could not mark for translation Python docstrings. Since some docstrings are vital for our command help system some method had to be devised to extract docstrings for the translation catalog. pygettext has the ability to locate and extract docstrings and it was introduced to acquire the documentation for our commands located in module and class docstrings. However pygettext was too large a hammer for this task, it lacked any fined grained ability to extract only the docstrings we were interested in. In practice it extracted EVERY docstring in each file it was presented with. This caused a large number strings to be extracted for translation which had no reason to be translated, the string might have been internal code documentation never meant to be seen by users. Often the superfluous docstrings were long, complex and likely difficult to translate. This placed an unnecessary burden on our volunteer translators. Instead what is needed is some method to extract only those strings intended for translation. We already have such a mechanism and it is already widely used, namely wrapping strings intended for translation in calls to _() or _negettext(), i.e. marking a string for i18n translation. Thus the solution to the docstring translation problem is to mark the docstrings exactly as we have been doing, it only requires that instead of a bare Python docstring we instead assign the marked string to the __doc__ variable. Using the hypothetical class foo as an example. class foo(Command): ''' The foo command takes out the garbage. ''' Would become: class foo(Command): __doc__ = _('The foo command takes out the garbage.') But which docstrings need to be marked for translation? The makeapi tool knows how to iterate over every command in our public API. It was extended to validate every command's documentation and report if any documentation is missing or not marked for translation. That information was then used to identify each docstring in the code which needed to be transformed. In summary what this patch does is: * Remove the use of pygettext (modification to install/po/Makefile.in) * Replace every docstring with an explicit assignment to __doc__ where the rhs of the assignment is an i18n marking function. * Single line docstrings appearing in multi-line string literals (e.g. ''' or """) were replaced with single line string literals because the multi-line literals were introducing unnecessary whitespace and newlines in the string extracted for translation. For example: ''' The foo command takes out the garbage. ''' Would appear in the translation catalog as: "\n The foo command takes out the garbage.\n " The superfluous whitespace and newlines are confusing to translators and requires us to strip leading and trailing whitespace from the translation at run time. * Import statements were moved from below the docstring to above it. This was necessary because the i18n markers are imported functions and must be available before the the doc is parsed. Technically only the import of the i18n markers had to appear before the doc but stylistically it's better to keep all the imports together. * It was observed during the docstring editing process that the command documentation was inconsistent with respect to the use of periods to terminate a sentence. Some doc had a trailing period, others didn't. Consistency was enforced by adding a period to end of every docstring if one was missing.
* Fixed object_name and object_name_plural internationalizationEndi S. Dewata2011-07-121-1/+1
| | | | | | | | | The object_name, object_name_plural and messages that use these attributes have been converted to support translation. The label attribute in the Param class has been modified to accept unicode string. Ticket #1435
* Add ignore lists to migrate-ds commandMartin Kosek2011-06-151-5/+61
| | | | | | | | | | | | | | When user migrates users/groups from an old DS instance, the migration may fail on unsupported object classes and/or relevant LDAP object attributes. This patch implements a support for object class and attribute ignore lists that can be used to suppress these migration issues. Additionally, a redundant "dev/null" file is removed from git repo (originally added in 26b0e8fc9809a4cd9f2f9a2281f0894e2e0f8db2). https://fedorahosted.org/freeipa/ticket/1266
* Handle LDAP search referencesMartin Kosek2011-06-101-3/+9
| | | | | | | | | | | | | LDAP search operation may return a search reference pointing to an LDAP resource. As the framework does not handle search references, skip these results to prevent result processing failures. Migrate operation crashed when the migrated DS contained search references. Now, it correctly skips these records and prints the failed references to user. https://fedorahosted.org/freeipa/ticket/1209
* Fix migration to work between v2 servers and remove search/size limits.Rob Crittenden2011-05-261-6/+14
| | | | | | | | | | | | Migration from a v2 server would fail because of our fake memberofindirect attribute. This isn't in any objectclass so would cause entries to fail to migrate. We can safely just remove it. Also remove any limits on time/size when searching for entries on the remote server. Otherwise only the number of entries configured in the local IPA server can be migrated. ticket 1124
* Fix style and grammatical issues in built-in command help.Rob Crittenden2011-03-041-2/+3
| | | | | | | There is a rather large API.txt change but it is only due to changes in the doc string in parameters. ticket 729
* Fix translatable strings in ipalib plugins.Pavel Zuna2011-03-011-1/+3
| | | | Needed for xgettext/pygettext processing.
* Typos in freeIPA messagesMartin Kosek2011-02-021-2/+2
| | | | | | | | | | This patch fixes several reported typos in IPA messages and in comments. Contributors file has been updated + the original author of the patch reporting the typos was added. https://fedorahosted.org/freeipa/ticket/848
* Change FreeIPA license to GPLv3+Jakub Hrozek2010-12-201-5/+5
| | | | | | | | | | The changes include: * Change license blobs in source files to mention GPLv3+ not GPLv2 only * Add GPLv3+ license text * Package COPYING not LICENSE as the license blobs (even the old ones) mention COPYING specifically, it is also more common, I think https://fedorahosted.org/freeipa/ticket/239
* Fix typo in migration documentationRob Crittenden2010-12-131-1/+1
|
* Add documentation to the migrate-ds command.Rob Crittenden2010-12-091-2/+34
| | | | ticket 539
* Do not migrate krbPrincipalKeyJakub Hrozek2010-12-071-0/+7
| | | | https://fedorahosted.org/freeipa/ticket/455
* Make the migration plugin more configurableJakub Hrozek2010-12-071-28/+108
| | | | | | | | | | | | This patch adds new options to the migration plugin: * the option to fine-tune the objectclass of users or groups being imported * the option to select the LDAP schema (RFC2307 or RFC2307bis) Also makes the logic that decides whether an entry is a nested group or user (for RFC2307bis) smarter by looking at the DNS. Does not hardcode primary keys for migrated entries. https://fedorahosted.org/freeipa/ticket/429
* Add labels for passwords, fix output of exceptions, fix passwd output.Rob Crittenden2010-12-021-1/+2
| | | | | | | | | | | Passwords didn't have internationalizable labels. Exceptions that occured during required input weren't printed as unicode so weren't being translated properly. Don't use output_for_cli() directly in the passwd plugin, use output.Output. ticket 352
* UUIDs: remove uuid python plugin and let DS always autogenerateSimo Sorce2010-10-281-3/+3
| | | | merge in remove uuid
* Handle an empty base_dn and no cn=ipaconfig in the ldap2 backend, fix migration.Rob Crittenden2010-09-281-6/+17
| | | | | | | | | | | | | We lacked good error messages if the user/group container you used doesn't exist. Add a --continue option so things can continue if you use a bad user/group container. This has the side-effect of letting you migrate just users or groups by using a bad container for the one you don't want. Fix a Gettext() error when displaying the migrated password message. ticket 289
* Handle errors raised by plugins more gracefully in mod_wsgi.Rob Crittenden2010-07-121-1/+4
| | | | | | | | | | | | This started as an effort to display a more useful error message in the Apache error log if retrieving the schema failed. I broadened the scope a little to include limiting the output in the Apache error log so errors are easier to find. This adds a new configuration option, startup_traceback. Outside of lite-server.py it is False by default so does not display the traceback that lead to the StandardError being raised. This makes the mod_wsgi error much easier to follow.
* Retrieve the LDAP schema using kerberos credentials.Rob Crittenden2010-03-171-1/+2
| | | | This is required so we can disable anonymous access in 389-ds.
* localize doc stringsJohn Dennis2010-03-081-21/+25
| | | | | | | | | | | | A number of doc strings were not localized, wrap them in _(). Some messages were not localized, wrap them in _() Fix a couple of failing tests: The method name in RPC should not be unicode. The doc attribute must use the .msg attribute for comparison. Also clean up imports of _() The import should come from ipalib or ipalib.text, not ugettext from request.
* Translatable Param.label, Param.docJason Gerard DeRose2010-02-241-7/+10
|
* Make error message in migration plugin unicode.Pavel Zuna2010-02-171-2/+2
|
* Expand the types of groups that can be migrated to support IPA v1 migrationsRob Crittenden2010-02-171-1/+1
|
* Add DS migration plugin and password migration page.Pavel Zuna2010-01-201-0/+374
generated by cgit (git 2.34.1) at 2025-09-09 20:43:32 +0000
flt = '(%s' % rules else: flt = '' for f in filters: if not f.startswith('('): f = '(%s)' % f flt = '%s%s' % (flt, f) if len(filters) > 1: flt = '%s)' % flt return flt def make_filter_from_attr(self, attr, value, rules='|', exact=True, leading_wildcard=True, trailing_wildcard=True): """ Make filter for ldap2.find_entries from attribute. Keyword arguments: rules -- see ldap2.make_filter exact -- boolean, True - make filter as (attr=value) False - make filter as (attr=*value*) leading_wildcard -- boolean, True - allow heading filter wildcard when exact=False False - forbid heading filter wildcard when exact=False trailing_wildcard -- boolean, True - allow trailing filter wildcard when exact=False False - forbid trailing filter wildcard when exact=False """ if isinstance(value, (list, tuple)): make_filter_rules = self.MATCH_ANY if rules == self.MATCH_NONE else rules flts = [ self.make_filter_from_attr(attr, v, exact=exact, leading_wildcard=leading_wildcard, trailing_wildcard=trailing_wildcard) for v in value ] return self.combine_filters(flts, rules) elif value is not None: value = _ldap_filter.escape_filter_chars(value_to_utf8(value)) if not exact: template = '%s' if leading_wildcard: template = '*' + template if trailing_wildcard: template = template + '*' value = template % value if rules == self.MATCH_NONE: return '(!(%s=%s))' % (attr, value) return '(%s=%s)' % (attr, value) return '' def make_filter(self, entry_attrs, attrs_list=None, rules='|', exact=True, leading_wildcard=True, trailing_wildcard=True): """ Make filter for ldap2.find_entries from entry attributes. Keyword arguments: attrs_list -- list of attributes to use, all if None (default None) rules -- specifies how to determine a match (default ldap2.MATCH_ANY) exact -- boolean, True - make filter as (attr=value) False - make filter as (attr=*value*) leading_wildcard -- boolean, True - allow heading filter wildcard when exact=False False - forbid heading filter wildcard when exact=False trailing_wildcard -- boolean, True - allow trailing filter wildcard when exact=False False - forbid trailing filter wildcard when exact=False rules can be one of the following: ldap2.MATCH_NONE - match entries that do not match any attribute ldap2.MATCH_ALL - match entries that match all attributes ldap2.MATCH_ANY - match entries that match any of attribute """ make_filter_rules = self.MATCH_ANY if rules == self.MATCH_NONE else rules flts = [] if attrs_list is None: for (k, v) in entry_attrs.iteritems(): flts.append( self.make_filter_from_attr(k, v, make_filter_rules, exact, leading_wildcard, trailing_wildcard) ) else: for a in attrs_list: value = entry_attrs.get(a, None) if value is not None: flts.append( self.make_filter_from_attr(a, value, make_filter_rules, exact, leading_wildcard, trailing_wildcard) ) return self.combine_filters(flts, rules) def find_entries(self, filter=None, attrs_list=None, base_dn=None, scope=_ldap.SCOPE_SUBTREE, time_limit=None, size_limit=None, normalize=True, search_refs=False): """ Return a list of entries and indication of whether the results were truncated ([(dn, entry_attrs)], truncated) matching specified search parameters followed by truncated flag. If the truncated flag is True, search hit a server limit and its results are incomplete. Keyword arguments: attrs_list -- list of attributes to return, all if None (default None) base_dn -- dn of the entry at which to start the search (default '') scope -- search scope, see LDAP docs (default ldap2.SCOPE_SUBTREE) time_limit -- time limit in seconds (default use IPA config values) size_limit -- size (number of entries returned) limit (default use IPA config values) normalize -- normalize the DN (default True) search_refs -- allow search references to be returned (default skips these entries) """ if base_dn is None: base_dn = DN() assert isinstance(base_dn, DN) if normalize: base_dn = self.normalize_dn(base_dn) if not filter: filter = '(objectClass=*)' res = [] truncated = False if time_limit is None or size_limit is None: (cdn, config) = self.get_ipa_config() if time_limit is None: time_limit = config.get('ipasearchtimelimit', [-1])[0] if size_limit is None: size_limit = config.get('ipasearchrecordslimit', [0])[0] if time_limit == 0: time_limit = -1 if not isinstance(size_limit, int): size_limit = int(size_limit) if not isinstance(time_limit, float): time_limit = float(time_limit) if attrs_list: attrs_list = list(set(attrs_list)) # pass arguments to python-ldap try: id = self.conn.search_ext( base_dn, scope, filter, attrs_list, timeout=time_limit, sizelimit=size_limit ) while True: (objtype, res_list) = self.conn.result(id, 0) if not res_list: break if objtype == _ldap.RES_SEARCH_ENTRY or \ (search_refs and objtype == _ldap.RES_SEARCH_REFERENCE): res.append(res_list[0]) except (_ldap.ADMINLIMIT_EXCEEDED, _ldap.TIMELIMIT_EXCEEDED, _ldap.SIZELIMIT_EXCEEDED), e: truncated = True except _ldap.LDAPError, e: self.handle_errors(e) if not res and not truncated: raise errors.NotFound(reason='no such entry') if attrs_list and ('memberindirect' in attrs_list or '*' in attrs_list): for r in res: if not 'member' in r[1]: continue else: members = r[1]['member'] indirect = self.get_members(r[0], members, membertype=MEMBERS_INDIRECT, time_limit=time_limit, size_limit=size_limit, normalize=normalize) if len(indirect) > 0: r[1]['memberindirect'] = indirect if attrs_list and ('memberofindirect' in attrs_list or '*' in attrs_list): for r in res: if 'memberof' in r[1]: memberof = r[1]['memberof'] del r[1]['memberof'] elif 'memberOf' in r[1]: memberof = r[1]['memberOf'] del r[1]['memberOf'] else: continue (direct, indirect) = self.get_memberof(r[0], memberof, time_limit=time_limit, size_limit=size_limit, normalize=normalize) if len(direct) > 0: r[1]['memberof'] = direct if len(indirect) > 0: r[1]['memberofindirect'] = indirect return (res, truncated) def find_entry_by_attr(self, attr, value, object_class, attrs_list=None, base_dn=None): """ Find entry (dn, entry_attrs) by attribute and object class. Keyword arguments: attrs_list - list of attributes to return, all if None (default None) base_dn - dn of the entry at which to start the search (default '') """ if base_dn is None: base_dn = DN() assert isinstance(base_dn, DN) search_kw = {attr: value, 'objectClass': object_class} filter = self.make_filter(search_kw, rules=self.MATCH_ALL) (entries, truncated) = self.find_entries(filter, attrs_list, base_dn) if len(entries) > 1: raise errors.SingleMatchExpected(found=len(entries)) else: if truncated: raise errors.LimitsExceeded() else: return entries[0] def get_entry(self, dn, attrs_list=None, time_limit=None, size_limit=None, normalize=True): """ Get entry (dn, entry_attrs) by dn. Keyword arguments: attrs_list - list of attributes to return, all if None (default None) """ assert isinstance(dn, DN) (entry, truncated) = self.find_entries( None, attrs_list, dn, self.SCOPE_BASE, time_limit=time_limit, size_limit=size_limit, normalize=normalize ) if truncated: raise errors.LimitsExceeded() return entry[0] config_defaults = {'ipasearchtimelimit': [2], 'ipasearchrecordslimit': [0]} def get_ipa_config(self, attrs_list=None): """Returns the IPA configuration entry (dn, entry_attrs).""" odn = api.Object.config.get_dn() assert isinstance(odn, DN) assert isinstance(api.env.basedn, DN) cdn = DN(odn, api.env.basedn) try: config_entry = getattr(context, 'config_entry') return (cdn, copy.deepcopy(config_entry)) except AttributeError: # Not in our context yet pass try: (entry, truncated) = self.find_entries( None, attrs_list, base_dn=cdn, scope=self.SCOPE_BASE, time_limit=2, size_limit=10 ) if truncated: raise errors.LimitsExceeded() (cdn, config_entry) = entry[0] except errors.NotFound: config_entry = {} for a in self.config_defaults: if a not in config_entry: config_entry[a] = self.config_defaults[a] setattr(context, 'config_entry', copy.deepcopy(config_entry)) return (cdn, config_entry) def has_upg(self): """Returns True/False whether User-Private Groups are enabled. This is determined based on whether the UPG Template exists. """ upg_dn = DN(('cn', 'UPG Definition'), ('cn', 'Definitions'), ('cn', 'Managed Entries'), ('cn', 'etc'), api.env.basedn) try: upg_entry = self.conn.search_s(upg_dn, _ldap.SCOPE_BASE, attrlist=['*'])[0] disable_attr = '(objectclass=disable)' if 'originfilter' in upg_entry[1]: org_filter = upg_entry[1]['originfilter'] return not bool(re.search(r'%s' % disable_attr, org_filter[0])) else: return False except _ldap.NO_SUCH_OBJECT, e: return False def get_effective_rights(self, dn, entry_attrs): """Returns the rights the currently bound user has for the given DN. Returns 2 attributes, the attributeLevelRights for the given list of attributes and the entryLevelRights for the entry itself. """ assert isinstance(dn, DN) principal = getattr(context, 'principal') (binddn, attrs) = self.find_entry_by_attr("krbprincipalname", principal, "krbPrincipalAux") assert isinstance(binddn, DN) sctrl = [GetEffectiveRightsControl(True, "dn: " + str(binddn))] self.conn.set_option(_ldap.OPT_SERVER_CONTROLS, sctrl) (dn, attrs) = self.get_entry(dn, entry_attrs) # remove the control so subsequent operations don't include GER self.conn.set_option(_ldap.OPT_SERVER_CONTROLS, []) return (dn, attrs) def can_write(self, dn, attr): """Returns True/False if the currently bound user has write permissions on the attribute. This only operates on a single attribute at a time. """ assert isinstance(dn, DN) (dn, attrs) = self.get_effective_rights(dn, [attr]) if 'attributelevelrights' in attrs: attr_rights = attrs.get('attributelevelrights')[0].decode('UTF-8') (attr, rights) = attr_rights.split(':') if 'w' in rights: return True return False def can_read(self, dn, attr): """Returns True/False if the currently bound user has read permissions on the attribute. This only operates on a single attribute at a time. """ assert isinstance(dn, DN) (dn, attrs) = self.get_effective_rights(dn, [attr]) if 'attributelevelrights' in attrs: attr_rights = attrs.get('attributelevelrights')[0].decode('UTF-8') (attr, rights) = attr_rights.split(':') if 'r' in rights: return True return False # # Entry-level effective rights # # a - Add # d - Delete # n - Rename the DN # v - View the entry # def can_delete(self, dn): """Returns True/False if the currently bound user has delete permissions on the entry. """ assert isinstance(dn, DN) (dn, attrs) = self.get_effective_rights(dn, ["*"]) if 'entrylevelrights' in attrs: entry_rights = attrs['entrylevelrights'][0].decode('UTF-8') if 'd' in entry_rights: return True return False def can_add(self, dn): """Returns True/False if the currently bound user has add permissions on the entry. """ assert isinstance(dn, DN) (dn, attrs) = self.get_effective_rights(dn, ["*"]) if 'entrylevelrights' in attrs: entry_rights = attrs['entrylevelrights'][0].decode('UTF-8') if 'a' in entry_rights: return True return False def update_entry_rdn(self, dn, new_rdn, del_old=True): """ Update entry's relative distinguished name. Keyword arguments: del_old -- delete old RDN value (default True) """ assert isinstance(dn, DN) assert isinstance(new_rdn, RDN) dn = self.normalize_dn(dn) if dn[0] == new_rdn: raise errors.EmptyModlist() try: self.conn.rename_s(dn, new_rdn, delold=int(del_old)) time.sleep(.3) # Give memberOf plugin a chance to work except _ldap.LDAPError, e: self.handle_errors(e) def _generate_modlist(self, dn, entry_attrs, normalize): assert isinstance(dn, DN) # get original entry (dn, entry_attrs_old) = self.get_entry(dn, entry_attrs.keys(), normalize=normalize) # generate modlist # for multi value attributes: no MOD_REPLACE to handle simultaneous # updates better # for single value attribute: always MOD_REPLACE modlist = [] for (k, v) in entry_attrs.iteritems(): if v is None and k in entry_attrs_old: modlist.append((_ldap.MOD_DELETE, k, None)) else: if not isinstance(v, (list, tuple)): v = [v] v = set(filter(lambda value: value is not None, v)) old_v = set(entry_attrs_old.get(k.lower(), [])) # FIXME: Convert all values to either unicode, DN or str # before detecting value changes (see IPASimpleLDAPObject for # supported types). # This conversion will set a common ground for the comparison. # # This fix can be removed when ticket 2265 is fixed and our # encoded entry_attrs' types will match get_entry result try: v = set(unicode_from_utf8(self.conn.encode(value)) if not isinstance(value, (DN, str, unicode)) else value for value in v) except Exception, e: # Rather let the value slip in modlist than let ldap2 crash self.log.error( "Cannot convert attribute '%s' for modlist " "for modlist comparison: %s", k, e) adds = list(v.difference(old_v)) rems = list(old_v.difference(v)) is_single_value = self.get_single_value(k) value_count = len(old_v) + len(adds) - len(rems) if is_single_value and value_count > 1: raise errors.OnlyOneValueAllowed(attr=k) force_replace = False if k in self._FORCE_REPLACE_ON_UPDATE_ATTRS or is_single_value: force_replace = True elif len(v) > 0 and len(v.intersection(old_v)) == 0: force_replace = True if adds: if force_replace: modlist.append((_ldap.MOD_REPLACE, k, adds)) else: modlist.append((_ldap.MOD_ADD, k, adds)) if rems: if not force_replace: modlist.append((_ldap.MOD_DELETE, k, rems)) return modlist def update_entry(self, dn, entry_attrs, normalize=True): """ Update entry's attributes. An attribute value set to None deletes all current values. """ assert isinstance(dn, DN) if normalize: dn = self.normalize_dn(dn) # generate modlist modlist = self._generate_modlist(dn, entry_attrs, normalize) if not modlist: raise errors.EmptyModlist() # pass arguments to python-ldap try: self.conn.modify_s(dn, modlist) except _ldap.LDAPError, e: self.handle_errors(e) def delete_entry(self, dn, normalize=True): """Delete entry.""" assert isinstance(dn, DN) if normalize: dn = self.normalize_dn(dn) try: self.conn.delete_s(dn) except _ldap.LDAPError, e: self.handle_errors(e) def modify_password(self, dn, new_pass, old_pass=''): """Set user password.""" assert isinstance(dn, DN) dn = self.normalize_dn(dn) # The python-ldap passwd command doesn't verify the old password # so we'll do a simple bind to validate it. if old_pass != '': try: conn = IPASimpleLDAPObject( self.ldap_uri, force_schema_updates=False) conn.simple_bind_s(dn, old_pass) conn.unbind() except _ldap.LDAPError, e: self.handle_errors(e) try: self.conn.passwd_s(dn, old_pass, new_pass) except _ldap.LDAPError, e: self.handle_errors(e) def add_entry_to_group(self, dn, group_dn, member_attr='member', allow_same=False): """ Add entry designaed by dn to group group_dn in the member attribute member_attr. Adding a group as a member of itself is not allowed unless allow_same is True. """ assert isinstance(dn, DN) assert isinstance(group_dn, DN) self.log.debug( "add_entry_to_group: dn=%s group_dn=%s member_attr=%s", dn, group_dn, member_attr) # check if the entry exists (dn, entry_attrs) = self.get_entry(dn, ['objectclass']) # get group entry (group_dn, group_entry_attrs) = self.get_entry(group_dn, [member_attr]) self.log.debug( "add_entry_to_group: group_entry_attrs=%s", group_entry_attrs) # check if we're not trying to add group into itself if dn == group_dn and not allow_same: raise errors.SameGroupError() # add dn to group entry's `member_attr` attribute members = group_entry_attrs.get(member_attr, []) members.append(dn) group_entry_attrs[member_attr] = members # update group entry try: self.update_entry(group_dn, group_entry_attrs) except errors.EmptyModlist: raise errors.AlreadyGroupMember() def remove_entry_from_group(self, dn, group_dn, member_attr='member'): """Remove entry from group.""" assert isinstance(dn, DN) assert isinstance(group_dn, DN) self.log.debug( "remove_entry_from_group: dn=%s group_dn=%s member_attr=%s", dn, group_dn, member_attr) # get group entry (group_dn, group_entry_attrs) = self.get_entry(group_dn, [member_attr]) self.log.debug( "remove_entry_from_group: group_entry_attrs=%s", group_entry_attrs) # remove dn from group entry's `member_attr` attribute members = group_entry_attrs.get(member_attr, []) assert all([isinstance(x, DN) for x in members]) try: members.remove(dn) except ValueError: raise errors.NotGroupMember() group_entry_attrs[member_attr] = members # update group entry self.update_entry(group_dn, group_entry_attrs) def get_members(self, group_dn, members, attr_list=[], membertype=MEMBERS_ALL, time_limit=None, size_limit=None, normalize=True): """Do a memberOf search of groupdn and return the attributes in attr_list (an empty list returns all attributes). membertype = MEMBERS_ALL all members returned membertype = MEMBERS_DIRECT only direct members are returned membertype = MEMBERS_INDIRECT only inherited members are returned Members may be included in a group as a result of being a member of a group that is a member of the group being queried. Returns a list of DNs. """ assert isinstance(group_dn, DN) if membertype not in [MEMBERS_ALL, MEMBERS_DIRECT, MEMBERS_INDIRECT]: return None self.log.debug( "get_members: group_dn=%s members=%s membertype=%s", group_dn, members, membertype) search_group_dn = _ldap_filter.escape_filter_chars(str(group_dn)) searchfilter = "(memberof=%s)" % search_group_dn attr_list.append("member") # Verify group membership results = [] if membertype == MEMBERS_ALL or membertype == MEMBERS_INDIRECT: user_container_dn = DN(api.env.container_user, api.env.basedn) # FIXME, initialize once host_container_dn = DN(api.env.container_host, api.env.basedn) checkmembers = set(DN(x) for x in members) checked = set() while checkmembers: member_dn = checkmembers.pop() checked.add(member_dn) # No need to check entry types that are not nested for # additional members if member_dn.endswith(user_container_dn) or \ member_dn.endswith(host_container_dn): results.append([member_dn, {}]) continue try: (result, truncated) = self.find_entries(searchfilter, attr_list, member_dn, time_limit=time_limit, size_limit=size_limit, scope=_ldap.SCOPE_BASE, normalize=normalize) if truncated: raise errors.LimitsExceeded() results.append(list(result[0])) for m in result[0][1].get('member', []): # This member may contain other members, add it to our # candidate list if m not in checked: checkmembers.add(m) except errors.NotFound: pass if membertype == MEMBERS_ALL: entries = [] for e in results: entries.append(e[0]) return entries (dn, group) = self.get_entry(group_dn, ['dn', 'member'], size_limit=size_limit, time_limit=time_limit) real_members = group.get('member', []) entries = [] for e in results: if e[0] not in real_members and e[0] not in entries: if membertype == MEMBERS_INDIRECT: entries.append(e[0]) else: if membertype == MEMBERS_DIRECT: entries.append(e[0]) self.log.debug("get_members: result=%s", entries) return entries def get_memberof(self, entry_dn, memberof, time_limit=None, size_limit=None, normalize=True): """ Examine the objects that an entry is a member of and determine if they are a direct or indirect member of that group. entry_dn: dn of the entry we want the direct/indirect members of memberof: the memberOf attribute for entry_dn Returns two memberof lists: (direct, indirect) """ assert isinstance(entry_dn, DN) self.log.debug( "get_memberof: entry_dn=%s memberof=%s", entry_dn, memberof) if not type(memberof) in (list, tuple): return ([], []) if len(memberof) == 0: return ([], []) search_entry_dn = _ldap_filter.escape_filter_chars(str(entry_dn)) attr_list = ["dn", "memberof"] searchfilter = "(|(member=%s)(memberhost=%s)(memberuser=%s))" % ( search_entry_dn, search_entry_dn, search_entry_dn) # Search only the groups for which the object is a member to # determine if it is directly or indirectly associated. results = [] for group in memberof: assert isinstance(group, DN) try: (result, truncated) = self.find_entries(searchfilter, attr_list, group, time_limit=time_limit,size_limit=size_limit, scope=_ldap.SCOPE_BASE, normalize=normalize) results.extend(list(result)) except errors.NotFound: pass direct = [] # If there is an exception here, it is likely due to a failure in # referential integrity. All members should have corresponding # memberOf entries. indirect = list(memberof) for r in results: direct.append(r[0]) try: indirect.remove(r[0]) except ValueError, e: self.log.info( 'Failed to remove indirect entry %s from %s', r[0], entry_dn) raise e self.log.debug( "get_memberof: result direct=%s indirect=%s", direct, indirect) return (direct, indirect) def set_entry_active(self, dn, active): """Mark entry active/inactive.""" assert isinstance(dn, DN) assert isinstance(active, bool) # get the entry in question (dn, entry_attrs) = self.get_entry(dn, ['nsaccountlock']) # check nsAccountLock attribute account_lock_attr = entry_attrs.get('nsaccountlock', ['false']) account_lock_attr = account_lock_attr[0].lower() if active: if account_lock_attr == 'false': raise errors.AlreadyActive() else: if account_lock_attr == 'true': raise errors.AlreadyInactive() # LDAP expects string instead of Bool but it also requires it to be TRUE or FALSE, # not True or False as Python stringification does. Thus, we uppercase it. account_lock_attr = str(not active).upper() entry_attrs['nsaccountlock'] = account_lock_attr self.update_entry(dn, entry_attrs) def activate_entry(self, dn): """Mark entry active.""" assert isinstance(dn, DN) self.set_entry_active(dn, True) def deactivate_entry(self, dn): """Mark entry inactive.""" assert isinstance(dn, DN) self.set_entry_active(dn, False) def remove_principal_key(self, dn): """Remove a kerberos principal key.""" assert isinstance(dn, DN) dn = self.normalize_dn(dn) # We need to do this directly using the LDAP library because we # don't have read access to krbprincipalkey so we need to delete # it in the blind. mod = [(_ldap.MOD_REPLACE, 'krbprincipalkey', None), (_ldap.MOD_REPLACE, 'krblastpwdchange', None)] try: self.conn.modify_s(dn, mod) except _ldap.LDAPError, e: self.handle_errors(e) # CrudBackend methods def _get_normalized_entry_for_crud(self, dn, attrs_list=None): assert isinstance(dn, DN) (dn, entry_attrs) = self.get_entry(dn, attrs_list) entry_attrs['dn'] = dn return entry_attrs def create(self, **kw): """ Create a new entry and return it as one dict (DN included). Extends CrudBackend.create. """ assert 'dn' in kw dn = kw['dn'] assert isinstance(dn, DN) del kw['dn'] self.add_entry(dn, kw) return self._get_normalized_entry_for_crud(dn) def retrieve(self, primary_key, attributes): """ Get entry by primary_key (DN) as one dict (DN included). Extends CrudBackend.retrieve. """ return self._get_normalized_entry_for_crud(primary_key, attributes) def update(self, primary_key, **kw): """ Update entry's attributes and return it as one dict (DN included). Extends CrudBackend.update. """ self.update_entry(primary_key, kw) return self._get_normalized_entry_for_crud(primary_key) def delete(self, primary_key): """ Delete entry by primary_key (DN). Extends CrudBackend.delete. """ self.delete_entry(primary_key) def search(self, **kw): """ Return a list of entries (each entry is one dict, DN included) matching the specified criteria. Keyword arguments: filter -- search filter (default: '') attrs_list -- list of attributes to return, all if None (default None) base_dn -- dn of the entry at which to start the search (default '') scope -- search scope, see LDAP docs (default ldap2.SCOPE_SUBTREE) Extends CrudBackend.search. """ # get keyword arguments filter = kw.pop('filter', None) attrs_list = kw.pop('attrs_list', None) base_dn = kw.pop('base_dn', DN()) assert isinstance(base_dn, DN) scope = kw.pop('scope', self.SCOPE_SUBTREE) # generate filter filter_tmp = self.make_filter(kw) if filter: filter = self.combine_filters((filter, filter_tmp), self.MATCH_ALL) else: filter = filter_tmp if not filter: filter = '(objectClass=*)' # find entries and normalize the output for CRUD output = [] (entries, truncated) = self.find_entries( filter, attrs_list, base_dn, scope ) for (dn, entry_attrs) in entries: entry_attrs['dn'] = [dn] output.append(entry_attrs) if truncated: return (-1, output) return (len(output), output) api.register(ldap2)
generated by cgit (git 2.34.1) at 2025-09-09 20:43:32 +0000