Use nss calls with timeout in extdom plugin

Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>

Expose nss timeout parameter. Use sss_nss_getorigbyname_timeout
instead of sss_nss_getorigbyname

Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>

Old implementation handles username and group names with
one common call. Character @ is used in the call to detect UPN.

Group name can legaly contain this character and therefore the
common approach doesn't work in such case.

Also the original call is less efficient because it tries to resolv
username allways then it fallback to group resolution.

Here we implement two new separate calls for resolving users and
groups.

Fixes: https://bugzilla.redhat.com/1746951
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>

Simulate getgrgid_r() timeout when packing list of groups user is a
member of in pack_ber_user().

Related: https://pagure.io/freeipa/issue/8044
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>

A return code LDAP_NO_SUCH_OBJECT will tell SSSD on the IPA client to
remove the searched object from the cache. As a consequence
LDAP_NO_SUCH_OBJECT should only be returned if the object really does
not exists otherwise the data of existing objects might be removed form
the cache of the clients causing unexpected behaviour like
authentication errors.

Currently some code-paths use LDAP_NO_SUCH_OBJECT as default error code.
With this patch LDAP_NO_SUCH_OBJECT is only returned if the related
lookup functions return ENOENT. Timeout related error code will lead to
LDAP_TIMELIMIT_EXCEEDED and LDAP_OPERATIONS_ERROR is used as default
error code.

Fixes: https://pagure.io/freeipa/issue/8044
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>

Coverity found the following issue:
Error: BAD_COMPARE (CWE-697): [#def1]
freeipa-4.6.5/daemons/ipa-slapi-plugins/ipa-extdom-extop/ipa_extdom_extop.c:121: null_misuse: Comparing pointer "threadnumber" against "NULL" using anything besides "==" or "!=" is likely to be incorrect.

The comparison is using the pointer while it should use the pointed value.

Fixes: https://pagure.io/freeipa/issue/7884
Reviewed-By: Christian Heimes <cheimes@redhat.com>

The user and group lookups done by the extdom plugin might need some
time depending on the state of the service (typically SSSD) handling the
requests.

To avoid that all worker threads are busy waiting on a connect or a
reply from SSSD and no other request can be handled this patch adds an
instance counter and an instance limit for the extdom plugin.

By default the limit will be around 80% of the number of worker threads.
It can be tuned further with the plugin option ipaExtdomMaxInstances
which must in set in ipaextdommaxinstances and should have an integer
value larger than 0 and lesser than the number of worker threads.

If the instance limit is reached the extdom plugin will return LDAP_BUSY
for every new request until the number of instance is again below the
limit.

Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>

The code in question was supposed to have the same license as the
rest of the plugin. Fix it by updating the comment header.

Signed-off-by: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Thomas Woerner <twoerner@redhat.com>

NSS_STATUS_RETURN is an internal value but GCC doesn't know that.

ipa_extdom_common.c:103:5: warning: enumeration value ‘NSS_STATUS_RETURN’ not handled in switch [-Wswitch]

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>

Refactor nsswitch operations in ipa-extdom-extop plugin to allow use
of timeout-enabled nsswitch calls provided by libsss_nss_idmap.

Standard POSIX nsswitch API has no way to cancel requests which may
cause ipa-extdom-extop requests to hang far too long and potentially
exhaust LDAP server workers. In addition, glibc nsswitch API iterates
through all nsswitch modules one by one and with multiple parallel
requests a lock up may happen in an unrelated nsswitch module like
nss_files.so.2.

A solution to the latter issue is to directly load nss_sss.so.2 plugin
and utilize it. This, however, does not solve a problem with lack of
cancellable API.

With SSSD 1.16.1, libsss_nss_idmap provides a timeout-enabled variant of
nsswitch API that is directly integrated with SSSD client side machinery
used by nss_sss.so.2. As result, this API can be used instead of loading
nss_sss.so.2 directly.

To support older SSSD version, both direct loading of nss_sss.so.2 and
new timeout-enabled API are supported by this changeset. An API to
abstract both is designed to be a mix between internal glibc nsswitch
API and external nsswitch API that libsss_nss_idmap mimics. API does not
expose per-call timeout. Instead, it allows to set a timeout per
nsswitch operation context to reduce requirements on information
a caller has to maintain.

A choice which API to use is made at configure time.

In order to test the API, a cmocka test is updated to explicitly load
nss_files.so.2 as a backend. Since use of nss_sss.so.2 would always
depend on availablility of SSSD, predictable testing would not be
possible without it otherwise. Also, cmocka test does not use
nss_wrapper anymore because nss_wrapper overrides higher level glibc
nsswitch API while we are loading an individual nsswitch module
directly.

As result, cmocka test overrides fopen() call used by nss_files.so.2 to
load /etc/passwd and /etc/group. An overridden version changes paths to
/etc/passwd and /etc/group to a local test_data/passwd and
test_data/group. This way we can continue testing a backend API for
ipa-extdom-extop with the same data as with nss_wrapper.

Fixes https://pagure.io/freeipa/issue/5464

Reviewed-By: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Simo Sorce <ssorce@redhat.com>
Reviewed-By: Robbie Harwood <rharwood@redhat.com>