summaryrefslogtreecommitdiffstats
path: root/ipatests/conftest.py
Commit message (Collapse)AuthorAgeFilesLines
* Make use of Azure Pipeline slicingStanislav Levin2019-07-161-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The unit tests execution time within Azure Pipelines(AP) is not balanced. One test job(Base) takes ~13min, while another(XMLRPC) ~28min. Fortunately, AP supports slicing: > An agent job can be used to run a suite of tests in parallel. For example, you can run a large suite of 1000 tests on a single agent. Or, you can use two agents and run 500 tests on each one in parallel. To leverage slicing, the tasks in the job should be smart enough to understand the slice they belong to. >The step that runs the tests in a job needs to know which test slice should be run. The variables System.JobPositionInPhase and System.TotalJobsInPhase can be used for this purpose. Thus, to support this pytest should know how to split the test suite into groups(slices). For this, a new internal pytest plugin was added. About plugin. - Tests within a slice are grouped by test modules because not all of the tests within the module are independent from each other. - Slices are balanced by the number of tests within test module. - To run some module within its own environment there is a dedicated slice option (could help with extremely slow tests) Examples. - To split `test_cmdline` tests into 2 slices and run the first one: ipa-run-tests --slices=2 --slice-num=1 test_cmdline - To split tests into 2 slices, then to move one module out to its own slice and run the second one: ipa-run-tests --slices=2 --slice-dedicated=test_cmdline/test_cli.py \ --slice-num=2 test_cmdline Fixes: https://pagure.io/freeipa/issue/8008 Signed-off-by: Stanislav Levin <slev@altlinux.org> Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
* Simplify ipa-run-tests scriptStanislav Levin2019-07-161-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a sort of rollback to the pre #93c158b05 state with several improvements. For now, the nodeids calculation by ipa-run-tests is not stable, since it depends on current working directory. Nodeids (tests addresses) are utilized by the other plugins, for example. Unfortunately, the `pytest_load_initial_conftests` hook doesn't correctly work with pytest internal paths, since it is called after the calculation of rootdir was performed, for example. Eventually, it's simpler to follow the default convention for Python test discovery. There is at least one drawback of new "old" implementation. The ignore rules don't support globs, because pytest 4.3.0+ has the same facility via `--ignore-glob`: > Add the `--ignore-glob` parameter to exclude test-modules with > Unix shell-style wildcards. Add the collect_ignore_glob for > conftest.py to exclude test-modules with Unix shell-style > wildcards. Upon switching to pytest4 it will be possible to utilize this. Anyway, tests for checking current basic facilities of ipa-run-tests were added. Fixes: https://pagure.io/freeipa/issue/8007 Signed-off-by: Stanislav Levin <slev@altlinux.org> Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
* Fix Pytest4.1+ warnings about pytest.configStanislav Levin2019-06-191-2/+8
| | | | | | | | | | | | pytest.config global is deprecated since Pytest4.1: https://docs.pytest.org/en/latest/deprecations.html#pytest-config-global https://github.com/pytest-dev/pytest/issues/3050 Fixes: https://pagure.io/freeipa/issue/7981 Co-authored-by: Christian Heimes <cheimes@redhat.com> Signed-off-by: Stanislav Levin <slev@altlinux.org> Reviewed-By: Christian Heimes <cheimes@redhat.com>
* Don't import ipaserver in conf.pyChristian Heimes2019-05-161-9/+3
| | | | Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
* Add option to remove lines from a fileChristian Heimes2019-04-021-0/+13
| | | | | | | | config_replace_variables() can now also remove lines from a file. Related: https://pagure.io/freeipa/issue/7860 Signed-off-by: Christian Heimes <cheimes@redhat.com> Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
* Make conftest compatible with pytest 4.xChristian Heimes2019-01-081-2/+8
| | | | | | | | pytest 3.6 has deprecated get_marker in 3.6. The method was removed in 4.x and replaced with get_closest_marker. Signed-off-by: Christian Heimes <cheimes@redhat.com> Reviewed-By: Thomas Woerner <twoerner@redhat.com>
* Fix pytest deprecation warningChristian Heimes2018-11-151-1/+1
| | | | | | | | | | | conftest uses the Function attribute of a pytest.Function object. Latest pytest has deprecated the attribute: _pytest.warning_types.RemovedInPytest4Warning: usage of Function.Function is deprecated, please use pytest.Function instead Signed-off-by: Christian Heimes <cheimes@redhat.com> Reviewed-By: Serhii Tsymbaliuk <stsymbal@redhat.com>
* Rename pytest_plugins to ipatests.pytest_ipaChristian Heimes2018-08-021-6/+6
| | | | | | | | | | pytest 3.7.0 doesn't like ipatests.pytest_plugins package. The string "pytest_plugins" is used as marker to load plugins. By populare vote and to avoid future conflicts, we decided to rename the directory to pytest_ipa. Fixes: https://pagure.io/freeipa/issue/7663 Signed-off-by: Christian Heimes <cheimes@redhat.com> Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
* Make fasttest pass without ~/.ipa/default.confChristian Heimes2018-03-161-1/+6
| | | | | | | | | | | Some fast tests depend on an api.env with realm, domain, and host. On machines without ~/.ipa/default.conf, the settings are not available. Provide dummy values to make tests pass. Closes: https://pagure.io/freeipa/issue/7432 Signed-off-by: Christian Heimes <cheimes@redhat.com> Reviewed-By: Rob Crittenden <rcritten@redhat.com> Reviewed-By: Aleksei Slaikovskii <aslaikov@redhat.com>
* Add marker needs_ipaapi and option to skip testsChristian Heimes2017-12-111-0/+10
| | | | | | | | | | | | | | | The new marker needs_ipaapi is used to mark tests that needs an initialized API (ipalib.api) or some sort of other API services (running LDAP server) to work. Some packages use api.Command or api.Backend on module level. They are not marked but rather skipped entirely. A new option ``skip-ipaapi`` is added to skip all API based tests. With the option, only simple unit tests are executed. As of now, freeIPA contains more than 500 unit tests that can be executed in about 5 seconds. Signed-off-by: Christian Heimes <cheimes@redhat.com> Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
* ipatests: do not finalize api when IPA is not configuredTomas Krizek2017-07-141-1/+2
| | | | | | | | | | | Pytest can be executed from a machine that doesn't have IPA configured. In this case, api can't be finalized because values such as basedn are unknown and missing. Fixes https://pagure.io/freeipa/issue/7046 Signed-off-by: Tomas Krizek <tkrizek@redhat.com> Reviewed-By: Martin Basti <mbasti@redhat.com>
* tox testing support for client wheel packagesChristian Heimes2017-04-121-1/+3
| | | | | | | | | | | | | Add tox infrastructure to test client wheel packages workflow: * build client packages * install client packages * ipa-run-tests --ipaclient-unittests under Python 2 and 3 * pylint of client packages under Python 2 and 3 * placeholder packages work as expected Signed-off-by: Christian Heimes <cheimes@redhat.com> Reviewed-By: Martin Basti <mbasti@redhat.com>
* Add options to run only ipaclient unittestsChristian Heimes2017-03-171-1/+33
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A new option for ipa-run-tests makes the test runner ignore subdirectories or skips tests that depend on the ipaserver package or on a running framework for RPC integration tests. The new option enables testing of client-only builds. $ ipatests/ipa-run-tests --ipaclient-unittests ... platform linux2 -- Python 2.7.13, pytest-2.9.2, py-1.4.32, pluggy-0.3.1 rootdir: /home/heimes/redhat, inifile: tox.ini plugins: sourceorder-0.5, cov-2.3.0, betamax-0.7.1, multihost-1.1 collected 451 items test_util.py ........ util.py .. test_ipaclient/test_csrgen.py ..............ssss... test_ipalib/test_aci.py ................... test_ipalib/test_backend.py ........ test_ipalib/test_base.py ............... test_ipalib/test_capabilities.py . test_ipalib/test_cli.py ... test_ipalib/test_config.py ............... test_ipalib/test_crud.py ............... test_ipalib/test_errors.py ....... test_ipalib/test_frontend.py ........................................ test_ipalib/test_messages.py .... test_ipalib/test_output.py ... test_ipalib/test_parameters.py ............................................................. test_ipalib/test_plugable.py ........ test_ipalib/test_rpc.py ......ssssssss test_ipalib/test_text.py ............................. test_ipalib/test_x509.py ... test_ipapython/test_cookie.py ............ test_ipapython/test_dn.py ........................... test_ipapython/test_ipautil.py .................................................................. test_ipapython/test_ipavalidate.py .......... test_ipapython/test_kerberos.py .............. test_ipapython/test_keyring.py .......... test_ipapython/test_ssh.py ............................... test_pkcs10/test_pkcs10.py ..... https://fedorahosted.org/freeipa/ticket/6517 Signed-off-by: Christian Heimes <cheimes@redhat.com> Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
* Print test env informationChristian Heimes2017-02-091-0/+11
| | | | | | | | Print api.env, uname, euid/egid, cwd and Python version when tests are run with -v (e.g. ipa-run-tests -v). Signed-off-by: Christian Heimes <cheimes@redhat.com> Reviewed-By: Martin Basti <mbasti@redhat.com>
* pytest: set rules to find test files and functionsChristian Heimes2017-01-121-6/+10
| | | | | | | | | 1e06a5195bafe0224d77371987f2509f5508ca2f removed pytest.ini. Without the ini file, pytest 3.x has suboptimal settings and no longer picks up all test functions and test files. Signed-off-by: Christian Heimes <cheimes@redhat.com> Reviewed-By: Milan Kubik <mkubik@redhat.com>
* Use pytest conftest.py and drop pytest.iniChristian Heimes2017-01-051-0/+76
Let's replace some ugly hacks with proper pytest conftest.py hooks. Test initialization of ipalib.api is now handled in pytest_cmdline_main(). Pytest plugins, markers and ignores are also moved into conftest.py. Additional guards make it possible to run tests without ipaserver installed. I added confcutdir to ensure that pytest does not leave our project space. Pytest used pytest.ini or setup.py before but pytest.ini is gone. Signed-off-by: Christian Heimes <cheimes@redhat.com> Reviewed-By: Milan Kubik <mkubik@redhat.com>
generated by cgit (git 2.34.1) at 2025-08-29 13:10:12 +0000