summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
...
* Make get_compression_type() reusableRadostin Stoyanov2017-08-281-14/+27
| | | | | Take out the code for getting compression type of tarball in separate function.
* Get layer size if not providedRadostin Stoyanov2017-08-282-7/+7
| | | | | | | Docker registry with Manifest v1 does not require the size of layers to be included. However, when this information is not provided we can use os.path.getsize() to get and show the size of tarball. We can also use this function for FileSource to provide consistent output messages.
* DockerSource: Split checksum and layersRadostin Stoyanov2017-08-284-19/+27
| | | | | | | | | | | | The current implementation store in one list: - checksum - checksum type - file path - file size However, the information about checksum and checksum type is only used to verify the content of tarball before it is being extracted. Splitting these into separate lists would allow us to reuse the function untar_layers() with FileSource.
* get_mime_type: Properly close stdout handleRadostin Stoyanov2017-08-281-5/+6
| | | | | This aims to fix the warning of Python3: ResourceWarning: unclosed file <_io.BufferedReader name=3>
* untar: Use unique name with virt-sandboxRadostin Stoyanov2017-08-281-0/+1
| | | | | | Specify unique name when creating Libvirt domain with virt-sandbox. Otherwise the default name "sandbox" will be used and this might result in collision with another instance of virt-bootstrap.
* Add regression testsRadostin Stoyanov2017-08-283-3/+757
| | | | | | These tests aim to verify the output of virt-bootstrap by creating tar archives which contain dummy root file system and call the function bootstrap(). The check the extracted root file system.
* Improve untar commandRadostin Stoyanov2017-08-281-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Add "--overwrite" to enforce the overwrite of existing files. Add the flag "--absolute-names" to disable the strip of leading '/'s This is used to get around the error "Cannot open:Permission denied" which occurs when the qemu driver is used by virt-sandbox. It is used for unprivileged users to create isolated environment in which tar is executed to extract the content from container image layers. In particular this error occurs when the tar archive contains symbolic link which has target path starting with '/'. Steps to reproduce: $ mkdir /tmp/foo $ cd /tmp/foo $ touch file $ ln -s /tmp/foo/file link $ tar -cf archive.tar link $ mkdir /tmp/foo/dest $ virt-sandbox -c qemu:///session \ -m host-bind:/mnt=/tmp/foo/dest \ -- /bin/tar xf /tmp/foo/archive.tar -C /mnt Error message: tar: link: Cannot open: Permission denied tar: Exiting with failure status due to previous errors
* Drop unit testsRadostin Stoyanov2017-08-287-1948/+180
| | | | | | | | | Unit tests were used to ensure that functions and methods work as expected. However, these tests are closely related to the implementation and will result in major changes after some refactoring. To reduce the amount of work needed to add new features or changes to the code most of these tests will be replaced with more abstract form of testing introduced in the following commits.
* Add a more to gitignoreCédric Bosdonnat2017-08-031-0/+4
|
* Add guestfs python binding dependencyCédric Bosdonnat2017-08-032-1/+5
| | | | | | Adding the python binding of libguestfs as dependency will allow switching from calls to the virt-* tools and potentially optimize processing time.
* Use requirements.txt for dependenciesCédric Bosdonnat2017-08-033-4/+5
| | | | | | | | | | | | | | | | | | | | Later on when we will introduce libguestfs dependency, we would need to add a dependency_links to setup.py. This is rather hacky and https://caremad.io/posts/2013/07/setup-vs-requirement/ recommends using requirements.txt. Thus, to install the package when all dependencies are already installed, the following command can be used: python setup.py install Otherwise, the recommended way is to use pip as follows: pip install -r recommends.txt -e . This will also download the guestfs python binding, build and install it. However this would be skipped if installed from a linux distro package if this one provides the egg-info.
* Split 'sources' module into filesRadostin Stoyanov2017-08-035-57/+118
| | | | | | | | | | | | | Improve readability by spliting the 'sources' module into separate files. Each file contains only one class. In addition update the mock statements in the unit tests to match these changes. Add recursive-include in MANIFEST.in to include virtBootstrap.sources module. Update the unit tests to match these changes.
* setup: Use find_packages()Radostin Stoyanov2017-08-031-1/+1
| | | | | | | The find_packages() [1] function returns a list of package names suitable for use as the packages argument to setup(). [1] http://setuptools.readthedocs.io/en/latest/setuptools.html#using-find-packages
* Use explicit importRadostin Stoyanov2017-08-035-39/+67
| | | | | Reduce the number of import statements and improve readability. Update the unit tests to match these changes.
* tests: Remove redundant argumentsRadostin Stoyanov2017-08-031-19/+22
|
* sources: Use default values of argumentsRadostin Stoyanov2017-08-031-6/+6
|
* sources: Set default format with a constantRadostin Stoyanov2017-08-032-2/+4
|
* tests: Add unit tests for the "utils" moduleRadostin Stoyanov2017-07-241-0/+699
|
* tests: Add unit tests for "progress" moduleRadostin Stoyanov2017-07-241-0/+112
|
* tests: Add unit tests for FileSourceRadostin Stoyanov2017-07-241-0/+171
|
* tests: Add unit tests for DockerSourceRadostin Stoyanov2017-07-241-0/+602
|
* tests: Add unit tests for 'virt_bootstrap' moduleRadostin Stoyanov2017-07-242-0/+503
|
* Use tox to test against multiple Python versionsRadostin Stoyanov2017-07-243-0/+25
| | | | | 'Tox' is tool that will check if the package installs correctly and all unit tests pass with different Python versions.
* Add support for setting root passowrd in qcow2Radostin Stoyanov2017-07-242-9/+32
| | | | | For containers bootstrapped in qcow2 format use the tool 'virt-edit' to insert the hashed root password in the shadow file of the last layer
* DockerSource: Show correct err msg on skopeo failRadostin Stoyanov2017-07-241-1/+1
| | | | | The first argument of CalledProcessError() must be the returncode not the command.
* DockerSource: Fix formula of download progressRadostin Stoyanov2017-07-241-3/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | When downloading image with multiple layers, the download progress value of every following layer should not start from 0. If we have 10 layers, downloading each of them should increase the total download progress by 10%. Assuming that the download and extraction are 50/50 of the total work. Then, downloading each of 10 layers will increase the progress value with 5% of the total work. When all layers are downloaded the progress value should be 50%. However, with the current formula the progress value of each layer starts from 0%. (E.g. when downloading 2nd layer of 10 the download progress starts from 0% instead of 5%.) This bug can be seen when downloading images with multiple layers of large size. Example: virt-bootstrap docker://rails /tmp/foo --status-only
* tests: Add test suiteRadostin Stoyanov2017-07-243-1/+16
|
* Add remapping ownership of files in rootfsRadostin Stoyanov2017-07-241-5/+122
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When Libvirt creates LXC container with enabled user namespace the ownership of files in the container should be mapped to the specified target UID/GID. The implementation of the mapping is inspired by the tool uidmapshift: http://bazaar.launchpad.net/%7Eserge-hallyn/+junk/nsexec/view/head:/uidmapshift.c Mapping values can be specified with the flags: --idmap Map both UIDs/GIDs --uidmap Map UIDs --gidmap Map GIDs Each of these flags can be specified multiple times. Example: virt-bootstrap docker://fedora /tmp/foo --uidmap 0:1000:10 --gidmap 0:1000:10 Will map the ownership of files with UIDs/GIDs: 0-9 to 1000-1009 The same result can be achived with: virt-bootstrap docker://fedora /tmp/foo --idmap 0:1000:10 Multiple mapping values can be specified as follows: virt_bootstrap.py docker://ubuntu /tmp/foo --idmap 0:1000:10 --idmap 500:1500:10 This will map the UID/GIDs: 0-9 to 1000-1009 and 500-509 to 1500-1509
* Update argument helpersRadostin Stoyanov2017-07-211-7/+4
|
* Specify logger name when executed directlyRadostin Stoyanov2017-07-211-1/+1
|
* readme: Fix typoRadostin Stoyanov2017-07-211-1/+1
|
* Drop need of root privileges to set root passwordRadostin Stoyanov2017-07-213-15/+41
| | | | | | | | | | | | | | These changes aim to avoid the requirement for root privileges when setting the password of root user on root file system. The "-R, --root" flag of chpasswd is using chroot to apply changes in root file system and this requires root privileges. [1] Instead compute hash of the root password using passlib [2] and insert the value in the /etc/shadow file in the rootfs. [1] https://en.wikipedia.org/wiki/Chroot#Limitations [2] http://passlib.readthedocs.io/en/stable/lib/passlib.hosts.html
* DockerSource: Add support for Manifest version 1Radostin Stoyanov2017-07-212-5/+20
| | | | | | | | | | | | | | | | Handle differences between version 1 and 2 of the Manifest file for Docker registry. Layers' blob sums in v1 are stored in a list "fsLayers" and the digest is stored in following filed with name "blobSum". [1] In v2 the layer list is stored in field with name "layers" and ordered starting from the base image (opposite order of v1). The digest is stored under a following field with name "digest". The size in bytes is included in a field with name "size". [2] [1] https://docs.docker.com/registry/spec/manifest-v2-1/#manifest-field-descriptions [2] https://docs.docker.com/registry/spec/manifest-v2-2/#image-manifest-field-descriptions
* Python 3/2 compatibility: Convert Byte-str to StrRadostin Stoyanov2017-07-212-8/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Encoded Unicode in Python 3 is represented as binary data. The difference with Python2 is that any attempt to mix text and data in Python 3.0 raises TypeError, whereas if you were to mix Unicode and 8-bit strings in Python 2.x, it would work if the 8-bit string happened to contain only 7-bit (ASCII) bytes, but you would get UnicodeDecodeError if it contained non-ASCII values. Reference: https://docs.python.org/release/3.0.1/whatsnew/3.0.html#text-vs-data-instead-of-unicode-vs-8-bit Example: Python 2: >>> b'foo.bar'.split('.') ['foo', 'bar'] Python 3: >>> b'foo.bar'.split('.') Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: a bytes-like object is required, not 'str' >>> b'foo.bar'.split(b'.') [b'foo', b'bar']
* DockerSource: Don't calc progress on short linesRadostin Stoyanov2017-07-181-0/+3
| | | | | Perform additional check of the input to avoid unexpected behaviour when string is passed instead of list or a list with short length.
* DockerSource: Remove '/' at the end of URIRadostin Stoyanov2017-07-181-1/+1
| | | | | | | | | This change aims to convert URI: docker:///repository/my_image/ to: docker://repository/my_image
* DockerSource: Encapsulate URI generationRadostin Stoyanov2017-07-181-14/+19
| | | | This change makes it easier to test the code.
* DockerSource: Encapsulate layers' info retrievalRadostin Stoyanov2017-07-181-3/+9
| | | | This change makes it easier to test the code.
* sources: Update doc stringsRadostin Stoyanov2017-07-181-2/+9
|
* utils: bytes_to_size remove trailing spaceRadostin Stoyanov2017-07-181-1/+1
| | | | | Remove traling space which occurs when the number is smaller than kibibyte.
* utils: size_to_bytes convert from intRadostin Stoyanov2017-07-181-2/+2
| | | | | | | | | | | | When converting 0 KB with string input the result will be string with zeroes. >>> print(size_to_bytes('0', 'KB')) 000000... Instead convert the string input to integer and then perform the conversion. Rename the variable from "string" to "number" to avoid confusion.
* Add script to run virt-bootstrap for hackersCédric Bosdonnat2017-07-182-0/+29
| | | | | | | run script takes care of setting the environment variables (only PYTHONPATH so far) needed to run virt-bootstrap without installing it. This wrapper script works in the same fashion than the ones in virt-manager or libvirt projects for consistency.
* DockerSource: Ignore short lines from skopeoRadostin Stoyanov2017-07-051-2/+2
| | | | | Avoid parsing short lines when detecting download progress from skopeo's output.
* Rename format_number() into bytes_to_size()Cédric Bosdonnat2017-07-051-2/+2
| | | | rename the function to match its counterpart size_to_bytes().
* Gather common utility functions in "utils" moduleRadostin Stoyanov2017-07-053-334/+369
| | | | | | Add new module to collect utility functions used in virt-bootstrap. Move the function definitions from "sources" and "virt_bootstrap" to the new module.
* get_image_details: Use connection optionsRadostin Stoyanov2017-07-051-4/+13
| | | | | Pass the "not_secure", "username" and "password" values to "skopeo inspect" when manifest is retrieved.
* Detect and log download progress of layersRadostin Stoyanov2017-07-051-1/+136
| | | | | | | | | | Parse skopeo's output messages to detect and log the donwload progress for each layer and update the progress of virt-bootstrap. Example: virt-bootstrap docker://ubuntu /tmp/foo --status-only Status: Downloading layer (2/5), Progress: 25.30%
* Add --status-only flagRadostin Stoyanov2017-07-051-3/+29
| | | | | When this flag is passed only the current state of virt-bootstrap will be shown along with the total bootstrap progress.
* Add new module to store the progressRadostin Stoyanov2017-07-053-21/+108
| | | | | | | | | | | | | | | | | | | | This module is used to store the progress of the bootstrap process and could be used by other applications to get status of virt-bootstrap as well as percentage of completion. Convert some info messages to use this new system for a clean reporting in client applications. Example usage: import virtBootstrap def show_progress(data): print("Status: %s, Progress: %.2f" % (data['status'], data['value'])) virtBootstrap.bootstrap(uri='docker://ubuntu', dest="/tmp/test1", progress_cb=show_progress)
* checksum: Log failuresRadostin Stoyanov2017-07-051-2/+8
|
generated by cgit (git 2.34.1) at 2025-09-27 09:23:05 +0000