Validate the raw.xz-ended image for testcloud
AbandonedPublic

Authored by jskladan on Jun 22 2016, 9:58 AM.

Details

Reviewers
lbrabec
lnie
Group Reviewers
testcloud
Summary

Add some code to testcloud to validate the raw.xz-ended image

Test Plan

Just run testcloud with a raw.xz-ended image and check whether it works well

Diff Detail

Repository
rTCLOUD testcloud
Branch
angel
Lint
Lint OK
Unit
Unit Tests OK
Build Status
Buildable 686
Build 686: arc lint + arc unit
lnie retitled this revision from to Validate the raw.xz-ended image for testcloud.Jun 22 2016, 9:58 AM
lnie updated this object.
lnie edited the test plan for this revision. (Show Details)
lnie added a reviewer: testcloud.
lbrabec requested changes to this revision.Jun 22 2016, 12:45 PM
lbrabec added a reviewer: lbrabec.
lbrabec added a subscriber: lbrabec.
lbrabec added inline comments.
testcloud/image.py
158–159

I see why to move os.path.exists() somewhere else, but this function then becomes merely a wrapper over shutil.copy() call. It is little obfuscating for me, I'd suggest to remove this function, any thoughts?

189

Popen() or check_output() from subprocess module would be better here. You can then call it with parameter cwd=config_data.STORE_DIR insead of os.chdir() and there is no need to create envirionment variable, instead use ['xz', '-d', meta_name] to call the xz decompression.

On top of that, Popen() or check_output() raises subprocess.CalledProcessError which you can catch a then raise apropriate TestcloudImageError exception.

201

There is probably no need of self.xz_bit variable, since I see only one usage of it. Instead, I would suggest not to use this variable and move the accompanying if statement.

Also, I would check the suffix with meta_name.endswith(".xz"), it is more readable.

202

Variable names chg_name and meta_name aren't too descriptive...

This revision now requires changes to proceed.Jun 22 2016, 12:45 PM
lnie added inline comments.Jun 23 2016, 9:25 AM
testcloud/image.py
201

Thanks for your hints to a so green been like me

201

Thanks for your great hint,I'm going to modify the unxz_image to something like this:
def _unxz_image(self,prexz_filename,xzto_name):
cmd = ['xz', '-d',prexz_filename]
store_dir=config_data.STORE_DIR
try:

proc = subprocess.Popen(cmd, cwd=store_dir)

except subprocess.CalledProcessError, e:

raise TestcloudImageError(e)

self.name=xzto_name
self.local_path = "{}/{}".format(config_data.STORE_DIR, self.name)

202

How about prexz_filename for meta_name and xzto_filename for chg_name

lnie added inline comments.Jun 23 2016, 9:28 AM
testcloud/image.py
189

Thanks for your great hint,I'm going to modify the unxz_image to something like this:
def _unxz_image(self,prexz_filename,xzto_name):

cmd = ['xz', '-d',prexz_filename]
store_dir=config_data.STORE_DIR
try:
    proc = subprocess.Popen(cmd, cwd=store_dir)
except subprocess.CalledProcessError, e:
    raise TestcloudImageError(e)
self.name=xzto_name
self.local_path = "{}/{}".format(config_data.STORE_DIR, self.name)
lnie added inline comments.Jun 24 2016, 6:59 AM
testcloud/image.py
158–159

Maybe we can do something else at the "wrapper" in the future?Though I have no idea about what they will be at this moment,but I tend to leave it here for the sake of symmetry : )

lnie updated this revision to Diff 2299.Jun 24 2016, 7:12 AM

update the diff using the hints getted from the package review

jskladan requested changes to this revision.Jun 24 2016, 10:56 AM
jskladan added a reviewer: jskladan.
jskladan added a subscriber: jskladan.
jskladan added inline comments.
testcloud/image.py
158–159

It does not really make sense to have this as a method. If we ever need a "wrapper" in the future, we'll create it then. This is only an unnecessary bloat right now.

188–192

I'd rather use check_output(), as Popen() is really meant to just be the underlying tool. Not a huge difference, but especially with python, it is for the best to use the highest level of abstraction available.

193–194

Having this kind of side effect in what is (by the name, and assumed intent) a "decompress x to y" method is not really a good idea. Move this to the prepare() method.

194

This is what os.path.join() is for.

199–205

I don't really see the usefullness of the variable. The code should be fine with just self.name instead of the prexz_fname.

201

Once again, use os.path.join()

202

I think it would be more semantic to switch the conditions in the if-clause, so the "less general" case is second. Just a proposal, though.

207–212

With the side-effect of _unxz_image removed, this can be simplified to something along the lines of:

if self.name.endswith(".xz"):
    if not os.path.exists(postxz_path):
        self._unxz_image(self.name, postxz_fname)
    self.name = postxz_fname
    self.local_path = os.path.join(config_data.STORE_DIR, self.name)
This revision now requires changes to proceed.Jun 24 2016, 10:56 AM
lnie added inline comments.Jun 27 2016, 8:02 AM
testcloud/image.py
199–205

yeah, just use self.name is better.The fault of my OCD,which I'm going to avoid: )

207–212

Looks pretty gentle compared to mine,thanks for your great hints to such a green been like me: )

lnie updated this revision to Diff 2305.Jun 27 2016, 8:14 AM

modify the diff code using the hints from the package review

jskladan requested changes to this revision.Jun 27 2016, 1:35 PM

I think it's almost there. The only real problem I have is with the _unxz_image() method, as described in the comment.

Thank you for doing this!

testcloud/cli.py
121 ↗(On Diff #2305)

Once again, I'd rather see os.path.join() here. I know that the testcloud's code is full of this antipattern, but let's make it right, with the new code we add.

160 ↗(On Diff #2305)

Please add docstring.

testcloud/image.py
184

I think it makes sense to leave the prexz_fname parameter in place, so the method can be used (and tested) with an other path than the name in self.file, it will also be a bit more readable, to pass in the name of the file to be decompressed explicitly.

On the other hand, I don't really understand the postxz_fname argument here - it is not used in any way, but it is required for the method call.

Also, please add docstring.

199–205

[0:-3] is the same as [:-3]. Not important, just a tip.

This revision now requires changes to proceed.Jun 27 2016, 1:35 PM

It seems like you wrapped changes for two different tickets into one diff, which ended up in both reviews (D898 and D905). Please make sure just the relevant changes are present in the Differentials.

lnie added a comment.Jun 28 2016, 6:48 AM

It seems like you wrapped changes for two different tickets into one diff, which ended up in both reviews (D898 and D905). Please make sure just the relevant changes are present in the Differentials.

yeah, I will try to avoid this by getting more familiar with git and phabricator :)

lnie updated this revision to Diff 2310.Jun 28 2016, 7:13 AM

update the diff using the hints from the review

jskladan requested changes to this revision.Jun 28 2016, 12:02 PM
jskladan added inline comments.
testcloud/image.py
199–211

Seems like you don't have your git in order. We agreed on just using self.name instead of prexz_fname.

This revision now requires changes to proceed.Jun 28 2016, 12:02 PM
lnie added inline comments.Jun 29 2016, 4:27 AM
testcloud/image.py
199–211

I think it makes sense to leave the prexz_fname parameter in place, so the method can be used (and tested) with an other path >than the name in self.file, it will also be a bit more readable, to pass in the name of the file to be decompressed explicitly.

I thought you changed your idea and wanted me to reuse prexz_fname ,and I think prexz_fname is better,after saw what you said above. So,shall we use prexz_fname instead of self.name? :)

jskladan added inline comments.Jun 29 2016, 7:40 AM
testcloud/image.py
199–211

No. Nowhere am I talking about having prexz_fname instead of self.name in the context of this method. The comments regarding unxz_image are relevant only to that method.

If you hover over a comment in Phabricator, it shows you the lines which are commented on (it might be more than one). This might be helpful.

Ad the prexz_fname comment - the way it was when I commented did not make any sense - you required a postxz_fname parameter, that was not even used at all, and worked on a side effect (value of self.name) instead. There, it really makes sense to either have no input param at all, or (as I suggested) have an input parameter describing what file is to be decompressed.

lnie added inline comments.Jul 12 2016, 4:33 AM
testcloud/image.py
199–211

Sorryyy for my late reply,If I'm understanding you correctly,bet I am,what you are saying is :change self._unxz_image(prexz_fname) to self._unxz_image(prexz_fname),right? : )

There, it really makes sense to either have no input param at all, or (as I suggested) have an input parameter describing what >file is to be decompressed.

That's a really good suggestion,I'd prefer it.

jskladan commandeered this revision.Jan 13 2017, 2:02 PM
jskladan abandoned this revision.
jskladan edited reviewers, added: lnie; removed: jskladan.

Abandoning, as there was no change on this for half a year