Repurposed Ansible module documentation code to fit our needs for documenting directives. This is a rough proof of concept -
and needs cleaned up before going into libtaskotron.
Details
- Reviewers
kparal tflink jskladan - Maniphest Tasks
- T214: all directives must document their return values
T142: Pull directive documentation into main libtaskotron documentation - Commits
- rLTRNae5cda6b72d2: refactor: updated blacklist modules
rLTRNa7d205681351: Added Ansible style docs to directives as a rough Proof of Concept.
rLTRNcd860d39c69d: Merged develop in and fixed conflicts.
rLTRNd844f88443fb: Moved doc_utils.py to docs/ and removed generated rst files from docs/source…
No testplan needed at this point. Just review and discussion on licensing.
Diff Detail
- Repository
- rLTRN libtaskotron
- Branch
- T142-directivedocs
- Lint
No Linters Available - Unit
No Unit Test Coverage
Overall, I like the output and the concept.
The license question aside, there are a couple of changes I'd like to see if we go forward with this:
- ignore .py[co] files so that docs can be built in dev environments
- don't add the <directive_name>.rst files to git - they're generated and don't need to be in version control
- get doc_utils.py out of the libtaskotron module and into a separate util module unless there's a reason to keep it there
- change rst template for directives to use the heading levels as specified in devguide.rst
The other thing I'm wondering about is whether statically defining the list_of_directive_modules and modules_by_category by hand is the best way to go about doing that. It looks like generating those files is a TODO?
| docs/source/check_module.rst | ||
|---|---|---|
| 1 ↗ | (On Diff #260) | I don't think that this should be pulled into generation - it's not a directive and should be documented with non-yaml docstrings |
tflink said:
ignore .py[co] files so that docs can be built in dev environments don't add the <directive_name>.rst files to git - they're generated and don't need to be in version control get doc_utils.py out of the libtaskotron module and into a separate util module unless there's a reason to keep it there change rst template for directives to use the heading levels as specified in devguide.rst
The other thing I'm wondering about is whether statically defining the list_of_directive_modules and modules_by_category by hand is the best way to go about doing that. It looks like generating those files is a TODO?
I can make it ignore .py[co] files easy enough. Also, the rst files being in git were something I didn't intend to do - thought I cleaned those out....
Most of the comments in the code are from Ansible - not me. So that TODO is something they're looking into writing. Though if it's something we want I could always contribute back upstream.
| docs/source/check_module.rst | ||
|---|---|---|
| 1 ↗ | (On Diff #260) | It wasn't supposed to be there - it got copied while I was messing with the file globbing logic and I didn't clean it out. Sorry for the confusion... |
- Moved doc_utils.py to docs/ and removed generated rst files from docs/source since they get rebuilt every time.
- Merged develop in and fixed conflicts.
I like the concept and the yaml-in-docstring format, my only question is about how the links show up in the rendered docs docs.
I've rendered an older version of the docs at http://tflink.fedorapeople.org/taskotron/demo-directive-docs/
The directives show up as a list of links off the index, under Directives Modules.
- Do we want the directive docs to all be on a single page?
- Do we want to have a different way of linking to the directives?
- should the rendered directive docs be committed in git?
Not commenting on any of the code. I'm not at all against documentation, but the form/format that is suggested here seems to just add a wall of text, that could easily be present in a separate file.
I would either like it to be a docstring (so it can have an use for the programmer), or moved out of the sources.
I am also not a big fan of:
# If a key doesn't apply to your module (ex: choices, default, or # aliases) you can use the word 'null', or an empty list, [], where # appropriate.
This should IMHO be done programatically using reasonable defaults in the parser. If we decide to keep the long text directly in the files, I'd like to be able to shorten it as much as possible.
Also, are there any real-life examples of the expected contents of DOCUMENTATION/EXAMPLES variables? I'm not really sure what would be the expected content - meaning what are "requirements", and what are "options". I guess "options" are the arguments that can/should be passed to the directive? If so, maybe renaming to "arguments" might make sense.
Not commenting on any of the code. I'm not at all against documentation, but the form/format that is suggested here seems to just add a wall of text, that could easily be present in a separate file.
I'm of a pretty different opinion in this case - keeping the docs with the code is a good way to make sure that the docs are updated when the code changes. Using a format like this (yamlish embedded in docstring) gives us more flexibility in how the docs are rendered without having to do weird things with parsing docstrings.
I would either like it to be a docstring (so it can have an use for the programmer), or moved out of the sources.
I don't understand why this is mutually exclusive to any other docstrings or why those would be required in this case (sphinx-style docstrings, not comments). I can't think of a case where directives would be included as a library, so what would be the usecase for rendering library-style docs like we do for other parts of libtaskotron?
Of the people using the directive documentation, I expect the vast majority to be people who are writing tasks and not hacking on the libtaskotron code itself. Therefore, I want to optimize for those users even if that means a bit of inconvenience for the people working on the directive code.
I am also not a big fan of:
Agreed, I think that should be in the template but it doesn't need to be in every directive.
This should IMHO be done programatically using reasonable defaults in the parser. If we decide to keep the long text directly in the files, I'd like to be able to shorten it as much as possible.
Not sure I understand what you're getting at. Are you talking about assuming that any missing data would be equivalent to null or empty list? I would prefer to have all the options listed in the yaml for the sake of being explicit
Also, are there any real-life examples of the expected contents of DOCUMENTATION/EXAMPLES variables?
No but I suppose that a more concrete example would be helpful here.
I'm not really sure what would be the expected content - meaning what are "requirements", and what are "options". I guess "options" are the arguments that can/should be passed to the directive? If so, maybe renaming to "arguments" might make sense.
Yeah, options are the arguments to the directive. That change makes sense to me.
Requirements aren't something that we would use a lot with the current set of directives but it would be something along the lines of "fedora 21+" or "aarch64 only" etc. - things that the directive needs to run but aren't arguments.
Some comments for the code. I went over it quite fast, so there might be other stuff to come out later.
| docs/doc_utils.py | ||
|---|---|---|
| 78 ↗ | (On Diff #350) | Just out of curiosity - do these lines divide some logical units, or is this just a separator between methods? If this is just a separator, please remove it. I'm not against keeping it as a separator for logically coherent pieces of code, but having it between all methods is quite distractive, and IMHO unnecessary. |
| 124 ↗ | (On Diff #350) | if the intention really is to traverse just "get files from module_dir and one level of directories inside it" something like this might be a bit more straightforward: files = [f for f in glob.glob("%s/*/*" % module_dir) if os.path.isfile(f)]
for f in files:
tokens = f.split("/")
module = tokens[-1]
category = tokens[-2]
if not category in categories:
categories[category] = {}
categories[category][module] = f
categories['all'][module] = f
return categories |
| 159 ↗ | (On Diff #350) | What is arg1 arg2? No positional arguments are used in the code using the parser. Maybe some leftover from past? |
| 206 ↗ | (On Diff #350) | if not os.path.basename(fname).endswith('.py') |
| 243 ↗ | (On Diff #350) | The for-cycle is not really needed, and can be easily replaced with: all_keys = sorted(doc['options'].keys()) If you decide to use this, line #221 (`all_keys = []`) can (should) be removed. |
| 304 ↗ | (On Diff #350) | Can you give an example of a situation, when this if-clause is True? AFAIK given the way you construct the argparser, you will always have module_dir set - either to MODULEDIR or to the value provided on the command line. |
| 310 ↗ | (On Diff #350) | I know what you are trying to achieve, but "required option" is IMHO a bit self-contradictory. If this option is required for the code to run, then it should be a positional argument. |
| 358 ↗ | (On Diff #350) | Please use some more explanatory variable name here. |
| 365 ↗ | (On Diff #350) | child.value.s.lstrip() is much more readable IMHO. I know that in your suggested format, the first character is always \n but either test directly for its presence, and then remove it, or use lstrip() (or maybe strip() even) instead. |
I'm of a pretty different opinion in this case - keeping the docs with the code is a good way to make sure that the docs are updated when the code changes. Using a format like this (yamlish embedded in docstring) gives us more flexibility in how the docs are rendered without having to do weird things with parsing docstrings.
But it is not really embedded in docstring, it is an arbitrary variable.
I don't understand why this is mutually exclusive to any other docstrings or why those would be required in this case (sphinx-style docstrings, not comments). I can't think of a case where directives would be included as a library, so what would be the usecase for rendering library-style docs like we do for other parts of libtaskotron?
I'm not saying it is mutually exclusive, just that if this wall of text will be present at the top of the file, it should be a docstring. If it is just an assignment to an arbitrary variable, it can (IMHO should) be moved to the end of the file.
Not sure I understand what you're getting at. Are you talking about assuming that any missing data would be equivalent to null or empty list? I would prefer to have all the options listed in the yaml for the sake of being explicit
Once again - if the text won't be at the beginning of the file, I do not really care. But IMO enforcing explicitly repetetively writing "This option has no XYZ" does not add much else than noise. The tool parsing the yaml can safely add the "nothing is here" defaults (null, empty list) on its own.
No but I suppose that a more concrete example would be helpful here.
Yeah, options are the arguments to the directive. That change makes sense to me.
Requirements aren't something that we would use a lot with the current set of directives but it would be something along the lines of "fedora 21+" or "aarch64 only" etc. - things that the directive needs to run but aren't arguments.
OK, that makes sense.
FWIW - this code was yanked directly from Anisble. It's a mashup of two files with some small tweaks to make it work. I didn't clean anything and I tried to change as little as possible - so it's going to need tidying up for sure. The goal was just to not spend much time on getting a proof of concept working.
I agree with the points in the review - and if we decide to use this code (which is the code that sparked the GPLv2+ vs GPLv3+ discussion) then I'll want to go through and reformat it to better fit our coding style.
| docs/doc_utils.py | ||
|---|---|---|
| 78 ↗ | (On Diff #350) | From what I could tell these lines broke up utility functions. I concur they don't need to be there. |
| 124 ↗ | (On Diff #350) | That seems sensible to me when we do a refactor. |
| 159 ↗ | (On Diff #350) | Couldn't tell you where they came from, they were there when I yanked the code. |
| 206 ↗ | (On Diff #350) | +1 for the refactor. |
| 243 ↗ | (On Diff #350) | That seems like a good way to go about it. |
| 304 ↗ | (On Diff #350) | It looks that way to me as well. |
| 310 ↗ | (On Diff #350) | Or set a default similar to MODULEDIR. |
| 358 ↗ | (On Diff #350) | Variable names are one thing I wanted to change for readability but I didn't take the time to go through it. Speed was what I was aiming for when copy-pasta-ing this code together. |
But it is not really embedded in docstring, it is an arbitrary variable.
I think you're getting a little nit-picky here. Yes, it is assigned to a variable and we could change it to a proper docstring and use AST parsing to extract it but I can't think of any good reason to actually do that.
I'm not saying it is mutually exclusive, just that if this wall of text will be present at the top of the file, it should be a docstring. If it is just an assignment to an arbitrary variable, it can (IMHO should) be moved to the end of the file.
I'm not sure what your objection is. Is it that the user-facing docs are at the top of the source files instead of things that are more dev-centric? AFAIK, putting documentation at the top of a file is pretty normal - the difference here is in style of docs and intended audience.
In the case of directive documentation, I assert that the user-facing docs are at least as important as dev-facing docs if not more important. The directives are the building blocks which task authors will use to write tasks and if Taskotron is successful, there will be far more people writing tasks than there will be writing directives.
Once again - if the text won't be at the beginning of the file, I do not really care. But IMO enforcing explicitly repetetively writing "This option has no XYZ" does not add much else than noise. The tool parsing the yaml can safely add the "nothing is here" defaults (null, empty list) on its own.
I'm not against the concept as long as we aren't spending too much time on the doc generating code. After a certain point, we might as well have written the thing from scratch ourselves.
Sorry for not responding so long. Here are my thoughts:
- I'm OK with this approach in general. It would be nice, though, if somebody could describe the largest benefits of using this ansible script compared to a standard rst-in-docstring documentation, because I'm not fully clear on that. The one benefit I see here is that we'll be forced to adhere to the specified directive documentation format. Is there anything else? (Please don't take this negatively, it's a well-meant question.)
- Even though I'm not happy about having a lot of non-code at the top of the file, I agree it's better to have it at the top rather than at the bottom.
- I see a benefit of putting it into module docstring instead of some global variables - some IDEs can automatically display it when working with these modules. But in this particular case (the directives are not directly imported and worked with too often), I think the real-usage difference is very small. If variables are much easier to operate with, let's keep using variables.
- I'd like to cut down the length (or better height) of the documentation as much as possible. So if the directive has no requirements, let's not use requirements: None or something like that. Let's just leave out that line completely. I've put more comments about specific lines inline.
- Can I use reST formatting inside directive documentation (the yaml structure)? If not, how do we do some important text formatting (putting something into bold, adding a hyperlink, etc)?
- IIUC the doc_utils.py is taken from Ansible, mostly unaltered. As long as it works OK and there are no serious issues with it, I don't think we need to rewrite/refactor it too much. It's not our code and it's used just for a specific purpose. Let's not waste too much time with it.
- I don't see return value documented inside DOCUMENTATION. That needs to be added, it's one of the most crucial pieces of information.
- I'm a bit confused by the names of the new files - doc_template/, doc_utils.py, DOCUMENTATION.yml. Is this going to be used also for other modules, or just for directives? If the latter is true, why not name it something like directives_template/, generate_directives_docs.py, template.yaml etc?
- I think the generated *.rst files don't need to be added to git.
- I'm getting some warnings when I run make html, e.g.:
/home/kparal/devel/taskotron/libtaskotron/docs/source/list_of_all_modules.rst:4: WARNING: toctree contains reference to nonexisting document u'bodhi_comment_directivec_module'
I'd like to get rid of those warnings, because other warnings might get overlooked if we keep these in. The ideal state is 0 warnings :-)
- Regarding options vs arguments debate - if I'm not mistaken, there are options (--verbose), parameters (--repeat 10) and arguments (FILE). If this is correct, the section should be called "Parameters:" :-)
| libtaskotron/directives/bodhi_comment_directive.py | ||
|---|---|---|
| 92 | This line might not be needed? | |
| 93–95 | I'd like these and similar guidelines to be put either into devguide or into BaseDirective (or both), I'd avoid it in all files for the sake of clarity and being succinct. | |
| 96 | I wonder, could this value be auto-generated? We save one line, and we won't get de-synced if some renaming occurs. | |
| 98–100 | It this concrete example, I'm a bit confused whether this should be a string or a list of strings (as the dashes suggest). The same goes for other sections in a similar style. | |
| 101 | I understand the motivation here, but I'm not sure if it's actually helpful. People should be always reading documentation matching their taskotron version. If they do not, the directives might be accepting different arguments, they might return different output, etc. The sole "version added" information brings too little benefit, in my view. I'd rather save one line. If we decide to keep this in, this should go to the very bottom of the documentation (important information higher than less important). That doesn't affect how it is rendered, but it affects how easily we navigate and read it when we're working with the source code. | |
| 102 | I'd remove this line. It's not important to the reader, and it has the same problems as "author" section in general source code - it's always outdated. git log is better. | |
| 105–108 | Let's move this section below options. | |
| 110 | Again, I'd move this line to devguide or BaseDirective documentation. | |
| 116 | I didn't know that YAML supports also lower-case versions of these keywords. But to keep it simpler, I'd use Python-style True and False. | |
| 120 | The same comment as above. The fact that it was added in 1.0 doesn't mean the parameter value didn't change in 1.1. | |
I'm OK with this approach in general. It would be nice, though, if somebody could describe the largest benefits of using this ansible script compared to a standard rst-in-docstring documentation, because I'm not fully clear on that. The one benefit I see here is that we'll be forced to adhere to the specified directive documentation format. Is there anything else? (Please don't take this negatively, it's a well-meant question.)
I think that the primary benefit is to force a directive format, making all the directive docs look the same. The advantage to having them inside the source file is that we don't need to update docs twice when changes are made and it's harder to forget something that stares you in the face when you make code changes :)
I don't see return value documented inside DOCUMENTATION. That needs to be added, it's one of the most crucial pieces of information.
Good point, that's something that has been on my mind but hasn't really been formalized yet.
I'd like to cut down the length (or better height) of the documentation as much as possible. So if the directive has no requirements, let's not use requirements: None or something like that. Let's just leave out that line completely. I've put more comments about specific lines inline.
Like I said in another comment, I'm not against the idea but I am more interested in getting docs soon than I am in optimizing right now. Whether we do it for the initial docs generation or not will depend on how much work would be required to make that happen.
Regarding options vs arguments debate - if I'm not mistaken, there are options (--verbose), parameters (--repeat 10) and arguments (FILE). If this is correct, the section should be called "Parameters:" :-)
WFM
| docs/doc_template/rst.j2 | ||
|---|---|---|
| 64 ↗ | (On Diff #350) | This will need to be updated |
| libtaskotron/directives/bodhi_comment_directive.py | ||
| 92 | It's a YAML artifact but I'm not clear on when it's needed and when it's not, some experimentation would be helpful here but for now, I'm OK with erring on the side of caution | |
| 93–95 | The only argument I can think of to keep it in every file would be to make it easier for new folks to figure out how to handle non-applicable keys. That being said, it's a pretty weak argument and I'm OK with keeping it in the devguide and/or BaseDirective | |
| 101 | I think the primary use case is to help people who land on the default link for docs instead of version-specific docs. I've found 'version added' very useful when using the Ansible docs but I also think that the lions share of people will be using a version of libtaskotron provided by us, so it's less of an issue. I think that "version added" is sufficient for the directive level. If there are changes/additions to parameters or return types, that can be noted on the return type or parameter docs. I'm OK with moving it farther down the list of information. | |
| 102 | WFM | |
| 116 | I understand where you're coming from but in this case, I think it would be better to stick with YAML's reserved words of 'true', 'false', 'yes' and 'no'. | |
| 120 | I agree that this is not enough information. What do you think about changing it to "Last Changed In:"? | |
| libtaskotron/directives/bodhi_comment_directive.py | ||
|---|---|---|
| 120 | Makes more sense. | |
@roshi answered some questions in T142#11 . I'll reply here.
I have managed to use inline markup inside these fields (bold, hyperlink) easily, but block elements are very tricky. You need to start the string with | first, to avoid further YAML formatting:
- |
code::
bar
foo
.. note:: fooSeparating paragraphs is also tricky, but can be done like this:
description:
- First paragraph.
- ""
- Second paragraph.The biggest problem here (except for remembering the workarounds) is the uncertainty of what input format is accepted and whether something is doable or not. I found out how to input the block elements after several hours. The conversion process goes like this:
YAML -> script that accepts only YAML formatted in a certain way (for example description must be a list, can't be a string) -> html (for certain fields like options) -> rst with some raw html blocks -> html
Because of this, it's hard to estimate how line breaks work, how block elements work, how to escape a quotation mark, etc.
Which means the only places you can't put reST is: the version and options.
The options field is a bit unfortunate, I'd imagine that a different option name highlight, or value formatting (like None) would help.
Also, the EXAMPLES - but that's meant to be code examples, so having reST there doesn't make a ton of sense anyway, IMO.
I imagine that separating an example from its description, and/or having several examples and their descriptions, is quite harder to read when all of it is put into a single preformatted block.
After working with the proposed directive system for a day, I'm not really happy about it. It would probably work satisfactorily in general, but it's way too difficult both in format and tooling and I wonder whether the common template is really worth it, or if a pure rst docstring would simply do. I'll post a simple example of pure rst docstring and the result and you can compare. I'll add the example into T142.
- handled merge conflicts with directives
- fix: apply "Returns" and "Raises" to directive docs
- refactor: cleaned up rst.j2 presentation
- add template above BaseDirective
- template: remove initial dashes
- template: document 'required' bug
- template: synchronize all indents to be 2 spaces
- fix: moved description format to string
- template: request type field for parameters
- template: found new bug in parameter description
- correct template description
- remove options to parameters, blacklist non-used directives temporarily
- rename comments to description
- fix parameter description bug with starting pipe
- fix line breaks in a parameter description
- move description column
- add parameter type column
- remove notes section
- remove requirements section
- pipe is not required when incluing quote mark inside string
- koji directive: try to finalize the documentation
- fix: added error checking for yaml
- escape html chars everywhere in parameter table
- document python_directive
- fix imports from future
- python_directive: move the raise section back
- document dummy_directive
- link errors in docs
- document resultsdb_directive
- python_directive: document how to create TAP
- don't document errors when required parameters are missing
- document createrepo_directive
- document resultsdb_directive
- print default and choice columns properly
- fix: don't render if module is in blacklist
- fix: moved directive docs to different directory
- refactor: renamed doc_utils
- bodhi_directive documented but UNTESTED
- bodhi_comment_directive documented but UNTESTED
- yumrepoinfo_directive documented but UNTESTED
- remove fixme, create ticket T258 instead
- fix a few errors in directive markup
- fix: create required docs directory if needed
- refactor: made code PEP8 compliant
@roshi - Just out of curiosity, is there any reason for not including the changes suggested in comment #11? I'm not trying to be mean, but it seems to be a bit more important than having the (already bad code) being "pep8 compliant".
I'd like naming to be improved a bit provided it's trivial to do so, see my second inline suggestion. In general, I think the patch is fine.
Mike, please don't forget to squash all the commits before you push to develop, otherwise it would be a mess. Either make it a single commit, or split it into two commits - code and documentation - I don't really mind.
| docs/generate_directive_docs.py | ||
|---|---|---|
| 46–50 | The template was already removed from disk, the code should be removed as well. I already pushed this change to the branch, just please don't forget to update it. | |
| docs/source/index.rst | ||
| 30 | I wonder if directives/list_of_directives wouldn't be more readable. The same applies for files named like docs/source/directives/koji_directive_module.rst (koji_directive.rst would be completely sufficient). | |
| libtaskotron/directives/bodhi_directive.py | ||
| 1–3 | For some reason the header got replaced by an older version. I pushed a fix. | |
I'd like to see the changes from comment #11 implemented, but I'm good with doing it in another diff. Created T266 to track that.
Josef implemented reST markup in parameters descriptions, it is available in D163. We can either merge it with this patch and push it together, or push them separately, I have no preference.
I made the pep8 changes as I scrolled through the page - only took a minute or two. I just wanted to clean up at least the giant # lines. Thought I'd do pep8 while I was there as a bonus - it wasn't meant to be a refactor.
I didn't take it as you being mean :) Also - I'm not avoiding making those changes - they should happen. This code will end up getting a solid refactor (though it's already been through a lot of changes) somewhere down the line. I was just focused on getting the minimum done we needed so that we could get the release pushed out. It works as is, and we need to finish the release.
This was pushed as 94b835cfb3.
Mike, please always keep the differential link inside the commit message, thanks.
The template was already removed from disk, the code should be removed as well. I already pushed this change to the branch, just please don't forget to update it.