summaryrefslogtreecommitdiffstats
path: root/lib/puppet/parser/lexer.rb
Commit message (Collapse)AuthorAgeFilesLines
* Fix #2364 - Associates the correct comment to the right statementBrice Figureau2009-07-101-8/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Due to the problem that we associate documentation in the lexer and not in the parser (which would be to complex and unmaintenable to do), and since the parser reads new tokens before reducing the current statement (thus creating the AST node), we could sometimes associate comments seen after a statement associated to this one. Ex: 1. $foo = 1 2. # doc of next class 3. class test { When we parse the first line, the parser can reduce this to the correct VarDef only after it lexed the CLASS token. But lexing this token means we already pushed on the comment stack the "doc of next class" comment. That means at the time we create the AST VarDef node, the parser thinks it should associate this documentation to it, which is incorrect. As soon as the parser uses token line number, we can enhance the lexer to allow comments to be associated to current AST node only if the statement line number is greater or equal than the last comment line number. This way it is impossible to associate a comment appearing later in the source than a previous statement. Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
* Make sure the parser sees the correct line numberBrice Figureau2009-07-101-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Careful inspection of the parser code show that when we associate a source line number for an AST node, we use the current line number of the currently lexed token. In many case, this is correct, but there are some cases where this is incorrect. Unfortunately due to how LALR parser works the ast node creation of a statement can appear _after_ we lexed another token after the current statement: 1. $foo = 1 2. 3. class test When the parser asks for the class token, it can reduce the assignement statement into the AST VarDef node, because no other grammar rule match. Unfortunately we already lexed the class token so we affect to the VarDef node the line number 3 instead of 1. This is not a real issue for error reporting, but becomes a real concern when we associate documentation comments to AST node for puppetdoc. The solution is to enhance the tokens lexed and returned to the parser to carry their declaration line number. Thus a token value becomes a hash: { :value => tokenvalue, :line } Next, each time we create an AST node, we use the line number of the correct token (ie the foo line number in the previous example). Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
* Fix #2333 - Make sure lexer skip whitespace on non-tokenBrice Figureau2009-06-121-1/+4
| | | | | | | | | Comments and multi-line comments produces no token per-se during lexing, so the lexer loops to find another token. The issue was that we were not skipping whitespace after finding such non-token. Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
* Removed extra whitespace from end of linesIan Taylor2009-06-061-3/+3
|
* Fixing #1755 - handling fully qualified classes correctly.Luke Kanies2008-11-261-1/+1
| | | | | | | This involves lexing '::class' tokens along with correctly looking them up from the Resource::Reference class. Signed-off-by: Luke Kanies <luke@madstop.com>
* Add a doc attribute to AST nodes and fill it with the last seen commentsBrice Figureau2008-11-171-8/+52
| | | | | | | | | | | | | The lexer maintains a stack of last seen comments. On blank lines the lexer flush the comments. On each opening brace the lexer enters a new stack level. On each block AST nodes, the stack is popped. Each AST nodes has a doc property that is filled with the last seen comments on node creation (in fact only on important node creation representing statements). Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
* Fix #1402 - Allow multiline commentsBrice Figureau2008-10-291-0/+5
| | | | Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
* Fixing #1098 - Multiline strings now correctly increment the line countLuke Kanies2008-10-041-0/+3
| | | | Signed-off-by: Luke Kanies <luke@madstop.com>
* Add parser for arbitrary expressionsBrice Figureau2008-10-011-4/+10
| | | | | | | | | | | | | | | | The expressions can be used in if 'test' and in the right side of assignements. The expressions can contain any number of sub-expressions combined by either arithmetic operators, comparison operators, or boolean operators. Random Usage Examples: $result = ((( $two + 2) / $one) + 4 * 5.45) - (6 << 7) + (0x800 + -9) or if ($a < 10) and ($a + 10 != 200) { ... }
* Add an append (+=) variable operator:Brice Figureau2008-09-211-0/+1
| | | | | | | | | | | | | | | | | | | The append variable operator can be used to append something to a variable defined in a parent scope, containing either a string or an array. The main use is to append array elements in classes to a variable globally defined in a node. Example: $ssh_users = ['brice', 'admin1'] class backup { $ssh_users += ['backup_operator'] ... } Signed-off-by: Brice Figureau <brice-puppet@daysofwonder.com>
* Fixes #1274 - allow class names to start with numbersJames Turnbull2008-08-171-1/+1
|
* Fixed #968 again, this time with tests -- parseonly works,Luke Kanies2008-02-131-1/+5
| | | | | including not compiling the configurations, and also storeconfigs is no longer required during parse-testing.
* Significantly refactoring the lexer, including adding Token and TokenListLuke Kanies2008-01-301-270/+391
| | | | | | classes for managing how the tokens work. I also moved they tests to RSpec, but I didn't rewrite all of them.
* Removing tons of unnecessary calls to "nil?" from the lexer.Luke Kanies2008-01-281-22/+6
|
* Removing the Id tags from all of the filesLuke Kanies2007-10-031-1/+0
|
* Removed TYPE token, replacing it with CLASSREF token, in the grammar and ↵Michael V. O'Brien2007-09-101-2/+1
| | | | lexer. Updated CLASSREF token regex in the lexer.
* Adding patch by Valentin Vidic to add the "+>" syntax for adding values to ↵luke2007-07-101-0/+1
| | | | | | parameters git-svn-id: https://reductivelabs.com/svn/puppet/trunk@2670 980ebf18-57e1-0310-9a29-db15c13687c0
* Adding better error reporting on unmatched brackets -- you will now get ↵luke2007-05-251-0/+32
| | | | | | notification of what was expected in most cases git-svn-id: https://reductivelabs.com/svn/puppet/trunk@2531 980ebf18-57e1-0310-9a29-db15c13687c0
* Adding #629 -- an undef keyword now existsluke2007-05-171-1/+2
| | | | git-svn-id: https://reductivelabs.com/svn/puppet/trunk@2522 980ebf18-57e1-0310-9a29-db15c13687c0
* Adding the functionality requested in ↵luke2007-04-191-1/+1
| | | | | | | | | | http://mail.madstop.com/pipermail/puppet-users/2007-April/002398.html . You can now retrieve qualified variables by specifying the full class path. git-svn-id: https://reductivelabs.com/svn/puppet/trunk@2393 980ebf18-57e1-0310-9a29-db15c13687c0
* Fixing #343. Collections and definition evaluation both now happen on every ↵luke2006-12-231-3/+5
| | | | | | iterative evaluation, with collections being evaluated first. This way collections can find resources that either are inside defined types or are the types themselves. git-svn-id: https://reductivelabs.com/svn/puppet/trunk@1967 980ebf18-57e1-0310-9a29-db15c13687c0
* Merging the changes from the override-refactor branch. This is a ↵luke2006-10-041-17/+79
| | | | | | significant rewrite of the parser, but it has little affect on the rest of the code tree. git-svn-id: https://reductivelabs.com/svn/puppet/trunk@1726 980ebf18-57e1-0310-9a29-db15c13687c0
* Reverting the work done in [1605] and [1606]. I have added it as a patch in ↵luke2006-09-151-0/+1
| | | | | | #271. git-svn-id: https://reductivelabs.com/svn/puppet/trunk@1610 980ebf18-57e1-0310-9a29-db15c13687c0
* This commit adds two important features (but which probably were notluke2006-09-151-1/+0
| | | | | | | | | | | | | | | | | | | | | | | worth the priority I suddenly placed on them). First, it adds search paths as I originally requested in #114. There is now a 'lib' setting, which can be used to tell Puppet where to find manifests. Any file you tell Puppet to parse will have its directory automatically added to the lib path. Also, Puppet will check the PUPPETLIB environment variable for further directories to search. Second, it converts the 'import' mechanism into a normal function, which means that you can now use variables and what-have-you in it. Of course, this function uses the lib mechanism. This is something that's always bothered me about the language, and having it fixed means you can do simple things like have custom code in the top scope for each operating system and then do "import os/$operatingsystem" to evaluate that code. Without this, you would either need a huge case statement or the code would need to be in a class, which often isn't sufficient. git-svn-id: https://reductivelabs.com/svn/puppet/trunk@1605 980ebf18-57e1-0310-9a29-db15c13687c0
* Adding "if/else" constructs. No operators, no elsif, but it is a good ↵luke2006-08-221-0/+3
| | | | | | start, anyway. git-svn-id: https://reductivelabs.com/svn/puppet/trunk@1483 980ebf18-57e1-0310-9a29-db15c13687c0
* Fixing #166. Function names are no longer reserved words.luke2006-06-081-3/+0
| | | | git-svn-id: https://reductivelabs.com/svn/puppet/trunk@1245 980ebf18-57e1-0310-9a29-db15c13687c0
* I appear to have object collection working, incredibly. This commit does ↵luke2006-05-131-1/+1
| | | | | | the collection from the database up to adding the objects to the current scope, which is what sends it to the client. git-svn-id: https://reductivelabs.com/svn/puppet/trunk@1190 980ebf18-57e1-0310-9a29-db15c13687c0
* The "collectable" syntax now works end-to-end -- the parser correctly ↵luke2006-05-131-0/+3
| | | | | | recognizes it, the AST objects retain the settings, the scopes do the right conversion, the interpreter stores them all in the database, and then it strips the collectable objects out before sending the object list to the client git-svn-id: https://reductivelabs.com/svn/puppet/trunk@1189 980ebf18-57e1-0310-9a29-db15c13687c0
* Creating a simplistic, generic function framework in the parser, so it is ↵luke2006-04-261-2/+3
| | | | | | now very easy to add new functions. There is a pretty crappy, hardwired distinction between functions that return values and those that do not, but I do not see a good way around it right now. Functions are also currently responsible for handling their own arity, although I have plans for fixing that. git-svn-id: https://reductivelabs.com/svn/puppet/trunk@1134 980ebf18-57e1-0310-9a29-db15c13687c0
* Changing "set" to "tag"luke2006-04-111-1/+1
| | | | git-svn-id: https://reductivelabs.com/svn/puppet/trunk@1106 980ebf18-57e1-0310-9a29-db15c13687c0
* Fixing the language side of #109. Added a "set" keyword.luke2006-04-111-0/+1
| | | | git-svn-id: https://reductivelabs.com/svn/puppet/trunk@1103 980ebf18-57e1-0310-9a29-db15c13687c0
* Change how names for nodes are specified: the 'node' keyword can be followed ↵lutter2006-04-041-1/+0
| | | | | | by a NAME or by single quoted text, i.e. fully qualified names for nodes must be enclosed in single quotes git-svn-id: https://reductivelabs.com/svn/puppet/trunk@1064 980ebf18-57e1-0310-9a29-db15c13687c0
* Manifests can now specify node names with fully qualified domain names, too.lutter2006-03-311-2/+3
| | | | git-svn-id: https://reductivelabs.com/svn/puppet/trunk@1047 980ebf18-57e1-0310-9a29-db15c13687c0
* Adding single-quote syntactical elementluke2006-02-161-4/+11
| | | | git-svn-id: https://reductivelabs.com/svn/puppet/trunk@923 980ebf18-57e1-0310-9a29-db15c13687c0
* removing "host" as a keyword; it was an alias for "node"luke2006-01-031-1/+0
| | | | git-svn-id: https://reductivelabs.com/svn/puppet/trunk@760 980ebf18-57e1-0310-9a29-db15c13687c0
* removing ruby shebang at the top of all of the filesluke2005-11-231-3/+2
| | | | git-svn-id: https://reductivelabs.com/svn/puppet/trunk@755 980ebf18-57e1-0310-9a29-db15c13687c0
* The new "include" syntax works now.luke2005-09-221-6/+7
| | | | git-svn-id: https://reductivelabs.com/svn/puppet/trunk@700 980ebf18-57e1-0310-9a29-db15c13687c0
* Successfully parsed my entire converted cfengine configuration; these are ↵luke2005-09-171-2/+2
| | | | | | all fixes for bugs i found as a result. I have not tried to execute the configuration yet. git-svn-id: https://reductivelabs.com/svn/puppet/trunk@687 980ebf18-57e1-0310-9a29-db15c13687c0
* tracked down some sticky bugs related to having false values and empty ↵luke2005-09-131-3/+4
| | | | | | strings; all fixed now, and all tests pass again, including the new tests that cover the bugs i found git-svn-id: https://reductivelabs.com/svn/puppet/trunk@652 980ebf18-57e1-0310-9a29-db15c13687c0
* removing if-related keywordsluke2005-09-021-3/+0
| | | | git-svn-id: https://reductivelabs.com/svn/puppet/trunk@618 980ebf18-57e1-0310-9a29-db15c13687c0
* replacing if statements with case statement, and adding defaults for both ↵Luke Kanies2005-08-241-0/+2
| | | | | | selectors and case statements git-svn-id: https://reductivelabs.com/svn/puppet/trunk@587 980ebf18-57e1-0310-9a29-db15c13687c0
* Moving all files into a consolidated trunk. All tests pass except the ↵Luke Kanies2005-08-231-0/+225
known-failing certificate test, but there appear to be some errors that are incorrectly not resulting in failurs. I will track those down ASAP. git-svn-id: https://reductivelabs.com/svn/puppet/trunk@576 980ebf18-57e1-0310-9a29-db15c13687c0
generated by cgit (git 2.34.1) at 2025-09-27 15:40:02 +0000