= Language-puppet

image:https://travis-ci.org/bartavelle/language-puppet.svg?branch=master["Build Status", link="https://travis-ci.org/bartavelle/language-puppet"]

A library to work with Puppet manifests, test them and eventually replace everything ruby.

.Basic usage:
puppetresources -p /where/your/puppet/files/are -o node.name.com

.Easy build instructions:
git clone https://github.com/bartavelle/language-puppet.git
cd language-puppet
# Add ~/.local/bin to $PATH
stack install

https://hub.docker.com/r/pierrer/language-puppet/[A docker image] is available.

== Puppetresources

The `puppetresources` command is a command line utility that let you interactively compute catalogs on your local computer.
It is much faster than its ruby counterpart, and has been designed for giving assistance to the Puppet catalog writer.
There are 4 different modes:

* `--node` will display all resources on screen in a nice user-friendly colored fashion.
* `--all` displays statitics and optionally shows dead code.
* `--parse` only goes as far as parsing. No interpretation.
* `--showcontent` to display file content.

Catalog can be verified using strict or more permissive rules.

=== Command line arguments

`-p` or `--puppetdir`::

This argument is mandatory except in `parse` mode. It must point to the base of the puppet directory (the directory that contains the `modules` and `manifests` directories).

`-o` or `--node`::

Enable the `node mode`. This let you specify the name of the node you wish to compute the catalog for.

`-a` or `-all`::

Enable the `stats mode`. If you specify `allnodes` it will compute the catalogs for all nodes that are specified in `site.pp` (this will not work for regexp-specified or the default nodes). You can also specify a list of nodes separated by a comma.
Combined with `--deadcode`, it will display the list of puppet files that have not been used.
This is useful as automated tests, to check a change didn't break something. You might want to run this option with `+RTS -N`.

`-t` or `--type`::

Filters the resources of the resulting catalog by type. Using PCRE regex is supported.

`-n` or `--name`::

Filters the resources of the resulting catalog by name. Using PCRE regex is supported.

`-c` or `--showcontent`::

If `-n` is the exact name of a file type resource defined in the catalog, this will display the file content nicely. Useful for debugging templates.
Example: `puppetresources -p . -o mynodename -n '/etc/motd' --showcontent`

`--loglevel` or `-v`::

Possible values are : DEBUG, INFO, NOTICE, WARNING, ERROR


Expects the url of a live PuppetDB.


Expects a path to a *fake* PuppetDB, represented as a YAML file on disk. This option is pretty slow but can be invaluable to test exported resources tricks.


Expects the path to the `hiera.yaml` file.


Expects a list of comma-separated modules. The interpreter will not try to evaluate the defined types and classes from this module. This is useful for using modules that use bad
practices forbidden by `puppetresources`.


When this flag is set, exported resources, catalogs and facts are saved in the PuppetDB. This is useful in conjunction with `--pdbfile`.


When this flag is set, exported resources are saved in the PuppetDB. This is useful in conjunction with `--pdbfile`.

`-j` or `--JSON`::

Displays the catalog as a Puppet-compatible JSON file, that can then be used with `puppet apply`.


Enable strict check.
Strict is less permissive than vanilla Puppet.
It is meant to prevent some pitfalls by enforcing good practices.
For instance it refuses to
- silently ignore/convert `undef` variables
- lookup an hash with an unknown key and return `undef`.


Disable the extra tests from `Puppet.OptionalTests`.


Enable `parse mode`. Specify the puppet file to be parsed. Variables are not resolved. No interpretation.


Output version information and exist.

=== Settings defaults using a yaml file

Defaults for some of these options can be set using a `/yourworkingdirectory/tests/defaults.yaml` file. For instance `OptionalTests` is checking that all users and groups are known. Because some of these users and groups might be defined outside puppet, a list of known ones is used internally. This can be overridden in that file using the key `knownusers` and `knowngroups`.

Please look at https://github.com/bartavelle/language-puppet/blob/master/tests/defaults.yaml[the template file] for a list of possible defaults.

== pdbQuery

The `pdbQuery` command is used to work with different implementations of PuppetDB (the official one with its HTTP API, the file-based backend and dummy ones). Its main use is to
export data from production PuppetDB to a file in order to debug some issue with `puppetresources`. Here is a list of command line arguments :

`-l` or `--location`::

The URL of the PuppetDB when working with a remote PuppetDB, a file path when working with the file-based test implementation.

`-t` or `--pdbtype`::

The type of PuppetDB to work with:

* dummy: a dummy PuppetDB.
* remote: a "real" PuppetDB, accessed by its HTTP API.
* test: a file-based backend emulating a PuppetDB.


Dump all facts, and store them in `/tmp/allfacts.yaml`.


Dump all nodes


Create a test DB from the current DB


Adds facts to the test DB for the given node name, if they are not already defined.


Output version information and exit.

== Unsupported Puppet idioms or features

puppet functions::
* the `require` function is not supported (see https://github.com/bartavelle/language-puppet/issues/17[issue #17])
* the deprecated `import` function is not supported (see https://github.com/bartavelle/language-puppet/issues/82[issue #82])

custom ruby functions::


v1.3.2 ()

  • Add stdlib validate_integer

v1.3.1 (2016/08/31)

  • Add --version to puppetresources and pdbquery
  • Version bump for some dependencies

v1.3 (2016/07/11)

This release is about dependencies versions upgrades.

Removed features

  • External Lua plugins are not a thing anymore

#v1.2 (2016/05/31)

New features

  • validate_numeric function
  • Preliminary implementation of variable captures in regexp matches
  • Ready for GHC 8 and corresponding stackage nightly

Bugs fixed

  • Fixed a deadlock when template code called the template or inline_template functions. It just stops now :(

# v1.1.5.1 (2016/03/14)

## New features

  • assert_private function
  • join_keys_to_values function

## Bugs Fixed

  • Several resource collector misbehaviour
  • Case expressions can now have multiple matchers of any kind as the same selector

v1.1.5 (2016/02/02)

New features

  • Added the pick_default function
  • merge now works with an arbitrary number of hashes
  • Added the hash function
  • puppet native type file resource accept selinux parameters
  • Added the shellquote function

Bugs fixed

  • create_resources can now create virtual and exported resources
  • puppet native type file fix for parameters sourceselect and recurselimit
  • Hiera array merge now only keeps unique values
  • merge now properly priorizes the lastest arguments

v1.1.4.1 (2015/11/15)

New features

  • Support for the new puppet contain function

Bugs fixed

  • Fix parser for search expression (see #132)
  • Fix logger set up (see #136)
  • Fix some regsubst (see #119)
  • Fix the template and inline_template functions (see #142)
  • Support lookups for expressions used in selector (TODO: arbitrary expression)
  • Fix a ruby 1.8 syntax error
  • Fix a case where resource overrides were not applied to the correct resource
  • Fix a bug where a class value would be incorrectly inferred as having a default value whereas it doesn’t


  • Replace parsec megaparsec (see #138)
  • All resource names are normalized. The leading :: is ignored (see #140)
  • Add CI using travis
  • Drop support for ghc < 7.10 explicitly

v1.1.4 (2015/09/07)

New features

  • The regsubst function now works with arrays.
  • The file variable is resolved in templates.
  • Support for function_x calls in templates.

Bugs fixed

  • Expressions such as (-1) are now supported.
  • Selectors recognize the undef token now.
  • Fixed a bug with parsing lines starting with ::.
  • Sanitize resource names in some missing instances to fix bugs when they were starting with ::.

v1.1.3 (2015/05/31)

New features

  • Support for the $settings variables.
  • Support for the to_yaml function in templates.
  • Settings can now be altered in the default YAML file.
  • Defaults and overriden facts are now controlled in the YAML file too.

v1.1.1.2 (2015/04/28)

Various packaging changes.

v1.1.1 (2015/04/20)

New features

  • Add ‘notify’ native type
  • Ability to provide defaults via a yaml file for some options
  • Added the ‘ensure_packages’ and ‘ensure_resources’ functions

Bugs fixed

  • Enable ‘package’ native type (issue #102)
  • Builds against GHC 7.10


  • Even in Permissive mode, don’t resolve unknown variable (see #103)
  • Add priority to the logger permissive output (see #106)
  • New hruby version
  • Rename option --ignoremodules into --ignoredmodules


  • Hiera config interpolation logs decrease from NOTICE to INFO

v1.1.0 (2015/03/11)

Critical bugs have been fixed, upgrade recommended.

New features

  • New dumpinfos debug function.
  • The interpreter can now run in a strict or permissive mode.
  • The new -a option accepts a comma separated list of nodes for gathering stats.
  • The new --noextratests option disable optional tests from Puppet.OptionalTests.
  • Implementation of member() from stdlib (see issue #100 for details)

Bugs fixed

  • Exported/virtual custom types are not expanded. This is a huge bug.
  • Class/define parameters that are explicitely set as undefined are now overriden by default values.
  • Empty resource groups are now rejected.
  • An existing resource can now be realized.


  • Hiera config interpolation logs decrease from WARN to NOTICE
  • Remove option --nousergrouptest
  • Ease the use of the puppetresources command options. See the README file for changes.

v1.0.1 (2014/11/13)

New features

  • Support for the join function.
  • Support for filtering json puppetresources output (fix issue #64)
  • Support for cmpversion in the templates.
  • The various chaining modes have been implemented.
  • Support for the is_bool function (Pierre Radermecker)
  • Support for concat and concat::fragment (Pierre Radermecker)

Bugs fixed

  • Fix array value extrapolation in string (issue #35)
  • ${var} without quotes will now be rejected by the parser (issue #78)


  • README moved to asciidoc (Pierre Radermecker)

v1.0.0 (2014/08/31)


Building without hruby is now unsupported.

New features

  • Support for Debian distribution detection in facter.
  • Support for the “~>” operator.
  • Support for mixed-case resource references.
  • Added the grep function.

Bugs fixed

  • Better support for –ignoremodules.
  • Fixed parsing of standalone $ characters in strings.

v0.14.0 (2014/06/12)

New features

  • Overhauled the dependency check system
  • Added an option to skip the user and group checks
  • Added an option to ignore some modules

Bugs fixed

  • Added vagrant, nagios, www-data, postgres and nginx to the list of known users.
  • Fixed how resource relationships were resolved with notify and before.
  • Fixed a problem where inheritance whould be used with :: prefix.
  • The defined function now works with classes.
  • All numbers are now strings in templates.

v0.13.0 (2014/05/21)

New features

  • Hacky support for scope.get_hash.
  • New stuff from the new parser (adding hashes, arrays, etc.).
  • Wrote a pure evaluation function, for unit tests and prisms.
  • Num and Fractional instances for Expression.
  • Numbers are now internally stored as numbers, just like the new parser does.
  • Add support for “structured facts”.
  • New stdlib functions: is_hash, has_key, size, values.

Bugs fixed

  • Puppetresources does not fail tests for file sources starting with file://.
  • Escaped characters were not properly handled in the parser.
  • Properly catch division by 0 (!!!!).
  • Got rid of the orphan instances … code is now a lot uglier.
  • Fixed settings of “title” and “name” in classes. The original puppet version only seems to do this when we declare in define style :(
  • Fixed associativity priority between =~ and and.

v0.12.3 (2014/03/13)

New featues

  • puppetresources now tests that groups and users are defined before being used in file, user, cron and exec.

v0.12.2 (2014/02/18)

New featues

  • Facts are now dumped in TestDB format by pdbquery.
  • The puppetresources command now has switch controlling the PuppetDB commit and “catalog update”.

v0.12.1 (2014/02/10)

New featues

  • Dead code finder in puppetresources.
  • CPU related facts.
  • puppetresources now exits with the proper error code.
  • puppetresources can now display some statistics about compilation times.
  • Bumped the version of the http-conduit dependency.

Bugs fixed

  • Fixed dependencies so that builds with GHC 7.8-rc1 work.

v0.12.0 (2014/02/07)

New featues

  • Builds against GHC 7.8-rc1.

v0.11.1 (2014/01/31)

Bugs fixed

  • Fixed build issues with strict-base-types version.

v0.11.0 (2014/01/30)

New features

  • Removal of the dedicated parsing threads.
  • Better default RTS options (for now, just the default allocation size)
  • Upgraded dependencies : aeson 0.7, attoparsec 0.11, lens 4, parsers 0.10, text 1.1, filecache 0.3, hruby 0.2

v0.10.6 (2014/01/25)

New features

  • New all nodes testing for puppetresources.
  • Added some uname related facts.
  • Added some lenses and prisms.

Bugs fixed

  • Parsing function calls without parens at the expression level is not allowed now.
  • Allow parsing of boolean facts from YAML files.
  • Allow resource references with array variables.
  • Fix spurious multiple includes error.
  • Fixed the implementation of some puppet functions.

v0.10.5 (2014/01/06)

Bugs fixed

  • Lambda blocks can now end with a bare function call
  • Fix version bounds with hslua and luautils

v0.10.4 (2013/12/18)

New features

  • Moved to the latest hruby version.
  • Updated the text bound

v0.10.3 (2013/12/03)

New features

  • The scope tracking system has been improved. It is now possible to know the original host of an imported resource, which helps a lot in case of resource clashes
  • is_virtual fact
  • new stdlib functions: flatten, str2bool, validate_absolute_path
  • Hiera support
  • JSON output that is compatible with “puppet apply”
  • New addfacts command for the pdbquery utility
  • Support for the classes variable in templates
  • Support for @instance variables in inline_template
  • Support for scope[‘key’] syntax in templates
  • Support for facts overriding with puppetresources

Bugs fixed

  • Deserialization problems with puppetDBs
  • Fixed several bugs with imported resources
  • Bug with relationships overrides that got stored as parameters
  • Importing several exported resources from the same class now works
  • Templates with an invalid encoding could crash the process
  • Yaml parse errors of the puppetdb file now throw errors

v0.10.2 (2013/10/27)

Bugs fixed

  • PVP support

New features

  • Support for properly setting instance variables before computing templates with native Ruby.

v0.10.1 (2013/10/27)

Bugs fixed

  • The TestDB file was never created.

v0.10.0 (2013/10/27)

New features

  • Map/each/filter functions with lambdas (not really tested)
  • Rewrite of the PuppetDB API
  • The whole scope stack is kept with each Resource, for easier debugging
  • Inclusion of three PuppetDB backends : dummy (no effect), TestDB (stored in a YAML file) and Remote (standard PuppetDB API)

Bugs fixed

  • This is a hack : variables declared in a parent (inheritance) can now be overriden. This is because inheritance is not handled like in Vanilla. As I do not really use inheritance, I am not sure if this is much of a breakage.

v0.9.0 (2013/08/15)

Huge rewrite !

v0.4.2 (2013/06/01)

New features

  • Functions ‘values’ and ‘keys’ from stdlib are now implemented.
  • hruby integration

Bugs fixed

v0.4.0 (2013/05/16)

New features

  • Big refactor of the PuppetDB API.
  • New “fake” PuppetDB used for testing
  • Support of the caller_module_name variable.
  • Support for a dumpvariable() function.
  • More details stored in the resource types, and in error messages.
  • User native type
  • Removal of the MissingH, filepath, monad-loop and directory dependency
  • Puppet booleans are now handled at parse stage
  • inline_template function

Bugs fixed

  • fqdn_rand now puppet perfect (at least for 32 bit max values)
  • Now depends on the built-in bytestring library that comes with GHC-7.6.1.
  • Aliases should now work as expected … I wish!
  • regexp_subs now works in a PCRE manner
  • Destination dependency can now be a variable resolving in an array.

v0.3.3 (2013/01/21)

New features

  • Tries to find calcerb.rb next to the executable.
  • Started cleaning imports …
  • It is now possible to write “top level” functions in lua.
  • Function getvar (stdlib)
  • TENTATIVE support for aliases.
  • Checks that file names don’t have trailing slashes.
  • Checks that exec commands are fully qualified if path is not defined.
  • New native type : package.

Bugs fixed

  • Fixed a ton of problems related to exported resources and relations.
  • Minor fix about zonerecord.
  • Resolving variable names starting with :: in templates
  • Fixed the file function.

v0.3.2 (2012/12/13)

The license has been changed to BSD3.

Bugs fixed

  • It is now possible to use expressions in include blocks. This is temporary, as include should be handled just like every other function.

v0.3.1 (2012/11/23)

New features

  • Yes, we can generate JSon catalogs now.

Bugs fixed

  • Several bugs about resource relationships.

v0.3.0 (2012/11/19)

New features

  • Resource relationships are somehow supported. The API is broken as a result.
  • Exported resources are now returned.

v0.2.2 (2012/11/12)

New features

  • A few statistics are exported.

v0.2.1 (2012/11/12)

Bugs fixed

  • The defaults system was pretty much broken, it should be better now.

New features

  • Basic testing framework started.
  • create_resources now supports the defaults system.
  • defined() function works for resource references.
  • in operator implemented for hashes.
  • Multithreading works.
  • The ruby <> daemon communication is now over ByteStrings.
  • The toRuby function has been optimized, doubling the overall speed for rendering complex catalogs.
  • Various internal changes.

v0.2.0 (2012/10/08)

New features

  • Lua integration for custom functions.
  • Automatically creates magic types based on the content of the modules.

Bugs fixed

  • Defaults parameters can now end with a comma.

v0.1.8.0 (2012/09/20)

New features

  • Refactoring of the PuppetDB API for interfacing with the facter library.
  • Support of exported resource resolution through PuppetDB ! This results in an API breakage.
  • Make binary distribution possible (ruby helper path).

Bugs fixed

  • Defines with spurious parameters, or unset mandatory parameters, should now be catched.
  • Exception handling for the HTTP failures.
  • Handles undefined variables in Ruby templates.
  • Undefined variables in Erbs now always throw exceptions. This is stricter than Puppet (which throws exceptions for “native” variables), but is I believe good practice.

v0.1.7.2 (2012/09/17)

New features

  • Preliminary support for PuppetDB

v0.1.7.1 (2012/09/14)

Bugs fixed

  • Various details have been modified since the official language documentation has been published.
  • Better handling of collector conditions.
  • Solves bug with interpolable strings that are not resolved when first found.

New features

  • Amending attributes with a collector.
  • Stdlib functions : chomp
  • Resource pretty printer now aligns =>.
  • Case statements with regexps.

v0.1.7 (2012/08/24)

Bugs fixed

  • Fix bug with ‘<’ in the Erb parser !
  • Assignments can now be any valid Puppet expression.
  • Proper list of metaparameters.

New features

  • Quick resolution of boolean conditions.
  • Start of the move to a real PCRE library.
  • Function is_domain_name.
  • New native types : zone_record, cron, exec, group, host, mount, file.

v0.1.6 (2012/08/01)

New features

  • Errors now print a stack trace (only works with profiling builds).
  • Nested classes.
  • generate() function.
  • defines with spurious top level statements now should work.
  • validate_* functions from puppetlabs/stdlib.

Bugs fixed

  • Metaparameters now include stages (not handled).
  • Resolving non empty arrays as boolean returns true.
  • Duplicate parameters are now detected.

v0.1.5 (2012/07/06)

Bugs fixed

  • Detection of spurious parameters when declaring parametrized classes now works.
  • Resource overrides with non trivial names should now work.
  • Require statements in required files would not be loaded.

v0.1.4 (2012/07/02)

New features

  • Basic native template function.
  • Added anchor as a native type for now. A better fix will be to just parse for defined types in the lib directory of modules.
  • Tentative defined() implementation. Will not work for resource references.
  • Functions md5, sha1, lowcase, upcase, split.

Bugs fixed

  • String comparison is not case insensitive.
  • Variable scope for inherited classes should now work.
  • Support for the $module_name variable (probably a bit buggy).
  • Proper location of a “define not found” error.
  • Parsing bug for single quoted strings and slashes.
  • Bug where a resource name is a variable that is actually an array.
  • Array indexing.
  • Top level variables are now supported in Erb.


  • Removed the title parameter from the catalog printing functions.
  • Used hslint a bit.