#puppet-mgmtgraph

Adds the :mgmtgraph face and puppet mgmtgraph subcommand to Puppet. It allows you to compile simple Puppet manifest into a data structure that is suitable for mgmt to consume.

Released under the terms of the Apache 2 License.

Authored by Felix Frank.

Installation

You need Puppet on the system that is supposed to translate code for mgmt. It is directly available in all major Linux distributions as a package. Otherwise, a generic way to get Puppet (applicable to Mac and Windows as well) is the Ruby gem:

gem install puppet

Once Puppet is functional, you can get this module from the Puppet Forge:

puppet module install ffrank-mgmtgraph

Note: You don't need root rights to do this. However, mgmt will likely run as root, and Puppet modules that were installed by non-root users are only available for the respective user.

Usage

It is no longer necessary to invoke puppet mgmtgraph directly, since it's possible to use mgmt's --puppet switch to load the resulting graph directly.

Still, for testing and debugging, the tool is still useful. It can also make sense to save a translated YAML graph to a file in order to cache it. After all, building the catalog is usually the most time consuming part when running mgmt from a Puppet manifest.

Manual invocation

Currently, the most useful invocation of puppet mgmtgraph targets single manifests of simple structure

puppet mgmtgraph --manifest /path/to/my.pp >/tmp/mygraph.yaml

The manifest can use modules from the configured environment, but please note that this likely clashes with current limitations.

With no manifest specified, puppet mgmtgraph will behave like puppet agent and receive the catalog from the configured master, using its agent certificate. (This works courtesy of the puppet catalog face.)

puppet mgmtgraph >/tmp/mygraph.yaml

A handy shortcut for testing simple manifests is the --code parameter

puppet mgmtgraph --code 'file { "/tmp/test": ensure => present } -> package { "cowsay": ensure => installed }'

Finally, run the graph through mgmt

mgmt run --file /tmp/mygraph.yaml

Conservative mode

In its default ("optimistic") mode, puppet mgmtgraph will emit as many native mgmt resources as possible. This will drop some attribute values to the floor, however. Consider the following simple manifest:

file { "/tmp/exchange_file": ensure => file, seltype => "tmp_t" }

As long as mgmt has no SELinux support, it can create and maintain the file, but will ignore its SEL context. In a more complex manifest context, this is likely inadequate to prepare the system for the synchronization of all dependent resources.

In order to make sure that mgmt applies such a catalog correctly, it has to resort to the puppet resource workaround that is used for resources that are not supported by mgmt at all (see below). This behavior is now available in the form of the conservative mode:

puppet mgmtgraph --conservative --code 'file { "/tmp/exchange_file": ensure => file, seltype => "tmp_t" }'

The mgmt integration has no way of passing this flag. However, eventually this mode will probably become the default, with an optional --optimistic flag to revert to the current default.

Collecting translation problems

In order to make the translation more effective, we are looking to add the most popular features from Puppet to mgmt. You can help by opening issues upstream at https://github.com/purpleidea/mgmt. For a handy list of errors from your manifests, use the following command:

puppet mgmtgraph stats

The stats command can be used just like print with respect to its parameters. However, instead of printing the graph structure for mgmt, it presents a consolidated list of translation issues.

This will help you determine which params that aren't yet natively supported in mgmt might be good candidates that you could send a patch for.

Example

Running:

puppet mgmtgraph stats --code 'file { "/tmp/test": ensure => present, seltype => "tmp_t" } -> package { "cowsay": ensure => installed }'

Will produce output that includes:

1x File[...] cannot translate attribute 'seltype', attribute is ignored

Since there is no equivalent for the file param named seltype in mgmt yet.

Limitations

Translation of virtual and exported resources is untested. Containment of supported resources in classes and defined types should work.

The set of supported catalog elements is still quite small:

• augeas
• ec2_instance
• exec
• file
• group
• mount
• notify
• package
• service
• user

For most of these, mgmt does not support all available properties and parameters. Whenever an attribute is ignored because of that, a warning message is printed during translation. There might be edge cases where this does not work reliably.

Resources of unsupported types are rendered into pseudo-resources of the special pippet type (a portmanteau of "pipe" and "puppet"). When mgmt processes a graph containing these nodes, it will hand them over to Puppet for synchronizing them.

When working with older versions of mgmt before 0.0.21, the --no-pippet flag can be specified in order to replace this technique with a legacy workaround. In this mode, the unsupported resources will be turned into exec resources of the following form instead:

exec:
- name: <type>:title
  cmd: puppet yamlresource <type> 'title' '{ param => value, ... }'
  ifcmd: puppet yamlresource ... --noop | grep -q ^Notice:

This means that testing the sync state of such a resource requires mgmt to launch a puppet yamlresource process. If that reports a change in noop mode, another puppet yamlresource is launched to perform the sync. (The jury is still out on whether the noop call is actually required.)

In conservative mode, this technique is also applied to resources that are generally translatable, but raise warnings about specific parameters or values.

Compatibility

Supports Puppet 5.x and 6.x.

Supports mgmt 0.0.21 and higher.

Supports mgmt 0.0.16 and higher when sticking to no-pippet mode (but no earlier releases).

Extending

See the DSL Guide.

TODO

• easier DSL (e.g. add a method to get at the namevar)
• find a way to properly model resource refreshing in mgmt

Alpha releases

0.6.2 2023-02-09

• fix a bug that would prevent agent mode from working right
• various plumbing improvements to get with the times

This release contains contributed code from Darragh Bailey (@electrofelix)

0.6.1 2020-02-01

• fix a bug that would lead to invalid pippet node structure

0.6.0 2020-01-31

• replaces the puppet yamlresource workaround with pippet per default
• adds the --no-pippet option for backwards compatibility
• raises version requirement for ffrank-yamlresource to 0.2.0
• raises version requirement for mgmt to 0.0.21 for full support

0.5.0 2020-01-29

• drop watchcmd from the "puppet resource" default translation
• drop support for mgmt < 0.0.16

0.4.5 2020-01-26

• fixes a metadata issue wrt. ffrank-yamlresource

0.4.4 2020-01-26

• converted to PDK format
• drop Puppet 4 support in favor of Puppet 6

0.4.3 2019-11-11

• supports more parameters to exec
• add support for mount, user, group

0.4.2 2019-02-12

• documentation improvements
• avoid spawning parameters when they get no value
• fix an issue where the application would abort with certain inputs
• add support for Puppet 6.x
• made sure to not try and run on Puppet 3 and older
• no longer warns when files use the default filebucket option
• fix issue with failing translations

This release contains contributions by James Shubin (@purpleidea)

Thanks!

0.4.1 2018-03-09

• add missing documentation

0.4.0 2018-03-09

• remove Puppet 3 support (not broken, just no longer tested)
• add the stats subcommand
• support more mgmt features (now requires mgmt 0.0.8)
• some fixes and improvements in edge case handling
• new resource type support: augeas, ec2_instance
• fix a bug with translating service resources
• fix metaparameter handling in the fallback pseudo-translator

This release contains code written by Johan Bloemberg (@aequitas).

Thanks for the contribution!

0.3.1 2017-02-20

• translate notify resources to mgmt's msg
• add the --conservative flag

0.3.0 2016-08-19

• turn resources that mgmt does not support into exec puppet yamlresource vertices
• print warnings about attributes that cannot be translated
• drop test coverage with Ruby 1.9.3
• ignore notify resources in the input
• adopt a dependency on the ffrank-yamlresource module

The idea to handle unsupported resources this way is from Daniele Sluijters (@daenney).

0.2.3 2016-07-12

• (#1) support relationships with classes and insances of defined types

0.2.2 2016-06-22

• generate a watchcmd in exec resources based on Puppet's runinterval
• fix the package type translation
• support ensure => directory in file resources
• generate deterministic names for relationship edges
• some bug- and doc fixes

This release contains contributed code from James Shubin.

0.2.1 2016-05-09

• support service resources
• support package resources
• more flexible support for relationship edges

0.2.0 2016-03-30

• (breaking) move output format from mgmt 0.0.2 to 0.0.3
• add mgmt compatibility information to README

0.1.0 2016-03-15

• switch implementation to the faces API
• add a DSL for defining resource translation rules
• add basic infrastructure like tests, README, metadata etc.

Pre-releases

0.0.1 2016-02-24

• initial publication