Forge Home

Modules

Puppet

Resources

PuppetEcosystemPuppet CommunityUpdates

osg

OSG Puppet module
Project URLRSS FeedReport issues

11,810 downloads

506 latest version

5.0 quality score

We run a couple of automated
scans to help you access a
module's quality. Each module is
given a score based on how well
the author has formatted their
code and documentation and
modules are also checked for
malware using VirusTotal.

Please note, the information below
is for guidance only and neither of
these methods should be considered
an endorsement by Puppet.

Version information

  • 6.1.0 (latest)
  • 6.0.0
  • 5.2.1
  • 5.2.0
  • 5.1.0
  • 5.0.0
  • 4.4.0
  • 4.3.1
  • 4.3.0
  • 4.2.2
  • 4.2.1
  • 4.2.0
  • 4.1.1
  • 4.1.0
released Jun 27th 2022
This version is compatible with:
  • Puppet Enterprise 2023.2.x, 2023.1.x, 2023.0.x, 2021.7.x, 2021.6.x, 2021.5.x, 2021.4.x, 2021.3.x, 2021.2.x, 2021.1.x, 2021.0.x, 2019.8.x, 2019.7.x, 2019.5.x, 2019.4.x, 2019.3.x, 2019.2.x, 2019.1.x, 2019.0.x
  • Puppet >= 6.0.0 < 8.0.0
  • , ,

Start using this module

  • r10k or Code Manager
  • Bolt
  • Manual installation
  • Direct download

Add this module to your Puppetfile:

mod 'treydock-osg', '6.1.0'
Learn more about managing modules with a Puppetfile

Add this module to your Bolt project:

bolt module add treydock-osg
Learn more about using this module with an existing project

Manually install this module globally with Puppet module tool:

puppet module install treydock-osg --version 6.1.0

Direct download is not typically how you would use a Puppet module to manage your infrastructure, but you may want to download the module in order to inspect the code.

Download
Tags: hpc, gridftp, osg, condor

Documentation

treydock/osg — version 6.1.0 Jun 27th 2022

puppet-osg

Puppet Forge Build Status

Table of Contents

  1. Overview
  2. Usage - Configuration examples and options
  3. Reference - Parameter and detailed reference to all options
  4. Limitations - OS compatibility, etc.
  5. Development - Guide for testing and contributing to the module

Overview

The OSG module manages the various components that make up the Open Science Grid software stack.

OSG Compatibility

Currently this module supports OSG 3.5.

The current version support matrix is as follows:

OSG Versions 3.3 3.4 3.5
puppet-osg 1.x yes no no
puppet-osg 2.x yes yes no
puppet-osg 3.x yes yes no
puppet-osg 4.x no yes no
puppet-osg 5.x no no yes

Usage

Classes

The public classes of this module are split into "roles". For example a CE would use the class osg::ce.

osg

The OSG class is required by all the other classes. This class sets parameters that are used by multiple "roles" and is the class responsible for configuring the OSG repos.

    class { 'osg': }

If all systems pull their grid-certificates from a shared filesystem then you instruct this module to install the empty-ca-certs package and symlink /home/osg/grid-certificates to /etc/grid-security/certificates. If this method is used some script must be executed on a regular basis to sync one system's certificates into the shared location. Such a script is currently outside the scope of this module.

    class { 'osg':
      shared_certs_path     => '/home/osg/grid-certificates',
      cacerts_package_name  => 'empty-ca-certs',
    }

The cacerts_package_ensure parameter can be used to ensure the latest CA certs package is installed. This is basically the same functionality as provided by the resources managed under the osg::cacerts::updater class.

    class { 'osg':
      cacerts_package_name   => 'osg-ca-certs',
      cacerts_package_ensure => 'latest',
    }

osg::cacerts::updater

The osg::cacerts::updater class by default will perform the following actions

  • Install osg-ca-certs-updater and fetch-crl packages
  • Configure /etc/cron.d/osg-ca-certs-updater
  • Start the osg-ca-certs-updater-cron service
  • Start the fetch-crl-cron service
  • Stop the fetch-crl-boot service

Example usage:

    class { 'osg':
      cacerts_package_name => 'osg-ca-certs',
    }
    class { 'osg::cacerts::updater': }

This class essentially performs the same role as setting osg::cacerts_package_ensure and osg::cacerts_other_packages_ensure to latest.

osg::ce

This class by default configures a HTCondor CE. The following example is to configure using HTCondor-CE that uses the SLURM batch system. This example also shows how to setup a host as the system that keeps the shared grid-certificates up-to-date.

    class { 'osg':
      site_info_resource        => 'SITE_CE',
      site_info_resource_group  => 'SITE',
      site_info_sponsor         => 'vo-name-here',
      site_info_site_policy     => '',
      site_info_contact         => 'Full Name'
      site_info_email           => 'admin@site.com'
      site_info_city            => 'Somewhere'
      site_info_country         => 'USA'
      site_info_longitude       => '-0.000000'
      site_info_latitude        => '0.000000'
    }
    class { 'osg::ce':
      batch_system              => 'slurm',
      hostcert_source           => 'file:///home/admin/osg/certs/ce/hostcert.pem',
      hostkey_source            => 'file:///home/admin/osg/certs/ce/hostkey.pem',
    }

osg::client

The osg::client class is intended to configure client systems, such as login nodes, to interact with the OSG software. The example below is a rough idea of how one would configure a client system to send both condor and condor-ce requests to a remote HTCondor-CE instance.

    class { 'osg::client'
      condor_schedd_host    => 'ce.example.tld',
      condor_collector_host => 'ce.example.tld:9619',
    }

The default behavior is to ensure both condor and htcondor-ce are installed but the services for each are disabled.

osg::cvmfs

The osg::cvmfs class will install and configure a system to access the CVMFS wide-area filesystem. The default parameter values should be enough to allow access to the filesystem.

    class { 'osg::cvmfs': }

Any host that needs to access /cvmfs should have this class assigned.

osg::gridftp

The osg::gridftp class by default will configure a system as a standalone OSG GridFTP server. Note that the osg::ce classes declares this class with standalone set to false, so do not include this class if the osg::ce class is assigned.

    class { 'osg':
      site_info_resource        => 'SITE_GRIDFTP',
      site_info_resource_group  => 'SITE',
      site_info_sponsor         => 'vo-name-here',
      site_info_site_policy     => '',
      site_info_contact         => 'Full Name'
      site_info_email           => 'admin@site.com'
      site_info_city            => 'Somewhere'
      site_info_country         => 'USA'
      site_info_longitude       => '-0.000000'
      site_info_latitude        => '0.000000'
    }
    class { 'osg::gridftp':
      hostcert_source       => 'file:///home/admin/osg/certs/gridftp/hostcert.pem',
      hostkey_source        => 'file:///home/admin/osg/certs/gridftp/hostkey.pem',
    }

osg::lcmaps_voms

The following example will setup LCMAPS VOMS to authenticate the GLOW VO and ban CMS production. The vo parameter will create osg::lcmaps_voms::vo resources and the users parameter will create osg::lcmaps_voms::user resources.

    class { 'osg::lcmaps_voms':
      ban_voms => ['/cms/Role=production/*'],
      ban_users => ['/foo/baz'],
      vos       => {
        'glow' => '/GLOW/*',
        'glow1 => '['/GLOW/chtc/*', '/GLOW/Role=htpc/*'],
      },
      users     => {
        'foo'   => '/fooDN',
        'foobar => ['/foo', '/bar'],
      }
    }

osg::lcmaps_voms::vo

This defined type populates /etc/grid-security/voms-mapfile. The dn value can be an Array or a String.

    osg::lcmaps_voms::vo { 'nova':
      dn => '/fermilab/nova/Role=pilot',
    }

osg::lcmaps_voms::user

This defined type populates /etc/grid-security/grid-mapfile. The dn value can be an Array or a String.

    osg::lcmaps_voms::user { 'rsv':
      dn => '/DC=org/DC=opensciencegrid/O=Open Science Grid/OU=Services/CN=rsv/ce.example.com',
    }

osg::squid

The osg::squid class will configure a system to run the Frontier Squid service. The example below installs squid and configures the firewall to allow access to squid on the host's eth0 private interface and allows squid monitoring on the eth1 public interface.

    class { 'osg::squid':
      private_interface => 'eth0',
      public_interface  => 'eth1',
    }

Be sure to define squid_location that points to the location of the squid server

    class { 'osg':
      squid_location => 'squid.site.com',
    }

The customize_template can be used to pass a site-specific template used to customize squid. The template provided by this module is very basic. The value in the example below will look in the site_osg Puppet module under templates/squid for the file customize.sh.erb. The value of customize_template is passed directly to the Puppet template function.

    class { 'osg::squid':
      customize_template  => 'site_osg/squid/customize.sh.erb',
    }

osg::utils

The osg::utils class will install utility packages from OSG.

Example:

    class { 'osg::utils':}

osg::wn

The osg::wn class will configure a worker node to work with the OSG software. This class currently has no parameters and performs the following actions:

  • Ensures the osg class is included (repo)
  • Ensures the osg::cacerts class is included
  • Installs osg-wn-client package
  • Installs xrootd-client

Example:

    class { 'osg::wn': }

Types

osg_local_site_settings

The osg_local_site_settings custom type will configure values in /etc/osg/config.d/99-local-site-settings.ini. Some of the values are set in this module's classes. One example of a value not currently managed (though may be in the future):

    osg_local_site_settings { 'Storage/se_available':
      value   => true,
    }

Note that boolean values of true and false are converted to the Python bool values of True and False.

osg_gip_config

The osg_gip_config custom type will configure values in /etc/osg/config.d/30-gip.ini. Example of setting your batch system to SLURM.

    osg_gip_config { 'GIP/batch':
      value => 'slurm',
    }

You can also remove the settings defined in 30-gip.ini and use the osg_local_site_settings type to define all configs in /etc/osg/config.d/99-local-site-settings.ini

    resources { 'osg_gip_config': purge => true }
    osg_local_site_settings { 'GIP/batch':
      value => 'slurm',
    }

This can be useful as the 99-local-site-settings.ini does not take precedence in GIP like it does with osg-configure (ref).

Reference

http://treydock.github.io/puppet-osg/

Limitations

Tested operating systems:

  • RedHat/CentOS 7

This module has only been thoroughly tested using OSG 3.5.

Development

Testing

Testing requires the following dependencies:

  • rake
  • bundler

Install gem dependencies

bundle install

Run unit tests

bundle exec rake spec

If you have Docker installed you can run system tests.

bundle exec rake beaker