pe_databases

A Puppet Module for Maintaining Your Puppet Enterprise Databases

Project URL RSS Feed Report issues

Module Stats

19,926 downloads

386 latest version

3.1 quality score

Version information

released Jan 31st 2024

Passed malware scan

This version is compatible with:

Puppet Enterprise 2023.8.x, 2023.7.x, 2023.6.x, 2023.5.x, 2023.4.x, 2023.3.x, 2023.2.x, 2023.1.x, 2023.0.x, 2021.7.x
Puppet >= 7.18.0 < 9.0.0
, , , , , , ,

Tasks:

reset_pgrepack_schema

Start using this module

Installation method

Add this module to your Puppetfile:

mod 'puppetlabs-pe_databases', '4.1.0'

Learn more about managing modules with a Puppetfile

Add this module to your Bolt project:

bolt module add puppetlabs-pe_databases

Learn more about using this module with an existing project

Manually install this module globally with Puppet module tool:

puppet module install puppetlabs-pe_databases --version 4.1.0

Documentation

puppetlabs/pe_databases — version 4.1.0 Jan 31st 2024

Table of Contents
Overview
Supporting Content

Overview

What does this module provide?

This module provides maintenance tasks to keep the pe-puppetdb database lean and fast

Usage

This module is bundled with all currently support versions of Puppet Enterprise, and enabled by default. When with Puppet Enterprise, see the following documentation for instructions on how to enable or disable.

https://puppet.com/docs/pe/latest/pe_database_maintenance.html#enable_pe_database_module

Items you may want to configure

Maintenance

This module provides systemd timers to pg_repack tables in the pe-puppetdb database. These times are configurable using the corresponding parameters, but default to:

facts tables are pg_repack'd Tuesdays and Saturdays at 4:30AM
catalogs tables are pg_repack'd Sundays and Thursdays at 4:30AM
reports table is pg_repack'd on the 10th of the month at 05:30AM on systems with PE 2019.7.0 or less
resource_events table is pg_repack'd on the 15th of the month at 05:30AM on systems with PE 2019.3.0 or less
other tables are pg_repack'd on the 20th of the month at 5:30AM

Classify the following parameters to change the values:

$facts_tables_repack_timer           = 'Tue,Sat *-*-* 04:30:00',
$catalogs_tables_repack_timer        = 'Sun,Thu *-*-* 04:30:00',
$other_tables_repack_timer           = '*-*-20 05:30:00',
$reports_tables_repack_timer         = '*-*-10 05:30:00',
$resource_events_tables_repack_timer = '*-*-15 05:30:00',

Please note that when using pg_repack as part of the pe_databases module, unclean exits can leave behind the schema when otherwise it should have been cleaned up. This can result in the messages similar to the following:

INFO: repacking table "public.fact_paths"
WARNING: the table "public.fact_paths" already has a trigger called "repack_trigger"
DETAIL: The trigger was probably installed during a previous attempt to run pg_repack on the table which was interrupted and for some reason failed to clean up the temporary objects. Please drop the trigger or drop and recreate the pg_repack extension altogether to remove all the temporary objects left over.

The module now contains a task reset_pgrepack_schema to mitigate this issue. This needs to be run against your Primary or PE-postgreSQL server to resolve this and it will drop and recreate the extension, removing the temporary objects.

Disable Maintenance

The maintenance systemd timers will perform a pg_repack on various pe-puppetdb tables to keep them lean and fast. pg_repack is a non blocking operation and should have no impact on the operations of Puppet Enterprise, however, if for some reason you experience issues you can disable the maintenance systemd timers. You can do so by setting pe_databases::disable_maintenance: true in your hieradata.

Deprecated functionality

Backups

WARNING: The backup functionality in this module has been removed. Please refer to the PE Backup and Restore documentation for details on how to backup. You should ensure the parameter pe_databases::manage_database_backups and any parameters from the pe_databases::backup class are removed from classification or hiera. You should also clean up associated crontab entries.

PE PostgreSQL Tuning

Recent versions of PE PostgreSQL included in all supported versions of Puppet Enterprise have superseded or improved upon the parameters previously tuned by this module, as such they are no longer required. Any classifications of the parameters relating to this functionality should be removed, however if set will harmlessly omit a warning of this deprecation.

Supporting Content

Articles

The Support Knowledge base is a searchable repository for technical information and how-to guides for all Puppet products.

This Module has the following specific Article(s) available:

Videos

The Support Video Playlist is a resource of content generated by the support team

How to Report an issue or contribute to the module

If you are a PE user and need support using this module or are encountering issues, our Support team would be happy to help you resolve your issue and help reproduce any bugs. Just raise a ticket on the support portal. If you have a reproducible bug or are a community user you can raise it directly on the Github issues page of the module puppetlabs/puppetlabs-pe_databases. We also welcome PR contributions to improve the module. Please see further details about contributing here

Reference

Classes

pe_databases: Tuning, maintenance for PE PostgreSQL.
pe_databases::pg_repack: Provides systemd timers to pg_repack tables in a given database

Defined types

pe_databases::collect: Create systemd units for repacking a given database type

Tasks

reset_pgrepack_schema: This task will remove and recreate the pg_repack extension

Classes

`pe_databases`

Tuning, maintenance for PE PostgreSQL.

Parameters

The following parameters are available in the pe_databases class:

manage_database_maintenance
disable_maintenance
install_dir
scripts_dir
facts_tables_repack_timer
catalogs_tables_repack_timer
other_tables_repack_timer
activity_tables_repack_timer
manage_postgresql_settings
manage_table_settings
reports_tables_repack_timer
resource_events_tables_repack_timer

`manage_database_maintenance`

Data type: Boolean

true or false (Default: true) Manage the inclusion of the pg_repack class

Default value: true

`disable_maintenance`

Data type: Boolean

true or false (Default: false) Disable or enable maintenance mode

Default value: false

`install_dir`

Data type: String[1]

Directory to install module into (Default: "/opt/puppetlabs/pe_databases")

Default value: '/opt/puppetlabs/pe_databases'

`scripts_dir`

Data type: String[1]

Directory to install scripts into (Default: "${install_dir}/scripts")

Default value: "${install_dir}/scripts"

`facts_tables_repack_timer`

Data type: String[1]

The Systemd timer for the pg_repack job affecting the 'facts' tables

`catalogs_tables_repack_timer`

Data type: String[1]

The Systemd timer for the pg_repack job affecting the 'catalog' tables

`other_tables_repack_timer`

Data type: String[1]

The Systemd timer for the pg_repack job affecting the 'other' tables

`activity_tables_repack_timer`

Data type: String[1]

The Systemd timer for the pg_repack job affecting the 'activity' tables

`manage_postgresql_settings`

Data type: Optional[Boolean]

Deprecated Parameter will be removed in future releases

Default value: undef

`manage_table_settings`

Data type: Optional[Boolean]

Deprecated Parameter will be removed in future releases

Default value: undef

`reports_tables_repack_timer`

Data type: Optional[String]

Deprecated Parameter will be removed in future releases

Default value: undef

`resource_events_tables_repack_timer`

Data type: Optional[String]

Deprecated Parameter will be removed in future releases

Default value: undef

`pe_databases::pg_repack`

Maintenance pg_repack

Parameters

The following parameters are available in the pe_databases::pg_repack class:

fact_tables
catalog_tables
other_tables
activity_tables
disable_maintenance
repack_log_level
enable_echo
jobs
facts_tables_repack_timer
catalogs_tables_repack_timer
other_tables_repack_timer
activity_tables_repack_timer
reports_tables_repack_timer
resource_events_tables_repack_timer

`fact_tables`

Data type: Array

Array of 'fact' tables to repack

`catalog_tables`

Data type: Array

Array of 'catalog' tables to repack

`other_tables`

Data type: Array

Array of 'other' tables to repack

`activity_tables`

Data type: Array

Array of 'activity' tables to repack

`disable_maintenance`

Data type: Boolean

true or false (Default: false) Disable or enable maintenance mode

Default value: false

`repack_log_level`

Data type: Enum['INFO','NOTICE','WARNING','ERROR','LOG','FATAL','PANIC','DEBUG']

Desired output level of logs

Default value: 'DEBUG'

`enable_echo`

Data type: Boolean

true or false (Default: true) Enabling echo output in logs

Default value: true

`jobs`

Data type: Integer

How many jobs to run in parallel

Default value: $facts['processors']['count'] / 4

`facts_tables_repack_timer`

Data type: String[1]

The Systemd timer for the pg_repack job affecting the 'facts' tables

Default value: $pe_databases::facts_tables_repack_timer

`catalogs_tables_repack_timer`

Data type: String[1]

The Systemd timer for the pg_repack job affecting the 'catalog' tables

Default value: $pe_databases::catalogs_tables_repack_timer

`other_tables_repack_timer`

Data type: String[1]

The Systemd timer for the pg_repack job affecting the 'other' tables

Default value: $pe_databases::other_tables_repack_timer

`activity_tables_repack_timer`

Data type: String[1]

The Systemd timer for the pg_repack job affecting the 'activity' tables

Default value: $pe_databases::activity_tables_repack_timer

`reports_tables_repack_timer`

Data type: Optional[String]

Deprecated Parameter will be removed in future releases

Default value: undef

`resource_events_tables_repack_timer`

Data type: Optional[String]

Deprecated Parameter will be removed in future releases

Default value: undef

Defined types

`pe_databases::collect`

Create systemd units for repacking a given database type

Parameters

The following parameters are available in the pe_databases::collect defined type:

database_type
command
disable_maintenance
on_cal
tables

`database_type`

Data type: String

The database to repack, uses titles from pg_repack.pp

Default value: $title

`command`

Data type: String

defined in pg_repack.pp

Default value: undef

`disable_maintenance`

Data type: Boolean

to disable maintenance mode (Default: false)

Default value: false

`on_cal`

Data type: String

values can be found in pg_repack.pp

Default value: undef

`tables`

Data type: Array

Array of tables to repack

Default value: undef

Tasks

`reset_pgrepack_schema`

This task will remove and recreate the pg_repack extension

Supports noop? false

Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/

TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION

1. Definitions.

"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.

"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.

"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.

"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.

"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.

"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.

"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).

"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.

"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."

"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.

2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.

3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.

4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:

(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and

(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and

(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and

(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.

You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.

5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.

6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.

7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.

8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.

9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.

END OF TERMS AND CONDITIONS

APPENDIX: How to apply the Apache License to your work.

To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "{}"
replaced with your own identifying information. (Don't include
the brackets!)  The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.

Copyright 2016 Nick Walker

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Quality checks

We run a couple of automated scans to help you assess a module’s quality. Each module is given a score based on how well the author has formatted their code and documentation and select modules are also checked for malware using VirusTotal.

Please note, the information below is for guidance only and neither of these methods should be considered an endorsement by Puppet.

Malware scan results

Scan Result

Passed

The malware detection service on Puppet Forge is an automated process that identifies known malware in module releases before they’re published. It is not intended to replace your own virus scanning solution.

Learn more about malware scans

Scan Detail

Module name:: puppetlabs-pe_databases
Module version:: 4.1.0
Scan initiated:: January 31st 2024, 5:14:10

Detections:

0 / 60

Scan stats:

59 undetected
0 harmless
1 failures
0 timeouts
0 malicious
0 suspicious
16 unsupported

Scan report:

View the detailed scan report

Modules

Discover

Contribute

Puppet

Premium features

Downloads

Resources

About Forge

Getting Started

pe_databases

Version information

This version is compatible with:

Tasks:

Start using this module

Documentation

Table of Contents

Overview

What does this module provide?

Usage

Items you may want to configure

Maintenance

Disable Maintenance

Deprecated functionality

Backups

PE PostgreSQL Tuning

Supporting Content

Articles

Videos

How to Report an issue or contribute to the module

Reference

Table of Contents

Classes

Defined types

Tasks

Classes

pe_databases

Parameters

manage_database_maintenance

disable_maintenance

install_dir

scripts_dir

facts_tables_repack_timer

catalogs_tables_repack_timer

other_tables_repack_timer

activity_tables_repack_timer

manage_postgresql_settings

manage_table_settings

reports_tables_repack_timer

resource_events_tables_repack_timer

pe_databases::pg_repack

Parameters

fact_tables

catalog_tables

other_tables

activity_tables

disable_maintenance

repack_log_level

enable_echo

jobs

facts_tables_repack_timer

catalogs_tables_repack_timer

other_tables_repack_timer

activity_tables_repack_timer

reports_tables_repack_timer

resource_events_tables_repack_timer

Defined types

pe_databases::collect

Parameters

database_type

command

disable_maintenance

on_cal

tables

Tasks

reset_pgrepack_schema

What are tasks?

Tasks in this module release

reset_pgrepack_schema

Change log

v4.1.0 (2024-01-31)

`pe_databases`

`manage_database_maintenance`

`disable_maintenance`

`install_dir`

`scripts_dir`

`facts_tables_repack_timer`

`catalogs_tables_repack_timer`

`other_tables_repack_timer`

`activity_tables_repack_timer`

`manage_postgresql_settings`

`manage_table_settings`

`reports_tables_repack_timer`

`resource_events_tables_repack_timer`

`pe_databases::pg_repack`

`fact_tables`

`catalog_tables`

`other_tables`

`activity_tables`

`disable_maintenance`

`repack_log_level`

`enable_echo`

`jobs`

`facts_tables_repack_timer`

`catalogs_tables_repack_timer`

`other_tables_repack_timer`

`activity_tables_repack_timer`

`reports_tables_repack_timer`

`resource_events_tables_repack_timer`

`pe_databases::collect`

`database_type`

`command`

`disable_maintenance`

`on_cal`

`tables`

`reset_pgrepack_schema`