neo4j puppet module

=====================

####Table of Contents

1. Overview
2. Setup
3. Usage
4. Development

Overview

Installs Neo4J (http://www.neo4j.com) on CentOS/RedHat/Ubuntu/Debian either from packages or from their distribution tarballs downloaded directly from the Neo4j website.

Setup

All of the setup for any of the configuration is done through via class neo4j. There are default options provided such that you can install Neo4j into /opt/neo4j and start listening on http://your_ip:7474/ by specifying:

include neo4j

To install the last enterprise edition, you can specify:

class { 'neo4j' :
  edition => 'enterprise',
}

See usage scenarios below for further usages.

Usage

Currently, all of the configuration is done through the class neo4j including clustered Neo4j server.

Here are some examples of different configurations that people could use this module to manage their Neo4j instances.

Install from source

class { 'neo4j' :
  install_method => 'archive',
  version        => '3.1.1',
}

Install from package

class { 'neo4j' :
}

Authentication and User Management

For information on what native roles are provided, creating and deleting custom roles, ldap config management, see https://neo4j.com/docs/operations-manual/current/security/authentication-authorization/.

Clustering

Starting from Neo4j version 3.1, causal clustering is enabled. HA clustering is still available, but - according my Neo4J's deep throat - it will become deprecated in future versions. So, if you're using the latest and greatest: go for Causal Clustering.

HA Clustering (>3.0)

Parameters available:

• ha_allow_init_cluster # Whether to allow this instance to create a cluster if unable to join.
• ha_branched_data_copying_strategy # Strategy for how to order handling of branched data on slaves and copying of the store from the master.
• ha_branched_data_policy # Policy for how to handle branched data.
• ha_broadcast_timeout # Timeout for broadcasting values in cluster.
• ha_configuration_timeout # Timeout for waiting for configuration from an existing cluster member during cluster join.
• ha_data_chunk_size # Max size of the data chunks that flows between master and slaves in HA.
• ha_default_timeout # Default timeout used for clustering timeouts.
• ha_election_timeout # Timeout for waiting for other members to finish a role election.
• ha_heartbeat_interval # How often heartbeat messages should be sent.
• ha_heartbeat_timeout # How long to wait for heartbeats from other instances before marking them as suspects for failure.
• ha_host_coordination # Host and port to bind the cluster management communication.
• ha_host_data # Hostname and port to bind the HA server.
• ha_initial_hosts # A comma-separated list of other members of the cluster to join.
• ha_internal_role_switch_timeout # Timeout for waiting for internal conditions during state switch, like for transactions to complete, before switching to master or slave.
• ha_join_timeout # Timeout for joining a cluster.
• ha_learn_timeout # Timeout for learning values.
• ha_leave_timeout # Timeout for waiting for cluster leave to finish.
• ha_max_acceptors # Maximum number of servers to involve when agreeing to membership changes.
• ha_max_channels_per_slave # Maximum number of connections a slave can have to the master.
• ha_paxos_timeout # Default value for all Paxos timeouts.
• ha_phase1_timeout # Timeout for Paxos phase 1.
• ha_phase2_timeout # Timeout for Paxos phase 2.
• ha_pull_batch_size # Size of batches of transactions applied onslaves when pulling from master.
• ha_pull_interval # Interval of pulling updates from master.
• ha_role_switch_timeout # Timeout for request threads waiting for instance to become master or slave.
• ha_server_id # Id for a cluster instance.
• ha_slave_lock_timeout # Timeout for taking remote (write) locks on slaves.
• ha_slave_only # Whether this instance should only participate as slave in cluster.
• ha_slave_read_timeout # How long a slave will wait for response from master before giving up
• ha_tx_push_factor # The amount of slaves the master will ask to replicate a committed transaction.
• ha_tx_push_strategy # Push strategy of a transaction to a slave during commit.

Causal Clustering (3.1, 3.2)

Parameters available:

• causal_clustering_array_block_id_allocation_size # The size of the ID allocation requests Core servers will make when they run out of ARRAY_BLOCK IDs.
• causal_clustering_catchup_batch_size # The maximum batch size when catching up (in unit of entries).
• causal_clustering_cluster_allow_reads_on_followers # Configure if the dbms.cluster.routing.getServers() procedure should include followers as read endpoints or return only read replicas.
• causal_clustering_cluster_routing_ttl # How long drivers should cache the data from the dbms.cluster.routing.getServers() procedure.
• causal_clustering_cluster_topology_refresh # Time between scanning the cluster to refresh current server’s view of topology.
• causal_clustering_disable_middleware_logging # Prevents the network middleware from dumping its own logs.
• causal_clustering_discovery_advertised_address # Advertised cluster member discovery management communication.
• causal_clustering_discovery_listen_address # Host and port to bind the cluster member discovery management communication.
• causal_clustering_expected_core_cluster_size # Expected number of Core machines in the cluster.
• causal_clustering_global_session_tracker_state_size # The maximum file size before the global session tracker state file is rotated (in unit of entries).
• causal_clustering_id_alloc_state_size # The maximum file size before the ID allocation file is rotated (in unit of entries).
• causal_clustering_initial_discovery_members # A comma-separated list of other members of the cluster to join.
• causal_clustering_join_catch_up_timeout # Time out for a new member to catch up.
• causal_clustering_label_token_id_allocation_size # The size of the ID allocation requests Core servers will make when they run out of LABEL_TOKEN IDs.
• causal_clustering_label_token_name_id_allocation_size # The size of the ID allocation requests Core servers will make when they run out of LABEL_TOKEN_NAME IDs.
• causal_clustering_last_applied_state_size # The maximum file size before the storage file is rotated (in unit of entries).
• causal_clustering_leader_election_timeout # The time limit within which a new leader election will occur if no messages are received.
• causal_clustering_log_shipping_max_lag # The maximum lag allowed before log shipping pauses (in unit of entries).
• causal_clustering_neostore_block_id_allocation_size # The size of the ID allocation requests Core servers will make when they run out of NEOSTORE_BLOCK IDs.
• causal_clustering_node_id_allocation_size # The size of the ID allocation requests Core servers will make when they run out of NODE IDs.
• causal_clustering_node_labels_id_allocation_size # The size of the ID allocation requests Core servers will make when they run out of NODE_LABELS IDs.
• causal_clustering_outgoing_queue_size # The number of messages waiting to be sent to other servers in the cluster.
• causal_clustering_property_id_allocation_size # The size of the ID allocation requests Core servers will make when they run out of PROPERTY IDs.
• causal_clustering_property_key_token_id_allocation_size # The size of the ID allocation requests Core servers will make when they run out of PROPERTY_KEY_TOKEN IDs.
• causal_clustering_property_key_token_name_id_allocation_size # The size of the ID allocation requests Core servers will make when they run out of PROPERTY_KEY_TOKEN_NAME IDs.
• causal_clustering_pull_interval # Interval of pulling updates from cores.
• causal_clustering_raft_advertised_address # Advertised hostname/IP address and port for the RAFT server.
• causal_clustering_raft_listen_address # Network interface and port for the RAFT server to listen on.
• causal_clustering_raft_log_implementation # RAFT log implementation.
• causal_clustering_raft_log_prune_strategy # RAFT log pruning strategy.
• causal_clustering_raft_log_pruning_frequency # RAFT log pruning frequency.
• causal_clustering_raft_log_reader_pool_size # RAFT log reader pool size.
• causal_clustering_raft_log_rotation_size # RAFT log rotation size.
• causal_clustering_raft_membership_state_size # The maximum file size before the membership state file is rotated (in unit of entries).
• causal_clustering_raft_messages_log_enable # Enable or disable the dump of all network messages pertaining to the RAFT protocol.
• causal_clustering_raft_term_state_size # The maximum file size before the term state file is rotated (in unit of entries).
• causal_clustering_raft_vote_state_size # The maximum file size before the vote state file is rotated (in unit of entries).
• causal_clustering_read_replica_refresh_rate # Read replica 'call home' frequency.
• causal_clustering_read_replica_time_to_live # Time To Live before read replica is considered unavailable.
• causal_clustering_relationship_group_id_allocation_size # The size of the ID allocation requests Core servers will make when they run out of RELATIONSHIP_GROUP IDs.
• causal_clustering_relationship_id_allocation_size # The size of the ID allocation requests Core servers will make when they run out of RELATIONSHIP IDs.
• causal_clustering_relationship_type_token_id_allocation_size # The size of the ID allocation requests Core servers will make when they run out of RELATIONSHIP_TYPE_TOKEN IDs.
• causal_clustering_relationship_type_token_name_id_allocation_size # The size of the ID allocation requests Core servers will make when they run out of RELATIONSHIP_TYPE_TOKEN_NAME IDs.
• causal_clustering_replicated_lock_token_state_size # The maximum file size before the replicated lock token state file is rotated (in unit of entries).
• causal_clustering_schema_id_allocation_size # The size of the ID allocation requests Core servers will make when they run out of SCHEMA IDs.
• causal_clustering_state_machine_apply_max_batch_size # The maximum number of operations to be batched during applications of operations in the state machines.
• causal_clustering_state_machine_flush_window_size # The number of operations to be processed before the state machines flush to disk.
• causal_clustering_string_block_id_allocation_size # The size of the ID allocation requests Core servers will make when they run out of STRING_BLOCK IDs.
• causal_clustering_transaction_advertised_address # Advertised hostname/IP address and port for the transaction shipping server.
• causal_clustering_transaction_listen_address # Network interface and port for the transaction shipping server to listen on.
• causal_clustering_unknown_address_logging_throttle # Throttle limit for logging unknown cluster member address.

Install Path

If you want to install to another directory, use the install_prefix parameter.

class { 'neo4j' :
  install_prefix => '/tmp',
}

Graphite

You can specifiy to send several metrics to a a graphite server.

• metrics_enabled # enable all supported metrics
• metrics_graphite_enabled # enable sending metrics to graphite
• metrics_graphite_interval # Set the graphite interval. Defaults to 3 min.
• metrics_graphite_server # Set the graphite server.
• metrics_prefix # set the prefix so you can find your metrics in graphite.

Enable/disable specific metrics:

• metrics_jvm_buffers_enabled
• metrics_jvm_gc_enabled
• metrics_jvm_memory_enabled
• metrics_jvm_threads_enabled
• metrics_neo4j_causal_clustering_enabled
• metrics_neo4j_checkpointing_enabled
• metrics_neo4j_cluster_enabled
• metrics_neo4j_counts_enabled
• metrics_neo4j_enabled
• metrics_neo4j_logrotation_enabled
• metrics_neo4j_network_enabled
• metrics_neo4j_pagecache_enabled
• metrics_neo4j_server_enabled
• metrics_neo4j_tx_enabled

Memory Tuning

Tuning of the page cache size

For caching the neo4j data as stored on disk in memory, sum up the size of all store.db files on disk and add 20% for growth.

• dbms_memory_pagecache_size

Tuning of the heap size

• dbms_memory_heap_initial_size
• dbms_memory_heap_max_size

JVM tuning properties

• dbms_jvm_additional_use_g1gc
• dbms_jvm_additional_omit_stacktrace_in_fast_throw
• dbms_jvm_additional_hashcode
• dbms_jvm_additional_commit_memory_to_process
• dbms_jvm_additional_unlock_experimental_vm_options
• dbms_jvm_additional_trust_final_non_static_fields
• dbms_jvm_additional_disable_explicit_gc

class { 'neo4j' :
  dbms_memory_heap_initial_size => '4g',
  dbms_memroy_heap_max_size     => '4g',
  dbms_memory_pagecache_size    => '4g',
}

Versions

All known versions >= 3.0.0 can be installed. For the list of current releases, you can go to http://neo4j.com/download/.

class { 'neo4j' :
  version => '3.1.1',
  edition => 'enterprise',
}

Development

Running tests

This project contains tests for both rspec-puppet and test kitchen to verify functionality. For detailed information on using these tools, please see their respective documentation.

Testing quickstart:

gem install bundler
bundle install
bundle exec rake {validate,lint,spec,strings:generate,guard}
bundle exec kitchen {list,converge,verify,test}

2018-08-21 - Release 2.0.2

• Fix typo in template neo4j.conf.general.erb
• Update rvm and puppet version in travis.yml

2017-02-22 - Release 2.0.1

• Fix to improve the quality score on puppetforge.
• Fixes to templates: some config settings are optional.
• Changed default group to 'neo4j'. Not all neo4j packages comply (yet).

2017-02-20 - Release 2.0.0

Summary

Completed parameter list, fixed documentation and added spec tests.

Improvements

• Removed option to install java from within module
• Added version indication to parameters whereever needed
• Completed parameter list: ALL possible settings should be in here.
• Default setup should make use of packages, whether it be rpm's or debs. The reason is, that no sane company is using a source install.
• Added spec testing
• Moving to test kitchen for integration testing

2016-11-15 - Release 1.2.4

Adaptations needed for install with rpm.

2016-11-14 - Release 1.2.3

Summary

Rewrite, targeted at neo4j 3.x users. Getting rid of existing shell scripts and jars. Although it is still possible to install neo4j using a tar.gz release, a packaged version is preferred.

2016-08-12 - Release 1.1.0

Summary

This is a minor release, targeted at neo4j 2.x users. We're not going to put any more effort in supporting this version, as most users (should) be migrating to 3.x anyway.

2016-02-02 - Release 1.0.3

Summary

Taken over module from Amos Wood

2014-10-13 - Supported Release 1.0.2

Summary

Added logical log parameter to configuration

Features

• Added control parameter keep_logical_logs.

2014-10-02 - Supported Release 1.0.1

Summary

Added changelog and feature for object cache new to version 2.1.5

Features

• Added control parameter cache_memory_ratio from version 2.1.5 object cache settings.

Bugfixes

• Switched to use function versioncmp instead.
• Added dependencies and requirements to metadata.

2014-06-27 - Release 1.0.0

Summary

The initial release

Features

Added custom configuration of:

1. Version 2.x.x or later
2. Community or Enterprise editions
3. JVM Wrapper Settings
4. File Buffer Cache Options (see Max DeMarzi's blog Scaling-Up)
5. Object Cache Options
6. New Relic java agent path specified (see http://www.newrelic.com)
7. Authentication Plugin installed and configured (see http://www.github.com)
8. Authenticated User Management

- Admin User
- Users and Passwords
- Read/Write or ReadOnly

1. High Availability Options