TL;DR
We’ve open-sourced a Puppet module to help manage the configuration of Sonatype Nexus instances. Check it out!
The Build Engineering team at Atlassian has been running Sonatype Nexus instances for a few years now. We use Nexus for storing our public and private artifacts on https://maven.atlassian.com/ (which receives 2.6 million requests/day), publishing to maven central and the various proxies that we have spread out across the world to support our build grid, and our global development teams. Sonatype’s support has been absolutely fantastic when we’ve hit hairy scaling issues. We’re also big users of Puppet where we manage our infrastructure as code. We originally managed our Sonatype Nexus instances by getting Puppet to manage the various xml files in the ‘sonatype-work/nexus/conf’ directory. This option had a few problems:
The need to restart Nexus to make sure new XML configuration files are processed
Ephemeral changes (staging repositories, for example) are lost after a Puppet run
When Nexus changes XML configuration files, Puppet will overwrite them and restart Nexus
Using XML for configuration is so passé
Having Nexus & Puppet fight over control of the configuration files isn’t a fun experience. The other alternative is to use Puppet Augeas, which allows more intelligent management of XML content, but this approach still shares several disadvantages and has its own:
The need to restart Nexus to make sure new XML configuration files are processed
Have to manage and maintain XML configuration templates and files
Introduce additional complexity to Puppet manifests
Thankfully, Nexus comes with a REST API, and we’ve taken advantage of it in writing a Puppet module that aims to address all of the above disadvantages. We’ve now open sourced this Puppet module so you can use it to manage your own Nexus instances in a Puppet managed environment.
In a nutshell, Puppet Module for Sonatype Nexus allows configuration to go from this:
#manifest/.../config.pp
file { ".../sonatype-work/nexus/conf/nexus.xml":
content => template('buildeng_nexus/common/opt/nexus/current/conf/nexus.properties.erb'),
owner => $buildeng_nexus::common::params::user,
group => $buildeng_nexus::common::params::group,
notify => Class['buildeng_nexus::common::service'],
}
#templates/.../sonatype-work/sonatype/conf/nexus.xml.erb
...
public
Public Repository
org.sonatype.nexus.proxy.repository.Repository maven2 IN_SERVICE
true
true
true
ALLOW_WRITE
true
true
file .../sonatype-work/nexus/storage/public
RELEASE
...
...
To this:
#manifest/.../config.pp
nexus_repository { 'public':
label => 'Public Repository',
provider_type => 'maven2',
type => 'hosted',
policy => 'release',
}
Usage
First of all you need to create a configuration file ‘$confdir/nexus_rest.conf’ (whereas ‘$confdir’ defaults to ‘/etc/puppet’):
The configuration file will provide the module with the required information about where Nexus is listening and which credentials to use to enforce the configuration. Obviously, it is recommended to manage the file within Puppet and limit the visibility to the root.
Any change is enforced through Nexus’ REST API. Hence, the Nexus service has to be running before any modification can be made. In general, any ordering between the
service { 'nexus': }
resource and resources provided by this module should be made explicit in the Puppet manifest itself. This module doesn’t express any autorequire dependency (‘soft dependency’) on the service resource itself—this is up to the user of the Puppet module. However, any resource provided by this module will wait a certain amount of time in order to give Nexus the chance to properly start up. The default timeout is 150 seconds and can be configured via the configuration file.
All resources are implemented as providers. This means that if you have a running Nexus instance you can simply inspect the current state with:
puppet resource
for example:
puppet resource nexus_repository
and copy & paste the result into your manifest file.
Global Configuration
The global configuration has been decomposed into different resources. The following examples show how to use them.
Note: the query string returned by Nexus contains encoded HTML entities. So submitting e.g. & via the REST interface will result in an new version of the query string where it is replaced with &. To avoid an ongoing war between Nexus and Puppet updating the configuration, this module will unescape the received query string. This can be subject to API breakages if Sonatype decides to change the behavior.
Note: the current implementation doesn’t support authentication at the proxy server. But code contributions are gratefully accepted!
nexus_smtp_settings { 'current':
hostname => 'mail.example.com',
port => 25,
username => 'jdoe',
password => present,
password_value => 'keepitsecret',
communication_security => none,
sender_email => 'nexus@example.com',
}
Repository Configuration
Note: the ‘position’ property in ‘nexus_repository_route’ is due to a workaround. Unfortunately, Nexus’ repository routes have no writable properties that can be used as an id. System instances are sorted and associated with catalog resources by matching their index in the sorted list with the ‘position’ property.
Since the instances are sorted by a randomly generated id, it may take two runs of ‘puppet apply’ for the resources to fall into place. Several ‘nexus_repository_route’ resources may be modified each time a new one is added.
Scheduled Tasks
You can easily manage all of your scheduled tasks from within Puppet. For instance the following resource would set up a task to empty the trash once a day:
nexus_scheduled_task { 'Empty Trash':
ensure => 'present', # present or absent
enabled => true, # true (default) or false
type => 'Empty Trash', # required, just use the name as provided in the user interface
alert_email => 'ops@example.com', # optional; use 'absent' (default) to disable the email notification
reoccurrence => 'daily', # one of 'manual' (default), 'once', 'daily', 'weekly', 'monthly' or 'advanced'
start_date => '2014-05-31',
recurring_time => '20:00',
task_settings => {'EmptyTrashItemsOlderThan' => '14', 'repositoryId' => 'all_repo'},
}
Notes:
‘type’ responds to the name or the id. If the type name is provided, the module will try to translate it to the type id; if that fails, the given name is passed through to Nexus. In case the given type name doesn’t work, try to provide the type id directly
Date and times are based on the timezone that is used on the server running Nexus. As Puppet should normally run on the same server this shouldn’t cause any trouble. However, when using the web UI on a computer with a different timezone, the values shown there are relative to that timezone and can appear different.
Be very careful with one-off tasks (ie. reoccurrence => ‘once’); due to the way Nexus works, it will reject any updates of the one-off task once the scheduled date has passed. This will cause your Puppet run to fail
Due to the complexity of the resource it is strongly recommended to configure the task via the user interface and use ‘puppet resource’ to generate the corresponding Puppet manifest.
Date and time related properties
Setting ‘reoccurrence’ to one of the following values requires to specify additional properties:
‘manual’ – no further property required
‘once’ – ‘start_date’ and ‘start_time’
‘hourly’ – ‘start_date’ and ‘start_time’
‘daily’ – ‘start_date’ and ‘recurring_time’
‘weekly’ – ‘start_date’, ‘recurring_time’ and ‘recurring_day’ (‘recurring_day’ should be a day of the _week_, e.g. ‘monday’, ‘tuesday’, …, ‘sunday’)
‘monthly’ – ‘start_date’, ‘recurring_time’ and ‘recurring_day’ (‘recurring_day’ should be a day of the _month_, e.g. 1, 2, …. 29, 30, 31 or ‘last’)
‘advanced’ – ‘cron_expression’
It is expected that ‘start_date’ matches ‘YYYY-MM-DD’ and ‘start_time’ / ‘recurrence_time’ match ‘HH:MM’ (including leading zeros). The ‘recurring_day’ accepts multiple values as a list (e.g. ‘[1, 2, 'last'])’.
Furthermore, you should keep your manifest clean and not specify properties that are not required (e.g. specify ‘cron_expression’ for a ‘manual’ task).
Limitations
Ruby and Puppet compatibility
The module has been tested to work with various Puppet and Ruby versions. Supported are
Ruby 1.8.7
Ruby 1.9.3
Ruby 2.0.0
and
Puppet 3.4
Puppet 3.5
It is very likely to work with any Puppet 3.x version. Support for Puppet 2.7.x has been dropped in favour of improved support for custom Puppet types and providers that is only available in Puppet 3.x.
Nexus compatibility
Furthermore, the module has been tested with the following Nexus versions:
Nexus Pro 2.9.x running on Ubuntu 12.04
A note on passwords
Due to the limitations of the Nexus REST API it is not possible to retrieve the current value of a password. Hence, Puppet can only manage the existence of the password but won’t notice when passwords change. Either way, passwords will be updated when attributes of the same resource change as well.
Contributing
1. Raise an issue
2. Fork it
3. Create your feature branch (git checkout -b my-new-feature)
4. Commit your changes (git commit -am ‘Add some feature’)
5. Push to the branch (git push origin my-new-feature)
6. Create a new pull request targeting master
At this point not all options covered by XML configuration are covered by this module, but the module is designed to be easily extensible and pull requests are welcome. This module could be capable of managing anything configurable through the Nexus UI
The post Announcing an open source Puppet Module for Sonatype Nexus appeared first on Atlassian Blogs.