The Reusable Ansible Role Problem

About 1 year ago, I build a Ansible playbook for the customers of 5xRuby.

When the customers grow it is hard to use the fork feature to maintenance the customer’s playbook.

I have to update the main version and sync the changes to the fork version for each customer. Therefore I decided to split the common parts to a single role repository.

Overview

In the current version, we have a playbook like below:

├── [1.0K]  README.md
├── [  96]  group_vars
│   └── [1.2K]  all.yml
├── [  96]  inventories
│   └── [ 309]  local
├── [ 480]  roles
│   ├── [  96]  5xruby_user
│   ├── [  96]  application
│   ├── [  96]  compile_env
│   ├── [  96]  deploy_user
│   ├── [  96]  init
│   ├── [ 128]  logrotate
│   ├── [ 160]  nginx_with_passenger
│   ├── [  96]  node
│   ├── [ 160]  postgresql_server
│   ├── [  96]  ruby
│   ├── [  96]  ssh
│   ├── [  96]  sudo
│   └── [ 128]  yum_install_commons
└── [ 467]  setup.yml

When the customer needs to customize their provision environment, you have to fork it and change the variables and some template.

But it is hard to update the playbook because it may cause some conflict.

Target

The Ansible Galaxy provides the dependencies manage feature to Ansible, it allows us to use roles/requirements.yml to manage it like below:

1- src: https://github.com/5xruby/ansible-ruby
2  version: 0.1.0
3- src: https://github.com/5xruby/ansible-nginx
4  version: 0.1.0

Before you run the playbook, you can use ansible-galaxy install -r roles/requirements.yml to automatically install these roles. And it also works well with Ansible AWX (a.k.a Ansible Tower)

That sounds good, but when I start work on it I got some problems.

NGINX Modules

For Rails project, we have more than one choice of the webserver.

If you use Puma you only require to configure NGINX as a reverse proxy.

But if you decide to use Passenger you have to compile it as an NGINX module.

It means if you want to support Puma and Passenger together, the NGINX role should include the Passenger tasks.

My first version is to use the include_tasks to add an extra module config when enabled Passenger.

But if you want to add more NGINX modules in the future? The NGINX role will grow and becomes another huge playbook.

Manual Dependencies

After many tries, I find an acceptable implement to resolve the problem.

1. Create a Fact nginx_module_options as empty array
2. Loop nginx_extar_modules to import_role to execute module related tasks
3. After the module’s source code downloaded, append configure options to nginx_module_options

In the playbook, you can configure dependencies like below:

1
2- src: https://github.com/5xruby/ansible-nginx
3  version: 0.1.0
4- src: https://github.com/5xruby/ansible-passenger
5  version: 0.1.0

And add NGINX module options to group_vars/all.yml as a default config to apply to all web hosts.

1nginx_extra_modules: ['passenger']

After resolving the NGINX module problem, another problem is coming.

The Role Dependencies

When I prepare the NGINX, Ruby, Node.js, and others required for deploy Rails. I configure the Rails role’s dependencies.

1dependencies:
2  - src: https://github.com/5xruby/ansible-nginx
3  - src: https://github.com/5xruby/ansible-ruby
4  - src: https://github.com/5xruby/ansible-node
5  - src: https://github.com/5xruby/ansible-passenger

When you use the playbook to run rails role, it starts the task from the nginx role.

It seems no problem, but you have to configure the nginx.conf and set the root to the application’s public directory.

If the nginx role is run before the rails role, you will get the NGINX start failed error.

The first version NGINX will create the root directory with customized owner and group, but it has some problems in this case. The deploy user is created by the rails role and it will unable to find the owner user which is not created yet.

But after clarifying the problem, it is a human design mistake.

“The NGINX is the dependencies of Rails?”

If you use the Puma, you can replace NGINX to any reverse proxy server. So the playbook didn’t dependent on the NGINX.

Final Produce

After about two days of hard work, I have a new playbook almost zero-config to deploy a server for Rails.

├── install.yml
├── group_vars
│  └── all.yml
├── inventory
├── playbooks
│  └── install-nginx.yml
│  └── install-postgres.yml
│  └── install-rails.yml
├── roles
│  └── requirements.yml
├── templates
└ ─── nginx.conf.j2

Everything is very simple, usually use import_role: nginx to add the necessary role.

If you need more customized you can override the variable (e.g. nginx_config_template) and put the customize template to templates/nginx.conf.j2

I only put a default NGINX config inside the NGINX role and put the customize nginx.conf in the playbook to enable the Passenger.

Conclusion

This is an interesting experience to “decouple” the provision script. As a programmer, we have a lot of rule can follow to decouple our program. But when you are working on the operation-side, how to create a reusable script and manageable?

But this work is only the start. I am considering the future upgrade after I finished this stage.

• How to clear the old versions?
• If the database has to upgrade, should I create a new server?
• If used for creating a Cloud Image (e.g. AMI) how to clear up?

The DevOps sounds very easy to put developer and operator together but make the work together still hard, I guess.