Ansible Development: Write Custom Modules, Plugins & Collections

Ansible Development: A Comprehensive Guide

Ansible is an open-source automation tool developed by Red Hat that enables IT professionals to automate tasks such as configuration management, application deployment, and task automation. This article delves into the essential aspects of Ansible development, covering its architecture, core concepts, and practical applications.

See also: Enhancing Ansible Documentation: Best Practices for Maintainable Playbooks

Understanding Ansible Architecture

Ansible's architecture comprises the following key components: Ansible Engine: This is the core component that executes the automation tasks defined in Ansible playbooks. Ansible Playbooks: Written in YAML, playbooks describe the desired state of the system and define the tasks to be executed. Ansible Inventory: This is a file that lists the hosts and groups of hosts that Ansible will manage. Modules: These are the units of work that Ansible executes. They can be anything from installing software to managing services. Plugins: These extend Ansible’s core functionalities, including action plugins, cache plugins, and callback plugins.

Ansible operates on a push model, where tasks are executed from a central control node to the managed nodes over SSH or WinRM, eliminating the need for agent software on the managed nodes.

Getting Started with Ansible

Ansible is favored for its simplicity and ease of use. Here’s how you can get started with Ansible: Installation: Ansible can be installed on any Unix-like system, including macOS and Linux. It can also be run on Windows under the Windows Subsystem for Linux (WSL).

   sudo apt update
   sudo apt install ansible
   

Setting Up an Inventory: The inventory file defines the hosts and groups of hosts to manage.

   [webservers]
   webserver1.example.com
   webserver2.example.com

[databases]
   dbserver1.example.com
   

Writing Your First Playbook: Ansible playbooks are written in YAML. Here’s a simple example that installs Nginx on web servers.

   ---
   - name: Install Nginx on web servers
     hosts: webservers
     become: yes
     tasks:
       - name: Install Nginx
         apt:
           name: nginx
           state: present
   

Executing Playbooks: Run your playbook with the ansible-playbook command.

   ansible-playbook -i inventory.ini playbook.yml
   

See also: Automating Jenkins Installation with Ansible

Core Ansible Concepts

Roles: Roles provide a way to organize playbooks and related files to be easily shared and reused.

   - hosts: webservers
     roles:
       - nginx
   

Variables: Variables allow you to store values that can be reused and overridden as needed.

   vars:
     http_port: 80
     max_clients: 200
   

Handlers: Handlers are tasks that are run when notified by other tasks.

   handlers:
     - name: restart nginx
       service:
         name: nginx
         state: restarted
   

Templates: Templates in Ansible are used to fill configuration files with variables.

   tasks:
     - name: Deploy configuration file
       template:
         src: templates/nginx.conf.j2
         dest: /etc/nginx/nginx.conf
       notify:
         - restart nginx
   

Advanced Ansible Usage

Ansible Tower: Ansible Tower is a web-based solution that makes Ansible even more powerful by providing role-based access control, job scheduling, and graphical inventory management. Dynamic Inventory: This is used to manage and interact with hosts that are not statically defined in the inventory file, such as cloud environments. Custom Modules and Plugins: While Ansible comes with many built-in modules, custom modules can be written to extend its functionality using any programming language, although Python is most common.

See also: Ansible Automation: Complete Guide to IT Automation with Playbook Examples

Integrating Ansible with Kubernetes

Ansible can manage Kubernetes clusters, automating tasks such as cluster provisioning, application deployment, and configuration management. The k8s module allows you to interact with Kubernetes resources directly from Ansible playbooks.

- name: Create a Kubernetes Pod
  hosts: localhost
  tasks:
    - name: Create a Pod
      kubernetes.core.k8s:
        state: present
        definition:
          apiVersion: v1
          kind: Pod
          metadata:
            name: nginx
            namespace: default
          spec:
            containers:
            - name: nginx
              image: nginx:1.14.2
              ports:
              - containerPort: 80

Conclusion

Ansible is a powerful tool for automating a wide range of IT tasks. Its simple, agentless architecture and human-readable automation language make it accessible to both beginners and seasoned professionals. Whether you are managing traditional infrastructure, containerized environments, or cloud services, Ansible provides the tools you need to automate and orchestrate your IT operations efficiently.

For more detailed guidance and advanced topics, refer to the comprehensive resources available in Red Hat’s documentation and the community-contributed materials on platforms like GitHub and Ansible Galaxy.

Custom Module (Basic)

#!/usr/bin/python
# library/my_module.py
from ansible.module_utils.basic import AnsibleModule
def main():
    module = AnsibleModule(
        argument_spec=dict(
            name=dict(type='str', required=True),
            state=dict(type='str', default='present', choices=['present', 'absent']),
        ),
        supports_check_mode=True
    )
name = module.params['name']
    state = module.params['state']
# Check mode
    if module.check_mode:
        module.exit_json(changed=True, msg=f"Would set {name} to {state}")
# Your logic here
    changed = False
    result = {"name": name, "state": state}
module.exit_json(changed=changed, **result)
if __name__ == '__main__':
    main()

# Use in playbook
- my_module:
    name: test
    state: present

Module with API Calls

#!/usr/bin/python
from ansible.module_utils.basic import AnsibleModule
from ansible.module_utils.urls import open_url
import json
def main():
    module = AnsibleModule(
        argument_spec=dict(
            url=dict(type='str', required=True),
            token=dict(type='str', required=True, no_log=True),
            data=dict(type='dict', default={}),
        )
    )
try:
        response = open_url(
            module.params['url'],
            method='POST',
            headers={'Authorization': f"Bearer {module.params['token']}",
                     'Content-Type': 'application/json'},
            data=json.dumps(module.params['data'])
        )
        result = json.loads(response.read())
        module.exit_json(changed=True, result=result)
    except Exception as e:
        module.fail_json(msg=str(e))
if __name__ == '__main__':
    main()

Custom Filter Plugin

# filter_plugins/my_filters.py
class FilterModule:
    def filters(self):
        return {
            'normalize_name': self.normalize_name,
            'to_cidr': self.to_cidr,
        }
def normalize_name(self, name):
        return name.lower().replace(' ', '-').replace('_', '-')
def to_cidr(self, ip, prefix=24):
        return f"{ip}/{prefix}"

- debug: msg="{{ 'My Server Name' | normalize_name }}"
  # my-server-name
- debug: msg="{{ '10.0.1.0' | to_cidr(16) }}"
  # 10.0.1.0/16

Custom Lookup Plugin

# lookup_plugins/my_lookup.py
from ansible.plugins.lookup import LookupBase
class LookupModule(LookupBase):
    def run(self, terms, variables=None, **kwargs):
        results = []
        for term in terms:
            # Your logic
            results.append(term.upper())
        return results

- debug: msg="{{ lookup('my_lookup', 'hello') }}"
  # HELLO

Collection Structure

myorg/
└── mytools/
    ├── galaxy.yml
    ├── plugins/
    │   ├── modules/
    │   │   └── my_module.py
    │   ├── filter/
    │   │   └── my_filters.py
    │   └── lookup/
    │       └── my_lookup.py
    ├── roles/
    │   └── setup/
    ├── playbooks/
    └── tests/

Testing Modules

# Unit test
python -m pytest tests/unit/
# Integration with Molecule
molecule test
# Sanity checks
ansible-test sanity --docker

Development Environment

# Clone ansible-core for development
git clone https://github.com/ansible/ansible.git
cd ansible
pip install -e .
# Create module development structure
mkdir -p library filter_plugins lookup_plugins

FAQ

Where do I put custom modules?

In library/ next to your playbook, or in a roles//library/ directory, or configure library path in ansible.cfg.

Do modules need to be Python?

Modules can be any language — Ansible sends JSON input and expects JSON output. Python is most common due to AnsibleModule helper.

How do I publish a collection?

ansible-galaxy collection build
ansible-galaxy collection publish myorg-mytools-1.0.0.tar.gz --server https://galaxy.ansible.com

Custom Module (Python)

#!/usr/bin/python
# library/my_module.py
from ansible.module_utils.basic import AnsibleModule
def main():
    module = AnsibleModule(
        argument_spec=dict(
            name=dict(type='str', required=True),
            state=dict(type='str', choices=['present', 'absent'], default='present'),
        ),
        supports_check_mode=True,
    )
name = module.params['name']
    state = module.params['state']
changed = False
    # Your logic here
if module.check_mode:
        module.exit_json(changed=changed)
module.exit_json(changed=changed, msg=f"Resource {name} is {state}")
if __name__ == '__main__':
    main()

Custom Filter Plugin

# filter_plugins/my_filters.py
class FilterModule:
    def filters(self):
        return {
            'normalize_hostname': self.normalize_hostname,
            'to_cidr': self.to_cidr,
        }
def normalize_hostname(self, value):
        return value.lower().strip().replace(' ', '-')
def to_cidr(self, ip, prefix=24):
        return f"{ip}/{prefix}"

# Usage
- debug:
    msg: "{{ 'My Server' | normalize_hostname }}"
    # Output: my-server

Custom Lookup Plugin

# lookup_plugins/my_lookup.py
from ansible.plugins.lookup import LookupBase
class LookupModule(LookupBase):
    def run(self, terms, variables=None, **kwargs):
        results = []
        for term in terms:
            results.append(term.upper())
        return results

Development Environment

# Clone ansible-core
git clone https://github.com/ansible/ansible.git
cd ansible
source hacking/env-setup
# Install test dependencies
pip install pytest pytest-mock
# Run module locally
python -m ansible.modules.my_module /tmp/args.json

Collection Development

# Create collection
ansible-galaxy collection init myorg.mytools
# Structure
myorg/mytools/
├── galaxy.yml
├── plugins/
│   ├── modules/
│   ├── filter/
│   ├── lookup/
│   └── action/
├── roles/
├── playbooks/
├── tests/
└── docs/

Testing Modules

# tests/unit/test_my_module.py
import pytest
from ansible.module_utils.basic import AnsibleModule
from plugins.modules.my_module import main
def test_module_present(mocker):
    args = {"name": "test", "state": "present"}
    mocker.patch.object(AnsibleModule, '__init__', return_value=None)
    mocker.patch.object(AnsibleModule, 'params', args, create=True)
    # ... test logic

FAQ

Where to put custom modules?

In library/ next to your playbook, in a role's library/, or in a collection's plugins/modules/.

How to debug modules?

Set ANSIBLE_KEEP_REMOTE_FILES=1 — Ansible leaves the module on the remote host. SSH in and run with Python directly.

Module vs Plugin?

Modules run on remote hosts (tasks). Plugins run on the controller (filters, lookups, callbacks, connection types).

Related Articles

• private Galaxy servers and Ansible • Jinja2 filters in Ansible templates • the Ansible handlers reference • Windows management with Ansible • switching users with Ansible become

See also

• Ansible Templates: Jinja2 Template Module Complete Guide • Ansible Plugins: Types, Usage & Custom Plugin Development Guide • Ansible Custom Modules: Write Your Own Module in Python (Complete Guide)