Creating a Custom Ansible Lookup Plugin in Python for Reading a File

Extending Ansible’s Capabilities with a Custom File Retrieval Python Lookup Plugin.

September 12, 2023

Access the Complete Video Course and Learn Quick Ansible by 200+ Practical Lessons

#ansible #plugin #custom #lookup #python #code #development #ansible.cfg

Introduction

Ansible, a powerful automation tool, offers a wide range of built-in modules and plugins that simplify infrastructure management. However, there are scenarios where you may need to extend Ansible’s capabilities by creating custom plugins tailored to your specific needs. In this article, we will explore creating and using a custom Ansible file lookup plugin, which allows you to retrieve file contents from your Ansible controller’s file system during playbook execution.

What is a Lookup Plugin?

Ansible lookup plugins are used to retrieve data dynamically during playbook execution. They allow you to fetch information from various sources, such as databases, APIs, or external files, and use that data in your Ansible tasks.

Unveiling the Custom File Lookup Plugin

Before delving into the details of the plugin, let’s take a closer look at the Python script provided at the beginning of this article. This script serves as an example of a custom Ansible file lookup plugin and consists of several essential components:

1. Python 3 Headers

from __future__ import (absolute_import, division, print_function)
__metaclass__ = type

These lines specify Python 3 headers required for compatibility when submitting this plugin to Ansible.

2. Documentation

DOCUMENTATION = r"""
  name: file
  author: Luca Berton <[email protected]>
  version_added: "0.1"
  short_description: read file contents
  description:
      - This lookup returns the contents from a file on the Ansible controller's file system.
  options:
    _terms:
      description: path(s) of files to read
      required: True
    option1:
      description:
            - Sample option that could modify plugin behaviour.
            - This one can be set directly ``option1='x'`` or in ansible.cfg, but can also use vars or environment.
      type: string
      ini:
        - section: file_lookup
          key: option1
  notes:
    - if read in variable context, the file can be interpreted as YAML if the content is valid to the parser.
    - this lookup does not understand globbing --- use the fileglob lookup instead.

This documentation block provides metadata about the plugin, including its name, author, version, a short description, and additional options. This information is crucial for documenting and understanding the plugin’s purpose and behavior.

3. Imports

from ansible.errors import AnsibleError, AnsibleParserError
from ansible.plugins.lookup import LookupBase
from ansible.utils.display import Display

The script imports modules and classes from Ansible, allowing it to handle errors, utilize Ansible utilities, and provide display output.

4. Initialization

display = Display()

The Display class is used to manage the display of messages and debugging information within Ansible. An instance of this class is created to facilitate logging.

5. Custom Lookup Module

class LookupModule(LookupBase):

Here, we define our custom lookup module, named LookupModule. It inherits from LookupBase, a base class for creating lookup plugins in Ansible.

6. Custom Method (run)

def run(self, terms, variables=None, **kwargs):
    # Plugin logic goes here

The run method is the core of the lookup plugin. It defines how the plugin operates when invoked in an Ansible playbook. In this example, it retrieves file contents from the controller’s file system and handles errors gracefully.

Please note that by design, Ansible Lookup plugins are historically usually used to pipe results into loops. Ansible expects a list as a return type. A common walkaround is to wrap our return value into a list, like:

return [ret]

The full code of the fileread.py Ansible lookup plugin looks like the:

# python 3 headers, required if submitting to Ansible
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type

DOCUMENTATION = r"""
  name: file
  author: Luca Berton <[email protected]>
  version_added: "0.1"
  short_description: read file contents
  description:
      - This lookup returns the contents from a file on the Ansible controller's file system.
  options:
    _terms:
      description: path(s) of files to read
      required: True
    option1:
      description:
            - Sample option that could modify plugin behaviour.
            - This one can be set directly ``option1='x'`` or in ansible.cfg, but can also use vars or environment.
      type: string
      ini:
        - section: file_lookup
          key: option1
  notes:
    - if read in variable context, the file can be interpreted as YAML if the content is valid to the parser.
    - this lookup does not understand globbing --- use the fileglob lookup instead.
"""
from ansible.errors import AnsibleError, AnsibleParserError
from ansible.plugins.lookup import LookupBase
from ansible.utils.display import Display

display = Display()

class LookupModule(LookupBase):

    def run(self, terms, variables=None, **kwargs):
      self.set_options(var_options=variables, direct=kwargs)
      ret = []
      for term in terms:
          display.debug("File lookup term: %s" % term)
          lookupfile = self.find_file_in_search_path(variables, 'files', term)
          display.vvvv(u"File lookup using %s as file" % lookupfile)
          try:
              if lookupfile:
                  contents, show_data = self._loader._get_file_contents(lookupfile)
                  ret.append(contents.rstrip())
              else:
                  raise AnsibleParserError()
          except AnsibleParserError:
              raise AnsibleError("could not locate file in lookup: %s" % term)
          if self.get_option('option1') == 'do something':
            pass
      return ret

Enabling lookup plugins

Ansible automatically enables all available lookup plugins. To activate a custom lookup plugin, you can do so by placing it in one of the following locations:

Adjacent to Your Playbook: You can include your custom lookup plugin in a directory called lookup_plugins located next to your playbook file.
Inside a Collection: If you’ve installed an Ansible collection, you can place your custom lookup plugin in the plugins/lookup directory within that collection.
Within a Standalone Role: Custom lookup plugins can also be included within a standalone Ansible role. Place the plugin in the appropriate lookup_plugins directory within the role.
Configured Directory Sources: If you have configured custom directory sources in your ansible.cfg configuration file, you can place the lookup plugin in one of those directories. The setting key is lookup_plugins under the [defaults] section or ANSIBLE_LOOKUP_PLUGINS environmental variable

Following these methods, you can activate and use custom lookup plugins in Ansible to extend its functionality according to your specific automation requirements.

Utilizing the Custom File Lookup Plugin

Now that we’ve examined the script’s components, let’s explore how to use this custom file lookup plugin effectively in Ansible playbooks. Here are the key steps:

Placing the Plugin File Save the Python script containing your custom file lookup plugin in a directory recognized by Ansible. Typically, you can place it in a directory named lookup_plugins located adjacent to your playbook file.
Using the Plugin in a Task In your Ansible playbook, use the lookup function to invoke your custom plugin. Specify the name of the plugin, such as ‘file,’ and provide the path(s) of the file(s) you want to read. For example:

name: Read File Contents ansible.builtin.debug: msg: “{{ lookup(‘fileread’, ‘/path/to/your/file.txt’) }}” This task reads the contents of the specified file and displays them using the Ansible debug module.

Ansible Playbook

The full Ansible Playbook code is the following:

exec.yml

---
- name: Exec lookup plugin
  hosts: all
  tasks:
    - name: Read file from plugin
      ansible.builtin.debug:
        msg: "{{ lookup('fileread', 'example.txt') }}"

example.txt

sample text

Execution

ansible-playbook -i inventory exec.yml      

PLAY [Exec lookup plugin] *******************************************************************************

TASK [Gathering Facts] **********************************************************************************
ok: [demo.example.com]

TASK [Read file from plugin] ****************************************************************************
ok: [demo.example.com] => {
    "msg": "sample text"
}

PLAY RECAP **********************************************************************************************
demo.example.com                  : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Conclusion

Creating custom lookup plugins in Ansible, such as the custom file lookup plugin discussed in this article, empowers you to extend Ansible’s functionality to meet your specific automation requirements. Whether you need to retrieve file contents, access data from external sources, or perform other dynamic operations, custom lookup plugins enable you to harness the full potential of Ansible in your infrastructure automation workflows. By understanding the structure of a lookup plugin and following the steps outlined in this article, you can streamline your automation tasks and achieve greater flexibility and control.

Subscribe to the YouTube channel, Medium, and Website, X (formerly Twitter) to not miss the next episode of the Ansible Pilot.