Ansible flatten Filter: Flatten Nested Lists & Remove Duplicates

Introduction

Ansible's flexibility in automation often involves handling intricate data structures. Nested lists, a common scenario, can pose challenges for efficient processing in playbooks. In this guide, we’ll explore how to manipulate and optimize nested lists in Ansible, leveraging powerful filters like flatten and unique to simplify workflows.

---

See also: Ansible Split String: Filter Guide for CSV, Delimiters & Lists

The Challenge: Nested Lists

Imagine this scenario: you have a data structure like the one below that needs processing:

input_data:
  list:
    [
      [
        { name: "foo" },
        { name: "bar" }
      ],
      [
        { name: "baz" },
        { name: "qux" }
      ],
      [],
      [
        { name: "quux" }
      ]
    ]

The structure has nested lists, empty elements, and requires transformation into a single, flat list for further tasks. Our goal: • Flatten the structure. • Remove unnecessary empty elements. • Ensure the integrity of the data for seamless automation.

---

Solution: Flatten and Optimize Lists

Ansible's flatten filter is your go-to tool for collapsing nested structures into manageable lists. Let’s see it in action.

Playbook Example

---
- name: Flatten and optimize nested lists
  hosts: localhost
  vars:
    input_data:
      list:
        [
          [
            { name: "foo" },
            { name: "bar" }
          ],
          [
            { name: "baz" },
            { name: "qux" }
          ],
          [],
          [
            { name: "quux" }
          ]
        ]
  tasks:
    - name: Flatten the nested list
      set_fact:
        optimized_list: "{{ input_data.list | flatten }}"
- name: Display the optimized list
      debug:
        msg: "Optimized List: {{ optimized_list }}"

---

Output

The playbook will transform the list into:

[
  { name: "foo" },
  { name: "bar" },
  { name: "baz" },
  { name: "qux" },
  { name: "quux" }
]

---

See also: Ansible Transform JSON Data: Filters for Parsing & Manipulating JSON

Advanced Techniques

Removing Duplicates

In cases where the flattened list contains duplicate entries, apply the unique filter:

optimized_list: "{{ (input_data.list | flatten) | unique }}"

Comparing Lists

To compare two lists and validate their content, use filters like map(attribute='name') and difference. This ensures all elements match between lists.

---

Use Cases

Dynamic Inventories: Simplify nested inventory data for efficient host management. Configuration Management: Process structured data for creating configurations. Data Cleanup: Normalize aggregated data from APIs or external sources.

---

See also: Ansible Troubleshooting: Fix Jinja2 Syntax & Inventory Errors

Conclusion

Mastering nested list handling in Ansible equips you to tackle complex data structures confidently. By using filters like flatten and unique, you can optimize playbooks, streamline workflows, and focus on automation tasks. Start applying these techniques to simplify your automation processes today!

Related Articles

• organizing hosts with Ansible inventory • the ansible.builtin.command reference