Ansible Search & Transform JSON: selectattr, map & combine (Guide)

By Luca Berton · Published 2024-01-01 · Category: troubleshooting

How to search, filter, and transform JSON data in Ansible. Use selectattr, map, combine, and json_query filters to process complex data structures.

Introduction

Handling JSON data is a common task in IT automation workflows. Ansible simplifies this process with filters like from_json, json_query, and selectattr. This guide demonstrates how to:

Parse JSON data into a Python dictionary.
Transform the data into a structured list.
Search for specific criteria (e.g., if a name has a guestState of "running").

---

Use Case: Verify If a Guest Is Running

Imagine receiving JSON data from an API or file, containing details about virtual machine states. Your task is to:

Transform this JSON data into a structured format.
Check if a specific name has its guestState set to "running".
Return true or false based on the search.

---

The Ansible Playbook

Here’s how to achieve this:

Playbook Example

---
- hosts: localhost
  gather_facts: false
  vars:
    search_name: "baz" # The name to search for
  tasks:
    - name: Define Raw JSON String (Example)
      set_fact:
        raw_json_string: >
          {
            "results": [
              { "instance": { "guest": { "guestState": "running" } }, "item": "foo" },
              { "instance": { "guest": { "guestState": "running" } }, "item": "bar" },
              { "instance": { "guest": { "guestState": "running" } }, "item": "baz" },
              { "instance": { "guest": { "guestState": "running" } }, "item": "qux" },
              { "instance": { "guest": { "guestState": "running" } }, "item": "quux" }
            ]
          }

    - name: Parse JSON string to dictionary
      set_fact:
        json_data: "{{ raw_json_string | from_json }}"

    - name: Transform JSON to formatted list
      set_fact:
        formatted_list: "{{ json_data.results | json_query('[].{name: item, guestState: instance.guest.guestState}') }}"

    - name: Check if guestState is running for a specific name
      set_fact:
        guest_running: "{{ formatted_list | selectattr('name', 'equalto', search_name) | selectattr('guestState', 'equalto', 'running') | list | length > 0 }}"

    - name: Display the result
      debug:
        msg: "Is guest running? {{ guest_running }}"

---

Explanation

Key Sections

Define Raw JSON String:
A sample JSON string is defined using set_fact. In real-world scenarios, this data may come from an API or file.

Parse JSON to Dictionary:
The from_json filter converts the JSON string into a Python dictionary for further processing.

Transform JSON to Formatted List:
The json_query filter restructures the JSON into a list of dictionaries with name and guestState attributes.

Search in the List:
The selectattr filter dynamically searches for items in the list:
selectattr('name', 'equalto', search_name): Filters items with the specified name.
selectattr('guestState', 'equalto', 'running'): Filters items with a guestState of "running".
The result is converted to a list and checked for length (> 0) to return true or false.

Display Results:
The debug module outputs whether the guest is running (true or false).

---

Running the Playbook

Save the Playbook:
Save the code as transform_and_search.yml.

Execute the Playbook:

   ansible-playbook transform_and_search.yml

Output:
If the search_name is "baz", the result will be:

     TASK [Display the result] ********************************************
     ok: [localhost] => {
         "msg": "Is guest running? true"
     }

If the search_name is "nonexistent", the result will be:

     TASK [Display the result] ********************************************
     ok: [localhost] => {
         "msg": "Is guest running? false"
     }

---

Key Benefits

Dynamic Search:

The selectattr filter provides a concise and efficient way to search data based on multiple conditions.

Reusable Code:

You can adapt this pattern to other JSON data transformations and searches.

Error Handling:

The search logic ensures accurate results and avoids processing empty or invalid data.

---

Conclusion

By combining JSON parsing with dynamic filtering, Ansible allows you to handle and query complex data structures efficiently. Whether you're automating virtual machine states or managing API responses, these techniques provide a robust solution. Try this approach in your automation workflows and unlock new possibilities with Ansible!

Filter with selectattr

- vars:
    employees:
      - { name: alice, dept: engineering, active: true }
      - { name: bob, dept: sales, active: false }
      - { name: charlie, dept: engineering, active: true }
  set_fact:
    engineers: "{{ employees | selectattr('dept', 'eq', 'engineering') | list }}"
    active: "{{ employees | selectattr('active') | list }}"
    inactive: "{{ employees | rejectattr('active') | list }}"

Extract with map

- set_fact:
    names: "{{ employees | map(attribute='name') | list }}"
    # ['alice', 'bob', 'charlie']
    upper_names: "{{ employees | map(attribute='name') | map('upper') | list }}"
    # ['ALICE', 'BOB', 'CHARLIE']

Chain Filters

# Active engineers' names
- set_fact:
    result: "{{ employees | selectattr('dept', 'eq', 'engineering') | selectattr('active') | map(attribute='name') | list }}"
    # ['alice', 'charlie']

Sort and Group

- vars:
    servers:
      - { name: web1, region: us, cpu: 4 }
      - { name: db1, region: eu, cpu: 8 }
      - { name: web2, region: us, cpu: 2 }
  set_fact:
    by_cpu: "{{ servers | sort(attribute='cpu', reverse=true) }}"
    by_region: "{{ servers | groupby('region') }}"
    us_servers: "{{ servers | selectattr('region', 'eq', 'us') | sort(attribute='name') | list }}"

Merge and Combine

- vars:
    defaults:
      port: 80
      workers: 4
      debug: false
      timeout: 30
    env_config:
      port: 8080
      debug: true
  set_fact:
    config: "{{ defaults | combine(env_config) }}"
    # { port: 8080, workers: 4, debug: true, timeout: 30 }

Deep Merge

- vars:
    base:
      database:
        host: localhost
        port: 5432
      cache:
        enabled: true
    override:
      database:
        host: db.production.com
  set_fact:
    merged: "{{ base | combine(override, recursive=true) }}"
    # database: { host: db.production.com, port: 5432 }, cache: { enabled: true }

Transform Structure

# List of dicts → dict
- vars:
    items:
      - { key: max_conn, value: 200 }
      - { key: timeout, value: 30 }
  set_fact:
    config_dict: "{{ items | items2dict(key_name='key', value_name='value') }}"
    # { max_conn: 200, timeout: 30 }

# Dict → list of dicts
- set_fact:
    config_list: "{{ config_dict | dict2items }}"
    # [{ key: max_conn, value: 200 }, ...]

Aggregate Data

- vars:
    orders:
      - { product: widget, qty: 10, price: 5.0 }
      - { product: gadget, qty: 3, price: 25.0 }
      - { product: widget, qty: 5, price: 5.0 }
  set_fact:
    total_qty: "{{ orders | map(attribute='qty') | sum }}"       # 18
    total_revenue: "{{ orders | map(attribute='price') | sum }}"  # 35.0
    max_order: "{{ orders | max(attribute='qty') }}"

Useful Filter Combos

Task	Expression
Unique values	`list \`	unique
Flatten nested	`list \`	flatten
Random item	`list \`	random
First/Last	`list \`	first / `list \`	last
Zip lists	`list1 \`	zip(list2)
Batch/chunk	`list \`	batch(3)
Default value	`var \`	default('fallback')

FAQ

selectattr vs json_query?

selectattr is simpler for flat list filtering. json_query (JMESPath) handles deeply nested structures and complex queries better.

How do I handle missing attributes?

# selectattr fails if attribute missing on any item
# Use default in map instead:
result: "{{ items | map(attribute='optional_key', default='N/A') | list }}"

Category: troubleshooting

Browse all Ansible tutorials · AnsiblePilot Home

AnsiblePilot — Master Ansible Automation

Popular Topics

About Luca Berton