Ansible Transform JSON Data: Filters for Parsing & Manipulating JSON

Introduction

Modern IT automation often requires handling complex JSON data, whether from APIs, configuration files, or dynamic inputs. Ansible, with its powerful filters like from_json and json_query, provides an elegant way to parse and transform JSON data into actionable formats. This guide walks you through parsing JSON data and transforming it into a structured list in an Ansible playbook.

---

See also: Ansible Search & Transform JSON: selectattr, map & combine (Guide)

The Scenario

You have a JSON string representing a nested structure, such as:

{
  "results": [
    { "instance": { "guest": { "guestState": "running" } }, "item": "foo" },
    { "instance": { "guest": { "guestState": "running" } }, "item": "bar" },
    { "instance": { "guest": { "guestState": "running" } }, "item": "baz" },
    { "instance": { "guest": { "guestState": "running" } }, "item": "qux" },
    { "instance": { "guest": { "guestState": "running" } }, "item": "quux" }
  ]
}

The goal is to transform this JSON into a structured list:

formatted_list:
  - { name: "foo", guestState: "running" }
  - { name: "bar", guestState: "running" }
  - { name: "baz", guestState: "running" }
  - { name: "qux", guestState: "running" }
  - { name: "quux", guestState: "running" }

---

The Playbook

Here’s how you can achieve this transformation:

Ansible Code

---
- hosts: localhost
  gather_facts: false
  tasks:
    - name: Define Raw JSON String (Example - in real use, this comes from a variable)
      set_fact:
        raw_json_string: >
          {
            "results": [
              { "instance": { "guest": { "guestState": "running" } }, "item": "foo" },
              { "instance": { "guest": { "guestState": "running" } }, "item": "bar" },
              { "instance": { "guest": { "guestState": "running" } }, "item": "baz" },
              { "instance": { "guest": { "guestState": "running" } }, "item": "qux" },
              { "instance": { "guest": { "guestState": "running" } }, "item": "quux" }
            ]
          }
- name: Parse JSON string to dictionary
      set_fact:
        json_data: "{{ raw_json_string | from_json }}"
- name: Transform JSON to formatted list
      set_fact:
        formatted_list: "{{ json_data.results | json_query('[].{name: item, guestState: instance.guest.guestState}') }}"
- name: Display formatted list
      debug:
        msg: "Formatted List: {{ formatted_list }}"

---

See also: Ansible flatten Filter: Flatten Nested Lists & Remove Duplicates

Explanation

Key Steps

Define Raw JSON String: • Use the set_fact module to define a raw JSON string. • In real scenarios, this JSON could come from an API, a file, or another source. Parse JSON String: • The from_json filter converts the JSON string into a Python dictionary, enabling further manipulation. Transform JSON Data: • Use the json_query filter to extract and restructure the JSON. • Query: [].{name: item, guestState: instance.guest.guestState}. • Extracts item as name. • Extracts instance.guest.guestState as guestState. Display Results: • The debug module outputs the transformed list for validation.

---

Output

After running the playbook, the debug task will display:

Formatted List: 
  - { name: "foo", guestState: "running" }
  - { name: "bar", guestState: "running" }
  - { name: "baz", guestState: "running" }
  - { name: "qux", guestState: "running" }
  - { name: "quux", guestState: "running" }

---

How to Use This Playbook

Save: • Save the code as a YAML file (e.g., transform.yml). Run: • Execute the playbook using:

     ansible-playbook transform.yml
     

Verify Output: • Ensure the transformed list is displayed correctly in the debug task.

---

See also: Ansible Split String: Filter Guide for CSV, Delimiters & Lists

Important Notes

• From JSON to Dict: Use the from_json filter only if the raw JSON is a string. If your JSON data is already a dictionary, you can skip this step. • Install JMESPath: The json_query filter requires the JMESPath Python library. Install it with:

  pip install jmespath
  

• Dynamic JSON Sources: Replace the hardcoded raw_json_string with actual data sources, such as API responses using the uri module.

---

Conclusion

Ansible’s from_json and json_query filters are powerful tools for handling and transforming JSON data. By leveraging these features, you can streamline complex data manipulations, making your playbooks more dynamic and efficient.

Start incorporating these techniques in your automation workflows today!

Parse JSON String

- set_fact:
    data: "{{ json_string | from_json }}"
# From command output
- command: curl -s https://api.example.com/status
  register: api_response
- set_fact:
    status: "{{ api_response.stdout | from_json }}"
- debug: msg="{{ status.version }}"

Convert to JSON

- debug:
    msg: "{{ my_dict | to_json }}"
# Pretty print
- debug:
    msg: "{{ my_dict | to_nice_json }}"

Filter and Transform

- vars:
    servers:
      - { name: web1, env: prod, cpu: 4 }
      - { name: web2, env: staging, cpu: 2 }
      - { name: db1, env: prod, cpu: 8 }
  set_fact:
    # Filter by attribute
    prod_servers: "{{ servers | selectattr('env', 'eq', 'prod') | list }}"
    # Extract names
    server_names: "{{ servers | map(attribute='name') | list }}"
    # Sum values
    total_cpu: "{{ servers | map(attribute='cpu') | sum }}"
    # Sort
    by_cpu: "{{ servers | sort(attribute='cpu', reverse=true) }}"

Restructure Data

- vars:
    users:
      - { name: alice, role: admin }
      - { name: bob, role: user }
      - { name: charlie, role: admin }
  set_fact:
    # Group by attribute
    by_role: "{{ users | groupby('role') }}"
    # Create lookup dict
    user_map: "{{ users | items2dict(key_name='name', value_name='role') }}"

Nested JSON Access

- vars:
    config:
      database:
        primary:
          host: db1.internal
          port: 5432
        replicas:
          - host: db2.internal
          - host: db3.internal
  debug:
    msg:
      - "Primary: {{ config.database.primary.host }}"
      - "First replica: {{ config.database.replicas[0].host }}"
      - "All replica hosts: {{ config.database.replicas | map(attribute='host') | list }}"

Combine/Merge Dicts

- vars:
    defaults: { port: 80, workers: 4, debug: false }
    overrides: { port: 8080, debug: true }
  set_fact:
    final_config: "{{ defaults | combine(overrides) }}"
    # Result: { port: 8080, workers: 4, debug: true }
# Deep merge
- set_fact:
    merged: "{{ dict1 | combine(dict2, recursive=true) }}"

dict2items and items2dict

- vars:
    config:
      max_connections: 200
      timeout: 30
      debug: false
  debug:
    msg: "{{ config | dict2items }}"
    # [{ key: max_connections, value: 200 }, ...]
# Filter dict entries
- set_fact:
    numeric_config: "{{ config | dict2items | selectattr('value', 'number') | list | items2dict }}"

Write JSON to File

- copy:
    content: "{{ my_data | to_nice_json }}"
    dest: /opt/myapp/config.json
  become: true

Common Transformations

| Task | Filter | |------|--------| | Parse JSON string | from_json | | To JSON string | to_json / to_nice_json | | Query nested data | json_query('expression') | | Filter list | selectattr('key', 'eq', 'val') | | Extract values | map(attribute='key') | | Sort | sort(attribute='key') | | Merge dicts | combine(other_dict) | | Dict to list | dict2items | | List to dict | items2dict | | Group items | groupby('key') |

FAQ

from_json vs from_yaml?

JSON is valid YAML, so from_yaml works on JSON too. Use from_json for clarity when you know the input is JSON.

How do I handle JSON with special characters?

Use to_json filter which properly escapes special characters.

Related Articles

• Ansible JSON Conversion Guide • Ansible set_fact Guide