Ansible JSON Query: Search & Extract Data with json_query
By Luca Berton · Published 2024-01-01 · Category: installation
How to search and extract JSON data in Ansible. Use json_query (JMESPath), from_json filter, and Jinja2 expressions to parse complex JSON structures.
Introduction
Ansible is a powerful tool used for automation and configuration management. While it is commonly associated with tasks like server provisioning and application deployment, its flexibility allows it to be used for a wide variety of tasks, even something as simple as searching for a specific band member in a list of bands. In this article, we'll create an Ansible playbook that searches for bands formed with a member named "Starr" and outputs the result.
See also: Mastering Ansible-Creator: Simplify Your Ansible Collection Development
Disclaimer
Full example: https://www.redhat.com/sysadmin/ansible-jinja-lists-dictionaries
The Playbook
Below is the Ansible playbook designed for this task. It includes a list of bands, each with its members, formation year, and the decade they were most active. The playbook filters through this list to find any band that has a member named "Starr."
---
- name: List bands formed with a member named Starr
hosts: localhost
gather_facts: no
vars:
bands:
- name: The Beatles
members:
- Lennon
- McCartney
- Harrison
- Starr
formed: 1960
decade: 60s
- name: The Eagles
members:
- Frey
- Henley
- Leadon
- Meisner
formed: 1971
decade: 70s
- name: Run DMC
members:
- Simmons
- McDaniels
- Mizell
formed: 1983
decade: 80s
- name: Red Hot Chili Peppers
members:
- Kiedis
- Smith
- Frusciante
- Balzary
formed: 1982
decade: 90s
- name: Destiny's Child
members:
- Knowles
- Rowland
- Williams
formed: 1990
decade: 00s
- name: Black Eyed Peas
members:
- Adams
- Lindo
- Gomez
formed: 1995
decade: 00s
tasks:
- name: Display bands with a member named Starr
ansible.builtin.debug:
msg: "{{ bands | selectattr('members', 'search', 'Starr') | map(attribute='name') | list }}"Explanation of the Code
- Playbook Structure:
- The playbook targets the
localhostand does not gather facts, as this is a simple data-processing task.
- Variables Section:
- The
bandsvariable is defined as a list containing information about several bands, including their names, members, formation year, and the decade they were active.
- Task:
- A single task is defined that uses the
ansible.builtin.debugmodule to filter through thebandslist and find any band with a member named "Starr." - The filtering is done using Jinja2 templating with the
selectattrfilter, which searches for "Starr" within thememberslist of each band.
Running the Playbook
To run this playbook, save it as list_bands.yml and execute the following command in your terminal:
ansible-playbook list_bands.ymlExpected Output
When the playbook is executed, the output will look like this:
PLAY [List bands formed with a member named Starr] *********************************************************************************************
TASK [Display bands with a member named Starr] *************************************************************************************************
ok: [localhost] => {
"msg": [
"The Beatles"
]
}
PLAY RECAP *************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0Analysis of the Output
As shown in the output, the playbook successfully found that "The Beatles" is the band with a member named "Starr." The playbook executed successfully, with Ansible confirming that the task was "ok," meaning it completed as expected without any changes, failures, or other issues.
Conclusion
This example Playbooknstrates how Ansible's flexibility extends beyond traditional IT automation tasks. By leveraging Ansible's powerful templating system, you can automate various types of data processing tasks. This playbook, while simple, showcases the potential of Ansible for managing and filtering data, opening up a wide range of possibilities for its use in unconventional automation tasks.
Basic json_query
- vars:
users:
- { name: alice, role: admin, active: true }
- { name: bob, role: user, active: false }
- { name: charlie, role: admin, active: true }
debug:
msg:
- "All names: {{ users | json_query('[].name') }}"
# ['alice', 'bob', 'charlie']
- "Admins: {{ users | json_query('[?role==`admin`].name') }}"
# ['alice', 'charlie']
- "Active admins: {{ users | json_query('[?role==`admin` && active].name') }}"
# ['alice', 'charlie']See also: Creating Ansible Collection Using ansible-creator and VS Code Ansible Extension
Nested Data
- vars:
infrastructure:
regions:
- name: us-east-1
servers:
- { hostname: web1, type: t3.micro }
- { hostname: web2, type: t3.small }
- name: eu-west-1
servers:
- { hostname: eu-web1, type: t3.micro }
debug:
msg:
- "{{ infrastructure | json_query('regions[].servers[].hostname') }}"
# [web1, web2, eu-web1]
- "{{ infrastructure | json_query('regions[?name==`us-east-1`].servers[].hostname') }}"
# [web1, web2]API Response Parsing
- uri:
url: https://api.example.com/v1/instances
return_content: true
register: api
- set_fact:
running_instances: "{{ api.json | json_query('instances[?state==`running`].{id: id, ip: private_ip, type: instance_type}') }}"
instance_count: "{{ api.json | json_query('length(instances[?state==`running`])') }}"See also: Ansible Debugger: Interactive Debug & Troubleshoot Playbooks
JMESPath Operations
- vars:
servers:
- { name: web1, cpu: 4, ram: 8 }
- { name: web2, cpu: 2, ram: 4 }
- { name: db1, cpu: 8, ram: 32 }
debug:
msg:
# Sort
- "{{ servers | json_query('sort_by(@, &cpu)[].name') }}"
# [web2, web1, db1]
# Max/Min
- "{{ servers | json_query('max_by(@, &ram).name') }}"
# db1
# Sum
- "{{ servers | json_query('sum([].cpu)') }}"
# 14
# Slice
- "{{ servers | json_query('[0:2].name') }}"
# [web1, web2]Comparison Operators
| Operator | Example | ||||
|---|---|---|---|---|---|
== | [?status==\active\] | ||||
!= | [?status!=\deleted\] | ||||
> | [?cpu>\4\] | ||||
>= | [?ram>=\8\] | ||||
&& | [?role==\web\ && active] | ||||
\ | \ | [?env==\prod\ \ | \ | env==\staging\] | |
! | [?!disabled] |
Pipe Expressions
# Chain operations with |
- debug:
msg: "{{ data | json_query('[?active] | sort_by(@, &name) | [].name') }}"json_query vs Jinja2 Filters
| Use Case | json_query | Jinja2 |
|---|---|---|
| Simple attribute | ✓ | ✓ map(attribute=) |
| Nested filtering | ✓✓✓ | Complex |
| Aggregation (sum, max) | ✓✓ | sum(), max() |
| Sorting | ✓✓ | sort(attribute=) |
| Readability | Better for complex | Better for simple |
FAQ
"JMESPath requires jmespath" error?
pip install jmespathWhy use backticks in json_query?
JMESPath uses backticks for literal values: [?name==\alice\]. In Ansible YAML, escape them or use different quote styles.
Can I use json_query in when conditions?
when: (data | json_query('[?status==`error`]') | length) > 0Basic json_query
- vars:
users:
- { name: alice, role: admin, active: true }
- { name: bob, role: user, active: false }
- { name: charlie, role: admin, active: true }
debug:
msg: "{{ users | json_query('[?role==`admin`].name') }}"
# ["alice", "charlie"]Install JMESPath
pip install jmespathCommon Queries
# Select all names
"{{ data | json_query('[].name') }}"
# Filter by condition
"{{ data | json_query('[?status==`active`]') }}"
# Nested access
"{{ data | json_query('servers[].network.ip') }}"
# First match
"{{ data | json_query('[?name==`web1`] | [0]') }}"
# Multiple fields
"{{ data | json_query('[].{host: name, addr: ip}') }}"Parse JSON from Command Output
- command: curl -s https://api.example.com/status
register: api_response
- set_fact:
status: "{{ api_response.stdout | from_json }}"
- debug:
msg: "Version: {{ status.version }}, Healthy: {{ status.healthy }}"Complex Nested JSON
- vars:
infrastructure:
datacenters:
- name: us-east
clusters:
- name: prod
servers: [{ name: web1, cpu: 4 }, { name: web2, cpu: 8 }]
- name: staging
servers: [{ name: stg1, cpu: 2 }]
- name: eu-west
clusters:
- name: prod
servers: [{ name: eu-web1, cpu: 4 }]
debug:
msg: "{{ infrastructure | json_query('datacenters[].clusters[].servers[].name') }}"
# ["web1", "web2", "stg1", "eu-web1"]Filter with Comparison
# Greater than
"{{ servers | json_query('[?cpu>`4`].name') }}"
# Contains
"{{ servers | json_query('[?contains(name, `web`)]') }}"
# Not equal
"{{ servers | json_query('[?status!=`offline`]') }}"
# AND conditions
"{{ servers | json_query('[?cpu>`2` && status==`active`].name') }}"
# OR (use pipe)
"{{ servers | json_query('[?role==`web` || role==`api`]') }}"json_query vs selectattr
# selectattr — simple single-attribute filter
"{{ users | selectattr('role', 'eq', 'admin') | list }}"
# json_query — complex nested queries
"{{ data | json_query('departments[?budget>`100000`].teams[?size>`5`].lead') }}"
# Use selectattr for simple, json_query for complexRead JSON File
# Local file
- set_fact:
config: "{{ lookup('file', 'config.json') | from_json }}"
# Remote file
- slurp: { src: /etc/myapp/config.json }
register: raw
- set_fact:
config: "{{ raw.content | b64decode | from_json }}"FAQ
json_query returns empty?
Check JMESPath syntax — backticks for literals: [?name==\alice\] not [?name=='alice'].
Can I use json_query without JMESPath?
No — json_query requires the jmespath Python library. Alternative: use Jinja2 selectattr, map, and reject filters.
How to pretty-print JSON?
- debug: msg="{{ my_dict | to_nice_json }}"
- copy:
content: "{{ my_dict | to_nice_json }}"
dest: /tmp/output.jsonRelated Articles
Category: installation