Introduction
AAP 2.6 significantly improves Kafka integration for Event-Driven Ansible with support for multiple topics, wildcard patterns, and better error handling. This makes Kafka-driven automation more practical for real-world enterprise environments.
What's New for Kafka in AAP 2.6
Multiple Topic Support
You can now subscribe to multiple Kafka topics in a single rulebook:
``yaml
sources:
- name: multi_topic_listener
type: ansible.eda.kafka
args:
host: kafka.example.com
port: 9092
topics:
- infrastructure.alerts
- application.deployments
- security.incidents
group_id: eda-consumer-group
`
Wildcard Pattern Matching
Use wildcards to subscribe to topic patterns:
`yaml
sources:
- name: infra_events
type: ansible.eda.kafka
args:
host: kafka.example.com
port: 9092
topic: "infrastructure.*"
group_id: eda-infra
`
This matches all topics starting with infrastructure.:
- infrastructure.cpu_alerts
- infrastructure.disk_warnings
- infrastructure.network_events
Improved Error Handling
Enhanced connection resilience with:
- Automatic reconnection on broker failures
- Configurable retry logic
- Better error reporting in EDA logs
- Graceful handling of topic deletion
Example: Multi-Topic Event Processing
`yaml
---
- name: Infrastructure event processor
hosts: all
sources:
- name: kafka_events
type: ansible.eda.kafka
args:
host: kafka-cluster.example.com
port: 9092
topics:
- monitoring.critical
- monitoring.warning
group_id: eda-monitoring
rules:
- name: Handle critical alerts
condition: event.topic == "monitoring.critical"
action:
run_job_template:
name: "Critical Remediation"
organization: "SRE Team"
- name: Log warning events
condition: event.topic == "monitoring.warning"
action:
run_job_template:
name: "Warning Logger"
organization: "SRE Team"
`
Architecture Considerations
Consumer Groups
Use consumer groups to distribute event processing across multiple EDA instances:
`yaml
Instance 1 and Instance 2 share a consumer group
Events are load-balanced between them
args:
group_id: eda-shared-consumers
`
Topic Design
Design your Kafka topics with EDA wildcard support in mind:
`
Good: hierarchical naming enables wildcards
infrastructure.network.alerts
infrastructure.compute.alerts
infrastructure.storage.alerts
Less useful: flat naming
network-alerts
compute-alerts
storage-alerts
``
Best Practices
1. Use consumer groups for high-availability EDA deployments
2. Design hierarchical topics to leverage wildcard subscriptions
3. Monitor consumer lag to ensure EDA keeps up with event volume
4. Set appropriate timeouts for rulebook actions
5. Test with high volume before production deployment