Download a file using an HTTPS proxy via environment variables - Ansible get_url and environment

How to download a file using an HTTPS proxy via environment variables with Ansible?

I'm going to show you a live Playbook with some simple Ansible code. I'm Luca Berton and welcome to today's episode of Ansible Pilot.

See also: Ansible environment Keyword: Set Environment Variables Per Task or Play

Download a file using an https proxy via env variables

• get_url module • http_proxy and https_proxy environment

The easiest way to download a file using an HTTPS proxy is via the get_url Ansible module and the environment variables. You could set the remote proxy via the http_proxy and https_proxy remote environment using the Ansible statement environment. This applies respectively to HTTP and HTTPS connections. The Ansible environment statement could be applied at the task level or play level.

## Playbook

Download a file using an HTTPS proxy via environment variables with Ansible Playbook. The following scenario uses the HTTPS proxy server http://proxy.example.com:3128.

code

---
- name: get_url module with proxy Playbook
  hosts: all
  become: false
  gather_facts: false
  vars:
    myurl: "https://releases.ansible.com/ansible/ansible-2.9.25.tar.gz"
    mycrc: "sha256:https://releases.ansible.com/ansible/ansible-2.9.25.tar.gz.sha"
    mydest: "/home/devops/ansible-2.9.25.tar.gz"
  tasks:
    - name: download file with proxy
      ansible.builtin.get_url:
        url: "{{ myurl }}"
        dest: "{{ mydest }}"
        checksum: "{{ mycrc }}"
        mode: '0644'
        owner: devops
        group: wheel
      environment:
        https_proxy: "http://proxy.example.com:3128"

execution

ansible-pilot $ ansible-playbook -i virtualmachines/demo/inventory download\ file/get_url_with_proxy.yml
PLAY [get_url module with proxy Playbook] *************************************************************
TASK [download file with proxy] *******************************************************************
changed: [demo.example.com]
PLAY RECAP ****************************************************************************************
demo.example.com           : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
ansible-pilot $

idempotency

ansible-pilot $ ansible-playbook -i virtualmachines/demo/inventory download\ file/get_url_with_proxy.yml
PLAY [get_url module with proxy Playbook] *************************************************************
TASK [download file with proxy] *******************************************************************
ok: [demo.example.com]
PLAY RECAP ****************************************************************************************
demo.example.com           : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
ansible-pilot $

before execution

ansible-pilot $ ssh devops@demo.example.com
Last login: Mon Feb 21 14:34:39 2022 from 192.168.251.111
[devops@demo ~]$ ls -al
total 16
drwx------. 4 devops wheel 111 Feb 21 14:29 .
drwxr-xr-x. 4 root   root   35 Jan  5 10:18 ..
drwx------. 3 devops wheel  17 Jan  5 10:22 .ansible
-rw-------. 1 devops wheel 560 Feb 21 14:33 .bash_history
-rw-r--r--. 1 devops wheel  18 Jul 26  2021 .bash_logout
-rw-r--r--. 1 devops wheel 141 Jul 26  2021 .bash_profile
-rw-r--r--. 1 devops wheel 376 Jul 26  2021 .bashrc
drwx------. 2 devops wheel  29 Jan  5 10:18 .ssh
[devops@demo ~]$

after execution

ansible-pilot $ ssh devops@demo.example.com
Last login: Mon Feb 21 14:36:10 2022 from 192.168.251.111
[devops@demo ~]$ ls -al
total 13964
drwx------. 4 devops wheel      140 Feb 21 14:36 .
drwxr-xr-x. 4 root   root        35 Jan  5 10:18 ..
drwx------. 3 devops wheel       17 Jan  5 10:22 .ansible
-rw-------. 1 devops wheel      572 Feb 21 14:35 .bash_history
-rw-r--r--. 1 devops wheel       18 Jul 26  2021 .bash_logout
-rw-r--r--. 1 devops wheel      141 Jul 26  2021 .bash_profile
-rw-r--r--. 1 devops wheel      376 Jul 26  2021 .bashrc
drwx------. 2 devops wheel       29 Jan  5 10:18 .ssh
-rw-r--r--. 1 devops wheel 14280306 Feb 21 14:36 ansible-2.9.25.tar.gz
[devops@demo ~]$

ansible-pilot $ ssh devops@proxy.example.com
[devops@proxy ~]$ sudo su
[root@proxy devops]# cat /var/log/squid/access.log
1645453749.115    598 192.168.251.205 TCP_TUNNEL/200 3952 CONNECT releases.ansible.com:443 - HIER_DIRECT/104.26.1.234 -
1645453770.840    412 192.168.251.205 TCP_TUNNEL/200 3955 CONNECT releases.ansible.com:443 - HIER_DIRECT/104.26.1.234 -
1645453779.292   8442 192.168.251.205 TCP_TUNNEL/200 14304919 CONNECT releases.ansible.com:443 - HIER_DIRECT/104.26.1.234 -
1645454103.403    135 192.168.251.205 TCP_TUNNEL/200 39 CONNECT releases.ansible.com:443 - HIER_DIRECT/104.26.0.234 -
1645454157.121    501 192.168.251.205 TCP_TUNNEL/200 3957 CONNECT releases.ansible.com:443 - HIER_DIRECT/104.26.1.234 -
1645454163.543   6413 192.168.251.205 TCP_TUNNEL/200 14304902 CONNECT releases.ansible.com:443 - HIER_DIRECT/104.26.1.234 -
1645454171.680    497 192.168.251.205 TCP_TUNNEL/200 3948 CONNECT releases.ansible.com:443 - HIER_DIRECT/104.26.1.234 -
[root@proxy devops]#

code with ❤️ in GitHub

Conclusion

Now you know how to use HTTPS proxy using environment variables with Ansible Playbook.

See also: Ansible Set Environment Variables: lineinfile for /etc/environment & .bashrc

Related Articles

• using become for sudo in Ansible • Ansible Inventory Guide • reading environment variables in Ansible playbooks

Proxy Configuration Methods

Method 1: environment in Playbook

- name: Download via proxy
  hosts: all
  environment:
    http_proxy: "http://proxy.example.com:8080"
    https_proxy: "http://proxy.example.com:8080"
    no_proxy: "localhost,127.0.0.1,.internal.com"
  tasks:
    - name: Download file
      ansible.builtin.get_url:
        url: https://example.com/file.tar.gz
        dest: /tmp/file.tar.gz

Method 2: Per-Task Environment

    - name: Download only this task via proxy
      ansible.builtin.get_url:
        url: https://example.com/file.tar.gz
        dest: /tmp/file.tar.gz
      environment:
        https_proxy: "http://proxy.example.com:8080"

Method 3: Group Variables

# group_vars/behind_proxy.yml
proxy_env:
  http_proxy: "http://proxy.example.com:8080"
  https_proxy: "http://proxy.example.com:8080"
  no_proxy: "localhost,127.0.0.1"

- name: Use proxy from group vars
  hosts: behind_proxy
  environment: "{{ proxy_env }}"
  tasks:
    - name: Install package
      ansible.builtin.dnf:
        name: nginx
        state: present

See also: Ansible Environment Variables: Set, Read & Manage env vars (Complete Guide)

FAQ

Does the proxy apply to the control node or target?

The environment directive sets variables on the target host where the task executes. For control-node proxy, set environment variables in your shell or ansible.cfg.

How do I use authenticated proxies?

Include credentials in the URL: http://user:password@proxy.example.com:8080. For security, store the proxy URL in Ansible Vault.

Which modules respect proxy environment variables?

Most modules that make HTTP requests respect http_proxy/https_proxy, including get_url, uri, dnf, apt, and pip.