I was looking through Reddit today and saw an interesting question about how to manipulate files with Ansible. I took a few minutes to throw together an example of how to convert a relatively simple file, a list of strings broken up by new lines, and output it as JSON. A similar method can be utilized to read any file into a variable in ansible by skipping the JSON parsing steps later in this tutorial.

Doing file conversions or data munging is an important skill to learn when growing in a DevOps position. While this is a fairly basic example, utilize this as a stepping stone to understanding more complex data structures. Starting with a relatively simple scenario like writing a script to convert a txt file to json is a great way to build experience with different data activities. Doing this conversion with Ansible is also a great way to get some baseline experience with a specific DevOps tool's syntax.

Related Articles

Simple Text to JSON Conversion with Ansible

Below is the Ansible code to read in a file name input.txt delimited by newline characters and then do a simple conversion to JSON writing to output.json.


- name: Simple Read file and output JSON
  hosts: localhost
  connection: local 
  become: false
  tasks:
  - name: Read Data File
    set_fact:
      data: "{{lookup('ansible.builtin.file', './input.txt').split('\n')}}"

  - name: Show debug data
    ansible.builtin.debug: var=data

  - name: Write output json file
    copy:
      dest: output.json
      content: "{{ data | to_json }}"

Task By Task Breakdown

The first task reads the input.txt file into a data variable that Ansible can use. It also splits the file into a list via the \n (newline) character.


- name: Read Data File
  set_fact:
    data: "{{lookup('ansible.builtin.file', './input.txt').split('\n')}}"

set_fact can be one of your best friends in Ansible. It allows you to take either data from what you have available in your playbook run, or lookup data and set it to a useable var for future tasks. Here I am calling lookup to read an input.txt file and saving it to a data var.


- name: Show debug data
  ansible.builtin.debug: var=data

ansible.builtin.debug lets you output the values of variables to the terminal during a playbook run. This code should not be utilized anywhere except for testing.


- name: Write output json file
  copy:
    dest: output.json
    content: "{{ data | to_json }}"

Finally, utilizing the built-in copy module, you can feed it a variable, like data, and then pipe that to a to_json jinja2 function which will then write your file out to dest.

The result of running this playbook will look something like this:


["/site/foo/", "/site/bar/"]

Complex Text to JSON Conversion with Ansible

Below is the Ansible code to read in a file name input.txt delimited by newline characters. After reading the file, a new variable is initialized name data_elements. A loop is then run to create new dictionary objects with a key of ‘name' and a value of each line of input.txt. Finally, a copy is run to write the data to disk under output.json.


- name: Complex Read file and output JSON
  hosts: localhost
  connection: local 
  become: false
  tasks:
  - name: Read Data File
    set_fact:
      data: "{{lookup('ansible.builtin.file', './input.txt').split('\n')}}"

  - name: Show debug data
    ansible.builtin.debug: var=data

  - name: Create a new array
    set_fact: 
      data_elements: []

  - name: Loop through lines and add data elements to array
    set_fact:
      data_elements: "{{ data_elements + [{'name': item}] }}"
    loop: "{{ data }}"

  - name: Show debug data
    ansible.builtin.debug: var=data_elements

  - name: Write output json file
    copy:
      dest: output.json
      content: "{{ {'records': data_elements} | to_json }}"

Task By Task Breakdown

Since most of the code here is the same, I will only focus on breaking out the differences.


- name: Create a new array
  set_fact: 
    data_elements: []

The block above sets up the empty data_elements array that we will utilize to map our file lines onto.


- name: Loop through lines and add data elements to array
  set_fact:
    data_elements: "{{ data_elements + [{'name': item}] }}"
  loop: "{{ data }}"

This is setting up a loop to loop over our initial data variable which was read from input.txt. It is appending new dictionaries into a larger array with a key of ‘name' and a value of the item in a loop.


- name: Write output json file
  copy:
    dest: output.json
    content: "{{ {'records': data_elements} | to_json }}"

Finally, we are doing the same write operation as the simple example, but this time we are creating a dictionary with a key of ‘records' and a value of the data_elements array. Basically, you are building a Python dictionary with a key of records and a value of data_elements and passing that to the ansible to_json function via a pipe"

This will have an output that looks like this:


{
    "records":[
      {
          "name":"/site/foo/"
      },
      {
          "name":"/site/bar/"
      }
    ]
}

Convert an Ansible List to JSON

Converting any in memory data structure to JSON is pretty simple in Ansible. Below is a code snippet that will help:


- name: Write output json file
  copy:
    dest: output.json
    content: "{{ your_variable | to_json }}"

Assume that your_variable is a list that contains the data that you would like to convert to JSON. You should be able to do this with most any variable in ansible, though your output results may vary depending on how structured your data is up front. If you have a large text blob in a variable, the to_json function will not automagically parse the data into something useable. If you have a complex dict or list that you would like to convert to JSON and potentially deserialize into a list again, this method works wonders.

Conclusion

As I stated at the beginning of the article, data manipulation in DevOps is a key skill to learn. Hopefully, this helps kickstart someone on their journey into becoming a DevOps engineer!