Extracting DMARC reports from MailCow (Dovecot)
or any other attachments, using doveadm and munpack
DMARC reports allow mail server admins to know how emails from their servers are being received. However, receiving the reports via email can make it difficult to see long-term trends and information.
Let's explore how to automate the extraction of DMARC reports from a MailCow server so that reports can be parsed and visualized by software such as parsedmarc. This method can be adapted for any type of attachment or report that you wish to receive and process through an automation pipeline, but my use case is DMARC so that's what I'm sticking with.
The first step in setting up automatic report extraction is having a way to identify the emails containing the DMARC reports. The easiest way is to have a dedicated email account for receiving the reports and then pulling the attachments for every email that lands in that account's inbox, but you could also set up filters in MailCow's settings that direct DMARC emails to a certain folder in an existing mail account.
For simplicity's sake, we'll be pulling all the emails in the Inbox folder of the email account dmarc-reports@example.com.
MailCow uses Dovecot for the SMTP backend, which is where the emails are stored. Dovecot provides a useful utility for interacting with saved emails; assuming that you're using the mailcow-dockerized deployment, you can get the ids of every email in our example inbox by running the following command from a commandline:
docker exec mailcowdockerized-dovecot-mailcow-1 doveadm search -u 'dmarc-reports@example.com' mailbox 'Inbox'
This executes the doveadm command in the dovecot container (the container name can be retrieved from docker ps but is fairly consistent); the main things to pay attention to are the email address that should match the user account of the user receiving the DMARC reports and the mailbox name at the end (Inbox in our case). You should receive output similar to following, each line contains the mailbox guid and the message uid separated by a space:
7a7a1e39178ced6386e002007aac2544 1
7a7a1e39178ced6386e002007aac2544 2
7a7a1e39178ced6386e002007aac2544 3
...
Knowing he ids, we can then use the same utility command to pull the contents of a specific email. For example, to get the text contents of the second email above we can use the following command (note the longer mailbox guid and the shorter message uid being carried over from the output above):
docker exec mailcowdockerized-dovecot-mailcow-1 doveadm fetch -u 'dmarc-reports@example.com' text mailbox-guid "7a7a1e39178ced6386e002007aac2544" uid "2"
We now have everything we need to pull the emails from MailCow, but we aren't quite done yet. DMARC reports are sent as attachments on the email files, so we need to process the email contents in order to extract the reports. Thankfully there's a tool for this, which makes our lives much easier: the aptly named munpack (short for mail unpack).
On Debian and most Debian-based Linux distributions, this is part of the mpack package which contains both munpack and its counterpart mpack. It can be installed using:
apt install mpack
Let's walk through an example of its syntax quickly so that you can adapt it to your use case:
munpack -f -t -C ./output_dir/ /absolute/path/to/input_file.txt
There are a few arguments being passed here:
-fforces replacement if a file with the same name as an extension already exists. In normal cases, this isn't usually desirable in case multiple attachments have the same filename, but for DMARC reports they're generally given unique timestamped names so using the-foption allows us to run the command multiple times while dumping the output to a single directory, which we'll probably end up doing in our automation pipeline.-textracts the text portions of multi-part MIME messages as if they were attachments. We do not want this, but it may prove useful to you later.-C ./output_dir/technically changes the directory that the command is running in. We're using it to set an output directory because themunpackcommand will dump attachment files to whatever directory it's running in./absolute/path/to/input_file.txtis our input file containing the email contents. It doesn't have to use an absolute path, but I would recommend it whenever you use the-Coption because it changes which directory is considered the reference point for relative paths.munpackcan also accept input fromSTDIN, which means we don't even need to save our email files to disk for us to extract the attachments.
Now that we have all the pieces, here's a script that puts it all together to get a list of all emails in a specific folder of an email account, pulls the email contents, and finally extracts the attachments and saves them to a specified output folder:
#!/bin/bash
email_username='dmarc-reports@example.com' # the username of the MailCow user who receives the DMARC reports
email_mailbox='Inbox' # the mailbox containing the DMARC reports
output_dir='/dmarc-reports' # where the DMARC report files will be saved
dovecot_container='mailcowdockerized-dovecot-mailcow-1' # The name of the dovecot docker container
# Get a list of email ids
email_ids="$(docker exec "$dovecot_container" doveadm search -u "$email_username" mailbox "$email_mailbox")"
# Pull emails from email ids and extract attachments
while read guid uid
do
docker exec "$dovecot_container" doveadm fetch -u "$email_username" text mailbox-guid "$guid" uid "$uid" \
| munpack -f -C "$output_dir"
done <<< "$email_ids"
This script can be saved to a file (e.g. pull-dmarc-reports.sh), made executable (e.g. chmod a+x), and then added to an automation pipeline or a scheduled job (e.g. crontab -e).
Now that you have all the DMARC reports neatly stored in a folder, you can feed them through parsedmarc to create analysis friendly aggregate reports and visualize them with a tool such as Grafana. If you want a ready-to-go solution, there's a dmarc-visualizer repo created by Debricked that spins up parsedmarc and Grafana with an ElasticSearch backend and a complete dashboard containing useful visualizations for understanding DMARC metrics and historical data.
Of course, there is a pretty clear security concern here: any email attachment sent to this email address will be downloaded automatically and passed through the pipeline that you set up. I would recommend a virus checker and some basic file validation as a minimum to ensure nothing nefarious is meandering through your pipeline, but your specific security concerns and risk tolerance will be unique to you.
I'm quite happy with this solution overall, which gives me a fairly robust way to automate a task that would otherwise be easy to forget amidst the chaos of life.
