Intro
I wanted to run a job on an Azure DevOps pipeline which did a backup of DNS zones on Cloudflare and write the results, in the form of a compressed tar file, into the ADO repository since everyone on the team has access to it and knows how to make a clone of the repo.
My first attempts produced some stunningly bad results. I was wiping out recently created files in the repo and such. That is very undesirable.
The solution
By “stealing with pride” from colleagues and such, I arrived at this AFAIK working solution. Here is the yaml file.
trigger: none
pool:
name: backup_agents
steps:
# next two lines needed so we can modify the git repo and add our backups
- checkout: self
clean: true
persistCredentials: true
fetchDepth: 1
- script: pip3 install -vvv --timeout 60 -r Cloudflare-backup/requirements.txt
displayName: 'Install requirements'
- script: python3 backup-all-zones.py
displayName: 'Run script'
workingDirectory: $(System.DefaultWorkingDirectory)/Cloudflare-backup
env:
CLOUDFLARE_API_TOKEN: $(cloudflare_api_token)
PYTHONPATH: $(System.DefaultWorkingDirectory)/Cloudflare-backup:$(System.DefaultWorkingDirectory)
- script: |
git config --global http.sslVerify false
git config --global user.email "[email protected]"
git config --global user.name "pipeline"
cd Cloudflare-backup
pwd
ls
git add backups/zones-*
git commit -m "adding todays backup files"
git push origin HEAD:refs/heads/main
schedules:
- cron: "47 23 * * *"
displayName: Run the script at 23:47 UTC
branches:
include:
- main
I’m not exactly where all the magic happens. I think the section at the top that does the self checkout must be important. Then, obviously, there are the git add/git commit -m/git push commands. I do not claim to understand the origin HEAD:refs/heads/main argument to git push. I just copied it from a working example.
And branches: include -main. I’m not sure what this does either.
I need a few more days of testing, to be really certain, but I no longer am reverting my repo to an old state as I was with my initial attempts which involved doing a git fetch and probably missed the self checkout step as well.
Conclusion
One day I hope to understand git. But that today is not today! nevertheless I got my ADO pipeline to add backup files to its own ADO repository! So that’s cool.