Intro
I’ve made a lot of mistakes and learned something from every one. I am trynig to pass on some of what I learned.
Limit of 1000 pipeline runs per week
This seems pretty crazy. In my unit we were going all gung-ho and imagining Azure DevOps pipelines as an elegant replacement for running cron jobs on our own linux servers. But something as simple as “I need to run this job every five minutes” seems to be overwhelming for a pipeline. What? Yes there is a hard limit of 1000 pipeline jobs a week. This limit is discussed here.
I wanted a job to run every six minutes, which will still hit that limit. So what I am trynig is to create two pipelines. Each is scheduled to run every 12 minutes. The yaml files are almost the same except in the one I sleep for six minutes. I also needed to remember to re-create the pipeline variables I was using.
Getting the raw logs from your pipeline runs
I wrote a little script to get the raw logs and copy them to a linux filesystem where I can use the linux command-line tools I know and love to examine them in bulk. The main point I wish to share right now is that it is not at all obvious that you need to use the get builds section of the api, not the log section! Who would have guessed? https://learn.microsoft.com/en-us/rest/api/azure/devops/build/builds/get-build-log?view=azure-devops-rest-7.1
Errors I am seeing in my pipeline
[error] We stopped hearing from agent dsc-adosonar-drjohns4servicescoresystems-agent-549c476959-whd72. Verify the agent machine is running and has a healthy network connection. Anything that terminates an agent process, starves it for CPU, or blocks its network access can cause this error. For more information, see: https://go.microsoft.com/fwlink/?linkid=846610
We still need to figure this one out. The error appears only randomly.
I also saw a lot of more subtle errors which amounted to my variables not being defined correctly in the yaml section. Indentation is important! I had variables set up secret environment variables amongst other things. The behavior which results does not always make it obvious what the root cause is.
Don’t run the pipeline for every commit
In your commit comment, put
[skip ci]
somewhere on its own line. This will avoid that the pipeline runs each time you do a commit, which quickly gets annoying.
References and related
How to use the Azure DevOps api to for instance fetch the raw logs
Microsoft’s api documentation pertinent to this topic: