The procps project for a few years has been hosted at Gitorious. With the announcement that Gitorious has been acquired by GitLab and that all repositories need to move there, procps moved along to GitLab. At first I thought it would just be a like for like thing, but then I noticed that GitLab has this GitLab CI feature and had to try it out.
CI here stands for Continuous Integration and is a way of automatically testing your program builds using a bunch of test scripts. procps already has a set of tests, with some a level of coverage that has room for improvement, so it was a good candidate to use for CI. The way GitLab works is they have a central control point that is linked to the git repo and you create runners, which are the systems that actually compile the programs and run the tests. The runners then feed back their results and GitLab CI shows it all in pretty red or green.
The first problem was building this runner. Most of the documentation assumes you are building testing for Ruby. procps is built using C with the test scripts in TCL, so there was going to be some need to make some adjustments. I chose to use docker containers so that there was at least some isolation between the runners and the base system. I soon found the docker container I used (ruby:2.6 as suggested by GitLab) didn’t have autopoint which mean autogen.sh failed so I had no configure script so no joy there.
Now my second problem was I had never used docker before and beyond that it was some sort of container thing so like virtual machines lite, I didn’t know much about it. The docker docs are very good and soon I had built my own custom docker image that had all the programs I need to compile and test procps. It’s basically a cut-down Debian image with a few things like gettext-bin and dejagnu added in.
Docker is not a full system
With that all behind me and a few “oh I need that too” (don’t forget you need git) moments we had a working CI runner. This was a Good Thing. You then find that your assumptions for your test cases may not always be correct. This is especially noticeable when testing something like procps which needs to work of the proc filesystem. A good example is, what uses session ID 1? Generally its init or systemd, but in Docker this is what everything runs as. A test case which assumes things about SID=1 will fail, as it did.
This probably won’t be a problem for testing a lot of “normal” programs that don’t need to dig a deep into the host system as procps does, but it is something to remember. The docker environment looks a lot like a real system from the inside, but there are differences, so the lesson here is to write better tests (or fix the ones that failed, like I did).
The runner and polling
Now, there needs to be communication between the CI website and the runner, so the runner knows there is something for it to do. The gitlab runner has this setup rather well, except the runner is rather aggressive about its polling, hitting the website every 3 seconds. For someone on some crummy Australian Internet with low speeds and download quotas, this can get expensive in network resources. As there is on average an update once a week or so these seems rather excessive.
My fix is pretty manual, actually its totally manual. I stop the daemon, then if I notice there are pending jobs I start the daemon, let it do its thing, then shut it down again. This is certainly not an optimal solution but works for now. I will look into doing something more clever later, possibly with webhooks.
- Gitorious is being acquired by GitLab and gitorious.org will shut down end of May (about.gitlab.com)
- Google To Shutdown Google Code, Concedes To GitHub (phoronix.com)
- GitLab Acquires Gitorious (developers.slashdot.org)