KDE: SoK: Revamp the Continuous Integration system progress report.

jenkins ci prototype

KDE jenkins ci prototype

Wow. What a journey. Poor CMake, my Guinea pig, while I figured out how to get it all working.

My quest is to accomplish : The continuous integration system is a piece of KDE.org infrastructure which compiles KDE software, executes tests included with the software and performs software quality tests. This is done when developers make commits to git.kde.org, and is used to ensure our software is in a releasable state and to detect regressions which are accidentally introduced from time to time. It can be found at build.kde.org. As part of this project you will be revamping the system to allow us to expand it to cover additional operating systems in the future, standardize the configuration of jobs and to improve the flexibility of the builds we perform. In addition you’ll also be responsible for setting up a new foundation for our Linux builds as the current one is running on a distribution which will soon be out of date.

Expected results: The CI system is prepared for future expansion, is able to use a new format for dependency and branching information, and is optionally able to communicate with Reviewboard.

Knowledge Prerequisite: Object oriented programming is mandatory. Student should also be aware of how to build KDE software by hand (using CMake). Familiarity with Python is a significant plus, as well as the JSON, XML and YAML data storage formats. The student should ideally be comfortable accessing a system over SSH, installing packages there and compiling software remotely without a graphical terminal.

I met all of the prerequisites, so I thought it would be a breeze. Hah. Thankfully, I love a challenge and this has been a super learning experience.

While doing my research I encountered this lovely concept called docker. It looks to be the answer for many of the above mentioned requirements, so I took the dive to learn docker. This being a large hurdle, as I have never worked with containers of any kind before… My first hurdles while learning docker was my data went poof every time I needed a new build, from a missing dependency or missed tool! Oh goodness, this will not do, in come volumes. Of course, that was a challenge until I figured out the UID and GID did not match. Volumes will be necessary with my future plans, so it all worked out :). Now I love docker so much, I for see myself containerizing everything!! I have now successfully built cmake+qt5+ECM+KF5 tier 1. This concludes Prototype 1.

Next steps:
Solution for Docker + SSH + rsync currently in scripts. Docker by design does not keep the same IP and I had to map the Hostname + IP on my slave. As this will change on every restart, this is not ideal.

I am certain I want to set up multi config for all build to accommodate MULTIOS and adjust the python scripts to check the OS and load config script accordingly.
Though I still need to figure out multi-config in itself as you can see kwindowsystem was an utter failure.

Compile all of KDE.

Future steps:
I want to break out each app into its own container which will satisfy: standardize the configuration of jobs and to improve the flexibility of the builds we perform.


Anything else I can think of to make this a great CI system.

A big thank you to Ben Cooksley, my mentor 🙂