Intel GFX CI
What Is It About?
We are continously testing i915 (a Linux kernel device driver for Intel Graphics) in an automated fashion, with a goal of having production ready upstream.
Our CI system is composed of multiple Intel machines spanning many generations and display types. You can find our full hardware list here:
This is the crux of this CI system. We test each patch that goes in our driver (i915) or in the DRM drivers test suite we use (IGT GPU Tools) before it lands in the repository and compare the results with the post-merge baseline (we filter out known bugs). This shifts the cost of integration on the people making the change and helps us avoid time-consuming bisection and reverts.
Since we accept patches through mailing lists, this is where you can find the results - they are sent out as a replies to the original mail. We run BAT and FULL IGT (if BAT is successful). On a usual day you can expect result emails for a BAT within 1 hour and for FULL IGT within 6 hours. You will receive the emails even if the run is successful.
Here are the mailing lists we currently support:
|Mailing List||List Info||Archive||Patchwork|
Trybot mailing list is for trying out changes on our infrastracture before they are ready for a review. It has the lowest priority of all the mailing lists. It supports both IGT and kernel patches.
IGT patches sould have
i-g-t tag in the subject, e.g.
[PATCH i-g-t] My
patch. This can be automated with:
git config --local format.subjectPrefix "PATCH i-g-t"
Testing Combined IGT And Kernel Changes
See Testing Combined IGT And Kernel Changes for details.
Git Trees Tested Post-Merge
We test many trees post-merge. Some of them are our baseline for pre-merge testing (drm-tip and IGT GPU Tools), while the others helps us to make sure that what we submit upstream works and catch any potential issues cased by changes in other drivers/trees before we actually integrate with them.
The testing is sparse, i.e. we poll the branch for changes periodically, and if something has changed we run it through our CI, even if that means multiple commits/merges.
|IGT GPU Tools*||docs||repo|
|Dave Airlie’s branch||???||repo|
*: IGT results are part of the drm-tip visualisation
Results Filtering And Bug Tracking
Bugs caught by our system are filled to the following bug trackers:
We also maintain a set of filters that tie those bugs to failures we see in our CI using machine names/types and patterns in dmesg/test output. This way we are able to filter known issues out of pre-merge results to decrease noise for developers, keep track of bug’s life cycle, and reproduction rate. The tool makes us able to confirm that a supposed fix is indeed fixing things.
You can browse the CI issues with the related data using our tool called cibuglog (updated hourly).
The Kinds Of Runs
Basic Acceptance Tests (aka BAT)
BAT is the most basic run in our portfolio - it consists of tests that help us ensure that the testing configuration is in a working condition. It gates all the sharded runs - if it breaks, then the build is deemed to broken to use more of the CI time on it. It utilizes fast-feedback.testlist, which you can find in the IGT repository.
Full IGT (aka sharded runs)
If the BAT run is succesful, then we continue with the sharded run. Much broader set of tests (everything you can find in IGT filtered through our blacklist.txt) is executed.
Those runs are complementary to the ones above, used mainly to gather extra data and increase coverage. As the name suggests, they are run when CI would be idle otherwise (i.e. there nothing to test for the regular pre-merge/post-merge).
- drmtip - Full IGT but on the same hosts that BAT uses, thus increasing coverege.
- KASAN - same as above but runs with The Kernel Address Sanitizer enabled.
- 100 re-runs - re-run BAT 100 times with fixed IGT and kernel for extra data on flip-floppers.
- IRC: #intel-gfx-ci @ freenode
- Mailing list: i915-ci-infra @ lists.freedesktop.org
- hardware/CI maintainer: Tomi Sarvela - tomi.p.sarvela @ intel.com
- cibuglog/metrics maintainer: Martin Peres - martin.peres @ intel.com