JMRI Code: Continuous Integration
"Continuous Integration" is the process of rebuilding a system every time it changes, so that you rapidly learn of new problems and are motivated to remove existing ones.
The core of our system is a series of Jenkins jobs that routinely build the code, run checks, and creates downloadable installers. That means that we can and do put changes into our users hands very quickly. There's a CI status page that shows the combined status of that.
We divide these into three groups:
- Mandatory checks that must be OK before the change will be merged.
- Optional checks that provide additional information on the code. These don't have to be completely cleared before a change is merged, but their observations should be considered and their metrics should be improving.
- Independent checks that are run by our Jenkins instance on a periodic basis to look for deeper problems.
Mandatory ChecksWe use three CI engines to test every proposed change entered into our GitHub code repository before it gets merged.
Travis will build any branch that contains a .travis.yml file. The build history for the main JMRI repository is at https://travis-ci.org/JMRI/JMRI.
Travis normally runs several jobs in parallel:
- The "graphical" one, which just runs AllTest with schema checks and intermittent tests disabled to reduce runtime; this also does coverage checks. A screen buffer (not a real screen) is used for all the GUI tests.
- The "static" one, which runs various static code checks such as he checks for valid line-ends, the check for proper ASCII characters, SpotBugs, etc,
- The "headless" one, which runs HeadlessTest with schema tests enabled
- And one that separately and individual runs the tests that are marked as requiring separate runnning.
You can enable Travis CI for your fork (personal GitHub repository) as well. After merging JMRI/master into your own fork, allow Travis CI to monitor your fork, and it will begin testing after your next push to GitHub. Follow the instructions at Travis CI to set up your forks.
You can prevent a Travis CI from running against a specific commit by including "[ci skip]" (including the brackets, but not the quotes) anywhere in the commit message if so desired.
Travis is controlled by the .travis.yml file. Much of the work is done by the scripts/travis.sh script.
GitHub ActionsWe use GitHub Actions to do two things on every pull into our main GitHub repositories.
- Run a series of static checks: Run Spotbugs and the compiler warning checks, make sure Javadocs build cleanly, etc.
- Run the JUnit tests on a Windows machine.
These checks are defined by files in the .github/workflows/ directory.
These normally run both on PRs to the main JMRI/JMRI repository, but also on pushes and PRs to your own repository. If you want to turn that off,
- On the page for your own repository, select "Settings"
- From the left sidebar, select "Actions"
- Pick on option for what can run. The bottom option turns everything off for you locally.
We use the WIP check to allow somebody to mark a PR as "Work in Progress" (WIP) by putting WIP in the title line. The PR will then not be merged until the WIP is removed.
We also run some advisory checkers. Although we don't require that they pass, we strongly recommend that people look at them and try to improve their metrics of test coverage, simplicity and understandability, etc.
CoverallsThe Coveralls check does a deep check of which parts of the code were run during the test steps, and which were left uncovered. This can be very useful information when writing tests to make sure your code is working properly. In general, you should try to increase the code coverage, even if just a bit, with every PR.
Code ClimateThe Code Climate points out places where the code appears to be complex in various ways. (It also provides coverage information, though Coveralls seems to do a better job of that.) We recommend you look at those results and make updates where they make sense, so that things are continuously improving, but not every suggestion it makes is worth it, or even appropriate. We don't require this to be clean before merging.
Independent Checks with JenkinsJMRI uses the Jenkins continuous integration engine for integration and test builds. This includes keeping our web site up to date as changes are committed, building installer packages from development builds, and building final releases.
Our primary Jenkins engine is at http://builds.jmri.org/jenkins/.
Because it's hosted outside the US, we force its Java locale
to US English by setting the
en_US in the master Jenkins
Specific Jenkins results of interest:
- Builds page, showing status of the builds done after every series of commits. Successful builds are followed by builds installers via the Packages job.
SpotBugs page, showing status of the more-extensive
SpotBugs tests done
periodically. There are also pages of the
most recent changes and the
most recent results.
For more information on JMRI's use of SpotBugs, see the separate SpotBugs page.
Jenkins Integration with NetBeans
If you are developing in the NetBeans 7.0 environment, it is possible to integrate feedback from the Jenkins Continuous Integration (CI) engine.
Integration into NetBeans is achieved by executing the following steps:
- Open NetBeans
- Switch to the 'Services' window (shortcut key 'Ctrl+5')
- Right-click (Ctrl-click on Mac) the entry 'Hudson
Builders' and choose 'Add Hudson Instance...'
- In the resulting pop-up, complete the following:
- Auto refresh every X minutes
- 60 (change from the default of 5 to avoid overloading the CI server)
From now on, the current status of the Jenkins builds can be reviewed in the 'Services' window by expanding the 'JMRI' entry under 'Hudson Builders'.