"Continuous Integration" is the process of rebuilding a system every time it changes, so that you rapidly learn of new problems and are motivated to remove existing ones.
The core of our system is a series of GitHub Actions and Jenkins jobs that routinely build the code, run checks, and creates downloadable installers. That means that we can and do put changes into our users hands very quickly. There's a CI status page that shows the combined status of that.
We divide these into three groups:
These checks can be run locally by developers before uploading changes to Github, see Running JUnit Tests.
We use multiple GitHub actions to test every proposed change entered into our GitHub code repository before it gets merged.
These checks are defined by files in the .github/ directory.
They normally run both on PRs to the main JMRI/JMRI repository,
but also on pushes and PRs to your own repository.
If you want to turn that off,
Note that if you turn this off, the "Actions" tab won't show in the GitHub web interface until you turn it back on.
This runs "AllTest", our test suite of over 30,000 JUnit tests on a Windows server. A screen buffer (not a real screen) is used for all the GUI tests. All tests must pass.
Reruns the JUnit test suite in "java.awt.Headless=true" mode to ensure that the parts of JMRI that are meant to run without a GUI really can. This is about 80% of the test suite. These are run on Linux. All tests must pass.
About a hundred JUnit graphical tests are run separately from AllTest to ensure they have a clean environment. These are run on Linux. All tests must pass.
This runs a series of static analysis checks:
Any errors or warnings fail this step.
There are a few additional actions that are used as process controls:
We also run advisory checks on every pull request (PR). Although we don't require that they have zero warnings, we strongly recommend that people look at them and try to improve their metrics of test coverage, simplicity and understandability, etc.
The Code Climate points out places where the code appears to be complex in various ways. (It also provides coverage information, though JaCoCo seems to do a better job of that.) We recommend you look at those results and make updates where they make sense, so that things are continuously improving, but not every suggestion it makes is worth it, or even appropriate. We don't require this to be clean before merging.
Code Climate is controlled by the .codeclimate.yml file.
JMRI uses the Jenkins continuous integration engine for integration and test builds. This includes keeping our website up to date as changes are committed, building installer packages from development builds, and building final releases.
Our primary Jenkins engine is at https://builds.jmri.org/jenkins/. Because it's hosted
outside the US, we force its Java locale to US English by setting the LC_ALL
environment variable to en_US in the master Jenkins configuration settings.
Specific Jenkins results of interest:
For more information on JMRI's use of SpotBugs, see the separate SpotBugs page.
If you are developing in the NetBeans 7.0 environment, it is possible to integrate feedback from the Jenkins Continuous Integration (CI) engine.
Within the NetBeans environment, reference is made to the Hudson CI engine - Jenkins is a "fork" of the Hudson code and supports the same API.
Integration into NetBeans is achieved by executing the following steps:
From now on, the current status of the Jenkins builds can be reviewed in the 'Services' window by expanding the 'JMRI' entry under 'Hudson Builders'.
ArchitectureTest
contains rules which seek to maintain and improve the overall system architecture of JMRI.
For example, the server classes should not depend on code within the system connection hardware types.
noClasses()
.that().resideInAPackage("jmri.jmris")
.should().dependOnClassesThat().resideInAPackage("jmri.jmrix..");
ArchitectureCheck uses FreezingArchRule to collect issues in the archunit_store directory.
Should new issues be encountered, they'll be reported as errors.
Please fix them rather than committing a larger store.
The Test classes also undergo an Architecture Test, TestArchitectureTest.
These tests can be run with
mvn -Dtest=jmri.ArchitectureTest,jmri.TestArchitectureTest,jmri.util.FileLineEndingsCheck test
Cucumber Tests allow
expected software behaviours to be specified in a logical language.
These are used to test the Web Server output ( in Firefox and Google Chrome ),
along with Application launches using simulation profiles for various hardware systems.
The Features and Step Definitions files are in the /java/acceptancetest folder.
The ant command runs all tests with the class name RunCucumberIT.class ,
so both the apps and jmri tests are performed.
ant cucumber