JMRI Code Swarm
Download the Video
You can download the 2009 JMRI Code Swarm here. Note that this is a large file (~15M) and is in zip format. After downloading, you will need to unzip it to extract the mpeg file, then play it in your favorite video player. It has been tested on Windows XP with Windows Media Player(V9) and Irfanview, on Ubuntu with Totem (after installing the Gstreamer codec and its plugins) as well as on the Mac.
The video is © copyright JMRI and the musical track is copyright AdHoc, see license details.
Introduction
What the heck is a Code Swarm?
Code Swarm is an open source visualization package that was released in 2008. It can be used to provide a dynamic video history of a software project, in this case JMRI.
Each time a developer "commits" (i.e. saves) a new file or a change to an existing one, a color-coded dot appears on the screen. The dots coalesce around the developer's name and the names move around the screen as others join in. The file dots fade away if there is no further activity on them, or flare up if changed again. The names fade as well if that developer has no more activity, or change to bold font if more contributions are made.
Although it sounds simple, it can be quite mesmerizing as the developers and files swirl around and it gives a very clear view of the increasing JMRI activity, especially over the last two years.
The resulting video gives a lively perspective of how JMRI has developed and who has been involved. The 4 minute JMRI Code Swarm covers the beginning, with the very first commits in June of 2001, through to early January 2009, a period of about seven and a half years and encompassing nearly 23,000 commits.
Here is an animated gif sample of about 1.5 secs from the video (covers about 2 weeks in April 2008):
After viewing the JMRI code swarm, one thing is abundantly clear, and that is the great debt we owe Bob Jacobsen. The large number of commits he has made is very evident and is an order of magnitude higher than any other developer. Knowing how quick Bob is to highlight others' contributions, I'm sure he would be the first to point out that his commits also include those of many others on the Acknowledgements list who submit files or patches and don't commit directly. Unfortunately, the code swarm can't separate out these contributions and attribute them to the true originators, but even if it could, Bob's own contributions are still huge.
Enjoy!
Also view our 2017 CodeFlower.
Technical Details
Warning! Any non-Geeks who have wandered into this section by mistake should stop now before any permanent damage is done to your intellect or personality. Proceeding further may result in glazed eyes, confusion and disorientation. Serious cases could result in an all-consuming interest in programming with the accompanying loss of social skills and deterioration in personal hygiene. Do not read while operating heavy machinery. Reader discretion is advised.
Data Source
- All of this was done in Windows XP, but the process will be essentially identical for other platforms.
- First, the project repository HEAD is checked out of CVS into a local directory. This can be done as a read-only version as described here. For Windows based systems, an excellent tool for this is TortoiseCVS
- Next, the commit data is extracted from the local repository into log files. This is obtained by issuing
the command line statement:
cvs log > activity.log
in each of the following subdirectories of the repository:- help
- java
- jython
- lib
- resources
- web
- xml
- The
activity.log
files now in each subdirectory have to be combined into a single file for processing. Pick one of them as the master and save a copy so you can go back if you make a mistake while combining the files. Then open each of the other files in turn and copy the entire file contents with the exception of the first two lines. (The first line contains the file name and the second line is blank). Paste the copied contents at the end of the master file and repeat for each of theactivity.log
files.
Code Swarm Frame Generation
- See this page for instructions on downloading the Code Swarm software using an SVN client and building from source. I use Eclipse, so I downloaded the Eclipse module from SVN and simply ran the software from within Eclipse.
- Once the Code Swarm software is ready and the log files have been combined, the video generation
process can begin.
The basic process described
here is used to generate the code swarm frames. The following steps give more detail and assume
the code swarm software has been downloaded into a directory called
codeswarm
:- Convert the master
activity.log
file to a properly formatted xml file. To do this, copy it to thecodeswarm/convert_logs
directory. (Note: Python must be installed and working before proceeding. If you don't have it, install it now). In a command session, change to theconvert_logs
directory and issue the command:
python convert_logs.py -c activity.log -o activity.xml
This will generate a file calledactivity.xml
in thecodeswarm/convert_logs
directory - This step is optional. Open the
activity.xml
file in a text editor and do a Replace All for each user id to replace it with the committer's name. The names are available on our SourceForge site. - One statement in the source code for code_swarm can be changed to make the Legend
more visible, especially when compressed video versions of the swarm will be generated.
In the
code_swarm.java
file, find thedrawLegend()
method and change the line
textFont(font)
to
textFont(boldFont)
- The code swarm config file
sample.config
, which comes with the program in thecodeswarm/data
directory, has the following options changed (all others are left at default):
InputFile=convert_logs/activity.xml
comment out:
FontSize=14
BoldFontSize=18MillisecondsPerFrame=21600000
uncomment and set:FramesPerDay=2
comment out all other ColorAssign statements
ColorAssign1="Documentation","(.*doc.*)|(.*htm.*)|(.*html.*)|(.*shtml.*)|(.*css.*)|(.*properties.*)", 0,0,255, 0,0,255
ColorAssign2="Java/Jython Files","(.*java.*)|(.*py.*)", 255,255,0, 255,255,0
ColorAssign3="Resource Files","(.*xml.*)|(.*dtd.*)|(.*gif.*)|(.*xsl.*)|(.*png.*)|(.*jpg.*)", 255,0,0, 255,0,0TakeSnapshots=true
DrawFilesSharp=true
DrawFilesFuzzy=false
DrawFilesJelly=true
ShowEdges=true
EdgeLength=50
PersonSpeed=1.50
PersonMass=20.0
HighlightPct=10
- Convert the master
- The
ColorAssign
statements above in the config file color-codes the files to be displayed as:
- Documentation
- in Blue; files: all help files, but not web site (doc, html/css) and language translations(properties)
- Java/Jython
- in Yellow; files: all program files (java) and scripts (py)
- Resources
- in Red; files: all decoder and programmer definitions (xml/dtd/xsl) and images (gif, png, jpg)
All other files types are considered Misc and are shown in gray.
- Create an empty folder under the
codeswarm
directory calledframes
- Execute
code_swarm
, which will open a graphics window showing the frames as they are generated and saved in thecodeswarm/frames
directory. They are saved as a series of.png
files, with names in the form ofcode_swarm-#####.png
where ##### starts at 0000 and counts up to the number of frames. This naming format is required for the benefit of the software that is used to convert the series of still pictures into a video.
Music
- If music is desired obtain it now.
- For this video, the musical track used is Sumbala from the album Toutes directions, copyright AdHoc and used under the Creative Commons license (attribution; non-commercial; share-alike). The music can be found at Jamendo.
Video Generation
- After
code_swarm
has created the frames, ffmpeg can be used to combine the frames and the mp3 file of the music into a single mpeg file. For Windows, a pre-compilied binary version is available here. It needs to be unzipped to a folder before use (for the purposes of these instructions, the folder name is assumed to beffmpeg
). - Copy the
codeswarm/frames
directory to theffmpeg/bin
directory. Also, copy the mp3 (sumbala.mp3
in this case) file to the sameffmpeg/bin
directory. Copying these files isn't really necessary but it keeps the path names short in the following command. - In a command session, change to the directory containing the
ffmpeg/bin
directory and issue the following command:
ffmpeg -i sumbala.mp3 -f image2 -r 24 -i frames\code_swarm-%05d.png -qmax 15 jmricodeswarm.mpg
This creates an mpeg file calledjmricodeswarm.mpg
in the same directory that is a combination of the video (made from the individual frames) and the musical sound track. - The quality of the video can be changed by modifying the
-qmax 15
setting in the above command. This value was chosen to give a balance between quality and file size (decreasing-qmax 15
improves quality but increases file size). For a high quality but much larger file, change-qmax 15
to-sameq
Also view our 2017 CodeFlower.