Accessibility navigation

Workflow automation for cycling systems

Oliver, H., Shin, M., Matthews, D., Sanders, O., Bartholomew, S. ORCID:, Clark, A., Fitzpatrick, B., van Haren, R., Hut, R. and Drost, N. (2019) Workflow automation for cycling systems. Computing in Science and Engineering, 21 (4). pp. 7-21. ISSN 1521-9615

Full text not archived in this repository.

It is advisable to refer to the publisher's version if you intend to cite from this work. See Guidance on citing.

To link to this item DOI: 10.1109/MCSE.2019.2906593


Complex cycling workflows are fundamental to numerical weather prediction (NWP) and related environmental forecasting systems. Large numbers of jobs are executed at regular intervals to process new data and generate new forecasts. Dependence between these forecast cycles creates a single never-ending workflow, but NWP workflow schedulers have traditionally ignored this—at the cost of efficiency when running “off the clock”—by enforcing a simpler nonoverlapping sequence of single-cycle workflows. Cylc (“Silk”)1 –3 is designed to manage infinite cycling workflows efficiently even after delays in real-time operation, or in historical runs, when cycles can typically interleave for much-increased throughput. Cylc is not actually specialized to environmental forecasting, however, and cycling workflows may also be useful in other contexts. In this paper, we describe the origins and major features of Cylc, future plans for the project, and our experience of Open Source development and community engagement.

Item Type:Article
Divisions:No Reading authors. Back catalogue items
Science > School of Mathematical, Physical and Computational Sciences > NCAS
Science > School of Mathematical, Physical and Computational Sciences > Department of Meteorology
ID Code:105320

University Staff: Request a correction | Centaur Editors: Update this record

Page navigation