Saturday, November 22, 2014

JGrass-NewAGE history - Version zero and version one

Jgrass-NewAGE (from now on, simply NewAGE) was conceived after the Adige River Authority was requesting a model for t he river Adige to help the managements of droughts. We decide to name the project Nuovo Adige. “Nuovo” since we already implemented a model, almost twenty years before for the same river, and that was the Adige model. Because the English translation of “Nuovo Adige” sounds very much like “new age”, this became soon the name of the model. The first model was implemented mostly by the group head by Alberto Bellin for hydrology and Aronne Armanini for the hydraulic part (I had some part in it, in designing the file organisation required by the model and suggesting some about the geomorphological unit hydrograph approach). The new model, however, had another ancestors: the real-time model operational at the Civil Protection of Province of Trento, also implemented by Alberto Bellin and collaborators, and including a snow model by some of my former students (Hydrologis, and Stefano Endrizzi). 

When I got the project, the first thought I did, was that, I had to abandon my beloved GIUH models for a more general one. We needed to estimate the discharges in multiple points of the river network, while GIUH gives the discharge just at the outlet of a basin, and this claimed, at least, for a generalisation of the usual GIUH for a system where multiple GIUH where used, each for any subbasin. (Well GIUH has also some limitations, but I will go back on this in a future post). 

If one browse the slides presented at the beginning of the project (In Italian) one could also notice that the emphasis of the project was not only on the models of physical processes, either conceptual or physical, but on the whole infrastructure of modelling, including a data base for storing the data (even the geographic data, of input as well as of output), and a visualisation system based on a GIS system (uDig) aimed to grow into a Decision Support System (DSS). 

Since the early nineties, I was in fact intrigued by the image of  a DSS system, found in a lost book of proceedings, which included a database, a system of visualisation, and models, and where all of the needed  concepts where already developed. Twenty years or so later I am still asking myself why the people who envisioned the picture, did not ever actually realised it in practice: but probably it was because a lot of tools (which I spent year to create consistently) were missing. 
Another key in the presentation was the inclusion in modelling of infrastructures like  water intakes, withdraw, dams and any other devices. An explicit treatment of flow in channel through a solid algorithm [Casulli, 1990] solving 1D de Saint Venant equation was also implemented.  
These features were known to be necessary to account for human action which, potentially, during low flows, could withdraw all river water for for irrigation and other uses [a clarification of the concept of Anthropocene at this local scale]. 

After the experience made with GEOtop, it was clear that modelling of such  systems could not anymore be implemented in the classical way as a monolithic program. We therefore look at OpenMI as a system to pursue a strategy of modelling by components.  However, presentation given at CUASHI 2008 biennial meeting (a must read including also some considerations about GEOtop) clarifies these and other design issues. 

From the point of view of the process mathematical descriptions, the plan was to “recycle” the snow model of the real time Adige model, to absorb the GEOtransf model [e.g. Majone et al., 2010] into the picture; to implement the estimation of Penman-Monteith (see also here the lectures by Dara Entekhabi) scheme for evapotranspiration, to recover all the tools of the Horton Machine for the geomorphometric analyses required for basin delineation. An ambitious idea was to include directly in modelling, through the use of GEOtools, the formats suitable for a direct GIS representation. 
Unfortunately, GEOtransf model was not available, due mainly to licensing issues, and we had to change the direction of our efforts, and we grabbed the Chris Duffy model [Duffy, 1996] as implemented in the Mantilla’s CUENCAS model. A couple of spatial interpolator for data measured at hydro-meteo stations were implemented, i.e. an ordinary Kriging, and Just Another Model Interpolator (JAMI), a simple bare-bone robust (which never break) method for making measure available in any catchment point. Finally an original description of the river networks, describing the topology, and governing the order of execution of the various modules, was implemented (here an early view of the watershed description, given as a Poster at EGU-Wien 2008, and here a more mature paper, just on the generalised Pfafstetter numbering in NewAGE).

We can call this above the zero version of NewAGE, which actually did not became never operational. It needed a testing calibration phase that, for various reasons could not be pursued. The financial support from the River Basin Authority terminated, and data base, model components, and whatever developed was closed in a drawer. A big lost occasion to have a new type of model working on River Adige!  What actually remains of version zero has to be archeologically retrieved. But we are doing it.

However, I (we) did not give up. Eventually we abandoned OpenMI in favour of OMS, and the reasons were explained in a previous post. Porting to OMS was done as well by Hydrologis. We also reduced our scope and concentrated just on the model components to improve them and verify their respondence to reality. 
To obtain this goal Giuseppe Formetta, in his start of Ph.D. implemented goodness of fit methods (GoF) and the calibration methods  DREAM and Particle Swarm that became two new OMS components. With them, a systematic analysis of the other components started.
We soon realised that Duffy’s model was not easy to calibrate (or, at that time, we were maybe not experienced in using the tools), and Giuseppe decided to implement a new entire module, based on Hymod. This was successfully accomplished, and the results are reported in Formetta et al, 2011.
Functional to this work was the incremental improvement of the Kriging (.) components, now including, besides Ordinary Kriging, Detrended Kriging, their local versions (accounting not for all the measurements points but just on the next neighbours), and five variograms models that can be fitted automatically at any time step having available data (also made by Giuseppe).

The refocused development of NewAGE was described in a concept paper actually published this year 2014.

The NewAGE zero had radiation simulation, however, the components was entirely rewritten for the short wave radiation on  the basis of Javier Corripio’s work.  The implementation was  initiated by Daniele Andreis, and enhanced, cleaned,  and completed with various contributions on how to estimate radiation attenuation by atmosphere and clouds by  Giuseppe Formetta. All of this work is summarised in this GMD paper (for some rehearsal on solar radiation, you can look at the slides referred here).

On the basis of the work on radiation, components for snow accumulation and melting were built and documented in another paper where the study was concerned about the Cache La Poudre basins close to Fort Collins, Co, another piece of Giuseppe Formetta's Ph.D. thesis

The actual state-of-art of NewAGE  includes also the implementation of a new version of the Penman-Monteith, its FAO counterpart,  and Priestley-Taylor formulas for the estimation of evapotranspiration (also implemented by Giuseppe F.). The potentialities in the latter components were not yet exploited as they could. But we will do it.

At present, therefore, NewAGE is constituted by a a set of components that can be used to simulated the whole hydrological cycle for any basin, from a few square kilometres to continental scale rivers. We can call the actual version JGrass-NewAGE version 1, but, actually is a set of components that can be arranged in various modeling solutions for various analyses that could not be easily obtainable with more traditional models.

The story is continuing and soon other components will be made available to the public, those available now are listed in a previous post, here.  A different view on the same concepts presented here, can be found here (seen from the Visualisation and Informatics perspective) and here (the idea of building a physico-statistical model).  If you resist and look the recent presentation given at Fort Collins you can see a rework of the ideas developed at CUASHI in 2008 and complete your overview.

The source code of the system can be found on Github.


V Casulli, Semi-implicit finite difference methods for the two-dimensional shallow water equations, Journal of Computational Physics 86 (1), 56-74, 1999

Formetta, G.; Mantilla, R.; Franceschi, S., Antonello A., Rigon R., The JGrass- NewAge system for forecasting and managing the hydrological budgets at the basin scale: models of flow generation and propagation/routing, Geoscientific Model Development Volume: 4 Issue: 4 Pages: 943-955, DOI: 10.5194/gmd-4- 943-201, 2011 

Formetta G., Antonello A., Franceschi S., David O. and Rigon R., The informatics of the hydrological modelling system JGrass-NewAge, 2012 International Congress on Environmental Modelling and Software Managing Resources of a Limited Planet, Sixth Biennial Meeting, Leipzig, Germany R. Seppelt, A.A. Voinov, S. Lange, D. Bankamp (Eds.) 2012-proceedings, 2012

Formetta G., Rigon R., Chavez J.L., David O., The short wave radiation model in JGrass-NewAge System, Geosci. Model Dev., 6, 915-928, 2013,

Formetta G., Antonello A., Franceschi S., David O., and Rigon R., Hydrological modelling with components: A GIS-based open-source framework, Environmental Modelling & Software, 5 (2014), 190-200

B Majone, A Bertagnoli, A Bellin, A non-linear runoff generation model in small Alpine catchments,  Journal of hydrology 385 (1), 300-312, 2010

Vrugt, J. A., ter Braak, C. J. F., Diks, C. G. H., Robinson, B. A., Hyman, J. M., Higdon, D., 2009. Accelerating Markov chain Monte Carlo simulation by differential evolution with self-adaptive randomized subspace sampling. International Journal of Nonlinear Sciences and Numerical Simulation 10 (3), 273-290. DOI:10.1515/IJNSNS.2009.10.3.273

No comments:

Post a Comment