Tuesday, December 28, 2021

Edward Wilson (1929-2021) - A storm in the Amazon

From "The diversity of life" 

"I sorted the memories this way and that in hope of stumbling on some pattern not obedient to abstract theory of textbooks. I would have been happy with any pattern. The best of science doesn't consist of mathematical models and experiments, as textbooks make it seem. Those come later. It springs fresh from a more primitive mode of thought, wherein the hunter's mind weaves ideas from old facts and fresh metaphors and the scrambled crazy images of things recently seen. ...


The storm grew until sheet lightning spread across the western sky. The thunderhead reared up like a top-heavy monster in slow motion, tilted forward, blotting out the stars. The forest erupted in a simulation of violent life. Lightning bolts broke to the front and then closer, to the right and left, 10,000 volts dropping along an ionizing path at 800 kilometers an hour, kicking a countersurge skyward ten times faster, back and forth in a split second, the whole perceived as a single flash and crack of sound. The wind freshened, and rain came stalking through the forest. In the midst of chaos something to the side caught my attention. The lightning bolts were acting like strobe flashes to illuminate the wall of the rain forest. At intervals I glimpsed the storied structure: top canopy 30 meters off the ground, middle trees spread raggedly below that, and a lowermost scattering of shrubs and small trees. The forest was framed for a few moments in this theatrical setting. Its image turned surreal, projected into the unbounded wildness of the human imagination, thrown back in time 10,000 years. Somewhere close I knew spear-nosed bats flew through the tree crowns in search of fruit, palm vipers coiled in ambush in the roots of orchids, jaguars walked the river's edge; around them eight hundred species of trees stood, more than are native to all of North America; and a thousand species of butterflies, 6 percent of the entire world fauna, waited for the dawn.

…..

The storm arrived, racing from the forest's edge, turning from scattered splashing drops into sheets of water driven by gusts of wind. It forced me back to the shelter of the corrugated iron roof of the open-air living quarters, where I sat and waited with the mateiros. The men stripped off their clothing and walked out into the open, soaping and rinsing themselves in the torrential rain, laughing and singing. In bizarre counterpoint, leptodactylid frogs struck up a loud and monotonous honking on the forest floor close by. They were all around us. I wondered where they had been during the day. I had never encountered a single one while sifting through the vegetation and rotting debris on sunny days, in habitats they are supposed to prefer. "

Edward O. Wilson, The Diversity of life, Penguin

Monday, December 27, 2021

DARTH4MED - A Digital eARth Twin of Hydrology for the prediction of water scarcity in the Mediterranean area


The DARTH4MED, D4M for short, project aims to be a high resolution twin of the hydrology and carbon cycle of the Italian peninsula. It is based on Po, WATZON and WATERSTEM projects, making treasure of previous modelling efforts like GEOtop and the GEOframe system, and GIS tools implementations like Jgrass and the Horton Machine toolbox. It builds upon state-of-art hydrological modelling case studies of various catchment sizes, from hillslope to Po and Blue Nile. It also draws on experiences in IT applied to hydrology with developments of the object modelling system, OMS.

D4M gives substance, both technical and scientific, to the Digital Earth metaphor and exploits it to improve the work of scientists and professionals, and to support open science. It aims to provide a shared infrastructure usable by scientists and users to investigate the processes involved in the water, energy and carbon budgets, WB, EB and CB, at a very fine spatial and temporal scale, 1 km2, hourly.

The GEOframe system already contains a sophisticated and complete set of modelling components, constituting a solid basis of comparison for innovative developments. Open API and training will be offered to anyone to advance the mathematical, statistical and numerical descriptions of hydrological and eco-hydrological processes with little programming effort. From this perspective, the project will be an experiment in participatory science, since the tools developed could be improved and given back by collaborative researchers. The method of multiple hypothesis testing will be the rule of scientific endeavour.

The core of the system will manage the interactions of groundwater, vadose zone, surface water, snow, vegetation, atmosphere, usually analyzed separately, and join them seamlessly in the continuum containing the feedbacks among the parts. On these bases researchers will be able to evaluate climate, hydrologic, pedological, ecological droughts.




D4M has several primary objectives, listed below:
  • To provide the core of a DE, defined as a Digital eARth Twin Hydrology system (a DARTH), to do hydrology by computer, with an infrastructure that allows partecipative hydrology and makes Earth system science practice easier for all the Italian Peninsula.
  • To improve the modelling of the water budget, WB, energy budget, EB, Vegetation and Carbon Cycle.
  • To provide forecasts for several variables, as detailed in the Synopsis.
  • To resolve some research questions, as presented in the Synopsis.
  • To give researchers sound tools on which to base their analysis of climate, hydrologic, pedological, ecological and agronomic droughts.
  • To provide a high level of abstraction and encapsulation for modelling services, so to allow improvements to parts of the DARTHs by anyone without disrupting the whole.
  • To give API and web services to final users, researchers, technical professionals, programmers, to connect their studies and products to the whole D4M, thus combatting the fragmentation of hydrological modelling through a participatory open platform.
Besides efficient algorithms, the effort will require the smart implementation of parallel computing infrastructures, which will remain mostly invisible to the users. All the infrastructure will be open source, built with open source tools and provided with open data.

The project was just submitted for the FIS call. Here below you find the proposal and the relevant annexes.
Compressing all the ideas in such a few words was quite difficult and the platform on which we had to upload the material with some issues (non accepting, for instance "()[]-/" and other characters. Some requirements quite stupid. The selection will be great. I obviously think that the gain for the country with such a project really great. Finger crossed and, if there are better projects, hope they'll win. 

Wednesday, December 15, 2021

DARTHs (Digital eARth Twin Hydrology systems)

There is a great hype about Digital Earth Twins (DETs) and EU, ESA, NASA and other institutions issue calls for building such IT infrastructures (ITI). This paper face the topic from a point of view of hydrologists who are concerned with the science content of these ITI. The Authors see in DETs an opportunity to make easier the work of scientists and professionals. However they claim that some aspect of making science should be respected. Mainly they are the hypothesis testing and estimation of errors in hindcasting or forecasting. Beside, the Authors claim that building a DET for Hydrology (called DARTH) is an enterprise that implies some choices about the implementation of models and of the infrastructure. DARTHs are not in fact just "models" and have requirements that need to be satisfied. Finally the Authors support the opinion of an open science oriented implementation of these ITI that also allows the participatory action of all the scientists that like to contribute (and look with suspect to science processing where just a few contribute to the core science). In turn also this options has requirements that should be reflected in the implementation.
To sum up, the Authors think that this is the right moment to push these ideas and desire to open a discussion with other colleagues. 

Final published paper: 



You can find the manuscript submitted to HESS discussions HESSD at the moment in our OSF repository, here.  

UPDATE: The paper had a positive first round of reviews that you can see here.  Below, please find the revised text with the supplemental material. 

Final published paper: 

Thursday, November 11, 2021

How to write a paper on a new hydrological model component

 Let’s try to keep the matter simple.  General rules apply:



Analyzing back the general scheme, in the case of software presenting, you need a specific part dedicated to the availability and delivery of the software. The main parts required here were already illustrated in explaining the Zero Notebook contents.

Because you are talking about scientific software your methodology has two parts. One related to the science you have to produce and one related to the science of writing good software.

Taking the example of Evapotranspiration. The science could be the one included in the sub-models you are implementing. Meaning, what is the science behind Priestley-Taylor, which the one behind FAO approach, and which the one one behind, for instance, our Prospero model ? Here the material is very large so you have to work usually by extracting the essentials and citing the literature. Part of it can easily fit actually inside the introduction. The informatics has to do with the way your system is built. Which is the framework you use, in our case, OMS3, and why you use it, instead of others. It also the system you are working with, like in our case GEOframe that provides ancillary tools. Finally the informatics can boil down to the algorithms and their organization in classes. Algorithms can be new or old and irrelevant. Just in the first case it is important to mention them with details, otherwise just a a little note can be done. Classes, assuming we are talking of some OO programming, have two scopes, one is to contain the algorithms, the other is to orchestrate the software relations in order to make easy the reuse of the softwares and their expansion. This part will be routine in future, maybe, but now it is not part of the common knowledge of hydrologist, and therefore it is worth to be explained if well engineered. In explaining classes and the overall working of the software using of UML diagrams is mandatory.

In a software paper, it is debatable what is the test of the contents. Let’s say that, because we are hydrologists, we need to test both the software running, and the models’ physics.

The software running test for who is programming in Java, like we do, is obtained through the appropriate Unit Tests and this part is commented, in case, inside the section which inherit from the Notebook Zero. For the physics we have, in turn, two modes. If we are solving problems, i.e. equations,  that have an analytical solution, then we have to reproduce the analytical results. Secondly the nasty reviewer, would also see that the model reproduces measurement. Getting some measurements to reproduce is then important. A third case is also ideally possible, which is that, no measurements are available and therefore eventually the model provide a possibility to test something that was never tried before. In this case it must be emphasized that the model makes possible something that before was not not, and we have to rely to some virtual, behavioural, experiment.

If measurements are involved, new methodological steps come in: explaining the case study, first. Secondly, not differently from other cases, we have to say if parameters to calibrate and to mention the techniques we use for doing it. Explaining how we assess the goodness of the results, and finally commenting the results are the rest of the story. An exceptionally good software that does not reproduce reality is simply not useful from the hydrology point of view, even if its implementation can still provide novelties worth to be explained. The physical test, however, should not extend to be very complicate but just functional to convince that the software is doing what it is designed to do. In the mentioned case of evapotranspiration, another issue is relevant, which is the comparison among sub-models or models alternative. It is clear that different models produce different results, so assessing in which case they work or work netter is important. However, that can be pursued with moderation in a “software presentation” paper, because this is clearly an argument which requires a paper by itself. For example , in a recent paper, Clark et al., 2021, they talk of “laugh test” for emphasizing this aspect.

At the end of the post, you have some ingredients and an idea of the procedure. To cook them together for a nice result is a little of art.  In general, a good example to follow is the WHETGEO paper.

Monday, November 8, 2021

GEOframe Winter School 2022 (GWS2022) - Save the dates Dec 20-22, 2021; January 10-14, 2022

December 20-22, 2021 - Installations and Informatics - Online

January 10 - 14 2022 - Onsite and Online

Scientific Committee: Prof. Riccardo Rigon, Ph.D.; Prof. Giuseppe Formetta, Ph.D; Ing. Niccolò Tubini, Ph.D., Ing. Marialaura Bancheri, Ph.D.

Organizing Committee: Ing. Concetta d’Amato, Eng. Shima Azimi, Ms. Sc. Martin Morlot, Ing. Daniele Andreis, Ing. Gaia Roati, Ing. Riccardo Busti (the fantastic group of our Ph.D. students)

Organizing Institutions:
  • Department of Civil, Environmental and Mechanical Engineering, University of Trento
  • Center Agriculture Food Environment, University of Trento
  • Institute for Agricultural and Forest Systems in the Mediterranean, National Research Council, Ercolano NA, Italy

GENERALITIES
GEOframe is a system for doing hydrology by computer. By saying that it is a system, we emphasize that it is not a model but an infrastructure that can contain many differentiated modelling solutions (some tens of that) that are built upon models components. This is because GEOframe leverage on the Object Modelling system-framework (v3) that allows to connect modelling components to solve a specific hydrological issue together and having many alternative for its mathematical/numerical description. This infrastructure allows adapting the tools to the problems and not viceversa. GEOframe has been applied to hydrological simulations from the point scale to large catchments as the Blue Nile, and among those is being deployed to the Po river (the largest in Italy) with great detail. GEOframe is open source and built with open source tools.

CONTENTS OF THE SCHOOL

GEOframe contains tens of components that cover rainfall-runoff, evaporation, transpiration, infiltration, terrain analysis tools, interpolation models, calibrations tools, and so on.
The Winter school is about using some of these tools to perform the hydrological budget of catchments. The core rainfall-runoff model are dynamical systems (systems of ordinary differential equations) and the school mainly treats their theory and their use in a contemporary way as summarized in these7 steps.
Besides the lectures and the hands-on sessions, the Summer School is the occasion for discussion and experience exchange among senior scholars and young researchers.

And Pizza party every night ! Just Kidding


PARTICIPANTS' BACKGROUND

Admissions are reserved to up to 30, PhD students and postdoctoral students, young researchers willing to learn the use of the GEOframe tools envisioned for the study of infiltration, energy budget, vegetation transpiration, water budget with process-based models
All students are asked to upload a CV and a motivation letter when applying.


WORKLOAD AND CREDITS

The Winter School which is to be held in English, consists of 8 hours/day of activities for 8 days. The first three days, 20-22 of December will be dedicate to the the installation of the new version of GEOframe-OMS system tools. Lectures will be brief, dedicated to informatics and the exploiting of the concepts of modeling by components, digital twin Earth and most of the time will be used for supporting participants’ installations.

The other five days (10-14 January) will cover:
  • Catchment and Hydrologic Response Unit delineation
  • Meteorological variables interpolation with Kriging techniques
  • Simple evapotranspiration methods
  • Rainfall-Runoff modelling (as explained in these 7 steps)

LOCATION & TIMING

University of Trento Polo Mesiano, H1 Room and Online. The three days on informatics and installations will be online. The others online and onsite. The time schedule will be 9-13 and 14-18 CET each of the days. Lectures and workout will be recorded and immediately post on the VIMEO Channel of the School and therefore they could be followed off line. Special agreement will be arranged for supporting abroad students with fuse issues.

PARTICIPATION COSTS

The cost is free for Students of the Hydrological Modelling Classes at the University of Trento, for Ph.D. students of the University of Trento DICAM and C3A programs, for the participants of the WATZON PRIN project and for all who wants to participate without having a certificate of GEOframe proficiency. Subscription to the class is necessary for anyone to receive the information to participate. For those who want the certificate, the Course costs 180 Euros. In any case the certificate is issued after the presentation of a small project of simulations for which appropriate tutoring will be given during and after the School.


CONTACTS

For further information write to: abouthydrology@google.com or to the Secretary of the Class dott. Lorena Galante, lorena.galante@unitn.it

OTHER INFORMATION

The GSS2022 talks and labs will be recorded and made publicly available during the School for self-training through the GEOframe blog (http://geoframe.blogspot.com). Information about past Schools can be found here.

GEOframe Soil Plants Atmosphere Continuum and hydrology Estimator (GEO-SPACE) essentials

GEO-SPACE (formerly known also as Lys-GEO in its its 1D implementation)  is intended to collect the growing set of GEOframe tools developed on the base of process-based philosophy. This can be found envisioned first in  Freeze and Harlan, 1969, and, fo instance well documented recently in Fatichi et al., 2016 and Paniconi and Putti, 2016. From a different point of view, it can be considered the upgrade of the GEOtop model, that still efficient and up-to-date, and more advanced than other similar models, was considered to be improvable from the algorithmic and informatics structure.  GEO-SPACE make leverage on the various common tools (components) shared with GEOframe-NewAGE and is made up  specifically of two main groups components, WHETGEO (mainly due, so far to the work of Niccolò Tubini) and the evaporation and transpiration, as follows from the work by Michele Bottazzi and Concetta D'Amato (ET-GEO). 



At present the development of GEO-SPACE (to become GEOtop 4.0) has still to achieve some goals, including the connection of plants treatment on WHETGEO 2D, the implementation of WHETGEO-3D, and so on. The current status of the project can be well described looking at the material presented at the Summer Schools on GEOframe that started in 2021 and will be held usually in week in middle June every year.  The most recent School addresses the more recent material. 

We remind here below, the general declaratory about GEOframe:

GEOframe is a system for doing hydrology by computer. By saying that it is a system, we emphasize that it is not a model but an infrastructure that can contain many differentiated modelling solutions (some tens of that) that are built upon models components. This is because GEOframe leverage on the Object Modelling system-framework (v3) that allows to connect modelling components to solve a specific hydrological issue together and having many alternative for its mathematical/numerical description. This infrastructure allows adapting the tools to the problems and not viceversa. In GEOframe particular attention has been dedicated to allow enhancements and additions writing the less code possible. The core code has been designed to open to addition and closed to modifications, thus allowing stability over time.  The systems contains tens of components that cover rainfall-runoff, evaporation, transpiration, infiltration, terrain analysis tools, interpolation models, calibrations tools, and so on. Every modelling paradigm is included, as, for instance process based modelling, lumped modelling, machine learning, or can be included. Spatially disjoint catchments can be modelled separately and joined together in a bigger model. GEOframe has been applied to hydrological simulations from the point scale to large catchments as the Blue Nile, and among those is being deployed to the Po river. GEOframe is open source and built with open source tools.

References

Fatichi, Simone, Enrique R. Vivoni, Fred L. Ogden, Valeriy Y. Ivanov, Benjamin Mirus, David Gochis, Charles W. Downer, et al. 2016. “An Overview of Current Applications, Challenges, and Future Trends in Distributed Process-Based Models in Hydrology.” Journal of Hydrology 537 (C): 45–60.

Freeze, R. Allan, and R. L. Harlan. 1969. “Blueprint for a Physically-Based, Digitally-Simulated Hydrologic Response Model.” Journal of Hydrology 9 (3): 237–58.

Paniconi, Claudio, and Mario Putti. 2015. “Physically Based Modeling in Catchment Hydrology at 50: Survey and Outlook.” Water Resources Research 51 (9): 7090–7129.

Tubini, Niccolò, and Riccardo Rigon. n.d. “Implementing the Water, HEat and Transport Model in GEOframe (WHETGEO): Algorithms, Informatics, Design Patterns, Open Science Features and 1D Deployment.Geoscientific Models Development Discussions.

Thursday, November 4, 2021

Seven Steps Into Catchments analysis by Hydrological models

 First step: Overall


Define the overall scope of the analysis. (e.g. Blöschl et al., 2019)
Document about the literature existing on the catchments. Including papers appeared in any type of studies.  Choose a large set of  performances indicators (depending on the model use and objectives) (Addor et al., 2017).  Set a strategy for assessing the results uncertainty and variability. (e.g. Clark et al., )

Second step: Geomorphology

Extract the catchment from DEM by state-of-art  analysis tools (as TauDEM or the Horton Machine Toolbox). Investigate if the surface catchment corresponds to the effective catchment (i.e. if including/excluding karst is an option). Control the surface water network extension (talk at the GSS2021). Analyze the presence and the number of lakes/reservoirs. Pay attention to endorheic catchments. Discuss the catchment connectivity.

Third step: Hydrological Data

Analyze the time series of available data and make a comparison  between rainfall and runoff amounts. Analyze any other time series or map time series, like  map of snow and evapotranspiration (Abera et al., 2017). Define the calibration set and the validation set.


Fourth step: Modelling Setup

Setup the space partitioning in hydrologic response units  (e.g. Dal Molin, 2021). Discuss the data density required (or possible) to give robust results or, viceversa, reduce your objectives to something achievable with the data available.   Choose the model among existing ones for adequacy not for legacy (Addor, 2019).  Choose a model structure as hypothesis zero (Fenicia and Kavetski, 2021, VimeoVideo).*  Setup the modelling solution (MS). Choose the better modelling structure (Clark et al., 2011).  Analyze the MS  parameters and discuss their variability.  Plan the model runs thinking to open science protocols (Hall et al., 2021). 

Fifth step: Modelling Execution

Executing the model, including discharges, snow and ET. Annotate the model performances issues. 

Sixth step: Results Delivery

Show the results appropriately (discharge analysis is not the only one quantity to watch at).  Analyze the performances of indicators. Add comments and discussion. Assess results confidence.

Seventh step: Deployment

Deploy the results for open science and public discussion (e.g. Hall et al., 2021). 

Notes

*With regards to the modelling structure, if you are using ODEs for modelling, consider a standard way to visualize and describe the model structure. As many know, the Extended Petri Net can be a sound way to do it. 

Some Slides and Videos about the above topics:

References

Abera, W., G. Formetta, and L. Brocca. 2017. “Modeling the Water Budget of the Upper Blue Nile Basin Using the JGrass-NewAge Model System and Satellite Data.” Hydrology and Earth System Sciences. http://nora.nerc.ac.uk/id/eprint/517346/.

Addor, Nans, Andrew J. Newman, Naoki Mizukami, and Martyn P. Clark. 2017. “The CAMELS Data Set: Catchment Attributes and Meteorology for Large-Sample Studies.” Hydrology and Earth System Sciences 21 (10): 5293–5313.

Addor, N., and L. A. Melsen. 2019. “Legacy, Rather Than Adequacy, Drives the Selection of Hydrological Models.” Water Resources Research 55 (1): 378–90.

Blöschl, Günter, Marc F. P. Bierkens, Antonio Chambel, Christophe Cudennec, Georgia Destouni, Aldo Fiori, James W. Kirchner, et al. 2019. “Twenty-Three Unsolved Problems in Hydrology (UPH) – a Community Perspective.” Hydrological Sciences Journal 64 (10): 1141–58.

Clark, Martyn P., Andrew G. Slater, David E. Rupp, Ross A. Woods, Jasper A. Vrugt, Hoshin V. Gupta, Thorsten Wagener, and Lauren E. Hay. 2008. “Framework for Understanding Structural Errors (FUSE): A Modular Framework to Diagnose Differences between Hydrological Models.” Water Resources Research, Water Sci. Appl., 44 (12): 2135.

Clark, Martyn P., Dmitri Kavetski, and Fabrizio Fenicia. 2011. “Pursuing the Method of Multiple Working Hypotheses for Hydrological Modeling: HYPOTHESIS TESTING IN HYDROLOGY.” Water Resources Research 47 (9). https://doi.org/10.1029/2010wr009827.

Dal Molin, Marco. 2021. “Improvement and Application of Flexible Frameworks for Modelling Regional Streamflow Variability.” Edited by Marco Schirmer Fabrizio Fenicia. Ph.D., Université de Neuchâtel.

Fenicia, Fabrizio, and Dmitri Kavetski. 2021. “Behind Every Robust Result Is a Robust Method: Perspectives from a Case Study and Publication Process in Hydrological Modelling.” Hydrological Processes 35 (8). https://doi.org/10.1002/hyp.14266.

Hall, Caitlyn A., Sheila M. Saia, Andrea L. Popp, Nilay Dogulu, Stanislaus J. Schymanski, Niels Drost, Tim van Emmerik, and Rolf Hut. 2021. “A Hydrologist’s Guide to Open Science.” Hydrol. Earth Syst. Sci. https://doi.org/10.5194/hess-2021-392.

Tuesday, October 26, 2021

The GEOframe Schools Index

The number of GEOframe schools has grown regularly during the last years, since 2019.  We think that it could be interesting to keep track of them and their material in a unique place, and this is it. The most recent are first, the initial one the last.

Rock balancing done by Peter Juhl, author of  "Center of Gravity: A Guide to the Practice of Rock Balancing." 

Links contains presentations, video, reference to code. Obviously the most recent material is more up-to-date but we did not cover the same topics along the years and therefore it could be interesting to browse also the old stuff. Please notice that the Winter School address Catchment modelling with lumped models (Hydrological Dynamical Systems), i.e. GEOframe-NewAGE modelling solutions; the Summer Schools are dedicated  to process-based modelling, i.e. WHETGEO and GEO-SPACE.  (Water, Heat, Energy and Transport in GEOframe, GEOframe Soil Plants Atmosphere Continuum Estimator).

GEOframe parts are also taught in the Classes of Hydrology (in Italian) and Hydrological Modelling (in English and Italian) of the Department of Civil and Environmental Engineering

Monday, October 25, 2021

Driven by Francesco and Olaf, one step in surrogate modelling

 This is one of the paper to which I was more spectator than a real player. However, my name appears among the contributors because I was somewhat crucial for the project to succeed. The idea that bother many is that hydrological models aren too complicate and we need more simple models that get almost the same results. This is the rational for this work that was part of Francesco Serafin Ph.D. Thesis. 

The main ideas was to get into OMS3/CSIP framework an artificial neural network (ANN) system that could surrogate the hydrological models. Surrogate means that it can reproduce most of the dynamic features of the model training the ANN and then making using it. The roadmap is simple but the realization involves several steps that the paper delineates.


So, here it is our paper and its abstract: Serafin, Francesco, Olaf David, Jack R. Carlson, Timothy R. Green, and Riccardo Rigon. 2021. “Bridging Technology Transfer Boundaries: Integrated Cloud Services Deliver Results of Nonlinear Process Models as Surrogate Model Ensembles.Environmental Modelling and Software[R], no. 105231 (October): 105231.

Environmental models are often essential to implement projects in planning, consulting and regulatory institutions. Research models are often poorly suited to such applications due to their complexity, data requirements, operational boundaries, and factors such as institutional capacities. This contribution enhances a modeling framework to help mitigate research model complexity, streamline data and parameter setup, reduce runtime, and improve model infrastructure efficiency. Using a surrogate modeling approach, we capture the intrinsic knowledge of a conceptual or process-based model into an ensemble of artificial neural networks. The enhanced modeling framework interacts with machine learning libraries to derive surrogate models for each model service. This process is secured using blockchain technology. After describing the methods and implementation, we present an example wherein hydrologic peak discharge provided by the curve number model is emulated with a surrogate model ensemble. The ensemble median values outperformed any individual surrogate model fit to the curve number model. 

Sunday, October 24, 2021

To potential postdocs

Are you interested to work with me or my colleagues, inTrento, as a post-doc in hydrology and Earth System Sciences ?  Well, there are two main ways to do it and all of it has to do with the source of funding. If the funds comes from our own research, we expect that the post-doc interacts actively with the topics on which we  are working on. Coming to me, there are many places in this blog where one can deduce what I am doing, but the characteristic of my research work is the commitment to build the GEOframe infrastructure. Therefore for applicants who wants to work with me, a good idea is to learn the use of GEOframe from the Winter School and the Summer School  we held every year. There is a plenty of material (slides, video, codes, food for thinking) that we have prepared with which willing people can self-instruct. We also, I mean the group of GEOframers, will be very proactive in helping you if you want to learn it. Start here or here. To sum up: do you want to work with me/us ? Sweat to learn GEOframe! In the meanwhile you'll do it we will have time to know each other and understanding if we can get along whilst creating for you a reputation independent from ours


A postdoc is though to be a pro-active person in the group making to advance the research, supporting Ph.D. Students, injecting new ideas and enthusiasm. Therefore learning to do what we do is just a prerequisite. We have expectations that they postdocs is able to pose research questions, suggest solutions, indicate new roads, leading the writing of papers for the best journals,  and make our small community more rich.

Researchers who are more mature in this can try to get a grant by themselves, for instance, getting a grant through a project proposal, like, for instance, the Marie Slowdowska Curie calls. In this case, it is more appropriate to say that we can host their research in Trento University DICAM or C3A. They, grant winners,  has  their own research to pursue which is not exactly the mine and, in a sense, it cannot be. But I can give them support, exchange ideas, walk with them along the road drawn. It can be fun for both, and I am sure Trento is a nice place to do it. It is obviously worth if there is some connection with what I know and I am practicing or someone of us is doing. 

An intermediate step would be to try to design, in advance, together,  the project, before presenting it. I am, we are, available to help.

For other information, please give a look to what I wrote for Ph.D. Student applicants. We are talking, usually, of different ages, different maturity, different goals but some arguments remain the same.

Please, remind that I am also committed to produce open source software (meaning we do code and we do it open source) and open science. The most open possible. People with medieval attitude to science (i.e. who hide it selfish) can avoid the discussions applying elsewhere.

Friday, October 22, 2021

Riccardo Busti Master Thesis on Using the GEOframe System to model the Brenta Catchment

 GEOframe-NewAGE is a tool that is becoming more and more tested on catchments. This Master Thesis presents the implementation of the system on the Brenta River. It presents some novelties, of which, some are visible and other more under the hood. The one visibile is the introduction of the possibility to model lakes, two of which are quite relevant for this part of the Brenta Catchment. The others are completely new algorithms for integrating the equations. 


Anyway, you can enjoy the Thesis and its contents just by reading it at this link. 

Thursday, October 21, 2021

Nobel Prize 2021 to Hasselmann and Manabe

 A good summary of Manabe and Hasselman work is in Real Climate.  It is worth to read. 



Klaus Hasselmann and Suki Manabe

The post is here and clarifies their outstanding contribution. 


Wednesday, October 20, 2021

Trento Hydrology Awareness Days (International Conference) - 13 October 2021

We planned for one day and a half of  presentations but eventually we occupied also the whole second day.  Here below, please find the presentations and the videos of the second THAD.  Also for this second day some remarkable guests: Fabrizio Fenicia (GS) and Christian Massari (GS)

The view of Trento from the Mesiano terrace


Tuesday 12
Lunch break



Trento University is a very nice place to study Hydrology: at the undergraduate level at the Course of Environmental and Land Engineering. At the Master level at the Courses of  and Environmental and Land Engineering and Environmental Meteorology. At the doctoral level in the course of DICAM or C3A.  Do not hesitate to contact us if you would like to have information.  As you've seen the Trento groups is open to collaborations and having scientific fun. 

Go to the first day

Trento Hydrology Awareness Days (International Conference) - 12 October 2021

 It happens that you have colleagues with who you want to start new collaborations.  It happens that you have colleagues-friends that work on the same subject you do and live few meters from you and you jokes with them at the cafeteria, talk of everything but rarely on what you are doing in research. It happens that you have some visiting that do exciting work and are in your lab and you would like to have a seminar from them, exchange ideas, and experiment the fun of talking about research. It happens that you all have many Ph.D. students that barely know each others.  It happens that you have an interesting project, WATZON, whose achievements should be presented. That's the origin of the Trento Hydrology Awareness Days (International Conference) 

Last weeks, on October 12 and 13. We have setup a small organizing committee (Giuseppe Formetta and Riccardo Rigon, plus our eldest PhD. students, Niccolò Tubini and Concetta D'Amato) and a small Scientific  Committee (Riccardo Rigon, Giuseppe Formetta, Alberto Bellin, Bruno Majone and Alessandra Marzadri).  Everybody was invited to talk about a topic of their research. The one they like, without restrictions. Twenty minutes each one that expanded sometimes to forty minutes with discussion.

Particular thanks have to be given to the visiting Scientists: Glen Tootle (GS), Venkataraman Lakshmi (GS), Chaopeng Shen (GS), Felipe de Barros (GS) who traveled for this and   Fabrizio Fenicia (GS) and Christian Massari (GS) who participated  online.  All our great Ph.D. students and postdocs who participated on site made their best. 

Here below, please find the first day program, with presentations and videos.

The Mesiano Building and environment where the talks were held



Monday 11
Lunch break



Friday, October 8, 2021

GEOframe Summer School 2021 material is ready for your browsing, inspection, peruse

The GEOframe Summer School, actually held in Autumn this year, is intended  to cover the the (distributed)-process-based hydrological modelling possible in the GEOframe system. This year the distributed part was  not so developed because the focus was on the 1D tools.  The School is intended to be held every year around mid-June.  The image (please let me know the source, which I know is an old book of which I have a copy but I cannot find) represents very well the hydrologic simulation  that our tools can model.  The material (slides, videos, codes, data) are now available to the public and you can find it addressed below. All the material is distributed as open source (the Java codes under GPL v3 license, the Python Codes and all the rest under a Creative Commons license). 



     
INDEX OF THE LECTURES for the inpatients (links to videos and material)

More details below. 

Specific Documentation

The specific documentation regards papers and thesis written on the GEOframe components used in this School. Other literature, of general interest, is provided within the presentations given during the course. Practical documentation for any of the tasks is provided by means of Jupyter Notebooks, of which the general ones are reported below.

 

Tuesday, October 5, 2021

The LysGEO modelling solution @ Italian Hydrological Society Hydrology days

 @ The Italian Hydrological Society Hydrology days, Concetta D'Amato presented her work on the LysGEO model. As some knows LysGEO put together the WHETGEO 1D component with the (revised) Prospero component. The first estimates infiltration, the second performs evaporation and transpiration. Together they constitute a soil-water-atmosphere model, that it is what LysGEO is. Or if you prefer, it is a tool to investigate the critical zone. 



LysGEO was already described elsewhere in the blog. However, in this case there is a relevant addition, derived from the work done utilizing the funding support of the WATSON cost action in Lausanne with Andrea Rinaldo e Paolo Benettin. They built a lysimeter whose seems to be the right experiment to test LysGEO. The presentation shows the first results (with almost no calibration). Clicking on the image above, you get the slides (in English). Here you can appreciate the presentation in Italian given by Concetta.   LysGEO is a product of the WATZON PRIN project.


Tuesday, September 28, 2021

Modelling the River Po (Poster @SII Hydrology days 2021)

 Wow, this will be the first meeting I will attend in more than two years. The are the Italian Hydrological Society (SII) days 2021 and will be held in Naples. We will be presenting a couple of interesting thing. This is the first poster on the modelling of the largest river basin in Italy, which is the topic of the poster here below. The poster should be self-explaining. 

Clicking on the above figure, you can download the high resolution pdf. 


Monday, September 20, 2021

Notes for an incoming hydrology book, Contemporary Hydrology - a Preface

 What’s on Earth is hydrology ? It is the science that studies the movements of water, from the atmosphere to the ground and into the ground and then back to the atmosphere again. This book does not aim to cover all the variety of issues and topics the movement of water causes but is mainly concerned with keeping the material simple and possibly concise. Insights are left to notes and to appropriate literature.

Everyone sees water moving on the Earth surface, maybe excluding those who lives in deserts, and it looks like the it should be easy to get a quantitative mathematical description of water movements: but this is not the case! Even the channelized flow, the simplest of the water flows, shows a great complexity, due to the turbulent nature of the flow and the variety of boundaries in which the water is “constrained”. I have put constrained among quotes “” because water is never really constrained. It goes everywhere, sometimes very slowly, compared to the subjective time of our human actions. Other  flows needs to cope with the medium where water flows, for instance the porous soils, the underground, the plants vessels.
More than that, water on Earth is usually liquid, solid, in form of ice or snow, vapor and each of the phase of water  interacts with and transform into the others. The complexity of phases is just one side of the multifaceted dynamics of water because the elementary physics or thermodynamics of water movements at the centimeter or smaller spatial scale (but how the small is small if, for instances, pores in soil can be micrometers and intercellular space in leaves nanometers ? ) dynamically organizes when we look at the movements at the meter scale and reorganizes at the decameters scale and reorganizes again at the hectare, and, again and again, at kilometers, up to the dimensions of continents. In this reorganization some feature of the elementary motion is loosed while some other feature emerges as fundamental. We could say that the core of contemporary hydrology is in getting a reason of these scales behavior in heterogeneous, i.e. affected by  a great variety of small causes, environment. Galilean science is the light to move among the variety. However hydrologists, the scientists who study and evolve Hydrology’s understanding, rarely can setup experiments (they do, obviously, in the labs) which are really answering the questions, because simplifications and distortion, implicit in other physical sciences branches do not work when you have to deal with a transpiring plant or when the inanimate (?) but complex channel networks forms. Hydrology is mostly observed and recorded, in a sequence of single cases, each one at best slightly different from the others and often very different from the others. Therefore from where we start ? A few certainties we have: mass conservation is the first. We give it for granted but it was recognized more than two centuries ago and its status was even more uncertain in hydrology up to Dalton’s research. The second is the energy conservation, which is in fact more elusive, since the difficulty to estimate some of its parts is bigger than the one to measure mass flow. Those two statements seem so obvious when  enunciated but when we come to measure the mass budget and the energy budget closure troubles begin. Historically, actually, their were not the center of scientists’ research, because this was concentrated merely on fluxes determination, like the river discharge, the evaporative transport, or groundwater flow, at most. However, let’s say, that this book is very much concerned with the closure of these budgets, than concentrating on any of the fluxes, on which actually various sub-disciplines flourished in the past.
Newtonian physics rules, when messing with hydrology, but also the explication of the second law of the dynamics is largely simplified, even in the recent treatments, because of the pitfalls it hides when dealing with simple mechanical systems but that reveals in their whole complexity when the case of water fluxes is the topic.
The nature of the measured hydrological quantities is usually presenting a  degree of randomness, even if their physics is known to be deterministic. This apparent randomness is certainly due to the high number of degree of freedom that the fluid nature of the water, either liquid or gaseous, brings, but also of the uncountable histories that water encounters in its travel. Probably it is not true randomness, is maybe random-mess, because of the structure of the medium water crosses.
If then it is true that determinism is at the core of the hydrological thermodynamics (or maybe saying “thermodynamics” we are contradicting ourselves?) the analysis of the matter lives at the faint light of statistics for almost every phenomenon.  Therefore, almost always hydrologists have hard times to deduce  patterns from very uncertain information. All of these issues in fact are those that makes this science one of the most fascinating of the contemporary science.

Sunday, September 19, 2021

What I did in research the last five years, 2016-2021

 In the last ten years I focused on building a reliable system for doing hydrology by computer. This systems learns from the implementation of the process-based GEOtop and is based on the framework developed by ARS/USDA called OMS3. The new system is called GEOframe


I have been concerned with the fact that many results claimed on the basis of computer simulations were, in fact, not properly reliable and verifiable, do to lack of software engineering, description of the internals of the tools and availability to researchers. Moreover, also the goodness experimental science is quite dependent on the capability and reliability of models.  In my experience with the model GEOtop so far, when there have been discrepancies between data and the model, most of the time the model was right and the experiment imprecise. Sometimes though was the model GEOtop (or other that we used) wrong and we worked to improve it. Models have to be robust, reliable, realistic and their results reproducible (R4). The new system GEOframe, which is built on these premises, is not a model though. It can fit several modelling solutions and it is actually is agnostic with respect to the methods. It is designed to offer a platform to compare different modelling strategies, lumped modelling, process-grid-based modelling and whatever but avoiding to redo any time the unnecessary.
We used some ML technique in the process of calibration and we implemented also a ANN framework (in OMS3) but that part is underdeveloped at that moment. (If we expand to much, we bleed).  

The older part of the GEOframe system contains lumped based types of models. However, we have recently implemented a solver of Richards 1D [0] and 2D (paper in writing) and 3D (software in deployment). The latter implement a new algorithm for integration of non linear PDE systems which always converges and can naturally switch between groundwater, vadose zone and surface water. Soil can be hot, warm and frozen  without problems (the latter is in deployment). Besides this I worked underground in having a better estimation of evaporation and transpiration. I published very little with respect the amount of work I did on these subjects, but disentangling the theory, the misconceptions, the scale issue was (is) not very easy, and took its time. I have a first (not completely satisfactory paper, from my point of view,  on this topic, just published on Water [1] , but better ones are going to be written in the Fall 2021 and in the 2022. Finally I worked on disentagling the theory of travel time residence time. We have some paper on it, since 2016 [2,3,4,5] which are also connected to a new way to categorize and, before of it, representing  lumped-semi-distributed  models [6] in order to be able to produce some quantitative reasoning about models structure. I did not pursued very much applied work but with GEOframe growing we were able to produce some nice applications on the Posina catchment [6] (~110 km2), Blue Nile (~175000 km2) [7]. These applications, could be the basis for a comparison of traditional and ML methods. We have also ongoing  the modelling of the largest river basin in Italy, the Po river (~75000km2) and the Nera catchment (closed at some hundreds of square kilometers),  being  very interesting because affected by karst.  Those  latter two catchments could be possible candidates for applying ML techniques and doing performance comparisons, once we have collected the appropriate data. Notably in the last work (actually since the implementation of GEOtop) working on the catchment meant for me working on the water budget of which the discharge is just one element completed by evaporation, transpiration and heat turbulent transfer. I viewed the use of the energy budget necessary to describe irrigation needs by crops and vegetation in this changing climate era, and in general as a tool to support a more complete view of the hydrological cycle.

Tuesday, August 31, 2021

Building a story or how a "narrative" is important in science I: in writing

 I found the below figure which I do not know the Author which I think can be useful to understand both what it is implied in writing a scientific paper and what is a theory, with respect to more simple analysis of data (or models, BTW). The first four arrangements of the data have some interest but, they do not capture much our interest. To do a gory example, is like to take an animal or a tree, separate them in parts and analyzing them from the point of view of the atoms it is made.  This information is real but it does not say anything crucial about the living being.  The being important think is realized when all the material is put together again (it would be great if the separation operation would be reversible) and it is analyzed in its "holistic" form and function. 


So, it is for the topic of a paper. Usually we have a problem and we dissect it using a "reductionist" approach, but we are not able to put again the whole together and make sense of it. The reductionist action is usually not trivial at all, as the figure could, on the contrary, suggest, and the Authors are usually exhausted after having applied the techniques for doing it which could be experimentally and/or mathematically very complex.  But writing the paper needs the vision of the whole, the story to tell, and the discovery, in the rough matter,  of the sense. It is not a bottom-up action but more a top-down one, where hypothesis and deduction comes before than induction.  
Therefore in writing a paper, we should first have an idea of the whole functioning, trying to get a working a logic from the elements we have in hands and see if they fit. At the first trial they will probably don't. Then we have to go back, refining our theory and trying again (and again).  After a while, the narrative, if we are patient and lucky, works. It is not necessary that it fully works. It is necessary that the final results is an appealing theory that can be further tested (or, better, falsified) in other cases and an improvement with respect to the actual knowledge. The steps forward are usually small. 

Wednesday, August 18, 2021

GEOframe Summer School 2021 (moved to early Autumn for this year)

September 27 - September 28, 2021/ October 4 - October 7, 2021

Scientific Committee: Prof. Riccardo Rigon, Ph.D.; Prof. Giuseppe Formetta, Ph.D; Ing. Niccolò Tubini, Ing. Concetta d’Amato, Ing. Marialaura Bancheri, Ph.D.

Organizing Institutions:

Department of Civil, Environmental and Mechanical Engineering, University of Trento
Center Agriculture Food Environment, University of Trento
Institute for Agricultural and Forest Systems in the Mediterranean, National Research Council, Ercolano NA, Italy

CONTENTS

The Earth’s Critical Zone (CZ) is defined as the heterogeneous, near surface environment in which complex interactions involving rock, soil, water, air, and living organisms regulate the natural habitat and determine the availability of life-sustaining resources (National Research Council, 2001). Clear interest in studying the CZ is spurred on by ever-increasing pressure due to the growth in human population and climatic changes.
Main topics will embrace the water flow (and heat transport) in porous media, the soil-plant-atmosphere continuum, and inverse problems. The aim of the course is to enable participants to run their own simulations with the GEOframe tools prepared to simulate the critical zone. They are process-based (e.g. Fatichi et al, 2016) tools, whose ambition is to simulate the processes of infiltration, heat transport and evaporation and transpiration. The GSS2021 deals mainly with the 1D tools and introduces the 2D ones called WHETGEO (1D and 2D), GEOframe-Prospero and LysGEO.
Besides the lectures and the hands-on sessions, the Summer School is the occasion for discussion and experience exchange among senior scholars and young researchers.
The School will be online on the Zoom platform.

PARTICIPANTS' BACKGROUND

Admissions are reserved to up to 30, PhD students and postdoctoral students, young researchers willing to learn the use of the GEOframe tools envisioned for the study of infiltration, energy budget, vegetation transpiration, water budget with process-based models

All students are asked to upload a CV and a motivation letter when applying.

WORKLOAD AND CREDITS
The Summer School which is to be held in English, consists of 6 hours/day of activities for 6 days. The first two days, 27, 28 of September the installation of the GEOframe-OMS system tools and the general characteristics of the system. Lectures will be brief, dedicated to informatics and most of the time will be used for supporting participants’ installations.
The other four days will cover simulation of infiltration with WHETGEO-1D and 2D, with Prospero Transpiration model, and with the LysGEO model. There will be lectures on the hydrological processes implemented and applications to use cases.

LOCATION
Due to the Covid-19 emergency all the activities will be held via Zoom.



PARTICIPATION COSTS

The cost is free for Students of the Hydrological Modelling Classes at the University of Trento, for Ph.D. students of the University of Trento DICAM and C3A programs, for the participants of the WATZON PRIN project and for all who wants to participate without having a certificate of GEOframe proficiency. Subscription to the class is necessary to receive the information to participate. For those who want the certificate, the Course costs 180 Euros. In any case the certificate is issued after the presentation of a small project of simulations for which appropriate tutoring will be given during and after the School.

CONTACTS

For further information write to: abouthydrology@google.com or to the Secretary of the Class dott. Lorena Galante, lorena.galante@unitn.it

OTHER INFORMATION

The GSS2021 talks and labs will be recorded and made publicly available during the School for self-training through the GEOframe blog (http://geoframe.blogspot.com).

Foreseen schedule

The details of the program are still to be defined

September 27-28:

These days are dedicated to those who never approached the GEOframe system and pursue the understanding of how it works. Who already knows how GEOframe works or have already installed it for different purposes than those of this School, can skip them

  • Introduction to the Object Modelling System and GEOframe Infrastructures (Verona 2022 environment)
  • Installation of OMS and GEOframe Verona
  • Brief introduction to Jupyter notebooks and Python
  • Few examples and Problem solving
October 4:

This morning is mostly dedicated to fill theory of the processes investigated by this School on GEOframe, meaning infiltration in soil, the basics of Richards/Richardson equation to which follow some exercises. The afternoon will be used to discuss issues related to the application of different boundary conditions, different parameterizations of the soil water retention curves.

Morning session
  • The Richardson-Richards equation
  • The equation and its parts, and three form of the equation
  • Soil Water Retention Curves
  • Hydraulic conductivity models
  • Numerical issues to keep in mind
Afternoon session
  • Practical session on Richardson-Richards equatio
  • one homogeneous layer
  • stratified layers
  • playing with boundary conditions
  • Presenting the results with Jupyter Notebooks
October 5:

This day is dedicated to discuss the problem of the surface boundary condition.

Morning session
  • Surface boundary condition and numerical issues
  • Practical session simulating:
  • Horton process
  • Dunnian process
  • Presenting the results with Jupyter Notebooks
Afternoon session
  • Individual exercises with support
October 6:

This day is dedicated to the bi-dimensional case of the Richardson-Richards equation and to present the radiation energy budget.

Morning session
  • Installing the software for building unstructured grids
  • Manage 2D unstructured grids.
  • Practical session on WHETGEO-2D on some pre-prepared cases
Afternoon session
  • Theory of radiation energy budget
  • Practical session on computing the radiation energy budget
October 7:

Day four is dedicated to the LysGEO model, evaporation and transpiration modelling and their coupling with R2.

Morning session
  • Evapotranspiration theory and equations in the Prospero model
  • Use of GEOframe - ET tools practices
Afternoon session
  • LysGEO theory
  • Practical session on LysGEO:
  • Comparison between potential ET and actual ET
  • Set different stress factors
  • Introducing vegetation traits

Specific Documentation

The specific documentation regards papers and thesis written on the GEOframe components used in this School. Other literature, of general interest, is provided within the presentations given during the course. Practical documentation for any of the tasks is provided by means of Jupyter Notebooks, of which the general ones are reported below.



Some essential about the Object Modelling System