Wednesday, December 23, 2020

Lysimiter GEO - Webinar II - An exercise step by step

 This follows the first webinar on Lysimeter Pro, a GEOframe modelling solution intended to estimate the 1D soil-vegetation-atmosphere fluxes using the GEOframe WHETGEO and GEOframe Prospero tools. No more explanations are required than those you already find in the previous webinar and in the Jupyter Notebooks inside the  OMS3 project at here. The OMS project contains all the executable, however you have to do some installations before using the GEOframe working environment

Please find the video of the webinar below.

Executing Lysimeter GEO from Ri Rigon on Vimeo.

The previous webinar here.  For any question please do not exitate to contact us using the GEOframe users google group: https://groups.google.com/g/geoframe-components-users

Friday, December 18, 2020

Lysimiter GEO - Webinar I

 Land-Vegetation-Atmosphere interactions are an exciting field of Hydrology. Within our system GEOframe, one branch of work is improving the physics of GEOtop and this talk shows some of the work we made to this goal. Lysimiter GEO builds a virtual lysimiter and modeling infiltration and energy transfer in soil and evaporation and transpiration. The infiltration is modeled by the component WHETGEO 1D (Water, HEat and Transport in GEOframe) that integrates the 1D Richards developed by Niccolò Tubini. The evaporation and Transpiration are modeled by the GEOframe component Prospero  developed by Michele Bottazzi  in his Ph.D. Thesis.  Lysimeter GEO, however, was completed by Concetta D'Amato who is pursuing her Ph.D. on these topics within the PRIN project WATZON.By clicking on the Figure below you can access the slides. 

If you want to run Lysimeter GEO, you have first to install the GEOframe 2021 environment.  Here below, please find the video of the talk.  The OMS project for all the run can be found on OSF here



The second webinar containing an exercise did step by step is in this new post. 

Thursday, December 17, 2020

Krigings of positive definite fields

 Kriging is a statistical technique used to interpolate spatial sparse data and obtain a field of measures. Interpolation of data (Mitas and Mitasova, 1999) is ubiquitous i hydrology and, in fact, the GEOframe system contains a solid Kriging interpolator.

One aspect that bother hydrologists is that often they have to treat with quantities that are positive definite and Kriging does not guarantees that you get from positive measurements positive values.
In fact, in literature there exist some trials to get a version of Kriging constrained to such values. Mainly three are the relevant contributions.

My preferred one is Szidarovszky et al., 1987 which takes the problem directly by definitions and solve it. However also Deutsch, 1996 use an iterative (trial and error method to obtain the same result). A more restrictive condition which impose the conservation of the total "mass" of the quantity modeled is pursued in Walvoort and de Gruiter (2001).


Other suggest, especially for the case of precipitation of using an indicator Kriging to detect locations where rain is falling from locations where it is not (e.g. Atkinson, 1998) or cokriging techniques.

References
Adhikary, Sajal Kumar, Nitin Muttil, and Abdullah Gokhan Yilmaz. 2017. “Cokriging for Enhanced Spatial Interpolation of Rainfall in Two Australian Catchments.” Hydrological Processes 31 (12): 2143–61.
Atkinson, Peter M. 1998. “Mapping Precipitation in Switzerland with Ordinary and Indicator Kriging.” Journal of Geographic Information and Deci Sion Analysis 2 (2): 65–76.
Deutsch, Clayton V. 1996. “Correcting for Negative Weights in Ordinary Kriging.” Computers & Geosciences. https://doi.org/10.1016/0098-3004(96)00005-2.
Mitas, Lubos, and Helena Mitasova. 1999. “Spatial Interpolation.” Geographical Information Systems: Principles, Techniques, Management and Applications 1 (2). http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.224.5959&rep=rep1&type=pdf.
Myers, Donald E. 1991. “Pseudo-Cross Variograms, Positive-Definiteness, and Cokriging.” Mathematical Geology. https://doi.org/10.1007/bf02068776.
Szidarovszky, F., E. Y. Baafi, and Y. C. Kim. 1987. “Kriging without Negative Weights.Mathematical Geology. https://doi.org/10.1007/bf00896920.
Walvoort, Dennis J. J., and Jaap J. de Gruijter. 2001. “Compositional Kriging: A Spatial Interpolation Method for Compositional Data.” Mathematical Geology 33 (8): 951–66.

Wednesday, December 16, 2020

One Year Grant on the management of River Po catchment and related water resources.

 To whom it may concern, River Po Authority, issued a grant for a one year to apply the system GEOframe to the river Po.  The person interested is requested to:

  • Implement and deploy the GEOframe system on the catchments of the District, including the main lakes and reservoirs
  • calibrate and verify the models
  • elaborate and the results 
  • Using and applying the interface Deltares FEWS/DEWS
  • Doing Analyses of the impacts of different choices of the water resources management also accounting for the climate change or modifications of land use
  • Improving the models
Previous knowledge of the tools above is preferred but not required. Good will is better than erudition.


The position is in Parma, at the District Authority but interactions with our group in Trento are foreseen to happen at daily basis. The working in Trento includes, at present, Prof. Riccardo Rigon, Prof. Giuseppe Formetta, one post-doc, one Ph.D. student, one master student. Dr. Christian Massari and Dr. Silvia Barbetta of C.N.R. IRPI in Perugia collaborate to the project, too.   If colleagues reading this post wish to join our crew, at present on a free basis (we have no that incredible money), but looking forward for a challenging collaborative research, they are welcomed. 
We are also looking for other funding resources to get new positions.
FYI, the District will probably hire many people on the next years, and therefore, our addiction, this position as well the others could be a way to learn some skills allowing to be overcome the selection to be hired.  We, at University of Trento, are also interested to people who could eventually be enroll for doctoral studies on the topics of the call. 

You can find the official information of the call at the site of the District Authority Further information about the system GEOframe can be obtained at the GSW2021

Third SII Hydrology Day Webinar - Il ruolo dell'idrologo nella pratica professionale e altro

 This is the third webinar of the 2020 Hydrology days held via Web. It was certainly a successful initiative and this third appointment maintain the interest high. Please below you will find the video registration of the conference


Wednesday, December 9, 2020

The decline of hydraulic conductivity with suction in plants

In the vein of understanding more on plants hydraulics I started from the (McDowell et al., 2008)} paper which is as much informative as imprecise in its two or three formulas which do not much with the concepts conveyedby the figures and the text. So definitely it is an inspiring paper but I would not start from it. Instead I would probably use as a first reading {(Venturas et al., 2017)} that is more recent and more consistent and which I would couple with Lehnebach et al., 2018), to read as acomplement. In this post, however, let's concentrate on the conductivity of stems decline with stems suction. In literature, it was found todecline with stem pressure according to a sigmoid form (I would say, an exceeding probability like s function).In literature various form were used to parametrize this behavior. Notably:

  • the exponential-sigmoid (Pammenter & der Willigen., 1998)
$$ K(\psi) = K_{max} \left( 1 - \frac{1}{1+ \exp( a(\psi-b) )}\right) $$
  • the Weibull (Rawlings \& Cure, 1985), Neufeld et al., 1992)
$$ K(\psi) = K_{max} e^{ - \left(\frac{\psi}{\alpha} \right)^\beta} $$
  • the Gomperz (Mencuccini \& Comstock, 1997)
$$ K(\psi) = 1 - a e^{-b e^{-c \psi}} $$
  • polynomial function (Pockman et al., 1995)
  • the power law (Wang et al., 2017)
$$ K(\psi) = \left(\frac{1}{2}\right)^{\left(\frac{\psi}{\psi_{50}}\right)^b}$$
However, for historical reasons, this decline is modeled~in plant physiology literature using PLC curves which represent the Percent Loss Conductivity. PLC have a one to one relation with hydraulic conductivity
which is given by the formula:
$$ K(\psi) = K_{max}\left(1-\frac{PLC(\psi)}{100}\right) $$
Some researchers found useful to treat differently the parameterization as, for instance, in Ogle et al., 2009, an indication that was taken in Duursma, 2017 for implementing their R software fitplc. The latter paper is interesting also for having tried to evaluate the error of estimation and the parameters they reproduce in their Table 1 for some plant species.

References

  • Nate McDowell, William T. Pockman, Craig D. Allen, David D. Breshears, Neil Cobb, Thomas Kolb, Jennifer Plaut, John Sperry, Adam West, David G. Williams, Enrico A. Yepez. Mechanisms of plant survival and mortality during drought: why do some plants survive while others succumb to drought?. New Phytologist178, 719–739 Wiley, 2008. Link
  • Martin D. Venturas, John S. Sperry, Uwe G. Hacke. Plant xylem hydraulics: What we understand current research, and future challenges. Journal of Integrative Plant Biology 59, 356–389 Wiley, 2017. Link
  • Romain Lehnebach, Robert Beyer, Véronique Letort, Patrick Heuret. The pipe model theory half a century on: a review. Annals of Botany 121, 773–795 Oxford University Press (OUP), 2018. Link
  • N. W. Pammenter, C. Van der Willigen.. A Mathematical and Statistical Analysis of the Curves Illustrating Vulnerability of Xylem to Cavitation. Tree Physiology 18 (1998).
  • J. O. Rawlings, W. W. Cure. The Weibull Function as a Dose-Response Model to Describe Ozone Effects on Crop Yields 1. Crop Science 25, 807–814 Wiley, 1985. Link
  • Howard S. Neufeld, David A. Grantz, Frederick C. Meinzer, Guillermo Goldstein, Gayle M. Crisosto, Carlos Crisosto. Genotypic Variability in Vulnerability of Leaf Xylem to Cavitation in Water-Stressed and Well-Irrigated Sugarcane. Plant Physiology 100, 1020–1028 American Society of Plant Biologists (ASPB), 1992.Link
  • M. Mencuccini, J. Comstock. Vulnerability to cavitation in populations of two desert species,Hymenoclea salsolaandAmbrosia dumosa from different climatic regions. Journal of Experimental Botany 48, 1323–1334 Oxford University Press (OUP), 1997. Link
  • William T. Pockman, John S. Sperry, James W. OLeary. Sustained and significant negative water pressure in xylem. Nature 378, 715–716 Springer Science and Business Media LLC, 1995. Link
  • Han Wang, I. Colin Prentice, Trevor F. Keenan, Tyler W. Davis, Ian J. Wright, William K. Cornwell, Bradley J. Evans, Changhui Peng. Towards a universal model for carbon dioxide uptake by plants. Nature Plants 3, 734–741 Springer Science and Business Media LLC, 2017. Link
  • Kiona Ogle, Jarrett J. Barber, Cynthia Willson, Brenda Thompson. Hierarchical statistical modeling of xylem vulnerability to cavitation. New Phytologist 182, 541–554 Wiley, 2009. Link
  • R. & Choat Duursma. fitplc - an R package to fit hydraulic vulnerability curves. J. Plant Hydraul. (2017)

Sunday, December 6, 2020

Why Excel is Evil

Why Excel is Evil is not a campaign against a commercial product. Instead, it is an analysis of the requirement for having a solid scientific product.  Thanks to Colin Caprani for this valuable contribution.


Monday, November 30, 2020

A note on Wilting Point (and water stress in vegetation models)

 The paper by Veihmeyer and Hendrickson seems definitive in its affirmation about both the field capacity and the wilting point. That they exist and depends upon the soil characteristics. Because I certainly know that field capacity is just an interpretation of phenomena and not some characteristics written in Mose’s tables, I have the doubt that also the second about the wilting point is at most an approximation. 

Gardner in his interesting 1965 paper, in fact says the following: "Whatever the exact value of the pressure potential at wilting, the wilting phenomenon is a consequence of a change in the mechanical or elastic properties of the plant leaf and appears to be associated with a rather definite value of the relative water content and water potential for leaves of a given species and given age for a given cell solute content.” 
This statement is much less assertive and leave large space to variation among plants types. 
Actually I was interested in the wilting point because $\theta_w$ is used in traditional standard models for determining the water stress induced by droughts on plants. Actually then, what I am interested in is not the “wilting” point, but the point at which stomata close with respect to this, the CLM model, in fact, acts in a different way:
  •  It uses suction instead than water content (i.e. formula 2.34 of the manual, Oleson et al, 2013)
  • It gives a table according to which different vegetation types close stomata at different values of suction (Table 8.1)


The second fact implies that the water content at which a plant type close its stomata varies with soil types (this is obtained by inverting the suction using the soil water retention functions). 
Clearly the two views cannot be both true when applied to transpiration. Either the one or the other must be the correct one (well, this is optimistic: probably they are both wrong). 
The reading of the Garner paper also suggests that anyway the wilting point, i.e. when plants wilts, depends upon the properties of plants’ cells, which actually could have a certain homogeneity among the various specie and the one of the wilting point  is a topic that needs further browsing of literature (but the issue related to stomata closure, maybe not). Anyway Gardner 1965 set it between -12 and -15 Bars, which is, actually a quite wide range. 

Since the arguing went to the topic of stresses it is also to remark that isohydric and anisohydric behave differently. Therefore this has to be reflect in the mathematical form of the stress which actually is not.

.... Still mumbling .......

News !!!

Finally it comes a very recent paper. Brought to my attention by Nunzio Romano, Chagas Torres et al. (2021) sortes out the problem with very recent analysis techniques. It shows that wilting point is effectively variable and depending both on soil types and plants. It also contains references to other recent literature I was not aware before. 

References

Chagas Torres, Lorena, Thomas Keller, Renato Paiva de Lima, Cássio Antônio Tormena, Herdjania Veras de Lima, and Neyde Fabíola Balazero Giarola. 2021. “Impacts of Soil Type and Crop Species on Permanent Wilting of Plants.” Geoderma 384 (February): 114798.

Keith W. Oleson, David M. Lawrence,Gordon B. Bonan, Beth Drewniak, Maoyi Huang, Charles D. Koven, Samuel Levis, Fang Li, William J. Riley, Zachary M. Subin, Sean C. Swenson, Peter E. Thornton, Anil Bozbiyik, Rosie Fisher, Colette L. Heald, Erik Kluzek, JeanFrancois Lamarque, Peter J. Lawrence, L. Ruby Leung, William Lipscomb, Stefan Muszala, Daniel M. Ricciuto, William Sacks, Ying Sun, Jinyun Tang, Zong-Liang Yang. 2013. “Technical Description of Version 4.5 of the Community Land Model (CLM).” NCAR.

Gardner, W. R. 1965. “DYNAMIC ASPECTS OF SOIL-WATER AVAILABILITY TO PLANTS.” Annu. Rev. Plant. Physiol.

Veihmeyer, F. J., and A. H. Hendrickson. 1950. “Soil Moisture in Relation to Plant Growth.” Annual Review of Plant Physiology 1 (1): 285–304. 

Sunday, November 22, 2020

Evaporation from a capillary tube coupled with the dynamics of flow

This is the result of some quick research of mine, initially stimulated by the model of evaporation from soil by Lehman and Or.  I did not read them and I am annotating them here in order not get them lost.  Among those I found, I would start from:


The next two, written the same Authors, have a theoretical part and experimental part. They looks like rigorous but a little intimidating at first. So I would let them for a second reading. 
  • Polansky, John. 2016. “An Experimental and Theoretical Investigation of Evaporating Meniscus Dynamics and Instabilities.” Carleton University. https://curve.carleton.ca/b5d63fef-d97d-4df1-88ae-6548ea935426.
  • Polansky, John, and Tarik Kaya. 2016. “Stability of an Evaporating Meniscus: Part II--Experimental Investigation.” International Journal of Thermal Sciences 105: 75–82.
Finally, the third group of papers contains some experiments and, I think they are worth a reading"
  • Luzar, Alenka, and Kevin Leung. 2000. “Dynamics of Capillary Evaporation. I. Effect of Morphology of Hydrophobic Surfaces.” The Journal of Chemical Physics 113 (14): 5836–44.
  • Leung, Kevin, and Alenka Luzar. 2000. “Dynamics of Capillary Evaporation. II. Free Energy Barriers.” The Journal of Chemical Physics 113 (14): 5845–52.
  • Leung, Kevin, Alenka Luzar, and Dusan Bratko. 2003. “Dynamics of Capillary Drying in Water.” Physical Review Letters 90 (6): 065502.
 Not having read them, I do not know how much they account properly for the evaporation demand, if it is fixed or if it is varying. In the latter case the equation for the capillary should be coupled with the atmosphere and what I can suggest, is to give a reading to the second chapter of dr. Michele Bottazzi dissertation. 

Monday, November 16, 2020

Life Expectation & Response Times explained

This is the continuation of the presentations that try to explain what residence time, travel time and response time are. The first presentation, on travel and residence times can be found here.


This presentation regards response times and their relation with life expectancies, which is an aspect quite unresolved so far and that was clarified to our knowledge for the first time in the supplemental material of the paper "On the relations among HDSys" by Rigon and Bancheri


Friday, November 13, 2020

Earth Observation For Water Cycle Science 2020

 I receive from Luca Brocca and I think the topic is interesting and the project mentioned worth to be followed. 

Dear colleagues


we have organised a discussion session at ESA conference EO FOR WATER CYCLE SCIENCE 2020  on Digital Twin Earth Hydrology (DTE Hydrology).
DTE Hydrology is a very recent activity (started one month ago) that aims at building an integrated and interactive system providing the best possible reconstruction and simulations of the water cycle and the hydrological processes and its interactions with human activities at unprecedented resolutions and accuracies (see also here for more information). The first implementation of DTE Hydrology will be carried out over the Po River Basin.

The session is on Tuesday 17 November 15:00-16:30 (UTC+1) (here the agenda).  The general agenda is obtained by clicking the image above.

Thursday, November 12, 2020

Proietti's lesson to hydrologists

 Below is an excerpt from the memories of Gigi Proietti (the book is from 2013) brought to my attention by the colleague Giovanni Pascuzzi



"When I began to take my first steps on stage, I did nothing but devote myself to the technical aspect of acting. Gassman said I was manic. And I thought: "from which pulpit the sermon comes".
Like someone who plays a scale after another maybe even quickly but without putting into it the warmth of invention, "er core" as the jazz musicians in Rome say.
I lacked the humility of those who put themselves at the service of the public, of those who are able to look at themselves with the eyes of another without hiding behind pure technique.
I had to get naked and instead I insisted on excessive virtuosity that covered every exchange, broke all communication. I continued with that attitude until I made my big discovery, the closest thing to an epiphany I have ever experienced: “I am fucking unsympathetic". Excessive technique made me unbearable. Today I notice it when I see some footage of that time.
Too many times, in the cinema world, I had heard myself say: "you are too good" which is different from "You are very good, outstanding". It hides the fact that you are unpleasant. When you overdo it, you get the opposite result: you do not pop on. And getting out of that trap that you  have set for yourself is not easy. You are confused and you ask yourself: "Should I do less?". But for someone who wants to give the best of himself, doing less is very difficult. Even on television the "experts" said I didn’t pop on camera. Then when I did Marshal Rocca they were silent." 

While this has to do with teaching and research ?  Both of them require possessing techniques (the standards, they would say a jazz musician) but you do not have to be possessed by the technique. The technique is a tool not the topic (unless your topic is the technique). 

Wednesday, November 11, 2020

Travel times and Residence Times explained

Long time ago it was believed that Residence times and travel times were the same concept. This is not true. Long time ago it was also assumed that they could be chosen as time invariant. Recent literature showed that this is not usually the case.  All of this is explained, hopefully in a clearly definitive way in the following presentation.


Yo can find it by clicking on the figure above. This presentation uses drawings, plots, in a way that it cannot be done in a paper (but would be useful sometimes) and can be considered to be a companion of part of our 2016 paper below. Other presentations will follow on the topics of response time and its relation to life expectation. The talk I gave for the WATZON project is here below

 

 

Rigon, Riccardo, Marialaura Bancheri, and Timothy R. Green. 2016. “Age-Ranked Hydrological Budgets and a Travel Time Description of Catchment Hydrology.” Hydrology and Earth System Sciences 20 (12): 4929–47.

Friday, November 6, 2020

The State-of-Art and the perspectives for next GEOframe research

These are the contents of an e-mail I sent to a friend and colleague to push forward our collaboration. Despite it has been written to a specific person, I think some of the topics can be on general interest for the topics it covers. At least for who is interested in Hydrological Modelling.

Dear Friend,
 
I am trying to simplify here the objectives of my research in order to see where we can find convergence of aims and goals.

Overall, I want to pursue a tight connection between the theory of hydrological processes and their sound (replicable, robust, reliable) implementations (see also here for other explanations). Actually, I work both on the theory and on the implementation. Soon in my career, I realized that a poor implementation of a correct theory often produces wrong results and moreover its incorrect falsification brings credits to flawed ideas (the case of  "tranchant" judgments on Richards equation, based on unreliable integrators is one of the cases). I also grew the idea that, differently from some other colleagues, I wanted to build not just doing-programming but programming-system products (e.g. - Brooks, 1975), i.e. reusable software on which other people can build new knowledge (yes, the idea that building on each others shoulders, and working that way, possibly having the couple of smart intuitions, we can arrive where only the giants, usually arrive). 


My main goal is to build better models than those existing, more controllable, and less prone to devastating bugs. My research tends to be more “methodological” than applied. It is exactly this approach that moved me towards OMS3. It, in comparison with other options, followed by successful colleagues, presents a clean design, support for models’ controllability, encapsulation of modelling solutions, easy and ordered reuse of modules, intrinsic documentation with annotations, support to technical issues, like parallelization and calibration, without overwhelming the hydrologist concentrated on physics. Besides, a wise use of components, could help to dose information to the users and set it visible only when and where required.

Coming to the practice of my research, I focused mainly to two types of models, those I call Hydrological Dynamical Systems (HDSys), mainly based on the solution of multiple coupled non-linear, non-autonomous, ordinary differential equations, and those that have the space variables explicit and solve partial differential equations. From a different perspective, my goal has been to cover entirely the various aspects of the hydrological budget (often including terrain and soil/sediment) not only focusing, as traditional models do, one some or one of the aspects, like discharge or evaporation, or infiltration, or groundwater. Coupling the water with the energy budget, has been an objective of both the types of modelling, especially, but not only, because temperature is easily measurable, even from remote sensing. Tracers, Nutrient and pollutants, were never deeply considered, except recently, but they are part of my modelling tools since many years.

For a community to grow around the previous ideas, some key tools are still missing. The points below summarize: i) what I envision is needed from different perspectives/users; ii) what futures OMS developments should potentially consider. Some of these points are already being reached with the work we did, some are planned to be implemented, for some others we need your support to reach a critical mass and make them happen.

Power User side:
  • Smoothing out some part of the process to deploy a modelling solution to a specific concrete catchment. With students of the GWSs and Hydrological modelling class the two processes of extracting the HRU and interpolating the data were instructive but too much detailed and cumbersome.
  • Using the console is easy and usually hassle-free. However, because we use mostly Jupyter notebooks for the treatment and analysis of inputs and outputs this is a further environment to learn. Using Docker and the command line inside Jupyter could be a choice but with Docker we had hard times on Windows. A console inside Jupyter would be the best choice.
  • In general, a convergence of our tools with those tools people use the most, as Jupyter, decrease the learning curve and developer commitment to bring in and maintain tools.
  • Some ancillary tools for “joining” catchment studied by different people are required
  • calibration revealed to be a time-consuming effort that needs parallelization and speed-up.
  • Probably a server or a “hub” to store and retrieve the collective work, including parameterisations, inputs and so on.
  • Manage the possibility of having multiple treatments of the same catchments would also be required sometimes in the future.

For institutional user
  • Connections to Delft-Fews could be an option to investigate.
  • Our group needs also to experiment with CSIP .
  • A distribution of all the material and the code, through some tool like Anaconda would be desirable.
  • They need dedicated interfaces. For they, modifying parameters and models structure should be not be an option as for researchers or power users
  • Scalability of the computing effort should be the standard (and this should include Net3)

Developers: 

  • Source code should be available on public repository like Github
  • To improve developer appropriate/specific documentation for them should be a continuous effort

Potential developer/researchers

  • When they come from Environmental Engineering or sciences, they usually are familiar with Python, R or Matlab. They do not have notions of OO programming, nor of basic software engineering background. Therefore, appropriate material providing all of this knowledge should be produced. I started with a Java for Hydrologist 101 but I am far to have completed it.
  • They are not comfortable to use tools like Git, Github, Docker, Unit tests, and other commons tools which are necessary for software carpentry and collaborative work. Therefore, some training course on these should be also provided .

The above is more a wish list which we are keeping in mind. Frankly we do not have yet the all the competence to treat them all. As you see I did not list any machine learning tool: but this does not mean that we are not looking to them. For the moment is just safe for us to concentrate to enhance and bring to an optimal state what we have and publish that ten of papers that we have in production on he work we have already done. We are looking for resources though and, resources arriving, we could also think to statistical/machine learning methods to be introduced. One thing to be remarked is that GEOframe-NewAGE can easily replace PRMS. The module we have, usually, are different from those PRMS has, but implementing them the very same way PRMS does should be VERY easy, if this is the goal. A greater integration with AGEs would be also advisable. The main differences to be treated for compatibility are the IO. For now, we often stick with complex data formats but abstract the algorithms from them is an objective we have in mind. Mostly we had to follow our way so far to be sufficiently comprehensive. To be sincere, IMHO, some parameterisation of the processes inside AGEs are simply old hydrology, not currently supported by researchers but, yes, still in use by practitioners (which worldwide use SWAT, though). In all I mentioned I forgot to mention the work by Daniele Dalla Torre that ported SWMM to OMS3. That is a thread that is, at present in a dead end but it can come back alive any time.

Below, I give further information on: i) the reasons why we use OMS; ii) the new components I mentioned before or we are going to develop.

When I arrived to OMS3, my most recent achievement was a stable version of GEOtop, a model that solves the water and energy budget, as its foundational paper told. After 15 years, GEOtop remains quite unique in the panorama of “process-based” models. It in fact includes what is usually present in other process-based models, i.e. an integrator of Richards, Groundwater and Surface water equations, with what usually appears in soil, vegetation, atmosphere models. Besides it has a solid model for snow height evolution, used operationally all over the Alps, and freezing soil, which constitutes a third type of process-based model usually cared by a different scientific community. I’ve certainly sinned arrogantly in doing what others still not do, even with much larger resources, and I will probably go to hell for that. GEOtop has a decently extended literature and I could have capitalized better its treasures, but I preferred to move on, because while I was getting GEOtop stable I’ve been also touching its limits.

Its monolitic structure made of thousands lines of code, made it not easily modifiable and improvable with incoming research and understanding.

Its ambition to cover all the areas of hydrological modelling have made exploding the number of input parameters a fact that most researchers found overwhelming. Introducing competing ideas to model some of the processes became practically impossible and any science advancement nullified.

That’s why I was looking from an intrinsically modular system that could resolve the above issues and boost collaborative work, and that’s why I moved to OMS3 in 2008.
I would have stick with that GEOtop objective but in the same year had quite unexpectedly financial support for studying the management of draughts of river Adige. GEOtop was impractical for that use because of its inability to be calibrated and some flaws in its subsurface-surface water interactions. Necessity brought to the implementation of GEOframe New Age version 0.

In the subsequent decade I and collaborators worked on the GEOframe model perspective, faster to calibrate and, nevertheless quite complete from the point of view of processes integration. For many ancillary parts of the system, it was reinventing the wheel again from scratch but this finally produced the mature product that GEOframe-NewAGE is today. It was conceived for using the natural spatial fractal - graph-like structure of rivers for distributing spatially the hydrologic response unit (HRU) physics and computation. The main driving idea behind GEOframe is that we do computation on a graph nodes which exchange mass and energy according to the interactions among parts described by graph's connections. These nodes can be spatially distinct entities, like hillslopes and HRU, or concurrent processes like discharge and transpiration. In principle an engine under the hood is responsible for distributing the computation along the graph, while the hydrologist takes care of describing the processes with the appropriate degree of refinement. This had a first implementation with Net3 but I believe can be improved in several directions. Before Net3, sure, river networks were schematized as graph, but their topology was hardcoded and no variation was possible in the spatial structure of the model without disrupting the whole. With Net3 the topological structure of the connections can be modified just before the run time, inserting or eliminating human infrastructures, diversions, new nodes of calculation, lakes, reservoirs.

Net3 opened also to the possibility for different researchers to work simultaneously on different part the catchments (actually of the graph) enabling the possibility for a sort of “crowd modelling action” to cover the whole Earth with GEOframe based modelling performed by a crowd of researchers or simply trained people. Clearly for this an infrastructural work is still missing but potentially it could provide a collective works that highly surpasses the present global scale hydrological applications which are based on rough characterization of parameters and scanty local reanalysis of data which is not possible, even for large research groups. The first application of this modeling strategy will be the application of the model to the river Adige, separated in a thousand or more HRU of which we have, so far, some work done by the students of my course of Hydrological Modelling. River Adige is relatively small (10^4 square kilometers) but a variety of climate situations and anthropic activities and settlements that make it very challenging to be modeled. Eventually the simulations will be extended to the whole Alps and beyond.

From the point of view of interacting components, GEOframe has many: the traditional set of tools coming from Hydrologis for terrain analysis; a set of Krigings for interpolation of hydrometeorological variables, the estimation of shortwave and longwave radiation including shadows and topography effects, interception of rainfall by canopies, three simplified models for snow water equivalent modelling, various tools for reservoirs-like modelling (a la PRMS), Muskingham-Cunge and 1d deSaint Venant propagation, Priestley-Taylor, Penman-Monteith and a new model called Prospero which implements a revision due to Penman-Monteith by Schymanski and Or. All the models can make use of LUCA and PSO tools for calibration and the dedicated papers constitute a guideline for their use.
Giuseppe and I just hired a coupled of Ph.D. student and, they, among the other stuff, will work on data assimilation and possibly on some OMS issues.
Did I abandoned then process based modeling? Not at all, the original plane to build the new GEOtop 4.0 is actually very alive.
The nucleus are the tools growing around WHETGEO. At present we have:
  • An integration of Richards 1D with and without temperature, decoupled and coupled with the Energy budget (return the soil temperature profile)
  • An integration of Richards equation 2D (hillslope profile, for instance). No coupling with the energy budget yet.

These tools make leverage on terrain analysis, radiation estimation, interpolation of data, estimation of Evaporation and Transpiration already present in GEOframe but have a gridded domain instead that an HRU separation. With respect to GEOtop, the integration algorithms are completely redesigned around the Newton-Casuli-Zanolli (NCZ) algorithm for Richards, and an appropriate implementation of the grids where topology and geometry are separated. In GEOtop a more traditional Newton-Krylov (NK) algorithm was used whose convergence is not granted a priori and the equations were written for a structured (regular) grid which causes artifacts into the results. Using appropriate design patterns, a twofold objective was obtained: to make room for changes in parameterisations of the equations and, and to maintain as simple as possible (but not too simple) the contents of the inputs. The use of standard formats like NetCDF for outputs contained the number of otherwise exploding output files. The input and output format were decoupled from the algorithms though, in order to maintain flexibility for changing the outputs format. As seen many components developed for GEOframe-NewAGE could be reused for WHETGEO and, in fact, in the foreseen future, also the Net3 infrastructure could be possible used to aggregate various hillslopes simulated by independent WHETGEO runs.
Because I have experience with them and notwithstanding the opinion of many colleagues, I do not see in fact that the lumped reservoir models can cope with those processes which have a well definite spatial history. Remote data is a new frontier and, besides, WHETGEO is fully able to exploit them.

Evolving Prospero will bring into WHETGEO green waters and vegetation and GEOtop 4.0 will be much greener than GEOtop 3.0. This achievemente is almost there. Carbon cycle evolution, forestry and crop, will be a set of ODEs attached to sites, either described as a grid cell or a HRU. Their mathematics is quite the same that for HDSys, with just a different interpretation of the parameters. Transport of tracers and pollutants, via the advection dispersion equation is also almost obtained, because these equations belong to the same family of the heterogeneous transport of heat in porous media that we have already implemented.
Giuseppe in his magic hat has already setup models for hillslope stability analysis that just wait for the 2D and 3D WHETGEO to be tested (the first) and fully implemented (the second). But the 3D solution is more a problem of drawing the grid than everything else with respect the 2D solver.
A possible threat to this WHETGEO is its computational burden. The algorithms are efficient but a high-resolution three-dimensional grid has potentially millions of nodes and simulations are time consuming. Parallelizing its core routines in a way that does not clash with the other forms of parallelism present if OMS3 will be a challenge.
Therefore, what I foresee in the next years is these tools to get maturity and to be used by a potentially large set of users. GEOtop 4.0, built on WHETGEO, Prospero, and other tools would be a really operational tool for instance for landslide risk early warning; for the detailed soil moisture account for precision, regenerative agriculture; for small catchments runoff and sediment production (the latter a feature to be implemented), and obviously would be a great tool for studying any aspect of the critical zone, in any climate past present or future. At the present we cannot foresee when we could add accurate modules for snow (like those or better than those already in GEOtop) but the modules already present in GEOframe could be used easily on a pixel base to surrogate them. Calibration tools for WHETGEO process-based modules, I think, will require some adjustment of the calibration tools now present in OMS3 and we did not try anything about yet.
This mail was pretty long but, I hope it serves to clarify my point of view and the legacy I have with my previous research, I also included Tim in the mail, because I thin he can be interested on many of the research I exposed. Thanks to the friendship we have, I hope the way to strengh our past collaboration in a few future objectives where we can find reciprocal satisfaction.

All the best,

ric (and the guys)

What is meant by programming a system product

 Here I just reproduce the very first pages of  The Mythical Man-Month: Essays on Software Engineering. Addison-Wesley which clarify the main concepts I experimented in my programming models.

"The Programming Systems Product One occasionally reads newspaper accounts of how two programmers in a remodeled garage have built an important program that surpasses the best efforts of large teams. And every programmer is prepared to believe such tales, for he knows that he could build any program much faster than the 1000 statements/year reported for industrial teams. Why then have not all industrial programming teams been replaced by dedicated garage duos? One must look at what is being produced. 



In the upper left of Fig. 1.1 is a program. It is complete in itself, ready to be run by the author on the system on which it was developed. That is the thing commonly produced in garages, and that is the object the individual programmer uses in estimating productivity. There are two ways a program can be converted into a more useful, but more costly, object. These two ways are represented by the boundaries in the diagram. Moving down across the horizontal boundary, a program becomes a programming product. This is a program that can be run, tested, repaired, and extended by anybody. It is usable in many operating environments, for many sets of data. To become a generally usable programming product, a program must be written in a generalized fashion. In particular the range and form of inputs must be generalized as much as the basic algorithm will reasonably allow. Then the program must be thoroughly tested, so that it can be depended upon. This means that a substantial bank of test cases, exploring the input range and probing its boundaries, must be prepared, run, and recorded. Finally, promotion of a program to a programming product requires its thorough documentation, so that anyone may use it, fix it, and extend it. As a rule of thumb, I estimate that a programming product costs at least three times as much as a debugged program with the same function. Moving across the vertical boundary, a program becomes a component in a programming system. This is a collection of interacting programs, coordinated in function and disciplined in format, so that the assemblage constitutes an entire facility for large tasks. To become a programming system component, a program must be written so that every input and output conforms in syntax and semantics with precisely defined interfaces. The program must also be designed so that it uses only a prescribed budget of resources—memory space, input-output devices, computer time. Finally, the program must be tested with other system components, in all expected combinations. This testing must be extensive, for the number of cases grows combinatorially. It is time-consuming, for subtle bugs arise from unexpected interactions of debugged components. A programming system component costs at least three times as much as a stand-alone program of the same function. The cost may be greater if the system has many components. In the lower right-hand corner of Fig. 1.1 stands the programming systems product. This differs from the simple program in all of the above ways. It costs nine times as much. But it is the truly useful object, the intended product of most system programming efforts."

The reading of this classic is a great pleasure:

Brooks, Frederick Phillips, and Frederick P. Brooks Junior. 1995. The Mythical Man-Month: Essays on Software Engineering. Addison-Wesley.

Friday, October 30, 2020

Use Git and Github

 When working to a software you should try its design at best, then write your code with or without an IDE and finally check and test your code. Let’s for the moment skip the check and test question which has its own development and concentrate on the first phases of your work. Your design was good but your coding goes trough various modifications and, sometimes, you get lost. That is the time to use a Version Control System, a tool that allows you to recover older versions of your code and track your code history.  This is what summarizes this nice and free course on Coursera.   A VCS is a highly recommended tool even if you work alone. there are many of VCSs but here we concentrate our attention on Git, the free and open source VCs designed and implemented by Linus Torvalds also knows as the initial developer  of the Linux OS and maintainer of its kernel. If you are one who prefer to read books, there is at least one open book on Git, and you can find it here. Otherwise, the Coursera videos are certainly worth to be seen. Coursera does not only introduces Git  but also Github, but we will be back on it later. An interesting short introduction to both Git and Github can be found in this short tutorial. However, if you want to face the topics orderly, please first complete the video lectures. 

Git is not useful when you just work alone but when you want to share your work with other. In this case you have to chose a repository and Github is exactly what you are looking for (also for repositories there  are alternatives but among the many we have chosen the more popular for Open Source Software). In this case you have to lear what Github is, how it works and how to send and document your code over there. The Coursera videos and the tutorial help you in this task.  Obviously if a VCS is important when you work alone, it is much more important when you work in a group where potentially many people cooperate simultaneously to the same project. The videos and the documentation cover also these case. Best practice are important (see also here)!

The above resources introduce you to the command line use of Git but eventually you want to do it inside a IDE. Because I and most of my collaborators use Eclipse (for Java programming) I also address you to the Lars Vogel tutorial where it is shown how to use EGit.  Other valuable resources are this and these other ones

Thursday, October 22, 2020

On putting plants in hydrological models in practice

 The work of Concetta D’Amato Ph.D. started with joining together a reasonable evapotranspiration model with a solid infiltration model building a virtual lysimeter.

The basic is to have the two models to connect, and we, in GEOframe, have them, due to the work of Michele Bottazzi and Niccolò Tubini. The first built Prospero, the GEOframe component that use a novel formulation of the Penman-Monteith solution for evapotranspiration, the second designed and coded a Richardz 1D solver based on a novel algorithm. 

Obviously attaching the two models was not as easy as it can be thought, since the goal was to have a coupled system in which transpiration depends on water content and water content depends on transpiration with feedbacks among the few systems. The presentation illustrate how the coupling works and some simulations. At present the simulations are virtual reality, meaning that they are not parametrized according to some real case, but we hope to find appropriate data in the Watzon project. The figure, under which, by clicking you’ll find Concetta’s presentation, illustrates the directions her research can take in the following years. One goal was to do better than Hydrus 1D, being more reliable, more robust, more flexible, multiplatform and open source. Maybe we already accomplished it :-) ?

Freezing soil requires new algorithms

Assuming you have got the physics right and you wrote the right equations, which is not given for granted when you deal with freezing soils, you have to solve the equations. This paper, deals with this last topic: given the freezing soil equation it implements a new algorithm to solve it. This algorithm was invented by Casulli and Zanolli in their 2010 paper. They called it nested Newton, we renamed it NCZ from Newton-Casulli-Zanolli. It was implemented for solving Richards equation which present a spiky term called hydraulic capacity, that poses serous challenges to the convergence of the solver. We extended here to a new equation with the same type of terms. In fact all equations that involve phase transitions have terms of this type. 


To someone a new algorithm for integration of some equation can seem a minor achievement but, while in some type of simulation, the numerical errors of traditional methods can be somewhat constrained, in most of the simulation they do not and tend to increase up to a point that any prediction either quantitative or qualitative becomes useless. Obviously our case has an enormous effect when dealing with simulations of permafrost areas under the threat of climate change. If this introduction makes you curious, you can find the the preprint at The Cryosphere page, by clicking on the Figure above.

Reference

Casulli, Vincenzo, and ZANOLLI. 2010. “A Nested Newton-Type Algorithm for Finite Colume Methods Solving Richards’ Equation in Mixed Form.” SIAM Journal of Scientific Computing 32 (4): 2225–73.

Tubini, Niccolò, Stephan Gruber, and Riccardo Rigon. “A Method for Solving Heat Transfer with Phase Change in Ice or Soil That Allows for Large Time Steps While Guaranteeing Energy Conservation.”

Sunday, October 18, 2020

It's time to revise the GIUH

GIUH has been a valuable approximate tool for getting the hydrologic response. A review can be found in:
The question of maximum discharges treated inside the theory can be found in
The role of hillslope and channels was treated in:
Jointly with my first paper that proved the role of the geometric/topological structure of the river network in forming the hydrologic response,
these papers constitute, on my side, quite a body of contribution on the topic.  There are other greater contributor and their work is cited in the review paper I cited for first.


However, the theory has some limitations. Its applicability is based on the:
  • assumption that the rainfall is uniformly distributed (but C. Cudennec and coworkers were able to generalize it, see review paper)
having some recipes to get the effective runoff (i.e. separation of the total rainfall in quick surface water and baseflow and, besides, in evapotranspiration). Another limitation derives from the fact that
That’s why I went back to consider simpler reservoirs systems to get a clue of the interplay of the acting processes. This research work brought to the studies on representation of these reservoir models, to get the good old models streamlined for their structure, e.g.
and see especially the interactions among processes. The preprint-paper
is (among other things) a trial to get all the types of lumped models related with understanding where the diverse theories can be plugged together. Maybe a little convoluted as way of thinking but hopefully effective.
Another remark regards that once upon a time I was looking and satisfied with discharge, now I try to check the water and the energy budget and, therefore, the overall budget. Let’s see what comes next and if I am able to close the circle.  For who is still interested to implement a GIUH solver, please look at here.

P.S. - In the whole GEOframe/NewAge stuff, river geomorphology is present through the connectivity of the Hydrologic Response Units. Hence geomorphology is not absent: it is just not present in the simple way allowed by the GIUH that permitted to obtain those remarkable semi-analytic results present in the cited papers. 

Thursday, October 15, 2020

Save the date ! The GEOframe Winter School 2021 (GWS2021)

The third edition of the Winter School on GEOframe (GWS2021) will be held between January 7 and 16, 2021 in Trento, Italy. The course is devoted to Ph.D. Students, Post-docs, Young researchers (and Professionals!) interested in estimating all the components of the hydrological cycle (rainfall, evapotranspiration, snow-melting, and river discharge). The system allows to work out catchments from very small to continental (e.g. Abera et al., 2017a,b), up to build operational solutions, as the one used in in Basilicata. The aim of the course is to enable participants to run their own simulations and, eventually, on their own catchments and estimate the hydrological budget components. Previous Winter school material is freely available at the GEOframe blog.
Compared to what was done in the past courses, there will be more practice and a more detailed work on evapotranspiration and rainfall-runoff. It will be much more focused on exercises and on getting the water budget performed under various hypotheses on models' structure.

Due to COVID-19/COVID-SARS-2 the lectures and the work will be limited to, at most, 10 people in presence but it will be possible to follow the class remotely either synchronous online and asynchronous online (we’ll use a “blended” type of teaching based on Zoom). There is no fee or subscription for the asynchronous classes (and no personal support, just the support coming through the geoframe users mailing list). Synchronous  classes with no personal support though require subscription and the payment of a small fee of 10 Euros or (for Italians) a subscription to the Italian Hydrological Society. However, who requires a certificate of participation, and want personal support, must pay a fee (150 euros). Getting the certificate will be subject to present a report and an analysis of a catchment performed with the GEOframe tools. Tourists are  welcomed but obviously real learning requires exercise and some effort.



Instructors will be

Participants


Topics:

Next Day/Task

Subscriptions for having credits for this class and support can be obtained all year long after January 15,  2021: please visit https://webapps.unitn.it/form/en/Web/Application/winterschool/GEOframeWS2021 and compile the form or write to abouthydrology@gmail.con with subject: GWS2021 if you need more information. After payment of the fee you can have individual support and gain the participation certificate if you setup and perform an exercise with the GEOframe tools. You can do it on a your own catchment or we can provide you with a catchment and the required data. 

How snowy are the Alps ?

 A recent preprint was submitted to The Cryosphere were a large group of colleagues scientists analyzed the snow precipitation in more than 2000 gauges all over the Alps. This research is not only important for assessing the effects of climate change but also will be a benchmark to other more local studies on snowfall. 

I think it could be a good reading for many, therefore I am sharing its information here. By clicking on the Figure you download the paper.

Friday, October 9, 2020

Introducing the Extended Petri Net using an example

 This is, in my intention, gentle introduction to the Extended Petri Net on which we wrote a paper recently. That is the definitive reference on the topic. However, a presentation  can smooth out some difficulties that the constraint a paper imposes do not allow. So, below, please find the presentation, bu clicking on the Figure. 

The same material was presented elsewhere in a more compressed form but here, it was expanded to make it more understandable. The presentation is live on my VIMEO channel and if you have half of an hour you can follow it. 



I hope you enjoy it. EPN are not only a clear representation for hydrological and, in general, compartmental model of the water budget but, also, van be use to represent any compartmental model and can become a "lingua franca" for who analyzes and implement models.

Thursday, October 8, 2020

Snowdrops

                                                            Do you know what I was, how I lived?  You know
                                                            what despair is; then
                                                            winter should have meaning for you.
                                                            
                                                            I did not expect to survive,
earth suppressing me. I didn't expect
to waken again, to feel
in damp earth my body
able to respond again, remembering
after so long how to open again
in the cold light
of earliest spring--

afraid, yes, but among you again
crying yes risk joy

in the raw wind of the new world.

Louise Glück

Thursday, October 1, 2020

WATZON seminar series - I

 The Project WATZON  has almost finished the first year of work. It was a troubled year,  since the COVID-19 pandemic and we could not meet and do the field work required. However, we learn to use better the resources Internet brought to us. Upon the initiative of Paolo Nasta we started a series of on-line conferences on the tools and topics of the project. He is providing in these days seminars about the use of Hydrus-1D, a leading software for estimating infiltration by using Richards equation.  He accepted his material to be uploaded on my VIMEO WATZON Channel.



To interested people, Paolo suggest the following material

Lecture 1: Introduction to Hydrus-1D - First part

Lecture 2: Introduction to Hydrus-1D-Second part
Lecture 3: Treating tracers with Hydrus-1d
Lecture 4 - Going deeper isotope transport in Hydrus-1D
Paolo Nasta also wrote: "In the attached Excel file I extracted Fig.6 from the  Stumpp's paper. In this exercise we consider only delta18O transport. First of all, we all need to measure delta18O for each rainfall event. This is quite unfeasible. But...we need to sample rainfall as much as we can in order to make reliable interpolation of delta18O contents in ech daily rainfall episode! Stable isotopes of water are reported in the delta notation as the δ-content (‰), which is a relative deviation from the international standard V-SMOW (Vienna-Standard Mean Ocean Water). Mostly, the δ-content is negative. It is not possible to calculate with negative “concentrations” in HYDRUS-1D, and therefore, the user has to add an arbitrary value to all isotope data for the simulations (input and observation data). Do not take the absolute values though!
In the PRIN WATZON activities, we sample the isotope concentration in the rainfall (very frequently!), plant, soil (at different depths) and groundwater table sporadically (it means every two weeks hopefully). So we use rainfall-isotope conc. as input.
We use the other sporadic isotope concentrations (in soil, plant, groundwater) as observation data for inverse modeling in Hydrus-1D. In inverse modeling we optimize the unknown parameters related to water flow (vG parameters, Feddes parameters) and solute transport (dispersivity, solute root uptake etc.).
Once we get the optimized parameters, we can have fun and run long-term simulations with known precipitation, known rainfall-isotope and calculate soil residence time, or travel times from rainfall to transpired water or to groundwater in each site. Please, let me know. I think it is getting quite clear so far.
"