Monday, June 5, 2017

A method for determining optimal observations for prediction

This is the seminar given in Trento on May 30th by Henk Dijkstra (GS). Henk is mainly an oceanographer but the methods he illustrates, especially the Bayesian tools he develops towards the end of his presentation can be useful also in hydrological cases, so I am very happy to host his talk here.
The discussion that followed is here:



The slides of the talk are here. And here is the paper by Kramer et al. (JPO 2012), Measuring the Impact of Observations on the Predictability of the Kuroshio Extension in a Shallow-Water Model.





Sunday, June 4, 2017

How to misinterpret photosynthesis measurements and develop incorrect ecosystem models

At recent EGU General Assembly in Wien, I saw an interesting presentation by Professor Ian Colin Prentice (GS) entitled: How to misinterpret photosynthesis measurements and develop incorrect ecosystem models. I believe I already cited some of his papers (in our Precise proposal and “Can we trust Climate models?”), however, I did not faced his thinking directly. I would lie if I said that I understood his point. I am far too ignorant of the Carbon cycle and the way to measure it to understand. However, I accept the challenge to to start somewhere, because understanding the carbon cycle helps certainly to understand evapotranspiration
Please find below some relevant picture of his slides and, just after the paper(s) he cited. Probably reading those papers can be a starting point to understand.
I. C. Prentice, X. Liang, B. E. Medlyn , and Y.-P. Wang, Reliable, robust and realistic: the three R’s of next-generation land-surface modelling, ACP, 2015 
Hoffman, F. M., J. T. Randerson, V. K. Arora, Q. Bao, P. Cadule, D. Ji, C. D. Jones, M. Kawamiya, S. Khatiwala, K. Lindsay, A. Obata, E. Shevliakova, K. D. Six, J. F. Tjiputra, E. M. Volodin, and T. Wu (2014), Causes and implications of persistent atmospheric carbon dioxide biases in Earth System Models, J. Geophys. Res. Biogeosci., 119, 141–162, doi:10.1002/2013JG002381.
H. D. Graven, R. F. Keeling, S. C. Pipe, P. K. Patra, B. B. Stephens, S. C. Wofsy, L. R. Welp, C. Sweeney, P. P. Tans, J. J. Kelley, B. C. Daube, E. A. Kort, G. W. Santoni, J. D. Bent Enhanced Seasonal Exchange of CO2 by Northern Ecosystems Since 1960, Science 2013
Wenzel, S., P. M. Cox, V. Eyring, andP. Friedlingstein (2014), Emergent constraints on climate-carbon cycle feedbacks in the CMIP5 Earth system models, J. Geophys. Res. Biogeosci., 119,794–807, doi:10.1002/2013JG002591 
Ainsworth EA1, Long SP, What have we learned from 15 years of free-air CO2 enrichment (FACE)? A meta-analytic review of the responses of photosynthesis, canopy properties and plant production to rising CO2., New Phytol. 2005 Feb;165(2):351-71.
Ning Dong, Iain Colin Prentice, Bradley J. Evans , Stefan Caddy-Retalic, Andrew J. Lowe, and Ian J. Wright, Leaf nitrogen from first principles: field evidence for adaptive variation with climate, Biogeosciences, 14, 481–495, 2017 doi:10.5194/bg-14-481-2017 
Zaehle, S., Medlyn, B. E., De Kauwe, M. G., Walker, A. P., Dietze, M. C., Hickler, T., Luo, Y., Wang, Y.-P., El-Masri, B., Thornton, P., Jain, A., Wang, S., Warlind, D., Weng, E., Parton, W., Iversen, C. M., Gallet-Budynek, A., McCarthy, H., Finzi, A., Hanson, P. J., Prentice, I. C., Oren, R. and Norby, R. J. (2014), Evaluation of 11 terrestrial carbon–nitrogen cycle models against observations from two temperate Free-Air CO2 Enrichment studies. New Phytol, 202: 803–822. doi:10.1111/nph.12697 

César Terrer, Sara Vicca,Bruce A. Hungate,Richard P. Phillips,I. Colin Prentice, Mycorrhizal association as a primary control of the CO2 fertilization effect, Science 2016 

Wednesday, May 24, 2017


This video presents part of the so called “Hazard Map” of Trentino Province in Italy. This is a work started in 2004 that aimed to substitute and simplify previous hazard maps. these maps are used for a variety of scopes that goes from urban and regional planning to civil protection scopes.
As reported in the Province website relative to the Hazard map, the following hazards are mapped:

Hydrological and geological hazards about:
  •  rivers; 
  •  torrents; 
  •  hillslope; 
  •  snow avalanches. 
Other hazards:
  • sismicity; 
  • unexploded bombs (after second world war); 
  • forest fires
The seminar was part of a short course held by Ing. Claudio Bortolotti, a former director of Civil Protection in Trentino, entitled: Integrated Civil Protection Systems.

The speakers were: dott. Mauro Zambotto, directory of Trento province Geological Service (TPGS), dott. Franco Daminato, and dott. Riccardo Campana, geologists at TPGS

I think that the seminar was interesting and highlight the practical use of many tools that I try to popularize to my students.

Saturday, May 20, 2017

ARS-AGEs is finally public

That is a news that I was waiting since a long time. AGEs is one of the other models that is based on the Object Modelling System infrastructure, and therefore a possible source of available components in our modelling based on GEOframe and JGrass-NewAGE tools.  I always beg for they to do this step, in order to have a clear basis on which to start collaborations and convergences. Finally they did.
Please, click on the image above for accessing the Bitbucket public repository.  They write:

"The Agricultural Ecosystem Services (AgES) model is a modular, Java-based spatially distributed environmental model which implements hydrologic/water quality simulation components under the Java Connection Framework (JCF) environmental modeling framework."

Actually, I do not like the word "JCF" which I do not know what exactly means, but is, anyway, a step forward openess that I appreciate.

Monday, May 8, 2017

Dolle's Water by Andrea Zanzotto

Now to console me 
with a long visit 
comes the water of Dolle 
that brought ten hills to the town 
fled among bees and their keen castles 
touched the sensitive shapes 
of an island of pure sand, 
now comes this water I long for 
because it shines through your 
twin limbs; 
because it lingered 
a long time in the shadowed coffer 
where the fig-tree stands guard 
and the sun no longer makes moss or fern, 
where the sky’s festive scenes 
are already open. 
Water ignorant of clay 
that already flows from its tangles, 
proud of the momentary red 
of flowers celebrated by this hour, 
you go lightly touching and probing 
the shyest solitudes: 
let it stay mine, 
for my snail’s lamp 
for the garden the dwarf sharecrops, 
water from the thickest alphabet 
water with its messages 
of noble invasion 
of stars returning from alps 
now heavy with silver, 
water promising 
a night cool as a tomorrow

(Translation form Italian from here)

Ora viene a consolarmi
con una lunga visita
l’acqua di Dolle
che portò dieci colline al paese
sfuggì tra le api e i lor castelli di acume
toccò le forme sensitive
di un’isola di pura sabbia,
ora viene quest’acqua ch’io sospiro
perché traspare dalle tue
membra gemelle;
perché a lungo
indugiò nello scrigno d’ombra
dove il fico s’affaccia guardiano
e il sole non fa più musco né felce,
dove sono già aperte
le scene da festa del cielo.
Acqua ignara della creta
che già fuoriesce dai suoi viluppi,
fiera del rosso momentaneo
dei fiori celebrati da quest’ora,
tu vai dovunque lambendo e tentando
le più ritrose solitudini:
lasciatemela mia,
per la mia lampadina di chiocciola
per l’orto di che il nano è mezzadro,
lei dal fittissimo alfabeto
lei che ha i messaggi
di nobili invasioni
degli astri che ritornano dalle alpi
ormai pingui d’argento,
lei che va promettendo
una notte fresca come un domani.

Friday, April 28, 2017

Quantify biological complexity - by John Baez

Just verbatim form the Azimuth blog (here). I place it also here for not forgetting it. Maybe later I will also comment it.

"Here’s a video of the talk I gave at the Stanford Complexity Group:


You can see slides here:

Biology as information dynamics.

Abstract. If biology is the study of self-replicating entities, and we want to understand the role of information, it makes sense to see how information theory is connected to the ‘replicator equation’ — a simple model of population dynamics for self-replicating entities. The relevant concept of information turns out to be the information of one probability distribution relative to another, also known as the Kullback–Liebler divergence. Using this we can get a new outlook on free energy, see evolution as a learning process, and give a clearer, more general formulation of Fisher’s fundamental theorem of natural selection.

I’d given a version of this talk earlier this year at a workshop on Quantifying biological complexity, but I’m glad this second try got videotaped and not the first, because I was a lot happier about my talk this time. And as you’ll see at the end, there were a lot of interesting questions. "

A new topic for a Ph.D. in Hydrology at University of Trento. Modelling water flows under phase transitions

This study starts from a pore scale view of flow in soil and aggregate it at the representative elementary volume, (REV), scale according to statistical assumptions, to obtain new forms of the Richards equation. Flows are assumed to happen under normal and/or freezing conditions and under evapotranspiration demand. Transitions from unsaturated to saturated conditions will be properly accounted in all types of flow. The theoretical work at the basis of this proposal is contained in Dall’Amico et al. 2011 and Tubini, 2017. At the beginning the system will be modeled by coupling the water budget equation and the energy budget equation, neglecting vapor mass budget, as usually done. The candidate should take care of integrating the equations with appropriate and sound numerical methods that guarantee mass and energy conservation, following the footsteps of the work by Casulli and Zanolli (2010) and work for possible extensions.

There are various possible further development of this research. One is to couple the water and energy budget with surface waters simultaneously solved, another is to deal with water vapor explicitly. Others developments could come ongoing.

The informatics behind the code will follow (and, in case co-develops) the developments pursued by dott. Serafin, Ph.D. work inside the Object Modelling System, version 3 or subsequent (OMS3, David et al., 2013), that will take care implicitly of execution of parallel processes and will provide various services to computation (e.g. Serafin, 2016).

All the code developed will be done in Github (or similar platform), inside the GEOframe community and will be Open Source according to the GPL v3 license.
The candidate will take care of implementing, besides the code, the appropriate procedures for continuous integration of the evolving source code, and s/he will be also asked to maintain a regular rate of commits to the common open platform. Despite these conditions, and being free and open source, the code will be intellectual property by the coder. This will be guaranteed also by the components-based infrastructure offered by OMS3, which allows to better define the contributions of anyone. (See also: For incoming students, The tales of open source codes).

The implementation part will be followed, accompanied by testing activities, either for mathematical consistency, than for physical consistency with experiments and field measurements. These will be made especially by Dr. Stephan Gruber (GS) group at Carleton University, where the candidate will be asked to spend some periods od his/her doctorate. Participation to experimental activities will not be intended to be purely passive, the candidate will be asked to actively participate as much as feasible and reasonable to any part of the research.

The Ph.D. student is intended to produce, besides working and tested codes, also at least three papers in major journals (VQR Class A), of which, at least one as first Author.  Duration of the doctoral studies could be three or four years.

This project can enter either the curriculum C (Environmental Engineering) or the curriculum A (Modelling and Simulation) of our doctoral school.

For information please refers to riccardo.rigon <at> unitn.it

Essential References

Casulli, V., & Zanolli (2010). A nested newton-type algorithm for finite volume methods solving Richards' equation in mixed form. SIAM J. SCI. Comput., 32(4), 2225–2273.

M. Dall’Amico, S. Endrizzi, S. Gruber, and R. Rigon, An energy-conserving model of freezing variably-saturated soil, The Cryosphere, 5, 469-484, 2011, doi:10.5194/tc-5-469-2011.

David, O., Ascough, J. C., II, Lloyd, W., Green, T. R., Rojas, K. W., Leavesley, G. H., & Ahuja, L. R. (2012). A software engineering perspective on environmental modeling framework design: The Object Modeling System. Environmental Modelling and Software, 39, 1–13. http://doi.org/10.1016/j.envsoft.2012.03.006

Serafin, F., About graphs, DSL and replicable research, 2016, http://abouthydrology.blogspot.co.at/2016/11/about-graphs-dsl-and-replicable.html

Tubini, N. (2017, March 31). Theoretical Progress in freezing-thawing process studies. (R. Rigon, F. Serafin, & S. Gruber, Advisors.).

Thursday, April 27, 2017

New Insights in Permafrost modelling (EGU Wien 2017)

This talk (and work) is another rearrangement of Niccolò Tubini work, and continues Matteo Dall'Amico Ph.D work. It derives the set of equations for water flow in freezing soils based on the same assumptions by Matteo but the results is slightly different at the end. 

Click on the figure above to see the presentation. Hope you enjoy it.

Dalton Prize 2017 to Dani Or

This is the video of Dani Or (GS) lecture for the prize he received at this year EGU Wien. Dani is an outstanding scientist and any of the things he does deserve attention and a reading. He talked about evaporation and others of his lectures were already linked in this blog.

Here below a presentation of Dani.


Here his lecture (unfortunately a little out of focus, but still visible. I hope that there will be an official, professional record from EGU).

Monday, April 10, 2017

Open-source software for simulating hillslope hydrology and stability

This is the material of the SC34/NH10.2 ECS: Open-source software for simulating hillslope hydrology and stability
by Giuseppe Formetta, Francesco Serafin, Riccardo Rigon, Raffaele Albano and Luigi Lombardo (co-conveners)

session of the  2017 EGU meeting in Wien.
For all of this work it is necessary to download a number of softwares.

Thursday, March 30, 2017

Modelling discharge in an Alpine basin with JGrass-NewAGE

This and a related post reports about the Master thesis by Niccolò Tubini and Stefano Tasin. It was a couple years ago that I graduate my last Master guy, and I am happy with these two graduations.
Stefano thesis is in Italian. So I am summarising it a little bit below.
JGrass-NewAGE has a a snow module that was developed by Giuseppe Formetta (GS). Giuseppe developed also a component called  Adige-Hymod for runoff estimation. The two were not tested conjointly (well, they were), and we would like to have a new case to understand more about the behaviour of the model and sharpen the methods we use with it.
 
Stefano did it, making leverage on the NewAGE database of river Adige and using, side by side with NewAGE, GEOtop as the true to reproduce in matter of snow. Other directions could have taken, but Stefano chose this one with excellent results. He had in mind a relatively small basin in the Norther part of Italy that was known to be dominated by snow (and glacier melt) and he wanted to investigate how much of discharge depends upon snow melting. The figure above is one of his results, which shows an excellent discharge fitting and quite impressive demonstration of how snowmelt counts in this case. Thinking that snow on the Alps is going to almost disappear cause the climate change, the basin will go to a quite large change in the discharge regime. It is foreseeable that winter discharge will grow in place of the summer ones, with possible modifications of the discharges distributions.
The thesis and the simulations files used are here.

Theoretical progress in freezing-thawing processes studies

This and a related post reports about the Master thesis by Niccolò Tubini and Stefano Tasin. It was a couple years ago that I graduate my last Master guy, and I am happy with these two graduations.
Niccolò thesis is about modelling permafrost.

I already worked on it during the Ph.D. thesis by Matteo Dall'Amico, obtaining interesting results, which were published in this 2011 paper. From it we built. Initially the idea was that the work by Matteo dall’Amico was clear enough to go directly to a full three-dimensional implementation of 3D algorithms on a unstructured grid. That was actually not the case an we had to rework all the theory. I do not want to waste its reading. So, if you want to know the story, please click here

The presentation Niccolò gave for his Master (Laurea) degree is here.

Wednesday, March 29, 2017

A field trip to Posina catchment

Since a couple of years we started to work on modelling Posina catchment in the Italian pre-alpine area. There University of Padua colleagues, Marco Borga heading the group, started hydrological measurements since many years. Posina is a small catchment (116 km$^{2}$) located in the Alpine foothills of the Veneto Region in Italy. The elevation difference of the basin is 1820 meters. The climate is characterised as wet, with annual precipitation of 1,645 mm and annual runoff of 1000 mm. For a detailed report on its hydrological budget please see Abera's et al. paper.
Finally we had the occasion to visit it with Marco Borga himself and Giulia Zuecco on March 22 afternoon. Please here below find a photographic synthesis with  some comment.

1 - The position of the gauge station at the Stancari outlet.
2 - A view of the position of Bazzoni gauge position.
3 - A view of the Ressi sub-catchment top with a few of the buckets
4 -A more panoramic view (was not a sunny day)
5 - A panorama view with Giulia, Marco and Marialaura from left to right.
6 - Going inside the catchment, to explore these channels (?) or hollow. Soil depth is not very large here. Many trees just did not succeeded to stay still due to the limited roots development. Here and there the bedrock outcrops, but in other position, especially in the concave parts, soil seems to be deeper. 
7 - Here the bedrock channel is visible. We can see also the soil section with the roots of the tree in the foreground.
8 - The V-shaped weir where discharge and water level is measured
9 - The water sampler to do isotopes estimations
10 - A view of the sampler - weir area from below. Please observe the big fallen tree that just slightly touched the box with the sampler when falling. 
11- The throughfall measurement site. 500 hundred buckets for 500 hundred square meters
12 - The stemflow measurements 
It was a nice a pleasant trip! We hope to continue the collaboration further. 

Thursday, March 16, 2017

Extreme Rainfall Distribution Estimation with Python (as a scripting language)

This post collects the Notebook used in my classes of Hydraulic Constructions and Hydrology for estimating the IDF curves. It is actually a work in progress.

For who wants to start with Python (for hydrologists), I suggest to give a look to my blog post "Python for Hydrologists". Here a brief summary, for the laziest.

There are a lot of resources to start with python, but for hydrologists, a recommendation is to use:
with a preference for the first one (keeping in mind differences between Python 2 and Python 3).
 Soil Physics with Python: Transport in the Soil-Plant-Atmosphere System, by Bittelli et al, is al, is a book on soil science which is quite appealing (as seen the TOC): the kindle version cost reasonably but I do not have it. Python programs are available here.

An overview of Scientific libraries and softwares of general use in Python is given at scipy.org, from which I extract these links to documentation:


3 - Estimating the parameters of a Gumbel distribution with

5 - Fitting the curves (some error here)

Data:

Paperopoli 2
Paperopoli 4

For who wants to do the same numerics with R, she should look "A few R scripts useful for hydrologists".

Friday, March 10, 2017

The tale of open source codes

Prologue

Why did I choose to produce with the people directly working with me (ph.D students, master students, postdocs) open source software ?

- because is good for science
- because I am paid by a public institution
- because it is a neutral conditions that can serve the rights of all the participants (in particular mine of freely use and modify the software at my will an defend myself from who, people or institution, would like to close the software, even against me). On the other side,  my intention is clearly that my projects serve as a seed for developments of my students (or others) who can freely use the products of my research and maintaining it alive beyond me and despite me. *

I use GPL (for its interpretation, see here) but many others licences could work.

A declaration

In this way, I think, I have the right to claim to be able to use or peruse the software outcomes from my group. I declare that I want to use “fair play” rules, but, it should be clear that these rules cannot extend to limit my research freedom. People who claim the participation to papers where they give no contribution, except having producing the code that we produced together, have wrong arguments. People who claim to be involved in projects or researchers, without any other reason that I want to use the software they contribute (under GPL), have wrong arguments.
Neither they can claim that I have to warn and tell them personally what I am going to do in my research with the common code, for having their consent.
They would be right to protest, only if I would not enlighten their contribution on previous work properly.
My research for my own belief is actually very public and its evolution too. It can be found at the abouthydrology blog. My core research is shared with my teamwork. This includes just the people of whom I have direct responsibility for age and rule (Master students, Ph.D. students and postdocs) and whom I sustain with funding, my own time and ideas.

With all others, including my masters, and my former students, colleagues, friends, women and men that like my research topics and achievements, and me, I can have collaborations. This means that we can share part of our views, beliefs, discussions, fightings, friendship, papers, parts of code. However our own agendas, in this imperfect world, do not coincide, and if they do, this happens for an incredibly short time. It seems it is a declaration of distance, but it is just consciousness of how life works, and the first step to start an effective and respectful collaboration.

Q&A

Can my students refuse to develop OS software ?
No, as soon as it is the product of common intellectual efforts in which they, maybe, write the code, but I will say what to write.

Do I start collaborations in which not OS code can be developed ?
Never say never. However there should be very strong reasons because, from my side, I to support this. Certainly in projects there could be partners that develop non open source software, but this falls in the responsibilities of who gives the financial support.

Is the requirement of open sourceness enough ?
No, it isn't. Open Sourceness is useless if not followed by good practices of using open repositories and collaborative modalities of action.

Can my students refuse to learn these practices ?
For the common work no. I am not responsible for the rest. I tend to fully book their time, though.

Do I start collaborations where these practices are not followed?
I would prefer not, but I do. Certainly collaborations can be at different levels and rarely they are about co-producing software. I would not participate to joint projects where I put ideas and expertise and others write closed codes, unless they pay me or my group a lot. Really a lot. I can participate to projects where other subjects put their ideas, or ideas from literature, in their own closed code, and I put mine in OS codes. However, the situation I prefer would be a common production, as a community, of open source codes.

My own use cases

Here below I summarised (with quite large simplifications) my software history in order to further justify what I wrote above.

Professor means who puts science, time, and money (funds derived by projects). Student means who puts time and science. Companies means they put time and money and business related efforts. Agents, Subjects are generic actors of the play (they can be either students, professors or someone else). Community is the informal group that happened to gather around the projects and, eventually,  evolve them.

Case 0

Professor Z writes the initial library. On top of that A builds radiation budget. Student B writes surface water flows. Student C implements soil-atmosfere interactions. Students D writes vadose zone components. Student E writes snow treatment. Student F rewrites snow components, then rewrites most of the codes interacting with student G and student H. Student I writes codes for landslides triggering treatment. Student G writes a small but important portion of the a little but successful part of the freezing soil hydrology. Professor L hires F. F continues to rewrite parts. G start a huge operation of cleaning the code, moving it to C++, uploading it to an open repository. C comes back and starts to use the code in his research and occasionally hires H to do some ancillary work for treating data. In meantime G  has founded a company where the common code is the basis of the business. M company, initially hired by L, works on the code to refactor and enhance it. M works collaboratively with G and F. M to setup continuos integration. Student N starts to produce executables for the main operating systems and eventually on Cooker (fictional name). M embraces immediately this philosophy.


Case 1

Professor Z writes the initial library. Z writes more than fifty tools for terrain analysis. Student A (not the same as above) ports them to a major Open Source GIS. Z and A start the construction of a new GIS, say JG. Initially JG contains just the the terrain analysis tools and some simple hydrological model. They start to do schools for financing their project. This works for some years. Student B, in the meanwhile, has joined the crew and A & B funded the company AB. They live with schools, supports from a main project of Z and other resources (a main research projects). A cleans the tools' suite and inaugurates the name JGT for them. Z uses JGT in his classes.
Z, A, and B decided to join the development of UGIS. Some research projects supports them together with resources raised by the company AB on its own. Students C and students D write some further modules. UGIS  funding disappears, and UGIS slowly becomes an almost inactive project. AB brings JGT to an intermediate product ST. Z continues to USE JGT in his classes. AB finally joins the development of a new GIS, say GS.
(In the middle,  A adds new tools, AB wrote an Android app, Aapp, and expands its business. Aapp is not  related to JGT, but worth to mention). During the years A and B get a Ph.D. whose topics are related to the GIS work. Student E with a small effort brings back JGT also to a platform, OI that Z uses with his students.

Case 2

Thanks to an unexpected financial support from project 00, Professor Z hires a five students to build from the scratch a new modelling platform. For this new software enterprise, he and company AB (funded by his former students A and B) chooses the open source framework OI. He hires former student C, to help software developments and former student D and E for the general management of the project and data gathering, respectively. C works more on improving and enriching JGT (see case 1 above) which serves as a basis for the terrain analysis functional to modeling. A and B develop a full suite of model components (the new paradigm) for: temperature and rainfall interpolation, rainfall-runoff, evapotranspiration and various tools to visualise components' inputs and outputs. AB also designs and populates an SQL database that contains all the data of the projects. The projects 00 ends. The Institution that supported the project close it in a drawer.

With other financial support, former 00 project's tools are maintained in life. Open source framework OI is changed for open source framework OM with a notable reduction of code lines (but it is a huge code effort, indeed, almost entirely on AB shoulders). With embracing OM, also starts a research collaboration with professor U and W.

Student E comes into play. He realises that rainfall-runoff does not work well. AB company has to survive on its own and cannot give very much support (https://vimeo.com/144089061). E implements a new rainfall-runoff model. AB, however, hires F for a small project where he works on radiation. Eventually, E refactors F's work and highly expands it. E adds a new snow modelling component and does/refactors evapotranspiration. In doing this (pouring sweat and blood) he, however, has the guidelines of the open sources codes already written. E spends some periods at U and W. E also refactors and enhances the Kriging code. Eventually E graduates and starts his career as post-doc elsewhere. In the meanwhile he finalises his research in a series of papers.
Student G comes. He does not have programming skills, but quietly learns to use the components of E and produces some interesting papers where E is co-author.
A new student, H, comes into the game. She works first on radiation on top of E code, then she starts to implement tools for travel time analysis and another rainfall-runoff component.
Student L comes. He  has a strong attitude for informatics. He brings-in new ways to manage projects. H and L implement the OpenOpenSoftware repository, and the site BeatifulGEO (names are fictional, but tools real). H refactors the old code,  and together with L (who, sort of, leads the learning process), introduces design patterns for increasing code reusability. L provides the trickery to have continuous integration on OpenOpenSoftware using GETIT and connects software deployment to ISTOREIT to store official versions of the components. Students M and N come in to stage and start to use the code. Professor Z (with the help of H) starts to use the components with his students for his classes. Student L evolves the original OM capabilities to allow for more flexibility and to increase the computational power of the models. H brings-in her models into the new infrastructure.

Discussion and Conclusions

The above is a summary (where, I say again, I simplified many passages) of my main software enterprises. Could have they been evolved all differently (and better) if I would not have applied an open source strategy ? Probably yes, but I should have constrained the students to a contract about the property of the software. In this way I would have deprived my students of parts of their own work.
At the same time, I could not have left the software simply to them. The histories themselves show that I built my own work and research on the software we develop, and being free to use it and modifying it was a necessity. If I have needed to ask permission to use it, to sign a contract or so with someone (for instance who gave financial support), all the development would have been much more difficult to pursue. The same apply for other Actors who invested time and resources in the software development just because it was open. They are usually singles or low budget companies that could not have afforded expenses related to other type of licenses and be subjected to limitation of the software use.
Other researchers used the model. Being it free and open source was a clearly an added value for them.

Keeping the software close and commercial, besides not having scientific reasons (which require the contrary), would have obliged me to change myself in a businessman and turned away from my science. There are several cases of scientists that turned to captains of companies. But, for instance Stephan Wolfram, a gifted scientist, did not give very much contributions to science after he devoted his energies to MathematicaMathematica (probably the best computing environment ever) itself is his main achievement (which is not depreciable), despite his own claims on "New kind of Science"s.
The overburden required for managing a commercial software is not for all and has its own dynamics, that personally I  could not bear.

The fact that my code is free and open source has allowed (not without difficulties) self-instruction of new incomers. Various Agents had the possibility to start experiments and investigate new directions of development. Nobody needed to ask for starting them. Asking is a process that would have decreased dramatically people or groups pro-activity.

The Community had benefits from this policy. In some cases, single Actors could have thought that their contribution was not recognised enough and did not give to them an advantage. Their argument is  flawed. All of them had advantages from the collaborative environment and nobody (me included) could have produced what s/he has achieved without building on the shoulder of others and other open source projects.

Forgetting the above, some feel that their work is not enough protected, and being all open source, newcomers can more easily jump in and take advantage of their work.
Uncertainty on future, a competitive society, the pure necessity to find something that pays you for a decent life incline even to bests to a moderate selfishness or a moderate parasitic behavior. They do not want to give back to the community, after having got a lot from it, and act defensively.

Well, this behavior is absolutely possible if their developments do not use the original code that was produced as GPL. In particular, the components strategy used in project 2 above allows for building on top of the open source material new, undisclosed material, that anyone can use for his/her own profit, with a non open license.
I have to warn, however, that if the moderate parasitism grows too much, enthusiasm that is always necessary decreases,  the projects die, the source of benefits disappears and the community falls.

I would say that a mild parasitism is functional to the community if it is necessary to sustain the collaborative Subjects, and if eventually the Subjects give something back to the community. Parasitic Subjects themselves act in favour of the community by spreading and advertising the products, and sooner or later this will be bring benefits back (so do not blame them, they are, in any case, part of the stream).

Some Subjects actually wants an opaque management of the GPL philosophy in which people maintain an informal (but they pretend recognized) property of the software that goes beyond the copyleft and the intellectual recognition of their contribution. This would imply, in their mind: preferential redirection of funds towards them; involvement in papers or conferences contributions that use their code; veto power towards actions of thirds.
These desiderata are based on misunderstandings. It is clear that they will be involved in papers, conference, and decision. Any (wo)man and community of good-will will apply this policy in their favour, if they do not grow too greedy. But these actions are not mandatory and not even necessary. GPL does not implies them.

To be more clear, especially in hydrology, the market out there treats our model and softwares as a fungible commodity, that is, the market tends to treat all the codes as equivalent or nearly so with no regard to who produced them. (I think this is wrong, highly wrong, when brought to an excess).
But also the internal market, inside the community, treats them as commodities, meaning that, it would be dysfunctional, it would cause a waste of precious time, but any contribution is perceived as a thing that can be replaced (this is part of the not said history of 0,1,and 2 projects). Everybody is important nobody is necessary.


The A. paradox

One common argument of reluctant open sourceres  is: “I did not have still tapped the results of my own work and I should share it (statement 1)”, or "if I share it, others will use it without me and I will have no personal gain(statement 2)".
The first danger can be overcome, by an appropriate delay of the disclosure of documentation and explanatory material (I would not argue that keeping industrial secrets is useless, in general, however). That is: it is matter of having strategies that prevent the negative cases. In our field, however, being everything perceived as a commodity (see above) nobody will care to use our model or achievement instead than another one that gives what is (wrongly perceived) as similar, especially if our code is not known. Being open source with proper support actions helps model spreading.
Besides, looking at my histories (see also here), software changes fast and is, by no means, immutable. Histories 0,1, and 2 are signed by change. So the advantage one has with a new code in hands is ephemeral. In my own estimates you have just a a year of advantage for small codes, and a few years of advantage with a large and complex code. This small advantage, if you are smart, can be appropriately managed and used to produce new and more innovative code and so on. (Open sourceness is against stagnation).
Often, however, it is not the the fear of far away threats that makes problems, but the fear of close by Agents. Guy A fear that B in the group who came in after her/him, will get positions or funding with his/her work. I would say that this could happen but it is difficult. In a fair (not fear) competition A always wins over B, if the quality of B can just be attributed to codes that A developed. The real problem is when B is much better that A. But in that case, having A work for B is not important. B will get rewards instead than A almost always. For A, the best thing, in the medium range, is to collaborate with B.
What, finally I really call the A. paradox is in statement (2). If it is so easy to grab your work, then it would be equally easy to anyone to replicate it. Therefore your work is not giving to you any competing advantage, even if you keep it secret for a while. If it is not easy to grab, then, who wants to use it proficiently needs you. So you are the winner, not because you keep your code top secret, but because all the issues it solves require a complex expertise that only you, the author can have. So ….

Epilogue

Professor eventually Z disappears. Not because he dies (please do exorcisms), but because his role, in the growing group of people around projects has become more and more marginal. Subjects also acquired maturity and as well as the will to maintain the advantages that the work has produced with respect to competitors.
This passage requires that the initially informal community establish as a formal Community (they wrote here for Academics) with its rules, etiquette, and wise management. This, in turn, requires Subjects coordinate and share alike their views, plan together new developments, plan events to make the common work to grow. Balkanisation of the code (which GPL could allow) and internal conflicts (never avoidable, having the Subjects different agendas) should be managed appropriately, and this requires clear agreements, smart actions, good will, and wise arguments.
If the community grows, everybody would be safer, because cooperating is better than competing (see also coopetition).
A partial adoption of the Open Source strategy is instead very useless. Open source codes that are practically not available (as those that are open source but not freely downloadable) cannot grow a healthy community and, sooner or later, die.

* A final note

Actually even if in my intention is a project also for my students, not a few of my students do not deeply endorse it. Reasons for this can be, maybe found in their personal history, the chemistry of their bodies and minds, or something else, which is hidden to me. So far, I  overreacts feeling myself betrayed, when they dismiss in what I believe it is right. So, probably my attitude is not is not correct. Sons do whatever they want, and probably they are right to try to find their way. So I have to conclude that the above is MY dream, and I will not be upset anymore, if my academic sons search their own in a different way.

Friday, February 24, 2017

Scale & Hydrology in 2020

This is the second lecture given at Potenza. The topic Salvatore Manfreda (GS) gave me is about how to move from one scale to another in Hydrological Modeling. I started from distribute modelling. My point there is that, for some tasks, distributed modelling can be scaled up to millions of square kilometers, and so,  upscaling theory is, in principle, not necessary.
But clearly this is a provocative statement I made just for pushing away some misconceptions.
Then I passed to consider other ways to upscale problems. Through simplifications, integration, heuristic thinking.  Eventually I gave a sight to "theories of all" that were so popular the last dacades and still remain possibilities and ideas to explore. By clicking on the figure you go on the presentation.

Wednesday, February 22, 2017

JGrass-NewAGE: the first Potenza lecture

This is the presentation of JGrass-NewAGE structure and achievements. A lot of posts were dedicated to it. But there is always space for new perspectives and details, since it is a work in progress where talented students of mine put all of their efforts.
JGrass-NewAGE has grown to a stable and operational set of OMS components, documented in the GEOframe blog. We also developed good practice for software design and traceability of our efforts meanwhile that could be interesting to know.
Aficionados will recognize (by clicking on the figure) that the presentation contains various topics already largely spread in other posts. However, there are a few small little things that could be interesting. Or, BTW, the arrangement given here to the matter, can clarify some choices that could have been seen obscure in other occasions (the slides are in Italian but contain link to other material and papers in English).

Monday, February 13, 2017

GRAL

Times ago we had the Gruppo Italiano delle Catastrofi Idrogeologiche (GNDC, Italian Group of Hydrological and Geological Hazards). But as the site testifies it languished. When Civil Protection beaome more dominant, not maybe the knowledge, but certainly the funding went in other directions (or was it just the natural fate of all things ?), and the all the initiatives stopped. The discussion never slept, and, BTW the Italian Hydrology is much stronger now than used to be.
So it is now the time for a new scientific initiative, with renovate objectives, to fill the gap between research and practice in defending our beautiful country from flooding. This is Gruppo Alluvioni. If they're roses they'll bloom (Time will tell).

Monday, February 6, 2017

Hydrology 2017

This year I decided to introduce strong news in my Hydrology course.  Not only a change of topics, but also a change of perspective. I increased widely the hours in the lab (up to 60%) of the class, and I arranged the lectures in a way that they could be followed by a three hour laboratory. Almost no lecture will be without numerical experiments. Another innovation is the use of Python instead of R.
I made this because of the large endorsement Python had among hydrologist and because:

  •  its object oriented structure is much more firm than the R one. 
  •  Besides, Python seems to be easy to learn by engineering students. 
  • Some of my colleagues seem to agree to converge toward the use of Python in their classes
R remains the first choice to do statistics. However, we have limited time. The class is 60 hours, and the material to convey a lot.
Here it is the foreseen schedule of the class:
Corso di Idrologia 2017

Legend: T - Theoretical lecture  - L - Laboratory class (this can include theoretical parts, but mostly students will exercise with tools)
  1. T - Introduction to the class
  2. T - A terrain analysis  primer. 
  3. L - Introduction to QGIS. Introduction to the JGrasstools in OMS.
  4. T - A little of Statistics and Probability. 
  5. L -  Delineation of catchments' characteristics with JGrasstools and QGIS.
  6. T - Precipitations. Mechanisms  of formation of precipitation. Ground based statistics. Extreme precipitations. 
  7. L - Intro to Python - Loading/reading files. Time series and their visualisation. (See Notebook 0 an 1 here.)
  8. T - Extreme precipitation statistics (parameters' estimation)
  9. L - Estimation of extreme distributions parameters. (See Notebook 2 to 5 here.)
  10. T -  Radiation (YouTube 2017). 
  11. L - Estimation of shortwave and longwave radiation in a catchment (data, executables, sim files are available through Zenodo. Who is interested in the source code and further information, plese refers to GEOframe or the Github GEOframe components site). 
    • A brief rehearsal of the matter given by Michele Bottazzi (M.B.) (YouTube)
    • Estimation of solar radiation with JGrass-NewAGE components (YouTube) by M.B. Part I
    • Estimation of solar radiation with JGrass-NewAGE components by M.B. (YouTube) Part II
  12. T - Spatial interpolation of environmental data
  13. L - Practical spatial interpolation of rainfall and temperature.  
  14. T - Water in soils. - Darcy-Buckhingham law- Soil water retention curves and hydraulic conductivity. 
  15. L - Numerical experiments on soil water retention curves and hydraulic conductivity.
  16. T -  Richards equation and its extensions.
  17. L - Experiments with a Richards 1D simulator
  18. T - Elements of theory of evaporation from water and soils - Dalton. Penman-Monteith. Priestley-Taylor
  19. T - Estimation of evaporation and Transpiration at hillslope scale
  20. L -  Estimation of evaporation and transpiration at catchment scale
  21. T - Water movements in a hillslope and runoff generation
  22. T - On the impact of climate change on the hydrological cycle (YouTube2017)
Verifications and tests 2017