Monday, February 3, 2020

4 Researchers Who Want to Develop Hydrological Models Without Reinventing the Wheel

Maybe you can save efforts and you do not need to reinvent the wheel. Therefore I wrote about our project that lasts since almost fifteen years. We use an open infrastructure, the Object Modelling System v. 3 and developed an open system, called GEOframe that has unmatched capabilities to be expanded while not wasting the old work done and maintaining your independence as developer.

The best way to understand OMS and see how it works is probably to give a look to the material of our last winter School on our system GEOframe. Specifically, this is the page of introduction to OMS. But also starting from installations of the material can give insights.
There you can find links to the OMS v3 pages at CSU and original material. Our choice of OMS and Java was long meditated. This was my assessment almost ten years ago. This is instead the assessment made by NIWA six years ago.

Here  you can find some thinking about the choice of programming  languages. We use, in practice, a mix of Python and Java. Python for treating and visualizing data. Java for writing our models.
When properly used Java is not so slow (twice slower of well written C++) and was favored by having an incredible suite of tools helping proper programming, project building and code maintenance that make the experience to develop in C++ a poor one. Python (specifically some of its libraries) and the Jupyter lab are instead a very nice experience for treating data and, besides, is supported by a vibrant community of developers, even among hydrologists, who really make the experience better and better everyday. Python is also becoming a “lingua franca” among scientists and this makes less difficult and steep to share results. I have a couple of pages to introduce Python to hydrologists that you can access starting from here. For what regards of Java one can start from here.
In any case Java is not mandatory for the use of OMS. Also FORTRAN and C++ codes can be used: NIWA did it with FORTRAN, but we never tried it.

OMS code is distributed with the MIT Open Source license. It is not available on a public repository though, but Olaf David, its chief architect, gave us access to it and does not have problem to give it to others. I recognize that this is a limitation but with more users, it would not be so difficult to get a branch of the source code in a public repository, I guess.
Our own source code is at the GEOframe components Github repository. The software is under GPL3 and the repository is open to anyone willing to collaborate.

The structure of the OMS allows to easily add components with any type of licence, the one more suited to you, and does not oblige you to release the software under the GPLv3 license.
Worth to mention is that OMS has a server side companion, CSIP of which you can find any information here.

Once Olaf David mentioned to me that the overall cost of OMS was closely 10 million of dollars. From my side I guess i invested around 1 million and more on developing GEOframe and its ancestors, Just to give a measure of the investment needed for such an enterprise. But building on top of them is now much less expensive and much more immediate.
Our current task are finishing the 1D-2D-3D solving of Richards equation (extended to treat groundwater and surface waters) and its coupling with the energy budget, evaporation and vegetation dynamics. This would replace our older GEOtop in two years, except for snow modelling for which we do not have started anything so far. We have also ongoing work for hillslope stability and a well established set of tools for doing lumped modelling, including the estimation of travel times and tracers/pollutants concentrations. Some other developments, more on the side of informatics were done by a former students of mine and are reported in his Ph.D. Dissertation. It contains also some work on neural networks.
For the future, an effort should be made to bring OMS to Java 9, which, I think, could make the code more understandable. Classes are a good thing but understanding the connection between them is what matters. Java 9 Modules should help to get the code more clean, and Gradle should do the rest for external libraries.  A few years go I started to dig into the code. I could not complete the task for my limited time but I believe one Java programmer can master OMS3 code in six intense months or so.  I also envision that, having a good project, OMS could be completely embedded in Jupyter lab making it usable through it making its use more familiar to a multitude of users. 

Obviously, we are willing to give any help, if you join us. I hope this information can help you to make your decisions.

3 comments:

  1. What about plans for adding tools for optimally ingesting remote sensing data and ground observations (via data assimilation) and human processes like irrigation? Some systems that do the work already exist in the land surface community like the NASA Land Information system https://lis.gsfc.nasa.gov/ but they are not based on OMS philosophy and changing and modifying the code can be very challenging. In addition they are suited for large scale applications with obvious limitations in small catchments and complex terrain. I think such a tool would be amazing and very powerful... ;) Christian Massari

    ReplyDelete
  2. Thanks for your efforts with porting applications to the OMS3 framework.
    Is there a GEOframe version for GEOtop? Or a component wrapper (Formetta 2013)?

    ReplyDelete
  3. There are 2 answers to this questions:
    1- There is a GEOtop wrapper that include the whole GEOtop as a blob. For this you have to wrote directly to Giuseppe Formetta giuseppe.formetta at unitn.it;
    2- We are slowly redoing GEOtop inside GEOframe starting from the integration and extension of Richards equation. Search the work of Niccolò Tubini for this in this blog or give a look to the geoframe components github site. There you will find Richards 1d alone, Richards 1d plus energy budget; Richards 2d and incoming will be Richards 3d. Integration with ET is an ongoing work with my younger Ph.D. student, Concetta D'Amato, starting from the work of Michele Bottazzi who is going to graduate soon. Hope this helps.

    ReplyDelete