Going Beyond the Present Stato-of-Art in hydrological Modeling. My point of view.

In the recent years there was a bloom of initiatives to support the modelling-by-component paradigms. A relevant European experience is OpenMI (http://www.openmi.org) which CUDAM and Hydrologis endorsed, and applied to a large modelling project, which was expected to produce a modeling system prototype of the river Adige. The system was intended to provide also the informatics for a decision support system to be built eventually. The OpenMI system, a product of the EU project HarmonIT, promised to improve collaboration among scientists, and to embed previous modelers' knowledge in a new framework. The choice of OpenMI came after an accurate analysis of competing frameworks, developed with hydrology and other science in mind. This analysis, which was partially borrowed from Rizzoli et al. [2005] included economic modeling, the agricultural domain (APSIM, STICS, CROPSYST, APES, and in general the results of the ongoing SEAMLESS EU project), the hydrology-water management milieu (TIME, OpenMI, IMT, OMS 2.2, and the Jupiter APIs). Some integrated environments such as those presented by SME, TARSIER, MODCOM, SWAT [for references, please see Argent and Rizzoli, 2004] and Wesselung at al. [1998] were also studied for comparison as well as products coming from the atmospheric sciences (i.e ESMF), and from the platform for high performance computing of the CCA Forum. A recent review of some of them was provided by Jagers [2010]. They were asked to respond to the requests exposed at the second biennial meeting of CUASHI in Boulder 2008 (http://www.slideshare.net/GEOFRAMEcafe/geoframe-a-system-for-doing-hydrology-by- computer).
These questions were essentially:
• Bringing to hydrological modelers the best of modern software technologies in terms of software architecture, design and engineering, code reusability, and collaborative information sharing
• Allowing third party revision of models and codes, thus improving the inspection of science embedded
in modeling. (Clearly in order to achieve this state, we needed to set a new model, where people can submit their code for review. In chain this caused the requirement - not yet fulfilled - to set of guidelines and rules to use as criteria for code quality)
• Creating models that could be used and perused by various users, for a variety of requirements (e.g. Rizzoli et al., 2005)
• Providing a collaboratively produced set of tools for calibration, data assimilation and statistical analysis, which are not by themselves the core physical science we pursue, but are necessary to exploit the possibilities open by the modern measurement systems, to supporting remote sensing and web-distributed resources, and therefore let scientists to concentrate on their science more than on tools. These tools which should however support all of the requirements for executing modern, cutting-edge science.
• Accessing data seamlessly and transparently, providing data marts, independent of the database layer
• Combining modules which work entirely in memory and step through time sequentially, making it possible to estimate non-linear environmental feedback systems.
• Integrating of models with the appropriate tools for gathering and formatting data deriving from heterogeneous sources, whilst promoting the use of standards such as those requested by INSPIRE, S@NY, the Open Geospatial Consortium, W3C and others. To accomplish this, further additions to the design requirements were deemed necessary, which entered the design requirements:
• the source code must be made available under open source license (to enable inspection of the internals of the models and progress of research)
• the platform should be language neutral (connecting code written in Java, C/C++, and FORTRAN) and platform neutral (i.e. working on MS Windows, Linux and MacOS)
• the platform should be built entirely by open source tools (as a desired approach, however not mandatory)
• interfaces to OGC data types should be included
• the platform should be compliant of INSPIRE directives for data formats and protocols
The initial choice for realizing the system was the OpenMI system (http://www.openmi.org, Gregersen et al., 2007), and since a binding language was needed to connect model components, Java was the choice, being open source, multiplatform and able to obtain performances comparable to compiled codes as demonstrated by the Mines Java Toolkit (http://inside.mines.edu/~dhale/jtk/index.html) and COLT (http://acs.lbl.gov/software/colt/) libraries, and by other sources (e.g. Knoll and Mirzaei, 2010). However, after the adoption of OpenMI 1.4, and having made possibly a large implementation of codes out of the group of OpenMI developers (numbering more that sixty model components included in the distribution of the GIS JGrass), CUDAM and Hydrologis decided to compare OpenMI with the OMS3 system (http://www.javaforge.com/project/oms) before switching to OpenMI 2.0 to which hydrologis contributed [e.g. Gijsbers et al., 2010]. Following this comparison, the decision to move to OMS3 was made. There were several reasons for this decision:
• The OpenMI system which has been implemented is not completely component-oriented: it is a model coupling framework and legacy centered.
• OpenMI 2.0 will be an improved platform, but the overhead it introduces in terms of boiler plate code threatens the normal modelist. OMS3 is much simpler and offers a more intuitive and cleaner design. After several years of development, there are just a few models that can be considered OpenMI (and these include CUDAM's newAge model and the Horton Machine components). Most of the models "ported to OpenMI" are just wrapped with an OpenMI interface, and not built upon OpenMI. This only manifests the difficulty of the task. Experiments in teaching modelers to build OpenMi components were frustrating and unsuccessful. A recent study (Lloyd, 2010) shows the quantitative overhead introduced by traditional modeling frameworks for model development, compared to the non-invasive and lightweight design of OMS3 models.
• OMS 3 adopts the meta data annotations for describing and managing the components [David et al, 2010a,b], which make any port, from any platform, much more simpler. CUDAM and Hydrologis had more that 60 components written in OpenMI 1.4 and bringing them to OpenMi 2.0 turned out to be a major undertaking. OpenMi offers a xml based component description, which is not as effective as the annotations in OMS3. While in principle OpenMI can adopt this strategy for component description, OMS3 is already has it implemented.
• OMS3 is considered lightweight and non -invasive [Lloyd et al., 2009]. The version of Hydrologis and CUDAM's NewAGE model contains around one third of the code of its OpenMi 1.4 counterpart. A simple OMS3 component contains little meta data overhead complementing the core computational code.
• OMS3 offers a larger academic base of modelers than OpenMI [e.g. Ascough et al., 2010]. CoLab [https://colab.sc.egov.usda.gov/], could potentially offer a rare pool of existing model components for reuse within other modular models. As a result it incorporates methods such as SCE, LUCA, FAST, DDS, and other approaches reflecting its academic origin. In addition OMS3 also reflects operational needs by providing ensemble simulation support, such as ESP.
• While OpenMI is mostly a component linking framework, it does not offer features such as simulation management for calibration, parameter estimation, uncertainty, and sensitivity analysis. They are not within its scope.
• OMS3 automatically supports multithreading and Amazon's elastic cloud computing environment [David et al., 2010c]. OpenMI 2.0 does not support parallelization and multi-threading. OMS3 also supports some of the more standard calibration methods required by environmental modelers.

• OpenMI 1.4 supports a Java execution system and console through JGrass. Although this is a great system, it needs to be upgraded to support OpenMI 2.0. This work will likely only be carried out by Hydrologis and CUDAM. A small but solid community of Java developers surrounds OMS3, which whom we can talk and cooperate. OMS3 currently offers a Java execution system and console and this community will eventually integrate OMS3 with the J-Grass console.
• OMS3 is a program which will exist for many years, being supported by a long term financial support from USDA. OpenMi promises to be the same, but is only financially supported by its participants.
• OMS3 plans to be compatible with CSDMS, in the sense that modules that will work with OMS3 will be easily scaled up to CSDMS, and supercomputers. Related to this effort, a OpenMI 2.0 wrapper generator was developed as a part of the OMS3 toolchain, to offer the use of any OMS3 component/model as an OpenMI model.
• OMS3 already provides tools for visualizing time series and doing simple statistical analysis. All major statistical checks analyzing models performance are built in (e.g Nash Suttcliffe, TRMSE, and many others).
Many of the tools formerly present in OpenMI have now been ported to OMS3 (see http://www.jgrasstools.org). OMS3 is provides a comprehensive library of process components originating in hydrology and agricultural sciences. Component based model implementations are currently being adopted for water supply forecasting in the western US (national Water and Climate Center Portland Oregon), for conservation effects assessment (USDA) using fully distributed watershed models, and for USDA-NRCS erosion modeling. Ongoing efforts at USDA focus on the integration of OMS3 into SOA and Cloud environments allowing a cost effective and scalable use of OMS3 based simulation models. While the software architecture behind the APES simulator (http://www.apesimulator.it/) promises to be as lightweight as OMS3, its software still remains undisclosed to modelers. Despite the declaration to be "framework independent", in practice, APES requires a third party engine to be executed and its generality is more theoretical than practical.

References

- Argent, R.M., Rizzoli, A.E., 2004. Development of multi-framework model components. In: Pahl-Wostl, C., Schmidt, S., Rizzoli, A.E., Jakeman, A.J. (Eds.), Transactions of the Second Biennial Meeting of the International Environmental Modelling and Software Society, Osnabru ? ck, Germany, vol. 1, pp. 365e370.

-Ascough II J.C. et al. Integrated Agricultural System Modeling Using OMS 3: Component Driven Stream Flow and Nutrient Dynamics Simulations. iEMSs 2010 International Congress (2010) pp. 1-9

-David O. et al. Object Modeling System v3.0, Developer and User handbook. Manual (2010) pp. 1-111

-David O. et al. Rethinking Modeling Framework Design: Object Modeling System 3.0. iEMSs 2010 International Congress (2010) pp. 1-9

-David O. et al. Cloud Services Innovation Platform (CSIP). Internal Draft (2010) pp. 1-17

-Donatelli et al., 2009 M. Donatelli, G. Russell, A.E. Rizzoli, M. Acutis, M. Adam, I. Athanasiadis, M. Balderacchi, L. Bechini, H. Belhouchette, G. Bellocchi, J.E. Bergez, M. Botta, E. Braudeau, S. Bregaglio, L. Carlini, E. Casellas, F. Celette, E. Ceotto, M.E. Charron-Moirez, R. Confalonieri, M. Corbeels, L. Criscuolo, P. Cruz, A.D. Guardo, D. Ditto, C. Dupraz, M. Duru, D. Fiorani, A. Gentile, F. Ewert, C. Gary, E. Habyarimana, C. Jouany, K. Kansou, M.J.R. Knapen, G.L. Filippi, P. Leffelaar, L. Manici, G. Martin, P. Martin, E.C. Meuter, N. Mugueta, R. Mulia, M.V. Noordwijk, R. Oomen, A. Rosenmund, V. Rossi, F. Salinari, A. Serrano, A. Sorce, G. Vincent, J.P. Theau, O. Therond, M. Trevisan, P. Trevisiol, F.K. Van Evert, D. Wallach, J. Wery and A. Zerourou, APES: The Agricultural Production and Externalities Simulator. In: F. Brouwer and M.K. Van Ittersum, Editors, Environmental and agricultural modelling: integrated approaches for policy impact assessment, Springer Academic Publishing (2009).

-Gijsbers, P., S. Hummel, S. Vanec ?ek, J. Groos, A. Harper, R. Knapen, J. Gregersen, P. Schade,
A. Antonello, and G. Donchyts. From OpenMI 1.4 to 2.0. In Proceedings iEMSs 2010, 2010.
Gregersen, J. B., P. J. A. Gijsbers, and S. J. P. Westen (2007), OpenMI: Open modelling interface, J. Hydroinf., 9(3), 175–191, doi:10.2166/ hydro.2007.023.
Jager, B.. Linking Data, Models and Tools: An Overview. iEMSs 2010 International Congress (2010) pp. 1-8

-Knoll and Mirzaei. Scientific computing with Java. Comput. Appl. Eng. Educ. (2010) vol. 18 (3) pp. 495-501

-Krause, P. Das hydrologische Modellsystem J2000, Schrft. FZ-Jülich, Bd. 29, 2001

-Krause, P., Kralisch, S., Flügel, W.A., 2005. Preface: model integration and devel- opment of modular modelling systems. Advances in Geosciences 4.

-Lloyd, W., O. David, J. C. Ascough II, K. W. Rojas, J. R. Carlson, G. H. Leavesley, P. Krause, T. R. Green, and L. R. Ahuja. An exploratory investigation on the invasiveness of environmen- tal modeling frameworks. InAnderssen, R. S., R. D. Braddock, and L. T. H. Newham, editors, 18th World IMACS Congress and MODSIM09 International Congress on Modelling and Simu- lation, pages 909–915. Modelling and Simulation Society of Australia and New Zealand and International Association for Mathematics and Computers in Simulation, July 2009.

-Nativi, S., Caron J., Davis, E., Domenico, B, Design and implementation of netCDF markup language (NcML) and its GML-based extension (NcML-GML) , Computer and Geosciences, (2005), 31(9), 1104-118

-Rizzoli, A.E., Svensson, M.G.E., Rowe, E.C., Donatelli, M., Muetzelfeldt, R., van der Wal, T., van Evert, F.K., Villa, F.:Modelling Framework (SeamFrame) requirements. SEAMLESS report no. 6, Dec 2005

-Wesselung, C. G., Karessenberg, D.-J., Burrough, P. A. and Van Deursen, W. P. A. (1996), Integrating dynamic environmental models in GIS: The development of a Dynamic Modelling language. Transactions in GIS, 1: 40–48. doi: 10.1111/j.1467-9671.1996.tb00032.x

Athen Hydrological Observatory

The Observatory, aims at freely releasing hydro-meteorological data (incl. rainfall, temperature, relative humidity, evaporation, air pressure, solar radiation, sunshine duration, wind direction and velocity) and hydrometric data, from the monitoring network of NTUA’s Laboratory of Hydrology and Water Resources, which is operational since 2005. The network covers the Greater Athens Area but also includes standalone stations in selected locations across Greece, in partnership with local municipalities and research centres.

MORFEO project meeting

The Morfeo project aimed at exploiting the remote sensing techniques for revealing various types of landslides, and built some almost operational tools for doing it. We had the closure of the project last tuesday and wednesday in Rome. We, together with the group of University of Bologna, lead by Enzo Farabegoli, worked on three Alpine catchments in three years of intense field work (Farabegoli), and computer work (us). Farabegoli provides very detailed field data, that could subsequently be used for driving GEOtop simulations. Remote sensing and field survey (see picture below, derived from GEOEye at 50 cm of resolution) were used ti derive, for instance, land use.

Trento's work (especially Mountain-eering) was to use all the information available to get a probabilistic forecasting of hillslope stability based on the methods described in Simoni et al., 2008.

Besides, we produce a working infrastructure around GEOtop to store environmental data (in Postgre-PostGIS plus Ramadda), visualizing them and use them for simulations, finally stored back in the database. A synthetic overview of the results can be found in the
very short overview I did in Rome.

The project is finished but actually some years will be necessary to properly use all the data collected.

Channel Initiation

The contribution of few days ago Where do channels begin ? looks to channels' forming processes. The scientific debate has another side which regards channels network extraction.

A recent paper by Orlandini et al. (2011), whose results where also presented at CATHY meeting last January, presents a concise but nice introduction to the topic, and a few results based on works made in the Dolomites. A parallel introduction can be found in my slides.

Certainly Orlandini and Moretti have the merit to have produced the most cleaned methods to produce channels delineation (all papers cited below), and in parallel, proper methods for estimating the upslope contributing areas of any point in a catchment. This is an essential ingredient of the whole story of proper watershed (and drainage directions) delineation and we implemented most of their work (not all of it unfortunately) in JGrass. The rest is kind of an epiphenomenon.

Reference

Orlandini, S., and G. Moretti (2009a), Determination of surface flow paths from gridded elevation data, Water Resour. Res., 45, W03417, doi:10.1029/2008WR007099.

Orlandini, S., and G. Moretti (2009b), Comment on ‘‘Global search algo- rithm for nondispersive flow path extraction’’ by Kyungrock Paik, J. Geophys. Res., 114, F04004, doi:10.1029/2008JF001193.

Orlandini, S., Tarolli, P., Moretti, G., Dalla Fontana, G. (2011). On the prediction of channel heads in a complex alpine terrain using gridded elevation data, Water Resources Research, 47, W02538, doi:10.1029/2010WR009648