A second post on the JGrasstools in row

Please give a look to the announcement given by Andrea Antonello. There are at least two thing to notice: The Java wrapper to GRASS, and the presentation by Andrea which reveals the beauty of the concept of OMSBox. Please follow the link to read.

The first official release of the JGrasstools is out !

Searching for contributors, at the jgrasstools site thanks to the continuous and dedicated work of Andrea Antonello and Silvia Franceschi, the "Hydrologis".

Personally, I will confirm my commitment in increasing the science of their content, and to enlarge their scope. It's a major step forward the next JGrass, and to a new generation of GISs.

Programming and Doing Matrix Algebra in Java

Java is a great programming language. I really appreciate the enormous efforts that were made in making programmer life better, and the elegance of the Object Oriented programming, as compared to the procedural way of thinking I was used to.
Java is special for that being incredibly large the number of things that should be managed for its professional use. A possible workflow for learning it, is presented here (pay attentions to looping links).

At the very end  most of the core work I have to do, is to do linear algebra. Recently the Mathematical Libraries for Java grew many and interesting.   Information can be found in Wikipedia at the List of Numerical Libraries (but most of the links do not work, you have to google the names).

.
Among the libraries cited there, recently we used the Parallel Colt in the work by Francesco Serafin. The merit of the page Parallel Colt was also to bring my attention on the theory of what I was looking for, and especially to the CSparse book by Tim Davis, and to the Golub work, and book.

A comprehensive overview of Java Matrix Packages is given at the Java-Matrix.org.
A comparison of Matrix packages performance was made here but a test with our tools will be necessary, sooner or later.  UJMP is the only one of all the packages that also has a companion paper that tries to describe the design of the library. For me a great add-value.  Also la4j seems promising. Both la4j and UJMP seems to be currently updated, while other efforts are more or less quiescent (including Parallel Colt).

I also observe that a possible use of new features of Java 8 could be of some interest for matrix multiplication. However the most informed answers suggested to cope with ready-made work.

Before doing anything, remeber that for many problem Apache Commons, as well as the more specialised Math Commons and the Google Guava library already have a solution.

A lot to learn indeed.

OpenFOAM ... new basic libraries for GEOtop ?

Matteo Dall'Amico, of Mountain-eering brought to my attention the OpenFoam community at:

http://www.openfoam.com/

OpenFoam is a set of models and libraries for simulations of fluid dynamics problems. From their own site:

"The OpenFOAM® (Open Field Operation and Manipulation) CFD Toolbox is a free, open source CFD software package produced by a commercial company, OpenCFD Ltd. It has a large user base across most areas of engineering and science, from both commercial and academic organisations. OpenFOAM has an extensive range of features to solve anything from complex fluid flows involving chemical reactions, turbulence and heat transfer, to solid dynamics and electromagnetics.
The core technology of OpenFOAM is a flexible set of efficient C++ modules. These are used to build a wealth of: solvers, to simulate specific problems in engineering mechanics; utilities, to perform pre- and post-processing tasks ranging from simple data manipulations to visualisation and mesh processing; libraries, to create toolboxes that are accessible to the solvers/utilities, such as libraries of physical models.
OpenFOAM is supplied with numerous pre-configured solvers, utilities and libraries and so can be used like any typical simulation package. However, it is open, not only in terms of source code, but also in its structure and hierarchical design, so that its solvers, utilities and libraries are fully extensible.
OpenFOAM uses finite volume numerics to solve systems of partial differential equations ascribed on any 3D unstructured mesh of polyhedral cells. The fluid flow solvers are developed within a robust, implicit, pressure-velocity, iterative solution framework, although alternative techniques are applied to other continuum mechanics solvers. Domain decomposition parallelism is fundamental to the design of OpenFOAM and integrated at a low level so that solvers can generally be developed without the need for any ’parallel-specific’ coding.
This section of the website describes some of the features of OpenFoam. The information is a little out of date, but more information will be added during 2010 to catalogue the full list of features of OpenFOAM. "

What could be important for us could be, not just the solvers, since we do solvers, but the tools and the rules. Soon in fact we will be facing to move our GEOtop C-based code to some object oriented language (possibly in a mmix of Java and C++). Object oriented programming has a lot of advantages with respect to procedural programming with respect to many topics, which can help us to do better programming and work more easily in cooperation. Since a full-Java solution could not be as efficient as we desire, using C++ could be the choice.

The lesson we learned from the past work is that large modelling efforts without a community supporting the various needs, is an overwhelming task. JGrass experience viceversa, taught us that one can learn a lot from a community of developers, even if they have slightly different scopes from ours, and even their presence help us.

Thus, OpenFoam, having a community, and having solved many of the problems we have with our own code, is a good candidate for substituting the old beloved Fluidturtle Libraries that I coded fifteen years ago

Deltares' Products become GPL !!!

From Edward Melger and Bert Jagers:

"If you're interested 2D/3D modeling of rivers, lakes, estuaries, coastal
and marine areas, join the Delft3D open source community at

http://oss.delft3d.nl

Here we facilitate the Delft3D open source community to share knowledge,
brainstorm on new features and build working relationships. The source
code of the Delft3D modules FLOW + MOR + WAVE is available as free
software under GNU General Public License (GPL).

By going open source Deltares strives to enhance collaboration, combine
the unique expertise of researchers worldwide and further expand the
Delft3D modelling suite.

Deltares remains committed to supplying high quality services to
consultancy firms, governmental organizations, universities and research
institutes worldwide. See also Services on the webportal.

If you have any questions, please do not hesitate to contact us.

Best regards,


Edward Melger and Bert Jagers
Deltares
Delft3D open source community website: http://oss.delft3d.nl"

The JGrass-NewAge system for forecasting and managing the hydrological budgets at the basin scale: the models of flow generation, propagation, and aggregation

A few year ago, I felt the necessity to built a less distributed model than GEOtop, but, at the same time, less lumped than my Peakflow model based in the GIUH theory. The model had to follow the new informatics envisioned in the GEOFRAME talk (see one of my first post for reference, and the post on adopting OMS3). The occasion was some financial support coming from the Adige river basin Authority. That, not only started the JGrass-NewAGE model, but also the migration of JGrass to the Eclipse Rich Client Platform, the implementation of a Postgres/Postgis database suited to contain a digital watershed model.

The first implementation of the model was based on the OpenMI, but as explained a couple of posts ago, we migrated to the OMS3 platform, and the second implementation of the model can be now found here.

The paper I am introducing, talk about the rainfallrunoff core of JGrass-NewAGE, and presents a discussion of its predictive capacity. The model focuses on the hydrological balance of medium scale to large scale basins, and considers statistics of the processes at the hillslope scale. The whole modeling system consists of six main parts: (i) estimation of energy balance; (ii) estimation of evapotranspiration; (iii) snow modelling; (iv) estimation of runoff production; (v) aggregation and propagation of flows in channel, and (vi) description of intakes, out-takes, and reservoirs. This paper details the processes, of runoff production, and aggregation/propagation of flows on a river network. The system is based on a hillslope-link geometrical partition of the landscape, so the basic unit, where the budget is evaluated, consists of hillslopes that drain into a single associated link rather than cells or pixels. To this conceptual partition corresponds an implementation of informatics that uses vectorial features for channels, and raster data for hillslopes. Runoff production at each channel link is estimated through a combination of the Duffy (1996) model and a GIUH model for estimating residence times in hillslope. Routing in channels uses equations integrated for any channels' link, and produces discharges at any link end, for any link in the river network. The model has been tested against measured discharges according to some indexes of goodness of fit such as RMSE and Nash Sutcliffe. The characteristic ability to reproduce discharge in any point of the river network is used to infer some statistics, and notably, the scaling properties of the modeled discharge.

The full paper is available at the GMMD site. Any comment from you is welcomed.

Why did you not choose a gauged basin ?

This is one question that very often reviewers write in commenting my papers, that most of the time, have a conceptual, if not theoretical attitude, on topics where field guys often dominated the scene in the past years. I am sorry: that I am, inclined to be theoretical.

The question is indeed a good question, but the answer is not trivial. Its contrary is: why you experimentalist do not use sound theoretical work to support your measurements ?

As a matter of facts, if our science pretends to be a physical science, experiments are simply necessary and fundamental. Even if, as I wrote in the past, often we "observe events" more than "designing sound controlled experiments" as Galilei would have required (Simply because this is not possible in relevant cases of our science).

However, let's assume I have finally done an experiment (and I did some in my life): what the reviewers would ask me ?

She would ask about the setting of the experiment. He would ask about the calibration of the instruments, and which instruments were used. They would require decent statistical inquires about the results, performing test of consistencies about them. But many times, the simple report of the measurement efforts (especially if considered massive, and difficult) is would be considered valuable enough to have a paper published.

There is clearly no "par condicio" (equal conditions) in this attitude. Reviewers will not request to an experimentalist other than her work is consistent in itself, and, obviously, that it will bring new evidences of confirmations of something in a matter that must be of interest (of course!).
Inverting the roles, they are not required to produce a sound physical theory of their findings. Even if they should, at least, give a look to the work of the more gifted modelers to support quantitatively their statements, and not allowed to silly built on qualitative (in the sense of poor quantitative) and subjective arguments, or on poor mathematics.

For instance, I am really tired to see in field works, at the edge of geomorphology and hydrology, experiments where data are interpreted with homogenous soil characteristics, with very roughly approximated hydraulic conductivities, and, when real measurements are performed, without any trial to assess error bounds, with unspecified instruments' calibration (even when it is known that they have highly non linear responses), and interpreted with art but on the premises od fundamentally flawed models. (Because, hidden or not, written in words or formulas, any interpretation is a model).

On the other side, when with collaborators, I could use data from highly advertised field experiments, I could often touch the indefiniteness of some of their aspects, and, in front of data not able to survive to any systematic analysis of consistence with regards to the delicate aspects we where investigating (quite unknown by definition of our work), I was several times overcome by frustration and disappointment.

However, I am frank, If I had to choose, I believe the current attitude of tolerance on experimental works is correct. Without any tolerance, no paper will be published or written, waiting for the ultimate one where all the things are performed properly, the theory sound, and its explanation crystal clear even to dummy minds. This would definitely block any development of any science, and I prefer the seed of a good idea inside a sea of garbage than no idea. (How many good ideas can you believe to have in life Riccardo, used to say my masters. One, two, maybe three, if you are really good).

But theoretical and conceptual work should be judged with the same attitude.

I think, that after the debate, and a delicate scrutiny, it would be better to let the community decide what is important or not. Otherwise, the most interesting papers could be eliminated from literature, while the most orthodox ones (and "false": modern Ptolemaic models) naturally proliferate and constitute a overwhelming bunch of "not even wrong" contributions. Therefore I vote for controversial but provocative papers to be published. I believe it is better to be wrong than nothing.

This attitude exposes also to other risks: that some groups of researchers, for a reason or another, and often in good faith, negatively influence sectors of a discipline, killing new ideas (which are usually confuse), and just reinforcing existing paradigms. But this is another story.


P.S. - Another additional, but often found statement, refers to thing that should have been done or tempted to have the paper accepted. Not infrequently these questions are of the type:

Please could you find the sense of life ?

Clearly a few words that can imply the involvement of many full life times of research ... without success, as history teaches. Please, my good old sweet reviewers, give me a break. Why you do so to me ?