Wednesday, September 17, 2014

GEOtop binaries for Mac OS

After a few hacking I was able to transport my compiled version of GEOtop and libraries to a different machine. It is still a rough approximation of what it can be. However, it works. No compiling required as before, just installations:

# Install the following packages (or verify you have them already):

Install,  Homebrew and other required libraries.

#Install needed libraries

brew install boost
brew install gdal
brew install proj

#Install GEOtop components and libraries: this require, at the moment, a little of hand working

Download the executable zipped file from here.

Once unzipped the file look at the directories created and place the file you find in the /Libraries and /usr directories as in the unzipped directory hierarchy.

As you can see, I chose to create a "/usr/local/GEOtop" directory where all the real executable are (has to be places). This directory is actually not a standard one, However, I placed a symbolic link into /usr/local/bin, which is a directory usually already present in the Terminal path.

#Download the examples

You can find them here. Unzip them  in your disrectory, for instance ~/GEOtop.

#Try geotop examples

cd to the directory (folder) where the tests are), for instance: ~/GEOtop/tests/test_sample_run/small_example

Issue:

$ geotop ./

If you did it well and the goddess of programmers assisted you, it should work, as it did for me and in Tim's machine.

Monday, September 15, 2014

Opinions on the GEOtop roadmap

Just opinions indeed. For who landed here occasionally, information on GEOtop can be found here.

1 - Infrastructure

Mountain-eering and Exact-lab companies are doing a visible and positive effort in having a clean C++ code, after the huge work made by Stefano Endrizzi and Stephan Gruber to arrive to GEOtop 2.0. However, I believe the work on C++  should make some further steps, which I delineate in one of my previous posts here.
I deem necessary to do at least the first step described there, but the others would be also necessary in order that many people can work collaboratively on the project, and to maintain what we have while producing alternatives, and progress. My solution is, obviously just a proposal, and others (that I do not know) could exist. I plan to work myself part time on this from now to February, but if I can have help from someone this will be great. While changing the basic structure of the model's main( ), if  I will be really able to do it in this favourable astral conjunction,  I will take notes, and provide information to everybody. 

2 - Algorithms

On the side of the algorithms I believe we can work in several directions.  Using an unstructured grid will be a necessary intermediate step to pass through. 
Obviously this would imply a subsequent complete rewrite of the code, and in this rewriting we could take the occasion to: 1 - solve Richards equation with Casulli’s method, and 2 - better integrate surface waters with it. If resources would be available, even freezing soil could be rewritten, and the successful 2011 paper already contains how to do it.

3 - Processes

Among the processes, it would be important if the work by Stefano Dellachiesa on vegetation, and if the work of Florian Marshall on contaminant transport could be really embedded in the GEOtop main stream. My claim for more OO structure above, will go also in the direction to include these achievements, without cluttering the code and/or making more complicate the IO.
Personally, and in perspective, I am interested to include in GEOtop a modeling of the low atmosphere, integrating Navier-Stokes equation. I did some preliminary approach to the problem working conjointly with Dino Zardi and Michael Dumbser. A lot of resources are needed but I can work to grab them.

4 - OMS - Web services

My personal academic interest is also to push further  the evolution of GEOtop’s (beyond step 1) informatics for embedding GEOtop in OMS. This would allow splitting GEOtop  in components that can be joined at run-time through a scripting language, i.e. Groovy. This would enormously  increase the ability to run different modelling solutions, facilitate simulations reports, incapsulate even more the code. I am not claiming here, or pursuing, that this must be the main road followed by the GEOtop community. A certain “genetic variability” in the GEOtop versions and ecology would be indeed of benefit, and in favour of the model eventual survival. A passage necessary for OMS integration will be that task 1above will be pursued. That I think, should be really accomplished right away.

5 - Commitment and Executables

I already manifested my opinion to the developer group. Which is the following: in this moment using GEOtop require the total commitment of the guy/gal the want to embrace it. This is not acceptable if we want that our product would have some spreading. In this moment compiling GEOtop, still remains a challenge for a new users, and even for expert users, because they usually have a lot of other things to do, and they are pissed out when something that they consider trivial does not work as they expect. 
Downloading the source code directly from trunk should work for developers, but not for users. A tagged version, known to compile under the three major operating systems, should be used instead. More than that, a tarred zipped file, with the source code should be made available without passing through git. The best experience for a user would be, however, to download directly the executable for her/his machine and run it. I think that this move, together with improving the manual and documentation, will really boost GEOtop use. And, IMHO is the presence of a lot users that create a fertile soil for businesses.  Certainly a different pathway would be to implement GEOtop as a remote web-service. Recently at CSU,  I was impressed on what te eRams platform can offer, and how it helps in getting data and simulations together, and make easier collaborative work of groups.

6 -Testing

GEOtop comes with a series of tests, and, I verified, all of them compile, and all except one converge to the right solutions. This is useful for making users confident of what they are doing, and developers certain that they did not screwed up the system with their last minute modifications.  However, a special attention should be given to compare GEOtop with the tests provided by the increasing community of process-based model developers. They call their virtual experiment “the superslab” and you can find its definition here [link to be set up].  Being engaged with those guys would be really important, and would bring furhter recognition the our model. We could possibly be able also to challenge that community and propose tests that cope with the cryosphere, since none of its models does it. 

7 - Manual

The manual is not out-of -date but not up-to-date too. Efforts to get it improved would be necessary. I will try to put some students on it, if the occasion comes.

8 -License and Community

GEOtop is GPL and I believe it has to remain GPL. However, not in name of a single Author, but most probably in name of many. The creation of a GEOtop by components, could help in this direction, in the sense that single components, could be copyrighted then by single persons or group different from the original ones. The establishment of some GEOtop foundation with some organisation, and which determines some rules of behavior and/or an etiquette to follow for GEOtop use, will be a desired goal for future. I consider myself not eligible for any position in it though. 

9 - Courses

If a few of the conditions above would be met, it would not be difficult to start some schools either in Europe and in the Americas (or elsewhere). This also should be a goal to be pursued. Schools for Ph.D. students and Researchers, could not be very expensive, however some income can be generated.  Courses for professionals should be more expensive ... but maybe these could be directed to use web-services, and therefore the real income could arrive from subsequent use. 


10 - And then ?


Then it is matter to find resources and man/months to have all of it working. 

Project CLIMAWARE

Today we submitted the CLIMAWARE project: CLIMatic change impacts on future Availability of WAter REsources and hydro- geological risks.
It is a proposal internal to UNITN but it involves so many different disciplines and Departments that represent a variety that normally can be found only in EU projects. Please below, find the abstract of the project, while we cross the fingers for its approval.
                                       


The pressure of human activities on the earth system, including the water cycle, is multifaceted and intrinsically interlinked with climate change due to the inherent complexity of natural and human processes. The interaction of humans with the environment is strongly non-linear and is dominated by multiple feedbacks and non-stationarities, which are difficult to identify and assess. Despite these complexities, the existing tools to deal with the many hazards connected to the interaction of humans with the environment are based on simple approaches, founded on the concept that the earth system is in a steady state, so that predictions on its evolution can be safely made either statistically, or through simplified models founded on the hypothesis the processes are stationary.
This proposal focuses on the interactions between climate change and human activities related to water, with a holistic view embracing physical, social and economic processes. It will consider, in particular, changes in water cycle components related to extremes and their implications in contiguous sectors. Here extremes include floods and land instabilities triggered by extreme precipitations, such as debris-flows and snow avalanches, but also different stress factors threatening the integrity of freshwater services, with adverse effects on agriculture, tourism, and energy production. New paradigms, approaches and tools will be developed in order to cope with the non-stationarity of water cycle processes and to study the entanglement between the physical processes and human activities. The proposal is interdisciplinary also regarding the physical aspects, and involves interaction between water related disciplines, such as climate science, meteorology, hydrology, fluid mechanics, and information technology for aspects related to distributed monitoring. Because of the challenges that climate change imposes, there is a need to increase general awareness of climate-related risks, and find ways to mitigate the impact on economic activities as well as to adapt to new scenarios. For mitigation and adaptation strategies to be effective, knowledge of the physical aspects of the water cycle should be combined and complemented with knowledge of social and economic processes that crucially affect the use and management of freshwater. Furthermore, it should consider that national governments, on the one hand, and the European Union, on the other, have started to enact laws aimed at countering the effects of climate change. The implementation of these measures and their harmonization with existing laws, however, is not without problems. This might require some legal creativity – and the interaction of law with other hard- and social sciences – to bring about effective results.

Project Description, highlighting the interdisciplinarity (excerpt)

Climate change mitigation and adaptation require substantial changes in production and consumption patterns, as well as in individual behaviors. This is particularly true for water- related sectors, which are a linchpin in the production of beverages, food and energy, and for many recreational activities. However, in economics, estimates of water resource availability are rarely made on a solid quantitative basis. Also, the systematic preemptive estimation of extremes caused by hydrological events is a science in its infancy. The recent advances in hydrology and water related sciences open interesting perspective for new methods and models to support a better informed decision making process at all levels, from local to national and international. Moreover, societal and economic concerns receive only marginal attention in the debate [1]. Arguably, the gap between science, society and policymakers undermines the decision-making process.

The interdisciplinary approach of this project, which combines insights from hard sciences with sociology, economics and law, represents an important step towards bridging the gap between science and society, and raising awareness of the benefits of cooperation between disciplines. The project studies the complex interplay between physical and human processes in controlling the distribution, in space and time, of water resources by considering both actual and possible future climatic and societal scenarios. The project is organized in five interconnected Tasks (Figure 1): Task 1 evaluates the drivers of water resource availability; Task 2 assesses climate change and human impacts on water resources and local hydro-geological extremes (underestimated according to a report by WMO [2]); Task 3 addresses the development of suitable tools for describing and modeling environmental granular flow extremes and sediment transport; Tasks 4 and 5 aim for better management of water resources, specifically, Task 4 uses the paradigm of virtual water to assess the regional impact of the food trade, while Task 5 tries to envisage effective societal responses to climate change.

References

1. Malnes, R.; Environmental Politics 17(4), 660-672, 2008.
2. http://www.wmo.int/pages/prog/drr/transfer/2014.06.12WMO1123_Atlas_120614.pdf

Wednesday, September 10, 2014

My CV and Five Papers that represent me

For who it may concern, I am posting my CV here. It collects what I did, obviously, and contains information already present here (there, in addition, you have a link to the papers).


Being in an introspective mood, if I would asked to choose five papers that more represent my work (and me), they wuld be not the most cited (among them), not those in Journals with the highest impact (well ... among them), and I would select probably:
Obviously I did decent work also between 1992 and 2006. Enjoy!

Monday, September 8, 2014

Improve GEOtop informatics !

I take the occasion of the actual structure of GEOtop main to argue about best practices in programming and introduce the topic of design patterns. For browsing the actual code you can see here , while my more specific comments are available at the GEOtop Developers forum

I argued that a much simpler  form for the main, should be represented by a scheme like the following (dots represent missing code, the code is assumed to be C++ - please be merciful with me: my knowledge of C++ details is very limited -  but, the arguments are for every generic language with support of object orientation):

where three main phases are distinguished: an initialization phase (with allocation and initialisation of variables), an execution phase (where the real stuff is done), and a closure phase (where all the memory if freed, and whatever is needed is performed). This best practice is a suggestion that comes from many recent numerical modeling efforts, like ESMF, OpenMI, OMS and allows an encapsulation of the processes in their own part of the project, but everybody, I think, can appreciate its simplicity and cleanness. 
A structure like this should be not very difficult to obtain in a few days of work of a proficient programmer. This structure does not alter the internal of the code and rationalise what is present now, introduce some encapsulation of the procedure,  but IT IS NOT object oriented (OO) programming.
Object oriented programming has to do with the choices of appropriate classes, and instantiation of the appropriate objects, and doing OO design, for implementing the code.

Who is suited to procedural programming, for instance interprets algorithms as “procedure calls”. My view, and understanding of the matter, is instead that procedures should be instantiation of classes, a.k.a. “objects”. Therefore, a proper evolution of the above could be:

In this second case, the organisation of the code remains the same. However the execution of the single procedure is written differently, i.e., for instance:

EnergyBalance.execute( ); 

This is intended to mean that EnergyBalance is an object and .execute() (please notice without arguments) is a method called on this object. One of the advantage of making the each procedure an object, has several argument in its favour. The first is that each object, now, contains its own inputs as fields, and encapsulation is certainly improved. This makes all the code easier to maintain, and the input data for each of the procedures easy to control and parse (out of the time loop, obviously). For a simpler, but a conceptually similar, example in Java, please see here.
Doing this step is not matter of a day, it certainly require more thinking and a sequence of modifications also in the structure of the data containers actually present in GEOtop which propagates down into the inner routines.

In case, also the print( ) method is part of a class, with the rational that with proper software engineering, it can be sent in parallel with the core calculation, on different processors, and therefore, interfering the less possibile with the ‘core’ computational time. I personally bet that, at moment, a consistent part of computational time of GEOtop is wasted in printing to screen data. So, maybe we spent months to have a fast algorithm for finding the solution, and that we use that time saved to put thousands of number on the screen or to write them to disk subtracting power to the computation. 

Actually, the one above is just the beginning of the story. Opening the way to classes, open the way to use some well known  Design Patterns to make the code more maintainable, flexible and evolvable. One case, obvious to all GEOtop aficionados is that, in reality, either the water budget and the other procedures, are a bundle of different alternatives. With OO there exists at least one neat scheme to treat this case, which is called, the “Strategy pattern” where, the object really chosen is one out of a group of options which are implementation of a common interface  (I am going to use the technical OO slang here). So actually, the "main( )" can be programmed to accept a generic type of solution (algorithm) to that problem (as specified by the abstract interface), and eventually this solution can be chosen at run time among the subclasses that implement it.  Solutions can then be added to the code without the need to modify (almost) anything else of the code than just adding a new subclass.

References (you will not find them easy to read ... but take it quietly, cross-check with the web, and something will pass slowly). Besides the basic, also adressed here fro Java, try to give a look to:

Gamma, E; Helm, R.; Johnson, R; Vlissides, J; Design Patterns: Elements of Reusable Object Oriented Software, Addison-Wesley, 1995

Martin Fowler's, UML Distilled: A Brief Guide to the Standard Object Modeling Language, Addison Wesley, 1999

Freeman, Eric T; Elisabeth Robson, Bert Bates, Kathy Sierra (2004). Head First Design Patterns. O'Reilly Media. ISBN 0-596-00712-4

Bruce Eckel, Thinking in patterns', notes. 

Various Authors, Design Pattern,  from Indian Student Association of University of Nebraska at Omaha
 

Friday, September 5, 2014

Installing GEOtop 2.0 on Mac OS X

I receive from Matteo Dall'Amico and I publish the following instruction to compile GEOtop 2.0 on Mac OS X. For Linux see this other post.

# Install the following packages (or verify you have them already):

#Install and compile MeteoIO libraries.

This will include to install the Developers tools,  Homebrew and oter required libraries. See instructions here

#Check if you have git, otherwise install
brew install git

#Install libraries

brew install pkg-config
brew install boost

# download GEOtop
git clone https://code.google.com/p/geotop/

# create the makefile

If you do not have installed before, from CMake website. When you start it a window as the one below appears:
Please, observe that, on top the path to source code has been specified, and just below, the path to binaries. click the Configure button to configure the Cmake. If not already present, add the path to MeteoIO libraires: /usr/local. After configuration, the screen should appear as below:

To process the file, Press Generate. This will generate, in the directory (folder) selected for binaries, the Makefile

# make the project

In terminal cd to the directories were binaries will be created anc check if the Makefile is present. Issue the command:

$ make

# copy either using Finder Window or Terminal commands the binaries files generated to a standard place.
I chose "/usr/local/GEOtop".
This directory is actually not a standard one, However, I placed a symbolic link into /usr/local/bin, which is a directory usually already present in the Terminal path.

$cd /usr/local/bin
$ln -s ../GEOtop/geotop geotop

#try geotop example. Due to previous operation

cd to the directory (folder) tests/test_sample_run/small_example (for instance this, among the others)
Issue:


$ geotop ./

If you did it well and the goddess of programmers assisted you, it should work, as it did for me.

Installing meteoIO on Mac OS

Matteo Dall'Amico (see previous post on GEOtop installation) also provided instructions for compiling MeteoIO on Mac and Linux). While Mac instructions can be found in this post, Linux instructions can be seen in a previous post

# Check the environment compiler: You are suppose to have installed the appropriate Developers tools (search for them in the App store). Be sure that you also download the command line tools.

#For Mac OS use Homebrew  (which needs to be installed first)
#Then write at the console, for instance for Make:
brew install make

#For any other package:
brew install subversion
brew install gdal
brew install proj
brew install doxygen

#install CMake from CMake website

# After having selected your directory for installing meteoIO libraries with the usual console commands
# download the source code from the SVN:

svn co https://models.slf.ch/svn/meteoio/trunk
#Start Cmake and a window like the one below will appear:


# Please observe that you have to indicate the directory (folder) where the source code is
# and the directory where you want the binaries. Is in this directory that the Makefile will be produced
#Click on Configure and a window like the following should appear:

#Please note that you can modify the flags just clicking on them. Be sure that you have also checked the proj.  The following fields should be "ON"


BUILD_SHARED_LIBS
PLUGIN_A3DIO
PLUGIN_ARCIO
PLUGING_ARPSIO
PLUGIN_GEOTOPIO
PLUGIN_GRASSIO
PLUGIN_PGMIO
PLUGIN_SMETIO
PLUGIN_SNIO
PROJ4 (then you have to verify the paths to LIBPROJ and to LIBPROJ_INCLUDE_DIR)

All the rest can be set to "OFF".

# Then, press  the butto to generate the makefile and exit from the console.

# In the terminal, goes in the directory where you create the Makefile and type:

make install

# (with "sudo" if you want to install it as super user).

Now move the libraries into /usr/local/lib.  They shoud be libmeteoio.2.4.3.dylib and the two symbolic links: libmeteoio.2.dylib and libmeteoio.dylib

After having installed meteoIO, in the needs to understand what it does, one can browse the examples in the "doc" directory. However, s/he can also built the doxygen documentation. For doing it, please first check you have doxygen installed. Subsequently:
  • using a terminal move to the directory where the meteoIo source code is located. 
  • execute "doxygen -g <configfile>" to create a configuration file, "configfile" for the project
  • execute doxygen <configfile> to create the documentation
  • Find the documentation in a new html directory created
  • Access it by clicking on the index.html file