Thursday, June 30, 2016

Celerity versus velocity and the travel time problem

A little of introduction

Velocity of a fluid is the variation of its particles' position with time. Celerity is the speed at which a signal (a wave) is transmitted through a medium. Velocity of particles determines flows (fluxes) out of a control volume. Celerity, as we will see, modify the particles' velocity field (intensity and extension).

Jeff McDonnell (GS), and Keith Beven (GS) (J&K) with their nose for interesting topics, recently wrote  a paper where they discuss the role of the two quantities with reference to travel time problems. Some other papers (e.g. Kirchner, 2016,  Hrachoitz et al, 2013) echo the same problem. 
One important point that Beven and McDonnell discuss is that transport of properties (passive tracers) depends on water molecules velocity and therefore between transported properties (like temperature, water isotopes, water age) and flow (assumed to be dominated by celerities) there can be various discrepancies.

Travel time, $T_r := t-\tau$ is described in literature by its probability $p(t-\tau|t) $, where $t$ is the actual time,  $\tau <  t$ is the injection time and $p(t-\tau|t)$ is the travel time distribution conditional to the actual time.  For details about travel time distribution, residence time and life expectancy of water particles, please take the time to read this paper. Implicitly, it is assumed that the probability is evaluated at the closure (boundary) of a control volume.  The distribution represents the ensable distribution of infinite random experiment that regards the motion of a single molecule or, after the hypothesis of ergodicity, the distribution of the travel time of a bunch of molecules, which are injected together in the system, in different positions.

Actually J&K argued about the mean travel time $<T_r >_t$ more than of its distribution. $<T_r >_t$ definition is:

$<T_r >_t := \int_0^\infty (t-\tau) p(t-\tau|t) d(t-\tau) $

J&K (and the other papers) said that $<T_r >_t$  is affected both by velocity and celerity but I do not think they went right to point. One central fact is that, as follow from Botter et al., 2010, 2011, discharge is estimated through a different probability distribution function, which is $p(t-\tau|\tau)$, conditional on injection time:

$ Q(t) = \int_0^t p(t-\tau|\tau) J(\tau) d\tau $

where $Q(t)$ is the total outflow from a control volume and $J(\tau)$ is the precipitation input. Therefore the breakthrough curve (the backward probability) and the fluxes are not, usually,  linearly related.
$p(t-\tau|\tau)$ and $p(t-\tau|t)$ are not completely independent and are related by the Niemi's relationship

Back to the basics

Let’s me re-discuss the topic from the scratch. When we have a velocity field, $u(\vec{x},t)$ in a fluid it can be either stationary or time varying. Assume it is stationary. Then:

$\frac{\partial u(\vec{x},t)}{\partial t} = 0$

which implies that $u(\vec{x},t) = u(\vec{x})$, i.e. the velocity field, does not depend on time, and it is called stationary. Does it imply that forces do not act ? No. Forces act, because still we can have gradient of velocity:

$\nabla u(\vec{x}) \equiv (\frac{\partial u(\vec{x})}{\partial x}, \frac{\partial u(\vec{x})}{\partial y}, \frac{\partial u(\vec{x})}{\partial z} ) \neq 0 $

So, when the we follow a particle moving (the so called Lagrangian view of fluid motion), it is locally accelerated, and it feels a force. However, forces do not change in time and are subject to stationary resistances (dissipation) that do not allow for the overall system to accelerate (increase or decrease the total kinetic energy). 
If a second particle is injected in the same place of the first one, it will have the same travel time. Particles (parcels, molecules) injected in different points will have different trajectories, all stationary, and the overall distribution of of travel times for a bunch of particles injected together (or, BTW, at a different time) is not time varying. It has to be remarked that the fluxes that come out from such a velocity field is not constant but reflect the mass (volume) of particle injected in the fields. Furthermore, we can say what is going in, will go out in a certain composition of ages which  is relatively easy to disentangle from the knowledge of the spatial field and the amount of injected matter.  
In the above case no net energy is transferred and we do not have to invoke celerity but just velocity. Also,  the integrated probabilities of which we talked at the beginning is: $p(t-\tau|\tau) = p(t-\tau|t) =p(t-\tau)$, i.e. the probabilities are time invariant and not conditional to something.

Now, assume we have a time varying velocity field, so that $u=u(\vec{x},t)$ and:

$ \frac{\partial u(\vec{x},t)}{\partial t} \neq 0$

This implies we have acceleration in (usually) many points which must be generated by a local force (which has to be time varying). The way to obtain it in a fluid is to generate a time-varying pressure gradient (yes, excluding time varying gravity, time varying electromagnetic fields, time varying applied work, time varying temperature or density gradient etc. , all things we usually not deal with in our hydrological problems). Actually it can be shown that these pressure variations, once imposed, can travel from one place to another (in ways that are different in different phenomena: see J&K for the enumeration of some cases) so that this pressure variation is the cause of the varying velocity field. The speed of these travelling pressures is the celerity we (and J&K and others) are talking about. Conjointly with velocity field variations, fluxes in previously unaffected portion of the control volume can be activated (in some part of the control volume particles can start to move or some part of the control volumes are invaded by new moving particles).
Therefore the distribution of particles’ travel times is affected in two ways: by mobilising water (and/or enlarging the sectional area) and by increasing (locally) water velocities.
From the above analysis is also evident that the phrase "celerity (of pressure waves) affects flows" is true, but in an indirect way, since flow is affected by velocity field and its spatial extension (no less, no more), but both are modified by pressure waves (the causal reason).

Consequences on the mean travel time

In the case of time varying velocity fields, distribution of the mean travel time can vary in time because the new activated areas can contain water of ages different from those activated previously, and because locally the velocity of water can increase (or decrease) at the passage of the pressure wave.
The latter effect can skew the distribution of travel time (to smaller or larger travel times), but unless the variation is such that the whole the control section is affected (which can be the case), it has not significant effects on the mean travel time, which is an integrated quantity.
Due to time variation in the velocity field, when we inject two water volumes in the same place at different times, their travel time is different, and consequently, any transported quantity is transported differently. The travel time distribution $p(t-\tau|t)$ is therefore time-variant, and we can attribute this variation to the effect of celerity, meaning that there are pressure waves propagating in the control volumes.  In my view this is a necessary and sufficient condition (mathematically speaking) but we can discuss how inputs variations affect water field velocity and special cases (mostly ideal ones) when the existing flow fields is just extended but not modified where previously existing.

Well, the velocity field is not superimposed, but caused by the quantity of water itself.

So far we discussed the cause of  varying velocity fields which I suggested was due to pressure waves. These, in case of water flow are usually generated by variations of the water amount itself. The velocity field, in fact, is not something independent of the water quantity and distribution and, on the contrary, depends on it. So when we inject water, we cause pressure waves that propagates with a law dependent on the medium: at the surface, in the vadose zone, or in the ground. (Therefore the idea of a stationary flow field with varying water inputs is not, in general, a credible assumption. It can be an approximation to take with care).

I think there is no mystery in all of it.

Post Scriptum (July 8/2016)

I realised that what I wrote was overlooking the normal view (and problems) researchers in the field have about travel time distribution.  The reference is the beautiful (a must read) work Dagan, 1984 cited below. Dagan and followers observe that there is diffusion and, moreover, stationary velocity field can be very heterogeneous.  I forgot diffusion, and, whilst accounting for the second, I did not stressed enough the consequences of heterogeneity on travel time distributions.  For this, the reading of Dagan's paper can be enough.  What I said, however, remains true.


Botter G, Bertuzzo E, Rinaldo A, 2010. Transport in the hydrologic response: Travel time distributions, soil moisture dynamics, and the old water paradox. Water Resources Research 46(3): n/a–n/a. ISSN 1944-7973. doi:10.1029/2009WR008371.

Botter G, Bertuzzo E, Rinaldo A, 2011. Catchment residence and travel time distributions: The master equation. Geophysical Research Letters 38(11): L11403. ISSN 1944-8007. doi:10.1029/2011GL047666.

Dagan, G. (1984). Solute transport in heterogeneous porous formations. Journal of Fluid Mechanics, 145, 151–177.

Hrachowitz, M., Savenije, H., Boogard, T. A., Tetzlaff, D., & Soulsby, C. (2013). What can flux tracking teach us about water age distribution patterns and their temporal dynamics?, Hydrol. Earth Syst. Sciences, 17, 533–564.

Kirchner, J. W. (2016). Aggregation in environmental systems – Part 1: Seasonal tracer cycles quantify young water fractions, but not mean transit times, in spatially heterogeneous catchments,  Hydrol. Earth Syst. Sciences, 279–297.

McDonnell, J. J., & Beven, K. J. (2014). The future of hydrological sciences: A (common) path forward? A call to action aimed at understanding velocities, celerities and residence time distributions of the headwater hydrograph. Water Resources Research, 50, 5342–5350.

Niemi AJ, 1977. Residence time distributions of variable flow processes. The International Journal of Applied Radiation and Isotopes 28(10–11): 855 – 860. ISSN 0020-708X. doi:

Tuesday, June 21, 2016

Is possible to get the runoff for European Countries ?

This is a conversation between two groups of the CLIMAWARE project of some general interest. Further information about global or continental scale hydrology can be found in some previous posts.

Dear Alberto and Riccardo

we are writing about the data we need for the project CLIMAWARE. 

The objective of out analysis is to estimate the impacts on agricolture of modification of:

1)  soil use
2)  anything due to the changed water availability as a consequence of the climate change. 

We are thinking to consider two time frames, the first up to 2020, the second up to 2040 (this because the estimation of soil use variation are available for these two dates and to do an analysis on the national scale for all the Countries of the European Community. 

Alberto's answer:

Dear Martina,

for what I know, there are not detailed projections at this scale. I saw studies on the largest basins (as  Danube) where all is solved by putting R = P- E (runoff = precipitation - potential evapotranspiration). For what I know, nobody looks at the national scale, for the reasons that the river basins do not coincide with nations areas.

There are regional studies (not about the countries but  for Southern Europe, for instance) made by Giorgi’s group or by Julich’s colleagues using their Earth system model. 

Bruno Majone answer:

Dear Martina,
what you are opening is a Pandora’s vessel. I try to explain it better. To deal with water availability at European scale, there are many different approaches. The simplest is to do a hydrological budget simply by using the available climatic projections. Alternatively, one can use hydrological models working at pan-european scale that can be fed directly by the projections themselves. For any of the choices the real problem is selecting the right forcings. There isn’t a reference dataset: there are various “ensemble” of simulations derived from various climate models, each one of them following a different pathways scenarios.  The reference initiative for Europe is Euro-cordex that gathers the modelling work of various research groups that perform experiments with global and regional climate models, and scale down the results from a coarse grid of 125 km to one of around 25 km (see the pdf). Considering the results of these models one should observe that usually they have strong biases that need to be corrected. Otherwise the water budgets can be extremely wrong. For what I know, inside the Euro-Cordex group there are groups that are examining and providing scenarios by using various techniques of bias correction, but I am not sure that they are already available and usable. 

If your interest is to have annually aggregated data by country, I personally would not explore thid road because I believe that the effort to contact the active groups, download huge datasets and precess them could be overwhelming.

I would try instead to contact groups that already have hydrological models setup at European scale and are already estimating climate change impacts. Alberto mentioned the Julich group, but another possibility could be JRC. They have their model LIFSFLOOD set up at 5*5 km of resolution (and it could be that they already have used it to do some impact studies).  At this point you should ask for they to send some gridded data of runoff and afterwards aggregate them at country level.

Monday, June 6, 2016

Java, Python, C/C++ or FORTRAN in scientific programming ?

I found this simple post by Michael Scharf on Quora  that discusses part of the question. It is easy to read, clear, and probably true. So I publish it verbatim (with some comment of mine, especially for FORTRAN)

"I have used  (Java, Python, C/C++ - my note) each of them for 15-20+ years. There is no best. They have different strengths and weaknesses.
  • C and C++ require a lot of discipline because you have to do memory management yourself.
  • C++ is extremely powerful but also very complex.
  • C and C++ are "dangerous" because, if you are not careful, your program can access and modify data that it is not supposed to touch.
  • Python is elegant and designed to be easy to use and read. It has the least distractions when it comes to syntax.
  • The syntax of C, C++ and Java look somewhat similar. Python looks different, it  uses indentation instead of {} to group code
  • Java has the best IDE support (e.g. eclipse or IntelliJ)
  • C and C++ are also statically typed, but the preprocessor can add a level of complexity that can make it difficult to be sure what actually happens.
  • In terms of speed C/C++ are fastest, but for most problems Java is very close in speed. Python can be slow, but if needed critical parts can be written in C. On modern execution speed is rarely the limit - cache behaviour, memory and disc access are the limits.
If you want to learn programming, I would learn Python first, then Java, then C and finally C++.
I personally would not recommend C++ because of its complexity. However, if your are disciplined and have a strict set of rules for a project, C++ can be fantastic.
I would use C only for low level stuff, like writing device drivers.

Java is good for large projects, provided you write good APIs and you are carefully modularizing your software.

Python is good for small projects. If the team and the software gets bigger, it can become hard to maintain unless you have a very good test coverage."

There is no mention of FORTRAN  because since many years, it is just a language that regards the niche of users that do numerics (of any type).  Here a discussion in StackOverflow about this language compared (more or less) to the other languages mentioned above. 
I never really used FORTRAN, so I am not the best person to  talk about it.  I actually moved out of it in 1988 when I decided to use a language with dynamic memory allocation, and I never went back to it.  Generically I believe its velocity is overestimated, especially considering that the average scientific programmer is a bad programmer, and a bad program in FORTRAN can be much slower than a program in any of the above languages.

I saw a lot of tools made in FORTRAN and some large models. But I am not sure it is the best choice for a large project.  Anyway, FORTRAN plus Python is the choice of many in these days. Still I believe it is a sub-optimal choice for managing software projects that go beyond programming a few algorithms, and personally I do not like its syntax and its convoluted object orientation. But I will never personally test if this is actually the truth. 

Moreover, it is true that "It's (FORTRAN's) array handling is nice, with succinct array operations on both whole arrays and on slices, comparable with matlab or numpy but super fast. The language is carefully designed to make it very difficult to accidentally write slow code -- pointers are restricted in such a way that it's immediately obvious if there might be aliasing, as the standard example -- and so the optimizer can go to town on your code. Current incarnations have things like coarray fortran, and do concurrent and forall built into the language, allowing distributed memory and shared memory parallelism, and vectorization. (cit. from Stack Overflow)". However,  it is this attitude that favors such a certain laziness in software design that I do not appreciate. 

For most people, however, is just matter, of what they were taught, i.e. a matter of legacy. They just have a lot of material to which they got used to, and do not want (or have the time) to change. So think that your initial choice can stick with you for most of your life, because what Peter Norvig says, has a lot of foundations (Teach yourself programming in then years).  On the other hand,  consider that, if your activity has to do with programming, during your life you will embrace  more than one programming language (but one will remain your main one). Not because it is mandatory (and for some jobs it is), but because you are this type of guy (gal) that likes this sort of things. 

So what would be my choice for students ? Well, for students in hydrology, I will choose Python and R. But for real life applications, I would chose Java first (as I did).  Eventually, my recent reflections tend to support the idea that the language (with its syntax and attitudes) is just one part of the problem, and that the real focus should be (without making of it an idol) the design of the code and its proper documentation (which has some peculiarities when the concern is a scientific activity).

Wednesday, June 1, 2016

gvSIG Festival

gvSIG is the platform where are migrating my contributions to GIS, thanks to Hydrologis. It is my intention, obviously, to embrace it from version 2.3, hoping that this will be the home sweet home I deeply desired in the last years. In preparing the arrival of the version that contains my tools, the developers had the idea of a nice initiative called gvSIG Festival where many developers were asked to explain their work.

There you can find a lot of useful information. Not specifically about hydrology but someone are. Clicking on the figure you will access the main page. Video Presentations are here.