2. Ron Gelaro; Data Assimilation Office/NASA GSFC
The singular vectors (SVs) of a dynamical system provide a mathematically rigorous, but tractable, approach to quantifying perturbation growth over finite time intervals. As such, SVs have been applied to a variety of problems in meteorology including the study of baroclinic instability, forecast error growth, sensitivity analysis, ensemble weather prediction and, most recently, as a means of identifying locations for collecting adaptive observations to improve numerical weather forecasts. Despite the successful application of SVs to such problems and the insights gained therein, there is confusion, as well as criticism, surrounding the definition, applicability and interpretation of SVs, especially when viewed in the context of traditional paradigms and conceptual models. The most common points of contention include the dependence of SVs on the choice of norm, their dynamical balance properties, their correspondence to "observable" atmospheric phenomena, their relationship to forecast and analysis errors, and their relevance to Lyapunov and super-Lyapunov growth. In this talk, we present an overview of the research evidence that addresses and, in many cases, defuses these issues, as well as highlights unresolved issues of ongoing research.
I will start by recalling some points which have struck me over 25 years as being particularly important for data assimilation (DA):
Incremental 4D-Var is one approach to DA which can be designed using these points. I will describe the design philosophy of the Met Office's scheme, which uses a simplified Perturbation Forecast model. The simplification and incrementing operations are an important part of the design; for instance the adjoint incrementing operator allows us to construct meteorologically plausible covariances between moisture, cloud and precipitation perturbations.
Alternatively, the Ensemble Kalman Filter is attractively simple in that physical insight is only needed in designing the forecast model and observation operators; there is no need for any adjoints (or Adjoint Workshops!?). All the multivariate covariances needed can be estimated directly from an appropriately constructed ensemble. The major weakness is that these estimates are inaccurate because of small ensemble sizes. To avoid spurious effects it is better to truncate covariances; this is best done in an "observation space" algorithm, where point to point covariances are explicitly calculated.
Looking ahead, there a two clear trends. The first is towards ever increasing volumes of remote sensing observations. The DA method will have to extract useful information from the tendencies and movements of tracers in such data. The second is towards increased computer power coming from a "Grid" of many linked computers, as well as better performance from supercomputers. I will speculate how well the two methods discussed will cope with these trends.
The development of adjoint and corresponding tangent linear versions of physical schemes for atmospheric models is often not straightforward. I will begin by reviewing the general issues of linearizing discontinuities often present in such schemes by using a simple stratiform precipitation model as an example. But for most of the talk I will describe an analysis of the linearization of a vertical diffusion scheme, for which discontinuities are only a minor issue. First I will show how direct linearization of the scheme is unsuitable. Next I will describe a generalizable technique for examining the problem in detail. Then I will describe some possible modifications to construct a more suitable tangent linear scheme, all of which have undesirable aspects. The required careful examination of this problem reveals undesirable aspects of the original nonlinear scheme as well as its tangent linear version. For such a scheme, it will therefore be demonstrated that both tangent linear and nonlinear versions should be developed simultaneously. I will conclude with a description of some ways to use Jacobians for facilitating adjoint-model development.
Models describing atmospheric flows possess the property that two or more initially slightly different initial states - each evolving according to the same physical laws - do, in general, over time develop into states no more similar than two or more randomly observed states of the atmosphere. This inherent error growth is not an artifact of the models, but is a consequence of the nonlinearity and instability of atmospheric dynamics. Such error growth coupled with the principal inevitability of initial-state errors implies that the atmosphere possesses a inherent lack of predictability.
The lack of atmospheric predictability manifests itself in an intrinsic uncertainty of weather forecasts made with atmospheric numerical weather prediction (NWP) models. This forecast uncertainty is further increased through errors within the model formulation itself.
The study of atmospheric predictability is primarily concerned with the question of how rapidly and by what physical processes do perturbations of small amplitude amplify (for certain scales and certain fields) in the atmosphere. Such questions (e.g., what is the rate of error growth) characterizing atmospheric predictability can be investigated by observing how quickly initially nearby model trajectories diverge; clearly, for realistic answers, NWP model behavior must be sufficiently close to observed atmospheric behavior.
In this talk work on atmospheric predictability will be reviewed and discussed. Early and more recent evidence for error growth, as well as estimates for growth rates will be presented. Given that the specific properties of error growth are dependent on both the structure of perturbations, as well as on the basic flow itself, ensemble prediction systems are aiming at predicting the case-dependent error evolution given initial-error estimates. In this process, appropriate initial-error estimates, as well as an appropriate strategy for constructing initially nearby model trajectories are necessary prerequisites. Both aspects, also in conjunction with their relationship to data assimilation, will be discussed from the perspective of atmospheric predictability.
The background error covariance matrix is a crucially important quantity in any data assimilation system. Unfortunately, its construction is far from straightforward. The statistical properties of background errors are difficult to diagnose. Moreover, the enormous size of the matrix makes it necessary to parameterize the statistics in a way that allows the matrix to be represented as a product of relatively simple, sparse matrices. Finally, the cross-correlations due to dynamical balances between mass and wind fields must be incorporated. A review of the main approaches to constructing background error covariance matrices will be presented. The emphasis will be on variational data assimilation for numerical weather prediction. The following issues will be discussed: diagnosis of background error using innovation statistics, forecast differences, and ensembles of analyses; parameterization of background error statistics, including ways to incorporate anisotropy and inhomogeneity; and methods for incorporating dynamical balance, including quasi-geostrophic and semi-geostrophic balance.
The Hessian matrix is the second derivative of the variational cost function, and its inverse is the theoretical analysis error covariance. In a recent article F.X. LeDimet, I.M. Navon and D.N. Daescu review the "Second Order Information in Data Assimilation". Several of the applications mentioned in their paper have in recent years been incorporated in the operational system at ECMWF, by Mike Fisher and colleagues.
In this tutorial the theory of the applications will be outlined and illustrated with examples from ECMWFs global 4D-Var. This will include Hessian pre-conditioning and rate-of-convergence, dynamical evolutions of the Hessian, diagnostics of 4D-Var performance from a Hessian perspective, Hessian sensitivity, and analysis error estimation.