<<  GRODNO STATE AGRAR UNIVERSITY Hackers  >>
GSI Data Assimilation System
GSI Data Assimilation System
Primary Developers
Primary Developers
Session Outline
Session Outline
GSI System
GSI System
Gridpoint Statistical Interpolation (GSI)
Gridpoint Statistical Interpolation (GSI)
Operational GSI Applications
Operational GSI Applications
Rapid Refresh Domain
Rapid Refresh Domain
GSI Code Flow
GSI Code Flow
Background Fields (xb)
Background Fields (xb)
Observations(yo)
Observations(yo)
Observations(yo) (cont
Observations(yo) (cont
Observation Operator (H)
Observation Operator (H)
Background Errors (B)
Background Errors (B)
L
L
Moisture analysis
Moisture analysis
Observation Errors
Observation Errors
Observation Quality Control
Observation Quality Control
Bias Correction of Radiance Data
Bias Correction of Radiance Data
B-O Histogram
B-O Histogram
Community Support
Community Support
Community GSI Code
Community GSI Code
Definition of Community Codes
Definition of Community Codes
GSI Community Release Timetable
GSI Community Release Timetable
Community GSI Code Repository
Community GSI Code Repository
Users Website
Users Website
GSI Users Guide
GSI Users Guide
On-line Tutorial (Beta Release)
On-line Tutorial (Beta Release)
On-line Documents (Beta Release)
On-line Documents (Beta Release)
GSI Web Brower
GSI Web Brower
O2R2O2R2O2R2O2R2O
O2R2O2R2O2R2O2R2O
GSI: Compile
GSI: Compile
Requirements
Requirements
GSI Directory
GSI Directory
Set environment
Set environment
Configuring GSI
Configuring GSI
Configuring GSI, cont
Configuring GSI, cont
Compiling GSI
Compiling GSI
Clean Compilation
Clean Compilation
Running GSI Users Guide: Chapter 3
Running GSI Users Guide: Chapter 3
To run GSI, you need:
To run GSI, you need:
Background
Background
Observation
Observation
Fixed files
Fixed files
Run script - structure
Run script - structure
Run Script - Experiment variables
Run Script - Experiment variables
Run Script - run directory
Run Script - run directory
Run Script : namelist
Run Script : namelist
Tuning and Diagnostics Users Guide: Chapter 4
Tuning and Diagnostics Users Guide: Chapter 4
Standard out file - stdout
Standard out file - stdout
stdout - structure
stdout - structure
stdout  example(1)
stdout example(1)
stdout  example (2)
stdout example (2)
Single Observation Test
Single Observation Test
Control Data Usage
Control Data Usage
Observation fit - statistic files
Observation fit - statistic files
Convergence Information
Convergence Information
Questions, Comments, Suggestions
Questions, Comments, Suggestions

: GSI Data Assimilation System. : Jamie Wolff. : GSI Data Assimilation System.ppt. zip-: 2050 .

GSI Data Assimilation System

GSI Data Assimilation System.ppt
1 GSI Data Assimilation System

GSI Data Assimilation System

Acknowledgement: NOAA, AFWA, NCAR

Hui Shao, Ming Hu, Laurie Carson, Louisa Nance, Xiang-Yu Huang, Bill Kuo Developmental Testbed Center John Derber, Russ Treadon NOAA/NCEP/Environmental Modeling Center

WRF-DA Tutorial, 20-22 July, 2009, Boulder, CO

2 Primary Developers

Primary Developers

NOAA/NCEP/EMC John Derber, Jim Purser, Russ Treadon, Wan-Shu Wu, Dave Parrish, Lidia Cucurull, Dave Parrish, Manuel Pondeca, Paul van Delst, Daryl Kleist, Xiujuan Su, Yanqiu Zhu, and others NASA/GMAO Ricardo Todling, Ron Errico, Runhua Yang, Ron Gelaro, Wei Gu, and others And NOAA/GSD, NCAR/MMM,

2

3 Session Outline

Session Outline

GSI System and Community Support (30 minutes, presented by Hui Shao) Installation, Running, and Diagnostics (30 minutes, presented by Ming Hu)

3

4 GSI System

GSI System

4

5 Gridpoint Statistical Interpolation (GSI)

Gridpoint Statistical Interpolation (GSI)

3D variational assimilation (same as WRF-VAR) J =1/2 (x-xb)TB-1(x-xb) + 1/2(H(x)-y0)T(E+F)-1(H(x)-y0) + JC Fit to background + Fit to observations + constraints x = Analysis (Output) xb = Background (Input) B = Background error covariance (Input) H = Forward model (within GSI) y0 = Observations (Input) E+F = R = Instrument error + Representativeness error (Input) JC = Constraint terms (within GSI)

5

6 Operational GSI Applications

Operational GSI Applications

* Collaborated with NASA/GMAO and others

System

Implementation date

Primary Developers

Physical SST retrieval

9/27/2005

NCEP/EMC*

NAM (regional)

6/20/2006

NCEP/EMC*

RTMA

8/22/2006

NCEP/EMC*

GFS (Global)

5/1/2007

NCEP/EMC*

HWRF

6/19/2007

NCEP/EMC*

Rapid Refresh (RR)

2010

NOAA/GSD

AFWA operational

2010 or 2011

NCAR/MMM

6

7 Rapid Refresh Domain

Rapid Refresh Domain

Air Force Weather Agency Domains

NAM

HWRF

7

8 GSI Code Flow

GSI Code Flow

Read in & distribute observations, background & background errors

Additional initializations

User input & initializations

Outer loop a) Set up right hand side of analysis equation (Compute observation innovations) b) Call inner loop Compute gradient information Apply background error Compute search direction Compute step size Update analysis increment

Write analysis & related output

8

9 Background Fields (xb)

Background Fields (xb)

Currently works for the following systems Global GFS GMAO global Regional WRF (ARW & NMM) - binary and netcdf NCEP RTMA

9

10 Observations(yo)

Observations(yo)

*Some of the data are restricted

Radiosondes Pibal winds Synthetic tropical cyclone winds Wind profilers Conventional aircraft reports ASDAR aircraft reports MDCARS aircraft reports** Dropsondes MODIS IR and water vapor winds GMS, METEOSAT and GOES cloud drift IR and visible winds GOES water vapor cloud top winds

Surface land observations* Surface ship and buoy observation SSM/I wind speeds QuikScat wind speed and direction SSM/I precipitable water SSM/I and TRMM TMI precipitation estimates Doppler radial velocities VAD (NEXRAD) winds GPS precipitable water estimates GPS Radio occultation refractivity profiles SBUV ozone profiles (other ozone data under test)

10

11 Observations(yo) (cont

Observations(yo) (cont

Regional GOES-11 and 12 Sounders thinned to 120km Channels 1-15 Individual fields of view 4 Detectors treated separately Over ocean only AMSU-A thinned to 60km NOAA-15 Channels 1-10, 12, 15 NOAA-18 Channels 1-8, 10-11, 15 AMSU-B/MHS thinned to 60km NOAA-15 Channels 1-3, 5 NOAA-16 Channels 1-5 NOAA-17 Channels 1-5 NOAA-18 Channels 1-5 HIRS thinned to 120km NOAA-17 Channels 2-15

Global GOES-11 and 12 Sounders thinned to 180km Channels 1-15 Individual fields of view 4 Detectors treated separately Over ocean only AMSU-A thinned to 145km NOAA-15 Channels 1-10, 12-13, 15 NOAA-18 Channels 1-8, 10-13, 15 METOP Channels 1-13, 15 AQUA Channels 1-6, 8-13, 15 AMSU-B/MHS thinned to 240km NOAA-15 Channels 1-3, 5 NOAA-16 Channels 1-5 NOAA-17 Channels 1-5 NOAA-18 Channels 1-5 METOP Channels 1-5 HIRS - thinned to 180km NOAA-17 Channels 2-15 METOP Channels 2-15 AIRS thinned to 180km AQUA 148 Channels

11

12 Observation Operator (H)

Observation Operator (H)

To use observation, GSI simulates observation using analysis variables observation operator (H) Can be simple interpolation to ob location/time. Can be more complex (e.g., radiation transfer). For radiances, GSI uses CRTM.

12

13 Background Errors (B)

Background Errors (B)

Space correlation computed using recursive filters in horizontal and vertical Multivariate relation Flow dependent variability in background error Background error variances modified based on 9 and 3 hour forecast differences: Variance increased in regions of rapid change Variance decreased in calm regions Global mean variance ~ preserved Being used for regional (US) surface analysis operationally.

13

14 L

L

Surface pressure background error standard deviation fields Valid at 00Z November 06, 2007

HPC Surface Analysis

b) with flow dependent re-scaling

rescaled

a) without re-scaling

as is

14

15 Moisture analysis

Moisture analysis

Option 1: univariate Temperature (blue) increment forces large increment in RH (shaded).

Option 2: multivariate Temperature (blue) increment forces increment in q (red). much smaller RH (shaded) increment.

16 Observation Errors

Observation Errors

Improved specification of observational errors Adaptive Tuning

After tuning

After tuning and smoothing

Before tuning

16

17 Observation Quality Control

Observation Quality Control

External platform specific QC Some gross checking in PREPBUFR file creation Optimal interpolation quality control (OIQC) on its way out Analysis QC Gross checks specified in input data files Variational quality control (VarQC) implemented operationally in Feb 2009

Number of data rejected by OIQC

VarQC weight (W)

17

18 Bias Correction of Radiance Data

Bias Correction of Radiance Data

NCEP uses a two step process for Tb Scan angle correction based on position Air mass correction based on predictors Predictors mean path length (local zenith angle determined) integrated lapse rate integrated lapse rate ** 2 cloud liquid water

18

19 B-O Histogram

B-O Histogram

DMSP15 July2004 : 1month before bias correction after bias correction

19

20 Community Support

Community Support

20

21 Community GSI Code

Community GSI Code

Goals: Provide current operational GSI capability to the research community (O2R) Provide a framework for distributed development of new capabilities & advances in data assimilation (R2O)

DTC

Research Community

System

Implementation date

Mode

Physical SST retrieval

9/27/2005

CRTM + analytical solution

NAM (regional)

6/20/2006

3D-VAR

RTMA

8/22/2006

2D-VAR

Global

5/1/2007

3D-VAR

HWRF

6/19/2007

3D-VAR

RR

Early 2010

AFWA operational

2010 or early 2011

21

22 Definition of Community Codes

Definition of Community Codes

Free and shared resource Ongoing distributed development by both research and operational communities Maintained under version control Periodic releases made available to the community Includes latest developments of new capabilities and techniques Centralized support Provided in collaboration with developers

22

23 GSI Community Release Timetable

GSI Community Release Timetable

Tasks

Timeline

Note

Beta release (Q1FY09)

Jun, 2009

Friendly user only (through the GSI website)

First release (Q1FY09)

Sep, 2009

With user support

Tutorial

Jun, 2010

Residential tutorial and hand-on practical session

23

24 Community GSI Code Repository

Community GSI Code Repository

Proposed Structure

Success of this structure will depend on communication and collaboration among all GSI developers, users, and DTC.

NCEP EMC Repository

24

25 Users Website

Users Website

http://www.dtcenter.org/com-GSI/users/

Release announcement System component descriptions Documentation Users Guide Presentations Registration Software downloads Bug fix reports User support information gsi_help@ucar.edu Tutorial information

25

26 GSI Users Guide

GSI Users Guide

Users Guide will be updated to be consistent with each new release to the community.

26

27 On-line Tutorial (Beta Release)

On-line Tutorial (Beta Release)

27

28 On-line Documents (Beta Release)

On-line Documents (Beta Release)

28

29 GSI Web Brower

GSI Web Brower

29

30 O2R2O2R2O2R2O2R2O

O2R2O2R2O2R2O2R2O

30

31 GSI: Compile

GSI: Compile

31

31

32 Requirements

Requirements

System required libraries FORTRAN 90/95 compiler C compiler Perl netCDF GSI system Download GSI system tar files (GSIbeta.tar.gz ) from http://www.dtcenter.org/com-GSI/users/index.php gunzip and untar tar zxvf GSIbeta.tar.gz (Should see GSI/. directory ) cd to GSI directory cd GSI

32

32

33 GSI Directory

GSI Directory

Compile rules

Compile scripts

Source code directory

Run directory

33

sorc

sorc

sorc

sorc

sorc

sorc

sorc

sorc

sorc

sorc

sorc

sorc

makefile

mains

libs

libs

libs

libs

libs

libs

libs

libs

libs

libs

bacio

crtm_gfsgsi

bufr

sfcio

gfsio

sigio

w3

mpeu

makefile

sp

34 Set environment

Set environment

WRF_DIR WRF needs to be compiled prior to compiling GSI GSI uses WRF I/O API libraries to do file input and output WRF directory specified: setenv WRF_DIR /home/user/WRFV3 netCDF If netCDF libraries are not located in the standard /usr/local , then setenv NETCDF path for netCDF For LINUX systems, make sure the netCDF libraries are installed using the same compiler (PGI, Intel, g95) that will be used to compile GSI.

34

34

35 Configuring GSI

Configuring GSI

To create a GSI configuration file for your computer: ./configure This script checks the system hardware and software (mostly netCDF), and then offers the user choices for configuring GSI: Choices for 32-bit LINUX operated machines are: 1. Linux i486 i586 i686, PGI compiler 2. Linux i486 i586 i686, Intel compiler 3. Linux i486 i586 i686, gfortran compiler Choices for IBM machines are: 1. AIX xlf compiler with xlc

35

35

36 Configuring GSI, cont

Configuring GSI, cont

configure.gsi Created by the ./configure command contains compilation options, rules, etc. specific to your computer can be edited to change compile options, if desired. At this time, the IBM option is well tested. Working on Linux option test. The arch/configure.defaults file can be edited to add a new option if needed.

36

36

37 Compiling GSI

Compiling GSI

To compile: ./compile >& compile_gsi.log To get compile help message: ./compile -h If the compilation is successful, it will create one executable under bin/: gsi.exe

37

37

38 Clean Compilation

Clean Compilation

To remove all object files and executables ./clean To remove all built files, including configure.gsi ./clean a Clean is recommended if compilation failed want to change configuration file

38

38

39 Running GSI Users Guide: Chapter 3

Running GSI Users Guide: Chapter 3

39

40 To run GSI, you need:

To run GSI, you need:

GSI Executable Background (first guess) file Observations Not needed for single observation experiment Fixed files (within GSI package) Run script (namelist included)

40

41 Background

Background

GSI can use WRF NMM input file in binary format WRF NMM input file in netcdf format WRF ARW input file in binary format WRF ARW input file in netcdf format GFS input file in binary format GMAO global model input file in binary format DTC has only tested regional analysis with WRF input On IBM: both binary and nedcdf format On Linux: netcdf format only

41

42 Observation

Observation

All observations have to be in BUFR format prepbufr: NCEP flavor BUFR Need NCEP BUFR library

42

43 Fixed files

Fixed files

Collection of statistic and control files under fix directory Background and observation errors berror_stats, errtable Observation data control file (info files) convinfo , satinfo Bias correction used by radiance analysis satbias_angle, satbias_in Radiance coefficient used by CRTM EmisCoeff.bin, CloudCoeff.bin

43

44 Run script - structure

Run script - structure

Ask for computer resources to run GSI Set environment variables for the machine Set experiments variables (experiment name, analysis time, background and observation) Check the definition of required variables Generate a run directory for GSI (working or temp directory) Copy GSI executable to run directory Copy background file to run directory Copy or link observations to run directory Copy fixed files to run directory Generate namelist for GSI Run the GSI executable Save the GSI analysis results

44

45 Run Script - Experiment variables

Run Script - Experiment variables

# analysis time (YYYYMMDDHH) ANAL_TIME=2008051112 # working direcotry, where GSI runs WORK_ROOT=./gsi/case # path and name of background file BK_FILE=./2008051112/bkARW/wrfout_d01_2008-05-11_12:00:00 # path of observations OBS_ROOT=./gsi/case PREPBUFR=./tutorialcases/data/newgblav.gdas1.t12z.prepbufr.nr # path of fix files FIX_ROOT=./GSI/fix # path and name of the gsi executable GSI_EXE=./GSI/bin/gsi.exe # which background error covariance and parameter will be used (GLOBAL or NAM) bkcv_option=NAM

45

46 Run Script - run directory

Run Script - run directory

Run directory Create a new directory for each run Copy or link data Executable Background Must be copied, will be over-written by analysis result Observations Link or touch if not exist Fixed files Link

46

47 Run Script : namelist

Run Script : namelist

Generated by running scripts Change namelist by editing run script Works for both global and regional analysis Detailed explanation in section 3.4

&SETUP miter=2,niter(1)=5,niter(2)=5, write_diag(1)=.true.,write_diag(2)=.false.,write_diag(3)=.true., qoption=1, gencode=78,factqmin=0.005,factqmax=0.005,deltim=$DELTIM, ndat=59,npred=5,iguess=-1, oneobtest=.false.,retrieval=.false.,l_foto=.false., use_pbl=.false., $SETUP / &GRIDOPTS JCAP=$JCAP,NLAT=$NLAT,NLON=$LONA,nsig=$LEVS,hybrid=.true., wrf_nmm_regional=.false.,wrf_mass_regional=.true., diagnostic_reg=.false., filled_grid=.false.,half_grid=.true.,netcdf=.true., regional=.true.,nlayers(63)=3,nlayers(64)=6, $GRIDOPTS

47

48 Tuning and Diagnostics Users Guide: Chapter 4

Tuning and Diagnostics Users Guide: Chapter 4

48

49 Standard out file - stdout

Standard out file - stdout

Includes lots of important information First place to look after any GSI run If successful Data distribution Optimal iteration Maximum and minimum of analysis fields If fails, which part of GSI has problem what is the possible reason for failure

49

50 stdout - structure

stdout - structure

read in all data and prepare analysis: read in configuration (namelist) read in background read in constant file (fixed file) read in observation partition background and observation data for parallel analysis optimal iteration (analysis) save analysis result

50

51 stdout  example(1)

stdout example(1)

Top of stdout: read in background End of stdout: write out analysis results

Read CRTM coefficient

0: rmse_var=T 0: ordering=XYZ 0: WrfType,WRF_REAL= 104 104 0: ndim1= 3 0: staggering= N/A 0: start_index= 1 1 1 785191608 0: end_index= 69 64 45 -1603623772 0: k,max,min,mid T= 1 309.9411316 264.5114136 289.7205811 0: k,max,min,mid T= 2 310.6200562 269.5698547 295.0413208 . 0: k,max,min,mid T= 45 510.0127563 472.5627441 494.7407227

0: Read_SpcCoeff_Binary(INFORMATION) : FILE: amsua_n15.SpcCoeff.bin; 0: SpcCoeff RELEASE.VERSION: 7.01 N_CHANNELS=15 0: Read_SpcCoeff_Binary(INFORMATION) : FILE: amsub_n15.SpcCoeff.bin; 0: SpcCoeff RELEASE.VERSION: 7.01 N_CHANNELS=5 0: Read_SpcCoeff_Binary(INFORMATION) : FILE: hirs3_n16.SpcCoeff.bin; 0: SpcCoeff RELEASE.VERSION: 7.01 N_CHANNELS=19 0: Read_SpcCoeff_Binary(INFORMATION) : FILE: amsua_n16.SpcCoeff.bin; 0: SpcCoeff RELEASE.VERSION: 7.01 N_CHANNELS=15

51

52 stdout  example (2)

stdout example (2)

0: PROGRAM GSI_ANL HAS ENDED. IBM RS/6000 SP

Read observations

Iteration

6: READ_LIDAR: bufr file date is 2007 12 20 0 11: READ_AIRS: bufr file date is 2007 12 20 0 5: READ_RADAR: vad wind bufr file date is 2007 12 20 0 1: READ_PREPBUFR: bufr file date is 2007 12 20 0 9: READ_BUFRTOVS: bufr file date is 2007 12 20 0 mhsbufr 5: n,lat,lon,qm= 1 47.04 246.01 -9 -9 -9 -9 -9 -9 -9 -9 2 3 -9 3 -9 2 -9 4: READ_PREPBUFR: messages/reports = 872 / 97301 ntread = 1

0: 0: GLBSOI: START pcgsoi 0: penalty,grad ,a,b= 1 0 0.160014047732364506E+05 0.724764165295960149E+06 0.132279098881E-02 0.00000000000E+00 0: pnorm,gnorm, step? 1 0 0.100000000000E+01 0.10000000000E+01 good 0: penalty,grad ,a,b= 1 1 0.150427003330115986E+05 0.608359222451359266E+06 0.124401271687E-02 0.83938151711E+00 0: pnorm,gnorm, step? 1 1 0.940086232815E+00 0.83938982252E+00 good

52

53 Single Observation Test

Single Observation Test

A good way to check ratio of background and observation variance pattern of background error covariance Setup in namelist SETUP oneobtest=.true., SINGLEOB_TEST maginnov=1.0, magoberr=1.0, oneob_type='t', oblat=20., oblon=285., obpres=850., obdattim= 2007081500, obhourset=0.,

53

54 Control Data Usage

Control Data Usage

From namelist and the links of observation files dfile(01)='prepbufr, dtype(01)='ps, dplat(01)=' ', dsis(01)='ps' dfile(02)='prepbufr dtype(02)='t', dplat(02)=' ', dsis(02)='t', dfile(27)='amsuabufr,dtype(27)='amsua,dplat(27)='n15,dsis(27)='amsua_n15', dfile(28)='amsuabufr,dtype(28)='amsua,dplat(28)='n16,dsis(28)='amsua_n16', From info file (convinfo ) otype type sub iuse twindow numgrp ngroup nmiter gross ermax ermin var_b var_pg ps 111 0 -1 1.5 0 0 0 5.0 3.0 1.0 10.0 0.000 ps 132 0 -1 1.5 0 0 0 5.0 3.0 1.0 10.0 0.000 ps 180 0 1 1.5 0 0 0 5.0 3.0 1.0 10.0 0.000 ps 181 0 1 0.25 0 0 0 5.0 3.0 1.0 10.0 0.000 ps 182 0 1 1.5 0 0 0 5.0 3.0 1.0 10.0 0.000 ps 183 0 -1 0.25 0 0 0 5.0 3.0 1.0 10.0 0.000 ps 187 0 1 0.25 0 0 0 5.0 3.0 1.0 10.0 0.000 ps 188 0 -1 0.25 0 0 0 5.0 3.0 1.0 10.0 0.000 t 120 0 1 1.5 0 0 0 7.0 5.6 1.3 10.0 0.000 t 126 0 -1 1.5 0 0 0 7.0 5.6 1.3 10.0 0.000 t 130 0 1 1.5 0 0 0 7.0 5.6 1.3 10.0 0.000 t 131 0 1 1.5 0 0 0 7.0 5.6 1.3 10.0 0.000 t 132 0 1 1.5 0 0 0 7.0 5.6 1.3 10.0 0.000

54

55 Observation fit - statistic files

Observation fit - statistic files

55

56 Convergence Information

Convergence Information

stdout 0: 0: GLBSOI: START pcgsoi 0: penalty,grad ,a,b= 1 0 0.160014047732364506E+05 0.724764165295960149E+06 0.132279098881E-02 0.00000000000E+00 0: pnorm,gnorm, step? 1 0 0.100000000000E+01 0.10000000000E+01 good 0: penalty,grad ,a,b= 1 1 0.150427003330115986E+05 0.608359222451359266E+06 0.124401271687E-02 0.83938151711E+00 0: pnorm,gnorm, step? 1 1 0.940086232815E+00 0.83938982252E+00 good More detail in fort.220 J= 0.000000000000000000E+00 0.000000000000000000E+00 0.000000000000000000E+00 0.000000000000000000E+00 0.111428969204742752E+05 0.262756287766784135E+04 0.256346518803503204E+04 0.921301367380113305E+02 0.244005730835839540E+04 0.000000000000000000E+00 0.000000000000000000E+00 0.000000000000000000E+00 0.000000000000000000E+00 0.000000000000000000E+00 0.000000000000000000E+00 0.000000000000000000E+00 0.000000000000000000E+00 0.000000000000000000E+00 0.253237990246203207E-01 0.225585966101596478E+04 20 items in cost function: 5 wind observation 6 satellite radiance observation 7 temperature observation 8 precipitable water obs. 9 specific humidity obs. 20 surface pressure observation

56

57 Questions, Comments, Suggestions

Questions, Comments, Suggestions

57

GSI Data Assimilation System
http://900igr.net/prezentacija/anglijskij-jazyk/gsi-data-assimilation-system-134799.html
c

661

29
900igr.net > > > GSI Data Assimilation System