forked from barronh/WHIPS
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathREADME.txt
1365 lines (1210 loc) · 49 KB
/
README.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
PROJECT TITLE: WHIPS
PURPOSE OF PROJECT: Provide a well-documented, easy-to-use general-purpose
processing module for processing satellite data
VERSION: 1.2.2 (8/28/13)
AUTHORS: oberman, maki, strom
CONTACT: [email protected]
ACKNOWLEDGEMENT POLICY
======================
Whenever you publish research or present results generated with WHIPS,
please include the following acknowledgement or an appropriate
equivalent:
We wish to thank the University of Wisconsin--Madison for the
use and development of the Wisconsin Horizontal Interpolation
Program for Satellites (WHIPS). WHIPS was developed by Jacob
Oberman, Erica Scotty, Keith Maki and Tracey Holloway, with
funding from the NASA Air Quality Applied Science Team
(AQAST) and the Wisconsin Space Grant Consortium Undergraduate
Award.
QUICK START
===========
1. Install the program and run the built-in test module to confirm
that it is working properly. Installation instructions can be found
in the file INSTALL.txt
2. Download whatever data you plan to process. Currently, the program
is designed to process OMI NO2 DOMINO level 2 data, OMI NO2 NASA level
2 data, MOPITT CO data, and MODIS AOD level 2 data. See the --filelist
argument documentation for more on aquiring data.
3. Navigate to the folder where whips.py installed or add it to
your path. It should be the /bin folder corresponding to the /lib
folder where your python packages live. Invoke it as:
whips.py --help
4. Follow the on-screen instructions, adding each of the required
parameters. If you need help with the projection attributes or the
output function attributes, invoke the built-in help as:
whips.py --AttributeHelp <function_name>
For detailed explanations of all parameters and attributes, see the
"Parameter Details" section below.
5. Invoke whips.py once for each output file you'd like to create.
Note that the software creates output files with only a single
timestep, so you'll need to invoke the command once for each timestep
(IE if you want a month with timesteps every day, you'll probably want
to write a shell script that calls the command once for each day)
6. Additionally, you may create a grid file for your chosen grid by
including the --includeGrid flag followed by the desired filename.
A file containing the gridcells used by the projection will be written
to the same directory as the standard output file.
7. Concatenate your outputs if desired (the authors recommend the NCO
operators at http://nco.sourceforge.net/ if you're using a netCDF
output format) and carry on!
INVOKING WHIPS - METHOD 1
============================================
Calling directly from the command line
--------------------------------------------
There are two primary methods of running whips. The first method
is to give whips all the information and settings it needs as flags
in a single command line call. The calls become very long; typing
them in by hand is therefore inadvisible. The script can easily
be batch run by building the call in a script.
For clarity and readability, line continuation characters are used to
place each attribute on a separate line. It is not required to break
up attributes like this, but the command line needs to see the
invocation as a single command, so if you want to break it onto
multple lines you must use line-continuation characters.
1A. Process MOPITT level 2 CO data, (Version 5)
----------------------------------------------
- Uses a 36km lambert conic conformal grid centered over North
America
- Writes out a 2D, 3D, and 4D parameter from the file
whips.py --directory /where/you/have/input/files \
--fileList MOP02T-20050101-L2V10.1.1.prov.hdf \
--filetype MOPITT_CO_NASA_HDF_V5 \
--gridProj lcc2par \
--mapFunc point_in_cell \
--outDirectory where/you/want/output \
--outFileName descriptive_name.nc \
--verbose True \
--interactive True \
--projAttrs xOrig:-2916000 yCell:36000 \
refLon:-97 refLat:40 nCols:162 nRows:126 stdPar2:45 \
stdPar1:33 xCell:36000 earthRadius:6370000 yOrig:-2268000 \
"inFieldNames:Time,Retrieved CO Mixing Ratio Profile,Retrieved CO Surface Mixing Ratio" \
outFieldNames:time,COprof,COsurf logNormal:False,True,True \
timeStart:00:00:00_01-01-2005 timeStop:23:59:59_01-01-2005 \
timeComparison:UTC fillVal:-9999.0 \
solZenAngCutoff:85 dayTime:True
2A. Process OMI level 2 DOMINO NO2 data, (Version 2)
---------------------------------------------------
- Uses a 36km lambert conic conformal grid centered over North
America
- explicitly specifies 3 input files
- includes an output grid file (this would be possible in any example,
but is only shown in this one)
- Writes out a 2D, and 3D parameter from the file
whips.py \
--directory /where/you/have/input/files \
--filetype OMI_NO2_KNMI_HDF_v2_0_postFeb2006 \
--fileList OMI-Aura_L2-OMDOMINO_2011m0901t1502-o37926_v003-2011m1012t121342.he5 \
OMI-Aura_L2-OMDOMINO_2011m0901t1641-o37927_v003-2011m1012t121458.he5 \
OMI-Aura_L2-OMDOMINO_2011m0901t1820-o37928_v003-2011m1012t121613.he5 \
--gridProj lcc2par \
--mapFunc regional_intersect \
--outDirectory /where/you/want/output \
--outFileName some_descriptive_name.nc \
--verbose True \
--interactive False \
--includeGrid /where/you/want/gridfile/output \
--projAttrs xOrig:-48 yCell:36 refLon:-97 refLat:40 \
nCols:30 nRows:32 stdPar2:45 stdPar1:33 xCell:36 \
earthRadius:6370 yOrig:-552 \
--outFuncAttrs \
inFieldNames:Time,AveragingKernel,TroposphericVerticalColumn \
outFieldNames:time,avKern,tropVCD \
timeStart:00:00:00_09-01-2011 \
timeStop:23:59:59_09-01-2011 timeComparison:UTC \
fillVal:-9999 cloudFractUpperCutoff:0.3 \
solarZenAngUpperCutoff:85 includePixelCount:False
3A. Process OMI level 2 NASA NO2 data (Version 1.2)
--------------------------------------------------
- uses a 36 km lambert conic conformal grid
- writes out 2 fields to an output file
- considers 2 input files
- looks at all the cornerfiles in the directory
whips.py \
--directory /where/you/have/input/files \
--fileList OMI-Aura_L2-OMNO2_2011m0430t1440-o36120_v003-2011m0501t043317.he5 \
OMI-Aura_L2-OMNO2_2011m0430t1619-o36121_v003-2011m0501t061955.he5 \
--filetype OMI_NO2_NASA_HDF_v1_2 \
--gridProj lcc2par \
--mapFunc regional_intersect \
--outDirectory /where/you/want/output \
--outFileName some_file_name.nc \
--verbose True \
--interactive False \
--projAttrs cornerDir:/directery/where/you/keep/cornerfiles \
cornerFileList: \
stdPar1:33 stdPar2:45 refLat:40 refLon:-97 xOrig:-2916000 \
yOrig:-2268000 xCell:36000 yCell:36000 \
nRows:126 nCols:162 earthRadius:6370000 \
inFieldNames:ColumnAmountNO2Trop,Time \
outFieldNames:tropVCD,time \
timeComparison:local timeStart:00:00:00_04-30-2011 \
timeStop:23:59:59_04-30-2011 cloudFractUpperCutoff:.3 \
solarZenAngUpperCutoff:85 fillVal:-9999.0 includePixelCount:False
4A. Process MODIS level 2 NASA AOD data (Version 1)
--------------------------------------------------
- uses a 36km Lambert Conical projection centered over the US
- writes out 2 fields to an output file
- considers all input files in the directory
- includes the number of level 2 pixels averaged for each level 3 pixel
whips.py \
--directory /where/you/have/input/files \
--filetype MODIS_AOD_NASA_Collection_5 \
--gridProj lcc2par \
--mapFunc point_in_cell \
--outDirectory /where/you/want/output \
--outFileName some_descriptive_name.nc \
--verbose True \
--interactive True \
--projAttrs xOrig:-2916000 yOrig:-2268000 \
xCell:36000 yCell:36000 nRows:126 nCols:162 \
refLat:40 refLon:-97 stdPar1:33 stdPar2:45 \
earthRadius:6370000 \
--outFuncAttrs \
timeStart:00:00:00_07-03-2008 \
timeStop:23:59:59_07-03-2008 \
timeComparison:UTC \
fillVal -9999.0 \
inFieldNames:Corrected_Optical_Depth_Land,Effective_Optical_Depth_Average_Ocean \
outFieldNames:AOD_Land,AOD_Ocean \
includePixelCount:True
INVOKING WHIPS - METHOD 2
============================================
Using a formatted input file
--------------------------------------------
The second method of running WHIPS is to specify all the options
and information it needs in a specially-formatted input file. The
command line call then consists only of specifying the location of
the input file WHIPS should read from.
Below is an explanatory example of the format:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
FAKE INPUT FILE
BEGIN
. This line begins with a period. That means it is a comment
. Comments will be ignored by WHIPS when processing the file
. All required flags must be specified with the flag name
. and the desired value. These flags must be in capital case.
. All possible flags (including optional flags) shown below:
DIRECTORY = /where/you/had/your/input/files
FILETYPE = some_file_type
GRIDPROJ = some_grid_projection
MAPFUNC = some_map_function
OUTDIRECTORY = /where/you/want/output/files
OUTFILENAME = name_of_output_file
VERBOSE = True_or_False
INTERACTIVE = True_or_False
OUTFUNC = some_output_function
INCLUDEGRID = /absolute/path/to/output/file/for/grid
. FILELIST is delimited by spaces. Leave out to use all files
. in DIRECTORY
FILELIST = list of files
. Projection and output function attributes are specified
. the same as flags. Just as with the command line call,
. the correct attributes must be present or the program
. will exit.
stdPar1 = 33
inFieldNames = ColumnAmountNO2Trop,Time
. Note that unlike with the command line call, quotations are
. not needed when whitespace is part of an attribute value
outUnits = Molecules cm^-2,seconds since epoch
. You can specify parameters in any order. This includes
. intermixing attributes and flags
time = Time
OUTFILENAME = whatever_we_want
. WHIPS input files must begin with BEGIN and end with END
END
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The examples below are equivalent to their counterparts
above in the section on method 1.
1B. Process MOPITT level 2 CO data, (Version 5)
----------------------------------------------
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
BEGIN
DIRECTORY = /where/you/have/input/files
FILELIST = MOP02T-20050101-L2V10.1.1.prov.hdf
FILETYPE = MOPITT_CO_NASA_HDF_V5
GRIDPROJ = lcc2par
MAPFUNC = point_in_cell
VERBOSE = True
INTERACTIVE = True
. Projection Attributes
xOrig = -2916000
yCell = 36000
refLon = -97
refLat = 40
nCols = 162
nRows = 126
stdPar2 = 45
stdPar1 = 33
xCell = 36000
earthRadius = 6370000
yOrig = -2268000
. output function attributes
inFieldNames = Time,Retrieved CO Mixing Ratio Profile,Retrieved CO Surface Mixing Ratio
outFieldNames = time,COprof,COsurf
logNormal = False,True,True
timeStart = 00:00:00_01-01-2005
timeStop = 23:59:59_01-01-2005
timeComparison = UTC
fillVal = -9999.0
solZenAngCutoff = 85
dayTime = True
OUTDIRECTORY = /where/you/want/output
OUTFILENAME = descriptive_name.nc
END
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
whips.py --inFromFile nameOfAboveFile.txt
2B. Process OMI level 2 DOMINO NO2 data, (Version 2)
---------------------------------------------------
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
BEGIN
DIRECTORY = /where/you/have/input/files
FILETYPE = OMI_NO2_KNMI_HDF_v2_0_postFeb2006
FILELIST = OMI-Aura_L2-OMDOMINO_2011m0901t1502-o37926_v003-2011m1012t121342.he5 OMI-Aura_L2-OMDOMINO_2011m0901t1641-o37927_v003-2011m1012t121458.he5 OMI-Aura_L2-OMDOMINO_2011m0901t1820-o37928_v003-2011m1012t121613.he5
GRIDPROJ = lcc2par
MAPFUNC = regional_intersect
VERBOSE = True
INTERACTIVE = False
OUTDIRECTORY = /where/you/want/output
OUTFILENAME = some_descriptive_name.nc
INCLUDEGRID = /where/you/want/output/grid/full/path.nc
. Proj attrs
xOrig = -48
yCell = 36
refLon = -97
refLat = 40
nCols = 30
nRows = 32
stdPar2 = 45
stdPar1 = 33
xCell = 36
earthRadius = 6370
yOrig = -552
. Output attrs
inFieldNames = Time,AveragingKernel,TroposphericVerticalColumn
outFieldNames = time,avKern,tropVCD
timeStart = 00:00:00_09-01-2011
timeStop = 23:59:59_09-01-2011
timeComparison = UTC
fillVal = -9999
cloudFractUpperCutoff = 0.3
solarZenAngUpperCutoff = 85
includePixelCount = False
END
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
whips.py --inFromFile aboveFileName.txt
3B. Process OMI level 2 NASA NO2 data (Version 1.2)
--------------------------------------------------
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
. somefilename.txt
. Input file for whips.py
BEGIN
. Flags
DIRECTORY = /where/you/have/input/files
FILETYPE = OMI_NO2_NASA_HDF_v1_2
FILELIST = OMI-Aura_L2-OMNO2_2011m0430t1440-o36120_v003-2011m0501t043317.he5 OMI-Aura_L2-OMNO2_2011m0430t1619-o36121_v003-2011m0501t061955.he5
GRIDPROJ = lcc2par
MAPFUNC = regional_intersect
OUTDIRECTORY = /where/you/want/output
OUTFILENAME = some_file_name.nc
VERBOSE = True
INTERACTIVE = False
. Parser attributes
cornerDir = /directory/where/you/keep/cornerfiles
cornerFileList =
. Grid attributes
stdPar1 = 33
stdPar2 = 45
refLat = 40
refLon = -97
xOrig = -2916000
yOrig = -2268000
xCell = 36000
yCell = 36000
nRows = 126
nCols = 162
earthRadius = 6370000
. Output function attributes
inFieldNames = ColumnAmountNO2Trop,Time
outFieldNames = tropVCD,time
timeComparison = local
timeStart = 00:00:00_04-30-2011
timeStop = 23:59:59_04-30-2011
cloudFractUpperCutoff = .3
solarZenAngUpperCutoff = 85
fillVal = -9999.0
includePixelCount = False
END
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
whips.py --inFromFile aboveFileName.txt
4B. Process MODIS level 2 NASA AOD data (Version 1)
--------------------------------------------------
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
BEGIN
DIRECTORY = /where/you/have/inputfiles
FILETYPE = MODIS_AOD_NASA_Collection_5
GRIDPROJ = lcc2par
MAPFUNC = point_in_cell
VERBOSE = True
INTERACTIVE = True
OUTDIRECTORY = /where/you/want/output
OUTFILENAME = some_descriptive_name.nc
. Proj attrs
xOrig = -2916000
yOrig = -2268000
xCell = 36000
yCell = 36000
nRows = 126
nCols = 162
refLat = 40
refLon = -97
stdPar1 = 33
stdPar2 = 45
earthRadius = 6370000
. Output attrs
inFieldNames = Corrected_Optical_Depth_Land,Effective_Optical_Depth_Average_Ocean
outFieldNames = AOD_Land,AOD_Ocean
timeStart = 00:00:00_07-03-2008
timeStop = 23:59:59_07-03-2008
timeComparison = UTC
fillVal = -9999.0
includePixelCount = True
END
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
whips.py --inFromFile aboveFileName.txt
PARAMETER DETAILS
=================
Each input attribute is explained here. Note that each output
function has a separate set of required inputs and that the
--outFuncAttrs is therefore broken down by output function. Make sure
you're referencing the details for the attributes relevant to the
function you want to use.
--help
REQUIRED: NO
DEFAULT: N/A
- Display the onscreen help message and exit the program.
--inFromFile /complete/path/to/input_file.txt
REQUIRED: NO
DEFAULT: N/A
- An input file specifying the input options and information
for WHIPS. The input file is an alternate interface -
everything that can be done with a command line call can
also be done by specifying the input file and calling the
input file using this flag.
- The input file syntax rules are as follows.
1) The file must have "BEGIN" at the top before specifying
anything else. Anything above the BEGIN line will be
ignored.
2) Comments are denoted by putting "." at the beginning of
a line. Comments are ignored when the input file is
parsed.
3) All required flags must be included. Values are assigned
using the following syntax:
FOO = bar
All flags must be in caps case. Optional flags may
be specified in this fashion as well.
4) All attributes for the projection, map and possibly
parser (right now only MOPITT requires parser attributes)
must be specified in the same syntax used for flags.
Case-sensitive, so the case of the attribute names must
exactly match that specified below
See above for both an illustrative example input file as well
as complete input files for the test cases.
--directory /path/to/input/directory
REQUIRED: NO
DEFAULT: the current working directory at time of invocation
- The input directory that the program will search for whatever
input files are specified. If those files are not found in
this directory, the programs behavior is governed by the
value of the --interactive flag
--fileList file1 [file2] [file3] ...
REQUIRED: NO
DEFAULT: The list of all files in --directory (non-recursive)
- The list of files that the program should attempt to
process for output. Most output functions (with the
exception of the function designed for MOPITT CO) are
designed to accept an arbitrary number of inputs and only
use that data which fits the requirements.
- filtering out files a priori which cannot contain the
information required, for instance files known to be on the
wrong date, saves computational power. It is advisable to
use this parameter to include only those files which may
contain the desired information wherever possible.
- Locations of data (current as of 09/21/12)
MOPITT - <http://eosweb.larc.nasa.gov/HPDOCS/datapool/>
NASA OMI - <http://mirador.gsfc.nasa.gov/>
KNMI OMI - <http://www.temis.nl/airpollution/no2.html>
MODIS AOD - <http://ladsweb.nascom.nasa.gov/data/search.html>
--filetype { HDFknmiomil2_generic, OMI_NO2_KNMI_HDF_v2_0_preFeb2006,
OMI_NO2_KNMI_v2_0_postFeb2006,
HDFnasaomil2_generic, OMI_NO2_NASA_HDF_v1_2,
HDFmopittl2_generic, MOPITT_CO_NASA_HDF_V5,
HDFmodisl2_generic, MODIS_AOD_NASA_Collection_5 }
REQUIRED: YES
DEFAULT: N/A
- The type of file we're attempting to read in. Must be one
of the options listed above.
- Filetypes have two primary roles. The first is to specify
the correct parser so that the file may be read by the
program. The second is to provide default values for the
attributes of the output function.
- Each filetype has a default output function considered most
appropriate for that filetype. Default attributes specified
will generally be targeted at this particular output function.
The default output function can always be overridden with
the --outFunc flag.
- Each parser has an associated filetype that ends in "_generic"
These filetypes still specify a default output function but do
not provide default values for any of the output function
parameters.
- Some filetypes require additional "parser parameters". These
can be passed in at the command line under the --projAttrs flag
and should be formatted just like any projection attribute
or output function attribute if using an input text file.
HDFknmiomil2_generic:
Parser: HDFknmiomil2
Default output function: OMNO2e_netCDF_avg
Description: The generic filetype for
the OMI NO2 data as processed by KNMI (the
DOMINO retrieval)
Output attributes supplied: None
Parser attributes required: None
OMI_NO2_KNMI_HDF_v2_0_preFeb2006
Parser: HDFknmiomil2
Default output function: OMNO2e_netCDF_avg
Description: The filetype for OMI NO2 as
processed by KNMI (version 2.0 of the DOMINO
retrieval). All input files should be before
Feb 2006 (when the vertical grid of the
meteorology model used in the retrieval changed).
Would probably work but has not been tested
for previous versions of the product.
Output attributes supplied:
overallQualFlag
cloudFrac
solarZenithAngle
time
longitude
pixIndXtrackAxis
outUnits
extraDimLabel
extraDimSize
Parser attributes required: None
OMI_NO2_KNMI_HDF_v2_0_postFeb2006
Parser: HDFknmiomil2
Default output function: OMNO2e_netCDF_avg
Description: The filetype for OMI NO2 as
processed by KNMI (version 2.0 of the DOMINO
retrieval). All input files should be after
Feb 2006 (when the vertical grid of the
meteorology model used in the retrieval changed).
Would probably work but has not been tested
for previous versions of the product.
Output attributes supplied:
overallQualFlag
cloudFrac
solarZenithAngle
time
longitude
pixIndXtrackAxis
outUnits
extraDimLabel
extraDimSize
Parser attributes required: None
HDFnasaomil2_generic
Parser: HDFnasaomil2
Default output function: OMNO2e_netCDF_avg
Description: The generic filetype for the OMI NO2
data as processed by NASA (the OMNO2 product).
Output attributes supplied: None
Parser attributes required:
cornerDir - the directory containing the
auxillary corner files that the
parser needs if the selected map
function makes use of them .
cornerFileList - A list of the corner files in cornerDir
that should be searched for corner files
that match the input file. Should be
comma-separated list of arbitrary length.
Set to empty string (right hand side of
equals sign or colon blank for input file
or command line, respectively) to use
all files in the cornerdir. Order does
not matter, files will be matched based
on orbit number.
OMI_NO2_NASA_HDF_v1_2
Parser: HDFnasaomil2
Default output function: OMNO2e_netCDF_avg
Description: The filetype for OMI NO2 as
processed by NASA (v1.2 of the OMNO2 product).
Would probably work but has not been tested
for previous versions of the product.
Output attributes supplied:
overallQualFlag
cloudFrac
solarZenithAngle
time
longitude
pixIndXtrackAxis
outUnits
extraDimLabel
extraDimSize
Parser attributes required:
cornerDir - the directory containing the
auxillary corner files that the
parser needs if the selected map
function makes use of them .
cornerFileList - A list of the corner files in cornerDir
that should be searched for corner files
that match the input file. Should be
comma-separated list of arbitrary length
HDFmopittl2_generic
Parser: HDFmopittl2
Default output function: unweighted_filtered_MOPITT_avg_netCDF
Description: The filetype for MOPITT CO
as processed by NASA.
Output attributes supplied: None
Parser attributes required: None
MOPITT_CO_NASA_HDF_V5
Parser: HDFmopittl2
Default output function: unweighted_filtered_MOPITT_avg_netCDF
Description: The filetype for version 5 of the
MOPITT CO retrieval as processed by NASA.
May work with previous versions of the product
but has not been tested.
Output attributes supplied:
time
longitude
solZenAng
surfTypeField
colMeasField
outUnits
dimLabels
dimSizes
Parser attributes required: None
HDFmodisl2_generic
Parser: HDFmodisl2
Default output function: MODIS_simp_avg
Description: The generic filetype for
the MODIS AOD data as processed by NASA.
Support for QA-mean averaging will be
added in a future update.
Output attributes supplied: None
Parser attributes required: None
MODIS_AOD_NASA_Collection_5
Parser: HDFmodisl2
Default output function: MODIS_simp_avg
Description: The filetype for level 2
MODIS Aerosol Optical Depth data
as processed by NASA (Collection 5)
Output attributes supplied:
time
extraDimSize
extraDimLabels
Parser attributes required: None
--gridProj {latlon, lcc2par}
REQUIRED: YES
DEFAULT: N/A
- The grid projection used to define the target grid (the grid
that we wish to regrid our data to). Must be one of the above
options. Further details on these options are as follows:
latlon - The Plate Caree projection, also known as
the "unprojected" projection. x and y are
mapped directly to longitude and latitude,
respectively.
REQUIRED PARAMETERS:
xCell - The size of a gridcell in the x
(longitude) direction. In degrees.
yCell - The size of a gridcell in the y
(latitude) direction. In degrees.
xOrig - The longitude of the lower-left
corner of the domain
yOrig - The latitude of the lower-left
corner of the domain
nRows - The number of rows in the grid.
nCols - The number of columns in the grid.
lcc2par - The Lambert Conic Conformal projection (2
parallel construction). x and y are
transformed and scaled, then mapped to
latitude and longitude via the projection.
The projection parameters must be accurate to
get the correct output grid. All required
parameters are available in the GRIDDESC file
associated with the MM3 modeling system. A
description of the GRIDDESC format can be
found at
http://www.baronams.com/products/ioapi/GRIDDESC.html
REQUIRED PARAMETERS:
stdPar1 - One of the 2 standard parallels
used to define the Lambert Conic
Conformal projection. Must be a
valid latitude, in degrees.
stdPar2 - The second standard parallel used
to define the Lambert Conic
Conformal projection. Set this
equal to the same value as stdPar1
if the single-parallel form of the
projection is being used. Must be a
valid latitude in degrees.
refLat - The reference latitude upon which
the projection is centered. This is
the YCENT value in the GRIDDESC
file. In degrees
refLon - The reference longitude upon which
the projection is centered. This
is BOTH the XCENT and PROJ_GAMMA
values in the GRIDDESC file. If
these values are not identical, do
not use this function. In degrees.
xOrig - The location of the origin in
projected x coordinates. This
is the XORIG value in the GRIDDESC
file. In same units as earthRadius.
yOrig - The location of the origin in
projected y coordinates. This
is the YORIG value in the GRIDDESC
file. In same units as earthRadius.
xCell - The x dimension of a cell, in
projected coordinates. In the same
units as earthRadius. This is the
XCELL value in the GRIDDESC file
yCell - The y dimension of a cell, in
projected coordinates. In the same
units as earthRadius. This is the
YCELL value in the GRIDDESC file
nRows - The number of rows in the grid.
nCols - The number of columns in the grid.
earthRadius - The assumed radius of the Earth
(assumed spherical). Must match
units used for xCell and yCell.
--projAttrs name1:value1 name2:value2 ...
REQUIRED: YES
DEFAULT: N/A
- The attributes required for the chosen projection. Must
have all the required attributes for that projection. The
required parameters for each projection are listed under
that projection's name above.
- Case-sensitive. Attribute names must EXACTLY match those
laid out above.
--mapFunc {point_in_cell, regional_intersect, global_intersect}
REQUIRED: YES
DEFAULT: N/A
- The mapping function that will be used to assign pixels to
cell(s). Certain datasets have restrictions on which
mapping functions are usable.
- Also responsible for computing the "geometric weight" of
pixels. That is, this function computes the weight unique
to a cell/pixel combination. At present, neither of the
functions provide this functionality.
point_in_cell - Maps pixels defined by a single
lat/lon (usually the cell center) to whatever
grid cell that point lies inside in projected
space. This assigns each pixel to a unique
grid cell. Grid cells are open on the top and
right sides and closed on the left and lower
sides (with directions defined according to
the projected coordinate system). This
function is supported by all currently
available input filetypes. NOTE: for
filetypes where regional_intersect is
available it is strongly recommended to use
regional_intersect over point_in_cell.
regional_intersect - Maps pixels (as defined by
pairs of geocoordinates that nominally
correspond to pixel corners) to ALL gridcells
intersected. No geometric weights are
calculated. This function is currently
supported by HDFnasaomil2 and HDFknmiomil2
filetypes only.
NOTE: regional intersect should NOT be
used for grids that wrap all the way
around the world longitudinally (IE the
east and west edges are the same meridian).
Such grids may give unexpected results,
especially if any pixels span the
discontinuity.
Pixels with one ore more vertices with
fill values coordinates are rejected
Makes several assumptions:
- Polar discontinuities not encountered
- Projection is NOT global
- grid is rectilinear in projected space.
- Pixels are convex polygons.
global_intersect - Maps pixels (as defined by
n pairs of geocoordinates that nominally
correspond to the n corners of the
pixel) to ALL gridcells intersected.
No geometric weights are calculated.
This is roughly the same as
the regional_intersect approach
save that it is slightly slower and
assumes that the grid wraps around the
world longitudinally.
Pixels with one ore more vertices with
fill values coordinates are rejected
NOTE: using this function with
grids where the east and west boundary
are NOT the same meridian is not
supported and may lead to unexpected
results.
Makes several assumptions:
- polar discontinuities not encountered
- Projection is global and cyclizes
along the east and west edges
- grid is rectilinear in projected space
- Pixels are convex polygons
--outFunc {OMNO2e_netCDF_avg,unweighted_filtered_MOPITT_avg_netCDF,
MODIS_simp_avg_netCDF}
REQUIRED: NO
DEFAULT: Depends on the value chose for --filetype
- The function that computes the output and writes the output
file. Functions are given significant freedom, but all
current functions take some kind of average and write it to
an output file.
- These functions are frequently designed around a particular
instrument or input format. Efforts are made to make them
as general as possible, but specialized output functions are
only guaranteed (and really should only be used) for the
parser types for which they have been designed. To this
end, the associated function will be chosen by default for
all filetypes, though it can always be overridden with this
command.
- See the documentaiton for the --filetypes flag to see what
the default output function for each filetype is.
- In all cases where a fieldname must be given for a
parameter it is the short name (the name used to access the
field through the parser) that must be given. The
fieldnames must correspond to the official field names
in the data file. These can usually be found in the data
documentation. Documentation for a few commonly processed
formats can be found at:
OMI (KNMI) - <http://www.temis.nl/docs/OMI_NO2_HE5_1.0.2.pdf>
OMI (NASA) - <http://toms.gsfc.nasa.gov/omi/no2/OMNO2_data_product_specification.pdf>
MOPITT - <http://www.acd.ucar.edu/mopitt/v5_users_guide_beta.pdf>
OMNO2e_netCDF_avg - Averaging algorithm based on the
NASA OMI level 2 to level 3 processing
algorithm. Designed for the OMI level 2
filetypes (HDFnasaomil2 and HDFknmiomil2) and
is the default for those filetypes use with
other filetypes is of questionable utility.
Outputs results to a netCDF file.
Further details available in the
official NASA documentation located at
<http://disc.sci.gsfc.nasa.gov/Aura/data-holdings/OMI/omno2e_v003.shtml>
Assumptions:
- Data is at most 3 dimensional.
- Invalid pixels are marked with an overall
quality flag.
- Timestamps are in the TAI93 format.
unweighted_filtered_MOPITT_avg_netCDF - Averaging
algorithm based on the NASA algorithm for
processing level 2 MOPITT CO data to level 3
MOPITT CO data. Designed for the MOPITT level
2 filetypes (HDFmopittl2 only at present) and
is the default for that filetype. Use with
other filetypes is discouraged.
Outputs results to a netCDF file.
IMPORTANT: Only 1 input file may be used with
this function. Conveniently, NASA currently
provides data in 1-day granules.
Further details available in the official NASA
documentation:
Deeter, Merritt N (2009). MOPITT (Measurements
of Pollution in the Troposphere) Validated
Version 4 Product Users Guide. Available from
<http://www.acd.ucar.edu/mopitt/products.shtml>
Assumptions/caveats:
- All fields are filtered based on the
number of valid layers present in the
specified column field. Including 2D
fields.
- Timestamps are in the TAI93 format.
MODIS_simp_avg_netCDF - Averaging
algorithm based on the NASA algorithm for
processing level 2 MODIS AOD data to level 3
MODIS AOD data. Designed for the MODIS level
2 filetypes (HDFmodisl2 and Collection 5) and
is the default for those filetypes. Use with
other filetypes is discouraged.
Outputs results to a netCDF file.
Further details available in the official NASA
documentation:
Hubanks, et al. (2008) MODIS Atmosphere L3 Gridded
Product Algorithm Theoretical Basis Document.
Available from
<http://modis-atmos.gsfc.nasa.gov/_docs/L3_ATBD_2008_12-04.pdf>
Assumptions/caveats:
- This averaging method does not factor in
QA weights. A more generalized QA-weighted MODIS
output function will be added in a future release.
- Timestamps are in the TAI93 format.
--outFuncAttrs name1:value1 name2:value2
REQUIRED: YES
DEFAULT: N/A
- The attributes required for the chosen output function
specified somehow. Can be either supplied by the filetype
or specified by the user. When the user and filetype both
supply output function attributes, the filetype takes
precedence. The "_generic" filetypes are supplied to
allow complete control of all output function attributes
if desired.
- To see which attributes are supplied by which filetypes,
see the documentation for the --filetype flag above
- If one of the "value" elements contains whitespace, enclose
the entire name:value pair in double quotes. For example: