Change Details - New Hourly Model Ceiling Statistics

31-May-2018 (beta): Set the default region to "E_US" (for which most models have data), and added the HRRR-relevent regions: 'ALL_HRRR', 'E_HRRR', and 'W_HRRR'.
20-Dec-2017 (beta) Enabled Dieoff plots. Moved code over from the latest visibility plots, including the (correct) statistics with a suffix '1', along with the old incorrect statistics. (See the 14-Dec-2017 change discussion for Visibility statistics.) Routine(s) changed from their versions in the beta visibility stats page): CurveParameters, DisplayFrame, Welcome.cgi, get_ceil_stats.cgi, make_station_summary.pl.
8-Dec-2017 (beta): The capability was added to insert a comma-separated list of METAR 4-letter names as the 'region'. Simply highlight the region and start typing the names (case insensitive). Also, a new statistic was added: NFlow = the number of metars for which the model forecasted a low (i.e., below the threshold) ceiling. (On the other hand, Nlow is the number of metars reporting low ceilings.) Routine(s) changed: get_ceil_stats.cgi, make_station_summary.pl (new).
28-Nov-2017 (production and beta): Queries for PODy/n and FAR were incorrect and have been fixed. Routine(s) changed: get_ceil_stats.cgi.
18-July-2017 (beta): Dieoff curve plotting has been added to the beta. Routine(s) changed: DisplayFrame, CurveParameters, Welcome.cgi, getModels.cgi (new).
9-March-2017 (production and beta, on new server): The java code calls getModels.cgi to get the list of available models. Also, the java code calculates default start and end dates. Routine(s) changed: StatsDisplay, CurveParameters, Welcome.cgi, getModels.cgi (new).
31-Aug-2016 (beta and production) Corrected an error for the RRrapx130 grid that was using the 'cloud base' field instead of the 'cloud ceiling' field. Eliminated processing for the RRrapx221 grid. Routine(s) changed: jet:~amb-verif/ceiling/ceil_driverHRRR.pl and jet:~amb-verif/ceiling/make_grib2_ciel_tmp_file.pl.
25-Aug-2016 (beta and production): Corrected an error that caused multiple METAR reports within the same hour to be matched with the model. Only the METAR report nearest the model valid time within each hour (valid time +/- 30 min) was supposed to matched with the model. This shouldn't change the statistics very much, but the correction does lower Ntot and Nlow. Ntot should now correctly match the number of METARs within the region that reported during the hour centered on the model valid time.
22-Aug-2016 (beta and production): Found an error in the ceiling calculation algorithm that's been there since the start. We had been taking the altitude (AGL) of the lowest reported level as the ceiling, even if the lowest level was SCT or FEW. We now take the lowest level coded as BKN, OVC, or VV. This was corrected starting 20-Aug-2016.
26-Aug-2015 (beta and production)

All beta changes moved to production.
Background horizontal lines changed from black to lightGray. Routine(s) changed: DisplayFrame.java.
16-July-2015 (beta): Added options to look at the verification statistics out on 3-hourly intervals out to 30 hours. Done in response to the RAP and HRRR forecast lengths being extended. Routine(s) changed: CurveParameters.
16-July-2014 (beta): Fixed to not put out data from incomplete averaging intervals. If there are gaps at either end of the averaging interval that together exceed 10% of the total averaging interval, the datum for that interval is set to 'missing'. Routine changed: get_ceil_stats.cgi
3-May-2012 20Z (beta and oper)

removed bad NAM results found earlier today. This creates a gap in results for non-RUC regions from 12Z 26-May-2011 through 18Z 2 May 2012.
(beta only) We now draw lines through data points that are separated by the minimum time interval between data on each curve. So, we can compare hourly RAP forecasts with 6-hourly NAM forecasts, and each will be a curve, not isolated data points.
Routine changed: get_ceil_stats.cgi
3-May-2012 (beta and oper): discovered a problem with NAM results for all regions other than RUC. All NAM regions were filled with data from the RUC regions. Fixed starting 0Z today. Check the Ntot variable to see if NAM has too many METARs for a given region. Need to eliminate the bad data.
1-May-2012 (beta) Added NAMceil -- NAM model, but using their special HGT level = 'cloud ceiling' field. New routines: NAMceil_driver.pl, fill_NAMceil_values.c, NAMceil_driver.c. Routine(s) changed: update_summaries.pl, get_iso_file2.pl
26-Jan-2012 (beta and production) Added RRrapx130 and RRrapx221 models. Changed to read HRRR files in wgrib2 (5x to 10x speedup!) Changed to read RR files in Rotated LatLon in wgrib2 (big speedup) Changed to use wgrib2/1.9.2c on jet, which correctly reads dx and dy. Routine(s) changed: find_wgrib2.sh (new), make_grib2_ciel_tmp_file.pl (new), get_grid.pl (new), ceil_driverHRRR.pl, ceil_driverRR.pl, get_iso_file3.pl, update_summaries3.pl, fill_ceil_values3.c, ceil_driver3.c, fill_rotLL_ceil_values3.c, ceil_rotLL_driver3.c.
25-Jan-2012 (beta and production): Made a fix to account for a change in the 'missing' value for ceilings (i.e., clear sky). Previously, this was negative. Now its some big number. So, ceilings > 1e10 are now coded as 60,000 ft, the same way negative ceilings were previously treated. Routine(s) changed: fill_rotLL_ceil_values3.c.
5-Jan-2012 (beta): The drop-down list for years now updates automatically to include the current year +1. Routine(s) changed: CurvesControl, lib.JYearBox.
26-Sep-2011 (beta) Added RR1h_dev2_prs. Routine(s) changed: ceil_driverRR.pl, get_iso_file3.pl. The list of models provided on the web interface is now populated by looking at the ceiling_sums database. Routine(s) changed: Welcome.cgi.
26-May-2011 (beta): We updated the processing for models: NAM dev1320 Op13 Op20 dev13 Bak13. Previously, we tried to process only one hour each hour; this caused occasional gaps when the db was too busy. We now process the 'current' hour (generally 5 hrs ago), then 1,3, and 8 hours previous to that. We also note if an hour/fcst has already been processed, so we don't re-process it. We also provide a second change to load obs into the ceiling db. To do this we added an entry to cron2's cron for 'get_obs.pl -3 wolphin'Routine(s) changed: update_summaries.pl, ceil_driver2.pl.
12-May-2011 (beta): We now allow a 'region' consisting of single METAR site id's (or several separated by commas). But this only works for 'unmatched' curves, and for the 'PODy' statistic. Routine(s) changed: CurveParameters.java, get_ceil_stats.cgi.
19-Oct-2010 (later, beta only): Now use a little 'x' in the upper right of each curve specifier to allow it to be removed. We no longer have a 'remove' button that removes the last curve. So, you can now remove any curve. Routine(s) changed: CurvesControl, DisplayFrame, CurveParameters, SimpleLine2, get_ceil_stats.cgi.
19-Oct-2010 (production and beta)

All beta changes moved to production.
Difference curves made optional.
Problem Java 6 update 22 causes Jbuttons that use html text to become blank, so I removed the html text on the buttons.
Routine(s) changed: CurvesControl, get_ceil_stats.cgi, Welcome.cgi (to use archive ceiling6.jar)
21-Oct-2009 (beta): Statistics, including standard error, added to the text window. Routine(s) changed: DisplayFrame.
20-Aug-2009 (production and beta)

All beta changes moved to production.
North American Model (NAM) added.
If more than 50% of the points on a line are missing, the line is plotted with symbols and lines, instead of just lines. (This was done to more easily display models that only produce forecasts every few hours, as opposed to every hour.) The way each line is plotted can (as always) can be changed by right-clicking on the line's legend. Routine(s) changed: DisplayFrame, SimpleLine2.
17-Aug-2009 (beta): Text window option added, that shows for each data point (averaged or not) the date, the value plotted, the number of hours in the average (Nhrs), the number of METAR sites below the threshold (Nlow), and the total number of METARs (Ntot). Routine(s) changed: get_ceil_stats.cgi, StatsDisplay, CurvesControl, DisplayFrame.
13-Aug-2009 (production and beta) Error corrected that caused the min and max dates in 'plot unmatched' to depend on the order of curves. N_in_avg added as a statistic. Date selection window now goes back to 2006, to include retro run periods. Routine(s) changed: DisplayFrame, CurveParameters, CurvesControl.
5-Dec-2008 (production and beta) All beta changes moved to production (ceiling3.jar) model Bak13 (on 13 km grid) from wjet added model dev13 (on 13km grid) added "model" persistence added statistic Bias added statistic FAR added statistic CSI added statistic HSS added The 'plot matching' function was adjusted to not requrie the same number of metar reports in compared models for each hour. This makes it easier to match persistence with other models (because persistence requires that a metar station report at exactly the same time n hours ago, and this doesn't always apply). It also makes it easier to match models on different grids, because models on 20km grids have a few (about 4) more metars than those on 13km grids. This is because at the edges a metar will be matched if it is within 10km of a grid point for a 20km model, but the METAR must be within 6.5km of a grid point for a 13km model. Routine(s) changed: get_ceil_stats.cgi, CurveParameters, routines that load models into the database
2-Dec-2008 (beta): Updated to get a list of models from the calling web page. Also, got rid of an annoying java bean info call in SGLabel that may have slowed down the appearence of plots. Routine(s) changed: Welcome.cgi, CurveParameters, StatsDisplay, CurvesControl, gov.noaa.pmel.sgt.SGLabel
25-Nov-2008: First version. This uses data from the 'ceiling' database on wolphin.

Prepared by Bill Moninger, Bill.Moninger@noaa.gov
Last modified: Thu May 31 21:09:15 UTC 2018