|
I'm wondering why grib files, such as those available on the FSL
server, aren't compressed? For years I have "known" that grib files
involved data compression and so had assumed that any further
compression would not be meaningful - but I finally experimented with
a simple unix "zip" compress (ala PKZIP on Windows machines) and lo
and behold it gave 40% compression of all my FSL 3D native grid grib
files. It seems that considerable bandwidth (not to mention storage
space) might be obtained by adding such a compression step. Certainly
other large files available on the Internet are generally available in
a compressed format for just that reason - and "zip" format
decompressers (for example) are almost universally available and
anyone who is getting a file from a server can be expected to have the
knowledge needed to do a decompression. I suppose server CPU time
needed for compression could be a factor, but that time seems pretty
modest [on my Athlon-processor Linux PC compression of a 55MB 3D
native grid FSL file took 8 seconds - and decompression took 3
seconds].
A recent posting here mentioned increased network traffic and server
loads at NCEP so I guess the same question could be asked of them,
since (for example) I found a "zip" compression of one of the ETA 3D
grib files decreased it's size by 68% ! So it's not like FSL is
unique in the meteorological community here, but I confess to not
understanding the rationale since in many other respects meteorology
has been highly aware of bandwidth issues (witness the abbreviation
jargon still used in NWS text forecasts, hearkening back to the old
teletype days). A nearly halved file size would seem to be very
beneficial to everyone.
|