GCP Data Retrieval

Web-Based

Talk about how to use extract page.

Server-Based

First option: If you are interested in retrieving whole days of second-by-second data, and you don't need up-to-the-minute data, you can simply use the existing compressed CSV files, which may be imported into any number of programs or processed through scripts directly on the server.

To send one of these files to the console or another program without creating a decompressed copy somewhere, this is as easy as:

	gzip -dc /usr1/eggsummary/1998/basketdata-1998-08-05.csv.gz
Obviously you can substitute a date of your own choosing.

Second option: If you want less than a day, more than a day, or very recent data, you can get the same CSV output from the basketran program. This program also gives you an option to include the parsed date, or to output a brief form ("-B") containing only the raw 'unix' time (seconds since Jan 1, 1970). Two examples are shown here:

	basketran -ES2005-03-01-12:00:00 -EE2005-03-01-12:00:05 /usr0/rawdata/2005-03/2005-03-01-noosphere
	basketran -B -ES2005-03-01-00:00:00 -EE2005-03-05-23:59:59 /usr0/rawdata/2005-03/2005-03-*
Note that these operations (particularly for large amounts of time and large numbers of input files) can take rather a long time (i.e. several minutes for the second example given above).

Author: [email protected] (Last updated 2005-04-01)