BINLOGs

A binlog contains the QUALITY and QUAL_VLD keywords for each "observation" for the entire mission

Binlogs are fits files.

There are two classes of binlogs defined - observable and data series.

Observable BINLOGs

There is potentially a binlog for each MDI observable, e.g. fd_V.
In practice they will be generated only for high-rate data.

These binlogs contain for each minute (or 30 seconds if this is ever appropriate for the observable) the QUALITY and QUAL_VLD keywords from the level 1 log files for a particular observable. These will be keyed to T_OBS.

At level 1 there is no question of merging, so the binlog values can be maintained/updated whenever the level 1 QUALITY and QUAL_VLD keywords are changed. A record of which observables depend on which DPCs will be consulted to remake the binlog values from the level 1 log files.

Data Series BINLOGs

When various data sources are merged at level 1.5 the question of which QUALITY to propagate into the binlog depends on the results of the gathers performed to create and perhaps recreate the level 1.5 values. This class of binlog is more appropriately associated with a data series than with observable (or DPC), e.g. fd_M_96m.

When the QUALITY and QUAL_VLD keywords in the level 1.5 log files agree with the values in the level 1 source log files, then the values from the level 1.5 log files can be used. Otherwise the level 1.5 gathers will have to be rerun before making the binlog.

The Data Series Binlogs must be updated any time a new level 1.5 data set is created. The reduction that creates the new level 1.5 data set ((gather_15qual?) should be responsible for updating the Data Series Binlogs (or seeing that they are).

There should be an entry in the binlog for every possibl3 time in the data series. The time grid should be T_REC, e.g. every 96 minutes for fd_M_96m.

Other Requirements & Issues

Changing QUALITY bits has various ramifications.

We require a way to ensure that level 1 modifications are registered and that reconstruction of the observable binlog files takes place whenever the keywords in the level 1 log files are changed. Downstream effects on level 1.5 are also expected.

This probably means that prob_upd will need to send some kind of notification when PROBLEM bits are changed. Changing PROBLEM bits (or DATA_SW bits if some other processing in redone) will generally require level 1.5 processing to be repeated for all data sets depending on the level 1 sources, because new QUALITY bits may result in different selection of source data. [It should be possible to track when downstream data are likely to be affected by looking at bit 1 in QUALITY.]

In addition the level 1.5 logs and binlogs will need to be regenerated after the new level 1.5s are ready. The new source data will have to be reevaluated as well.

The issue of locking files hasn't been addressed.

Names are needed for the binlogs. As is some way of making sure they are secure, reconstructable, and that they maintain integrity.

Back to QUALITY.
Author JTH; Last modified by KRL on 4 Jun, 1998