EBZ June 1998 Not yet approved by RogerC TELEMETRY HANDLING ------------------ REAL-TIME ACQUISITION OF DATA: There are 4 sockets between mdi_acq and the ecs computers. These sockets are defined by the file /mdisw/egse_v5.2/dbs/TM_CONFIG.MDI. VC0 housekeeping data, spacecraft data VC1 low rate science data (5k) VC2 high rate data (160k) CMD commanding socket Raw data comes in CCSDS packets, which are organized bytes of data with headers that include the length, packet number, sequence number, and time. If we see an error message about the sequence number being off, it is because we lost a packet (or several packets depending on the sequence number that we got versus what was expected). Packets are small - it takes many packets to produce an image. Bad packets are taken out and put into a separate file. Data packets come in and a copy of the data is made - one copy is temporarily stored in a buffer and one copy is written to disk. One copy of the data is stored in a shared memory circular buffer whose length is set by environment variables. The buffer has several sections, one for each type of data (hk,s/c,5k,160k). The latest packets are always saved in this buffer. This buffer is used for updating the housekeeping screens. The other copy of the data is written to disk. Low rate data is written to slotted files and high rate data is written to non-slotted files. Low rate data is slotted in a file according to time. There is one file per data type per day. A template file of the correct length for each type of data. These templates are copied at the turn-over for a new day and the new template becomes a decom file and current data is written to it. These decom files are the files that are on the General Telemetry Page and must be open for the data to be saved. The slotted files are in /md80/log. High rate data is written to a file in the order that the data is received, but it is not slotted by time since we get so little high rate data each day. The data is written to a file until the file is full. The file is a .tfr file and an index file called a .tfrx file is made and tells where in the .tfr file specific data is. ACQUISITION OF DATA THAT DOESN'T COME IN REAL_TIME: Real time data that come over the telemetry sockets are handled by the mdi_acq process. If the data doesn't get to the egse, the .QKL and .REL files (high rate gets only .QKL files) are ftp'ed to mdisas from the ECS computer. The high rate quick-look data come to us in an unusable format so IDL>remake_tfr should be run manually on the data to make them into .tfr files and also to create the .tfrx file. The mdi_acq process receives the telemetry packets and does two other things with the data besides slotting it; it updates the housekeeping screens and it distributes the telemetry packets to some of the IDL programs that generate the EOF web pages. The low rate data ftp'ed by ECS computer are handled by the tm_mgr. TM_MGR: - .REL and .QKL file handling: Normally, .REL and .QKL files are ftp'ed into mdisas by the ecs computer. They live in /md80/ecs/from_ecs/rel and /md80/ecs/from_ecs/qkl. Occasionally, an ftp process fails, in which case human intervention is required to copy the file into the appropriate directory on mdisas. The tm_mgr is NOT a daemon; rather, it is invoked by another process whenever files are ready for its ingestion. When the tm_mgr ingests a .REL or .QKL file, it leaves a log entry in /md80/tmp/tm_log describing its actions as it parses the file. The tm_mgr is invoked indirectly by cron. Every twenty minutes, the mdi crontab on mdisas runs /mdisw/bin/dinnerbell, which polls the incoming ecs directories and invokes the tm_mgr on each incoming file. tm_mgr is run on new files that have not been touched in at least five minutes, to allow the ftp process time to complete. When tm_mgr finishes a file in /mdisw/ecs/from_ecs/(rel|qkl), it transfers the raw file to /mdisw/ecs/done_ecs. Dinnerbell uses some subsidiary jobs to handle its polling. The program /mdisw/bin/pavlov is a general purpose file queue handler. Pavlov keeps track of which files in /mdisw/ecs/from_ecs/(rel|qkl) have had processing attempted on them; and invokes the tm_mgr on the interesting files. The tm_mgr is invoked only once per file per modification of the file; so if a file causes a tm_mgr error, it will sit harmlessly in the incoming directory (and not be processed again and again). Pavlov, in turn, uses /mdisw/bin/bailey to keep track of the contents of the directories it examines. - .REL and .QKL File Processing Lore: Because a given TM packet can (nominally) arrive three separate times -- in realtime, in .QKL, and in .REL -- slots in the slotted TM files may be overwritten. In case of conflict, the mdi_acq and tm_mgr processes must decide whether to overwrite old data with the newer stuff. Historically, issues surrounding overwriting of data have been central to several problems with the tm_mgr. TM Raw files are named after the time at which they nominally begin. This means that raw files whose time interval includes midnight contin data for two separate slotted files. The directories /md80/log/* are all mirrored with Mons's mdisw system. Hence, avoid making extra or temporary files in them (or the extra files will be shipped around the country). If there is a problem with tm_mgr, it can probably be found in the /md80/tmp/tm_log/* files -- they contain a complete record of the tm_mgr's actions for the last month or so. - Data Reprocessing Procedure: If a slotted file needs to be regenerated, you should: * Move the old slotted files from /md80/log/* to somewhere else (not in the /md80/log tree) for safekeeping. * Drop all the old .REL and/or .QKL files for the target day, and the LAST .REL and/or .QKL file from the day before, into the appropriate /md80/ecs/from_ecs/(rel|qkl) directory. * Invoke dinnerbell as mdi, or wait for cron to do the same.