AaPd      HH $ d igaH7Kff@dFootnote TableFootnote**. . / - :;,.!?  ]0*% ^;WhTOCdHeading",==   EquationVariablese    = = M. > Ti > ?es  ?ime @M. @ca F AelvG Ad. Belv0 BTi" E$ E% Fn' F( G* G U e Z U=  V>  W? X@  YA  ZB  ]E  ^F  _G  <$lastpagenum><$monthname> <$daynum>, <$year>"<$monthnum>/<$daynum>/<$shortyear> ;<$monthname> <$daynum>, <$year> <$hour>:<$minute00> <$ampm>d "<$monthnum>/<$daynum>/<$shortyear> <$monthname> <$daynum>, <$year> "<$monthnum>/<$daynum>/<$shortyear>  <$fullfilename>  <$filename>  <$paratext[Title]>Fo  <$paratext[Heading]>  <$curpagenum>   <$marker1>!  <$marker2>0  (Continued) Heading & Page<$paratext> on page <$pagenum> Pagepage <$pagenum> See Heading & Page%See <$paratext> on page<$pagenum>.  Table & Page'Table<$paranumonly> on page<$pagenum> + (Sheet <$tblsheetnum> of <$tblsheetcount>)  Figure & Page (Figure<$paranumonly> on page<$pagenum>A Section & Page%Section<$paranum> on page<$pagenum>E Equation Number* Heading<$paranumonly> l ~AFl l thne>l $mohnl ;l Aol "Aol >l $ye>l qq/<$TOCyeal ss LOF$fil uutleLOT l xx IXl >!A Y ?1 tin, e- _$. `g* s1 upB e\ AAe<2  3 4 '5 >6  7 b8 |s9 |: | ; |n< |= |> |e? |a@ |pA |qB |C |rD |nE |lF |G |H |I |J |K 9L M >N O oP R S T U V W AoX Y Z ![ "Ae\ #J] $A$ea_ &As` 'A fiJ (:a )ALOb *c +Jd ,Je -J!f .J?g /J ,h 0V_i 1V.gj 2Vsk 3JpBl 4Jm 5Jn 6? 2 3o 7p 8@'q 9@r :@6s ;t <u =@sv >w ?@ x @Dny Az B{ CJe| DVa} EJp~ FJq GJ HJr IJn JJl KJ L Ma Na Oa Pa Qa R> S To U V?3  WT XU YV ZW [J \> ]H ^J _> `H aJ$ b>A cH` d^i e> fH g> hH i> jHJ k>fK l;? m n?4 i o ps q?J5 B r s2 t uoL v=p wq xr |Js }Jt ~Ju v w ?x6  Z [e \a dp eq f gr hn il j k l m n omm9d3no H:_3N `@4ozn H:_3N `@H H Footnotedpqv>H~6qrp >H~6 >UU` "<$paranum><$paratext><$pagenum> ` "<$paranum><$paratext><$pagenum> )US UT` "<$paranum><$paratext><$pagenum> HnO `@rqsp HnO `@HwjHwjTable of Contents SpecificationLH-srtp H- UU` "<$paranum><$paratext><$pagenum> H `@tsup eH `@HEHEList of Figures SpecificationH6utvp lH6 UU` "<$paranum><$paratext><$pagenum> HS `@vup HS `@HEHEList of Tables SpecificationdwxyH~xyw H~ ` 1, 23 ` $<$symbols><$numerics><$alphabetics> $p` > Level3IX &` m> Level2IX >0` UT Level1IX C` atLSymbols[\ ];Numerics[0];A;B;C;D;E;F;G;H;I;J;K;L;M;N;O;P;Q;R;S;T;U;V;W;X;Y;Z L` Ta <$pagenum> ts Hl@mV `@yxw Hl@mV `@HuHuIndex SpecificationHE:_@ `@5zo{n tHE:_@ `@HMHM Single LinepH'%6{znifFootnoted}~1cnQ\(~} a1cnQ\( `  Hhnyh~} oHhnyh~ d HH HH &1` (Use of DataSet Names u6 ` lcsPhilip H. Scherrer > X UU` vSOI-TN-112 - 31 March 1993 w` l1-Notation for formal syntax of dataset names: 0` I;#Name parts are shown in italics. L` <$BCharacters (letters or punctuation) not in italics are literal. ` 4The symbol "::=" should be read as "consists of". ` %Brackets {} denote optional parts. e` 'The symbol "<>" means concatenation. ` no A "|" should be read as "or". `  ` 1Jdatacollection_name ::= dataset_name where the member_numbers are ranges. ` 0dataset_name ::={dataseries_name,}data_selector ` Gdataseries_name ::= program_selector{,class_selector},series_selector ` ,program_selector ::= prog:member_selector ` 5member_selector ::= member_name{[member_number]} -` ammember_name ::= name h;` > .name ::= alphabetic | name<>alphanumeric I`  member_number ::= number xW`  0&number ::= digit | number<>digit e` Aclass_selector ::= {class_selector,}class_name:member_selector is` . class_name ::= name he ` d -series_selector ::= series:member_selector ` no4data_selector ::= {sel:}var_list{[slice_selector]} "` atvar_list ::= name{,var_list} "|` s 0slice_selector ::= record_range{,array_slice} 1` amrecord_range ::= range ` e range ::= number{-number} ` as,array_slice ::= axis_range{,array_slice} ` axis_range ::= range s` leNotes. & rA dataset_name consists of a data_selector optionally preceded by a dataseries_name. The %  sdata_selector must contain at least a var_name. If a dataseries_name is present it will always 3@ r start with "prog:". 0M dipThe basic unit of data that is handled by the SOI DSDS is a dataseries with a single value for [ laseach dataseries_name member_number. While a datacollection may then be specified as a set elei@ _lVof DSDS dataseries, the chunk that is processed at a time is a single series.  icgWhile any of the member_numbers in the datacollection_name may be ranges which imply a ` d _sceHH HH & fset of dataseries is the basic unit that is handled by the DSDS and SDS_open and may not have @ am4more than one member number value for each series. st0 a jA dataset may be an entire dataseries or it may be a subset of the series by specifying a >@  0slice_selector. ThaUT UT` taUSAGE { `The formal name specification for SOI data product names does not give sufficient understanding as m\of the intended usage to allow either real use or development of the system. Therefore, the @ SDYfollowing description is intended to show the intent of the way to use this name format.   t^The purpose of the SOI naming format is to enable identification of chunks of SOI and related  adata independent of the particular storage location (disk, tape, etc.) or the particular storage @ protocol (FITS, CDF, etc.).  dCertain rules of usage and limitations are designed to make the actual use of SOI data as practical  n ]as possible while bounding the problem of the DSDS and the pipeline and exploratory analysis n@ s>programs in handling the expected very large data collection.  e_WOne of the goals is to enable the development of program modules that can be used in a cat+ ro`production pipeline environment, in stand-alone analysis programs written to the same standards a9 se^as the pipeline programs, and in GUI analysis environments such as SGI Explorer, Khoros, IDL, G e aor a TAE driven analysis front-end. Since these analysis environments provide their own view of mU en[the world and each have differing expectations of the ability of computing modules to gain larc (d[knowledge of their surroundings, we have decided to build SOI SSC analysis programs in two q fparts or levels. Actually, we have divided one of these levels into further divisions to (hopefully)  n ^enhance program maintenance capabilities but that division is hidden for the purposes of this @ s discussion. ha d aWe call these two levels the interface level and the strategy level. The interface level is p t ^considered to be a wrapper around the strategy module that handles ALL connections with the  n `program's environment. The goal is to produce a single interface level linkable module that is nm xp^appropriate to make a unix "exec"able program suitable for each analysis environment needed.  roaThe particular environment connections that we will shield from the strategy modules include the  ngYDSDS catalog database, staging SOI datasets by the DSDS, command line parameters, stderr t@ alDerror logging, stdout message printing, program return values, etc. ly oneThis isolation is accomplished by defining a fixed calling interface for all strategy modules. This c#  tZcalling interface is similar to the main program calling interface in unix which provides 1 al]connections with the command line and the programs environment variables. Since the various ? coawrappers we will need will in general not want strategy modules to send output to the pre-opened M raai/o streams, but will want to do particular handling of communications with the user under their [ op[own control, we add some message i/o like functions in addition to handling command line i@ pacontrol parameters. on ileStrategy modules are where the action is. It is the strategy modules that actually are written with Id HH gHH (  on`specific knowledge of the particular problem at hand. The interface level has simply performed  inVsome services for the strategy modules that provide a clean connection with a virtual $@ ec environment. c>` !e \In particular, strategy modules communicate with the interface level in the following ways: d X t XThe strategy module code includes pre-defined lists of expected control or command line bf  pZarguments. These arguments are provided as initializations of an external scope array of t  m[structures. Each member of the array describes one expected control parameter. the strattro Zegy module specifies (at compile time) the key name that will be used for that parameter,  teZthe default value to use if the parameter is not specified (or a flag indicating that the  Xparameter must be specified before starting the strategy level module), and probably we k rtXwill also supply max and min appropriate values and the type of parameter so that a GUI so@ e Dcan do some first level validity checking (or radio-dial building).  "roWThe strategy module includes the definition of an external pointer named DOIT which is ace "ow^initialized to point to the strategy module itself. The interface module will use that exter@ "nd2nal scope variable to invoke the strategy module.  $ a\The interface module will provide a KEYlist which is a linked list of structures containing o $beYkey_name:value pairs. It will examine the list of expected arguments, the command line, (@ $heYshell environment, possibly files in $HOME, and possibly files in other standard places. u( r WThe interface module will provide an empty KEYlist for return values from the strategy pec6@ in9module that will be reported to the calling environment. H &o ]The interface module will provide a function which takes a string and puts it into a history V@ &fi"log appropriate to the interface. h '\The interface module will provide a function which takes a string and puts it into an error Dv '[log appropriate to the interface. These string functions will take the same kinds of arguill ']ments as printf. In the shell callable interface module in cases where printf is a function n 'l Xrather than a macro, the history function will probably just be printf. These functions ke '. [could actually be provided via external function pointers just like the strategy module is ll @ 'blSprovided to the interface module. That might actually be a more extensible model. WTh )YThe interface module may later provide other services. To allow graceful upgrading of o )reZinterface modules and strategy modules in a working system, the strategy module will also  )akVprovide an external variable called STRAT_VERSION (or something like that) which the @ )Ointerface module can use to determine the expectations of the strategy module. uts` * DLThe expected flow of logic in the interface module would be something like: s ` + k>1. Build empty KEY lists for control and return parameters. .` ,rf@2. Process the argument list provided by the strategy module. l >` -crFa. If simple numeric type, get it from the caller, maybe check it, N` .. -and add it or its default to the KEYlist. i^` /ikBb. If a DATASET type, the interface module must do a bit more. n 0haO1.Call the dataset name parse routine which splits a dataset name into its ce |@ 0rocomponent parts. de duHH mHH ! ) 1xtN2.Use the prog member_name and the environment to determine the directory @ 1rf'naming rule for datasets in that prog. tat&` 2gyP3.Put the dataset name and the directory template or rule into the KEYlist. le6 3g ]3.Call the strategy module (which returns a status) with the KEYlists and string reporting D@ 32 functions. e aT` 4dea4. On return from the strategy module, check status and report failure to calling environment. ld` 5, Y5.Examine the return KEYlist and report any values present to the calling environment. UT UT` 6yp TEMPLATE r 7o kA dataseries is a collection (one or more) files that is handled as a unit by the DSDS. The DSDS its 7 0_or a distribution of data from the DSDS or a dataseries to be supplied to the DSDS must reside H 7gwithin a unix filesystem directory. There is a well defined mapping between a dataseries_name rog 7he`and a directory. That mapping is via a rule specified in a template string. The template is a g. 7 2Yconcatenation of literal characters and place-holders for class member_names and @ 73Emember_numbers. The rule for use of the template is as follows: n` 8 -Any literal characters are copied unchanged. ` 9reTAny construct {class_name} is replaced by the matching member_name. l% :, cAny construct {#class_name} is replaced by the matching member_number converted to 3@ : r)a string via a sprintf with format "%d". M ;ne]Note that an empty template produces an empty string for the directory name. Thus, an empty d[@ ;a ?template can be used to always refer to the current directory. ideu < 7iThe dataset_name processing function will operate on a KEYlist with a key_root name also @ < 7$provided. The prototype might be: s ` =ed/int parse_name( KEY *params, char *key_root); ` >atand it might be used as: r` ?s $if ( !parse_name(call_keys, "in")) ` @mecomplain_to_someone(); or  Ae hThe key_root name, e.g. "in" will be the keyword used to locate the dataset_name in the An AlaiKEYlist. All related keyword names will be built from the root_key by appending a suffix of the l Aepbform "_parsed_name_part", e.g. the template itself will be available via the keyword "in_rule" if #@ A%dthe root_key is "in". =` BatfThe parse_name function that uses the template will take a dataset_name and do the following: W` Cef`0.Look for pre-existing parameters root_key_dataseries and root_key_selector. wg` DEY*if they are found, use them below, else: dm =HH eHH ) b Em1.Separate the dataseries_name and the data_selector parts of the dataset_name and @ EPmake keylist entries root_key_dataseries and root_key_selector. na& Fl2.Separate the dataseries_name into its component classes. The prog class must always be uf4@ F7present when the dataseries_name is not empty. itD Gabd2.5Build keylist entries for each class of the form, e.g. "in_prog", "in_level", "in_series". If atR GWsuch entries were already present in KEYlist use the ones from KEYlist rather than the `@ GLo4just learned ones from the dataseries_name. rip Hyq3.If the prog class is not there, i.e. there is no dataseries_name present, the directory name ~ Hdis an empty string. This means a dataset_name which does not start with prog: will @ H E"default to the current directory. ` I \4.Step through the template using the template rules above to construct a directory name. e` JyZ5.The directory is added to the KEYlist with keyword root_key_wd, e.g. "in_wd".  K iX6.If any of the member_numbers were ranges the name being parsed is a pr Kd]datacollection_name rather than a dataset_name. The parse routine should always l K.gYprepare keyword entries of the form "in_series_sn", "in_series_fsn", and "in_series_lsn"  K uVwhere the "fsn" and "lsn" mean "first sequence number" and "last sequence number" and  Ks_[will be the same if a range was not given but rather a single sequence number. If a range o  KXwas found, the "sn" parameter, e.g. "in_series_sn" should be set to an invalid sequence  Khi]number, e.g. -1 or perhaps minus number in range. If any of the member_numbers were o K I]ranges, the eventual parse function return value should indicate that the name referred to a e, K Jcdatacollection rather than a dataset by perhaps returning the product of all the range : KIfWlengths so that the calling program can examine the return value to see if an SDS_open H@ KcoBwill be possible without further qualification of the name parts. b L lfThe function will then parse the data_selector part of the name. While we may someday expand p LMthe syntax to allow a list of variable names, FOR NOW WE WILL ONLY ALLOW ONE e~ LmVARIABLE NAME in the data_selector. The keywords root_key_select and root_key_slice  Lfcould contain the two parts of the selector. But I suggest that we do a bit more right here since we  Le.mwill need it later. That would be to split the slice spec into its component axis_ranges. Until we s Lse[decide how to know the axis names, we can use only the first axis_range and leave the rest aco Lr icombined. Since the first axis_range present will be interpreted as a record or image number, I o Lprdsuggest we extract the following keyword parameters from the data_selector (using "in as an ho@ Lca example): ` M in_select-> variable name en` Na_"in_fsn-> first record number  ` Oxp!in_lsn-> last record number x` Pf Ain_sn-> (in_fsn == in_lsn ? in_fsn : -(in_lsn - in_fsn +1)) ,` Q t;in_slice-> any remaining part of the slice_selector. ct F R\In addition to building the working directory, e.g. "in_wd" the parse function must build a wT RghZbasename that will be the leading part of the actual data filename. The basename will be b@ RonDconstructed from the series_selector by the fixed template: o |` Ss,"{series}.{#series}" din tHH HH ` TsucThe resulting string will be stored in KEYlist as "root_key_basename", e.g. "in_basename". "` Uex9At this point the parse function is done and can return. mEUT UT` V NACTUAL FILE NAMES _ W GfThere are several aspects to the actual file names associated with a datacollection_name or a m Winudataset_name. It is a dataseries_name that maps to a set of files. A dataset_name then defines R{ Wtona set of files since it is contained within a single dataseries. A datacollection is a collection of  Wadgsets of files. They will have to be handled one at a time. Thus the DSDS, the pipe, or the strategy s W bkmodules will have to expand a datacollection_name into its implied set of dataseries_names  Wcprior to actually doing anything with them. I assume that the above parse function can be used in @ Wsu&all three areas to decide what to do. & XoolOnce a specific dataseries is specified, the collection of files is EXACTLY the set that is matched cF XUTVif e.g. "wd" and "basename" were shell variables, by the shell wildcard expansion of: f Yss$wd/$basename.*  ZdThe actual files containing data will depend on the data storage protocol. There are several cases f Zetathat we can specify now: In the following I will use the shell variable example method above to d+F ZdaKhave specific examples rather than using the root_key_word method. y wE` [dlFITS Tables W \thTA column in a fits table maps directly to a dataset as described above. I suggest tae \ Qthat we ignore the desired case of a complex dataset that describes multiple colas \hiWumns or fields from a fits table at the start. The fits table file which contains the su@ \tosimple dataset has the name: ` ]pe$basename.fits ies` ^ tFITS images  _thTFits image files will be stored ONLY in simple fits files with one image per file. ar@ _1The image files, one per record number, will be: $` `$basename.$select.$sn.fits s c` al CDF datasets a s bThQCdf datasets are stored as a set of files. I believe they have a basename that o% be Omatches the "name of the cdf" dataset. with several extensions appended in the les3@ b t sagtdecide that just as there is a template for each prog since the prog determines the set of classes, wL@ sroLthere is a well defined set of search keys available for each prog. tef usSome of the keys are obvious: class member_names and member_numbers., var_names, axis t@ utiGnames, axis lengths, axis first and delta physical values, axis units. on  wnOther keys will include, dates, access rights, etc. I expect that there will be thousands or tens-of-F w8thousands of dataseries vs. millions of images. t xna\When new data is put into the DSDS it could be by one of two paths, operator action such as r x) bloading the raw telemetry data in which case only the tape label info is needed as keys, and from  xie`strategy modules called by shell or pipeline parents. When creating new data, strategy modules ag@ x ashould: a # T1.Make a KEYlist containing the new name parts, directory must be consistent with ro@ #detemplate for the prog. e f` |A2.Call a function that builds a dataseries == dataset name m(f }d ,3.Build the datafiles with SDS functions. 8` ~=4.Return a set of catalog key values in the return KEYlist lR n bThen the calling program (shell or pipeline) should build a request to the DSDS to take this data `@ dsZand put it somewhere using the catalog keys provided to help find its name in the future. z na]If we want to install data pre-existing in some format, we can simply make a simple strategy r@ ) 7module program that extracts the key info and returns. apeUT UT` deDISTRIBUTED DATA  ie_When we distribute data we will send only DSDS owned dataseries. We would package the ag  abdataseries with its name and also provide, together or apart, the template that we used to map to  rohfiles. We could provide install instructions to extract the datafiles from the distribution media/file ld  =Xformat using one of the templates. I can even imagine our own SFDU format that has the F Rekdataseries_name, the dataseries, and a sample template, and perhaps a shar file to extract it. ordD 6ix e6ix n `  6)o6)matwe6) )6.1& o6.1& tuh  Running H/F 2  6A=pe6A=pd oy 6A=pA66bdase6 6 I  6 I  h ou= #  of  8  Running H/F 1  hH)}H)g o oH))H0$yQ} aH0$yQ amh as 3/15/94  amH\hr}tH\hrH\hr\hrH}HHH I } H I h  #  of  8  6d& 1&H$ ' H$ UU`  HUX ( HUX UUh se2PHS - 3/31/93 4  112.DataSet Names  HH) HH  ` t 7{  Hl@D@ `@8{n Hl@D@ `@HuHu Double LineHz %9n hr Double Line : H  1;1H I11HT%  <n Single Lineh   =  &   HZ>n TableFootnote )d Leftd Rightd n ReferenceXd  d   d  d ddddd }7Firstd pTOCdw@IX5<L8fffff5  CBullet. f6  CellHeading. ~~f7D  References References. ~~ f8E aEquationEquation Number E:(EQ )t. f9 $  Body. f:PTitleBody. Fi f;PTitleBody.  f=PTitleBody. HHf> Body4. f?QfHeading Body. $$f@ H.fZ.CStep. $fA $.Bullet Bullet Symbol\t. HHfB Body4. $fC $.Bullet Bullet Symbol\t. $$fD H.Z.CStep. ~ fEHQ  u~3HeadingH:\t..\t FirstBody. fF@  Purpose FirstBody.. $$fG H.Z.CStep. lfH Bo4Body3r. fffISE  ffbo 1Step Step Number S:.\tStep. $fJ .. $.6.fH.Body2. fK >Extract. ~~fLD u FirstBody Body. ~;fMQ  u~ ReferenceHead 1Heading Rule\t\tReferencesAppendix. fNHQ  u~2HeadingH:\t.\t FirstBody. fOFA  e>. Figure Table Top Step NumberfF:FIGURE .\tBody. fffPS yff Step Step Number S:.\t. fQ  TableFootnote. ~;fRHQ  uf~1Heading 1Heading RuleH:\t.\t FirstBody. $fS .f$.6.H.Body2 . fT@ Abstract1Heading. ~;fUAQ  muGU <~ AppendixHead 1Heading RuleA:\tAppendix \tAppendix. ~fV $.6.fH.Z.~.Body3 . fW@  TNTagAbstract. Z fX@ AuthorTNTag. lfY dyBody3r. Z fZ@ AuthorTNTag. f[  xheader. A:f\   footer left. ~f] $.6.H.Z.~. Body3. $f^ .$.6.orH.TNBody2. f_   header first. f`   footer right. ~fa $.6. H.xZ.er~.A:.Body3. fffffd ff.  ff 3HeadingTOC. fffe  ffdy 2HeadingTOC$. cff~  1HeadingTOC. cfg er r FigureLOF. cfh r TableTitleLOT. ffi  SeparatorsIX. fj  SortOrderIX. $fk Level3IX.  fl fLevel2IX. fm  Level1IX. fn   GroupTitlesIX. fo aIndexIX. fs ra s Header. fu .   Footer. f{T  TableTitleT:Table : . f| f.$.6.H.Z. Data.notes. f}  CellHeading. f~ CellBody. f $Body. fPTitleBody. f Footnote. fQHeading Body. f CellBody. fT  TableTitleT:Table : . f $Formula. fPAppendix Body. fQ f SubHeading . Body. 666 f  $caption.  f $Body. Bof $.Openinga. f $.Opening. f .$.6.H.ApZ. Data.notes.    Bullet Symbolu   Bullet Symbol Equation Number$ First Letter 3Appendix   Callout Bo  Default Bold Run-In Heading   Step Number      Default Bold et       t  Superscript  Subscript oEmphasisoEquationVariables nt@@  a q g aThin bMedium cDouble dThick@ e Very Thine fSingle    a a a a a a a a a c{H}~}H}~}H}~} H}~}H}~}Format A   a b a a a a a b{H}~}tH}~}H}~}H}~}H}~}Format B ) 9 * )uComment =MB o = U > V ? W oEq@ X sHA Y /B Z #E ]  &F ^  )G _  d BlackT!WhiteddAۀRedddGreendd BluedCyandMagentaid DoYellow M.Times.P Times-Roman M.Helvetica.BHelvetica-Bold M.Times.B Times-Bold M.Helvetica.P Helvetica M.Times.BITimes-BoldItalic M.Courier.B Courier-Bold M.Times.I Times-Italic M.Courier.P}CourierM.Helvetica.BIHelvetica-BoldOblique Courier HelveticaTimesRegularaRomanMediumRegularBold Regular~ObliqueItalicMdiZsd<؁]  $ &_CڢWO1 r@}I<+џi$jl)u[e׺h2LUzC&V4 G)RsMSp\ӧTr8a<-Uk{ʿ>'P}[Ξ*3 ~oa/ms_i#YƒF,ŝlW