The DEIMOS project (Deep Extragalactic Imaging Multi-Object Spectrograph) aspires to deliver a very complex, powerful spectrograph to Keck-II in 1998. The mechanical and optical complexity of the instrument, and the large CCD mosaic, present challenges at every level, from design to deployment and data reduction.
At the DEIMOS PDR I presented a proposal to model most of the instrument's components and operating characteristics in a relational database, logging actual operating parameters to this database during observing. Several objectives might be achieved if this could be done successfully:
See the DEIMOS PDR documents for background information, especially Chapter 9 which contains the DEIMOS database proposal. In this chapter are outlined our reasons for using existing Sybase servers at Lick for development and possibly for deployment; there are also preliminary schema designs for the capture of static and dynamic information about DEIMOS.
After the PDR, as I considered the complexity of the software and hardware system we are building, it occurred to me that the database could be used to solve some aphysical data management problems for the project. We were likely to be adding several hundred new keywords to the Keck (KTL) instrument control system, all of which can be expressed as FITS keywords. (KTL uses a keyword/value model to control dome, telescope, and instrument; many of the keyword/value pairs are captured in image headers written at the observatory. Lick uses a similar system). FITS namespace is crowded; many of these keywords may exist already, used for some other instrument built at Lick; several programmers and engineers needed access to a dynamic, shared, centralized and authoritative source of keyword nomenclature, syntax, and semantics. It seemed an ideal target for a database application, treating FITS keywords as entities to be modelled.
STB (Save the Bits) software from NOAO had been installed at Keck-I some months previously, and I had written and installed some tools there for ingesting STB headers automatically into a Sybase schema provided by NOAO. Part of this schema was a table of FITS keywords. It seemed that this table design could be significantly improved, and that an improved version could serve many purposes simultaneously , far beyond the simple archiving of image headers.
I commenced work on a FITS keyword schema using Sybase, with the help of Mr. Steve Allen (who is far more expert in FITS semantics and usage than I). Our objective was to capture FITS keyword syntax intelligently, in a generalized way, so that complexities like the internal structure of some keyword values, or the multiple occurrence of indexed keywords, would be economically and consistently represented. Our original goal was to produce complete formatted documentation of all new DEIMOS keywords, automatically, for the Critical Design Review and for delivered manuals.
Our first target was clean HTML documentation automatically generated for a set of FITS keywords. Our generated documents should show the format of each keyword as well as its semantics, and express explicitly any relationships with other keywords (i.e. NAXISn where N is controlled by the value of NAXIS). It should also show the provenance of the keyword (FITS namespace is nonunique, and different institutions may use the same keyword differently); we invented a concept called context to disambiguate keywords of the same name but different semantics.
Our swift success in producing HTML documents encouraged us to proceed with the generation of sample headers. From there, it was a small step to the production of FITS I/O code (source which is very tedious to hand-craft, yet not difficult to autogenerate once you have a database of headers). A trivial extension of our code added the ability to generate Sybase DDL (data definition language) for the creation of tables whose schema corresponds to FITS tables and headers; from there it was only natural to transform existing Sybase tables into FITS table extensions.
At this point in the project we have a tool set which
all from one modest schema (three tables) whose data are easily maintainable via our fosql forms interface or by any commercial or public interface to Sybase. It would not be terribly difficult to convert the schema and tools to Oracle or Postgres95, if Sybase were not available.
Having explored the nature of FITS keywords rather deeply, and having incorporated Sybase table definitions into the keywords database, we were forced to admit that the keywords database was not just a list of FITS keywords, but something far more general. For lack of any more concise term I chose to (ab)use the term meme (from semantics studies) to describe what it was we were documenting. A meme, in our specialized sense, is a unit of meaning in a system where FITS keywords and/or databases are used to express information. A FITS keyword, or an element of a complex FITS keyword, is a meme; so is a field in a database table. There is hierarchy among memes; a FITS header is a meme which owns a list of other memes.
Having the concept of meme handy, my next thought was that some rather complicated design decisions (about the DEIMOS control system overall) could be expressed by modelling the travel of memes through the hardware/software/human system composed of the telescope, dome, instrument, observing computers, etc. Every unit of meaning or information originates somewhere, and is passed by some medium, in some format, through various hands, until it ends up in one or several destinations. For example, the CCD temperature is measured by a sensor and read by a controller, which reports it on demand to the control computers as a keyword value. The meme TEMPDET, in other words, travels from hardware agents to software agents. One of the places it ends up is in a FITS header, when the image is written to disk and/or tape. It might also end up on a screen display, and in a log file.
It seemed to me that we could express our design decisions economically and flexibly by means of a schema relating agents and memes. Capturing the origin and flight path of any piece of information in an online database would assist us in
I proposed this to the other project members and proceeded to draft a prototype database of Agents, and the tables necessary to represent the flow of memes between Agents.
It's my hope that the Agents and Meme-flow databases will prove useful for the DEIMOS project (and possibly other Lick instrument projects) by expediting the design and documentation process, as well as (potentially) providing powerful help features for telescope operators and astronomers. The Memes (keywords) database, I hope, has wider application. It's our ambition to continue the collection and taxonomy of FITS keywords, documenting FITS headers of many different types from many different institutions, and offering this library of FITS keywords publicly via Web interfaces. Ideally I would like to provide means for other institutions to contribute new FITS keywords to the library; and ideally I would like other institutions to use our code, documentation, and sample data generators freely for their own purposes.