SkyView in a Jar

SkyView-in-a-Jar provides users with a local SkyView system on their own machines. Users can generate FITS, GIF, JPEG, ... images from major surveys in any requested geometry, resample and mosaic their own data and create their own surveys. SkyView-in-a-Jar comes completely ready to use as a single file. In this default configuration almost all surveys accessible through the SkyView web pages are available. It is easy to add new surveys using local files or the Virtual Observatory Simple Image Access protocol.

The code itself is also extremely extensible. Users can add new samplers, projections, and coordinate systems. Even the classes which define the basic algorithms for finding surveys, selecting images within the survey, and mosaicking the images can be overridden by the sophisticated user. Users can insert their own pre- and post- processing functions to modify the input data or the resulting image.

This document provides a comprehensive User's Guide to SkyView-in-a-Jar.

2. Setup.

SkyView-in-a-Jar is self-contained within a single executable JAR file. JAR files are a modified version of ZIP files used to store Java code. However JARs can include data and program control files as well as the Java code. SkyView-in-a-Jar contains all of the control files needed to access the surveys available from SkyView's standard Web interface. For a few surveys the survey data itself is included. For these surveys not even network connections are required. The other surveys are accessed through standard Web protocols.

Users can download the JAR file from

http://skyview.gsfc.nasa.gov/jar/skyview.jar

and then run the file using the command

java -jar skyview.jar key1=arg1 key2=arg2 ...

The same JAR file is used for all operating systems. This program requires at least a Java 1.5 (sometimes called Java 5.0) environment to run. Java distributions for most major operating systems are available at

http://www.java.sun.com.

E.g., to create a DSS and FIRST images of 3c273 simply enter

java -jar skyview.jar survey=DSS,FIRST position=3c273

The program creates output1.fits and output2.fits. The first contains an image resampled from the DSS and the second contains data from FIRST.

There are many options controlling how to generate and resample the image. These are discussed briefly in the command summary section and then in more detail in the context of how SkyView works.

The SkyView Java classes in the JAR file can be used in other applications as well. Simply include the JAR file in your CLASSPATH. The only difference between a regular and executable JAR file is that the later contains a manifest that indicates how to start up when the -jar option is used in the java command. The SkyView Java application could also be started up with

java -cp pathto/skyview.jar skyview.executive.Imager key1=arg1 ...

The Imager class could be instantiated within other Java applications as well as being invoked from the command line.

Later versions of the JAR files may start a different class, but this method of using the Imager class should still work, and the command line arguments will be upwardly compatible.

3. SkyView-in-a-Jar and ImageJ

Starting with version 1.9, the SkyView JAR also contains a complete distribution of the ImageJ image display and analysis tool. ImageJ is a very powerful public domain software package developed by Wayne Rasband (wsr@nih.gov) at the National Institutes of Health. SkyView currently uses ImageJ functions for creating quicklook images in GIF, JPEG and other formats, including various scaling and color table operations. ImageJ has many other capabilities for users may to analyze images created by SkyView. Hundreds of plug-in modules are also available that provide powerful functionality in diverse areas.

Users should be able to start ImageJ itself with the SkyView JAR using the command

java -cp skyview.jar ij.ImageJ [other arguments]

or they can download the ImageJ JAR file at http://rsb.info.nih.info/ij. Much more information on ImageJ is available at that site. Future versions of the SkyView are likely to use additional ImageJ functionality. The SkyView JAR includes both the ImageJ

class and source code. Note however that the class files are copied directly from the ImageJ distribution version v1.34s and have not been recompiled from the source.

One caveat: ImageJ does not currently accept FITS files with 8-byte real numbers. If you are interested in using ImageJ for further analysis of images, you may wish to set the float option for any images you produce since 8-byte reals are now the default for SkyView.

4. Command Summary

This section describes the command line arguments to the SkyView-in-a-Jar application. Each command line argument consists of a key=value pair. The command keys are case-insensitive. In some cases, particularly when the value is a file or class name, the values may be case-sensitive. Generally, to associate more than one value with a key, the values are separated by commas.

SkyView-in-a-Jar Command Line Arguments
Key	Value
Survey	The survey or surveys indicate which surveys the user wishes to generate an image for. If more than one survey is desired, the values should be separated by commas. The list of all surveys known (by default) to the program can be generated by invoking the application with no arguments. This will give a quick description of the arguments and a list of known surveys. Survey names are case insensitive. The survey user is special. It indicates that one or more files indicated in the userfile argument are to be used as the survey. Using the survey=user and userfile=file1,file2, arguments, users can resample and mosaic local data files. Survey=DSS,2MASS-K
Output image geometry
Position	The position argument gives the center position for which images are requested. It may be either a coordinate pair, or a target name. The position argument or the Lon/Lat pair is required. Position=3c273 Position=’10 20 30, -15 16 22’ Note that the quotes are not part of the actual value: they are needed to ensure that the value is treated as a single string. Your operating system may have other methods for enclosing embedded spaces in an argument.
Lon	This argument gives the Longitude or right ascension of the center of the image. Lon/Lat arguments supersede the position if both are specified.
Lat	This argument gives the Latitude or declination of the center of the image.
Coordinates	This argument gives the coordinate system used for the output image. The first letter in the value gives the type of coordinates. If a number is appended then this is the epoch of the equinox of the coordinate system. Recognized values include J -- Julian equatorial coordinates B -- Besselian equatorial coordinates ICRS -- International Coordinate Reference System coordinates. Gal -- Galactic coordinates H -- Helioecliptic coordinates E -- Ecliptic coordinates. The single letter values may be combined with an epoch, e.g., J2000, B1950, E2020.123 The default coordinate system is J2000. Coordinates=J2000 Coordinates=Galactic
Projection	The three letter designation used to specify the projection of the output image. This uses the suffix in the FITS CTYPE keywords. Supported projections include Car, Tan, Ait, Zea, Sin and Csc. The default is Tan. Projection=Tan
Equinox	If the coordinate system only specified a coordinate type, the equinox can be specified separately. Equinox=1980
Scale	Scale gives the size of pixels in the output image and corresponds to the CDELT keywords. Scale may be specified as a scalar, in which case the pixels are square, or two values may be given if rectangular pixels are desired. The scale is always positive. Output images follow the astronomical convention where the longitude decreases along a horizontal line. The scale is specified in degrees per pixel. The default scale depends upon the survey. If you do not specify an explicit scale, and you choose more than one survey, then the scale may differ among surveys. If there is no survey defined default, then the default scale is 1 per pixel. A user may specify the scale of individual pixels, or the size of the image as a whole, but not both. Scale=0.0025 Scale=0.0002777777 Scale=0.0002777777,0.0005
Resolver	When a target name is specified rather than specific coordinates, the name must be resolved into coordinates. There are two widely used name resolvers available: NED and SIMBAD. The application invokes these through a HEASARC web service. This argument can be used to specify which resolver or resolvers are to be used. If both resolvers are specified, then the second is checked only if the object is not found at the first. Values are: SIMBAD, NED, NED-SIMBAD and SIMBAD-NED. The default is SIMBAD-NED. Resolver=NED
Pixels	This gives the number of pixels in the output image. If specified as a single value, the number of pixels will be the same in both dimensions. The default is 300. Pixels=500 Pixels=1280,640
Rotation	The argument indicates that the output image is rotated by the specified angle (in degrees). By default there is no rotation of the image. Rotation=45.0
Graphic outputs
Quicklook	This argument is used to give the format for quicklook/graphic outputs. The supported formats are: GIF, JPEG (or JPG), TIFF and BMP. A value of JPG is assumed if there are other graphic keywords and this argument is not specified. The output format for three color images (see RGB below) is always JPEG. The argument value is not case sensitive. Quicklook=gif
Scaling	This argument controls the scaling between the data and the displayed images. Recognized values include Linear -- The output pixels are linearly scaled from the input. Sqrt -- Output scaled as the square root of the input pixels. Negative pixels are scaled to the minimum. Log -- Output scaled logarithmically from the input. Negative pixels scaled to the minimum. HistEq -- Output scaled so that each output bin has the same number of pixels. scaling=Sqrt
LUT	The argument allows the user to read in a lookup table which will be used to translate pixels values to color. There are three sources for look up tables. ImageJ defines the following lookup tables internally: Fire, Ice, Red, Yellow, Green, Blue, Cyan, Magenta, Grays, Spectrum, Red/Green, and 3-3-2 RGB. There are also a set of lookup tables distributed with the JAR file to mimic standard IDL color tables. These may be specified using the file name colortables/xxx.bin where xxx is the IDL color table name with spaces replaced by hyphens. (These are also available using the syntax as the old SkyView Batch interface using the Batch Compatibility feature). Users may also create their own color tables. The only format currently supported is a 768 byte file comprising the 256 red values followed by 256 green and 256 blue. When using the internally defined color tables, the case does not matter, but for many operating systems if mixed case files names are used, the argument must be in the correct case. Note that the program will always check for files with the argument in lower-case first. If you have two LUTs xxx.lut and xXx.lut, then the program will always choose the former even when the user specifies lut=xXx.lut. If no color table is specified a gray scale image is produced. lut=’3-3-2 RGB’ lut=colortables/Stern-Special.bin lut=mylutDirectory/myLUT.lut
Invert	Invert the output. The value of this argument is ignored and may be elided. The output image is given with the color table used in inverse order (or with a white rather than black background if no color table is specified). Invert
Min	Set the minimum value for the output image. Before scaling, all values below the minimum will be set to the minimum. This affects only the graphic image, not any FITS data. min=0.
Max	Set the maximum value of the output image. Before scaling, all values above the maximum will be set to the maximum. This affects only the graphic image, not any FITS data. max=256
RGB	Create a color image from three surveys using the first as the red color, the second as green and the third as blue. The output image is always created as a jpg, e.g. output_rgb.jpg. The value of the argument is ignored and may be elided. RGB
Resampling
Ebins	This argument is used only when sampling three-dimensional images which have a set of energy values at each pixel. It must have three comma separated numeric values. The first gives the offset of the first output energy bin, the second gives the size of the output bins relative to the original bins, and the third gives the number of output bins. E.g., suppose we have 10 channels in the input data. We wish to eliminate the top and bottom channels and rebin the remaining 8 channels into 4 output bins in the output image. Then we have Ebins=1,2,4 Note that the offset is 0-based. The bin size does not need to be an integer, the output bins will be integrated over the input bins assuming flux to be uniformly distributed over the energy bin. In SkyView, 3-D surveys normally specify a default for the Ebins value which gives a 2-D output image.
Sampler	This argument specifies the sampling method to be used. The value consists of an algorithm designator which may in some cases be suffixed with an integer giving the order of the sampler. Currently supported samplers include: Nearest Neighbor (NN), Bi-linear interpolation (LI), Splines from 2-5 order (Spline or Spline3, Spline2, Spline4, Spline5), Lanczos interpolation with a specified kernel size (Lanczos, or LanczosN where N >= 2, the default is 3), and an exact-area resampling using Sutherland-Hodges clipping (Clip). The default is nearest neighbor. Sampler=Lanczos5 Sampler=Spline Sampler=Clip
Output file name and FITS format
Output	The name (or stem) of the output image or images. The user may specify the full path of the output image x.fits or just a stem x. If more than one image is being produced, then ordinal numbers will be added so that the output files will be x1.fits, x2.fits. If compression is requested the .gz extension should not be specified, it will be added automatically. The default is output. Output=~/myoutputs/test.fits Output=c:\datafiles\3c273
Compression	This flag indicates that the output files are to be gzip compressed. The default is not to compress the data. The content of the value is ignored and the value can be omitted. Compression=T Compression
Float	This flag indicates the output files should be 4 byte reals. The default is 8 byte reals. The content of the value is ignored and can be omitted. Float=T Float
NoFITS	Suppress the FITS output. The value of the argument is ignored and may be elided. This argument may be used in conjunction with graphics arguments if only a quicklook (e.g., GIF or JPEG) is desired. nofits
Survey definition
Userfile	This argument is used to specify one or more files that are treated as the constituent image files of the user survey. Users can resample and mosaic local files easily using this argument and the user survey. There is no default. Userfile=myfile1.fits,myfile2.fits
SurveyXML	This argument allows the user to specify one or more survey XML descriptors that users can query (see below for details of the format). Use this argument to specify individual files and the XmlRoot argument if you wish to specify an entire directory. SurveyXML=surveys/mysurv1.xml,surveys/mysurv2.xml
XmlRoot	This argument gives a directory where survey XML files are to be found. Each .xml file in the directory will be treated as a survey descriptor file. Currently only one directory is permitted. XMLRoot=/survey/docs/xml
SurveyManifest	The Survey manifest is a file (or Java resource) that lists survey XML files. By default the Survey Manifest is given in the system resource surveys/survey.manifest that is included in the JAR. SurveyManifest=mysurvey/docs/mysurvey.manifest
Caching of survey files
Cache	When a survey’s data is retrieved from a remote location, it is cached in the local file store. The cache argument specifies one or more directories in which the application should look for the remote data before attempting to retrieve it. If a file matching the desired name is found, it is used directly. The first directory in the cache is used to save any files that need to be retrieved. If no cache is specified the current directory is used as the cache. Each survey can be cached into a different directory by specifying a unique cache value in the Settings element of the Survey descriptor file or a common cache can be used for multiple surveys. To have the survey definition for the cache location have priority over a user specified cache location, set the cache location in the Images area of the survey definition rather than in the Settings area. If no cache area is specified the current working directory is used. Cache=/surveycache/cache1,/surveycache/cache2
PurgeCache	This argument indicates that cache files are to be purged after the processing of that survey is completed. Only survey files actually downloaded in this request will be deleted. Other files in the cache are left unchanged even files used in processing. The value of the argument is not used. All files needed for a given image will be downloaded before any files are purged. PurgeCache=T PurgeCache
Classes used in processing
SurveyFinder	The class used to find surveys. This class must implement the skyview.survey.SurveyFinder interface. By default this is the class skyview.survey.XMLSurveyFinder but it may be overridden by knowledgeable users.
ImageFinder	The class used to find the which survey image to sample for a given output image pixel. This class must inherit from the skyview.geometry.ImageFinder abstract class. The default is skyview.geometry.imagefinder.RecursiveImageFinder
PreProcessor	One or more classes implementing the skyview.geometry.Processor interface that are to be used in processing (presumably of the input images) before mosaicking takes place. If a user needs to communicate parameters to a pre (or post) processor, additional Settings can be defined which are used to transmit the information. No preprocessing is performed by default and no classes appropriate for preprocessing are currently provided. An input image smoother might be an appropriate preprocessor class.
Mosaicker	A class implementing the Processor interface that fills in the pixel values in the output image. The default mosaicker is the class skyview.geometry.Mosaicker.
Deedger	A de-edger is a specific type of Post-Processor which allows the user to do additional processing on the image after it has been mosaicked. Currently the skyview.geometry.Deedger and several other deedging classes can be used or the user can write their own class. Supplied classes are: skyview.geometry.Deedger which de-edges using the default algorithm (currently using the BoundaryMedian) skyview.geometry.deedger.BoundaryMedian sets the median change across the boundary of two image to 0 skyview.geometry.deedger.BoundaryAverage sets the average change across boundaries to 0 skyview.geometry.deedger.ImageMedian sets the median pixel value for the regions derived from different input images to a common value. All of these de-edgers add a constant to each of the images. More complex algorithms which add a slope or more complex flat-fielding could be coded. When survey images are mosaicked together there may be artifacts at the edges of the images where the different backgrounds of the images merge. The deedger attempts to minimize these artificats. Deedging is turned on by default for some surveys (e.g., the DSS). It can be turned off for those surveys with the value null for post-processing. The de-edger is a kind of PostProcessor in many cases a user could use the PostProcessor setting equivalently. However using the Deedger setting is preferred since it enables and survey defaults to set de-edging on or off without confusion with other post-processors.
PostProcessor	One or more classes defining the Processor interface which process the image after mosaicking. De-edgers (see above) and a graphics processor are currently available. The class skyview.ij.IJProcessor handles the interface to ImageJ functionality. A call to this processor will normally be added whenever one of the graphic arguments is specified so that users do not need to specify this argument explicitly.
SettingsUpdaters	This argument is used to indicate classes that might wish to look over the arguments that have been specified before image processing begins. E.g., one could build a class that translated arguments from French to the English values expected in the program. Two classes are specified in the default settings: skyview.executive.BatchCompatibility and skyview.ij.IJProcessor. The first translates arguments from the old SkyView batch interface to comparable values for this program. The second fills in default values when the user specifies some graphics keywords (e.g., the graphics postprocessor and the graphic format). Classes specified in this setting should implement the skyview.executive.SettingsUpdater interface. SettingsUpdaters=skyview.executive.BatchCompatibility,skyview.ij.IJProcessor

4.1 Default arguments.

When the Java application begins, default settings are loaded. These default settings are normally read in through the system resource skyview.settings. If users wish to provide their own default settings, they can override the settings provided in the JAR file by creating a skyview.settings file in the current directory, or by pointing to a file in any location (and of any name) using the environment variable SKYVIEW_SETTINGS.

The settings file is a simple ASCII file. Each line can be a blank, a comment line

(first character of the line is #) or specify a setting. Here is what an example file looks like:

# An example skyview.settings file.

Pixels=500

Projection=Sin

Coordinates=J2000

Sampler=Lanczos

Equinox=2000

Resolver=NED-SIMBAD

XMLroot=$SKYVIEW_XMLROOT

Mosaicker=skyview.geometry.Mosaicker

SurveyFinder=skyview.survey.XMLSurveyFinder

ImageFinder=skyview.geometry.imagefinder.RecursiveImageFinder

SettingsUpdaters=skyview.executive.BatchCompatibility,skyview.ij.IJProcessor

Note the definition for the XMLRoot (the directory where survey XML files may be found). When a value begins with a $ the value of the corresponding environment variable will be used but the line will be ignored if the environment variable is not defined. This allows users to override the default settings using environment variables using two lines in the settings file, e.g.,

Projection=Sin

Projection=$PROJ

will use a Sine projection unless the user overrides it by specifying the PROJ environment variable.

5. Example requests.

java -jar skyview.jar position=3c273 survey=2mass-j,2mass-k,2mass-h

This returns three 300x300 pixel images named output1.fits, output2.fits, and output3.fits with the survey default pixel size..

1a.

java -jar skyview.jar position=3c273 survey=2mass-j,2mass-k,2mass-h \

quicklook=jpg nofits

Get the same images but only as JPEGs

1b.

java -jar skyview.jar position=3c273 survey=2mass-j,2mass-k,2mass-h \

quicklook=jpg scaling=log

Get the same images but only as both FITS and JPEGs. Scale the JPEGs logarithmically.

1c.

java -jar skyview.jar position=3c273 survey=2mass-j,2mass-k,2mass-h rgb

Get the FITS images along with a 3-color JPEG.

java -jar skyview.jar position=3c273 survey=2mass-j \

scale=.0005 pixels=1200

This returns a single file 1200x1200 pixel file named output.fits with pixels of 1.8.

java -jar skyview.jar position=3c273 survey=2mass-j \

cache=/local/data/2masscache

This returns a 300x300 file but looks in the cache directory /local/data/2masscache to see if needed survey files have been downloaded there first. In the previous examples all survey files were placed in the current directory. Images are generated much faster if needed survey data is already present in the cache. If the underlying data is not already in the cache it will be placed in the specified cache location.

java -jar skyview.jar position=3c273 survey=2mass-j \

cache=/local/data/2masscache purgecache

This returns the same image as the previous request, but deletes any survey files downloaded from the 2MASS site for this request. However any files that might have been downloaded in previous requests are left untouched.

java -jar skyview.jar position=3c273 survey=user \

userfile=ff1.fits,ff2.fits,ff3.fits scale=.003 \

sampler=Lanczos4 deedger=skyview.geometry.Deedger

This request looks at the three local files ff[1-3].fits and tries to create a mosaic from them using a fourth order Lanczos resampler. Boundaries between the input images are to be masked using the standard deedger class.

java -jar skyview.jar position='horsehead nebula' \

postprocessor=null output=horsehead \

float coordinates=B1950

This asks for a DSS image of the Horsehead nebula but turns de-edging off. The output image should use 4-byte reals and be in B1950 coordinates.

6. The Cache

When a user requests an image from a survey whose source files are on another computer, the application first downloads the files onto the local computer and then resamples the data. By default the downloaded files are copied into the current directory and are left there when the application is complete and serves as a cache for subsequent requests. Often users make several requests near a given location, perhaps changing the pixel size or resampling method used. When these subsequent requests are made, the application notes that the needed files are already downloaded and immediately begins resampling.

If a user is sure that a survey file is not going to be needed again, or if space is at a premium, the user can request that the survey files be deleted by specifying the PurgeCache setting. This will delete all files downloaded for a given survey in the current request. However if multiple files are needed for a given survey, then all files for the survey will be downloaded and the output image produced before the downloaded files are deleted.

While the advantages of keeping the survey files available can be substantial, having them in the current directory can be a nuisance if more than a few requests are made. In this case the user will likely wish to specify a cache directory to save these files, so they do not obscure other files in the current directory. A cache may be specified in the settings file or in each command using the Cache setting to specify one or more cache directories.

Users who create their own survey data sets (or who modify the XML survey files distributed with the application) can create different cache directories for each survey. By setting the cache directory in the <Settings> area of the survey description the user specifies a default cache directory which can be overridden on the command line. If the user specifies a cache in the <Images> area, then this setting cannot be overridden by the user. Note that more than one cache directory can be specified. All of the cache directories are searched when looking to see if a file is already available, however any files that are downloaded will be placed in the first directory.

6.1 Caching the Digitized Sky Survey.

The Digitized Sky Survey data is handled somewhat differently than most surveys. Rather than trying to download entire plates of data, which would take forever even if it were permitted by the STScI servers, SkyView requests small tiles from these images. It then builds the image the user requests by mosaicking the tiles together. The tiles are cached like any other survey. The headers for each tile indicate the original image from which it was derived.

7. Dependencies on Remote Systems

While all data processing occurs in the local system, the SkyView-in-a-Jar application may fail if remote resources are not available. The underlying data for a few surveys are included inside the JAR file (e.g., 408 MHz and EGRET). These are generally low resolution surveys. The application will fail if it is unable to retrieve data for other surveys. For many surveys that data is fetched from the SkyView server at skyview.gsfc.nasa.gov. However the DSS, DSS2 and FIRST surveys link to archives that the Space Telescope Science Institute, and the 2MASS and NEAT surveys use archives at JPL. A number of other surveys, e.g., the SDSS, now use the VO Simple Image Access protocol, and link to URLs described therein. The <Images> area of the survey description will generally indicate the dependencies.

The only other remote dependency for the application is in the resolution of target names. If the user specifies a target rather than a pair of coordinates, the Name resolver/coordinate converter service at the HEASARC is used to resolve this name into coordinates. The HEASARC service in turn calls the SIMBAD and NED name resolvers. The HEASARC service may also be used if the user enters the coordinates using characters other than numerics, signs, colons, commas and decimal points (e.g,

10h30m10.3s).

8. Compatibility with the SkyView Batch Interface

For the convenience of users of the old SkyView batch interface, the SkyView-in-a-Jar application supports most of the arguments of that interface insofar as they related to the generation of FITS files. The following are supported as input settings:

SkyView Batch keyword	SkyView-in-a-Jar setting
VCOORD	Position
SCOORD	Coordinates
MAPROJ	Projection
PIXELX,PIXELY	If both specified, converted to a Pixels
EQUINX	Equinox
SFACTR	Size
COLTAB	LUT
ISCALE	Scaling
file	Output

The values are also converted appropriately. E.g., MAPROJ=Gnomonic is translated to Projection=Tan.

Generally results from the new interface should be very similar to the old SkyView interface. There is one major exception. The IDL code which the old interface used ran almost entirely in single precision arithmetic. The Java classes use double precision arithmetic exclusively with an option for converting the final result to single precision. Users must specify the Float setting to get single precision results. The difference in precision and other small differences in the algorithms mean that there can be small differences in the values returned. E.g., when using a near-neighbor resampling, positions near the edge of one pixel might move onto an adjacent pixel.

There format of the FITS headers differs in considerably between the two versions. Substantially more information about the processing is included in the new version including the files that were used in generating the output mosaic.

9. SkyView-in-a-Jar Operations.

In this section we describe how the SkyView Java application works. This includes short descriptions of some of the algorithms used.

The basic goal of SkyView is relatively simple. Given a set of one or more existing images comprising a survey, SkyView generates a new image in a user-specified geometry. The output user image may overlap one or more of the survey images, and may overlap different images in different locations. This section discusses how the application accomplishes this task.

9.1 Getting defaults and user arguments.

When the application starts, a set of default parameters is established. The java.executive.Settings class is used to convey the current parameter settings. The default parameters are read in from a configuration file or system resource. User arguments are then processed and override any defaults. System parameters are stored in a HashMap but both the put and get methods convert the key to lower case, so that the keys are not case sensitive.

9.2 Updating Settings.

One or more classes may wish to look at the settings before processing begins to fill in further defaults, to allow synonyms or for other reasons. These classes should be specified in the SettingsUpdaters setting. Currently the BatchCompability class allows syntax similar to the old Batch interface while the IJProcessor fills in missing settings when graphics have been requested.

9,3 Locating surveys.

Once the user arguments have been parsed, the application looks for surveys. Using the default SurveyFinder there are four possible sources of surveys:

· The application searches for a survey manifest resource. This manifest gives the names of other resources or XML files where each file defines one survey. In this initial pass the XML is only parsed far enough to find the names of the surveys. Surveys can have several names. When the system tries to find the surveys the user is looking for the match will be done in a case-insensitive fashion. The source manifest is where the default surveys are normally found when the executable JAR file is used.

· The second location where surveys may be found is determined by the setting for the SourceXML setting. This may be set on the command line, or if the user sets the environment variable SKYVIEW_SOURCEXML then the value of that variable is used. When the SourceXML is set, that directory is searched and each file of type .xml is treated as a survey definition file. Users can use this method to add a set of surveys to their SkyView implementation. It is also used when the SkyView application is run outside of the JAR file.

· Users can specify one or more survey files directly using the SurveyXML setting. Each field in the SurveyXML is assumed to be a Survey definition file and is parsed.

· Finally, the user can specify a User survey by setting the SurveyFile setting. Each field of the SurveyFile is treated as a survey image file. Users can generate an image from this survey by using the survey name, User.

9.4 Looping over the surveys.

Once the set of available surveys has been determined, the application loops over the surveys the user specified in the Survey setting. If the name the user has specified is not found, then an error is signaled and processing continues with the next survey.

When a survey is found, the corresponding survey definition file is parsed. Part of the survey definition file is a set of survey default settings, e.g., the default scale for the image, or whether the image will be de-edged. These settings do not override settings already made, so that survey settings do not override the overall defaults or user defined settings.

9.5 Getting candidate survey images

The first step in processing a survey is determining a list of candidate survey images that may be used in generating the output image. If a survey uses the VO Simple Image Access Protocol (SIAP, see http://us-vo.org), then the candidates are all images returned in the initial SIAP request. If the survey explicitly lists the images, then the application passes through these images and uses the ImageSize characteristic to find images that might overlap part of the user’s image. If this is a request for the User survey, then all of the images the user specified are candidates.

9.6 Building images.

At this point, a survey image is only a candidate for processing. Often there will be many more candidates than will actually be used. If the images are present locally this is not an issue, but it may seriously delay processing if we download a number of images that are not actually needed. To ensure that only needed data are retrieved from remote sites, remote images are normally represented initially as proxy images. Proxy images have the same geometry (or a very good approximation) as the real image, but the image data is not filled in. If and when the application decides to sample the data in the image, it downloads the remote data and converts the proxy into a real image.

The SIA protocol allows for fields that define the geometry of the images returned which makes it easy to build proxies without actually retrieving data. For other surveys we find that the geometry of the images is largely constant with only the center position changing. This allows us to build spells to enable the creation of proxy images and their transformation to real images.

The Survey XML descriptor file controls this process. It can specify either an explicit list of image files or spells that define the content of the survey, or an ImageGenerator class that will create this list dynamically. E.g., if the IRAS100 survey uses local data, then its survey XML file lists 430 files explicitly. In the SkyView-in-a-Jar distribution, the IRAS data is retrieved over the net, so the survey XML file includes sufficient information to generate proxy images for the survey. These proxies are used to find out which images are to be resampled and which pixels in the output image are to be resampled from which input image. Only when we actually begin resampling does the program actually download the data.

The SDSS-I survey uses an ImageGenerator class that in turn uses the SIA protocol to find candidate images around the users requested location. The SIA call returns sufficient information to generate proxy images so we only actually retrieve the SDSS images we are going to use.

An ImageFactory class is also defined in the survey XML file. It shows how to convert the file name/spell String into an actual image object. The FitsImageFactory just reads in a FITS file, but the CachingImageFactory first looks to see if files are present in the cache, and if not creates a proxy image as described above.

Users can create their own ImageGenerators and ImageFactorys to meet special requirements for new surveys they might want to include.

9.7 Finding the best survey image for each output pixel.

Once the set of candidate images is determined, the application determines the image to sample for each pixel in the output image. This uses an object of the ImageFinder class. The default, BorderImageFinder is described here. The position of the four corners of the output image is determined for each of the candidate input images. The best candidate image for each of the four pixels is then determined. The best input image is where the output pixel location is located in the image but furthest from the edge of the input image.

E.g., consider first pixel (x,y=0,0) of the output image and assume there are three candidate images each of which is a 500x500 image. The position corresponding to the center of output pixel 0,0 is transformed into the pixel coordinates of the three candidate images. If the pixel coordinates corresponding to the output pixel in three candidate inputs are (-80, 300), (40,40) and (250,35), then the second image is the best. The output pixel location is not within the first image, and is only 35 pixels from the edge of the third, but 40 pixels from the edge of the second.

If all four corners match the same input image, then that image is assumed to be the best image for all of the interior pixels. If different images are best (or perhaps there is no corresponding image for one of the pixels), then the rectangle is broken up into four sub-rectangles and the process is repeated recursively until we have either checked each pixel individually, or we get common input images for each corner of some sub-rectangle.

In the case where the output image is comparable to or larger than the input images, the initial checking will be done on rectangles no larger than half the size of the input images. When the input image selection splits into sub-rectangles, any candidate image that was not found on any of the four corners is not passed down to the next level of the recursion.

Consider generating an all sky map from the IRAS100 survey which has 430 12x12 degree square images. The application first breaks the output image into 6x6 degree squares and checks each of these rectangles against all 430 images. In most cases it will not find the same best image for all four corners of the 6x6 degree squares, so it next looks at 3x3 degree squares. However, in this second step it only looks at candidate images on which at least one of the four pixels at the previous step was on the image (though not necessarily the best image). So in this second stage we are typically looking at only three or four candidates making the search much more efficient.

The end-point of this process is an integer array with the same dimensions as the user output image but where the value at each pixel is the best input image. Special flag values are used to indicate that this output pixel is not matched by any survey image. An output pixel may also be outside of the standard projection coverage, e.g., outside the oval of an all-sky Aitoff projection, or the circle defined in the Orthographic (SIN) projection.

In many cases the user may have selected images from a set of surveys which have identical or near identical geometry (e.g., IRAS100 and IRAS25, or 2MASS-J and 2MASS-K). These are called geometry twins and are identified in the survey descriptor files. If two geometry twins are processed in succession, then the calculation of the best image from the first twin processed is re-used for the later twin (or triplets, or ).

9.8 Preprocessing

At this point the program allows for image pre-processing. The user can define a class (or classes) that can modify the data that are used for sampling. A preprocessor class has access to the input images, the index associating images with output pixels and the output image. Only the geometry of the output image has been specified, the data so far are nil.

No preprocessing classes are currently provided. One useful class might be a smoother that replaces the input images, with smoothed versions.

9.9 Validating the image.

Up to this point in the processing of the image, only the coordinate system of the input image has been needed. To avoid downloading images that may not actually be used, remote images use proxy images that do not have data values associated. Even if the image is available locally, only the FITS header is read rather than the data values if this can be done efficiently. As we begin to process each survey image, the image is validated, i.e., we download the image if it is a remote URL and then populate the actual input data array.

While the proxy image should have a geometry very similar to the real image, it may not be quite identical. If a request for a region is repeated and the survey images were cached on the first request, then the actual image geometry will be used at all stages in the second request which may result is slightly different boundaries between the regions sampled from the various input images.

9.10 Resampling setup.

Once the best image for each output pixel has been selected, the program begins to process each input image. For some samplers there is some processing of the input image done before any resampling occurs. E.g., the spline resampler needs to calculate the spline coefficients to be used. In this case if the input image is much larger than the output image, then spline coefficients may only be computed on a sub-region of the input. The minimum and maximum x and y values for the output image that are being sampled in this input image are computed. These bounds are projected into the input coordinate system and the spline is computed only for this sub-region. Currently this behavior is enabled only for tiled surveys.

9.11 Sampling in energy.

If the input survey has depth in the energy dimension, the data is first sampled in energy according to the Ebin’s specification before spatial sampling. A default specification is normally included in the survey default to produce a 2-d output image. Thus unless the user specifies that they want a 3-d output image, they get a 2-d image.

If there is no energy dimension to the data, this step is skipped.

9.12 Sampling in space.

The program then loops over the output pixels associated with the current input image and computes the value using the selected resampler. For most resamplers the center of the output pixel is projected into the plane of the input image. For the clipping sampler, the corners of the pixel are projected. The output pixel is then populated with the resampled value according to the requested algorithm. If the output image has depth in the energy dimension, the sampling is performed at each output energy channel.

Currently the spline sampler cannot process 3-d images.

9.13 Post-Processing

After the data have been resampled, post-processing may be performed. The interface for post-processing is exactly the same as for pre-processing, however at this point the output image has been populated and we would anticipate that most post-processing will perform some filtering of the output image while preprocessing is expected to primarily affect the input image or images. Examples of post-processor are de-edging and graphics processing

9.13.1 De-edging

Since survey data may have different background values in the different survey images, the edges of the image may show up as an unsightly artifact. If the user wishes to minimize the edges, then a de-edger may be run on the output image. .

The algorithm used by the default deedger attempts to minimize discontinuities along borders. The output image is divided into regions defined by the input image from which the pixels were sampled. These regions are placed into adjusted and unadjusted sets. The region with maximum number of pixels is placed in the adjusted set and all other images are initially unadjusted. The program then looks for the longest border between an adjusted and unadjusted region. Borders are defined by looking at each pixel and seeing if any of the four neighboring pixels is from another region (diagonals are not considered). The median of the shifts as we move across the border from the adjusted to the unadjusted regions is computed. An offset is applied to the unadjusted region to set the median shift across the border to 0. The unadjusted region is moved into the adjusted set, and the process iterates until all regions have been adjusted. Other classes use somewhat different algorithms to compute the offsets for images: use the average rather than the median shift at the border, adjusting the medians of all pixels that come from an image, not just the boundary pixels.

The FITS headers include information on the adjustments made when de-edging is applied.

9.13.2 Graphics Processing

All graphics outputs are also produced in post-processing. Appropriate functions in the ImageJ library are called to produce the desired data. Graphics procdssing does not affect the FITS data. If Min or Max settings were given, then any values below or above the limits are replaced by the limit value. The data is then scaled using the appropriate scaling method. Logarithmic (Log) and Square-root (Sqrt) scaling scale negative values to 0 (or the minimum positive value for log). The result are images with values ranging from 0 to 255. These data are then converted to the requested graphics output.

If users request an RGB image from three input surveys, then each of these surveys is processed in turn and the scaled results are stored until the last survey is processed. Then the three images are combined to create a single RGB JPEG file.

9.14 Data conversion and writing the FITS file.

All of the internal computations are done with the data as a one-dimensional double array of values. Just before writing the data out, the application checks to see if the user wished to have only 4-byte reals (float’s in C or Java-speak) rather than the default 8-byte data (double’s). If so the data is converted. The program then curls the data into the appropriate dimensionality and uses the nom.tam.fits library to write the FITS image. The survey descriptor defines survey-dependent information that is included in the FITS header.

The user can elect not to create a FITS image by specifying the NoFITS setting.

9.15 Cache cleanup.

If a user requested data that was not present in local files, this data is first downloaded into the first cache directory (the current directory by default). By default downloaded survey files are left for possible use in subsequent requests, but if PurgeCache has been set, then these files are deleted as the processing for each survey is completed. However only the files actually downloaded during this request will be purged. Files already present in the cache will be left untouched.

10. Survey Definition File

Each survey (other than the User survey) used in SkyView is defined by a survey definition file. This is an XML file that pulls together all information needed to use the survey. This section describes the structure of the file. [Note: A user can define SurveyFinder object with behavior that they specify.]

There is no formal schema for the survey file format but we suggest the following order for the top elements of the file.

Survey

ShortName

Name

Description

Settings

MetaTable

OnlineText

FITS

Images

10.1 ShortName

The short name element gives one or more short names for the survey. Short names are used as the values of the Survey keyword in the SkyView-in-a-Jar application and therefore it is recommended that they not contain embedded blanks. If multiple short names are to be specified they are separated by commas. All survey description files are read at program startup until the short name is found, so putting this first can speed up the program a little.

Examples:

<ShortName>EGREThard,EGRET100-10000</ShortName>

10.2 Name

The name element gives a longer name for the survey generally spelling out acronyms and such that may have been used in the short name.

Example:

<Name>Two Micron All-Sky Survey: K band </Name>

10.3 Description

This gives a text description of the survey. This may include HTML and if so should be coded as a CDATA element.

Example

<![CDATA[

This LISA Galactic survey is a low-resolution image

of the gravitational background in

the Galactic plane as described in <a href=someurl>Lisa

Galactic Plane Handbook

</a>

]]>

</Description>

10.4 Settings

The settings element is used to specify survey specific settings. Within the settings element there may be 0 or more sub-elements of the form

<key>value</key>

When processing of a specific survey begins, the Settings element is parsed and the Settings hash is updated with any key/value pairs found. The case of the key is not distinguished but case may be important in the value. Note that XML parsers require that the opening and closing case of the keys be consistent within the file.

Common survey specific settings include the default image scale (Scale), other surveys that have the same geometry (GeometryTwin), the default energy binning for 3-d surveys (EBins), and post-processing options, notably the de-edger.

Survey settings will not override existing user specified settings.

<GeometryTwin>SurveyA,SurveyB,SurveyC</GeometryTwin>

<Deedger> skyview.geometry.Deedger </Deedger>

</Settings>

Since the survey settings do not override user specified settings, the default de-edging of this survey will be turned off when the user specifies Deedger=null.

10.5 MetaTable

The metatable element includes a set of metadata elements that give an overall description of the survey. SkyView attempts to give some common information about all surveys, energy range, resolution, and these are grouped in the metatable.

The metatable information is normally included in any FITS file produced and elements may be displayed in other interfaces.

Example:

NASA IPAC/Jet Propulsion Laboratory

</Provenance>

<Copyright> Public Domain </Copyright>

<Regime> Infrared </Regime>

<Coordinates> Equatorial </Coordinates>

<Projection> Gnomonic(TAN) </Projection>

<![CDATA[

Wheelock, et al., 1991, IRAS Sky Survey Atlas Explanatory

Supplement.

]]>

</Reference>

</MetaTable>

10.6 OnlineText

The online text element is used to include text that is to be shown on Web pages associated with a generated image. The online text may include HTML tags and if so should be enclosed a CDATA element. Variables prefixed by $ will be substituted for in the text by the appropriate values. These include:

$ra The right ascension of the center of the image in decimal degrees

$raH The hours of right ascension

$raM The minutes of right ascension

$raS The seconds of right ascension (including fraction)

$dec The declination of the center of the image in decimal degrees

$decD The degrees (and sign) of declination

$decM The minutes of declination

$decS The seconds of declination (including fraction)

$sizeD The requested size of the image in degrees.

$sizeM The requested size of the image in minutes.

$sizeS The requested size of the image in seconds.

Example:

<![CDATA[

<a href=http://somewhere/cgi-bin/query?POS=$ra,$dec&SIZE=$sizeD>Link </a> to the associated source catalog.

]]>

</OnlineText>

No software currently uses this element and no data is supplied in these elements in the survey descriptions included in the Jar.

10.7 FITS

The FITS element includes information to be included in the FITS header generated with the image.

Example:

<FITS>

<![CDATA[

SURVEY = 'IRAS 100 micron'

BUNIT = 'MJY/SR ' / INTENSITY

COMMENT Note that the values are not adjusted for any change in pixel

COMMENT size from the original pixel size of 90 x 90 arcseconds

BLANK = -2000000000 / TAPE VALUE FOR EMPTY PIXEL

ORIGIN = JPL-IPAC / INSTITUTION

TELESCOP= 'IRAS' /

INSTRUME= 'ISSA-FLD' / IRAS SKY SURVEY ATLAS

EDITED = T / SCANS EDITED

APPCAL = T / CALIBRATION CORRECTION 25 MICRON

DE-ZODY = T / DE-ZODIED IMAGE

GLBL-D = T / APPLIED GLOBAL PARAMETERS

LOCAL-D = T / LOCAL DESTRIPER

ASBLANK = T / ASTEROID BLANKING

]]>

</FITS>

11. The < Images> Element

The main business of the survey file is in the Images element and so we give it a whole section to itself. There are two alternate configurations: an explicit list of the images in the survey, or a link to a class that will generate a list of images for a given user request. Most of the fields in the Images element (with the exception of Image elements) are parsed when survey processing begins and used to update or set entries in the Settings hash. However, unlike the fields in the Settings element, these settings override any existing settings.

11.1 Explicit Image Lists

In this case, the Images element will contain a set of Image elements. Each element has a text value with four blank delimited tokens: the filename or spell, a longitude and latitude, and the epoch of the image. The coordinate system for the longitude and latitude can be specified in the SurveyCoordinateSystem setting and defaults to J2000. In addition to the Image elements there should at a minimum be an ImageFactory element and an ImageSize element. The image size gives the size in degrees of the smaller dimension (if not square) of the images. This is used in the process of determining the best image for each output pixel.

The ImageFactory gives the class which given the appropriate spell or file name creates the image. The skyview.survey.FitsImageFactory takes a filename and simply reads the file to create the input image. If a <FileNamePrefix> element is included Images element, then this prefix is prepended to the file name in the Image element before being passed to the FitsImageFactory.

The skyview.survey.CachingImageFactory assumes that it is being given a spell that tells it both how to retrieve the data as a URL as well as sufficient metadata to create a proxy image. The spell consists of 10 or 12 comma delimited fields. These are:

1. The URL to retrieve the data from

2. The file name to store the cached file as

3. The longitude of the center of the image

4. The latitude of the center of the image

5. The projection used in the image

6. The coordinate system used in the image

7. The width of the image in pixels

8. The height of the image in pixels

9. The width of a pixel in degrees

10. The height of a pixel in degrees

..or..

9-12. The CD matrix elements for the image.

For projections (Aitoff, Cartesian and CSC) where the reference value is normally fixed, the third and fourth tokens are the pixel offsets for the image, while for other projections the reference pixel is assumed to be at the center of the image.

The user familiar with FITS will note the similarity of fields 3-10 (or 12) to standard FITS WCS keywords. Note that the spell does not need to use the same coordinate system as the other fields in the Image elements.

If the SpellPrefix or SpellSuffix values are specified in the Images element, then the Image spell string is appropriately extended. Usually much of a spell is constant for a given survey.

11.2 Image Generators

Rather than specifying each individual image, the survey file can describe an ImageGenerator class that will generate the images. An Image generator can generate either files or spells.

There are specialized image generators for the DSS and 2MASS surveys. Users may wish to examine these classes for extension to other special surveys, but the most useful generator is likely to be the SIAPGenerator. The SIAP generator uses the Virtual Observatory Simple Image Access protocol and can be adapted to use many of the existing SIAP services with appropriate entries in the Images element.

The key fields are:

· SiapURL: defines the base URL for the Simple Image Access service.

· SiapProjection: The projection used in images returned by the service

· SiapCoordinates: The coordinate system used in images returned by the service

· SiapNaxis: The size (in pixels) of image returned by the service

· SiapScaling: The size (in degrees) of the pixels returned by the service

· SiapMaxImages: The number of images returned to include as candidates.

Alert readers will note the correspondence of most of the fields to the spell parameters and to the FITS WCS fields. SIAP services can return WCS information for each image explicitly and to the extent that they do so (e.g., almost all return the size of the image in pixels), the user need not supply this information. However many (most) SIAP services do not supply all needed information. The fields above remedy any deficit.

Not all SIAP services are good candidates for immediate inclusion as a service. SkyView will use all images returned as candidates even if the images are quite heterogeneous. Only SIAP services that return relatively homogeneous images can be used with some kind of initial filtering. However it is often easy to modify the SIAPGenerator to do such filtering and we anticipate that this class will have filtering capabilities added.

The third example below shows the power of the SIAPGenerator. The entire SDSS survey is included as a survey with just a few lines of text.

11.3 Example <Images> elements

Listing local files:

/surveydata/iras/iras100/

</FilePrefix>

skyview.survey.FitsImageFactory

</ImageFactory>

... 425 more Image elements ...

</Images>

Listing remote URLs:

<SpellPrefix> http://skyview.gsfc.nasa.gov/survey/iras100/images/

</SpellPrefix>

<ImageFactory> skyview.survey.CachingImageFactory </ImageFactory>

... 425 more images ...

</Images>

Using an Image generator.

<![CDATA[

http://casjobs.sdss.org/vo/DR3SIAP/siap.asmx/getSiapInfo?FORMAT=image/fits&BANDPASS=I&

]]>

</SiapURL>

<ImageFactory> skyview.survey.CachingImageFactory </ImageFactory>

<ImageGenerator> skyview.survey.SIAPGenerator</ImageGenerator>

</Images>

12. A Simple Survey Definition File

While a survey XML definition file can be complex, most data is optional. A minimal survey definition need only include the short name of the survey, a default scaling, the image size and the names and locations of the included files.

<ShortName> MySimpleSurvey </ShortName>

</Settings>

<ImageFactory> skyview.survey.FitsImageFactory</ImageFactory>

</Image>

</Survey>

This describes a survey with three images along the equator (at RAs of 0, 10 and 20 degrees) which have a minimum dimension of 10 degrees and 0.1 x 0.1 degree pixels.

13. Contents of the JAR file

The SkyView JAR file contains files from a number of directories as well as indidual files used by the program. This section discusses the content of the JAR for users who may wish to create their own customized versions of the JAR. You can view the contents of the JAR file using the command jar tf skyview.jar.

13.1 The Class Files

Class files are found in three directory trees:

skyview/... contains the class files for the class files developed in SkyView. This includes all the code relating to geometric transformations, image surveys and the main program for the JAR file.

nom/... contains the methods for reading and writing FITS files and utility I/O functions.

ij/... contains the classes and a few data files) for ImageJ.

13.2 Survey Data and Metadata

The JAR file contains metadata descriptions for many surveys and actual survey data for a few. Metadata descriptions in the format described above are found in surveys/xml. FITS data for a number of surveys is found in surveys/data.

The survey manifest file gives a list of all of the surveys included in the JAR and is found in surveys/survey.manifest.

13.3 Control Files

A number of control and miscellaneous files are found in the JAR.

The skyview.settings files contains the default settings for the program.

IJProps.txt contains settings used by ImageJ.

The META-INF/MANIFEST.MF file is included in the JAR using a command like jar fm skyview.jar manifest.fil (the JAR command renames the file when it is inserted). It contains a single keyword value pair that gives the class which is to be used when the JAR is executed (currently skyview.executive.Imager).

The onlineinfo.txt file contains help text which is displayed when the program is invoked without any arguments.

13.4 Source code

Source code is found in the source/... hierarchy for all Java classes defined in the JAR file.