Over the past couple of days we have discovered problems in accessing two of our remote survey data sets. The UKIDSS simple image access service has been non-responsive. We’re hoping it comes up again soon. You can see the red warning on the SkyView home page where we assess the aliveness of remote resources.
A user has alerted us to a more subtle and serious problem with the SDSS surveys. If they requested a large (0.5 degree) region they found that many of the pixels in their image were blank. If they requested the same image later most of those pixels were filled in. A few missing pixels might require three image requests to be filled in. It appears that the SDSS is supplying incorrect descriptions of the coverage of images in their SIA service. We have informed the SDSS of the issue but while it is being worked on we have created a version of SkyView, v3.2.1, which has a workaround for the issue. You can try it out now — and if you are using the SDSS for large images we strongly recommend that you do — and anticipate that it will be made the default version in the next few days.
The SDSS problem arises from how SkyView mosaics data from multiple images together. In the first stage an image generator defines a set of candidate images that are to be considered. For remote services, where we are using the SIA protocol to get data, all of the images returned by the SIA service are considered as candidates. We then consider each pixel in the output image to be produced and decide which of the candidates it should be sampled from. The usual approach — and the one used for the SDSS — is to pick the candidate where we will be furthest from the edge of the image.
To make this determination we need to know the sky coverage of each candidate. If we have already downloaded an image so that it exists in the cache we can and do use information in the FITS file itself. If we haven’t downloaded the image yet, we normally use the coverage information provided in the SIA metadata returned with the image URL. Typically there will be a lot of candidate images that aren’t going to be used in the mosaic and we don’t want to have to download them all.
Since the SIA metadata is wrong, when we chose the candidate image, it often turned out that that image could not be resampled at the given pixel. However the SDSS SIA metadata generally did point us to include the right images, just not the proper boundaries for each one. So after we downloaded the data we could recompute the best source image and that would work. Internally the class we use to find the image to resample a given pixel is called an ImageFinder. The default ImageFinder is the Border ImageFinder. For the SDSS we have created a special ImageFinder, LocalBorder, which iteratively reruns the image finding operation until all of the images that are used in the mosaic are downloaded before we do the image finding. Typically this requires three iterations for large images. However we still only download the images that we actually need, and only once. The image finding is generally pretty fast, so the time penalty for this kludge is modest. We only do the actual resampling with the final data.
However for this to work, the SIA information has to be at least in the correct ball park. Generally it appears that the SIA-specified center of the image is deep within the actual image. In this case
we anticipate that our workaround will at least greatly mitigate if not always cure the problem.
Note that the SDSS problem is particularly acute for those using SkyView-in-a-Jar. Since many popular regions are already available in the cache for the web site, the problem may not show up there. However jar users are building up their own caches from scratch.
The jar file that addresses these issues can be found at http://skyview.gsfc.nasa.gov/v3.2.1/jar/skyview.jar. Note that to get the correct survey description file users will need to use the setting:
xmlurlprefix=http://skyview.gsfc.nasa.gov/v3.2.1/jar/
until this version is released since the default for this setting points to the current release which currently has the older version of the SDSS survey description.