This is a minor release mostly to fix the small glitches and omissions we found in our first operational use of our new release process.
The one substantive change allows DSSx requests to continue when a corrupt tile is found in the underlying data. DSSx data are compressed in 500×500 or 768×768 pixel tiles that are restored as needed when rendering a DSS image. There are about 1000 tiles for each plate scan and we have a total of about 7,000 plate scans. About six years ago we tried to decompress all data and found 19 bad tiles. We were able to get 9 of them updated (see the 2008/02/06 entry in http://skyview.gsfc.nasa.gov/current/help/whatsnew.html). However users could run into one of the 10 remaining bad tiles which heretofore caused the entire SkyView request to fail. Now the bad tile is rendered as zeroes but processing continues and a warning message is given in the results. Thanks to Jim M for noting the problem.
Note that the address for the SkyView home page is now
http://skyview.gsfc.nasa.gov/current/cgi/titlepage.pl. This gives the current version. You can get to earlier versions by replacing
current with a specific version, currently
v3.0.1. We anticipate keeping at least the last few versions around when possible.
During the past 24 hours we have released our new SkyView organization discussed in our prior article. All of the old URLs for queries should continue to work though most are now redirected. We did run into a couple of surprises so that it took us longer to switch all requests over to the new system than expected.
There are a few visible changes to our home page, one new catalog, the Stripe82VLA survey, and a couple of changes that may mostly affect SkyView-in-a-Jar users. While incremental changes in SkyView have often been made with limited or no visibility, we now anticipate that any permanent change will be reflected in the version visible on the home page.
Please let us know if you run into any issues.
Next week we will be making a major new release of SkyView. This is primarily a structural release where we have centralized the organization of SkyView‘s files. Previously the operational SkyView services used code scattered through a number of directories: CGI, jar, scripts, documents…. Now all of these are collected under a single directory and relative references are used. A number of scripts and documents that were used in the operational system were brought fully within our code management system. So it will be easy to have multiple, independent versions of SkyView available simultaneously. SkyView releases will be delivered into directories named after the version used, e.g., this new release is designated
v3.0.0. [Version 1 of SkyView was our original IDL-based system, Version 2 refers to the existing organization of the Java-based code. Typically the middle number in the version increments when new functionality is released, while the third number is updated when we add surveys or make minor bug fixes or updates.] The
current directory will be linked to whatever is the current operational release. Typically users will only ever reference this
current directory but when a new release is made users will still be able to run the old release by explicitly specifying the version in the SkyView URL they invoke (or they can get a specific version of SkyView-in-a-Jar by downloading the jar from the appropriate directory).
While there will be new URL addresses for most pages, all the old addresses for data requests will be linked to the new locations so existing scripts should not be affected.
There are a few changes: The code for reading survey data has been changed so that if the survey data is 4-byte floating point, it need not be converted to 8-byte floating point. This allows use of much larger input files without exceeding memory constraints. It should also speed things up a bit. We’ve also added in some support for hierarchical imaging where we store an image a multiple resolutions. Both of these changes are in preparation for supporting a set of surveys in the GOOOS regions where some of the source data is very high resolution HST data.
We’ve also added one new small survey, the VLA Stripe 82 survey, a VLA survey of a region where the SDSS coverage is especially deep.
You can try out the new version right away. Next week we’ll begin redirecting the standard URLs here. Some of the old code that we have distributed to access SkyView programmatically does not handle redirect requests so the
pskcall URL it invokes uses a server side redirect that is transparent to the client. Other URLs will redirected using HTTP 301 permanent redirects that any modern clients should handle.
Please let us know if you have any concerns or if you run into any problems.
A couple of years ago the Sloan Digital Sky Survey underwent a major reorganization and completely revamped its data. One of the changes that was made in the transition from SDSS DR7 to SDSS DR8 was that new releases were compressed in BZIP2 format which provided a better overall compression ratio.
Unfortunately, the BZIP2 format is not supported by Java native IO libraries in the same way that the GZIP format is. So we needed to make some changes to SkyView to accommodate the new compression. Our first idea was to incorporate a new BZIP2 library into the SkyView JAR. There is a JAR file available through the Apache Commons (there’s a JAR file in the binary downloads) and we tried using that. It worked, but it didn’t take us long to notice that it was quite slow. Much slower than we were used to with GZIP files. Also, the Apache library although very liberally licensed is under a license unlike the other public domain libraries that we have incorporated into the jar (notably the ImageJ library we use to generate quick look images).
When we investigated using the bunzip2 command that we found on our system we noted that this seemed to be much faster — fast enough the decompression no longer dominated the time required to generate images. However it was unclear how commonly users would have access to bunzip2.
Given this situation we decided to provide a dual path to enable users to use bzip2 files. If there is a bunzip2 executable available, then the user can define a logical name BZIP_DECOMPRESSOR to point to that executable. SkyView will spawn off a process to run the decompression using that command. If you don’t tell it, SkyView does not look for the bunzip2 command in any specific location — we certainly don’t know where it would be found on a Windows or Mac though on Unix machines it often seems to be in
If there is no executable available, then SkyView will look for the Apache commons classes. However to minimize the licensing issues we did not include these in the JAR. It’s up to the user of the JAR to add them into the class path. Normally users will execute SkyView as an executable JAR. The executable JAR defines the classpath such that it’s hard to override by a user. Suppose you have the Apache Commons JAR as bzip2.jar, then SkyView needs to be run as
java -cp skyview.jar:bzip2.jar skyview.executive.Imager [regular arguments] …
to get the new compression capabilities into the path. You can’t just do
java -cp bzip2.jar skyview.jar [regular arguments]
So you can use bzip2 files within SkyView though it requires a little work and we might have found better ways to do this.
However we do need to apologize to our users for never documenting these changes in the user’s guide or in the blog till now. It doesn’t do much good to add a capability if no one knows how to use it!
If you have any questions about BZIP2 files please let us know.
SkyView is back up and running! We appreciate your patience while we were offline.
Here’s a random image submitted by a SkyView user to our Image Gallery. Enjoy!
We have updated the VLA FIRST data to include many new observations made available at the MAST archive. These are available immediately at our web site. If you would like to use the new data using SkyView-in-a-Jar please download the latest SkyView jar file.
Posted in Notices
SkyView Survey Data Status Table
You may have noticed that the SkyView homepage has changed. We wanted to add a new feature and decided it was time for a little re-arranging.
The new feature is a table listing SkyView local and remote surveys and their current availability status. Most survey data used for image generation are stored locally on SkyView systems but when images are generated from surveys listed in this table the data are transferred from remote servers. From time to time data availability (both local and remote) is interrupted and image queries fail. We monitor these remote server connections throughout the day. If there are connection issues we update the table and try to notify our contacts at the appropriate institution as soon as possible.
A new release of the Swift BAT all sky survey is now available. This has data from the first 70 months of the Swift mission. It is much more sensitive than the 9 month surveys that we had heretofore. Both signal to noise and flux maps are provided. In the web interface flux maps are provided as links off of SNR map results, but both sets of maps are available directly through the batch interfaces. This is the inverse of what we used to have, but the SNR maps seem the better map to use when asking the question of whether anything is being detected at a given location.
Data are now divided into 8 energy bands (compared to 4 before) and a combined SNR image is also available. The surveys available are:
||Frequency (EHz) |
|Sum (SNR only)
You can specify the survey using either the band or the energy range. E.g.,
BAT-SNR-1, BAT SNR 1, or BAT SNR 14-20 all reference the same data. To get a flux map, substitute Flux for SNR in any of the aliases above, e.g., BAT Flux 14-20.
Note that case is ignored in survey aliases.
Thanks to the BAT team for the data. If you would like access to the nine-month images please let us know.
SkyView now provides access to the Planck all-sky survey data in nine frequency bands from 30 to 857 GHz. Currently we are using the data as released by the Planck team–a single file for each band. This requires SkyView to read relatively large files and can take longer than the typical survey. We are looking at tiling the Planck data to enable more rapid response when users are interested in only a fraction of the sky.
This week we will be adding all-sky surveys from the Planck mission to SkyView. Planck provides nine all-sky images in the range from roughly 30-1000 GeV. While many Planck science goals relate to cosmology, Planck’s resolution approaches that of IRAS, so that Planck images may be of interest for imaging the the far infrared. Planck has roughly twice the resolution and substantially higher sensitivity than WMAP at the frequencies over which they overlap. Planck data extends to almost an order of magnitude higher frequency than the ~100 GeV cutoff for WMAP and the higher frequency data has higher resolution. WMAP does extend to slightly lower frequencies than the longest wavelength data from Planck.
These new surveys represent the first release of Planck data. Data has been downloaded from the IRSA’s Planck archive.
Planck data is stored in SkyView as a single all-sky HEALPix image. For the high frequency data the data file is 600 MB which means that it can take a little while to query Planck images. The three lower frequency bands are a bit faster since they are only 1/4 the size. We will be looking at creating tiles for for the Planck data if the delays seem unacceptable.
The following images give the WMAP-W image and Planck 100 GHz image of the 10 degrees of sky at the Galactic Center. These are at roughly the same frequency. While there is excellent agreement between the two, the superior resolution of the Planck data is immediately apparent.
Congratulations to ESA and the Planck teams in Europe and the US for their very impressive results.
WMAP-W image of the Galactic Center
Planck 100 GHz image of Galactic Center