A couple of years ago the Sloan Digital Sky Survey underwent a major reorganization and completely revamped its data. One of the changes that was made in the transition from SDSS DR7 to SDSS DR8 was that new releases were compressed in BZIP2 format which provided a better overall compression ratio.
Unfortunately, the BZIP2 format is not supported by Java native IO libraries in the same way that the GZIP format is. So we needed to make some changes to SkyView to accommodate the new compression. Our first idea was to incorporate a new BZIP2 library into the SkyView JAR. There is a JAR file available through the Apache Commons (there’s a JAR file in the binary downloads) and we tried using that. It worked, but it didn’t take us long to notice that it was quite slow. Much slower than we were used to with GZIP files. Also, the Apache library although very liberally licensed is under a license unlike the other public domain libraries that we have incorporated into the jar (notably the ImageJ library we use to generate quick look images).
When we investigated using the bunzip2 command that we found on our system we noted that this seemed to be much faster — fast enough the decompression no longer dominated the time required to generate images. However it was unclear how commonly users would have access to bunzip2.
Given this situation we decided to provide a dual path to enable users to use bzip2 files. If there is a bunzip2 executable available, then the user can define a logical name BZIP_DECOMPRESSOR to point to that executable. SkyView will spawn off a process to run the decompression using that command. If you don’t tell it, SkyView does not look for the bunzip2 command in any specific location — we certainly don’t know where it would be found on a Windows or Mac though on Unix machines it often seems to be in
If there is no executable available, then SkyView will look for the Apache commons classes. However to minimize the licensing issues we did not include these in the JAR. It’s up to the user of the JAR to add them into the class path. Normally users will execute SkyView as an executable JAR. The executable JAR defines the classpath such that it’s hard to override by a user. Suppose you have the Apache Commons JAR as bzip2.jar, then SkyView needs to be run as
java -cp skyview.jar:bzip2.jar skyview.executive.Imager [regular arguments] …
to get the new compression capabilities into the path. You can’t just do
java -cp bzip2.jar skyview.jar [regular arguments]
So you can use bzip2 files within SkyView though it requires a little work and we might have found better ways to do this.
However we do need to apologize to our users for never documenting these changes in the user’s guide or in the blog till now. It doesn’t do much good to add a capability if no one knows how to use it!
If you have any questions about BZIP2 files please let us know.
SkyView is back up and running! We appreciate your patience while we were offline.
Here’s a random image submitted by a SkyView user to our Image Gallery. Enjoy!
We have updated the VLA FIRST data to include many new observations made available at the MAST archive. These are available immediately at our web site. If you would like to use the new data using SkyView-in-a-Jar please download the latest SkyView jar file.
Posted in Notices
SkyView Survey Data Status Table
You may have noticed that the SkyView homepage has changed. We wanted to add a new feature and decided it was time for a little re-arranging.
The new feature is a table listing SkyView local and remote surveys and their current availability status. Most survey data used for image generation are stored locally on SkyView systems but when images are generated from surveys listed in this table the data are transferred from remote servers. From time to time data availability (both local and remote) is interrupted and image queries fail. We monitor these remote server connections throughout the day. If there are connection issues we update the table and try to notify our contacts at the appropriate institution as soon as possible.
A new release of the Swift BAT all sky survey is now available. This has data from the first 70 months of the Swift mission. It is much more sensitive than the 9 month surveys that we had heretofore. Both signal to noise and flux maps are provided. In the web interface flux maps are provided as links off of SNR map results, but both sets of maps are available directly through the batch interfaces. This is the inverse of what we used to have, but the SNR maps seem the better map to use when asking the question of whether anything is being detected at a given location.
Data are now divided into 8 energy bands (compared to 4 before) and a combined SNR image is also available. The surveys available are:
||Frequency (EHz) |
|Sum (SNR only)
You can specify the survey using either the band or the energy range. E.g.,
BAT-SNR-1, BAT SNR 1, or BAT SNR 14-20 all reference the same data. To get a flux map, substitute Flux for SNR in any of the aliases above, e.g., BAT Flux 14-20.
Note that case is ignored in survey aliases.
Thanks to the BAT team for the data. If you would like access to the nine-month images please let us know.
SkyView now provides access to the Planck all-sky survey data in nine frequency bands from 30 to 857 GHz. Currently we are using the data as released by the Planck team–a single file for each band. This requires SkyView to read relatively large files and can take longer than the typical survey. We are looking at tiling the Planck data to enable more rapid response when users are interested in only a fraction of the sky.
This week we will be adding all-sky surveys from the Planck mission to SkyView. Planck provides nine all-sky images in the range from roughly 30-1000 GeV. While many Planck science goals relate to cosmology, Planck’s resolution approaches that of IRAS, so that Planck images may be of interest for imaging the the far infrared. Planck has roughly twice the resolution and substantially higher sensitivity than WMAP at the frequencies over which they overlap. Planck data extends to almost an order of magnitude higher frequency than the ~100 GeV cutoff for WMAP and the higher frequency data has higher resolution. WMAP does extend to slightly lower frequencies than the longest wavelength data from Planck.
These new surveys represent the first release of Planck data. Data has been downloaded from the IRSA’s Planck archive.
Planck data is stored in SkyView as a single all-sky HEALPix image. For the high frequency data the data file is 600 MB which means that it can take a little while to query Planck images. The three lower frequency bands are a bit faster since they are only 1/4 the size. We will be looking at creating tiles for for the Planck data if the delays seem unacceptable.
The following images give the WMAP-W image and Planck 100 GHz image of the 10 degrees of sky at the Galactic Center. These are at roughly the same frequency. While there is excellent agreement between the two, the superior resolution of the Planck data is immediately apparent.
Congratulations to ESA and the Planck teams in Europe and the US for their very impressive results.
WMAP-W image of the Galactic Center
Planck 100 GHz image of Galactic Center
We’re changing the way the User Image Gallery is organized so that you can use it more effectively. Previously we’d set up the gallery so that the last 9 images were on the first page, the previous nine on the next and so forth. It could be very difficult to get to specific gallery images if you didn’t remember the exact URL to them. Now we organize the gallery by date and you can easily jump to a specific day. If more than 20 gallery images are taken on a day, then there are multiple pages for the day. You can still browse the latest images, or you can jump to any date since we started the gallery on October 7, 2008.
There will be a day or two as things are reorganized when new gallery images may not appear. We apologize for the inconvenience and anticipate getting everything working soon.
Please let us know if you have any questions or suggestions. We hope this makes it easier for everything to use the often beautiful and sometimes perplexing images that you have saved.
Posted in Notices
A typo in the survey description had disabled the GRANT/SIGMA survey. This is now fixed. I apologize for the error.
Posted in Notices
Tagged bug, GRANAT, SIGMA
SkyView users may now generate images using the public UKIRT Infrared Deep Sky Survey (UKIDSS) data. While UKIDSS data is available in only a fraction (14%) of the sky, it goes much deeper and has higher resolution than the 2MASS data in the same wavebands. E.g., here is a comparison of 3-color images of the Galactic Center region (1000 pixels with a 0.1 degree field of view) in 2MASS and UKIDSS generated by straightforward SkyView requests (using a Clip resampler is the only non-default).
A 3-color image of the Galactic Center in 2MASS KHJ colors
A 3-color image of the Galactic Center in UKIDSS KHJ
It’s easy to see how much deeper the UKIDSS data goes. SkyView uses the latest public release for the UKIDSS data. This is the DR8 release for most of the sky, and the DR7 for the UKIDSS Galactic Plane Survey. Many thanks to the UKIDSS team including Mike Read and Mark Holliman in setting up access for us. UKIDSS coverage is different in the different bands while the survey is ongoing and the Y band coverage will always be limited.
UKIDSS data is served by the Wide Field Astronomy Unit (WFAU) at the Institute for Astronomy in the University of Edinburgh. SkyView accesses the data using the Virtual Observation Simple Image Access protocol and caches images as users request them. As the cache of images at SkyView grows the time to generate images will diminish.
Internally UKIDSS data required that SkyView add support for the ZPN, zenithal polynomial, projection which is what UKIDSS uses. Another article will discuss use of the ZPN projection.