New Release of SkyView

Next week we will be making a major new release of SkyView. This is primarily a structural release where we have centralized the organization of SkyView‘s files. Previously the operational SkyView services used code scattered through a number of directories: CGI, jar, scripts, documents…. Now all of these are collected under a single directory and relative references are used. A number of scripts and documents that were used in the operational system were brought fully within our code management system. So it will be easy to have multiple, independent versions of SkyView available simultaneously. SkyView releases will be delivered into directories named after the version used, e.g., this new release is designated v3.0.0. [Version 1 of SkyView was our original IDL-based system, Version 2 refers to the existing organization of the Java-based code. Typically the middle number in the version increments when new functionality is released, while the third number is updated when we add surveys or make minor bug fixes or updates.] The current directory will be linked to whatever is the current operational release. Typically users will only ever reference this current directory but when a new release is made users will still be able to run the old release by explicitly specifying the version in the SkyView URL they invoke (or they can get a specific version of SkyView-in-a-Jar by downloading the jar from the appropriate directory).

While there will be new URL addresses for most pages, all the old addresses for data requests will be linked to the new locations so existing scripts should not be affected.

There are a few changes: The code for reading survey data has been changed so that if the survey data is 4-byte floating point, it need not be converted to 8-byte floating point. This allows use of much larger input files without exceeding memory constraints. It should also speed things up a bit. We’ve also added in some support for hierarchical imaging where we store an image a multiple resolutions. Both of these changes are in preparation for supporting a set of surveys in the GOOOS regions where some of the source data is very high resolution HST data.

We’ve also added one new small survey, the VLA Stripe 82 survey, a VLA survey of a region where the SDSS coverage is especially deep.

You can try out the new version right away. Next week we’ll begin redirecting the standard URLs here. Some of the old code that we have distributed to access SkyView programmatically does not handle redirect requests so the pskcall URL it invokes uses a server side redirect that is transparent to the client. Other URLs will redirected using HTTP 301 permanent redirects that any modern clients should handle.

Please let us know if you have any concerns or if you run into any problems.

Posted in Announce, releases | Tagged , , , | 2 Comments

SkyView, Bzip2 and the SDSS

A couple of years ago the Sloan Digital Sky Survey underwent a major reorganization and completely revamped its data. One of the changes that was made in the transition from SDSS DR7 to SDSS DR8 was that new releases were compressed in BZIP2 format which provided a better overall compression ratio.

Unfortunately, the BZIP2 format is not supported by Java native IO libraries in the same way that the GZIP format is. So we needed to make some changes to SkyView to accommodate the new compression. Our first idea was to incorporate a new BZIP2 library into the SkyView JAR. There is a JAR file available through the Apache Commons (there’s a JAR file in the binary downloads) and we tried using that. It worked, but it didn’t take us long to notice that it was quite slow. Much slower than we were used to with GZIP files. Also, the Apache library although very liberally licensed is under a license unlike the other public domain libraries that we have incorporated into the jar (notably the ImageJ library we use to generate quick look images).

When we investigated using the bunzip2 command that we found on our system we noted that this seemed to be much faster — fast enough the decompression no longer dominated the time required to generate images. However it was unclear how commonly users would have access to bunzip2.

Given this situation we decided to provide a dual path to enable users to use bzip2 files. If there is a bunzip2 executable available, then the user can define a logical name BZIP_DECOMPRESSOR to point to that executable. SkyView will spawn off a process to run the decompression using that command. If you don’t tell it, SkyView does not look for the bunzip2 command in any specific location — we certainly don’t know where it would be found on a Windows or Mac though on Unix machines it often seems to be in /usr/bin.

If there is no executable available, then SkyView will look for the Apache commons classes. However to minimize the licensing issues we did not include these in the JAR. It’s up to the user of the JAR to add them into the class path. Normally users will execute SkyView as an executable JAR. The executable JAR defines the classpath such that it’s hard to override by a user. Suppose you have the Apache Commons JAR as bzip2.jar, then SkyView needs to be run as
java -cp skyview.jar:bzip2.jar skyview.executive.Imager [regular arguments] …
to get the new compression capabilities into the path. You can’t just do
java -cp bzip2.jar skyview.jar [regular arguments]

So you can use bzip2 files within SkyView though it requires a little work and we might have found better ways to do this.

However we do need to apologize to our users for never documenting these changes in the user’s guide or in the blog till now. It doesn’t do much good to add a capability if no one knows how to use it!

If you have any questions about BZIP2 files please let us know.

Posted in Discussion | Tagged , , | 2 Comments

We’re Back!

SkyView is back up and running!   We appreciate your patience while we were offline.

Here’s a random image submitted by a SkyView user to our Image Gallery.  Enjoy!

 

Posted in Discussion, Notices | 7 Comments

Updated VLA FIRST Survey

We have updated the VLA FIRST data to include many new observations made available at the MAST archive. These are available immediately at our web site. If you would like to use the new data using SkyView-in-a-Jar please download the latest SkyView jar file.

Posted in Notices | Tagged | Leave a comment

SkyView Main Page Update

SkyView Survey Data Status Table

SkyView Survey Data Status Table

You may have noticed that the SkyView homepage has changed. We wanted to add a new feature and decided it was time for a little re-arranging.

The new feature is a table listing SkyView local and remote surveys and their current availability status. Most survey data used for image generation are stored locally on SkyView systems but when images are generated from surveys  listed in this table the data are transferred from remote servers. From time to time data availability (both local and remote) is interrupted and image queries fail. We monitor these remote server connections throughout the day. If there are connection issues we update the table and try to notify our contacts at the appropriate institution as soon as possible.

Posted in Discussion | Leave a comment

Swift BAT 70 Month Survey Available in SkyView

A new release of the Swift BAT all sky survey is now available. This has data from the first 70 months of the Swift mission. It is much more sensitive than the 9 month surveys that we had heretofore. Both signal to noise and flux maps are provided. In the web interface flux maps are provided as links off of SNR map results, but both sets of maps are available directly through the batch interfaces. This is the inverse of what we used to have, but the SNR maps seem the better map to use when asking the question of whether anything is being detected at a given location.

Data are now divided into 8 energy bands (compared to 4 before) and a combined SNR image is also available. The surveys available are:

Band Energy (keV) Frequency (EHz)
1 14-20 3.38-4.84
2 20-24 4.84-5.80
3 24-35 5.80-8.46
4 35-50 8.46-12.1
5 50-75 12.1-18.1
6 75-100 18.1-24.2
7 100-150 24.2-36.3
8 150-195 36.3-47.2
Sum (SNR only) 14-195 3.38-47.2

You can specify the survey using either the band or the energy range. E.g.,
BAT-SNR-1, BAT SNR 1, or BAT SNR 14-20 all reference the same data. To get a flux map, substitute Flux for SNR in any of the aliases above, e.g., BAT Flux 14-20.

Note that case is ignored in survey aliases.

Thanks to the BAT team for the data. If you would like access to the nine-month images please let us know.

Posted in Announce, Notices | Tagged , , | Leave a comment

Planck Now Available Through SkyView

SkyView now provides access to the Planck all-sky survey data in nine frequency bands from 30 to 857 GHz. Currently we are using the data as released by the Planck team–a single file for each band. This requires SkyView to read relatively large files and can take longer than the typical survey. We are looking at tiling the Planck data to enable more rapid response when users are interested in only a fraction of the sky.

Posted in Announce | Tagged , | 2 Comments

Planck data in SkyView

This week we will be adding all-sky surveys from the Planck mission to SkyView. Planck provides nine all-sky images in the range from roughly 30-1000 GeV. While many Planck science goals relate to cosmology, Planck’s resolution approaches that of IRAS, so that Planck images may be of interest for imaging the the far infrared. Planck has roughly twice the resolution and substantially higher sensitivity than WMAP at the frequencies over which they overlap. Planck data extends to almost an order of magnitude higher frequency than the ~100 GeV cutoff for WMAP and the higher frequency data has higher resolution. WMAP does extend to slightly lower frequencies than the longest wavelength data from Planck.

These new surveys represent the first release of Planck data. Data has been downloaded from the IRSA’s Planck archive.

Planck data is stored in SkyView as a single all-sky HEALPix image. For the high frequency data the data file is 600 MB which means that it can take a little while to query Planck images. The three lower frequency bands are a bit faster since they are only 1/4 the size. We will be looking at creating tiles for for the Planck data if the delays seem unacceptable.

The following images give the WMAP-W image and Planck 100 GHz image of the 10 degrees of sky at the Galactic Center. These are at roughly the same frequency. While there is excellent agreement between the two, the superior resolution of the Planck data is immediately apparent.

Congratulations to ESA and the Planck teams in Europe and the US for their very impressive results.

WMAP galactic center

WMAP-W image of the Galactic Center

Planck 100 GHz image of Galactic Center

Planck 100 GHz image of Galactic Center

Posted in Discussion, Notices | Tagged , , | 1 Comment

Gallery changes

We’re changing the way the User Image Gallery is organized so that you can use it more effectively. Previously we’d set up the gallery so that the last 9 images were on the first page, the previous nine on the next and so forth. It could be very difficult to get to specific gallery images if you didn’t remember the exact URL to them. Now we organize the gallery by date and you can easily jump to a specific day. If more than 20 gallery images are taken on a day, then there are multiple pages for the day. You can still browse the latest images, or you can jump to any date since we started the gallery on October 7, 2008.

There will be a day or two as things are reorganized when new gallery images may not appear. We apologize for the inconvenience and anticipate getting everything working soon.

Please let us know if you have any questions or suggestions. We hope this makes it easier for everything to use the often beautiful and sometimes perplexing images that you have saved.

Posted in Notices | Tagged | Leave a comment

Error accessing GRANT-SIGMA survey

A typo in the survey description had disabled the GRANT/SIGMA survey. This is now fixed. I apologize for the error.

Posted in Notices | Tagged , , | Leave a comment