SkyView erroneously linking to SDSS DR8 data

Although the survey documentation indicates that SkyView has been using the SDSS DR9, it has been using the DR8 data internally. The main difference is that there are some astrometric corrections in the DR9. The releases since the DR9 have not affected the imaging survey.

The SkyView web site should now be using the DR9 and we will be releasing a new SkyView -in-a-Jar shortly. Our apologies for the error.

Posted in Notices | Comments Off on SkyView erroneously linking to SDSS DR8 data

SkyView v3.5.1: CopyWCS and SIA fixes

We have just released SkyView v3.5.1 which has a couple of fixes that shouldn’t affect most people. A bug in the handling of the CopyWCS keyword has been fixed. CopyWCS is a keyword which allows a SkyView-in-a-Jar user to specify that the output geometry for some request is to be the same as some existing image. E.g., if you have a WISE image of a given region and you want a DSS image of exactly the same spot, you can use CopyWCS instead of position, size, pixels, scale, projection, coordinates and such.

The other change is to some of the metadata returned when making VO Simple Image Access requests. All input parameters are now described as strings which is consistent with SIA specification.

Posted in releases | Tagged | Leave a comment

SkyView V3.5.0 released: Planck DR3 Data

We have just made SkyView v3.5.0 the default version for the system. In addition to the changes discussed in the prior post, we’ve updated the Planck surveys to use the latest DR3 release.

This version also fixes a bug that a user discovered where the Spline sampler would sometimes fail with some surveys — notably surveys where the source data is in the HEALPix projection.

Posted in Notices, releases | Tagged , | Comments Off on SkyView V3.5.0 released: Planck DR3 Data

SkyView v3.5.0: Compound surveys

We announced a test release of SkyView v3.4.5 a few weeks ago, but in trying out the new version we recognized that it did not do what we wanted. In particular we had added new surveys that were the Stokes parameters for Planck data. These included that base Q and U Stokes data as well as derived quantities like the total polarized intensity, the polarization fraction and the position angle of the polarization vector.

While the data were being extracted properly, the behavior of our data when it was smoothed was not really what we expected. Basically we were computing these derived quantities right away and smoothing the derived pixels. However this emphasizes the noise in the data. What we should have been doing is smoothing the Q and U (and possibly I) data and then calculating the appropriate functions on these smoothed images.

To enable just this behavior, we added a new kind of SkyView survey: a compound survey. Compound surveys include one or more base component surveys, and a formula for combining the bases to create the ultimate result. E.g., for the total polarization, the component surveys are the Q and U surveys, and these are combined by an element that calculates sqrt(q2+u2) at each point.

When we ask for the total polarization we first compute the Q and U images for the geometry the user asked for. This includes any smoothing, or any other processing that affects the pixel value. Then we compute the final image by applying the combining element). For this final image we calculate any grid, contour or other overlays.

This is a fairly substantive change so it’s taken us a while to test it out. We also decided it meant that we’d be making this 3.5.0 and abandoning our released version of 3.4.5. We hope to release this new version in the coming week.

Posted in Uncategorized | Leave a comment

SkyView V3.4.5: New Surveys and Projection

We’re announcing a test release of SkyView V3.4.5. The new version includes the DRAO 22 MHz survey of most of the sky. This is now the lowest energy survey in SkyView. See Rogers et al, 1999 for details. In addition to missing much of the southern hemisphere the area around a few of the brightest radio sources has been excised since the survey was primarily intended to study diffuse emission.

We’ve also updated the Planck survey data and added the ability to get various polarization measures for most of the Planck frequencies. For all but the two highest energy bands users can get the total intensity, I, the Stokes polarization parameters, Q and U, the total polarized intensity, PI, the fractional polarization, PI/I, and the position angle, PA, of the polarization. Note that the position angle is currently with respect to Galactic coordinates even if you sample the survey in equatorial coordinates — we’re working on updates that will transform the angles appropriately. Future versions of SkyView should have new ways to display these data, e.g., to show positional angle overlays over other maps.

Finally, we also added the Mollweide (Mol) projection as one of SkyView‘s supported projections. This is a fixed projection similar to the Hammer-Aitoff projection in that it is an equal area projection that projects the entire sky into an ellipse. However, unlike the Aitoff, lines of constant latitude are projected into horizontal lines in the projection. Apparently this makes it popular in the CMB community.

Update: In testing out all of the new Planck image planes, we finally had to confront the known issue of the slow generation of Planck images — we’d mentioned this when we first added Planck data. Our test images are usually just a small fraction of the sky, but because of the way Planck data are stored, a single HEALPix file, SkyView was reading in the entire image. We’ve added a new feature to this release where a HEALPix image can be split into tiles and only the tiles that are needed for a given request will be read. We’re currently splitting the HEALPix image into 48 tiles. For a typical small image the new approach can be 10-20 times faster than reading the entire survey.

At least from the perspective of our software this is a bit different than the tiling used in many of our other surveys. We usually have separate FITS files covering regions typically with some overlap between the images. Internally in SkyView each of these files is treated as a separate Image object. SkyView mosaics these images together. In this new feature, there is only a single Image covering the entire sky but the data from that image is segmented and we read a segment only at the first request for data from it.

Posted in Announce, releases | Tagged , , , | Leave a comment

Updated survey documentation for SkyView v3.4.3

We have updated the survey documentation page. It had gotten out of date and was missing some survey information. The Contents listing the available surveys by wavelength regime had been incomplete but is now generated automatically from all of the meta data.  The figure summarizing the surveys in resolution-frequency space has been updated. (No core functionality of SkyView has changed.)

Posted in Documentation, Notices, Uncategorized | Comments Off on Updated survey documentation for SkyView v3.4.3

SkyView in a Jar file size fixed

The SkyView in a Jar file has been overly large lately. (We mistakenly included a bunch of unnecessary files when building the jar last time.) This has now been fixed so that the file size is back down to roughly 7 MB. The code has not changed.

Posted in Uncategorized | Leave a comment

Amazon AWS access to SkyView blocked.

One or more users running on Amazon Web Services accounts have been sending a steady and unsustainable stream of requests to SkyView, overloading the servers and severely degrading SkyView performance for others. We’ve temporarily blocked some SkyView access from any Amazon AWS doamin. We expect that this issue not an intentional attack so we have tried to limit the impact and will remove the block in few days.

Generally speaking we advise users to queue their requests, only submitting the next request when the previous one finishes. While one can gain a bit with two queues anything more than that can certainly affect overall SkyView performance. Alas, this is a pretty small site with just a couple of machines underlying everything.

Our apologies to those who have been affected.

Posted in Notices | Leave a comment

WISE and 2MASS move to HTTPS

Access to data at IRSA recently has moved to requiring HTTPS URLs. This has broken SkyView’s access to the WISE and 2MASS surveys. We have just updated the URLs in the current version of SkyView to re-enable access to these popular surveys.

Note that since this bypassed our regular release procedures users of SkyView in a Jar may wish to delete any wise.xml.gz or 2mass.xml.gz files in their skycache directories to ensure that they get the latest survey description files with the updated URLs.

Posted in Notices | Tagged , , | Leave a comment

Recent Heavy SkyView Usage

For the past several weeks there has been very intense usage of SkyView by a small number of users which has degraded responsiveness for other users. Most of the requests seem to be coming from virtual machines using Amazon Web Services so we are not clear who or what is causing the issue. The requests seem temporarily at least to have abated but we are looking at options for throttling or forbidding access should they start up again.

We apologize to our users for the recent poor performance and hope to be nimbler in addressing such concerns in the future.

If you are planning on making some large number of SkyView requests you may find that using SkyView-in-a-Jar is more convenient and less intrusive for other users. You download the jar file and run requests directly. This will often be more efficient — e.g., you can generate just FITS files if that is what you want — and allows many custom options not available through the web interface. With the exception of the Mellinger Optical survey, all surveys available on the Web interface can be accessed using SkyView in a Jar.

Posted in Discussion, Notices | Tagged , | Leave a comment