SkyView v3.5.3 updates and bug fixes

SkyView has been updated to v3.5.3 today.  This version has changes to the following surveys:  GLEAM now includes the Galactic plane data as well as the Magellanic Clouds; and the UKIDSS survey is now using DR11 (previously DR8) and including the hydrogen H_2 line (“1-0S1”) where available.

Bug fixes include:  exception handling when requests to remote servers time out;  command-line parsing of Planck polarization compound observables (e.g, “planck-030 PI”);  a version option for the command line skyview-in-a-jar to see the code version without having to open the jar (just ‘java -jar skyview.jar version’).

Posted in Uncategorized | Tagged , | Comments Off on SkyView v3.5.3 updates and bug fixes

SkyView V3.5.2: Updates and Bug Fixes

SkyView version 3.5.2 is now available for testing. There are two changes to surveys and one to the underlying code. The data for the GLEAM surveys has been updated with substantial improvements in coverage. The problem with the SDSS surveys noted in the prior post is fixed in the associated SkyView-in-a-Jar file everything points to DR9 now.

The code change involves the Spline Sampler. Previously this sampler did nothing special when it encountered a NaN in the input data. It simply used that value. The way the Spline sampler works is that if first computes a grid of spline coefficients and then uses those to interpolate at requested positions. However any arithmetic operation involving a NaN results in a NaN, so if there are NaNs at the edges of an image, they’ll tend to propagate throughout the image.

This is the case for the GB6 radio survey (at least), so if you used the Spline sampler on this survey, nothing was returned — or more precisely an array of NaNs. David Green noted this behavior and we’ve changed how SkyView handles NaNs in splines.

There is now a new setting, SplineNaNs, which you can use in the SkyView Jar. The default — used in Web pages — is SplineNaNs=min. This replaces all NaNs with a value 10% of the total range less than the minimum value. Other options are max, which uses a value 10% greater than the maximum value, skip, which duplicates the prior behavior (i.e., we do nothing to handle the NaNs). Or you can specify a number (other than 1) to use as the value for the NaNs.

We’re still testing v3.5.2 but it should be released in a few days.

Posted in Announce, Discussion, Notices, releases | Tagged , , | Comments Off on SkyView V3.5.2: Updates and Bug Fixes

SkyView erroneously linking to SDSS DR8 data

Although the survey documentation indicates that SkyView has been using the SDSS DR9, it has been using the DR8 data internally. The main difference is that there are some astrometric corrections in the DR9. The releases since the DR9 have not affected the imaging survey.

The SkyView web site should now be using the DR9 and we will be releasing a new SkyView -in-a-Jar shortly. Our apologies for the error.

Posted in Notices | Comments Off on SkyView erroneously linking to SDSS DR8 data

SkyView v3.5.1: CopyWCS and SIA fixes

We have just released SkyView v3.5.1 which has a couple of fixes that shouldn’t affect most people. A bug in the handling of the CopyWCS keyword has been fixed. CopyWCS is a keyword which allows a SkyView-in-a-Jar user to specify that the output geometry for some request is to be the same as some existing image. E.g., if you have a WISE image of a given region and you want a DSS image of exactly the same spot, you can use CopyWCS instead of position, size, pixels, scale, projection, coordinates and such.

The other change is to some of the metadata returned when making VO Simple Image Access requests. All input parameters are now described as strings which is consistent with SIA specification.

Posted in releases | Tagged | Leave a comment

SkyView V3.5.0 released: Planck DR3 Data

We have just made SkyView v3.5.0 the default version for the system. In addition to the changes discussed in the prior post, we’ve updated the Planck surveys to use the latest DR3 release.

This version also fixes a bug that a user discovered where the Spline sampler would sometimes fail with some surveys — notably surveys where the source data is in the HEALPix projection.

Posted in Notices, releases | Tagged , | Comments Off on SkyView V3.5.0 released: Planck DR3 Data

SkyView v3.5.0: Compound surveys

We announced a test release of SkyView v3.4.5 a few weeks ago, but in trying out the new version we recognized that it did not do what we wanted. In particular we had added new surveys that were the Stokes parameters for Planck data. These included that base Q and U Stokes data as well as derived quantities like the total polarized intensity, the polarization fraction and the position angle of the polarization vector.

While the data were being extracted properly, the behavior of our data when it was smoothed was not really what we expected. Basically we were computing these derived quantities right away and smoothing the derived pixels. However this emphasizes the noise in the data. What we should have been doing is smoothing the Q and U (and possibly I) data and then calculating the appropriate functions on these smoothed images.

To enable just this behavior, we added a new kind of SkyView survey: a compound survey. Compound surveys include one or more base component surveys, and a formula for combining the bases to create the ultimate result. E.g., for the total polarization, the component surveys are the Q and U surveys, and these are combined by an element that calculates sqrt(q2+u2) at each point.

When we ask for the total polarization we first compute the Q and U images for the geometry the user asked for. This includes any smoothing, or any other processing that affects the pixel value. Then we compute the final image by applying the combining element). For this final image we calculate any grid, contour or other overlays.

This is a fairly substantive change so it’s taken us a while to test it out. We also decided it meant that we’d be making this 3.5.0 and abandoning our released version of 3.4.5. We hope to release this new version in the coming week.

Posted in Uncategorized | Leave a comment

SkyView V3.4.5: New Surveys and Projection

We’re announcing a test release of SkyView V3.4.5. The new version includes the DRAO 22 MHz survey of most of the sky. This is now the lowest energy survey in SkyView. See Rogers et al, 1999 for details. In addition to missing much of the southern hemisphere the area around a few of the brightest radio sources has been excised since the survey was primarily intended to study diffuse emission.

We’ve also updated the Planck survey data and added the ability to get various polarization measures for most of the Planck frequencies. For all but the two highest energy bands users can get the total intensity, I, the Stokes polarization parameters, Q and U, the total polarized intensity, PI, the fractional polarization, PI/I, and the position angle, PA, of the polarization. Note that the position angle is currently with respect to Galactic coordinates even if you sample the survey in equatorial coordinates — we’re working on updates that will transform the angles appropriately. Future versions of SkyView should have new ways to display these data, e.g., to show positional angle overlays over other maps.

Finally, we also added the Mollweide (Mol) projection as one of SkyView‘s supported projections. This is a fixed projection similar to the Hammer-Aitoff projection in that it is an equal area projection that projects the entire sky into an ellipse. However, unlike the Aitoff, lines of constant latitude are projected into horizontal lines in the projection. Apparently this makes it popular in the CMB community.

Update: In testing out all of the new Planck image planes, we finally had to confront the known issue of the slow generation of Planck images — we’d mentioned this when we first added Planck data. Our test images are usually just a small fraction of the sky, but because of the way Planck data are stored, a single HEALPix file, SkyView was reading in the entire image. We’ve added a new feature to this release where a HEALPix image can be split into tiles and only the tiles that are needed for a given request will be read. We’re currently splitting the HEALPix image into 48 tiles. For a typical small image the new approach can be 10-20 times faster than reading the entire survey.

At least from the perspective of our software this is a bit different than the tiling used in many of our other surveys. We usually have separate FITS files covering regions typically with some overlap between the images. Internally in SkyView each of these files is treated as a separate Image object. SkyView mosaics these images together. In this new feature, there is only a single Image covering the entire sky but the data from that image is segmented and we read a segment only at the first request for data from it.

Posted in Announce, releases | Tagged , , , | Leave a comment

Updated survey documentation for SkyView v3.4.3

We have updated the survey documentation page. It had gotten out of date and was missing some survey information. The Contents listing the available surveys by wavelength regime had been incomplete but is now generated automatically from all of the meta data.  The figure summarizing the surveys in resolution-frequency space has been updated. (No core functionality of SkyView has changed.)

Posted in Documentation, Notices, Uncategorized | Comments Off on Updated survey documentation for SkyView v3.4.3

SkyView in a Jar file size fixed

The SkyView in a Jar file has been overly large lately. (We mistakenly included a bunch of unnecessary files when building the jar last time.) This has now been fixed so that the file size is back down to roughly 7 MB. The code has not changed.

Posted in Uncategorized | Leave a comment

Amazon AWS access to SkyView blocked.

One or more users running on Amazon Web Services accounts have been sending a steady and unsustainable stream of requests to SkyView, overloading the servers and severely degrading SkyView performance for others. We’ve temporarily blocked some SkyView access from any Amazon AWS doamin. We expect that this issue not an intentional attack so we have tried to limit the impact and will remove the block in few days.

Generally speaking we advise users to queue their requests, only submitting the next request when the previous one finishes. While one can gain a bit with two queues anything more than that can certainly affect overall SkyView performance. Alas, this is a pretty small site with just a couple of machines underlying everything.

Our apologies to those who have been affected.

Posted in Notices | Leave a comment