SkyView V3.3.2: Links to Mosaicking Inputs

Version 3.3.2 of SkyView has been released with two significant changes: updated memory limits, and links to data inputs.

The parameters used to set the heap size limits for SkyView web requests have been updated so that users will generally be able to create larger images than before — although the exact size is a complex combination of he source data format, the requested output region and other traffic on the site. Images up to 10,000 x 10,000 pixels should be feasible though you may run out of time rather than memory — the connection will drop if nothing happens in 20 minutes. One particular problem was that SkyView ran out of memory if you requested multiple Planck surveys (which are very large single files). You can now retrieve as many Planck datasets as you like! Thanks for Ignacio Cisneros for alerting us to this problem.

The big change for users is a new button that will appear below every image. One of the defining characteristics of SkyView is that it always mosaicks and resamples the input survey data into whatever geometry the user requests. We now provide a button so users can easily see what input data was used for this resampling/mosaicking. This will usually give you a link to the source data that was used in generating the SkyView image. This isn’t necessarily the fundamental source data, it’s just whatever dataset SkyView uses for a particular survey. Currently there are five cases depending on how the data was derived:

  1. From the DSS (or DSS2) plate scans. Links to the directories in which the DSS tiles are found are given.
  2. From a HiPS image (see the recent releases). A link to the base HiPS directory and the properties file for the HiPS is given. Note however that the base directory is not required to be a working URL itself.
  3. From FITS files where SkyView keeps a complete copy of the survey locally. Links to the SkyView survey copy is given.
  4. From FITS files which have been copied from a remote archive into a SkyView cache. Links to the cached files are given.
  5. From FITS files which for copyright or technical reasons cannot be distributed to the public. No links are given.

Two caveats: Many of these datasets have some level of copyright. Please look at the documentation for the survey for more information. Also, to forestall a profusion of browser windows, we have defined a particular browser window in which the link information will be displayed. If you use a tabbed browser, then the second and subsequent times you click on one of the link buttons the tab may be updated, but probably will not be given focus. So it may look like nothing has happened….

You may also note that the detailed SkyView users guide is now available directly from the home page. You don’t need to go into the help pages to see it.

Internally this version considerably reorganizes the various Mosaicker classes minimizing duplication of code. The listing of which files were used in the FITS headers has been somewhat reorganized and now gives the number of pixels resampled from each image.

As always if you have questions or comments please let us know.

Posted in Announce, releases | Tagged , , | Leave a comment

SkyView V3.3.1: Swift UVOT, UltraVista and CFHT HiPS data

We are about to release SkyView version 3.3.1.  This new version includes extended support for HiPS data and has many new surveys.  The primary new datasets are counts, intensity and exposure maps for the Swift UVOT instrument in seven filters.  All data for the first 11.5 years of the mission have been combined into HiPS for each filter.  For the most observed filters a bit under 3% of the sky is observed.  There is a fair bit of overlap but also many differences in the coverage of the various filters.

m81uvot

M81 in UVOT U, UVW1 and UVW2 filters

We have also added in a number of other HIPS surveys from the UltraVista and CFHT surveys. These data are just a few of the many data in Hiearchical Progessive Survey (HiPS) format that are currently available through the web. We anticipate making more available in future data. HiPS joins the VO Simple Image Access Protocol in enabling SkyView to access data from remote
sites.

Caching of HiPS data is now supported and some some issues that disabled special access to a UVOT overlay capability have been resolved.

Posted in Announce, releases | Tagged , , , , | Leave a comment

SkyView v3.3.0: Swift XRT surveys

Versions 3.3.0 of SkyView was released today.  This includes an all-sky mosaic of the Swift XRT observations.  Over 129,000 XRT observations were combined to give coverage of just under 6% of the sky.  A set of three surveys were produced: the combined counts, the total exposure, and the intensity map that is the ratio of the first two.   This image ofM31 combines the XRT intensity map with the WISE 22 micron image — contrasting the very different sources of X-ray and infrared emission.

XRT and WISE image of M31

XRT Intensity (green and blue) and WISE (red) in a 1.2 degree field centered on M31.

A few other changes were made in the release, notably many references to HTTP  on our sites were changed to HTTPS.   Although there should be an automated redirection, we are trying to purge the old HTTP addresses since occasionally they cause problems.

Posted in Announce, Notices, releases | Tagged , , | Leave a comment

SkyView and HiPS

Next week we’ll be releasing version 3.3.0 of SkyView.  The big change from the user perspective will be the new Swift XRT combined image.  This is similar in spirit to the venerable PSPC surveys — we’ve added all of the XRT counts and exposures and created all sky maps based on the sums — and an intensity map as the ratio of the two.   From the programming side though the big change is that SkyView will be supporting a new survey format: the Hierarchical Progressive Survey or HiPS.

Traditionally SkyView simply takes a set of images which normally have overlaps and mosaics them to the users specifications.   So long as there is a little overlap between them, we don’t care too much about how the source image tiles are organized.  In HiPS the survey data is required to match a very specific tiling. The organization is strongly tied to the HEALPix pixelization.  This is a way of defining pixels in the sky such that pixels are all of equal area and the average latitude values of the pixels are restricted to a relatively small set — i.e., the pixels are centered on lines of constant latitude.  Both these characteristics are convenient in analysis of CMB datasets where HEALPix originated.  The pixelization starts with 12 top level (or level 0) pixels and then breaks them up recursively into higher level pixels.  So there are 48 level 1 pixels, 192 level 2 pixels, 768 level three pixels and so on.  In HiPS each tile corresponds to a specific HEALPix pixel but the tile also will have Nx and Ny that is some power of 2. So all of the pixels in the tile are also HEALPix pixels but of a higher order.

http://healpix.sourceforge.net/images/gorski_f1.jpg

Levels 0-3 HEALpix pixels.

HEALPix pixelizaton

E.g., in the Swift XRT HiPS images the highest resolution tiles correspond to the 786,432 level 8 HEALPix pixels.  Each tile is a 512×512 image so that each pixel in a tile image  is a level 17 HEALPix pixel.  A client can find the level 8 pixels in their region of interest and download only the tiles of interest.

The other big feature of HiPS is that not only do we have the high-resolution tiles, we also precompute tiles of lower resolution.  E..g., we also have level 7 tiles where we’ve averaged the high-resolution pixels 2×2 and also combined four of the tiles into one tile that covers four times the area.   For the Swift data we’ve generated tiles down to level 3 and also a specially formatted all sky image with even lower resolution.  These choices mimic those of the  CDS Hipsgen tool (including with the latest versions of Aladin) which we used to generate the exposure maps.  We used our own software to build the counts and intensity maps.

A client reading a HiPS file can easily zoom in and out a la Google Earth and Sky or the WWT.  These technology is supported by Aladin and Aladin-lite and can be seen in the ESASky and the MAST data portal.  Our goal in SkyView is not so much providing this kind of dynamical interface as enabling the user to generate images to a clear specification, but we use the multiple resolutions too.  SkyView resamples the lowest resolution higher than the user requested resolution.

Many archives have generated HiPS of their datasets and provide them on the Web. We anticipate providing access to many of these datasets in upcoming versions of SkyView — and we’ll be publishing any HiPS we generate.  If you have a particular interest in getting some dataset into our system please let us know.

 

Posted in Discussion, Documentation | Tagged , , , | 1 Comment

Code 400, “Bad Request” error

Our system administrators have resolved a problem where users would occasionally get a code 400, “Bad Request” message when accessing SkyView (and other HEASARC sites). Tracking tools required by NASA policy were creating cookies that were too large for our servers to accept.  See  this earlier article (https://skyview.gsfc.nasa.gov/blog/index.php/2014/07/16/v3-1-7-skyview-add-google-analytics/) for more background.  We apologize for any inconvenience caused. Please let us know if you are still seeing issues.

Posted in Discussion | Tagged , | Leave a comment

SkyView V3.2.3 Beware the Ides of March!

Over the next few days (as early as noon 3/15) the SkyView servers will be transitioning to requiring secure HTTP, i.e., HTTPS, URLs to access SkyView services.  This is part of a mandated transition of all NASA (and indeed all federal) web sites. Web users should see little difference.  If you come in using a non-HTTPS URL you will be automatically redirected to the correct URL.  However users who access SkyView through batch services may see a difference since the software that accesses SkyView may not automatically follow the redirect.  In particular, Java’s network library  will not automatically follow redirects which transition from HTTP to HTTPS.   Java (and any other framework that exhibits similar behavior) tools can be programmed to explicitly follow redirects, or the HTTPS URL can be given directly.

Since SkyView-in-a-Jar is written in Java it is sensitive to this change.  When you are using SkyView-in-a-Jar it retrieves survey data from SkyView using information in survey description files. Until version 3.2.3 those files had HTTP URLs.  With this version all the survey description files have been updated to use HTTPS URLs.  We have updated the survey XML files to use HTTPS addresses and it had been our thought that users of recent jar files would be OK since the program would detect the new survey description files and download them automatically.  However we have since recognized that the base URL for the survey description files needs to be updated.  So we recommend all SkyView-in-a-Jar users update to the latest version of the Jar (at least version 3.2.3).

Update: 3/17:  Small updates have been made to the Jar that may fix issues seen by Windows users of SkyView-in-a-Jar.  Our apologies for not catching these earlier.

Posted in Announce, releases | Tagged , | Leave a comment

SkyView update: v3.2.2 including HI4PI data

On Tuesday Dec. 13, 2016, we updated SkyView to v3.2.2 in order to include the new HI4PI radio survey. This dataset is the atomic neutral hydrogen (HI) column density map derived from HI4PI (|Vlsr| < 600 km/s). It includes the Effelsberg-Bonn HI Survey (EBHIS) of the northern sky but completes the sky coverage with the addition of southern sky data from Parkes.

Note that a previously-downloaded jar file of v3.2.1 should still pick up this new survey automatically. This update also includes some changes to the internal meta data handling, but this should be transparent to users.

Posted in Announce, releases, Uncategorized | 2 Comments

SkyView downtime October 4, 2016

SkyView was down from about 9:00am to 2:30pm today due to a problem encountered while updating the SkyView servers.  SkyView operations should now be back to normal. We apologize for the inconvenience.

Posted in Discussion, Notices | Leave a comment

NASA SkyView vs SkyView

We often get email requesting  help downloading and using SkyView on the iPhone and other devices.  We thought we would try to clear up some apparent confusion.  These questions refer to a smart phone app that is not associated with NASA SkyView but has the same name – SkyView (or SkyView Free).  Information about this app can be found at the Apple App Store and the Terminal Eleven site.

Posted in Discussion | 2 Comments

SkyView V3.2.1: SDSS and AKARI fixes

On Wednesday we released SkyView version 3.2.1 to fix the SDSS survey. It includes a new capability where when we are using an SIA service we can apply a regular expression transformation to the URLs being returned. We use this to transform SDSS image tile URLs from http://www.sdss3.org/… to https://www.sdss.org/…. The setting SiapUrlTransform expects two comma-separated fields which are interpreted as arguments to the Java String.replaceAll(String,String) method in modifying the input URL string.

We’ve had this fix for several days but were unable to release it. SkyView was unable to see remote HTTPS URLs due to GSFCs firewall settings. As soon as that was addressed by our system managers we made the release.

Shortly after we communicated some concerns with the publishers of the AKARI data regarding issues with their SIA service. They were using Ecliptic coordinates in places where ICRS coordinates were required. SkyView had implemented special code to address this. The AKARI team very quickly implemented fixes on their end, and this morning we removed the special handling we had been performing for AKARI. We delivered this as a modification to the v3.2.1 survey file rather than a new release to minimize the time that AKARI was off-line. SkyView in a Jar users who are using version 3.2.0 or later should get the new survey description next time they access AKARI data. Users of older versions of SkyView will need to upgrade to access AKARI data properly.

Users may have seen problems with accessing AKARI data for several hours this morning until we updated our software to recognize their fixes.

Posted in Notices, releases | Tagged , , | 2 Comments