TGSS issue

A couple of users noticed that requests for data from the TGSS ADR1 survey were not working. We’ve tracked this down to a change in the URLs used at the archive in the Netherlands used to store the TGSS data. They have moved to using secure HTTP (i.e., HTTPS urls), and were providing redirection from the old URLs to the new ones. While browsers and many programs will automatically follow such redirections, Java programs will not follow a redirect automatically if the scheme changes, as here where it goes from http to https. We’ve updated the URLs we were using and all should now be working.

Since we’ve fixed this without making a full formal release, if you use SkyView-in-a-Jar you may need to clear out the tgss.xml.gz file in your skycache but otherwise no action should be needed by users. We’ll be doing the full release soon at which point the tgss.xml.gz file will update automatically.

Our apologies to anyone inconvenienced by this and many thanks to those who pointed out the problem.

Posted in Notices | Tagged , | Leave a comment

SkyView V3.3.4: GLEAM Survey and new features

GLEAM image of 2 degree region at declination -80, RA 0

SkyView V3.3.4 has just been released. The new version includes a set of maps from the GaLactic and Extragalactic All-sky MWA Survey (GLEAM). The GLEAM survey was taken at the Murchison Widefield Array (MWA) and covers most of the sky south of 25 degrees north, about 25,000 square degrees.

The image above is an example field: two degrees around RA=0,Dec=-80. SkyView includes four wider bands from the GLEAM survey ranging from 71 to 231 MHz. These and many narrower bands are available from the GLEAM Cutout Service. More details are available from the survey documentation in SkyView or the GLEAM website itself.

There are a few new capabilities in SkyView as well. A new setting, TrackedInputs, can be used to transfer values from the input FITS files to the outputs. The value should be one or more comma-separated FITS keywords which are assumed to be real-valued. The pixel-weighted averages of these keywords are computed as the input files are read, and the averages are written out to the output file. The new keyword is used in the GLEAM survey to average the beam characteristics.

A new survey name, Average, may be used when generating RGB images. Previously, when one wished to create an RGB image from only two surveys, one could pick only two colors, or one could use one of the surveys in two of the RGB colors. Users now have a third choice, to use the average image of the two inputs as the third color. This can give a pleasing image while still evenly weighting the two input surveys. Only one Average survey can be specified and there should be two other surveys specified. The position of the Average survey indicates which color is to be averaged. E.g.,

     java skyview.jar survey=average,dss2red,dss2blue rgb

would use the average image for the red, the dss2red image for the green, and the dss2blue image for the blue.

A new smoother is also available in skyview.data.ThresholdMedianSmoother. This smoother (which can be used as a postprocessor when using SkyView-in-a-Jar), does a median filtering of the image using some specified box size. However the median filtering is thresholded, so that a pixel is replaced with the median value only if it deviates from the median by more than some specified offset or ratio. This filter is essentially what is used in the popular Dust And Scratches filter in PhotoShop. One could, e.g., use this filter to get rid of point sources in an image while keeping a diffuse background. There are lots of options in this filter. E.g.,
one can set the threshold to 0 to do median filtering, request thresholding by offset or ratio, and threshold in only one direction,
i.e., for high values but not low as well as setting the box size for the median calculation.

Posted in Announce, releases | Tagged , , , | Leave a comment

SkyView V3.3.2: Links to Mosaicking Inputs

Version 3.3.2 of SkyView has been released with two significant changes: updated memory limits, and links to data inputs.

The parameters used to set the heap size limits for SkyView web requests have been updated so that users will generally be able to create larger images than before — although the exact size is a complex combination of he source data format, the requested output region and other traffic on the site. Images up to 10,000 x 10,000 pixels should be feasible though you may run out of time rather than memory — the connection will drop if nothing happens in 20 minutes. One particular problem was that SkyView ran out of memory if you requested multiple Planck surveys (which are very large single files). You can now retrieve as many Planck datasets as you like! Thanks for Ignacio Cisneros for alerting us to this problem.

The big change for users is a new button that will appear below every image. One of the defining characteristics of SkyView is that it always mosaicks and resamples the input survey data into whatever geometry the user requests. We now provide a button so users can easily see what input data was used for this resampling/mosaicking. This will usually give you a link to the source data that was used in generating the SkyView image. This isn’t necessarily the fundamental source data, it’s just whatever dataset SkyView uses for a particular survey. Currently there are five cases depending on how the data was derived:

  1. From the DSS (or DSS2) plate scans. Links to the directories in which the DSS tiles are found are given.
  2. From a HiPS image (see the recent releases). A link to the base HiPS directory and the properties file for the HiPS is given. Note however that the base directory is not required to be a working URL itself.
  3. From FITS files where SkyView keeps a complete copy of the survey locally. Links to the SkyView survey copy is given.
  4. From FITS files which have been copied from a remote archive into a SkyView cache. Links to the cached files are given.
  5. From FITS files which for copyright or technical reasons cannot be distributed to the public. No links are given.

Two caveats: Many of these datasets have some level of copyright. Please look at the documentation for the survey for more information. Also, to forestall a profusion of browser windows, we have defined a particular browser window in which the link information will be displayed. If you use a tabbed browser, then the second and subsequent times you click on one of the link buttons the tab may be updated, but probably will not be given focus. So it may look like nothing has happened….

You may also note that the detailed SkyView users guide is now available directly from the home page. You don’t need to go into the help pages to see it.

Internally this version considerably reorganizes the various Mosaicker classes minimizing duplication of code. The listing of which files were used in the FITS headers has been somewhat reorganized and now gives the number of pixels resampled from each image.

As always if you have questions or comments please let us know.

Posted in Announce, releases | Tagged , , | Leave a comment

SkyView V3.3.1: Swift UVOT, UltraVista and CFHT HiPS data

We are about to release SkyView version 3.3.1.  This new version includes extended support for HiPS data and has many new surveys.  The primary new datasets are counts, intensity and exposure maps for the Swift UVOT instrument in seven filters.  All data for the first 11.5 years of the mission have been combined into HiPS for each filter.  For the most observed filters a bit under 3% of the sky is observed.  There is a fair bit of overlap but also many differences in the coverage of the various filters.

m81uvot

M81 in UVOT U, UVW1 and UVW2 filters

We have also added in a number of other HIPS surveys from the UltraVista and CFHT surveys. These data are just a few of the many data in Hiearchical Progessive Survey (HiPS) format that are currently available through the web. We anticipate making more available in future data. HiPS joins the VO Simple Image Access Protocol in enabling SkyView to access data from remote
sites.

Caching of HiPS data is now supported and some some issues that disabled special access to a UVOT overlay capability have been resolved.

Posted in Announce, releases | Tagged , , , , | 1 Comment

SkyView v3.3.0: Swift XRT surveys

Versions 3.3.0 of SkyView was released today.  This includes an all-sky mosaic of the Swift XRT observations.  Over 129,000 XRT observations were combined to give coverage of just under 6% of the sky.  A set of three surveys were produced: the combined counts, the total exposure, and the intensity map that is the ratio of the first two.   This image ofM31 combines the XRT intensity map with the WISE 22 micron image — contrasting the very different sources of X-ray and infrared emission.

XRT and WISE image of M31

XRT Intensity (green and blue) and WISE (red) in a 1.2 degree field centered on M31.

A few other changes were made in the release, notably many references to HTTP  on our sites were changed to HTTPS.   Although there should be an automated redirection, we are trying to purge the old HTTP addresses since occasionally they cause problems.

Posted in Announce, Notices, releases | Tagged , , | Leave a comment

SkyView and HiPS

Next week we’ll be releasing version 3.3.0 of SkyView.  The big change from the user perspective will be the new Swift XRT combined image.  This is similar in spirit to the venerable PSPC surveys — we’ve added all of the XRT counts and exposures and created all sky maps based on the sums — and an intensity map as the ratio of the two.   From the programming side though the big change is that SkyView will be supporting a new survey format: the Hierarchical Progressive Survey or HiPS.

Traditionally SkyView simply takes a set of images which normally have overlaps and mosaics them to the users specifications.   So long as there is a little overlap between them, we don’t care too much about how the source image tiles are organized.  In HiPS the survey data is required to match a very specific tiling. The organization is strongly tied to the HEALPix pixelization.  This is a way of defining pixels in the sky such that pixels are all of equal area and the average latitude values of the pixels are restricted to a relatively small set — i.e., the pixels are centered on lines of constant latitude.  Both these characteristics are convenient in analysis of CMB datasets where HEALPix originated.  The pixelization starts with 12 top level (or level 0) pixels and then breaks them up recursively into higher level pixels.  So there are 48 level 1 pixels, 192 level 2 pixels, 768 level three pixels and so on.  In HiPS each tile corresponds to a specific HEALPix pixel but the tile also will have Nx and Ny that is some power of 2. So all of the pixels in the tile are also HEALPix pixels but of a higher order.

http://healpix.sourceforge.net/images/gorski_f1.jpg

Levels 0-3 HEALpix pixels.

HEALPix pixelizaton

E.g., in the Swift XRT HiPS images the highest resolution tiles correspond to the 786,432 level 8 HEALPix pixels.  Each tile is a 512×512 image so that each pixel in a tile image  is a level 17 HEALPix pixel.  A client can find the level 8 pixels in their region of interest and download only the tiles of interest.

The other big feature of HiPS is that not only do we have the high-resolution tiles, we also precompute tiles of lower resolution.  E..g., we also have level 7 tiles where we’ve averaged the high-resolution pixels 2×2 and also combined four of the tiles into one tile that covers four times the area.   For the Swift data we’ve generated tiles down to level 3 and also a specially formatted all sky image with even lower resolution.  These choices mimic those of the  CDS Hipsgen tool (including with the latest versions of Aladin) which we used to generate the exposure maps.  We used our own software to build the counts and intensity maps.

A client reading a HiPS file can easily zoom in and out a la Google Earth and Sky or the WWT.  These technology is supported by Aladin and Aladin-lite and can be seen in the ESASky and the MAST data portal.  Our goal in SkyView is not so much providing this kind of dynamical interface as enabling the user to generate images to a clear specification, but we use the multiple resolutions too.  SkyView resamples the lowest resolution higher than the user requested resolution.

Many archives have generated HiPS of their datasets and provide them on the Web. We anticipate providing access to many of these datasets in upcoming versions of SkyView — and we’ll be publishing any HiPS we generate.  If you have a particular interest in getting some dataset into our system please let us know.

 

Posted in Discussion, Documentation | Tagged , , , | 1 Comment

Code 400, “Bad Request” error

Our system administrators have resolved a problem where users would occasionally get a code 400, “Bad Request” message when accessing SkyView (and other HEASARC sites). Tracking tools required by NASA policy were creating cookies that were too large for our servers to accept.  See  this earlier article (https://skyview.gsfc.nasa.gov/blog/index.php/2014/07/16/v3-1-7-skyview-add-google-analytics/) for more background.  We apologize for any inconvenience caused. Please let us know if you are still seeing issues.

Posted in Discussion | Tagged , | Leave a comment

SkyView V3.2.3 Beware the Ides of March!

Over the next few days (as early as noon 3/15) the SkyView servers will be transitioning to requiring secure HTTP, i.e., HTTPS, URLs to access SkyView services.  This is part of a mandated transition of all NASA (and indeed all federal) web sites. Web users should see little difference.  If you come in using a non-HTTPS URL you will be automatically redirected to the correct URL.  However users who access SkyView through batch services may see a difference since the software that accesses SkyView may not automatically follow the redirect.  In particular, Java’s network library  will not automatically follow redirects which transition from HTTP to HTTPS.   Java (and any other framework that exhibits similar behavior) tools can be programmed to explicitly follow redirects, or the HTTPS URL can be given directly.

Since SkyView-in-a-Jar is written in Java it is sensitive to this change.  When you are using SkyView-in-a-Jar it retrieves survey data from SkyView using information in survey description files. Until version 3.2.3 those files had HTTP URLs.  With this version all the survey description files have been updated to use HTTPS URLs.  We have updated the survey XML files to use HTTPS addresses and it had been our thought that users of recent jar files would be OK since the program would detect the new survey description files and download them automatically.  However we have since recognized that the base URL for the survey description files needs to be updated.  So we recommend all SkyView-in-a-Jar users update to the latest version of the Jar (at least version 3.2.3).

Update: 3/17:  Small updates have been made to the Jar that may fix issues seen by Windows users of SkyView-in-a-Jar.  Our apologies for not catching these earlier.

Posted in Announce, releases | Tagged , | Leave a comment

SkyView update: v3.2.2 including HI4PI data

On Tuesday Dec. 13, 2016, we updated SkyView to v3.2.2 in order to include the new HI4PI radio survey. This dataset is the atomic neutral hydrogen (HI) column density map derived from HI4PI (|Vlsr| < 600 km/s). It includes the Effelsberg-Bonn HI Survey (EBHIS) of the northern sky but completes the sky coverage with the addition of southern sky data from Parkes.

Note that a previously-downloaded jar file of v3.2.1 should still pick up this new survey automatically. This update also includes some changes to the internal meta data handling, but this should be transparent to users.

Posted in Announce, releases, Uncategorized | 2 Comments

SkyView downtime October 4, 2016

SkyView was down from about 9:00am to 2:30pm today due to a problem encountered while updating the SkyView servers.  SkyView operations should now be back to normal. We apologize for the inconvenience.

Posted in Discussion, Notices | Leave a comment