Problems accessing WISE data

For the past couple of weeks SkyView has not been able access data from the WISE archive. SkyView queries for  WISE images in regions already in our cache worked fine, but users who requested a new region got back a blank image.    This behavior seems to have started around June 27.

The problem arose when WISE inadvertently changed the URL’s they published the data under (using the VO Simple Image Access Protocol) to a format that was really only intended for internal use.  In particular it used a non-standard port number, rather than the standard HTTP port 80. The firewall that our web servers are behind does not permit access to unregistered, non-standard ports, so our servers were unable to download new data.

The WISE team fixed the problem immediately upon being notified of it and all should be back to normal.  We apologize for the time it took to discover this problem.  The normal behavior requests in cached regions concealed the problem from our regular checks.

Posted in Uncategorized | 4 Comments

SkyView cache problems

A data disk used to cache survey data filled up over the weekend resulting in the inability to generate images for many surveys. We have cleared space on the disk and SkyView is now back to normal.  We apologize for the inconvenience.

Posted in Notices | Leave a comment

Features in the Gallery: Klingon Forcefields?


Since there are more than 100 surveys in SkyView, it shouldn’t be surprising that we are not experts in the characteristics of each one. A blog reader noted that there is a very unusual feature in the WISE data near around 8 55 39, 19 36 44 and asked what it was. Off hand I couldn’t say. Here’s a one degree image from the WISE 3.4 micron data from the Gallery — though I admit that I put it there.

Feature in WISE 3.4 data
Unusual Feature is WISE 3.4 Survey (1 degree centered at 8 55 39, 19 36 44)

It’s really quite an extraordinary object — but what is it, where does this grid pattern come from? It may be helpful to show how we can try to track this down.  [The vertical boundary towards the right of the image is a failure of the algorithm that tries to match the backgrounds of adjacent images.  That often has problems when there are very bright objects in the field but otherwise has nothing to do with the anomaly.]

The first step is to try to see if it’s real, i.e., could it just be some kind of glitch. Fortunately WISE has four different bands, so we can look at each of those. And indeed each of those shows something similar overall though the details vary quite a bit. It suggests that there really was something there.

If there was a real object there, then the only way it could cause this kind of weirdness was if was so bright that it overwhelmed the  optics and electronics of the detector. Our first step is to check the bright star catalog. We can do that in SkyView itself, or at the HEASARC (using the BSC5P catalog). There don’t seem to be any objects in the field so the anomaly isn’t something like Sirius. By the bye that indicates that the apparently very bright star we see in the top left of the image is actually fainter than about 7th magnitude or so.

If it’s a bright natural object we’re running out options but we still have moving objects: planets, asteroids and such. The elongation of the object suggests a moving object. Only a few planets are likely to be bright enough though. If it is a planet then it should be close to the ecliptic plane.  We convert the RA,Dec input to ecliptic coordinates (You can use the coordinate converter at the HEASARC for example.. We find that the ecliptic latitude is only about 2 degrees, very reasonable for a planet. Now we need to go to find detailed ephemerides for the bright planets. Let’s try the Horizons service at JPL. Infrared satellites don’t look close to the Sun usually, so we’ll skip Mercury and Venus. We really only need to look at Mars, Jupiter and Saturn.

The survey metadata indicate that the WISE data were taken in 2010, so we generate an ephemeris for Mars for that year. With a bit of luck we find a hit immediately. On April 27, 2010, Mars was at exactly the location we see the anomaly. At that time is was about apparent magnitude 0.5, so it was likely a thousand or more times brighter than the bright star we see in the image. That star is already showing very serious diffraction spikes, so its not surprising that something vastly brighter causes real problems.

Where does the grid come from? It seems like the WISE data is a sum of many images taken in quick succession. We get diffraction spikes on each one. As Mars moves the spikes form the pattern we see. Or maybe it’s something else entirely. With such a bright object very weird things may be happening in the electronics. But at least we know what we’re seeing.

Posted in Discussion | Tagged , , | 1 Comment

New INTEGRAL Galactic Plane Survey in SkyView

A new INTEGRAL Galactic plane survey has been added using data provided by R. Krivonos and colleagues.  (See paper and web site).   This survey combines 9 years of INTEGRAL IBIS observations from December 2002 through January 2011 into a single Galactic Plane image. A total of 135 megaseconds of exposure is included in the observations used. Survey data is generated for the Galactic plane in the region |b| <= 17.5. This survey is sensitive to about 1 millicrab in the hard X-ray range from 17-80 keV.

INTEGRAL Galactic Plane Survey: The Galactic Center


These data are based on observations with INTEGRAL, an ESA project with instruments and science data centre funded by ESA member states (especially the PI countries: Denmark, France, Germany, Italy, Switzerland, Spain), Poland and with the participation of Russia and the USA.

Posted in Announce | 2 Comments

SkyView updates

We are releasing a new version of SkyView today (2012-06-25). We’ve a separate article on the new INTEGRAL Galactic Plane surveys, but we’ve made some changes that affect many of the survey datasets. For each survey that we support, SkyView uses a survey description file, a custom XML format which includes both metadata on the survey (all the documentation you see), and the information about where the original survey images are stored. Heretofore we’ve used a separate file for each survey, e.g., the SDSSi had a separate file from the SDSSg. In the new version we support parametrized survey description files where we substitute values for parameters in the file. E.g., the text in the raw file survey.xml might include a line like

<Frequency> ${F} EHz </Frequency>

When we tell SkyView to use this file we would use the name survey.xml?F=2.7 and the raw XML would be rendered as

<Frequency> 2.7 EHz </Frequency>

Many surveys (SDSS, BAT, Fermi, IRAS, … ) have multiple bands where where the differences in the survey descriptions are easy to parametrize. In this new release the number of survey description files has dropped from over 100 to a little over 50.

While this saves a little space in the SkyView JAR, the major advantage is that it means we don’t have to repeat the common information, so we should be able to keep the survey descriptions more accurate over the long run. However it’s possible we missed some element where we’ve left a survey specific item in and not parametrized it properly. Please let us know if you see any problems.

Another change in the latest version is that we’ve made much more extensive use of the ability that SkyView has always had of providing links to related SkyView images in the results for a primary image. If you look at the new INTEGRAL Galactic plane surveys, the BAT data or the PSPC pointed surveys, you will see links to additional imagery. From the web page you can directly request intensity/flux maps for the INTEGRAL, BAT, RASS and PSPC data, but the results have links to exposure and counts maps, that are not available from the web page directly. You can invoke these directly in the batch interface however.

This allows us to keep the Web form a little cleaner. We’ve tied together the RASS Count and Intensity maps where they had previously been presented as separate surveys. We’ve also now have space to bring back the PSPC 0.6 degree cutoff maps. This is a mosaic of the high resolution cores of the ROSAT PSPC pointed observations.

With about 130 total surveys making it possible to find all of the surveys is becoming a real driver in our development.

If you are using one of the surveys that is not directly available and find the new organization to be a problem please contact us.

Posted in Documentation, Notices, releases | Tagged , , | Leave a comment

SkyView using Xamin

Recently the HEASARC released a new more powerful interface to its catalog and archive, Xamin. We have just updated SkyView so that
SkyView catalog queries of tables in the HEASARC databases are now being made through this new Xamin interface rather than our older Browse interface. If you click on the “Link to HEASARC catalogs” in the results page to query HEASARC database, you will now start an Xamin session rather than run a Browse query.

Although the rows returned should be identical, the user interfaces for the two systems are very different so expect some changes if you click through. With Xamin you will start the web interface and you’ll get a browser page which has two panes, the query pane and a resuls pane. The query pane shows the query parameters that were used and allows you to follow up with any further queries that you are interested in. You may see Xamin processing your query, if it takes a while to get the results.

If you queried one catalog, the results pane will be the table of matching rows. If you queried many catalogs, then the results pane will show the number of matches for each catalog that had at least one. You can click on any row to see the results for that catalog.

Note that for both Browse and Xamin, you can get more rows from the query run at the HEASARC site than you saw in the image overlay. The catalog queries search a circle inside which the SkyView image is enscribed. So for a typical square image the catalog search looks at π/2 more area.

Please let us know if you have any questions or encounter any problems.

Posted in Discussion, Notices, releases | Tagged , , , | Leave a comment

New Fermi Surveys

We’re pleased to be able to provide a new set of Fermi surveys. Fermi is the highest energy survey in SkyView and heretofore we had been providing two bands of Fermi data, from 100 to 5450 MHz and 5450 MHz to 300 GeV. Fermi’s resolution is a very strong function of energy. With the bands we’d originally chosen, low energy, low resolution photons blurred the images of most sources. While the top band had very reasonable resolution, it had relatively few photons. The new Fermi data is broken into 5 bands:

  1. 30-100 MeV
  2. 100-300 MeV
  3. 300-1000 MeV
  4. 1-3 GeV
  5. 3-300 GeV

This seems to give a cleaner separation of the low energy/resolution data while keeping enough photons in the higher energy bands to really show the sky. This all sky image shows data from bands 3-5.

Fermi RGB all sky image

RGB image using Fermi bands 3, 4 and 5. Click for high res version

Harder gamma-ray sources show as blue. A myriad Galactic and high-latitude sources show.

We’ve imaginatively named these surveys Fermi 1, 2, 3, 4 and 5 in order of increasing energy. They contain all of the Fermi data available through the beginning of April 2012. The Fermi sky exposure is now considered to be sufficiently uniform that we are providing these surveys as counts maps rather than intensity maps. We’ll be adding exposure information soon.

Posted in Documentation, Notices | Tagged , | 4 Comments

WISE All-Sky Data Available in SkyView

SkyView is now serving the data from the WISE All-Sky Data release. The main difference with the preliminary release is coverage on the ~40% of the sky that was missing in the earlier release. However the entire dataset has been reprocessed using updated calibrations and such.

We download WISE data from IRSA in large tiles, caching each tile so that we never have to download the same tile twice. It can take a while for WISE images to be generated if there has been no request in that region before, even when you are requesting only a very small region. It can easily take a minute or more to get the files we need to generate your image. As our cache grows you’ll be more likely to hit it and access should be much faster. Once data are in the cache it takes only a couple of seconds to generate a typical image.

The WISE data give us a view of the infrared sky comparable to the DSS optical images in the optical. We anticipate WISE being one of SkyView’s most popular surveys.

Posted in Announce | Tagged , | 6 Comments

SkyView Surveys Summary: I

Recently we’ve been updating and systematizing some of the metadata we have on SkyView surveys, trying to make sure that we have consistent, quantitative description of each one. Some of the key metadata are the resolution of the survey on the sky, sensitivity and sky coverage. It’s quite difficult to provide single numbers for these in a consistent way. E.g., for lots of surveys the resolution varies depending upon where you are. In some cases it even depends upon the spectrum of the source.

So take this with a grain (or maybe even a spoonful) of salt, but here’s a graph of the nominal resolution of SkyView’s surveys as a function of wavelength.

The resolution of SkyView surveys as a function of frequency

Resolution of Surveys Available in SkyView


The surveys span about 18 orders of magnitude in frequency. That’s pretty amazing. If you want to get a sense of how big that range is, consider the ratio of the distance to the nearest stars to your height… An immense difference — but only 1% of the range of SkyView‘s frequency coverage!

Looking at the graph we can split surveys into three categories based on their resolution: High resolution surveys have better than 10″ resolution. Most of these are concentrated near the optical, but there are a couple of outliers. Medium resolution surveys, with resolutions of about 1’ are the most common. These surveys have resolution comparable to our eyes. A fair number of surveys have resolutions of a degree or more. This doesn’t necessarily mean they are less valuable. Sometimes you want to look at the forest and not the trees.

Posted in Discussion | 1 Comment