Batch Queries Moving to Xamin

On October 2, 2012 we will begin sending batch queries to our  Xamin interface.  Internal tests have shown Xamin about twice as fast as the current interface.

The switch will be accomplished by internally redirecting batch queries sent to the current URL to an Xamin URL.  Non-batch queries (used by our Browse interface) are not affected.  The redirection will be done internally within our site so that web clients will not see an HTTP 30x redirection response.

There are some small differences in the formats of data returned discussed in a previous post, but we hope that this transition should be largely invisible to users.  If you want to switch over early, you can query the URL

http://heasarc.gsfc.nasa.gov/xamin/BrowseBatch

rather than the

http://heasarc.gsfc.nasa.gov/cgi-bin/W3Browse/w3query.pl

URL currently invoked.  After next Tuesday batch requests to the w3query.pl URL will be transparently redirected to Xamin.  If you want to invoke the old behavior after next week you can switch to using

http://heasarc.gsfc.nasa.gov/cgi-bin/W3Browse/w3query_noredir.pl

for your batch requests.

Versions of our batch scripts that will point consistently to either Xamin or Browse are available at http://heasarc.gsfc.nasa.gov/FTP/software/web_batch.  We recommend that you use the wget versions of scripts if possible since the simple webquery.pl tool used as an alternative has no ability to negotiate firewalls or redirections.

 

Xamin and batch queries

Over the next couple of weeks we plan for Xamin to take over responsibility for batch queries sent to the HEASARC. In our initial tests the Xamin queries ran about 4 times faster.  While we’ve made some efforts to minimize the differences between Xamin and the current Browse-based results there are still a few.

The differences include:

  • The name of the column giving the offset between the target position and the row in the table is _offset, not Search_offset
  • The value of the offset column does not include the central position.  E.g., if you have asked for rows near the position 00 40 24, 40 32 , the current offset column might contain 0.235 (00 40 24, 40 32) but it will just be 0.235 in the new version.
  • Class codes will be returned for the HEASARC classifications, not the string value corresponding to the class code.
  • The mission name is not given when the list of available tables is shown.  In Xamin not all tables are associated with missions (some are associated with regimes or object types).  We added a ‘archive’ column to keep the format similar.
  • Error messages are very different.  E.g., If a user requests a VOTable output, Browse still gives an HTML error message when there is an error retrieving data, but Xamin returns an VOTable with the error status.  If there are errors when the user is making a FITS or Excel format request, then there may be no output where Browse might have returned an HTML error message.
  • There are numerous detailed format differences in the Excel and FITS outputs.  E.g., Xamin uses a binary table format while Browse uses ASCII tables.

There are some new features.  Since we are limited to the interface that the Xamin batch script provides only a few of the new capabilities of Xamin are available, but a few do leak though e.g.,

  • The fields displayed can contain expressions of fields, not just one or more column names: fields=’SomeText’, ra+dec,cos(ra/57)
  • Results can be sorted by multiple columns (sortvar=col1,col2) and in both ascending and descending order (sortvar=col1:+,col2:-).

Please let us know if you have any problems with the new format.  If you would like to test this out before we release it change the URL the batch script points to from http://heasarc.gsfc.nasa.gov/cgi-bin/W3Browse/w3query.pl to http://heasarc.gsfc.nasa.gov/xamin/BrowseBatch.

 

HEASARC VO Cone Search to Use Xamin

Starting next week, queries to the HEASARC using the Virtual Observatory Cone Search protocol will use our Xamin interface rather than a script that invokes Browse. While the results for queries should be the same, the VOTables returned are not byte identical (e.g., they include differing white space). For a few tables changes have been made to column definitions to meet the requirements of the cone search protocol. In some cases ID columns that were returned as integers are now considered to be strings to meet the spec. Please get in touch with us if you have any problems. Our internal tests show the Xamin queries running about twice as fast.