Xamin and batch queries

Over the next couple of weeks we plan for Xamin to take over responsibility for batch queries sent to the HEASARC. In our initial tests the Xamin queries ran about 4 times faster.  While we’ve made some efforts to minimize the differences between Xamin and the current Browse-based results there are still a few.

The differences include:

  • The name of the column giving the offset between the target position and the row in the table is _offset, not Search_offset
  • The value of the offset column does not include the central position.  E.g., if you have asked for rows near the position 00 40 24, 40 32 , the current offset column might contain 0.235 (00 40 24, 40 32) but it will just be 0.235 in the new version.
  • Class codes will be returned for the HEASARC classifications, not the string value corresponding to the class code.
  • The mission name is not given when the list of available tables is shown.  In Xamin not all tables are associated with missions (some are associated with regimes or object types).  We added a ‘archive’ column to keep the format similar.
  • Error messages are very different.  E.g., If a user requests a VOTable output, Browse still gives an HTML error message when there is an error retrieving data, but Xamin returns an VOTable with the error status.  If there are errors when the user is making a FITS or Excel format request, then there may be no output where Browse might have returned an HTML error message.
  • There are numerous detailed format differences in the Excel and FITS outputs.  E.g., Xamin uses a binary table format while Browse uses ASCII tables.

There are some new features.  Since we are limited to the interface that the Xamin batch script provides only a few of the new capabilities of Xamin are available, but a few do leak though e.g.,

  • The fields displayed can contain expressions of fields, not just one or more column names: fields=’SomeText’, ra+dec,cos(ra/57)
  • Results can be sorted by multiple columns (sortvar=col1,col2) and in both ascending and descending order (sortvar=col1:+,col2:-).

Please let us know if you have any problems with the new format.  If you would like to test this out before we release it change the URL the batch script points to from http://heasarc.gsfc.nasa.gov/cgi-bin/W3Browse/w3query.pl to http://heasarc.gsfc.nasa.gov/xamin/BrowseBatch.

 

Xamin passes query milestone

The vast majority of the queries that the HEASARC receives are through automated interfaces that use either the VO cone search protocol or the Batch protocol we developed for Browse (long before the VO). Starting this week cone searches are running the Xamin interfaces. Usually there are more batch interface queries than cone searches, but there is a lot of variation. Yesterday we had a few more cone search requests and as a result Xamin satisfied more query requests than Browse for the first time, just under 10,000 Xamin queries versus just over 7,000 in Browse.

We’re looking at moving the Batch requests to Xamin in the next couple of weeks. Preliminary tests show it is up to 4x faster. When we do that Xamin will be handling over 90% of the HEASARC’s query load.

Xamin 1.6 released

A new version of Xamin, 1.6, was released this morning. The most noticable changes are a new notice facility and links to the Xamin Blog in the help menu. We will use the notice facility to inform users of new features and scheduled downtimes. Error messages are now displayed when SAMP connections (e.g., to TOPCAT or Aladin) fail, and the HTTP headers in data responses now give a suggested time to the next query which is set based upon the current system load. Some modifications to the cone search responses ensure better compliance to the VO Cone Search standard and more compatibility with the format used in Browse-base cone searches.

HEASARC VO Cone Search to Use Xamin

Starting next week, queries to the HEASARC using the Virtual Observatory Cone Search protocol will use our Xamin interface rather than a script that invokes Browse. While the results for queries should be the same, the VOTables returned are not byte identical (e.g., they include differing white space). For a few tables changes have been made to column definitions to meet the requirements of the cone search protocol. In some cases ID columns that were returned as integers are now considered to be strings to meet the spec. Please get in touch with us if you have any problems. Our internal tests show the Xamin queries running about twice as fast.

Querying Vizier From Xamin

The Vizier system hosted at the CDS is an unmatched source for astronomical tables with nearly 10,000 resources and over 20,000 tables. All public Vizier tables are queryable through Xamin.

 

First you have to select a table (or tables).

If you know the designation of the Vizier table you can just enter it directly. E.g., the USNO B catalog is catalog I/284 and you can just type that name in the ‘Enter table name’ box of the Tables Explorer. For the first couple of characters as you type, you may see a menu of HEASARC tables that start with the same letters. You can just ignore these.

Selecting tables with direct entry

Entering a Vizier table name directly

Hit return after you’ve entered the table name.

Or you can use the Available tables tree. All Vizier tables are listed within the External tables tree. We open the External tables. Vizier tables are the first entry. If we open this up we get the nine Vizier catalog type categories, with Roman numeral prefixes I through IX, mirrored catalogs, B, and many catalogs associated with specific journals. These have a prefix the starts with J and includes a contraction of the journal name, e.g., J/ApJ for Astrophysical Journal. We can open up any of these. If there are more than 50 resources included for that category, then these resources are broken up into subcategories in the appropriate order (e.g., the resource number for the numbered tables, or by journal volume for journal tables) into groups of 25. If we need to, we open up this level of the tree to see the actual Vizier resources.

Some resources include multiple tables so that they are still branch nodes in the table tree. They can be opened up to get the individual tables for the resource. Other Vizier resources have only a single table, so the resource is a leaf node.

Selecting tables with direct entry

Selecting a Vizier table using the Tables tree

 

Once you see the table you want you can double click it, click on the + icon or drag it into the selected tables area to select it.

If you know a bit about what you are looking for, or have some specific information like a bibcode, it may be more efficient to enter that in the Table Explorers, Search by Keywords box. That will look for matching Vizier entries, and you can select the table from the pane that come up when you click on the nearby Search button.

Selecting tables with direct entry Finding a Vizier table using the keyword search]

 

Just click on the + icon to select a table from the list of returned tables.

You can use all three of these mechanisms to select Vizier tables in the Tables Explorer. If you want to get documentation on a Vizier table, you can click on the resource name in either the Available or Selected tables trees. The documentation shows at the bottom of the query pane. You may want to enlarge that part by dragging the divider up to view the document.

Once you’ve selected the table you can, in principle, do any kind of query. However only positional constraints and relatively simple constraints on columns can be handled by Vizier itself. If you want to specify complex constraints or a cross-correlation, Xamin may try to download the table and use a local copy. For many Vizier tables with a few hundred of thousand rows, that will work fine, but Vizier has many very large tables and so a naive query may time out.

 

Let us know if you have any questions regarding Vizier queries in Xamin.

Comments Policy

We welcome comments and discussion in this blog. All comments are reviewed before publication.  Comments should be relevant and use appropriate language. comments that appear to be comment spam will be eliminated. If you do not see your comment and wish to ask why, please send us E-mail using our feedback system.