This morning we had a user making very heavy usage of Xamin sending multiple cone search requests per second. Requests piled up in the database and this slowed down the system considerably. Requests that normally took a second could take several minutes. Metadata queries in the interactive versions of Xamin would often time out so that the Web interface was not really working.
We have temporarily disabled access for this IP address.
We’ll be looking at how we can meter usage more effectively in the future and apologize to those adversely affected.
The vast majority of the queries that the HEASARC receives are through automated interfaces that use either the VO cone search protocol or the Batch protocol we developed for Browse (long before the VO). Starting this week cone searches are running the Xamin interfaces. Usually there are more batch interface queries than cone searches, but there is a lot of variation. Yesterday we had a few more cone search requests and as a result Xamin satisfied more query requests than Browse for the first time, just under 10,000 Xamin queries versus just over 7,000 in Browse.
We’re looking at moving the Batch requests to Xamin in the next couple of weeks. Preliminary tests show it is up to 4x faster. When we do that Xamin will be handling over 90% of the HEASARC’s query load.