13 December 2012 in Journal of Physics: Conference Series
The Open Science Grid (OSG) supports a diverse community of new and existing users in adopting and making effective use of the Distributed High Throughput Computing (DHTC) model. The LHC user community has deep local support within the experiments. For other smaller communities and individual users the OSG provides consulting and technical services through the User Support area. We describe these sometimes successful and sometimes not so successful experiences and analyze lessons learned that are helping us improve our services. The services offered include forums to enable shared learning and mutual support, tutorials and documentation for new technology, and troubleshooting of problematic or systemic failure modes. For new communities and users, we bootstrap their use of the distributed high throughput computing technologies and resources available on the OSG by following a phased approach. We first adapt the application and run a small production campaign on a subset of "friendly" sites. Only then do we move the user to run full production campaigns across the many remote sites on the OSG, adding to the community resources up to hundreds of thousands of CPU hours per day. This scaling up generates new challenges – like no determinism in the time to job completion, and diverse errors due to the heterogeneity of the configurations and environments – so some attention is needed to get good results. We cover recent experiences with image simulation for the Large Synoptic Survey Telescope (LSST), small-file large volume data movement for the Dark Energy Survey (DES), civil engineering simulation with the Network for Earthquake Engineering Simulation (NEES), and accelerator modeling with the Electron Ion Collider group at BNL. We will categorize and analyze the use cases and describe how our processes are evolving based on lessons learned.
6 June 2025 in Astronomy & Astrophysics
Context. In a series of publications, we describe a comprehensive comparison of Event Horizon Telescope (EHT) data with theoretical models of the observed Sagittarius A* (Sgr A*) and Messier 87* (M87*) horizon-scale sources. Aims. In this article, we report on improvements made to our observational data reduction pipeline and present the generation of observables derived from the EHT models. We make use of ray-traced general relativistic magnetohydrodynamic simulations that are based on different black hole spacetime metrics and accretion physics parameters. These broad classes of models provide a good representation of the primary targets observed by the EHT. Methods. We describe how we combined multiple frequency bands and polarization channels of the observational data to improve our fringe-finding sensitivity and stabilization of atmospheric phase fluctuations. To generate realistic synthetic data from our models, we took the signal path as well as the calibration process, and thereby the aforementioned improvements, into account. We could thus produce synthetic visibilities akin to calibrated EHT data and identify salient features for the discrimination of model parameters. Results. We have produced a library consisting of an unparalleled 962 000 synthetic Sgr A* and M87* datasets. In terms of baseline coverage and noise properties, the library encompasses 2017 EHT measurements as well as future observations with an extended telescope array. Conclusions. We differentiate between robust visibility data products related to model features and data products that are strongly affected by data corruption effects. Parameter inference is mostly limited by intrinsic model variability, which highlights the importance of long-term monitoring observations with the EHT. In later papers in this series, we will show how a Bayesian neural network trained on our synthetic data is capable of dealing with the model variability and extracting physical parameters from EHT observations. With our calibration improvements, our newly reduced EHT datasets have a considerably better quality compared to previously analyzed data.