19 November 2026 in ACM Transactions on Information Systems
Transformer-based sequential recommender systems are retrained frequently, yet their training is dominated by cross-entropy (CE) over item catalogs that can reach millions of items. Its logit tensor scales with catalog size × batch size × sequence length, forcing practitioners to trade these quantities against each other or use negative sampling. However, how to allocate memory among negatives, batch size, and sequence length remains an open question. We study this trade-off with nearly 1,000 SASRec training runs on six datasets and find that no single memory-related hyperparameter should be maximized in isolation. Scaling both the number of negatives and batch size outperforms maximizing only one, while the choice between full CE and CE with negative sampling is dataset-dependent. We further introduce a memory-efficient and GPU-optimized replacement for CE with negative sampling that avoids materializing the logit tensor. It speeds up training by up to 3.5× and reduces memory by 4–24× relative to PyTorch CE with negative sampling. Reallocating memory saved by our replacement and Cut Cross-Entropy, which we adapt to sequential recommender systems, into larger batches, longer sequences, and more negatives improves NDCG@10 by up to 54.7% under the same memory budget. We release the Triton kernel and a public dataset of all training runs at https://github.com/On-Point-RND/MemoryEfficientSRS .
11 December 2023 in Russian Journal of Earth Sciences
The article analyzes histograms of the distribution of the main hydrometeorological parameters of the Caspian Sea for 1980-2021-air temperature, atmospheric pressure, absolute humidity, atmospheric precipitation and wind speed according to the MERRA-2 atmospheric reanalysis. Histograms were constructed for the entire study region of the Caspian Sea (36 •-48 • N, 45.625 •-55 • E), as well as separately for land and sea areas. The extremeness criteria were calculated based on the normal distribution and the real histogram for all 5 main meteorological parameters. A comparison was made of histograms obtained from MERRA-2 data and from weather station at Derbent located on the coast of the Caspian Sea. Distributions of average monthly wind speed, integral water vapor content and water vapor content in clouds over the Caspian Sea were also constructed according to microwave radiometry (SMMR, SSM/I) data for the time interval 1980-2021.