Tolga Topal

Mr. Tolga Topal

orcid

Research Focus & Metrics

Academic Career Length

2Years (2025-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
12 December 2025 in Preprints.org
Verified
Preprint
Open access
PDF file
TTTolga Topal
Tolga Topal
Information theory underpins modern communication, computation and complex systems, yet the structure of its governing inequalities remains an active area of research. This paper revisits the concept and mathematical foundations of Information Theory Laws, that is, constraints applied to the entropy function. Starting from Shannon’s seminal framework, we review the evolution from basic linear inequalities– i.e., polymatroid axioms– to the discovery of non-Shannon-type inequalities, which revealed that the Shannon region does not overlap with its closure region for n ≥ 4. We outline the geometric and algebraic representation of entropy spaces, discuss the critical threshold where complexity escalates, and highlight the role of machine-assisted verification in uncovering new inequalities that are not of Shannon-type. By tracing historical milestones and computational advances, this work provides a structured recollection of the spectrum of information inequalities and their implications for converse coding theorems and related applications. The study emphasizes the interplay between the theoretical developments and computational tools of the field in shaping the landscape of information theory.

13 May 2026 in Preprints.org
Verified
Preprint
Open access
PDF file
TTTolga Topal
Tolga Topal
Shannon entropy and Kolmogorov complexity describe complementary facets of information. We revisit Q2 from 27 Open Problems in Kolmogorov Complexity: whether all linear information inequalities including non‑Shannon‑type ones admit $\mathcal{O}(1)$-precision analogues for prefix‑free Kolmogorov complexity. We answer in the affirmative via two independent arguments. First, a contradiction proof leverages the uncomputability of $K$ to show that genuine algorithmic dependencies underlying non‑Shannon‑type constraints cannot incur length‑dependent overheads. Second, a coding‑theoretic construction treats the copy lemma as a bounded‑overhead coding mechanism and couples prefix‑free coding (Kraft's inequality) with typicality (Shannon-McMillan-Breiman) to establish $\mathcal{O}(1)$ precision; we illustrate the method on the Zhang-Yeung (ZY98) inequality and extend to all known non‑Shannon‑type inequalities derived through a finite number of copy operations. These results clarify the structural bridge between Shannon‑type linear inequalities and their Kolmogorov counterparts, and formalize artificial independence as the algorithmic analogue of copying in entropy proofs. Collectively, they indicate that the apparent discrepancy between statistical and algorithmic information manifests only as constant‑order effects under prefix complexity, thereby resolving a fundamental question about the relationship between statistical and algorithmic information structure.

17 December 2025 in Preprints.org
Verified
Preprint
Open access
PDF file
TTTolga Topal
Tolga Topal
In this work, we present a refined interpretation of the Neural Scaling Laws that is inspired by phase transitions. Our starting point is from the paper: A Resource Based Model For Neural Scaling Laws. Indeed, there is both empirical and theoretical backing for the \( \ell \propto N_p^{-1/3} \) power law. Our formalization through the combined use of propositional logic and an SMT-solver allows us to draw new perspectives on the learning curve. As formulated and relied on for their internal consistency in the prior work, the critical depth conjecture is strengthened in our work. Rooted in a combination of: logic and empirical/theoretical insights, we draw a three regime profile of the Neural Scaling Laws. Ultimately, our physics-inspired proposal of Neural Scaling Laws profiles as follows: 1) structural phase, where we argue that, the loss scales following: \( \ell \propto N_p^{-2/3} \), 2) Above the \textit{critical depth}, a redundancy phase, with a loss following the classical: \( \ell \propto N_p^{-1/3} \) (where most of current LLMs operate). Finally, 3) an optimized trajectory where depth is fixed and a scaling is based on width following: \( \ell \propto N_p^{-1/2} \).

13 August 2026 in SSRN Electronic Journal
Verified
Preprint
Open access
TTTolga Topal
Tolga Topal
We address three open problems from the 27 Open Problems in Kolmogorov Complexity concerning the Gray-Wyner profile of a pair of strings. (Q7) The Gray-Wyner profile is characterized as the set of integer triples that satisfy the polymatroid inequalities together with the fixed complexities of the pair, up to logarithmic precision. (Q8) the Gray-Wyner, Wyner, and extended profiles are shown to be pairwise incomparable, while the extended profile strictly more informative than either. (Q9) We construct a pair of strings and a universal machine for which a random oracle always changes the Gray-Wyner profile, answering this question in the negative.

16 May 2025 in Preprints.org
Verified
Preprint
Open access
PDF file
TTTolga Topal
Tolga Topal
This technical report explores various methods for characterizing the complexity of strings, focusing primarily on Algorithmic Complexity (AC), also known as Kolmogorov Complexity (K). The report examines the use of lossless compression algorithms, such as Huffman Coding and Run-Length Encoding, to approximate AC, contrasting these with the Coding Theorem Method (CTM) and Block Decomposition Method (BDM) approaches. We conduct a series of experiments using leaked passwords as data, comparing the different methods across various alphabet representations i.e.: ASCII and binary. The report highlights the limitations of Shannon Entropy (H) as a sole measure of complexity and argues that AC offers a more nuanced and practical approach to quantifying the randomness of strings. We conclude by outlining potential research avenues in areas such as source coding, cryptography, and program synthesis, where AC could be used to enhance current methodologies