ON THE DISTRIBUTION OF CONTINUED FRACTIONS AND MODULAR SYMBOLS

Yuri I. Manin, Matilde Marcolli

Max–Planck–Institut für Mathematik, Bonn, Germany

Abstract. Using the techniques introduced by D. Mayer, we prove an extension of the classical Gauss–Kuzmin theorem about the distribution of continued fractions, which in particular allows one to take into account some congruence properties of consecutive convergents. We then study some averages related to modular symbols.

§0. Introduction and summary

0.1. Continued fractions. We start by fixing the notation which will be used throughout the paper. Considering first k_{1}, …, k_{n} as independent variables, put for n ≥ 1

[k_{1}, …, k_{n}] := \frac{1}{k_{1} + _{\frac{1}{k_{2}+... \frac{1}{kn}}}} = \frac{P_{n}(k_{1}, …, k_{n})}{Q_{n}(k_{1}, …, k_{n})}  (0.1)

where P_{n}, Q_{n} are polynomials with integral coefficients which can be calculated inductively from the relations

Q_{n+1}(k_{1}, …, k_{n}, k_{n+1}) = k_{n+1}Q_{n}(k_{1}, …, k_{n}) + Q_{n−1}(k_{1}, …, k_{n−1}),
P_{n}(k_{1}, …, k_{n}) = Q_{n−1}(k_{2}, …, k_{n})  (0.2)

It is convenient also to put formally Q_{−1} = 0, Q_{0} = 1 which is compatible with (0.2), (0.1).

From (0.2) one readily sees that

[k_{1}, …, k_{n−1}, k_{n} + x_{n}] =

\frac{P_{n−1}(k_{1}, …, k_{n−1}) x_{n} + P_{n}(k_{1}, …, k_{n})}{Q_{n−1}(k_{1}, …, k_{n−1}) x_{n} + Q_{n}(k_{1}, …, k_{n})} = ( \begin{matrix}P_{n−1} & P_{n} \\ Q_{n−1} & Q_{n} \end{matrix} ) (x_{n})  (0.3)

where we use the standard matrix notation for fractional linear transformations defining the action of GL(2) on P^{1}:

z ↦ \frac{az + b}{cz + d} = ( \begin{matrix}a & b \\ c & d \end{matrix} ) (z)

If α ∈ (0, 1) is an irrational number, there is a unique sequence of integers k_{n}(α) ≥ 1 such that α is the limit of [ k_{1}(α), …, k_{n}(α) ] as n → ∞. Moreover, there is a unique sequence x_{n}(α) ∈ (0, 1) such that

α = [ k_{1}(α), …, k_{n−1}(α), k_{n}(α) + x_{n}(α) ]

for each n ≥ 1. Rational numbers in (0, 1] can be accommodated by allowing finite sequences of k_{i} ≥ 1 complemented by zeroes, and similarly for x_{n}(α).

We can specialize (0.3) at the point α and get by induction

α = ( \begin{matrix}0 & 1 \\ 1 & k_{1}(α) \end{matrix} ) … ( \begin{matrix}0 & 1 \\ 1 & k_{n}(α) \end{matrix} ) (x_{n}(α)).  (0.4)

Put

p_{n}(α) := P_{n}(k_{1}(α), …, k_{n}(α)), q_{n}(α) := Q_{n}(k_{1}(α), …, k_{n}(α))

so that p_{n}(α)/q_{n}(α) is the sequence of convergents to α. Denote also

g_{n}(α) := ( \begin{matrix}p_{n−1}(α) & p_{n}(α) \\ q_{n−1}(α) & q_{n}(α) \end{matrix} )

For further use, we reproduce a description of the total set of matrices g_{n}(α). Put

Red_{n} = \{ ( \begin{matrix}0 & 1 \\ 1 & k_{1} \end{matrix} ) … ( \begin{matrix}0 & 1 \\ 1 & k_{n} \end{matrix} ) | k_{1}, …, k_{n} ≥ 1; k_{i} ∈ Z\} ,  (0.5)

and Red := ∪_{n≥1} Red_{n} ⊂ GL(2, Z).

In the terminology of [LewZa1], Red_{n} consists of reduced matrices of length n. The following properties of Red are proved in that paper:

(i) A matrix in GL(2, Z) is reduced iff it has non–negative entries which are non–decreasing downwards and to the right.

(ii) The length l(g) of a reduced matrix g and its representation in the form (0.5) are uniquely defined.

(iii) Reduced matrices are hyperbolic and have two distinct fixed points on the real axis. Every conjugacy class g of hyperbolic matrices in GL(2, Z) contains reduced representatives. They all have the same length l(g), and there are exactly l(g)/k(g) of them where k(g) is the maximal integer such that g = h^{k(g)} for some h.

0.1.1. Generalized Gauss problem. Consider a subgroup of finite index G ⊂ GL(2, Z) and the coset space P = GL(2, Z)/G with the left transitive action of GL(2, Z) on it:

t ↦ ( \begin{matrix}a & b \\ c & d \end{matrix} ) (t)

For any x ∈ [0, 1], t ∈ P, n ≥ 0 put

m_{n}(x, t) := measure of the set {α ∈ (0, 1) | x_{n}(α) ≤ x, g_{n}(α)^{−1}(t_{0}) = t} (0.6)

where t_{0} is the base point of P, the coset of G. Notice that x_{n}(α) = g_{n}(α)^{−1}(α) so that m_{n} is essentially the pullback of the Lebesgue measure on (0, 1) × P with respect to the operator g_{n}(α) acting upon α and t simultaneously. Notice also that g_{n}(α)^{−1} is another notation for the n–th power of the shift operator

T : (α, t) ↦ (\frac{1}{α} − [\frac{1}{α}] , ( \begin{matrix}−[1/α] & 1 \\ 1 & 0 \end{matrix} ) (t))

The first result of this paper is the following generalization of the Gauss–Kuzmin– Lévy formula:

0.1.2. Theorem. Assume that Red (t) = P for each t ∈ P. Then the limit m(x, t) = lim_{n→∞} m_{n}(x, t) exists and equals

m(x, t) = \frac{1}{|P| log 2} log (1 + x).  (0.7)

A proof is given in §1 below. It starts with a straightforward generalization of the Gauss–Kuzmin inductive expression of m_{n+1} through m_{n}. To write it down, consider first the following sets: for y ∈ (0, 1), s ∈ P put

M_{n}(y, s) := {β ∈ (0, 1) | x_{n}(β) ≤ y, g_{n}(β)^{−1}t_{0} = s}.

Then we have, using (0.4) and neglecting the rationals which have measure zero:

M_{n+1}(x, t) = ∐_{k=1}^{∞} \{M_{n} (\frac{1}{k} , ( \begin{matrix}0 & 1 \\ 1 & k \end{matrix} ) (t)) − M_{n} (\frac{1}{x + k} , ( \begin{matrix}0 & 1 \\ 1 & k \end{matrix} ) (t))\} .

Therefore,

m_{n+1}(x, t) = ∑_{k=1}^{∞} \{m_{n} (\frac{1}{k} , ( \begin{matrix}0 & 1 \\ 1 & k \end{matrix} ) (t)) − m_{n} (\frac{1}{x + k} , ( \begin{matrix}0 & 1 \\ 1 & k \end{matrix} ) (t))\} . (0.8)

Derivating (0.8) in x, we get the following equation for the densities, which introduces an important operator L, the generalized Gauss–Kuzmin operator:

m′_{n+1}(x, t) = (Lm′_{n})(x, t) := ∑_{k=1}^{∞} \frac{1}{(x + k)^{2}} m′_{n} (\frac{1}{x + k} , ( \begin{matrix}0 & 1 \\ 1 & k \end{matrix} ) (t)) .  (0.9)

In the classical case, when |P| = 1, at least four ways to deduce (0.7) from (0.9) are known: two “elementary” deductions going back to R. Kuzmin and P. Lévy respectively (see e.g. [Sch]), and two functional analytic methods, based upon the spectral analysis of the operator L formally defined by (0.9): see [Ba], [BaYu] and [May].

Clearly, the limiting measure must be an eigenfunction of L corresponding to the eigenvalue 1, and both analytic proofs show the convergence of L^{n}m′_{0} to this limiting measure by establishing that 1 is the multiplicity one eigenvalue with maximal modulus for the extension of L to an appropriate function space. K. Babenko realizes L as a self–adjoint operator in a Hilbert space, whereas D. Mayer works in the context of nuclear and trace class operators in Banach spaces.

Of these four proofs, we were able to generalize to our context only Mayer’s method. Babenko’s representation seems to be inadequate for proving convergence. However, it might still be useful for numerical calculations, and we present it in §1.3.

The operator L and its deformation L_{s} ((1.1) below) were introduced and studied also in a recent paper [ChMay], of which we became aware only after the first draft of this paper was written.

0.2. Modular curves and geodesics. Our study of the generalized Gauss measure (0.7) was motivated by the relationship between continued fractions and one–dimensional homology of modular curves. We will start by recalling the basic features of this relationship in the form given in [Man1]; see also [Mer] for additional information.

Let G_{0} be a subgroup of finite index in the fractional linear group P SL(2, Z) = SL(2, Z)/(±1). It determines the noncompact modular curve G_{0} \ H (where H is the upper complex half–plane). This curve admits a smooth compactification by a finite number of cusps which are in a natural bijection with the set G_{0} \ P^{1}(Q). Let X_{G_{0}} (or more precisely, X_{G_{0}}(C)) denote this compactification, φ the respective covering map.

For any two points α, β ∈ \overline{H} := H ∪ P^{1}(Q) we can define a real homology class (“modular symbol”) {α, β}_{G_{0}} ∈ H^{1}(X_{G_{0}}, R) by integrating the lifts of differentials ω of the first kind on X_{G_{0}} along the geodesic path connecting α to β:

∫_{{α,β}} ω := ∫_{α}^{β} φ^{∗}(ω).  (0.10)

Modular symbols satisfy the basic additivity and invariance properties:

{α, β}_{G_{0}} + {β, γ}_{G_{0}} = {α, γ}_{G_{0}},

∀g ∈ G_{0}, {gα, gβ}_{G_{0}} = {α, β}_{G_{0}}.  (0.11)

The integrals (0.10) can be related to finite, stably periodic, or general infinite continued fractions, depending on the arithmetical nature of the ends α, β. We will briefly treat these three cases separately.

(i) Finite continued fractions. Assume first that α, β are cusps. It is known that in this case the modular symbol represents a rational homology class (Manin– Drinfeld’s theorem).

By additivity, it suffices to look at the integrals of the form ∫_{0}^{α}, α ∈ Q. Let

g_{k} := ( \begin{matrix}p_{k−1}(α) & p_{k}(α) \\ q_{k−1}(α) & q_{k}(α) \end{matrix} ) , k = 1, …, n, α = \frac{p_{n}(α)}{q_{n}(α)}.

Then, again by additivity, we have

∫_{0}^{α} φ^{∗}(ω) = ∑_{k=1}^{n} ∫_{p_{k−1}(α)/q_{k−1}(α)}^{p_{k}(α)/q_{k}(α)} φ^{∗}(ω) = − ∑_{k=1}^{n} ∫_{{g_{k}(0),g_{k}(i∞)}} ω.  (0.12)

Finally, in view of (0.11), the k–th integral in (0.12) depends only on the class of g_{k} in G_{0} \ P SL(2, Z).

Thus, (0.12) establishes a connection between the distribution of modular symbols {α, β}_{G_{0}} and the distribution of pairs of consecutive convergents to α and β in P_{0} := G_{0} \ P SL(2, Z).

Notice that there is a slight discrepancy with 0.1.1 where we dealt instead with G ⊂ GL(2, Z) and GL(2, Z)/G. In order to reduce the current problem to the former one, we can first replace G_{0} by its lift to SL(2, Z) and then denote by G the

subgroup generated by this lift and ( \begin{matrix}−1 & 0 \\ 0 & 1 \end{matrix} ). We will have a natural identification

P_{0} = G \ GL(2, Z), and this set in turn identifies with P in 0.1.1 under the map g ↦ g^{−1} which is implicit in (0.6).

If α and/or β in (0.10) is real irrational, the integral diverges at this end. In this case, it is natural to study its asymptotic behavior. We will define the “limiting modular symbol” by the following expression whenever it makes sense:

\{\{∗, β\}\}_{G_{0}} := lim \frac{1}{T (x, y)} \{x, y\}_{G_{0}} ∈ H^{1}(X_{G_{0}}, R).  (0.13)

Here x, y ∈ H are two points on the geodesic joining α to β, x is arbitrary but fixed, T (x, y) is the geodesic distance between them, and the limit is taken as y tends to β. In §2 we will prove that if the limit does exist, it depends neither on x nor on α, hence the first argument is replaced by ∗, whereas the double curly brackets remind about the limit. We will discuss the relation of this symbol to continued fractions in two situations.

(ii) Stably periodic continued fractions. Among geodesics with two irrational ends there is an important subclass consisting of geodesics that connect two fixed points of a hyperbolic element in G_{0}. More precisely, any hyperbolic g ∈ G_{0} has two fixed points α^{±}, repelling and attracting, on the real line. Let Λ_{g}^{±} be the respective eigenvalues, 0 < Λ_{g}^{+} < 1. The oriented geodesic in H connecting α_{g}^{−} to α_{g}^{+} is g–invariant, and the action of g induces on it the shift by the geodesic distance λ(g) := log Λ_{g}^{−}.

For any point x on this geodesic, the image of its segment [x, gx] is a parameterized closed loop on X_{G_{0}} missing the cusps. (The supporting set–theoretic loop is ran over exactly k(g) times where k(g) is the maximal k such that g is a k–th power in G_{0}). The homology class of this loop is {0, g(0)}. When we integrate from a fixed x to g^{n}x, we run over the parameterized loop n times, the geodesic length of the path is n λ(g), and its homology class is n {0, g(0)}. Therefore the limiting modular symbol (0.13) exists and equals

\{\{∗, α_{g}^{+}\}\} = \frac{\{0, g(0)\}}{λ(g)}.  (0.14)

The most important generating function for closed geodesics is the Selberg zeta function. However, it encodes only the lengths of closed geodesics in the hyperbolic metric. The usual modular symbol in the numerator of (0.14) depends only on the class of g modulo [G_{0}, G_{0}] and is additive in g (see [Man1], Prop. 1.4). Perhaps, one can construct a generating function for (0.14) as a combination of Selberg’s zetas with abelian characters.

The usual Selberg’s zetas were studied in [May], and then in [LewZa1] for GL and SL separately. It turned out that they could be represented as Fredholm determinants det (1 − L_{s}) and det (1 − L_{s}^{2}) respectively. Literally the same is true in our generalized setting, when subgroups G or G_{0} are introduced. This is proved in [ChMay], and we supply a brief discussion of this in §3.

As we have briefly explained, the distribution of continued fractions and modular symbols at cusps is encoded in the eigenvalues of L_{1}, 1 being the dominant value producing the distribution (0.7).

From the identity det (1 − L_{s}) = Z(s) it follows that the zeroes of Z(s) are exactly those values for which the deformed operator L_{s} has eigenvalue 1.

J. Lewis and D. Zagier produced also an in–depth study of the respective eigenfunctions for the full modular groups GL(2, Z) and SL(2, Z).

(iii) General infinite continued fractions. For this case, we will prove in §2 two results. Namely, we will establish that with an additional assumption the limiting modular symbol (0.13) exists only in a weak sense and and is zero.

0.2.1. Theorem. Assume that Red(t) = P_{0} for each t ∈ P_{0} (transitivity condition). Then (0.13) weakly converges to zero.

For the precise description of the sense in which this convergence holds, see §2.3 and in particular, (2.21).

This vanishing can be compared with a well known interpretation of the Selberg trace formula for compact surfaces: quantum mechanical averages for the geodesic flow can be calculated as if this flow were classically concentrated on closed geodesics.

The transitivity assumption for Red will be checked in §2 in the case G_{0} = Γ_{0}(N). The next result of §2 concerns a series of averaging formulas of a different kind.

Drawing on a lemma of P. Lévy, we will explain in §2 how to calculate averages (over [0, 1]) of some functions of α defined by the sums over all pairs of consecutive convergents of α. Here we will state an interesting particular case, providing averages of weighted modular symbols.

Fix a prime number N > 0 and put G_{0} = Γ_{0}(N). We will assume that the genus of X_{G_{0}} = X_{0}(N) is ≥ 1, otherwise our identities become trivial. Consider a Γ_{0}(N)– invariant differential φ^{∗}(ω) on H which is a cusp eigenform for all Hecke operators and denote by L_{ω}^{(N)}(s) (resp. ζ^{(N)}(s)) its Mellin transform (resp. Riemann’s zeta) with omitted Euler N–factor. More precisely, the coefficients of L_{ω}^{(N)}(s) are Hecke eigenvalues of φ^{∗}(ω)/dz.

0.2.2. Theorem. We have for Re t > 0:

∫_{0}^{1} dα ∑_{n=0}^{∞} \frac{q_{n+1}(α) + q_{n}(α)}{q_{n+1}(α)^{1+t}} ∫_{\{0,q_{n}(α)/q_{n+1}(α)\}} ω =

[\frac{ζ(1 + t)}{ζ(2 + t)} − \frac{L_{ω}^{(N)}(2 + t)}{ζ^{(N)}(2 + t)^{2}} ] ∫_{0}^{i∞} φ^{∗}(ω).  (0.16)

0.3. Related problems and comments. (i) In [Man2], it was shown that after a choice of Schottky uniformization, the Arakelov geometry of a complex curve X at arithmetical infinity can be described in terms of the hyperbolic geometry of geodesics not on X itself, but rather in the hyperbolic handlebody having X as the boundary at infinity. It would be interesting to clarify the statistical aspects of the closed and infinite geodesics in the same vein as above and to relate them to Arakelov geometry. We hope to return to this question later.

(ii) The quotient P GL(2, Z) \ P^{1}(R) can be identified with the space of classes of continued fractions modulo the equivalence relation “k_{n+n_{0}}(α) = k_{n+n_{1}}(β) for some n_{0}, n_{1} and all n”. Classical results on various averages like

lim_{n→∞} n^{−1} ∑_{i=1}^{n} f(k_{i}(α))

state that these averages are almost everywhere constant functions on this space.

On the other hand, this space and its finite coverings corresponding to subgroups G ⊂ P GL(2, Z) constitute the boundary of the analytic moduli stack of elliptic curves whose irrational points are invisible in algebraic geometry (only cusps admit classical algebraic interpretation). According to the emerging general philosophy, this boundary is a bridge to the world of non–commutative geometry, in particular, the limiting geometric objects parameterized by this boundary must be “non–commutative elliptic curves”, perhaps endowed with an additional structure and represented by objects like triangulated categories of their coherent sheaves. For some explanations, see [Soi], [Man4].

Our study concerns rather the geometry of the boundary itself, which also must be considered as (a tower of) “non–commutative modular curves” in Connes’ spirit. From this perspective, the modular complex introduced in [Man1] and further studied in [Mer], [Gon] and other papers, provides a definition of the (co)homology of this boundary compatible with passage to the limit. In the general framework of noncommutative geometry this definition looks however somewhat ad hoc. It would be interesting to relate it to the standard homological constructions of non– commutative geometry.

In the same spirit, the identity (0.16) and its generalizations considered in §2.1– 2.2 show that a considerable part of the theory of automorphic functions in the upper half plane can be recast as the study of averages of certain functions defined on the boundary R as sums of the type (2.1) representing a class of boundary functions with automorphic behavior.

For another family of similar phenomena, see [Za] and [LawZa].

It is interesting to remark in this context that the Gauss–Kuzmin operator L_{s} vaguely looks like a “Hecke operator at the arithmetical infinity”, and has some properties that might be expected of such an operator.

§1. Gauss–Kuzmin operator

1.1. Operator L_{s}. Consider the operator L_{s} acting on functions of two variables (x, t) and formally given by

(L_{s}f)(x, t) := ∑_{k=1}^{∞} \frac{1}{(x + k)^{2s}} f (\frac{1}{x + k} , ( \begin{matrix}0 & 1 \\ 1 & k \end{matrix} ) (t)) .  (1.1)

The variable x here varies in a subset of C stable with respect to all maps x ↦ (x + k)^{−1}, k = 1, 2, . . .. In our context this subset will always contain [0, 1]. The variable t belongs to a finite GL(2, Z)–set P endowed with a base point t_{0}. The parameter s here is real and > 1/2. We are mostly interested in the case s = 1. In §3 we will allow s to take complex values. Finally, f will vary in a linear space of functions stable with respect to L_{s} and containing the function m′_{0}(x, t) = δ_{t,t_{0}} (cf. 0.1.1).

For the proof of Theorem 0.1.2, we want to create a functional analytic context in which the machinery of Krasnoselskii’s theorem as stated in [May], 7.25, becomes applicable. To this end we make the following choices, slightly generalizing Mayer. (They were also made in [ChMay]).

(i) Definition domain. It will be D × P where D := {z ∈ C | |z − 1| < 3/2}. We will call the subsets D×{t} sheets. Notice that each map z ↦ (z +k)^{−1} transforms D strictly into itself.

(ii) Functional spaces. We shall consider the complex Banach space B_{C} := V_{C}(D × P) consisting of functions holomorphic on each sheet and continuous on its boundary. We shall also consider the real Banach space B := V (D×P) of functions holomorphic on each sheet and continuous on its boundary, real at the real points of each sheet. Both spaces are endowed with the supremum norm. These spaces obviously contain δ_{t,t_{0}} and are stable with respect to L_{s} for real s > 1/2. The space B_{C} is also stable with respect to L_{s} with Re s > 1/2.

(iii) Positive cone. Denote by K ⊂ B the cone consisting of functions taking non–negative values at real points of each sheet. We have K ∩ −K = 0 (K is proper) because a nonzero analytic function cannot vanish on an interval. We also have B = K − K (K is reproducing) because f = (f + r) − r, and if r is large and positive, f + r, r ∈ K. Finally, functions positive at all real points of all sheets form the interior of K.

We write f ≤ g if f − g ∈ K.

(iv) L_{s} is K–positive. This means that L_{s}(K) ⊂ K which is obvious.

1.1.1. Lemma. Assume that P contains no proper invariant subsets with respect to the operators Red (see (0.5)).

Then for each nonzero f ∈ K there exist two real positive constants a, b and an integer p ≥ 1 such that a ≤ L_{s}^{p}f ≤ b.

Proof. The upper bound is trivial. Assume that for some f and all p the lower bound is zero. This means that for each p ≥ 1, L_{s}^{p}f vanishes at some point (x_{p}, t_{p}) with x_{p} real in the closure of D. Since in (1.1) all summands are non–negative at real points, when f ∈ K, we see that f must vanish at all points contained in the set ∪_{p≥1}Red_{p}(x_{p}, t_{p}).

From our assumption it follows that for some q and any t ∈ P, Red_{q}(t) = P. By downward induction we deduce first that for any p, L_{s}^{p}f has real zeroes on all sheets. Then, again by downward induction, one sees that for each t there exists a sequence of integers q_{n} → ∞ and real points y_{n} in D such that f(x, t) vanishes at all x ∈ ∪_{n}Red_{q_{n}}(y_{n}). But the intersection of the latter set with [0, 1] is dense in [0, 1]. A nonvanishing holomorphic function cannot have as many zeroes. This contradiction proves our assertion.

1.1.2. Example. Take for G the subgroup generated by the lift of Γ_{0}(N) and the sign change. We will show that Red_{3}(t) = P for any t. In fact, elements of P can be thought of as points of the projective line over Z/N, that is, formal quotients of residues mod N that can be represented by pairwise prime integers ([Man1]). Moreover, this encoding can be chosen compatible with the usual action of GL(2) upon P^{1}.

Let us break these points into three groups:

(I) {u/1 | (u, N) = 1}.

(II) {du/1 | d/N, d > 1, (u, N) = 1}.

(III) {1/du} | d/N, d > 1, (u, N) = 1.

Let us say that s can be obtained from t in one step, if s ∈ Red_{1}(t), that is, s = (t + k)^{−1} for some k. The following statements are straightforward, and taken together, prove our claim.

From any single element of I∪II we can obtain in one step all elements of the set I∪III.

From any single element of III one can obtain in one step an element of II, by adding zero and inverting. Hence from the total III one can obtain the total II in one step.

1.1.3. Lemma. L_{s} : B_{C} → B_{C} is a nuclear operator of order zero, in particular compact and trace class for Re s > 1/2.

Proof. The reasoning is the same as in [May], and we only sketch it. Denote the k–th summand in (1.1) by (π_{s,k}f)(x, t). Each π_{s,k} is nuclear, and ∑_{k} ∥π_{s,k}∥ converges for Re s > 1/2. In fact, the spectrum of π_{s,k} can be easily calculated.

Let z_{k} be the unique fixed point of γ_{k} := ( \begin{matrix}0 & 1 \\ 1 & k \end{matrix} ) in D, and μ_{i}^{(k)} the spectrum

of the permutation induced by this matrix on P. Then the spectrum of π_{s,k} is {(−1)^{n}(z_{k} + k)^{−2(s+n)}μ_{i}^{(k)}}, n ≥ 0.

1.2. Proof of the generalized Gauss–Kuzmin Theorem. We have now checked all the conditions for the applicability of the Theorem 7.25 of [May]; see also [KraLS] for more details.

Using this theorem, we conclude that there exists exactly one eigenfunction of norm one f_{s} of L_{s} in the interior of K and its eigenvalue λ_{0,s} is positive and simple. All other eigenvalues have strictly lesser modulus. For any f ∈ B and ϵ > 0, we have

L_{s}^{n}f = const λ_{0,s}^{n} f_{s} + O(c(ϵ)(q + ϵ)^{n}λ_{0,s}^{n})  (1.2)

as n → ∞, where q = q(L_{s}) < 1 is the spectral margin of L_{s}.

In the case s = 1 we know a positive eigenfunction: this is Gauss’ density \frac{1}{x + 1},

independent of t. The normalization constant is straightforward.

This argument completes the proof of the generalized Gauss–Kuzmin theorem. We will, however, provide some more details about the deduction of (1.2), because this technique can be useful also in the treatment of lower eigenvalues.

The basic result that ensures the existence of eigenfunctions in certain invariant cones is the following ([KraLS], Theorem 9.2):

If K is a cone in a real Banach space B satisfying \overline{K − K} = B and L is a compact operator with LK ⊆ K, and with positive spectral radius r(L), then r(L) is an eigenvalue of L with a corresponding eigenfunction in K.

We also recall some results which enable us to establish that the top eigenvalue of an operator with an invariant cone is simple, and when the rest of the spectrum is of strictly smaller absolute value.

Following [May], we say that an operator L is u–bounded, with respect to a function u ∈ K, if for any f ∈ K there exists some n > 0 and a, b > 0 such that

au ≤ L^{n}f ≤ bu

(Since we allow a power of L, this is a weaker definition of u–boundedness than the one on p. 110 of [KraLS], cf. their remark on p. 111.)

Lemma 1.1.1 shows that L_{s} is u–bounded with respect to the constant function u(x, t) = 1. The lower bound guarantees the positivity of the spectral radius r(L_{s}) and hence the applicability of the Theorem 9.2 of [KraLS]. This fact follows from Lemma 9.2 of [KraLS].

We then have the following result ([KraLS], Theorem 11.1). Assume that the cone K is reproducing and the K–positive operator L is u–bounded. Assume moreover that L has an eigenvalue λ_{0} > 0 with an eigenvector f ∈ K. Then the eigenvalue λ_{0} is simple. A proof of this fact can be obtained using Lemma 11.1 of [KraLS]. This is actually a simple result of linear algebra which uses only the fact that f is an interior point of the cone (nowhere vanishing), and that iterates of the operator L map boundary points of the cone different from {0} to interior points of the cone.

Furthermore, Theorem 11.4 of [KraLS] shows that with the same hypothesis as in the previous result, every eigenvalue λ of L different from λ_{0} satisfies |λ| < λ_{0}. This result follows from the observation that if the operator L is u–bounded then it is also f–bounded, where again f is the eigenfunction in K with eigenvalue λ_{0}. Then, if h is an eigenfunction with eigenvalue λ the estimate

−α(λ_{0} − ϵ)f ≤ λh ≤ α(λ_{0} − ϵ)f

for some ϵ > 0 follows easily, where α > 0 is the smallest positive number such that −αf ≤ h ≤ αf is satisfied. This gives |λ| ≤ λ_{0} − ϵ.

Since in our case we know that the operator L_{s} is compact, the previous result implies that all the other eigenvalues λ (hence all the points in the spectrum of L_{s}) satisfy the estimate |λ| < qλ_{0}, for some q < 1.

Finally, we have a result on the convergence of iterates, cf. Theorem 15.4 of [KraLS].

The cone K in our case contains some ball of positive radius. In this case, Theorem 9.11 on p. 97 of [KraLS] ensures that the adjoint operator L^{∗} acting on the dual Banach space B′ has an eigenfunctional f^{∗} in the adjoint cone K^{∗} of linear K–positive functionals, with eigenvalue λ ≤ r(L) where r(L) is the spectral radius of L. In our case L_{s} has an eigenvalue λ_{0} = r(L_{s}) and a corresponding eigenfunction f which is an interior point of the cone K. Thus, if L^{∗}f^{∗} = λf^{∗}, for a non-trivial f^{∗} ∈ K^{∗}, and if moreover f^{∗}(f) > 0, then λ = λ_{0}, because

λ_{0}f^{∗}(f) = f^{∗}(λ_{0}f) = f^{∗}(Lf) = (L^{∗}f^{∗})(f) = λf^{∗}(f).

Assume that the operator L has a simple eigenvalue equal to the spectral radius, λ_{0} = r(L), and the remaining part of the spectrum lies in the disk |λ| < q r(L) for some q < 1. Let f ∈ K be the eigenfunction of the eigenvalue λ_{0}. Let f^{∗} be an eigenfunctional for L^{∗} in K^{∗}, with eigenvalue λ_{0} as above, satisfying f^{∗}(f) = 1. Then the sequence of iterates

f_{n+1} = Lf_{n}

converges to the eigenfunction f in the following sense.

Define the operators Uh := f^{∗}(h)f, and U^{⊥}h := h − f^{∗}(h)f. We have

lim_{n} \frac{∥U^{⊥}f_{n}∥}{∥Uf_{n}∥} = 0

and the rate of convergence is estimated by

\frac{∥U^{⊥}f_{n}∥}{∥Uf_{n}∥} ≤ c(ϵ)(q + ϵ)^{n} \frac{∥U^{⊥}f_{1}∥}{∥Uf_{1}∥},

for arbitrarily small ϵ > 0. In other words, the iterates converge as fast as a geometric progression with ratio arbitrarily close to the spectral margin q = q(L), cf. §15.2 of [KraLS], and in particular Theorem 15.3, where a more refined estimate of the coefficient c(ϵ) is also given.

Sometimes similar techniques may be applied to the study of the second eigenvalue, by supplying a suitable real Banach space with an invariant cone for the operator h ↦ L_{s}h − λ_{0,s}f^{∗}(h)f, cf. [May].

1.3. The integral kernel operator. In this subsection, we consider the formal operator (1.1) in another functional space, and show that it admits there a representation with integral kernel, generalizing that of [Ba], [BaYu].

We will have to assume additionally that the GL(2, Z)–set P (see 1.1) is such that the action

t ↦ ( \begin{matrix}0 & 1 \\ 1 & k \end{matrix} ) (t)

depends only on k mod N for an appropriate integer N. This assumption is satisfied for instance when G is a congruence subgroup. In the following, we fix such N.

With this assumption, (1.1) can be written as

(L_{s}f)(z, t) = ∑_{p=−N+1}^{0} ∑_{k: k≥1, k≡p (N)} \frac{1}{(k + z)^{2s}} f (\frac{1}{k + z} , ( \begin{matrix}0 & 1 \\ 1 & p \end{matrix} ) (t)) (1.3)

Our new functional space H will consist of functions f(z, t) holomorphic on the sheets {Re z > −1/2} × t.

We first recall a useful identity which is the essential ingredient in the arguments of [Ba], [BaYu], [May], namely

∑_{k≥1} \frac{1}{(k + z)^{2s}} exp (\frac{−ξ}{k + z} ) = ξ^{−s+1/2} ∫_{0}^{∞} η^{s−1/2}e^{−η(z+1/2)} \frac{J_{2s−1}(2\sqrt{ξη})}{2 sinh(η/2)}dη. (1.4)

(This is the formula (111) of [May] rewritten in a way that looks more similar to the corresponding formula in [Ba], [BaYu].)

For our purpose, it is useful to consider also the following corollary of (1.4).

1.3.1. Lemma. Let p ∈ {−N + 1, …, 0}. We have

∑_{k: k≥1, k≡p (N)} \frac{1}{(k + z)^{2s}} exp (−\frac{ξ}{k + z} ) =

N^{2s−2}ξ^{−s+1/2} ∫_{0}^{∞} η^{s−1/2} exp (−η (z + p + N/2)) \frac{J_{2s−1}(2\sqrt{ξη})}{2 sinh (Nη/2)} dη.  (1.5)

Proof. We have

∑_{l≥1} \frac{1}{(p + lN + z)^{2}} exp (−\frac{ξ}{p + lN + z} ) =

\frac{1}{N^{2}} ∑_{l≥1} \frac{1}{(l + (z + p)/N)^{2}} exp (−\frac{ξ/N}{l + (z + p)/N} .) .

Plugging (1.4) in and redenoting Nη as the new η, we get (1.5). It is convenient to write the functions f(z, t) in the form

f(z, t) = ∑_{j=0}^{|P|−1} f_{j}(z)δ_{j}(t),

with respect to a basis of delta functions. We have

δ_{j} (( \begin{matrix}0 & 1 \\ 1 & p \end{matrix} ) (t)) = ∑_{l} A_{jl}(p) δ_{l}(t),

where A(p) is the matrix representing the permutation of the sheets. With this notation, we can write L_{s} in the form

(L_{s}f)(z, t) = ∑_{j=0}^{|P|−1} ∑_{p=−N+1}^{0} ∑_{k: k≥1, k≡p (N)} f_{j} (\frac{1}{k + z} ) \frac{1}{(k + z)^{2s}} δ_{j}(( \begin{matrix}0 & 1 \\ 1 & p \end{matrix} ) (t)) =

∑_{j,l=0}^{|P|−1} ∑_{p=−N+1}^{0} ∑_{k: k≥1, k≡p (N)} f_{j} (\frac{1}{k + z} ) \frac{1}{(k + z)^{2s}} A_{jl}(p)δ_{l}(t).

We now introduce the following operators. Let L denote the Fourier–Laplace transform

(Lg)(z, t) := ∫_{0}^{∞} e^{−ξz}g(ξ, t)dξ.

We also define the multiplication operator

(T g)(ξ, t) := e^{−ξ/2}g(ξ, t),

as in [Ba], [BaYu], and

(Sg)(ξ, t) := S(ξ) g(ξ, t),

with

S(ξ) = \frac{(1 − e^{−Nξ})^{1/2}}{ξ^{−1/2+s}} .

The operator LT is an isometric isomorphism between the space L_{2}((0, ∞) × P) and H endowed with the norm

∥f∥^{2} := \frac{1}{2π} ∑_{j=0}^{|P|−1} ∫_{−∞}^{∞} |f_{j}(0 + iy)|^{2}dy.

Define the matrix function

Θ_{jl}(ξ) := e^{−(N−1)ξ/2} ∑_{p=−N+1}^{0} e^{−ξp}A_{jl}(p).  (1.6)

We can and will choose its square root with complex valued real analytic entries Θ_{jl}^{1/2}(ξ):

Θ_{jl}(ξ) = ∑_{k=0}^{|P|−1} Θ_{jk}^{1/2}(ξ)Θ_{kl}^{1/2}(ξ),

for all ξ ∈ (0, ∞). We define

Θ̂_{jl}(ξ) := e^{−(N−1)ξ/4}Θ_{jl}^{1/2}(ξ).  (1.7)

The results of the following Lemma and Proposition represent the analog in our setting of the main result of [Ba], see also 7.4.2 of [May].

1.3.2. Lemma. For a function f(z, t) = ∑_{j} f_{j}(z) δ_{j}(t) such that

f_{j} = LT S^{−1}h_{j},

we can write the operator L_{s} in the form

(L_{s}f)(z, t) = ∑_{l=0}^{|P|−1}( ∑_{j=0}^{|P|−1} ∫_{0}^{∞} e^{−ηz}S(η)^{−1}Θ_{jl}(η)e^{−η/2}e^{−Nη/4}×

∫_{0}^{∞} κ̃(ξ, η)e^{Nξ/4}e^{−ξ/2}h_{j}(ξ)dξdη) δ_{l}(t)  (1.8)

with a function κ̃(ξ, η) satisfying κ̃(ξ, η) = κ̃(η, ξ).

Proof. Assume that the functions f_{j}(z), for j = 0, …, |P| − 1, are in the range of the operator LT, namely

f_{j} = LT g_{j},

for some function g_{j} in L_{2}(0, ∞). This means that we can write

f_{j}(z) = ∫_{0}^{∞} e^{−ξz}e^{−ξ/2}g_{j}(ξ)dξ.

We now apply (1.5) and obtain

= N^{2s−2} ∫_{0}^{∞} ξ^{−s+1/2} ∫_{0}^{∞} η^{s−1/2}e^{−η(z+p+N/2)} \frac{J_{2s−1}(2\sqrt{ξη})}{2 sinh(Nη/2)}e^{−ξ/2}g_{j}(ξ)dξdη.

With our previous definition of Θ(ξ) as in (1.6) we can write

∑_{p} ∑_{k: k≥1, k≡p (N)} f_{j} (\frac{1}{k + z} ) \frac{1}{(k + z)^{2s}} A_{jl}(p) =

N^{2s−2} ∫_{0}^{∞} e^{−ηz} ∫_{0}^{∞}(\frac{η}{ξ})^{1/2−s} \frac{J_{2s−1}(2\sqrt{ξη})}{2 sinh(Nη/2)}e^{−(ξ+η)/2}Θ_{jl}(η)g_{j}(ξ)dξdη.

Now, for g_{j}(ξ) = S(ξ)^{−1}h_{j}(ξ), this can be rewritten as

∫_{0}^{∞} e^{−ηz}S(η)^{−1}Θ_{jl}(η) ∫_{0}^{∞} \frac{N^{2s−2}J_{2s−1}(2\sqrt{ξη})}{\sqrt{2} sinh(Nη/2)^{1/2}}e^{−(ξ+η)/2}e^{−Nη/4} \frac{g_{j}(ξ)}{ξ^{1/2−s}} dξ dη =

∫_{0}^{∞} e^{−ηz}S(η)^{−1}Θ_{jl}(η) ∫_{0}^{∞} κ̃(ξ, η)e^{−(ξ+η)/2}e^{−Nη/4}e^{Nξ/4}h_{j}(ξ)dξ dη,

where we have set

κ̃(ξ, η) := \frac{N^{2s−2}J_{2s−1}(2\sqrt{ξη})}{2 sinh(Nη/2)^{1/2} sinh(Nξ/2)^{1/2}} .

Now the final step.

1.3.3. Proposition. On the range of R := LT S^{−1}Θ̂, the operator L_{s} satisfies

R^{−1}L_{s}R = M,

where M is the integral kernel operator

(Mζ)(η, t) = ∑_{i=0}^{|P|−1} ∫_{0}^{∞} ∑_{j=0}^{|P|−1} M_{ij}(η, ξ)ζ_{j}(ξ)dξ δ_{i}(t),

with ζ(ξ, t) = ∑_{j} ζ_{j}(ξ)δ_{j}(t). The integral kernel is of the form

M_{ij}(η, ξ) = κ(η, ξ) ∑_{r} Θ̂_{jr}(ξ)Θ̂_{ri}(η),

where the function

κ(ξ, η) = e^{(N/4−1/2)(ξ+η)}κ̃(ξ, η)

still satisfies κ(ξ, η) = κ(η, ξ), but the integral kernel is in general not symmetric. Proof. For a function

f(z, t) = ∑_{j=0}^{|P|−1} f_{j}(z)δ_{j}(t) = ∑_{j=0}^{|P|−1} ∫_{0}^{∞} e^{−ξz}e^{−ξ/2}S(ξ)^{−1} ∑_{i=0}^{|P|−1} Θ̂_{ij}(ξ)ζ_{i}(ξ)dξ δ_{j}(t),

we have

(L_{s}f)(z, t) = ∑_{l=0}^{|P|−1}(L_{s}f)_{l}(z)δ_{l}(t),

with (L_{s}f)_{l}(z) of the form

∑_{j=0}^{|P|−1} ∫_{0}^{∞} e^{−ηz}Θ_{jl}(η)e^{−η/2}S(η)^{−1} ∫_{0}^{∞} e^{−Nη/4}κ̃(ξ, η)e^{Nξ/4}e^{−ξ/2} ∑_{i=0}^{|P|−1} Θ̂_{ij}(ξ)ζ_{i}(ξ)dξdη.

We can write this equivalently as

∑_{k,i=0}^{|P|−1} ∫_{0}^{∞} e^{−ηz}Θ̂_{kl}(η)e^{−η/2}S(η)^{−1} ∫_{0}^{∞} M_{ik}(η, ξ)ζ_{i}(ξ)dξdη,

where we define

M_{ki}(η, ξ) := e^{(N/4−1/2)(η+ξ)}κ̃(η, ξ) ∑_{j=0}^{|P|−1} Θ̂_{ij}(ξ)Θ̂_{jk}(η).

Thus, we have obtained

L_{s} LT S^{−1}Θ̂ ζ = LT S^{−1}Θ̂ M ζ.

1.4. Remarks about the l–adic case. Let l be a prime number, Z_{l} (resp. Q_{l}) the ring of l–adic integers (resp. the field of all l–adic numbers). Put also

\overline{Z}_{l} := \{\frac{n}{l^{r}} | n, r ∈ Z, r ≥ 0\} ∩ (0, l) ⊂ Q, Z_{l} := \overline{Z}_{l} \ \{1, 2, …, l − 1\}. (1.9)

Every irrational α ∈ Q_{l} has a unique representation in the form

α = \frac{a_{−r}}{l^{r}} + ⋯ + \frac{a_{−1}}{l} + a_{0} + a_{1}l + ⋯ + a_{s}l^{s} + …, a_{i} ∈ \{0, 1, …, l − 1\}.

The l–adic norm of this α is |α|_{l} = l^{r} if a_{−r} ≠ 0. We then put

[α]_{l} := \frac{a_{−r}}{l^{r}} + ⋯ + \frac{a_{−1}}{l} + a_{0} ∈ \overline{Z}_{l}.  (1.10)

In more invariant terms, [α]_{l} is the unique element in \overline{Z}_{l} such that |α − [α]_{l}| < 1.

Repeating the usual reasoning, one sees that each irrational l-adic α with |α|_{l} < 1 determines a unique sequence of k_{n}(α) ∈ Z_{l} and x_{n}(α) ∈ lZ_{l} \ {0} such that

α = [ k_{1}(α), …, k_{n−1}(α), k_{n}(α) + x_{n}(α) ]  (1.11)

for each n ≥ 1. We get thus the formalism of the theory of continued fractions in the l–adic setting, in which R, Z, [0, 1) are replaced respectively by Q_{l}, \overline{Z_{l}}, lZ_{l}. Notice that the successive convergents are still rational numbers, but the incomplete quotients k_{n}(α) generally are not integral.

It will be convenient to restrict ourselves to irrational α in Q_{l}^{∗} \ Z_{l}^{∗}. In this case all k_{n}(α) will belong to Z_{l}.

The shift operator is given by T : α ↦ α^{−1} − [α^{−1}]_{l}, and it transforms Q_{l}^{∗} \ Z_{l}^{∗} into itself.

The definition of a (deformed) transfer operator, however, presents interesting new problems. We can consider two basic options.

(A) We can try to define the formal transfer operator by the classical formula

(L_{s}f)(x) := ∑_{k∈Z_{l}} \frac{1}{(x + k)^{2s}} f (\frac{1}{x + k} )  (1.12)

in which Z is replaced by Z_{l}.

We could have included a second argument t, but did not do it in order to focus on the peculiarity of (1.12) apparent already in this straightforward version of the classical setting. Namely, we should not imagine f as a function taking real or

complex values. In fact, otherwise \frac{1}{(x + k)^{2s}} will be defined only at rational points

x and will not tend to zero as k runs over Z_{l} since from the archimedean viewpoint, Z_{l} is a dense set inside [0, l) so that (1.12) will tend to diverge, unless f is highly discontinuous and tends to zero when the denominator of the argument tends to infinity. But this last property will be lost after an application of L_{s}.

However, Z_{l} is discrete and unbounded in the l–adic sense, so that (1.12) still makes sense as an operator in various l–adic function spaces. For example, one can consider the space of analytic functions on lZ_{l} represented by convergent series ∑_{n≥0} a_{n}x^{n}.

This remark, and parallels with the theories of l–adic uniformization and Drinfeld modules, suggest the following problems.

(i) Find a natural Banach space of l–adic functions in which (1.12) would define a compact operator.

Since compact operators are nuclear in the l–adic theory, this would allow us to define l–adic Selberg’s zeta values at integral points 2s > 0 as det (1 − L_{2s}).

(ii) Define the set of l–adic reduced matrices by the same prescription as (0.5), but this time with k_{i} running over Z_{l}.

Can one find a characterization of this set similar to that given in [LewZa1] and reproduced in 0.1? Assuming we know an l–adic zeta, can one find an Euler product for it similar to (3.1) below?

(iii) Again assuming a positive answer to the first question, is there an eigenfunction with eigenvalue 1 of L_{1}? Can one find its measure–theoretic interpretation in terms of Mazur’s theory of l–adic integration? (See e. g. [Man5], §8 and 9).

At this point, we may notice that the passage from T to L_{1} in the classical case implicitly involves integration with respect to the additively invariant measure, since these operators are adjoint via the obvious bilinear form determined by this integration, cf. formula (2.19) below.

It is well known that this invariant measure becomes inadequate for integrating l–adic valued functions, for the simple reason that the smaller is, say, an l–adic ball, the larger is its invariant measure l–adically.

Instead, the l–adic integration invented by B. Mazur for treating l–adic L– functions, utilizes finitely additive functions on open/closed subsets of, say, Z_{l} which take values in bounded subsets of finite–dimensional l–adic spaces. Such measures produce linear functionals on the spaces of functions satisfying the Lipschitz condition. See [Man5], §8 and §9, where a more general class of measures of moderate growth is introduced and studied as well.

Hence another option, perhaps more natural than the formal prescription (1.12) is:

(B) Define and study the transfer operators on various spaces of l–adic measures, as well as appropriate modifications of the questions (i)–(iii).

§2. Calculation of certain averages

2.1. P. Lévy’s lemma. Let f be a complex valued function defined on pairs of coprime integers (q, q′) such that q ≥ q′ ≥ 1 and f(q, q′) = O(q^{−ε}) for some ε > 0. Put for α ∈ (0, 1]

l(f, α) = ∑_{n=1}^{∞} f(q_{n}(α), q_{n−1}(α)).  (2.1)

2.1.1. Proposition (P. Lévy, 1929). We have

∫_{0}^{1} l(f, α)dα = ∑′ \frac{f(q, q′)}{q(q + q′)}.  (2.2)

Sums and integrals in (2.1), (2.2) converge absolutely and uniformly.

A notational convention: prime at the summation sign as in (2.2) refers to the domain q ≥ q′ ≥ 1, (q, q′) = 1.

Proof. This Proposition is an immediate consequence of the following statement. For any q ≥ q′ ≥ 1 with (q, q′) = 1 there exists a unique n ≥ 0 such that one

can find α ∈ (0, 1] with q_{n}(α) = q, q_{n−1}(α) = q′. Moreover, all such α form a

semi-interval of length \frac{1}{q(q + q′)}.

In fact, assume that such an α and n exist, and let p_{n}(α)/q_{n}(α), p_{n−1}(α)/q_{n−1}(α) be the respective convergents to α, then we have

p_{n−1}(α) q_{n}(α) − p_{n}(α) q_{n−1}(α) = (−1)^{n}.

Together with the conditions p_{k}(α) ≤ q_{k}(α) this allows us to reconstruct n uniquely by induction and shows that all α with this property fill the semi-interval

\frac{p_{n−1}(α)z + p_{n}(α)}{q_{n−1}(α)z + q_{n}(α)}, z ∈ (0, 1].

(cf. (0.3)). Conversely, for any (q, q′) we can start with complementing this line by p ≤ q, p′ ≤ q′ to a reduced (2, 2)–matrix with determinant ±1, and then produce the continued fraction for p/q with neighboring convergents p/q, p′/q′. This proves the lemma.

It is often convenient to have the summation domain in (2.2) extended to all q ≥ q′ ≥ 1. One can do this by first extending f to this domain in the following way: choose a function κ : N → Z and a number t and put

F (q, q′) := κ(d) d^{−t} f(q, q′), d := g. c. d. (q, q′).  (2.3)

Then

∑′ \frac{f(q, q′)}{q(q + q′)} = ζ(κ, t)^{−1} ∑_{q≥q′≥1} \frac{F (q, q′)}{q(q + q′)},  (2.4)

where ζ(κ, t) := ∑_{d≥1} κ(d) d^{−t}. (This remark is also contained in [L].)

We will now combine this with the results of [Man1] in order to prove Theorem 0.2.2.

2.2. Averaging weighted modular symbols. In this subsection we keep the notation explained before the statement of the Theorem 0.2.2. In particular, modular symbols refer to the group Γ_{0}(N). We start with the identity (20) in [Man1]:

∑_{d/m} ∑_{b=1}^{d} ∫_{{0,b/d}} ω = (σ(m) − c_{m}) ∫_{0}^{i∞} φ^{∗}(ω).  (2.5)

Here (m, N) = 1, ϕ^{∗}(ω)/dz is a cusp form for Γ_{0}(N) with eigenvalue c_{m} with respect to the Hecke operator T_{m}, σ(m) is the sum of the divisors of m.

Multiply this identity by m^{−2−t} and sum over all m prime to N:

∑_{m: (m,N)=1} \frac{1}{m^{2+t}} ∑_{d/m} ∑_{b=1}^{d} ∫_{\{0,b/d\}} ω = [ ∑_{m: (m,N)=1} \frac{σ(m)}{m^{2+t}} − L_{ω}^{(N)}(2 + t)] ∫_{0}^{i∞} φ^{∗}(ω).

(2.6)

Any symbol \{0, \frac{q′}{q} \}, (q, q′) = 1, occurs in the guise \{0, \frac{b}{d}\} for some d/m only when q divides m, and then exactly τ(mq^{−1}) times where τ is the number of divisors. Hence the integration path in the left hand of (2.6) can be rewritten in the following way:

∑_{m: (m,N)=1} ∑_{q/m} \frac{τ(mq^{−1})}{m^{2+t}} ∑_{q′≤q, (q,q′)=1}\{0, \frac{q′}{q} \} =

∑_{n: (n,N)=1} \frac{τ(n)}{n^{2+t}} ∑_{q: (q,N)=1} \frac{∑_{q′≤q, (q,q′)=1}\{0, \frac{q′}{q} \}}{q^{2+t}} =

ζ^{(N)}(2 + t)^{2} [ ∑_{q: (q,N)=1} \frac{1}{q^{2+t}} ∑_{q′≤q, (q,q′)=1}\{0, \frac{q′}{q} \}] .  (2.7)

Moreover, the first series inside the square brackets in (2.6) equals

ζ^{(N)}(1 + t) ζ^{(N)}(2 + t).

Hence (2.6) divided by ζ^{(N)}(2 + t)^{2} can be rewritten as

∑_{q: (q,N)=1} \frac{1}{q^{2+t}} ∑_{q′≤q, (q,q′)=1} ∫_{\{0, \frac{q′}{q} \}} ω =

[\frac{ζ^{(N)}(1 + t)}{ζ^{(N)}(2 + t)} − \frac{L_{ω}^{(N)}(2 + t)}{ζ^{(N)}(2 + t)^{2}} ] ∫_{0}^{i∞} φ^{∗}(ω).  (2.8)

The left hand side of (2.8) can be represented as the right hand side of (2.2) with the function f(q, q′) which vanishes for q/N and otherwise equals

f(q, q′) = \frac{q + q′}{q^{1+t}} \{0, \frac{q′}{q} \}.  (2.9)

Let us define a new function f̃(q, q′) by the same formula (2.9) for all relatively prime (q, q′). Since N is prime, we have \{0, \frac{q′}{q} \} = \{0, i∞\} for N/q. Therefore, writing (2.2) for this f̃, and integrating φ^{∗}(ω), we get on the left hand side the same expression as on the left hand side of (0.13). The right hand side becomes the sum of the right hand side of (2.8) and

∑_{d=1}^{∞} \frac{ϕ(Nd)}{(Nd)^{2+t}} ∫_{0}^{i∞} φ^{∗}(ω) = [\frac{ζ(1 + t)}{ζ(2 + t)} − \frac{ζ^{(N)}(1 + t)}{ζ^{(N)}(2 + t)}] ∫_{0}^{i∞} φ^{∗}(ω)  (2.10)

where ϕ is the Euler function. This sum equals the right hand side of (0.16). This completes the proof.

2.2.1. Comments and variations. The distribution of modular symbols was studied by D. Goldfeld ([Gol1], [Gol2]) who has found interesting connections between the conjectural asymptotic behavior of certain sums involving such symbols and other number–theoretical problems, e. g. the abc–conjecture. One of Goldfeld’s conjectures reads:

∑_{\begin{matrix}c^{2}M^{2}+d^{2}≤X \\ c≡0 (N) \end{matrix}}\{0, \frac{b}{d}\}_{N} ∼ R(iM) X

as X → ∞, where the sum is taken over matrices in Γ_{0}(N), R(iM) := ∫_{iM}^{i∞}, and both sides are considered as functionals on the space of Γ_{0}(N) cusp forms of weight two.

D. Goldfeld and C. O’Sullivan introduced a class of Eisenstein series twisted by modular symbols and established their analytic properties. The simplest series of this kind can be represented as the right hand side of (2.2) if one chooses for f the following function (depending on z, s as parameters):

\frac{f(q, q′)}{q(q + q′)} := χ(q) \{0, \frac{q′}{q} \}_{N} ∑_{g∈A_{q,q′}} Im (gz)^{s}

where A_{q,q′} is the set of matrices in Γ_{0}(N) with the second column (q′, q)^{t}.

Here is another class of quite simple functions f that might produce interesting specializations of (2.2):

\frac{f(q, q′)}{q(q + q′)} := \frac{χ_{1}(q) χ_{2}(q′)}{q^{s_{1}}q^{′s_{2}}}.

They lead to some identities involving double logarithms at roots of unity at the right hand side of (2.2). As Goncharov has shown in [Gon], relations between these numbers can be described in terms of the modular complex for Γ_{1}(N). This stresses the relevance of the modular symbols in the study of the distribution of continued fractions.

Our last example is a function that was introduced in [AlZa]:

\frac{f(q, q′)}{q(q + q′)} := x^{∑ k_{j}(q/q′)−1} q log_{2} q.

2.3. Proof of the Theorem 0.2.1. We now return to the notation and conventions explained in the paragraph around formula (0.13). In particular, β is real irrational. We start with proving that whenever the limit (0.13) exists, it does not depend on α (independence on x ∈ H with fixed α is obvious).

We will compare the behavior of (0.13) for two geodesics Γ_{1}, Γ_{2} ending at β. It suffices to consider the case when Γ_{1} starts from i∞, whereas Γ_{2} starts from some real α < β.

Denote by p_{n}(β)/q_{n}(β) = p_{n}/q_{n} the convergents to β. If n is large enough and has the appropriate parity, the respective convergents will have the following positions on the real line:

\frac{α + β}{2} < \frac{p_{n−1}}{q_{n−1}} < β < \frac{p_{n}}{q_{n}} .  (2.11)

Besides, we always have

|\frac{p_{n−1}}{q_{n−1}} − β| > |\frac{p_{n}}{q_{n}} − β| .  (2.12)

Let [a, b] generally denote the geodesic joining a to b. Define two sequences of points in H by

z_{n} := Γ_{1} ∩ [\frac{p_{n−1}}{q_{n−1}} , \frac{p_{n}}{q_{n}} ] , ζ_{n} := Γ_{2} ∩ [\frac{p_{n−1}}{q_{n−1}} , \frac{p_{n}}{q_{n}} ] .  (2.13)

From (2.11)–(2.13) it follows that

\frac{1}{2q_{n}q_{n+1}} < Im z_{n} < \frac{1}{2q_{n−1}q_{n}} ,

and if moreover n is large enough,

\frac{θ}{2q_{n+2}q_{n+1}} < θ Im z_{n+1} < Im ζ_{n} ≤ \frac{1}{2q_{n−1}q_{n}}

where θ is some fixed constant between 0 and 1. The easiest way to convince oneself of this is to look at a picture containing all the relevant geodesics.

The geodesic distance from any fixed x_{1} ∈ Γ_{1} to z ∈ Γ_{1} equals −log Im z + O(1) as z → β. The similar distance from a fixed x_{2} ∈ Γ_{2} to ζ ∈ Γ_{2} to β equals −log Im ζ + O(1) as ζ → β.

Taking into account our inequalities and the additivity of modular symbols, we obtain

\frac{1}{T (x_{1}, z_{n})} \{x_{1}, z_{n}\} = \frac{1}{T (x_{2}, ζ_{n}) + O(1)} [\{x_{2}, ζ_{n}\} + O(1)].  (2.15)

From this and (2.14) it follows that both limits exist or otherwise simultaneously, and have a common value whenever they both exist.

Moreover, according to the Khintchin–Lévy theorem we have for almost all β

log q_{n} = Cn(1 + o(1)), C = \frac{π^{2}}{12 log 2}  (2.16)

as n → ∞. Hence for almost all β we can replace the limit (0.13) by

lim_{n→∞} \frac{1}{2Cn} \{i∞, z_{n}\} = lim_{n→∞} \frac{1}{2Cn} ∑_{i=1}^{n} \{\frac{p_{i−1}(β)}{q_{i−1}(β)}, \frac{p_{i}(β)}{q_{i}(β)}\} .  (2.17)

Temporarily fixing n, we will consider the sum of modular symbols on the right hand side of (2.17) as a function of β, and then prove that the resulting sequence of functions weakly converges to zero in the L_{2}–sense. For this, we need the following lemma.

2.3.1. Lemma. Let P_{0} be a finite left GL(2, Z)–set such that Red^{−1}(t) = P_{0} for each t ∈ P_{0}. Let φ : P_{0} → H be a function with values in an R–vector space, t_{0} ∈ P_{0}. Then we have

lim_{n→∞} \frac{1}{n} ∑_{i=1}^{n} φ(g_{i}(x)^{−1} t_{0}) = \frac{1}{|P_{0}|} ∑_{s∈P_{0}} φ(s),  (2.18)

where the limit is taken in the sense of weak convergence in L_{2}([0, 1] × P_{0}), and

g_{k}(x) = ( \begin{matrix}p_{k−1}(x) & p_{k}(x) \\ q_{k−1}(x) & q_{k}(x) \end{matrix} ) .

Proof. Consider the shift

T (x, t) = (\frac{1}{x} − [\frac{1}{x}] , ( \begin{matrix}−[1/x] & 1 \\ 1 & 0 \end{matrix} ) (t))

We denote by λ the measure on [0, 1]×P_{0} given by the standard Lebesgue measure on [0, 1] and the counting measure on P_{0}.

The Gauss–Kuzmin operator L = L_{1} that we discussed in §1 is the adjoint of this shift T, in the sense that, for any function h ∈ L_{1}([0, 1] × P_{0}, λ) and any f ∈ B_{C} we have

∫_{[0,1]×P_{0}} f · Lh dλ = ∫_{[0,1]×P_{0}}(f ∘ T) h dλ.  (2.19)

The eigenfunctional of of L^{∗} denoted f^{∗} ∈ K^{∗} in §1.2 can be taken as

h ↦ ∫_{[0,1]×P_{0}} h(x, t)dλ(x, t).

From 1.2 it follows that for any h ∈ B_{C} we have strong convergence

lim_{n→∞} \frac{1}{n} ∑_{k=1}^{n}(L^{k}h)(x, t) = \frac{1}{|P_{0}| log 2} \frac{1}{1 + x} ∫_{[0,1]×P_{0}} h dλ.  (2.20)

According to (2.19), this is equivalent to the convergence

∫_{[0,1]×P_{0}} \frac{1}{n} ∑_{k=1}^{n} f(T^{k}(x, t))h(x, t)dλ(x, t) →

\frac{1}{|P_{0}| log 2} (∫_{[0,1]×P_{0}} \frac{f(x, t)}{1 + x}dλ(x, t)) ∫_{[0,1]×P_{0}} h dλ,

for any f ∈ B_{C} and any test function h ∈ B_{C}. If we consider a function f(x, t) = φ(t), independent of x ∈ [0, 1], we obtain that for any t

\frac{1}{n} ∑_{k=1}^{n} φ(g_{k}(x)^{−1}t) → \frac{1}{|P_{0}|} ∑_{s∈P_{0}} φ(s)

weakly in L_{2}, because among the test functions we have all polynomials.

This is equivalent to (2.18) for C–valued functions and therefore also for vector– valued ones.

We can now conclude the proof of the Theorem 0.2.1. If our modular curve is G_{0} \ \overline{H}, we put P_{0} = P SL(2, Z)/G_{0} and consider P_{0} as a left GL(2, Z)–set as explained in the Introduction.

Since modular symbols are left G_{0}–invariant, we can find a function φ and t_{0} ∈ P_{0} such that

φ(g_{k}(β)^{−1} t_{0}) = \{g_{k}(β)(0), g_{k}(β)(i∞)\} = \{\frac{p_{k−1}(β)}{q_{k−1}(β)}, \frac{p_{k}(β)}{q_{k}(β)}\} .

It follows from (2.18) that the weak limit (2.17) is

\frac{1}{2C |P_{0}|} ∑_{k}\{h_{k}(0), h_{k}(i∞)\}

where h_{k} now run over a complete set of representatives of P_{0}. But this last sum vanishes. In fact, let

σ = ( \begin{matrix}0 & −1 \\ 1 & 0 \end{matrix} ) .

Then {h_{k}σ} as well is a complete system of representatives, and

{σ(0), σ(i∞)} = −{0, i∞}.

Let us stress that the pointwise behavior of (2.17) might be wildly oscillating. We proved only that for any measurable set E we have

lim_{n→∞} \frac{1}{n} ∫_{E} ∑_{i=1}^{n} \{\frac{p_{i−1}(β)}{q_{i−1}(β)}, \frac{p_{i}(β)}{q_{i}(β)}\} dβ = 0.  (2.21)

§3. Selberg’s zeta function

3.1. Notation. In this section we explain the definition of Selberg’s zeta for subgroups of finite index G ⊂ GL(2, Z) and G_{0} ⊂ SL(2, Z), their representation as Fredholm determinant, and relations to geodesics on modular curves. We closely follow [LewZa1], pp. 3–6, whose version requires only minor modifications. For a much more comprehensive treatment, see [ChMay].

As in [LewZa1], for g ∈ GL(2, Z) put

D(g) = Tr (g)^{2} − 4 det (g), N(g) = (\frac{Tr (g) + D(g)^{1/2}}{2} )^{2} ,

and call g hyperbolic if Tr (g) and D(g) are positive. A hyperbolic matrix is primitive if it is not a nontrivial power of an element of GL(2, Z). For a hyperbolic g set

χ_{s}(g) = \frac{N(g)^{−s}}{1 − det (g) N(g)^{−1}} .

As above, put P := GL(2, Z)/G and denote by ρ_{P} the natural representation of GL(2, Z) in the space of functions on P. Finally, put

Z_{G}(s) := ∏_{g∈Prim} ∏_{m=0}^{∞} det [1 − det (g)^{m} N(g)^{−s−m} ρ_{P}(g)]  (3.1)

where Prim is a set of representatives of all GL(2, Z)–conjugacy classes of primitive hyperbolic elements of GL(2, Z).

For G_{0} ⊂ SL(2, Z), we define Z_{G_{0}}(s) in the same way, replacing Prim by Prim_{0}, a set of representatives of all SL(2, Z)–conjugacy classes of primitive hyperbolic elements of SL(2, Z), and P by P_{0}.

3.2. Theorem. We have

Z_{G}(s) = det (1 − L_{s}), Z_{G_{0}}(s) = det (1 − L_{s}^{2})  (3.2)

where L_{s} is given by (1.1) and considered as a nuclear operator in the space B_{C}.

We give only a sketch of formal calculations for Z_{G}(s). Using notation as in the proof of Lemma 1.1.2, we have

−log det (1 − L_{s}) = ∑_{l=1}^{∞} \frac{Tr L_{s}^{l}}{l} = Tr (∑_{l=1}^{∞} \frac{1}{l} (∑_{n=1}^{∞} π_{s,n})^{l}) =

Tr ( ∑_{g∈Red} \frac{1}{l(g)}π_{s}(g)) = ∑_{g∈Hyp} \frac{1}{k(g)} χ_{s}(g) τ_{g}.  (3.3)

Here we define the operator π_{s}(g) for a reduced matrix g as in (0.5) as the product of the respective π_{s,k_{i}}, and l(g) means its length. Hyp denotes a set of representatives of all conjugacy classes of hyperbolic matrices, and k(g) the maximal integer such that g = h^{k(g)}. The last piece of notation is

τ_{g} := Tr (ρ_{P}(g)) = card {t ∈ P | g(t) = t}.  (3.4)

The appearance of τ_{g} is explained by the fact that our π_{s}(g) acts as the tensor product of the similar operator π_{s}(g) for the case G = GL(2, Z) and of ρ_{P}, and our trace is the product of the respective traces.

Using the properties (i)–(iii) of Red summarized at the end of 0.1, we can keep rewriting (3.3):

∑_{g∈Hyp} \frac{1}{k(g)} χ_{s}(g) τ_{g} = ∑_{g∈Prim} ∑_{k=1}^{∞} \frac{1}{k} \frac{N(g)^{−ks} τ_{g^{k}}}{1 − det (g)^{k} N(g)^{−k}} .  (3.5)

On the other hand, from (3.1) we find:

−log Z_{G}(s) = ∑_{g∈Prim} ∑_{m=0}^{∞} Tr ∑_{k=1}^{∞} \frac{1}{k} det (g)^{mk} N(g)^{−(s+m)k} ρ_{P}(g^{k}) =

∑_{g∈Prim} ∑_{k=1}^{∞} \frac{1}{k} \frac{N(g)^{−ks} τ_{g^{k}}}{1 − det (g)^{k} N(g)^{−k}}  (3.6)

This finishes the formal argument which differs from that of [LewZa1], §1, only by the presence of τ_{g}. The subsequent check of convergence in [LewZa1] and the argument of §3 concerning SL generalize in the same straightforward way.

Finally, in order to interpret (3.6) in the language of closed geodesics, it remains only to notice that if G ⊂ GL(2, Z) is the lift of G_{0} ⊂ P SL(2, Z) as in the Introduction, then G_{0} \ H can be naturally identified with GL(2, Z) \ (H × P), and any closed geodesic on the respective modular curve is covered by the geodesics [α_{g}^{−}, α_{g}^{+}] lying on those sheets which are left invariant by the respective hyperbolic matrix g ∈ G, in agreement with (3.4).

References

[AlZa] J. C. Alexander, D. Zagier. The entropy of a certain infinitely convolved Bernoulli measure. J. Lond. Math. Soc., (2), 44 (1991), 121–134.

[Ba] K. I. Babenko, On a problem of Gauss. Dokl. Akad. Nauk SSSR, Tom 238 (1978) No. 5, 1021–1024. (English translation: Soviet Math. Dokl. Vol.19 (1978) N.1, 136–140.)

[BaYu] K. I. Babenko, S. P. Yurev, On a problem of Gauss. Sel. Math. Sov. Vol. 2 (1982) N.4, 331–378.

[ChMay] C.-H. Chang, D. Mayer. Thermodynamic formalism and Selberg’s zeta function for modular groups. Regular and Chaotic Dynamics, vol. 5, No 3 (2000), 281–312.

[Gol1] D. Goldfeld. Zeta functions formed with modular symbols. In: Proc. Symp. Pure Math.. vol. 66, 1 (1999), 111–121.

[Gol2] D. Goldfeld. The distribution of modular symbols. In: Number Theory in Progress (A. Schinzel’s Festschrift, vol. 2, Walter de Gruyter, Berlin–New York, 1999, 849–865.

[Gon] A. Goncharov. The double logarithm and Manin’s complex for modular curves. Math. Res. Letters, 4(1997), 617–636.

[KraLS] M. Krasnosel’skij, Je. Lifshits, A. Sobolev. Positive linear systems. Heldermann Verlag, 1989.

[L] P. Lévy. Sur les lois de probabilité dont dépendent les quotients complets et incomplets d’une fraction continue. Bull. Soc. Math. France, 557 (1929), 178–194.

[LawZa] R. Lawrence, D. Zagier. Modular forms and quantum invariants of 3–manifolds. In: Asian J. Math. 3 (1999) (Atiyah’s Festschrift), No. 1, 93–107.

[LewZa1] J. Lewis, D. Zagier. Period functions and the Selberg zeta function for the modular group. In: The Mathematical Beauty of Physics, Adv. Series in Math. Physics 24, World Scientific, Singapore, 1997, pp. 83–9.

[LewZa2] J. Lewis, D. Zagier. Period functions for Maass wave forms. Preprint.

[Man1] Yu. Manin. Parabolic points and zeta-functions of modular curves. Math. USSR Izvestija, vol. 6, No. 1 (1972), 19–64, and Selected Papers, World Scientific, 1996, 202–247.

[Man2] Yu. Manin. Explicit formulas for the eigenvalues of Hecke operators. Acta Arithmetica, 24 (1973), 239–249.

[Man3] Yu. Manin. Three–dimensional hyperbolic geometry as ∞–adic Arakelov geometry. Inv. Math., 104 (1991), 223–244.

[Man4] Yu. Manin. Mirror symmetry and quantization of abelian varieties. Preprint math.AG/0005143

[Man5] Yu. Manin. Periods of parabolic forms and p–adic Hecke series. Math. USSR Sbornik, 21:3 (1973), 371–393, and Selected Papers, World Scientific, 1996, 268–297.

[May] D. Mayer. Continued fractions and related transformations. In: Ergodic Theory, Symbolic Dynamics and Hyperbolic Spaces, Eds. T. Bedford et al., Oxford University Press, Oxford 1991, pp. 175–222.

[Mer] L. Merel. Intersections sur les courbes modulaires. Manuscripta Math., 80 (1993), 283–289.

[O’S] C. O’Sullivan. Properties of Eisenstein series formed with modular symbols. J. reine u. angew. Math., 518 (2000), 163–186.

[Sch] F. Schweiger. Ergodic theory of fibred systems and metric number theory. Clarendon Press, 1995.

[Soi] Y. Soibelman. Quantum tori, mirror symetry and deformation theory. Preprint math.QA/0011162

[Wa] M.S. Waterman, Remarks on invariant measures for number theoretic transformations. Monatshefte für Mathematik, 79 (1975) 157–163.

[Za] D. Zagier. Vassiliev invariants and a strange identity related to the Dedekind eta–function. Preprint MPIM 99–78.