Rational billiards and flat structures

Howard Masur^{∗} and Serge Tabachnikov^{†}

Contents

1 Billiards, polygonal billiards, rational billiards 2 1.1 General billiards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Polygonal billiards . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.3 Examples: a pair of ellastic point-masses on a segment and a triple of

point-masses on a circle . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.4 Unfolding billiard trajectories, rational polygons . . . . . . . . . . . . 6 1.5 Example: billiard in the unit square . . . . . . . . . . . . . . . . . . . 7 1.6 Rational billiard determines a flat surface . . . . . . . . . . . . . . . . 9 1.7 Minimality of the billiard flow in rational polygons . . . . . . . . . . 11 1.8 Rational billiards and interval exchange maps . . . . . . . . . . . . . 14 1.9 Flat metrics and quadratic differentials . . . . . . . . . . . . . . . . . 15

2 Teichmuller space, strata of quadratic differentials and SL(2, R) action 19 2.1 Teichmuller space and mapping class group . . . . . . . . . . . . . . . 19 2.2 Compactifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 2.3 Strata of quadratic differentials . . . . . . . . . . . . . . . . . . . . . 20 2.4 SL(2, R) action on quadratic differentials . . . . . . . . . . . . . . . . 21

3 Ergodicity 22 3.1 Veech nonuniquely ergodic example . . . . . . . . . . . . . . . . . . . 22 3.2 Ergodicity in almost every direction . . . . . . . . . . . . . . . . . . . 28 3.3 A combinatorial construction . . . . . . . . . . . . . . . . . . . . . . 31 3.4 Further ergodicity results . . . . . . . . . . . . . . . . . . . . . . . . . 35 3.5 Ergodicity of general polygonal billiards . . . . . . . . . . . . . . . . 36 3.6 Construction of polygons with ergodic billiard flow . . . . . . . . . . 40

∗University of Illinois at Chicago †University of Arkansas, Fayetteville

4 Periodic orbits 40 4.1 Periodic directions are dense . . . . . . . . . . . . . . . . . . . . . . . 40 4.2 Counting saddle connections and maximal cylinders . . . . . . . . . . 46

5 Veech groups 47 5.1 Definition and examples of Veech groups . . . . . . . . . . . . . . . . 47 5.2 Veech dichotomy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 5.3 Asymptotics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 5.4 Covers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

6 Miscellaneous results 56 6.1 Stable periodic trajectories . . . . . . . . . . . . . . . . . . . . . . . . 56 6.2 Encoding billiard trajectories. Polygonal billiards have zero entropy . 58 6.3 Complexity of billiard trajectories in rational polygons . . . . . . . . 61 6.4 Periodic trajectories in some irrational billiards . . . . . . . . . . . . 65 6.5 A non-periodic trajectory that is not dense in the configuration space 67

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

Acknowledgements

Both authors were supported by the National Science Foundation grants. The second author is grateful to Max-Planck-Institut in Bonn for its hospitality.

1 Billiards, polygonal billiards, rational billiards

1.1 General billiards

The topics of this survey are plane polygonal billiards, rational billiards, and their generalization to quadratic differentials or flat structures. However, to put the expositon into a perspective, we start with some generalities on the billiard dynamical systems (see, e.g., [10], [30], [48]).

A billiard table Q is a Riemannian manifold with a piece-wise smooth boundary ∂Q. The billiard dynamical system in Q is generated by the free motion of a masspoint (thought of as the billiard ball) subject to the elastic reflection in the boundary. This means that the point moves along a geodesic line in Q with a constant (say, unit) speed until it hits ∂Q. At a smooth boundary point the billiard ball instantaneously reflects so that the tangential component of its velocity remains the same while the normal component changes sign. In dimension two, the case of interest in this paper, the reflection is described by the familiar law of geometrical optics: the angle of incidence equals the angle of reflection. If the billiard ball hits a non-smooth boundary point (a corner) its further motion is not defined (with a few exceptions mentioned below).

The time-t billiard map acts on tangent vectors to Q, namely, on those tangent vectors whose trajectories undergo finitely many reflections in the boundary and avoid corners on the time interval [0, t].

Thus the billiard flow is the geodesic flow on a manifold with boundary; this flow is discontinuous at the moments when the billiard ball reflects in the boundary. Denote by σ the billiard reflection at a smooth boundary point x; then σ is the linear reflection of the tangent space T_{x}Q in the hyperplane T_{x}(∂Q). It is convenient to identify the tangent vectors (x, v) and (x, σ(v)) for every smooth boundary point x and every tangent vector v ∈ T_{x}Q. These identifications being made, the billiard flow becomes continuous at the smooth boundary points; however the discontinuities, corresponding to the corners of ∂Q, persist.

One may consider the billiard flow as a Hamiltonian or as a Lagrangian dynamical system. This point of view will not play role in the present paper, so we mention it only in passing. The cotangent bundle T^{∗}Q carries the canonical symplectic structure ”dp ∧ dq”; using the identification of the tangent and cotangent spaces induced by the Riemannian metric, one obtains a symplectic structure on T Q. The billiard flow preserves this symplectic structure. The dual, Lagrangian viewpoint is that the billiard trajectories extremize the distance between their points. For example, a closed three-link billiard trajectory in Q has an extremal length among the geodesic triangles whose vertices lie on the boundary ∂Q.

Assume that Q is geodesically convex and compact. In the spirit of geometrical optics one may consider the billiard transformation acting on the oriented geodesics in Q (thought of as rays of light): the geodesic ray that contains a segment of the trajectory of the billiard ball, oriented in the direction of its motion, is sent to the ray that contains the next segment of this trajectory after the reflection in the boundary. The billiard transformation preserves the canonical symplectic structure of the space of oriented geodesics obtained from the symplectic structure in the cotangent bundle by symplectic reduction.

The described construction reduces the dimension of the phase space by one: the unit tangent bundle is replaced by its quotient space of geodesic rays, and the billiard flow – by the billiard map. Taking the first intersection point of a geodesic ray with the boundary, one also describes the phase space of the billiard map as the space of the unit tangent vectors (x, v) ∈ T Q whose foot point x is a smooth boundary point and the vector v has the inward direction.

This paper concerns the dynamics of billiards. The billiard flow has an invariant measure (indiced by the invariant symplectic form): this measure in the unit tangent bundle is the product of the measure on Q induced by the Riemannian volume form and the Lebesgue measure on the unit tangent spheres. One can prove that for every billiard table the billiard flow is defined for all times on a subset of full measure in the unit tangent bundle of Q.

1.2 Polygonal billiards

Informally speaking, the theory of mathematical billiards can be partitioned into three areas: convex billiards with smooth boundaries, billiards in polygons (and polyhedra) and dispersing and semi-dispersing billiards (similarly to differential geometry in which the cases of positive, zero and negative curvature are significantly different). These areas differ by the types of results and the methods of study: in the former a prominent role is played by the KAM theory and the theory of area preserving twist maps; the latter concerns hyperbolic dynamics and has much in common with the study of the geodesic flow on negatively curved manifolds. The recent progress in the study of polygonal billiards is mostly due to applications of the theory of flat structures on surfaces (a.k.a. quadratic differentials) and the study of the action of the Lie group SL(2, R) on the space of quadratic differentials which is the main topic of this paper.

From now on we will be considering plane polygonal billiards; this means that Q is a compact polygon in the plane (in general, not necessarily convex or simply connected). We describe the invariant measure μ for the billiard transformation induced by the above mentioned invariant measure of the billiard flow.

The phase space of T consists of the inward unit tangent vectors (x, v) whose foot point x is an interior point of a side of Q. Let t be the length parameter along the perimeter of Q and let θ be the angle made by v with the respective side. Then the invariant measure is given by the next formula:

μ = sin θ dθ dt.

One often needs to consider parallel beams of billiard trajectories in polygons. For example, every even-periodic trajectory includes into a strip consisting of a oneparameter family of parallel periodic trajectories of the same period and length. An odd-periodic trajectory also includes into a parallel strip, this time consisting of trajectories whose period and length is twice as great. Note that the billiard reflection in a side of Q transforms a parallel beam to another family of parallel trajectories, and that the width of the beam remains the same.

Fig. 1

Part of the interest in polygonal billiards comes from the fact that they are closely related to problems of mechanics. We discuss one such simple example below.

1.3 Examples: a pair of ellastic point-masses on a segment and a triple of point-masses on a circle

Consider two points with masses m_{1} and m_{2} on the unit segment [0, 1]. The points may elastically collide and reflect from the end points of the segment (”walls”). The configuration space of the system is given by the inequalities 0 ≤ x_{1} ≤ x_{2} ≤ 1 where x_{1} and x_{2} are the coordinates of the points. Rescale the variables: x̄_{i} = \sqrt{m}_{i}x_{i}, i = 1, 2. The configuration space is now a right triangle in the (x̄_{1}x̄_{2})-plane with the acute angle tan^{−1}(\sqrt{m_{1}/m_{2}}).

Consider a collision of the points. Let v_{1}, v_{2} be the velocities before, and u_{1}, u_{2} – after the collision. The conservation of momentum and energy laws read:

m_{1}u_{1} + m_{2}u_{2} = m_{1}v_{1} + m_{2}v_{2},
m_{1}u_{1}^{2}/2 + m_{2}u_{2}^{2}/2 = m_{1}v_{1}^{2}/2 + m_{2}v_{2}^{2}/2.

In the rescaled coordinates the velocities are rescaled by the same factors, therefore

\sqrt{m}_{1}ū_{1} + \sqrt{m}_{2}ū_{2} = \sqrt{m}_{1}v̄_{1} + \sqrt{m}_{2}v̄_{2},
ū_{1}^{2} + ū_{2}^{2} = v̄_{1}^{2} + v̄_{2}^{2}.

The latter equation says that the magnitude of the vector (ū_{1}, ū_{2}) does not change after the collision; the former one says that the scalar product of this vector with the vector (\sqrt{m}_{1}, \sqrt{m}_{2}) is preserved as well. The vector (\sqrt{m}_{1}, \sqrt{m}_{2}) is tangent to the side of the configuration triangle given by the equation x̄_{1}/\sqrt{m}_{1} = x̄_{2}/\sqrt{m}_{2}.

Thus the configuration trajectory reflects in this side according to the billiard reflection law. Likewise one considers collisions with the walls x = 0 and x = 1: they correspond to the billiard reflections in the other two sides of the configuration triangle. One concludes that the dynamical system of two elastic particles on a segment is isomorphic to the billiard in a right triangle whose shape depends on the ratio of the masses of the particles.

Similar arguments show that the system of three elastic point-masses on the circle with a fixed center of mass is isomorphic to the billiard in an acute triangle (we learned this example from E. Gutkin). Let the masses be m_{1}, m_{2}, m_{3}; then the angles α_{i} of the triangle are given by

tan α_{i} = m_{i}\sqrt{\frac{m_{1} + m_{2} + m_{3}}{m_{1}m_{2}m_{3}}} , i = 1, 2, 3.

In the limit m_{3} → ∞ one obtains the previous example of two point-masses on a segment.

Many other mechanical models reduce to billiards, in particular, the model of gas in a closed vessel as a collection of elastic balls in a compact domain. Such models are of great importance in statistical physics.

1.4 Unfolding billiard trajectories, rational polygons

Instead of reflecting a billiard trajectory in a side of the billiard polygon Q one may reflect Q in this side and unfold the trajectory to a straight line. This process is iterated at each reflection: each succesive copy of the billiard polygon is obtained from the previous one by the reflection in the side, met by the straightened trajectory. This unfolding method has many applications in the study of polygonal billiards.

Fig. 2

Let A(Q) be the group of motions of the plane generated by the reflections in the sides of Q. Denote the reflection in the side s by σ_{s}. Notice that for every two sides s and t of Q one has:

σ_{σ_{s}(t)} = σ_{s}σ_{t}σ_{s}.

It follows that every copy of Q involved in the unfolding is the image of Q under an element of the group A(Q). The product of an even number of elements of this group preserves orientation while an odd number reverses it.

To keep track of the directions of billiard trajectories in Q consider the group G(Q) that consists of the linear parts of the motions from A(Q). This subgroup of the orthogonal group is generated by the reflections in the lines through the origin which are parallel to the sides of the polygon Q. The group G(Q) acts on the unit circle. When a billiard trajectory in Q reflects in a side s its direction is changed by the action of the element of G(Q) which is the projection of σ_{s} to G(Q).

Definition 1.1 A billiard polygon Q is called rational if the group G(Q) is finite and irrational otherwise.

If Q is a rational polygon then the group G(Q) is the dihedral group of symmetries of a regular polygon. A necessary condition for Q to be rational is that all its angles are rational multiples of π. It is also sufficient if the boundary is connected; that is, the polygon is simply connected. The set of plane n-gons can be considered as a subset in R^{2n} (each vertex has two degrees of freedom). We give the space of n-gons the subspace topology. Rational polygons are dense in the space of polygons.

A given billiard trajectory in a rational billiard table will have only finitely many different directions; this finite collection of directions plays the role of an integral of motion. More precisely, let p be the composition of the projection of the unit tangent bundle Q × S^{1} on S^{1} and the projection of S^{1} to the quotient space S^{1}/G(Q). Then the function p is constant along every billiard trajectory in a rational polygon.

We illustrate the unfolding procedure in the simplest example of a rational polygon, the square.

1.5 Example: billiard in the unit square

Unfolding a trajectory one obtains a line in the plane which is tiled by the unit squares, the images of the original square Q under the action of the group A(Q). Two lines in the plane correspond to the same billiard trajectory in Q if they differ by a translation through a vector from the lattice 2Z + 2Z.

Consider the fundamental domain of the group 2Z + 2Z which is the square made of four copies of Q. Identify the opposite sides to obtain a flat torus. A billiard trajectory becomes a geodesic line on this torus; every geodesic line has a constant slope λ. The unit tangent bundle of the torus is represented as the union of the tori, ennumerated by the slopes λ; the geodesic flow on each torus is a constant flow.

The dynamics on an individual invariant torus depends on whether λ is rational or irrational: in the former case the geodesic flow is periodic, and in the latter it is ergodic, and in fact, uniquely ergodic. In particular, a billiard trajectory with a rational slope is periodic, while the one with an irrational slope is dense in the square.

One can also analyse periodic trajectories. The unfolding of such a trajectory is a segment in the plane whose end-points differ by a vector from 2Z+2Z. Every parallel trajectory is also periodic with the same period and length.

Assume that an unfolded periodic trajectory goes from the origin to the lattice point (2p, 2q). If p and q are coprime this is a prime periodic trajectory, and if p and q have a comon multiplier then the periodic trajectory is multiple. The length of the trajectory is 2\sqrt{p^{2} + q^{2}}, and to a choice of p and q there correspond two orientations of the trajectory. Thus the number of (strips of) parallel trajectories of length less than L equals the number of pairs of integers, satisfying the inequality p^{2}+q^{2} < L^{2}/2.

This is the number of lattice points inside the circle of radius L/\sqrt{2}, centered at the origin. In the first approximation, this number equals the area πL^{2}/2, and to take only prime periodic trajectories into account one divides this estimate by π^{2}/6. Thus one obtains a quadratic asymptotic estimate on the number of periodic trajectories of the length not exceeding a fixed number. We will see in a later section that this result holds for general rational polygons.

What works so well in the above example is the fact that the images of Q under the group A(Q) tile the plane. A similar consideration applies to rectangles, equilateral triangles and right triangles with an acute angle π/4 or π/6. These polygons are called integrable, and the billiard flow reduces to a constant flow on a torus. Note that in all these cases one may define the extension of a billiard trajectory through a vertex of the billiard polygon.

Although integrable polygons are exceptional, some of the features of the billiard dynamics in the integrable case extend to general rational polygons. These results will be discussed in succeeding sections.

One class of rational billiards for which one can make such precise statements is the class of almost integrable billiards, intermediate between integrable and general rational polygons, studied by Gutkin ([21]). A polygon Q is called almost integrable if the group A(Q) is a discrete subgroup of the group of motions of the plane. There are exactly four such groups generated by the reflections in the sides of the four integrable polygons. An almost integrable polygon is one that can be drawn on the corresponding lattice.

Fig. 3

Given an almost integrable polygon, the billiard flow decomposes into directional flows F_{θ} (just as in the case of a square – see next section for a detailed discussion). Choose a basis e_{1}, e_{2} of the respective lattice. A direction is called rational if it is given by a vector a_{1}e_{1} + a_{2}e_{2} with a_{1}/a_{2} ∈ Q. Gutkin proved that, similarly to the square case, the following conditions are equivalent:

(i) θ is an irrational direction;

(ii) F_{θ} is minimal;

(iii) F_{θ} is ergodic;

(iv) F_{θ} is aperiodic, that is, F_{θ}^{t} ≠ id for all t ≠ 0.

1.6 Rational billiard determines a flat surface

The following construction of a flat surface from a rational billiard table plays a central role in the present study (see [14], [31], [32], [45] and [34]; the later paper was the first to relate billiards in polygons and quadratic differentials).

Let Q be a rational polygon. The group G(Q) is the dihedral group D_{N} generated by the reflections in the lines through the origin that meet at angles π/N where N is a positive integer. This group has 2N elements, and the orbit of a generic point θ ≠ kπ/N on the unit circle consists of 2N points. Let the angles of Q be πm_{i}/n_{i} where m_{i} and n_{i} are coprime integers. Then N is the least common multiple of the denominators n_{i}.

Consider the unit tangent bundle Q × S^{1}, the phase space of the billiard flow, and let M_{θ} be the subset of points whose projection to S^{1} belongs to the orbit of θ under D_{N}. Then, M_{θ} is an invariant surface of the billiard flow in Q. This invariant surface is a level surface of the above mentioned function p, ”the integral of motion”.

Assume that θ ≠ kπ/N and ennumerate the angles in the D_{N}-orbit of θ on the unit circle counterclockwise: θ = θ_{1}, θ_{2}, ..., θ_{2N}. The surface M_{θ} is obtained from 2N copies of Q, namely, Q × θ_{i} ⊂ Q × S^{1}, i = 1, ..., 2N by gluing their sides according to the action of D_{N}.

Consider 2N disjoint and parallel copies of Q in the plane. Call them Q_{1}, ..., Q_{2N} and orient the even ones clockwise and the odd ones counterclockwise. Choose an index i = 1, ..., 2N and a side s of Q_{i}; reflect the direction θ_{i} in this side. The resulting direction is θ_{j} for some j = 1, ..., 2N. Glue the side s of Q_{i} to the identical side of Q_{j}. After these gluings are made for all values of i and all choices of the side s of Q_{i}, the sides of all the polygons Q_{i} are pasted pairwise, and the gluings agrees with the orientation. The result is an oriented compact surface that depends only on the polygon Q, but not on the choice of θ, and we denote it by M. The billiard flows on M in different directions are obtained one from another by rotation.

For example, if Q is a square then N = 2 and the result is a torus made of four identical squares. If Q is a right triangle with an acute angle equal to π/8 then the surface M_{θ} is obtained from a regular octagon, the result of gluing 16 copies of the triangle, by pairwise gluing its opposite sides; this surface has genus 2.

Fig. 4

The genus of M is given in the next lemma.

Lemma 1.2 Let the angles of a billiard k-gon be πm_{i}/n_{i}, i = 1, ..., k where m_{i} and n_{i} are coprime, and N be the least common multiple of n_{i}’s. Then

genus M = 1 + \frac{N}{2} (k − 2 − ∑ \frac{1}{n_{i}} ).

Proof. We need to analyse how the gluings are made around a vertex of Q. Consider the i-th vertex V with the angle πm_{i}/n_{i}. Let G_{i} be the group of linear transformations of the plane generated by the reflections in the sides of Q, adjacent to V. Then G_{i} consists of 2n_{i} elements.

According to the construction of M the number of copies of the polygons Q_{j} that are glued together at V equals the cardinality of the orbit of the test angle θ under the group G_{i}, that is, equals 2n_{i}. Originally we had 2N copies of of the polygon Q, and therefore, 2N copies of the vertex V; after the gluings we have N/n_{i} copies of this vertex on the surface M.

It follows that the total number of vertices in M is N ∑ 1/n_{i}. The total number of edges is Nk, and the number of faces is 2N. Therefore the Euler characteristic of M equals

N ∑ \frac{1}{n_{i}} − Nk + 2N = 2 − 2g

where g is the genus, and the result follows. □

The billiard flow on M is obtained from the constant flows in the directions θ_{i} in the polygons Q_{i}. The result is a (unit) vector field on M with singularities at the vertices. The above proof shows that the i-th vertex of M is the result of gluing 2n_{i} copies of the angle πm_{i}/n_{i} which sums up to an angle of 2πm_{i}. One may realize such a singularity geometrically as follows. Take m_{i} copies of a Euclidean upper half plane H_{j} and m_{i} copies of a lower half plane L_{j}; j = 1, ..., m_{i}. Then glue the positive real axis of H_{j} to the positive real axis of L_{j} and glue the negative real axis of L_{j} to the negative real axis of H_{j+1} (m_{i} + 1 = 1). The result is a singularity with a total angle of 2πm_{i}. We will call it a cone angle 2πm_{i} singularity. From this description one can also see that for any direction θ, there are 2m_{i} separatrices emanating from the singularity in direction θ.

It is easy to describe the set of polygons for which all the angles are 2π, that is, m_{i} = 1; then the singularities of M are removable. The sum of interior angles of a k-gon is π(k − 2). Thus if m_{i} = 1 for all i then

\frac{1}{n_{1}} + ... + \frac{1}{n_{k}} = k − 2.

This equation has only four solutions with n_{i} ≥ 2, considered up to permutations:

(\frac{1}{3}, \frac{1}{3}, \frac{1}{3}), (\frac{1}{2}, \frac{1}{4}, \frac{1}{4}), (\frac{1}{2}, \frac{1}{3}, \frac{1}{6}), (\frac{1}{2}, \frac{1}{2}, \frac{1}{2}, \frac{1}{2}).

These solutions correspond to the already mentioned integrable polygons, and in each case the invariant surface M is a torus with a constant flow.

To summarize the construction, given a rational billiard polygon Q, one constructs a compact surface M whose genus is given by the above lemma. This surface inherits a flat metric from Q with a finite number of cone-type singularities, corresponding to the vertices of Q, with cone angles multiples of 2π. The billiard flow on M is a constant flow in a fixed direction with singularities at the cone points.

1.7 Minimality of the billiard flow in rational polygons

The next result on the minimality of the billiard flow in a rational polygon is much easier than the stronger results on ergodicity described later in the paper. We discuss it because it serves a model for these harder theorems.

Definition 1.3 A flow is called minimal if any of its orbits is dense in the phase space.

Denote the billiard flow on M in the direction θ by F_{θ}.

Definition 1.4 A saddle connection in direction θ is an orbit of F_{θ} that goes from a singularity to a singularity (possibly, the same one) and has no interior singularities.

A saddle connection is also called a generalized diagonal since it corresponds to a billiard orbit in direction θ that goes from a vertex to another vertex.

We may decompose a saddle connection β as a sum of its horizontal part h_{β} and its vertical part v_{β}.

Definition 1.5 A metric cylinder in direction θ is the isometric image of a straight Euclidean cylinder in R^{3} consisting of closed geodesics in direction θ. A metric cylinder is maximal if it cannot be enlarged.

We already noted that a periodic orbit in direction θ for the billiard flow included into a strip of parallel periodic orbits thus giving rise to a metric cylinder in direction θ. The following lemmas and theorem are standard (see [47]), and the proofs do not use the group structure coming from billiards. Thus they will hold verbatim for the general situation of flat structures (that will be formally defined shortly).

Lemma 1.6 If M has singularities, then the boundary of a maximal metric cylinder consists of a finite number of saddle connections in direction θ.

Proof. If a metric cylinder fills the surface, then M is the flat torus which has no singularities. Thus we can assume the maximal cylinder has a boundary. The obstruction to enlarging a maximal cylinder are saddle connections on the boundary. □

Lemma 1.7 Suppose α^{+} is a trajectory, infinite in the positive direction. Let β be an interval perpendicular to α^{+} with initial point P_{0} on α^{+} as one of its endpoints. Then α^{+} returns to β after P_{0}.

Proof. Since there are a finite number of singularities, there are a finite number of trajectories starting at points of β that hit a singularity before crossing β again. By shortening β to a subinterval β′ with one endpoint P_{0} and some other endpoint Q_{0}, we can assume that no trajectory leaving β′ hits a singularity before returning to β. Now flow the interval β′ in the positive direction. The interval sweeps out rectangles of increasing area. Since the area of the surface is finite, the interval β′ must return and overlap β. If α^{+} itself does, we are done. Otherwise the trajectory leaving Q_{0} returns to β′ and some trajectory leaving a point Q_{1} ∈ β′ returns to P_{0}. We now consider the interval β′′ ⊂ β′ with endpoints P_{0} and Q_{1} and apply the previous analysis to it. Flowing in the forward direction it must return to β and now α^{+} itself must return to β. □

Theorem 1.8 For all but countably many directions θ the flow F_{θ} is minimal on the surface M.

Proof. Since there are only finitely many singularities and countably many homotopy classes of arcs joining the singularities, it is clear that there are only countably many saddle connections (the same statement follows from the fact that the group A(Q) is countable.) We claim that the flow F_{θ} is minimal if there is no saddle connection in direction θ.

By Lemma 1.6 there cannot be a metric cylinder for then there would be a saddle connection. Suppose there was an infinite trajectory l_{θ} in direction θ which is not dense. Let A ≠ M be the set of ω limit points of l_{θ}. Then A is invariant under the flow F_{θ}. Since A ≠ M one can choose a trajectory γ on ∂A; let P_{0} be its initial point. We will show that γ is a saddle connection. If not then γ is infinite in at least one of the two directions. We show that there is an open neighborhood of P_{0} contained in A, a contradiction to P_{0} being a boundary point.

Let β be a perpendicular arc with P_{0} an endpoint. It is enough to show that there exists a segment [P_{0}, Q] ⊂ β which is contained in A. For then doing this on both sides we would have our open neighborhood. Now Lemma 1.7 implies that γ hits β again at some P_{1}. If the interval [P_{0}, P_{1}] ⊂ A we are done. Suppose not. Then there exists Q_{1} ∈ [P_{0}, P_{1}] which is not in A. Since A is closed, there is a largest open subinterval I_{1} ⊂ [P_{0}, P_{1}] containing Q_{1} which is in the complement of A. Let P_{2} BE the endpoint of I_{1} closest to P_{0}. Then P_{2} ∈ A and the trajectory through P_{2} must be a saddle connection. For if it were infinite in either direction, it would intersect I_{1}. Since A is invariant under the flow, this contradicts that I_{1} misses A. □

Projecting back to the rational billiard polygon Q we see that for all but countably many directions every billiard trajectory is dense in Q. Approximating a general billiard by a rational one we obtain the next result on topological transitivity of polygonal billiards ([31]).

Definition 1.9 A flow is called topologically transitive if it has a dense orbit.

Theorem 1.10 For every k ≥ 3 there exists a dense G_{δ} subset in the space of simply connected k-gons that consists of polygons with topologically transitive billiard flows.

Proof. Identify the phase space of the billiard flow in each k-gon with D^{2} × S^{1}, and assume that this identification depends continuously on the polygon. Let B_{i} be a countable basis for the topology of D^{2} × S^{1}. Denote by X_{n} the set of k-gons Q such that for each open set U in the phase space, there exists a billiard trajectory starting in U that visits all the images of the sets B_{1}, ..., B_{n} in the phase space of the billiard flow in Q. Each set X_{n} is open and their intersection is a G_{δ} set.

Let us show that this intersection is dense. Let Y_{m} be the set of rational k-gons with the angles πm_{i}/n_{i}, m_{i} and n_{i} coprime, such that the least common multiple of n_{i}’s is at least m. For every polygon Q ∈ Y_{m} the invariant surface M_{θ} is 1/m-dense in the phase space. Therefore for every n there exists m such that for every Q ∈ Y_{m} the surface M_{θ} intesects all the images of the sets B_{1}, ..., B_{n} in the phase space of the billiard flow in Q. Since M_{θ} has a dense trajectory for all but countably many θ the space Y_{m} lies in X_{n}. Finally, Y_{m} is dense in the space of k-gons for every m, and so X_{n} is dense for every n as well. It follows that ∩X_{n} is dense by Baire’s theorem.

Let Q be a billiard polygon in ∩X_{n}. We claim that there is a dense billiard trajectory in the phase space of Q. Let U be a compact domain in the phase space D^{2} × S^{1}. Suppose inductively we have chosen a compact neighborhood U_{n−1} ⊂ U. Since Q ∈ ∩X_{n}, one can find a billiard trajectory starting in U_{n−1} in Q that visits B_{1}, ..., B_{n}. By continuity there is a neighborhood U_{n} ⊂ U_{n−1} such that each trajectory in U_{n} visits B_{1}, …, B_{n} within time T_{n}. Choose a phase point v ∈ ∩U_{n}. Then the trajectory of the vector v is dense in the phase space of Q. □

1.8 Rational billiards and interval exchange maps

In the case of rational billiards the billiard flow on the invariant surface can be reduced to a one-dimensional transformation; this reduction is a particular case of the reduction of the billiard flow to the billiard transformation.

Definition 1.11 Let (I_{1}, ..., I_{n}) be a partition of the interval [0, 1) into nonintersecting semiclosed intervals, enumerated from left to right, and let σ be a permutation of n element. The respective interval exchange transformation T : [0, 1) → [0, 1) is a transformation whose restriction to every I_{i} is a parallel translation and such that the intervals T (I_{1}), ..., T (I_{n}) follow from left to right in the order σ(1), ..., σ(n).

Clearly, an interval exchange transformation preserves the Lebesgue measure on the unit interval. For example, the exchange of two intervals [0, a) and [a, 1) is identified with the rotation of the circle R/Z through 1 − a.

The reduction of the billiard flow in a fixed direction θ to an interval exchange goes as follows. Recall that the invarinat surface M was constructed by pasting pairs of equal and parallel sides of 2N copies of the billiard polygon Q. Choose one side from each such pair and call these segments s_{1}, ..., s_{m}. Arrange these segments along a line and let I be their union. Give each segment the measure determined by the orthogonal projection on the direction, perpendicular to θ. Since ”the width of a beam” is an invariant transversal measure of a constant flow, the billiard transformation induces a piecewise isometry T of the segment I. If the segments are appropriately oriented then T is orientation preserving. It remains to rescale I to the unit interval and to define T at its points of discontinuity so that it is continuous on the left. The result is an interval exchange transformation.

The reduction of the billiard flow in a fixed direction to an interval exchange transformation is by no means unique. For example, assume that the flow F_{θ} is topologically transitive. Choose an interval I in M, transverse to the flow, and give it the measure determined by the orthogonal projection on the direction, perpendicular to θ. Then the first return map to I along the trajectoires of F_{θ} is orientation and mesure preserving, that is, an interval exchange transformation. Note that since F_{θ} is topologically transitive, the first return map is defined no matter how small I is. The points of discontinuity of the first return map correspond to the orbits of the flow through the singular points of the flow F_{θ}.

1.9 Flat metrics and quadratic differentials

As we have seen a rational billiard defines a closed surface formed by gluing Euclidean polygons isometrically along their edges. The vertices of the polygon correspond to points with cone angle singularities of the metric. In this section we generalize this notion to what we will call flat structures with parallel line fields, or just flat structures for short, so that the set of rational billiards of a given genus is a subset of the space of flat structures. The main reason for this generalization is that the group SL(2, R) acts on the space of flat structures while not preserving the space of rational billiards. The study of this action allows to make conclusions about flat structures in general, which then give us results about rational billiards in particular.

Let M be a compact C^{∞} surface and Σ a finite set of points in M. On M \ Σ we require coordinate charts v = (x, y) such that the transition functions on the overlaps are of the form

v → v + c or v → −v + c.

That is, the transition functions are translations and reflections in the origin followed by translations. Since these preserve the Euclidean metric this allows one to define a locally Euclidean metric on M. In addition these transition functions preserve families of parallel lines in the plane so that for each direction θ, there is a well-defined foliation F_{θ} of the surface consisting of lines in direction θ.

Note that if the transition functions are all translations, then the line field defines a vector field and in each direction θ we have a flow F_{θ}. Now the same topological theorem Theorem 1.8 that holds for rational billiards hold in the more general case of a flat structure.

In a neighborhood of a point in Σ there are polar coordinates (r, θ) such that the metric can be written as

ds^{2} = dr^{2} + (crdθ)^{2}

where c is a half integer. We say that the metric has cone type singularity with cone angle 2πc. The curvature κ at a singular point is defined by the formula

κ = 2π − 2πc.

We may concretely describe the metric in a neighborhood of a singular point by gluing together half planes as described above. Notice that our previous discussion shows that a rational polygon determines such a flat structure with transition functions all of the first type (parallel translations) and c an integer.

Another way to define the same structure would be to begin by requiring that M have Gaussian curvature zero, or equivalently, it is locally isometric to R^{2} with the Euclidean metric, away from a finite set of points Σ and cone type singularities near points of Σ, where c is an arbitrary real. This in itself is not enough, for it does not rule out the possibility of rotations in the transition functions, which would not allow for parallel line fields. One way to guarantee the correct structure in this language is to consider parallel translation with respect to the connection determined by the metric. The global obstruction to parallel translation being well defined is the holonomy group which is a subgroup of O(2). If the holonomy group is either trivial or {I, −I} then parallel translation is well-defined and M possesses a parallel line field. The existence of such a line field implies that the cone angles are multiples of π. Pick one such line field and call it the vertical line field. In the neighborhood of each point of M \ Σ we can construct a chart which is an orientation preserving isometry and takes the vertical line field to the vertical line field in R^{2}. The change of coordinate functions between these charts then have the desired form.

The same structure can also be defined in terms of complex analysis. Recall that a Riemann surface or a complex analytic structure on a surface M consists of an atlas of charts (U_{α}, z_{α}) where {U_{α}} is a covering of M by open sets, z_{α} : U_{α} → C is a homeomorphism and if U_{α} ∩ U_{β} ≠ ∅, then z_{α} ∘ z_{β}^{−1} : z_{β}(U_{α} ∩ U_{β}) → C is complex analytic. The maps z_{α} are called uniformizing parameters.

A meromorphic quadratic differential ϕ assigns to each uniformizing parameter z_{α} a meromorphic function ϕ_{z_{α}}(z_{α}) with the property that

ϕ_{z_{β}}(z_{β})(\frac{dz_{β}}{dz_{α}})^{2} = ϕ_{z_{α}}(z_{α})

in U_{α} ∩ U_{β}. Then the quadratic differential ϕ(z)dz^{2} is invariantly defined on M.

Although the value of ϕ is not well-defined, the set of zeroes and poles of ϕ and their orders are. It is a classic result in Riemann surface theory that ϕ has a finite number of zeroes with orders k_{i} and poles of order l_{i} satisfying

∑ k_{i} − ∑ l_{i} = 4g − 4.

In this paper we will assume that if there are poles then they are simple. This implies that the norm or area of ϕ defined by

||ϕ|| = ∫_{M} |ϕ(z)||dz|^{2}

is finite. Notice that the area element |ϕ(z)||dz^{2}| is well-defined independently of choice of coordinates. We may equally well consider ϕ to be a holomorphic quadratic differential on the surface punctured at the poles.

A quadratic differential determines a metric with the length element |ϕ(z)|^{1/2}|dz|. If the quadratic differential has a simple pole then the metric is not complete on the punctured surface.

The vertical trajectories of ϕ are the arcs along which

ϕ(z)dz^{2} < 0

and the horizontal trajectories the arcs along which

ϕ(z)dz^{2} > 0.

We now indicate why a flat structure (with a parallel line field) and quadratic differential define the same object. First suppose ϕ is a quadratic differential on M and p ∈ M is a point which is neither a pole nor zero. We may choose a uniformizing parameter ζ in a neighborhood of p with p corresponding to ζ = 0. We may then choose a branch of ϕ^{1/2}(ζ) near ζ = 0 and define a new uniformizing parameter w by

w = w(ζ) = ∫_{0}^{ζ} ϕ^{1/2}(τ)dτ.

Then, in the w coordinates, the quadratic differential is given by ϕ_{w}(w) ≡ 1. Such coordinates are called natural. If w and w′ define natural coordinates in overlapping neighborhoods, then

w′ = ±w + c

and so one has a flat structure. The Riemannian metric is the flat metric |dw|.

At a zero of ϕ of order k (at a simple pole take k = −1) the quadratic differential can be written as

(\frac{k + 2}{2})^{2} ζ^{k}dζ^{2}

for a choice of coordinate ζ. Then

w(ζ) = ζ^{\frac{(k+2)}{2}} ,

where the uniformizing parameter w is defined as above. The full angle 0 ≤ arg ζ ≤ 2π is subdivided into k + 2 equal sectors

\frac{2π}{k + 2}j ≤ arg ζ ≤ \frac{2π}{k + 2}(j + 1), j = 0, ..., k + 1.

Every sector is mapped by w(ζ) onto the upper or lower halfplane. Thus a zero of ϕ of order k corresponds to a singular point of a flat structure with the cone angle π(k + 2). The pre-images of the horizontal lines are the horizontal trajectories of the quadratic differential ϕ, and the trajectory structure has the form of a k + 2 pronged singularity.

Fig. 5

Conversely, a flat structure with a parallel line field defines a quadratic differential. Let v = (x, y) and v′ = (x′, y′) be coordinates in overlapping charts so that

v′ = ±v + c.

Setting z = x + iy and z′ = x′ + iy′, the transition functions are complex analytic, so define a Riemann surface structure on M \ Σ, where Σ is the set of singularities. We may then define a quadratic differential ϕ on this Riemann surface by assigning the constant function 1 to the parameter z. The πc cone angle singularity corresponds to a zero of order c − 2.

There is an additional description of these structures via the theory of measured foliations, developed by Thurston. This description will not play any role here and we will not dwell on it.

We will go back and forth between the terminology of quadratic differentials and flat structures with parallel line fields.

We finish the section with an example taken from [59]: the Riemann surface corresponding to the billiard in a regular n-gon with n odd is conformally equivalent to the Fermat curve x^{n} + y^{n} = 1, and the respective quadratic differential is dx^{2}/y^{4}; see [3] for a similar description of Riemannian surfaces corresponding to rational triangles.

2 Teichmuller space, strata of quadratic differentials and SL(2, R) action

2.1 Teichmuller space and mapping class group

A general reference for Teichmuller spaces, and compactifications is the paper of Bers [6]. Let M be a surface of genus g with n punctures. We assume 3g − 3 + n ≥ 0. Teichmuller space T_{g,n} is the space of equivalence classes of complex structures X on M where X_{1} ∼ X_{2} if there is a biholomorphic map from X_{1} to X_{2} which is isotopic to the identity on M. One may define the Teichmuller distance function d_{T} (·, ·) on T_{g} by

d_{T} (X_{1}, X_{2}) = 1/2 inf log K(f),

where the infimum is taken over all quasiconformal maps f isotopic to the identity on M and K(f) is the maximal dilatation of f as measured by the complex structures X_{1}, X_{2}. The mapping class group Mod(g, n) is the group Diff^{+}(M)/Diff_{0}(M), the group of orientation preserving diffeomorphisms modulo those isotopic to the identity. It acts on T_{g,n} by pull-back; given a complex structure X defined by coordinate charts (U_{α}, z_{α}) and f ∈ Mod(g, n), we find a new complex structure f · X defined by the atlas of coordinate charts (f(U_{α}), z_{α} ∘ f^{−1}) The quotient space R_{g,n} = T_{g,n}/Mod(g, n) is the moduli space of Riemann surfaces.

2.2 Compactifications

The moduli space R_{g,n} is well-known not to be compact. It is possible to deform a Riemann surface by pinching along one or more disjoint simple closed curves, by letting the hyperbolic length of the curves go to 0. The resulting surface then has nodes or punctures and may not be connected. For example if a closed surface of genus g is pinched along a single closed curve γ and γ does not disconnect M, the resulting surface has genus g − 1 with 2 punctures, while if γ is dividing, the resulting surface has two components each of which has one puncture and the sum of their genera is g. We can compactify R_{g,n} by adjoining the moduli spaces of surfaces obtained in this fashion.

Denote the compactification by R̄_{g,n}. The topology has the following property. Let X_{0} be any surface in the compactification and X_{n} → X_{0}. Remove any neighborhood U of the punctures of X_{0}. Then for large enough n, there is a conformal embedding of X_{0} \ U → X_{n}.

The compactification by these moduli spaces is well-behaved with respect to quadratic differentials. For suppose ϕ_{n} are unit norm quadratic differentials on X_{n} which converge to X_{0} ∈ R̄_{g,n}. Then there is a subsequence of ϕ_{n} which converges uniformly on compact sets of X_{0} via the conformal embedding to an integrable quadratic differential ϕ_{0}. However it may be the case that ϕ_{0} ≡ 0 on one or more components of X_{0}. This issue will be discussed further in the section on periodic orbits.

2.3 Strata of quadratic differentials

Fix a surface M of genus g. Let σ = (k_{1}, …, k_{j}, ϵ) be an j + 1 tuple where k_{i} are integers which are either positive or −1 satisfying ∑ k_{i} = 4g − 4 and ϵ = ±. Consider the set of quadratic differentials with zeroes of order k_{i}, and simple poles if k_{i} = −1, the sign + if the quadratic differential is the square of a holomorphic 1 form, and − otherwise. These spaces can be empty (see [43] for necessary and sufficient conditions) and need not be connected – [53] and [35]. We say that two such quadratic differentials are equivalent if there is a homeomorphism of M isotopic to the identity taking singular points to singular points of the same order and which at other points has the same local form as the change-of-coordinate functions. The resulting space is denoted by Q(σ) and one can prove that this is a manifold. In the case that all k_{i} = 1 (and ϵ = −1) Q(σ) is called the principle stratum. For fixed g, the union of the Q(σ) where all k_{i} > 0, is the space of all quadratic differentials Q_{g} over T_{g}. It is a well-known part of Teichmuller theory that T_{g} is a complex manifold and Q_{g} is the cotangent bundle, although we will not make use of this structure in this paper. The strata Q(σ) are not closed subsets of Q_{g}, unless there is a single zero (n = 1). This is because a sequence in Q(σ) may collapse a pair of lower order zeroes into a higher order one.

The group Mod(g) acts on each Q(σ). The quotient is denoted by QD(σ) and will play a crucial role in this paper. Since Mod(g) does not act freely, the space QD(σ) has the structure of an orbifold.

2.4 SL(2, R) action on quadratic differentials

The group SL(2, R) acts on quadratic differentials by linear transformations of local coordinates, preserving each stratum. If {u_{α}} is an atlas of natural coordinates defining a quadratic differential ϕ, and A ∈ SL(2, R) then {Au_{α}} defines a new family of natural coordinates for a quadratic differential Aϕ. We require that A takes singularites of ϕ to singularities of Aϕ of the same order.

The following one parameter subgroups of SL(2, R) are of special interest:

g_{t} = ( \begin{matrix}e^{t/2} & 0 \\ 0 & e^{−t/2} \end{matrix} ) , r_{θ} = ( \begin{matrix}cos θ & sin θ \\ − sin θ & cos θ \end{matrix} ) , h_{s} = ( \begin{matrix}1 & s \\ 0 & 1 \end{matrix} ) ,

referred to as the geodesic or Teichmuller, circular and horocyclic flows, respectively.

The effect of the flow ϕ → g_{t}ϕ is to stretch along the horizontal trajectories of ϕ by a factor of e^{t} and contract along the vertical trajectories by e^{t}. The flow contracts the transverse measure of the horizontal foliation and expands the transverse measure of the vertical foliation. Associated to g_{t} is the map f_{t} of the underlying Riemann surfaces X → X_{t}, called the Teichmuller map. It is the famous theorem of Teichmuller that given any homeomorphism f : X → Y of Riemann surfaces of finite type, there is a unique Teichmuller map in the homotopy class which minimizes the maximal dilation.

The action of r_{θ} on a quadratic differential ϕ is the same as multiplying ϕ by e^{2iθ}. This multiplication defines an action of the circle R/πR on the set of quadratic differentials. Note that the action of r_{θ} leaves the flat metric invariant, but changes the vertical line field.

The action on saddle connections is as follows. Let

A = [ \begin{matrix}a & b \\ c & d \end{matrix} ]

be a matrix in P SL(2, R). Let β be a saddle connection, and let h_{1} and v_{1} be its horizontal and vertical components with respect to ϕ. Then the horizontal and vertical components h_{2} and v_{2} with respect to Aϕ are given by matrix multiplication; namely

[ \begin{matrix}a & b \\ c & d \end{matrix} ] · [ \begin{matrix}h_{1} \\ v_{1} \end{matrix} ] = [ \begin{matrix}h_{2} \\ v_{2} \end{matrix} ] .

The orbit of a quadratic differential under P SL(2, R) is called a Teichmuller disc and has been considered by numerous authors – see [36], for example. In a later section we will examine certain examples of Teichmuller discs that arise in the socalled Veech billiards and their generalizations.

The action of SL(2, R) commutes with the action of Mod(g) and thus descends to an action on QD(σ). It is this action we will study in some detail.

It is possible to define an SL(2, R) invariant measure μ_{0} on each QD(σ) which is absolutely continuous with respect to the orbifold structure. We show how to do this for the principle stratum QD(σ). The measure is defined locally. A saddle connection of ϕ_{0} persists under small perturbation of ϕ_{0}. There are a finite number of such saddle connections β_{1}, …, β_{n} whose horizontal and vertical components (h_{β_{1}}, v_{β_{1}}, …, h_{β_{n}}, v_{β_{n}}) serve as local coordinates for Q(σ) in a neighborhood of ϕ_{0}. Specifically, pass to a double cover π : M̃ → M, ramified over the zeroes of ϕ_{0}. There is an involution τ of M̃ which interchanges the sheets. We choose the β_{i} such that their lifts to M̃ form a basis for the first homology of M̃, odd with respect to the involution τ. The measure μ_{0} is then Lebesgue measure on R^{2n} pulled-back to Q(σ) via these local coordinates. One sees that μ_{0} does not depend on choice of basis β_{i}, so actually defines a measure. It is easily seen to descend to QD(σ) and to be SL(2, R) invariant. The measure was shown to be finite and ergodic on the principle stratum in [37] and then on each component of general stratum in [52], [53]. An interesting question is to classify all ergodic SL(2, R) invariant measures on QD(σ).

3 Ergodicity

3.1 Veech nonuniquely ergodic example

Definition 3.1 A foliation F is uniquely ergodic if it is minimal and the transverse measure is unique up to scalar multiplication.

Unique ergodicity is equivalent to the following condition. Let G be any foliation with transverse measure, transverse to F. For any point x_{0}, the transverse foliation G defines an arc length l(t) on the leaf of F through x_{0} in either direction. The pair (F, G) defines a measure μ on M. For every continuous function f(x) on M and every point x_{0} we have

lim_{T →∞} \frac{1}{T} ∫_{0}^{T} f(l(t))dt = ∫_{M} f(x)dμ(x).

The classical example of a uniquely ergodic foliation is the irrational flow on the flat torus. In [51] Veech constructed examples of minimal and not uniquely ergodic dynamical systems that are skew products over irrational rotations of the circle. Take two copies of the unit circle and mark off a segment J of length 2πα in the counterclockwise direction on each with one endpoint at 0. Now take θ irrational and consider the following dynamical system. Start with a point p, say in the first circle. Rotate counterclockwise by 2πθ until the first time the orbit lands in J; then switch to the corresponding point in the second circle, rotate by 2πθ until the first time the point lands in J; switch back to the first circle and so forth. Veech showed that for any irrational α there are an uncountable set of θ such that this system is minimal and not uniquely ergodic.

We give geometric description of Veech’s examples that was worked out by John Smillie jointly with the first author (unpublished). In order to do that we first give a geometric criterion of how to find nonergodic foliations. This criterion was given in [42]. To set the notation let E_{0}, E_{1}, … be a sequence of subsets in M so that E_{n} ∩ E_{n+1} = ∅ and each E_{n} is bounded by a set of saddle connections all in the same direction θ_{n}. Assume the directions θ_{n} converge to θ_{∞}. Let F_{θ_{∞}} be the foliation in direction θ_{∞}. Rotate the coordinate system so that θ_{∞} is the vertical direction. Define a system of partitions P_{n} = [A_{n}, B_{n}] of M as follows. If n is even, then A_{n} = E_{n} and B_{n} = E_{n}^{c} (complement of E_{n}), and if n is odd, then A_{n} = E_{n}^{c} and B_{n} = E_{n}. Let h_{n} be the sum of the horizontal components of the saddle connections separating A_{n} and B_{n}. Let μ be the measure defined by the flat structure.

Theorem 3.2 Suppose that

• (i) lim_{n→∞} h_{n} = 0

• (ii) for some c, c′, 0 < c ≤ μ(A_{n}) ≤ c′ < 1

• (iii) ∑_{n=1}^{∞} μ(A_{n}ΔA_{n+1}) < ∞

Then the vertical foliation F_{θ_{∞}} is nonergodic.

Proof. Let

A_{∞} = lim inf A_{n} = {x : ∃N such that for n ≥ N, x ∈ A_{n}}.
B_{∞} = lim inf B_{n} = {x : ∃N such that for n ≥ N, x ∈ B_{n}}.

We first show that A_{∞} and B_{∞} satisfy

• (1) μ(M \ (A_{∞} ∪ B_{∞})) = 0.

• (2) A_{∞} ∩ B_{∞} = ∅

• (3) μ(A_{∞}ΔA_{n}) → 0 as n → ∞.

• (4) 0 < μ(A_{∞}) < 1.

By (iii) and the Borel-Cantelli Lemma the set of x′ which are in infinitely many A_{n}ΔA_{n+1} has μ measure 0. ¿From this we have (1). Statement (2) is immediate. To see (3) note that

A_{∞}ΔA_{n} ⊂ ∪_{i=n}^{∞}A_{i}ΔA_{i+1}

so that

μ(A_{∞}ΔA_{n}) ≤ ∑_{i=n}^{∞} μ(A_{i}ΔA_{i+1}).

Hypothesis (iii) implies that the right hand side goes to 0 as n goes to infinity, proving (3). Statement (4) follows from (3) and (ii).

Returning to the proof of Theorem 3.2 , we can assume that the vertical foliation F_{θ_{∞}} is orientable: if not, we may replace M by an orientable double cover and replace A_{∞} and B_{∞} by their lifts. Now let f_{t} be the flow along vertical leaves.

We claim that for any t the set A_{∞} is μ a.e. invariant; that is,

μ(f_{t}(A_{∞})ΔA_{∞}) = 0.

Suppose on the contrary, that for some t_{0}, we have μ(f_{t_{0}}(A_{∞})ΔA_{∞}) = δ > 0. By (3) and (i) we may choose n large enough so that

μ(A_{∞}ΔA_{n}) < δ/8 and e^{t_{0}}h_{n} < δ/8.

Since f_{t_{0}} is μ preserving, the first inequality above and the assumption give

μ(f_{t_{0}}(A_{n})ΔA_{n}) ≥ δ − 2δ/8 = 3δ/4.

Thus at time t_{0}, 3δ/8 of the measure of A_{n} flows to its complement. However at most e^{t_{0}}h_{n} < δ/8 of measure can cross the boundary of A_{n}, a contradiction, proving the claim. We would like to conclude that A_{∞} is flow invariant, although by the claim it is only a.e. invariant for each time. The theorem is a consequence of the following general lemma.

Lemma 3.3 Let f_{t} be a flow on a space X preserving a probability measure μ. Suppose there is a set A such that for every t, μ(f_{t}(A)ΔA) = 0. Then there is a set A′ invariant under f_{t} with μ(AΔA′) = 0.

Proof. Let λ be Lebesgue measure on R. Let

A′ = {x : f_{t}(x) ∈ A for λ a.e.t}.

It is clear that A′ is f_{t} invariant. We will show that μ(AΔA′) = 0. Let

C_{0} = {(x, t) : x ∈ A} and C_{1} = {(x, t) : f_{t}(x) ∈ A}.

For every t we have

μ({x : (x, t) ∈ C_{0}ΔC_{1}}) = μ(f_{t}(A)ΔA) = 0.

This implies that (μ × λ)(C_{0}ΔC_{1}) = 0. By Fubini there is a set X′ of full μ measure so that for all x ∈ X′,

λ({t : (x, t) ∈ C_{0}ΔC_{1}}) = 0.

If x ∈ X′ ∩ A then (x, t) ∈ C_{0} so that λ({t : (x, t) ∉ C_{1}}) = 0. Therefore the set {t : f_{t}(x) ∈ A} has full λ measure so x ∈ A′. If x ∈ X′\A then the set {t : (x, t) ∉ C_{1}} has full λ measure so that x ∉ A′. Thus AΔA′ is contained in the complement of X′ so μ(AΔA′) = 0. This finishes the proof of the Lemma and therefore the Theorem. □

Now we wish to return to the Veech examples which will be described in terms of quadratic differentials. We start with a square torus T; that is, the unit square with lower left vertex at (0, 0) with opposite sides identified. Equivalently, it is R^{2} modulo the integer lattice. Fix a point (x_{0}, y_{0}) ∈ R^{2}. For integers m, n let w be a segment in R^{2} joining (0, 0) to (x_{0} + m, y_{0} + n), projected to T. Take two copies of T, each slit along w, and identify the positive side of w on one copy to the negative side in another. It is easy to see that this results in a quadratic differential ϕ on a surface M of genus 2. Since the total angle around each of the points (0, 0) and (x_{0} + m, y_{0} + n) is 4π, they correspond to zeroes of order 2 of ϕ. The surface of genus 2 is partitioned into 2 sheets M_{w}^{+} and M_{w}^{−} separated from each other by the union of the two slits.

Theorem 3.4 ([51]) Suppose x_{0}, y_{0} are not rationally related; that is, for any pair of rationals, r_{1}, r_{2}, the equation r_{1}x_{0} + r_{2}y_{0} = 0 implies r_{1} = r_{2} = 0. Then there exists uncountably many directions θ such that the flow on M in direction θ is minimal and not ergodic.

In the special case that x_{0} = 0 and y_{0} = α, so that the slit is along the vertical axis, the resulting surface M has two metric cylinders in the vertical direction, one for each sheet. Choose the core curve of each metric cylinder. The first return map to this pair of curves for the flow in direction θ is precisely the dynamical system studied by Veech.

This same dynamical system can be described in terms of rational billiards by taking a rectangle of length 2 and width 1, with an interior slit of length α from the midpoint of a horizontal side.

Fig. 6

Proof. Suppose w′ is another segment on T with the same endpoints as w. Suppose w and w′ intersect an odd number of times in their interior so they divide each other into an even number of pieces. This is equivalent to saying that w and w′ are homologous mod 2 on T. The sheet interchange measured by c = (M_{w}^{+} ∩ M_{w′}^{−}) ∪ (M_{w}^{−} ∩ M_{w′}^{+}) is a union of an even number of parallelograms with sides on w and w′ (here w and w′ are thought of as vectors). Thus the area of c is at most |w × w′|.

Fig. 7

Now fix a sequence of positive numbers ρ_{j} with ∑ ρ_{j} < ∞. We will build an infinite directed tree with each vertex leading to 2 further vertices. At level j there will be 2^{j−1} vertices. Each vertex will correspond to a pair of integers (p, q) which will determind a slit with endpoints (0, 0) and (x_{0} + p, y_{0} + q).

For any pair (p, q) form the quotient (p+x_{0})/(q+y_{0}), the slope of the corresponding slit. Let δ_{j} be the minimum distance between any two slopes as (p, q) varies over the vertices at level j. Now inductively, suppose we have determined the tree up to level j. For each vertex (p, q) at level j we will choose two vertices (p′, q′_{)} and (p′′, q,′′) at level j + 1, and call these the two children of (p, q). To find such children begin by choosing an even integer d such that

\frac{ρ_{j}}{(q + y_{0})(q + y_{0} + d)} < δ_{j}/4.

Then consider the inequality

d|(p + x_{0})n − (q + y_{0})m| < ρ_{j}

Since x_{0} and y_{0} are not rationally related, neither are p_{0} + n and q + y_{0}, and so there are infinitely many coprime solutions (m, n). Choose any two of them and set

p′ = p + dm and q′ = q + dn,

calling the two resulting two pairs the children. A direct calculation shows that

δ′_{j} = |\frac{p + x_{0}}{q + y_{0}} − \frac{p′ + x_{0}}{q′ + y_{0}} | < δ_{j}/4.

That is to say, the distance between the slope of a slit and either child is bounded by δ_{j}/4. Then the distance between the slopes of the two children of (p, q) is at most δ_{j}/2, which implies that

δ_{j+1} < δ_{j}/2.

Let w be the segment corresponding to the parent and w′ the segment of a child. Since d is even, w′ is homologous to w mod 2. In fact w′ − w represents d times the primitive class of (m, n). As we have seen the area of the sheet interchange c is bounded by |w′ × w|, and an easy calculation shows that this is bounded by

|(p + x_{0})n − (q + y_{0})m]| < ρ_{j}.

Thus, in the situation of Theorem 3.2, we have constructed a sequence of partitions satisfying (iii). Now for any geodesic in the tree, the sequence of ratios (p + x_{0})/(q + y_{0}) is a Cauchy sequence hence converging to some θ_{∞}. From this, condition (i) of Theorem 3.2 follows. Condition (ii) holds since the partition divides the surface into two pieces of equal areas. Thus by Theorem 3.2 the limiting foliation F_{θ_{∞}} is not ergodic.

We now show that, in fact, there are uncountably many limiting directions. Since there are only countably many directions that are not minimal, we then can conclude that there are uncountably many limiting directions which are minimal. Since each pair (p, q) has two children there are clearly uncountably many sequences. We therefore need to show that any two limits are distinct. Suppose r_{j} = (p_{j} + x_{0})/(q_{j} + y_{0}) → θ and a distinct sequence r′_{j} = (p′_{j} + x_{0})/(q′_{j} + y_{0}) → θ′. Suppose the sequences differ for the first time at stage j, so that the slopes r_{j} and r′_{j} satisfy

|r_{j} − r′_{j}| ≥ δ_{j}.  (4.5)

Then by (4.3) and (4.4) we have

|θ − \frac{p_{j} + x_{0}}{q_{j} + y_{0}} | ≤ ∑_{k≥j} δ′_{k} < ∑_{k≥j} \frac{δ_{k}}{4} < \frac{δ_{j}}{2} ,

and similarly |θ′ − r′_{j}| < δ_{j}/2. Combined with (4.5), we have θ ≠ θ′. □

The above theorem gives uncountably many minimal nonergodic directions. By requiring α to satisfy a Diophantine condition, it is even possible to show that there is a set of such examples of positive Hausdorff dimension. The main result of the next section however says that the set of nonergodic directions must have Lebesgue measure 0.

3.2 Ergodicity in almost every direction

Theorem 3.5 ([34]) For any quadratic differential ϕ (rational billiard) the set of θ ∈ [0, 2π] such that r_{θ}ϕ has minimal but non-uniquely ergodic vertical foliation has Lebesgue measure 0.

The proof of this theorem uses the SL(2, R) action on quadratic differentials and some combinatorial constructions related to quadratic differentials. Let QD_{ϵ} be the set of quadratic differentials all of whose saddle connections have length at least ϵ. Recall R_{g} is the moduli space of Riemann surfaces of genus g.

Proposition 3.6 QD_{ϵ} is compact in QD.

Proof. QD_{ϵ} is clearly closed in QD since if ϕ_{n} → ϕ and ϕ has a geodesic segment of length less than ϵ so does ϕ_{n} for n large. It is therefore enough to show that QD_{ϵ} lies over a compact set in R_{g}. If not, there is a sequence ϕ_{n} ∈ QD_{ϵ} of quadratic differentials lying on Riemann surfaces X_{n} going to infinity in R_{g}. Passing to subsequences we may assume that X_{n} converges to a Riemann surface X_{∞} with nodes or punctures acquired by pinching along a set of disjoint simple closed curves α_{1}, …, α_{p}. By passing to a further subsequence we can assume that ϕ_{n} converges uniformly on compact sets to an integrable quadratic differential ϕ_{∞} on X_{∞}.

Thus ϕ_{∞} has at most simple poles at the nodes. There is therefore a curve α_{j} homotopic to a puncture with ϕ_{∞} length at most ϵ/2. For large n the uniform convergence implies that α_{j} has ϕ_{n} length at most ϵ/2, a contradiction. □

Definition 3.7 A quadratic differential ϕ is called divergent if the Riemann surfaces on which g_{t}ϕ lie, eventually leave every compact set of R_{g} as t → ∞.

Theorem 3.8 Suppose the vertical foliation F_{ϕ} of ϕ is minimal but not uniquely ergodic. Then ϕ is divergent.

Proof. Suppose the theorem is not true so that for some sequence t_{n} → ∞ the Riemann surfaces X_{n} = X_{t_{n}} converge in R_{g} to some X_{0}. By passing to a further subsequence we may assume that ϕ_{n} = g_{t_{n}}(ϕ) converges to some ϕ_{0} on X_{0}. Let f_{t_{n}} be the corresponding Teichmuller map. Denote by Σ and Σ_{0} the sets of zeroes of ϕ and ϕ_{0}, rspectively.

The normalized transverse measures on the topological foliation F_{ϕ} form a finite dimensional convex set for which the extreme points ν_{i} are mutually singular ergodic measures – see [58]. Let ν_{0} be the transverse measure of F_{ϕ}. We have ν_{0} = ∑_{i=1}^{m} c_{i}ν_{i}, for constants c_{i}. Pick a segment I of a horizontal trajectory of ϕ so that ν_{i}(I) ≠ ν_{j}(I) for i ≠ j. Let μ the transverse measure to the horizontal foliation defined by ϕ. This means that the area element |ϕ(z)dz^{2}| = ν_{0} × μ. Let E_{i} ⊂ X be the disjoint sets of generic points of the measure ν_{i} × μ for the interval I. We have (ν_{i} × μ)(E_{i}) = 1. Each E_{i} is a union of leaves of F_{ϕ}.

For each i let A_{i} ⊂ X_{0} denote the set of accumulation points of f_{t_{n}}(x) ∈ X_{n} for x ∈ E_{i}. Now let U ⊂ X_{0} be any open set. We claim that there is a j such that

U ∩ A_{j} ≠ ∅.

To prove the claim, choose an open W so that W̄ ⊂ U. Since X_{n} → X_{0}, for each n, W may be considered as a subset of X_{n} with area bounded below by δ > 0. Consider W_{n} = f_{t_{n}}^{−1}(W) ⊂ X. Since f_{t_{n}} preserves area, (ν_{0} × μ)(W_{n}) > δ. Then for each n, there is a j = j(n) such that

(ν_{j} × μ)(E_{j} ∩ W_{n}) = (ν_{j} × μ)(W_{n}) ≥ \frac{δ}{mc_{j}} .

Choose a j so that this inequality holds for infinitely many n. This implies that there exists some x ∈ E_{j} ∩ W_{n} for infinitely many n. Since f_{t_{n}}(x) ∈ W, f_{t_{n}}(x) has an accumulation point in W̄, proving the claim.

Partition X_{0} into a finite number of rectangles in the natural coordinates of ϕ_{0} whose sides are horizontal and vertical segments. Adjacent rectangles meet along a common horizontal or vertical segment of a side. Each point of Σ_{0} is required to be a vertex of a rectangle. For each pair of adjacent rectangles R_{1}, R_{2} choose open sets U_{i} ⊂ R_{i} with the property that if y_{i} ∈ U_{i} there is a vertical segment l_{i} with one endpoint y_{i} such that l_{1}, l_{2} are two sides of a coordinate rectangle contained in R_{1} ∪ R_{2}. For any two points y_{1}, y_{2} in the same coordinate rectangle there are vertical segments l_{i} with endpoints y_{i} which are the vertical sides of a coordinate rectangle. Now by the claim, by passing to further subsequences of t_{n}, for each open set U_{i} we may choose y_{i} ∈ U_{i} and x_{i} ∈ ∪_{j=1}^{m}E_{j} such that y_{i} = lim f_{t_{n}}(x_{i}).

Fig. 8

Now suppose U_{1}, U_{2} are such open sets contained in either adjacent or the same rectangle with corresponding y_{i} and x_{i}. We claim that x_{1} and x_{2} must belong to the same E_{j}. Assume otherwise. By renumbering assume x_{i} ∈ E_{i}, i = 1, 2. Let l_{i} be the vertical segments with endpoints y_{i} and which are vertical segments of a coordinate rectangle. They have the same length. Each l_{i} is the limit of vertical segments l_{i,n} of equal length of g_{t_{n}}ϕ with one endpoint f_{t_{n}}(x_{i}). Each l_{i,n} is the image under f_{t_{n}} of a vertical segment L_{i,n} ⊂ E_{i} passing through x_{i}; L_{i,n} have equal length which go to ∞ with n. Therefore the number of intersections of L_{i,n} with I goes to ∞ and since L_{i,n} ⊂ E_{i},

lim_{n→∞} \frac{card(l_{i,n} ∩ f_{t_{n}}(I))}{card(l_{2,n} ∩ f_{t_{n}}(I))} = lim_{n→∞} \frac{card(L_{1,n} ∩ I)}{card(L_{2,n} ∩ I)} = \frac{ν_{1}(I)}{ν_{2}(I)} ≠ 1.

But since l_{i,n} has limit l_{i} and every horizontal segment of ϕ_{0} that intersects l_{1} intersects l_{2} and vice versa,

\frac{card(l_{1,n} ∩ f_{t_{n}}(I))}{card(l_{2,n} ∩ f_{t_{n}}(I))}

can be made arbitrarily close to 1 by taking n large enough. This is a contradiction, proving the claim.

Any two rectangles R_{1}, R_{n} can be connected by a chain of rectangles R_{1}, …, R_{n} where R_{i} and R_{i+1} are adjacent along a side. This fact and the last claim imply that there is a single index j such that x_{i} ∈ E_{j} for all x_{i}.

Choose some index i ≠ j and x ∈ E_{i} \ Σ. Let y ∈ X_{0} be an accumulation point of f_{t_{n}}(x). By passing to further subsequences of t_{n} we can assume that there is a single rectangle R such that f_{t_{n}}(x) ∈ R and lim_{n→∞} f_{t_{n}}(x) = y. Let U_{k} ⊂ R be one of the open sets found previously. We may find vertical segments l ⊂ R with one endpoint y and l′ ⊂ R with endpoint y_{k} = lim f_{t_{n}}(x_{k}) such that l, l′ are two vertical sides of a coordinate rectangle contained in R. We may find vertical segments l_{n} and l_{n′} of g_{t_{n}}ϕ of equal length, l_{n} → l and l′_{n} → l′, l_{n} has endpoint f_{t_{n}}(x) and l′_{n} has endpoint f_{t_{n}}(x_{k}). As before

\frac{card(l_{n} ∩ f_{t_{n}}(I))}{card(l′_{n} ∩ f_{t_{n}}(I))}

can be made arbitrarily close to 1 by taking n large enough, and yet since x and x_{k} are points of E_{i} and E_{j} respectively, the ratio has limit

\frac{ν_{i}(I)}{ν_{j}(I)} ≠ 1,

and we have our contradiction. □ The proof of Theorem 3.5 follows from the above theorem and the next

Theorem 3.9 The set of θ ∈ [0, 2π] such that r_{θ}ϕ is divergent, has Lebesgue measure zero.

3.3 A combinatorial construction

In order to prove this theorem we will need to discuss certain a combinatorial construction. We fix a stratum QD(σ).

Definition 3.10 Two saddle connections are disjoint if their only common points are vertices.

Denote by p_{0} = p_{0}(σ) the maximum number of disjoint saddle connections. A set of disjoint saddle connections Γ is called a system. By |Γ| we mean the maximum length of any member of Γ. By a complex K we mean a subset of M consisting of disjoint saddle connections and triangles, such that a triangle is in K if and only if its sides are in K. An ϵ complex is a complex all of whose saddle connections have length at most ϵ. We denote the (topological) boundary of K by ∂K. Then ∂K is a system of saddle connections.

Note that if K is a complex, then K \ ∂K (viewed as a subset of M) is a (possibly empty) domain. We say that the complexity of a complex is j if it has j saddle connections, including those on the boundary. The main construction is as follows.

Proposition 3.11 If K is a complex of complexity i and σ is a saddle connection crossing ∂K or disjoint from K, then there is a complex K′ of complexity j > i with K ⊂ K′ and |∂K′| ≤ 3|∂K| + |σ|.

Proof. If σ is disjoint from K, we may just add σ to K. Thus we may assume that σ crosses ∂K at a point Q on a segment ω with endpoints A, B. In following σ from Q into the exterior of K to its other endpoint P, suppose σ hits P before again crossing ∂K. We may form a polygon with no interior singularities whose sides are ω, a possibly broken geodesic joining A to P and a possibly broken geodesic joining B to P. The edges and diagonals of this polygon have lengths bounded by |σ| + |ω|. Since ω is a boundary edge of K not all of the edges and diagonals of the polygon can be in K. We may therefore add an edge that is not in K.

Fig. 9

Thus assume that before next hitting P, σ again crosses ∂K at Q_{1}, an interior point of a segment ω_{1}. Let the endpoints of ω_{1} be denoted C, D. Let σ̂ be the subsegment of σ joining Q to Q_{1}. Now move σ̂ parallel to itself with one endpoint moving along ω toward A , the other along ω_{1} toward D. This can be done until for a first time a segment σ_{1} parallel to σ̂ either contains a singularity P_{0} in its interior or is an endpoint of ω or an endpoint of ω_{1}, say A. We have |σ_{1}| ≤ |σ| + |ω| + |ω_{1}|. In the first case we build a polygon as before, with σ_{1} replacing σ. In the second case also replace σ with σ_{1} and also replace ω with ω_{1} and P with A. Then we may use σ_{1}, A, and ω_{1} to form the desired polygon. □

Definition 3.12 Let ϵ < C. A saddle connection γ is (ϵ, C) isolated if it has length less than ϵ and every saddle connection that crosses γ in its interior has length greater than C.

Since any two (ϵ, C) isolated curves are disjoint, the number of such curves is bounded above by p_{0}.

Definition 3.13 For a quadratic differential ϕ let n_{ϵ}(ϕ) be the maximum number of simplices in a connected ϵ complex.

Definition 3.14 For a stratum QD(σ) let N(ϵ, C) be the set of quadratic differentials in the stratum which possess (ϵ, C) isolated saddle connections.

Proposition 3.15 Let μ denote Lebesgue measure on the circle. Suppose there is a set S of angles θ of positive measure so that r_{θ}ϕ is divergent for θ ∈ S. Then there are • (1) a sequence of times T_{i} → ∞

• (2) a sequence of sets S_{i} ⊂ S and a number δ > 0 such that μ(S_{i}) ≥ δ

• (3) a sequence ϵ_{i} → 0

• (4) a positive constant C such that g_{T_{i}}r_{θ}ϕ ∈ N(ϵ_{i}, C) for θ ∈ S_{i}.

The idea behind this Proposition is that since the Riemann surface of g_{t}r_{θ}ϕ eventually leaves every compact set, there will be times T_{i} after which there will always be a segment of length less than ϵ_{i} in the corresponding metrics. For a fixed C, however, the sets N(ϵ_{i}, C) do not form a neighborhood basis of infinity so it may not be the case that for sufficiently large T_{i} that there will be always (ϵ_{i}, C) isolated segments. Nevertheless by passing to slightly smaller sets than S_{i} we will be able to make the statement for isolated segments.

Proof. Choose a sequence ϵ_{i} → 0. Since for each ϵ_{i} the set QD_{ϵ_{i}} is compact, for each divergent ray r_{θ}(q) there are times T_{i} such that g_{t}r_{θ}(q) ∈ QD \ QD_{ϵ_{i}} for all t ≥ T_{i}. Then T_{i} can be chosen so that this relationship holds for all θ ∈ S′ where S′ ⊂ S and μ(S′) > μ(S)/2.

Consider now triples of sequences (ϵ_{i}, T_{i}, S_{i}) such that ϵ_{i} → 0, T_{i} → ∞, μ(S_{i}) bounded away from 0 and g_{T_{i}}r_{θ}ϕ ∈ QD \ QD_{ϵ_{i}} for θ ∈ S_{i}. The set of triples is nonempty for we can take S_{i} = S′ for all i. Choose a sequence of triples that maximizes

min_{i} min_{θ∈S_{i}} n_{ϵ}(g_{T_{i}}r_{θ}ϕ).

For the corresponding sequence, for each i and θ ∈ S_{i} let C_{i,θ} be the length of the shortest saddle connection that crosses a boundary segment of the ϵ_{i} complex of g_{T_{i}}r_{θ}ϕ. For each i, let m_{i} be a number so that

μ({θ ∈ S_{i} : C_{i,θ} ≥ m_{i}}) = μ(S_{i})/2.

We claim that the numbers m_{i} are bounded below. Suppose not. Choose a subsequence converging to 0 and for that subsequence let

S′_{i} = {θ ∈ S_{i} : C_{i,θ} < m_{i}},

so μ(S′_{i}) = μ(S_{i})/2 is bounded below, and let

ϵ′_{i} = 3ϵ_{i} + m_{i} → 0.

Fix θ ∈ S′_{i}. Since there is an ϵ_{i} complex and a saddle connection of length at most m_{i} crossing it, by Proposition 3.11 we can find an ϵ′_{i} complex on g_{T_{i}}r_{θ}ϕ with more simplices, contradicting the maximality of our sequence of triples. Thus the set of m_{i} is bounded below, by some C > 0. If we replace S_{i} with the set of θ such that C_{i,θ} ≥ m_{i} ≥ C, every saddle connection crossing the ϵ_{i} complex has length at least C, proving the proposition. □

Let α be a geodesic segment. For some θ_{α} , the segment α is vertical with respect to r_{θ_{α}}ϕ. Rotate the coordinate system so that θ_{α} = 0. Denote by l(t, θ) the length of α with respect to the structure g_{t}r_{θ}ϕ, so that |α|_{ϕ} = l(0, 0). In these coordinates

l(t, θ) = l(0, 0)(e^{t} sin^{2} θ + e^{−t} cos^{2} θ)^{1/2}.

Fix t ≥ 0 and choose ϵ > 0. Define two intervals I_{α} ⊂ J_{α} ⊂ (−π/2, π/2) with respect to the rotated coordinates:

I_{α} = {θ : | sin θ| ≤ ϵ/(ve^{t/2})},
J_{α} = {θ : | sin θ| ≤ C/(2ve^{t/2})}.

Lemma 3.16 For θ ∉ I_{α} one has: l(t, θ) ≥ ϵ. There are constants ϵ_{1}, T, K independent of α such that for ϵ < ϵ_{1} and t > T we have μ(I_{α})/μ(J_{α}) < Kϵ. In addition if l(t, θ) < ϵ for some θ, then l(t, θ) < C for all θ ∈ J_{α}.

Proof. The first statement is immediate from the definition. If l(t, θ) < ϵ for some θ then, since l(t, θ) attains its minimum at θ = 0,

ve^{−t/2}| cos θ| ≤ ve^{−t/2} ≤ l(t, θ) < ϵ.

For θ ∈ J_{α}, we then have

l(t, θ) ≤ (C^{2}/4 + ϵ^{2}e^{−t})^{1/2} ≤ C/2 + ϵ.

If we choose ϵ_{1} = C/2 and ϵ < ϵ_{1} then l(t, θ) < C. This proves the last statement.

Let I′_{α} and J′_{α} be the images of I_{α} and J_{α} under the sine function. If C/(2ve^{t/2}) < 1/2 then J′_{α} is contained in the interval (−1/2, 1/2) and μ(I′_{α})/μ(J′_{α}) = 2ϵ/C. Since the arcsine function restricted to the interval (−1/2, 1/2) is Lipschitz and has a Lipschitz inverse, it changes the lengths of intervals by a bounded amount. Thus we can choose a constant K depending on C such that μ(I_{α})/μ(J_{α}) < Kϵ as claimed.

To ensure that C/(2ve^{t/2}) < 1/2 choose T such that C/(2me^{T/2}) < 1/2 where m is the length of the shortest saddle connection on ϕ. Since v ≥ m and t > T, the inequality holds and the proof of the lemma is finished. □

The last technical statement we need is as follows.

Proposition 3.17 There are constants T, ϵ_{1}, K′ such that for t > T and ϵ < ϵ_{1} one has:

μ({θ : g_{t}r_{θ}ϕ ∈ N(ϵ, C)}) < K′ϵ.

Proof. Fix T and ϵ < ϵ_{1} as in the preceding lemma. Consider all saddle connections α which are (ϵ, C) isolated with respect to g_{t}r_{θ}ϕ for some θ. For each such α let Î_{α} be the smallest interval of θ′s for which α is (ϵ, C) isolated. Then Î_{α} ⊂ I_{α} ⊂ J_{α}.

Construct a new open interval Ĵ_{α} as follows: the left-hand endpoint of Ĵ_{α} will be halfway between the lefthand endpoints of Î_{α} and J_{α}, and the righthand endpoint of Ĵ_{α} will be halfway between the righthand endpoints of Î_{α} and J_{α}. The point of this construction is the fact that if Ĵ_{α} and Ĵ_{β} intersect, then either J_{α} intersects Î_{β} or J_{β} intersects Î_{α}. Assume the first. We can find θ ∈ J_{α} ∩ Î_{β} such that β is (ϵ, C) isolated with respect to g_{t}r_{θ}q. Now, with respect to the metric of g_{t}r_{θ}ϕ, the saddle connection α has length less than C, and β cannot cross any geodesic segment of length less than C. Thus α and β are disjoint. The maximum number of disjoint segments is p_{0}. Therefore no θ can lie in more than p_{0} segments Ĵ_{α}. The sum of the lengths of the Ĵ_{α} is at most 2πp_{0}. The sum of the lengths of the J_{α} is at most 4πp_{0}. The sum of the lengths of the intervals I_{α} is at most 4Kπϵp_{0}, and the intervals I_{α} cover the set of θ for which g_{t}r_{θ}ϕ ∈ N(ϵ, C). Thus we can take K′ = 4Kπp_{0}. □

Proof of Theorem 3.9. Assume the theorem is false. Then choose constants ϵ_{i} → 0, times T_{i} → ∞, a constant C, and sets S_{i} so that Proposition 3.15 holds. In particular, μ(S_{i}) ≥ δ > 0. Let K′, T be as in Proposition 3.17; then, for ϵ_{i} small enough and T_{i} > T large enough, we have: μ(S_{i}) < K′ϵ_{i}. However, for ϵ_{i} small enough, one has: K′ϵ_{i} < δ, a contradiction. □

3.4 Further ergodicity results

The measure 0 result in Theorem 3.5 was improved in [41] to give a statement about Hausdorff dimension.

Theorem 3.18 For any quadratic differential ϕ the set of θ ∈ [0, 2π] such that r_{θ}ϕ has minimal but non-uniquely ergodic vertical foliation has Hausdorff dimension at most 1/2.

On the other hand positive Hausdorff dimension turns out to be typical as was shown in [42]. Recall the stratum QD(σ) and SL(2, R) ergodic invariant measure μ_{0} on each component of QD(σ). Assume that σ ≠ (∅; +), (1, −1; −), (−1, −1, −1, −1; −), that is, the quadratic differentials do not define a flat torus, once punctured torus, or four times punctured sphere.

Theorem 3.19 For each component C of QD(σ) there is a δ = δ(C) > 0 such that for μ_{0} almost all ϕ ∈ C, the set of θ ∈ [0, 2π] such that r_{θ}ϕ has minimal but non-uniquely ergodic vertical foliation has Hausdorff dimension δ.

3.5 Ergodicity of general polygonal billiards

Now we apply Theorem 3.5 to polygonal billiards. As before, the collection of plane polygonal regions with n vertices and a given combinatorial type is identified with a subset of R^{2n} and is given a subspace topology. Recall that G(Q) denotes the group generated by reflections in the sides of a polygon Q.

The next result is in the spirit of Theorem 1.10 but is stronger.

Theorem 3.20 ([34]) Let X be a closed subset of the space of billiard tables with the property that for any number N the set of rational tables Q ∈ X with card(G(Q)) ≥ N is dense. Then ergodic tables in X form a dense G_{δ} subset.

Proof. We can assume that X ⊂ R^{2n} is compact. Each Q ∈ X is a polygonal region in the plane. We can assume that the area of Q is 1. Let P X be the bundle whose base space is X and the fiber P_{Q} over Q ∈ X is the phase space of the corresponding billiard table Q. Note that P X ⊂ X × R^{2} × S^{1}. Let μ_{Q} the product of area measure on Q with unit Lebesgue measure on S^{1}, and let ϕ_{t} be the billiard flow on P X.

Choose a sequence of continuous functions f_{1}, f_{2} … on P X which, when restricted to each P_{Q}, are dense in L^{2}(P_{Q}). We make the further assumption that if v is an outward and v′ is the corresponding inward vector on the boundary of Q, then f_{i}(v) = f_{i}(v′). Let E(i, n, T) be the set of Q ∈ X for which

∫_{z∈P_{Q}}(\frac{1}{T} ∫_{0}^{T} f_{i}(ϕ_{t}(z))dt − ∫_{P_{Q}} f_{i}dμ)^{2}dμ < 1/n.

Let

E(i, n) = ∪_{T =1}^{∞}E(i, n, T),

and let

E = ∩_{i=1}^{∞} ∩_{n=1}^{∞} E(i, n).

The set of Q ∈ X for which ϕ_{t} restricted to P_{Q} is ergodic is precisely E – see [44]. We will prove that

• (1) The sets E(i, n, T) are open, and

• (2) For a given i and n, there is an N such that E(i, n) contains all rational tables Q for which card(G(Q)) > N.

Assuming these statements, the theorem is proved as follows. The first stetement implies that the sets E(i, n) are open. The second one implies that the E(i, n) are dense. Then E is a G_{δ}, and it follows from the Baire category theorem that E is dense in X.

Statement (1) is a consequence of the next result.

Lemma 3.21 Let T > 0 be fixed, and let f be a continuous function on P X respecting the boundary identifications. Then

∫_{z∈P_{Q}}(\frac{1}{T} ∫_{0}^{T} f(ϕ_{t}(z))dt − ∫_{P_{Q}} fdμ_{Q})^{2}dμ_{Q}

depends continuously on Q.

Proof. For Q ∈ X let a(Q) = ∫_{P_{y}} fdμ_{Q}. Clearly, a(Q) depends continuously on Q.

Replace the function f by the function f̃ defined as follows: for z ∈ P_{Q} let f̃(z) = f(z) − a(Q). Then the proof of the lemma reduces to the proof of the continuity of the following function:

c(Q) = ∫_{z∈P_{Q}}[\frac{1}{T} ∫_{0}^{T} f̃(ϕ_{t}(z))dt]^{2}dμ_{Q}.

The difficulty is that, due to discontinuities of the billiard flow at the corners, the billiard orbits of close velocity vectors may diverge. Roughly speaking, this difficulty is overcome by deleting the set of velocity vectors whose trajectories hit a corner; this set has zero measure, and the time T trajectory depends continuously on the velocity vector in its complement.

More specifically, for z ∈ P_{Q} ⊂ P X introduce an auxiliary function l(z) as the infimum of the distance from the time T trajectory of vector z to the set of vertices of the polygon Q. Clearly l(z) depends continuously on z, and the zero level set of l(z), that is, the set of velocity vectors whose time T trajectory hits a vertex, has zero measure.

Let ϵ > 0 be fixed, and let M = sup f̃. For a neighborhood N_{1} of Q in the space of polygons denote by Q_{1} the intersection of all polygons from N_{1} (note that Q_{1} is not a polygon). We can choose N_{1} so small that the area of Q_{1} is at least 1 − ϵ/6M^{2}.

Let C_{δ} ⊂ P_{Q} be the set of velocity vectors z for which l(z) ≥ δ. Choose δ small enough so that the measure of C_{δ} is at least 1 − ϵ/6M^{2}. Let D_{δ} ⊂ C_{δ} consist of pairs (x, v) ∈ R^{2} × S^{1} with x ∈ Q_{1}. Note that the measure of D_{δ} is at least 1 − ϵ/3M^{2}. Since Q_{1} ×S^{1} is contained in P_{R} for all R ∈ N_{1}, we can identify N_{1} ×D_{δ} with a subset of P X in a natural way. Let l̄(R) be the infimum of l(z) for z ∈ (P_{R} ∩ (N_{1} × D_{δ})). Now l̄ is continuous and l̄(Q) = δ. We can find a neighborhood N_{2} ⊂ N_{1} so that for all R ∈ N_{2}, l̄(R) > 0. Let d(R) denote

d(R) = ∫_{z∈(P_{R}∩(N_{1}×D_{δ}))}[\frac{1}{T} ∫_{0}^{T} f̃(ϕ_{t}(z))dt]^{2}dμ_{R}.

Then

|d(R) − c(R)| ≤ M^{2}μ(P_{R} \ (P_{R} ∩ (N_{1} × D_{δ}))) ≤ ϵ/3

for each R ∈ N_{2}. For R ∈ N_{2} the function f(ϕ_{t}(z)) depends continuously on t ∈ [0, T] and z ∈ D_{δ}. Thus we can find a neighborhood N_{3} of Q in which d varies by less than ϵ/3. Then, for R ∈ N_{3},

|c(Q) − c(R)| ≤ |c(Q) − d(Q)| + |d(Q) − d(R)| + |d(R) − c(R)| ≤ ϵ.

This completes the proof of continuity of c and the proof of the Lemma. □

To finish the proof of the Theorem one needs to establish statement (2); this statement follows from the next lemma.

Lemma 3.22 Fix n > 0 and let f be a continuous function on P X. Choose δ > 0 so that if |θ_{1} − θ_{2}| < δ then |f(θ_{1}) − f(θ_{2})| < 1/2n. Let N ≥ 2/δ, and let Q be a rational polygon with |G(Q)| ≥ N. Then for sufficiently large T one has:

[∫_{z∈P_{Q}}[\frac{1}{T} ∫_{0}^{T} f(ϕ_{t}(z))dt − ∫_{P_{x}} fdμ_{Q}]^{2}dμ_{Q}]^{1/2} < 1/n.

Proof. Since Q is fixed we will drop the subscripts from P, G and μ. For θ ∈ S^{1} let u(θ) = 1/|G| ∫_{z∈M_{θ}} f(z)dA where dA is the area measure on M_{θ}; here, as before, M_{θ} is the invariant surface of the billird flow in direction θ. For z ∈ M_{θ} let u′(z) = u(θ). For z ∈ P let v_{T} (z) = 1/T ∫_{0}^{T} f(ϕ_{t}(z))dt. The quantity which appears in the lemma is the norm in the space L^{2}(P) of the function v_{T} − ∫ fdμ. We claim that

lim_{T →∞} ||v_{T} − u′|| = 0.

To prove the claim, notice that the surfaces M_{θ} are parametrized by θ ∈ S^{1}/G. We evaluate the norm by integrating first with respect to M_{θ} and then with respect to θ:

||v_{T} − u′|| = [|G| ∫_{θ∈S1/G} \frac{1}{|G|} ∫_{z∈M_{θ}}(v_{T} (z) − u′(z))^{2}dAdθ]^{1/2}.

Let

w_{T} (θ) = [\frac{1}{|G|} ∫_{z∈M_{θ}}(v_{T} (z) − u′(z))^{2}dA]^{1/2}.

Then

||v_{T} − u′|| = [|G| ∫_{θ∈S1/G} w_{T} (θ)^{2}dθ]^{1/2}.

For a given θ the ergodicity of ϕ restricted to M_{θ} implies that lim_{T →∞} w_{T} (θ) = 0. Theorem 3.5 implies ergodicity for almost all θ. Since the functions w_{T} are bounded and converge pointwise almost everywhere to 0, they converge to 0 in norm. This completes the proof of the claim.

The second claim is that

||u′ − ∫ fdμ|| ≤ 1/2n.

Now ||u′ − ∫ fdμ|| is equal to the norm of u − ∫ fdμ in L^{2}(S^{1}). Let θ_{1} and θ_{2} be points in the circle at which u assumes its minimum and maximum values m and M, respectively. Note that u is constant on the orbits of G. The distance between neighboring points in a G orbit is less than 2/|G| < 2/N < δ. By replacing θ_{2} by g(θ_{2}), where g ∈ G, we may assume that |θ_{1} − θ_{2}| < δ. It follows from the continuity assumption on f that since |θ_{1} − θ_{2}| < δ, then |u(θ_{1}) − u(θ_{2})| < 1/2n. Since u is defined by averaging f, ∫_{P} f = ∫_{S^{1}} u. Thus m ≤ ∫ f ≤ M; hence |u(θ) − ∫ f| ≤ 1/2n and ||u − ∫ f|| ≤ 1/2n. This completes the proof of the claim.

We now complete the proof of the lemma. Choose T sufficiently large so that ||v_{T} − u′|| ≤ 1/2n. Then

||v_{T} − ∫ f|| ≤ ||v_{T} − u′|| + ||u′ − ∫ f|| ≤ 1/2n + 1/2n = 1/n,

and we are done. □

3.6 Construction of polygons with ergodic billiard flow

Ya. Vorobets gave a constructive description of a topologically massive set of polygons with ergodic billiard flow – see [56]. We describe his result without proof.

Definition 3.23 Let ϕ(N) be a positive function of a positive integer N whose limit is zero as N → ∞. Let Q be a plane k-gon with angles α_{1}, ..., α_{k} between the adjacent sides. We say that Q admits approximation by rational polygons at the rate ϕ(N) if for every n > 0 there is N > n and positive integers n_{1}, ..., n_{k}, each coprime with N, such that |α_{i} − πn_{i}/N| < ϕ(N) for all i.

The next theorem gives an explicit estimate of how well a polygon should be approximated by rational ones to guarantee ergodicity of the billiard flow.

Theorem 3.24 Let Q be a polygon that admits approximation by rational polygons at the rate

ϕ(N) = (2^{2^{2^{2^{N}}}} )^{−1}.

Then the billiard flow in Q is ergodic.

The paper [56] also contain constructive versions of other results on polygonal billiards and flat surfaces. In particular it gives a different proof of the quadratic upper bound growth rate for the number saddle connections, a result we will discuss in more detail in the next section.

4 Periodic orbits

4.1 Periodic directions are dense

Suppose β is a closed geodesic of a quadratic differential ϕ that does not pass through a zero. As we have observed, β can be moved parallel to itself and sweeps out a cylinder of homotopic geodesics; we called such a cylinder a metric cylinder. The boundary of the metric cylinder is a union of parallel saddle connections. In the special case of the flat torus, there are closed geodesics in a dense set of directions, and, of course, for each such direction the closed geodesics fill the surface. For surfaces of higher genus, the boundary of the metric cylinder contains saddle connections. We have already seen that for a rational billiard, there are periodic trajectories that leave a side orthogonally. The main objective of this section is to prove the following theorem,

Theorem 4.1 ([38]) For any quadratic differential ϕ_{0} there is a dense set of directions θ ∈ S^{1} such that ϕ_{0} has a metric cylinder in the direction θ.

Corollary 4.2 For any rational billiard table there is a dense set of directions with a periodic orbit in that direction.

Theorem 4.1 was strengthened in the following result of Boshernitzan, Galperin, Kruger and Troubetzkoy – see [9].

Theorem 4.3 For any quadratic differential ϕ_{0} there is a dense set of vectors in the tangent space to the surface such that the orbit determined by that vector is closed.

The theorem is proved in [9] for rational billiards. The proof holds verbatim for the general case of a quadratic differential.

Recall that in the compactification of R_{g} we allow pinching Riemann surfaces along p ≤ 3g − 3 simple closed curves to produce Riemann surface X_{0} with nodes or punctures. Also recall that if ϕ_{n} is a sequence of unit norm quadratic differentials on X_{n}, we can pass to a subsequence and assume that ϕ_{n} converges uniformly on compact sets of X_{0} to a finite norm quadratic differential ϕ_{∞} on X_{0}. We single out the a set of limiting quadratic differentials.

Definition 4.4 A limiting quadratic differential ϕ_{∞} on X_{0} is called exceptional if

(a) some component of X_{0} is a torus and ϕ_{∞} defines the flat metric on the torus or

(b) some component of X_{0} is a punctured sphere and ϕ_{∞} has no zeroes or

(c) ϕ_{∞} ≡ 0 on every component of X_{0}.

We note that in case (b), since ϕ_{∞} has no zeroes, there must be exactly four punctures at which ϕ_{∞} has a simple pole, and at any other puncture, ϕ_{∞} is regular. The importance of this definition is illustrated by the following proposition. We are indebted to Yair Minsky for suggesting the proof. A different proof appeared in [38] based on work of [37].

Proposition 4.5 There exists M > 0 with the property that if X_{n} → X_{0} in R̄_{g} and ϕ_{n} are quadratic differentials on X_{n} which converge uniformly on compact sets of X_{0} to an exceptional ϕ_{∞}, then for large enough n, ϕ_{n} has a metric cylinder of length at most M.

Proof. In cases (a) and (b) there is a dense set of directions such that ϕ_{∞} has a metric cylinder in that direction. Choose one such metric cylinder and a neighborhood of the punctures such that the cylinder contains a closed curve missing that neighborhood. The uniform convergence of ϕ_{n} to ϕ_{∞} on the complement of this neighborhood implies that for large n, ϕ_{n} has a closed regular geodesic with some uniformly bounded length.

Thus we can assume we are in case (c). For any ϵ > 0 and for each loop C surrounding a puncture of X_{0} we have |C|_{ϕ_{n}} < ϵ for large n. The loops C divide X_{n} into p annuli B_{i} homotopic to the pinching curves and ϕ_{n} → ϕ_{∞} uniformly on X_{0} \ (∪B_{i}). Since ϕ_{∞} ≡ 0, the ϕ_{n} area of X_{0} \ (∪B_{i}) goes to 0 and so the ϕ_{n} area of ∪B_{i} goes to 1. For large n, for at least one annulus B, the ϕ_{n} area of B is at least 1/(3g − 3). This annulus has two boundary components C.

Fix such a large n. If there is no metric cylinder in the homotopy class of C then we may let α be the unique ϕ_{n} geodesic homotopic to the loop C. Then α passes through singularities, at some of which the angle is in excess of π. We will arrive at a contradiction. First assume that α is embedded. For each r let N(r) be the r neighborhood of α; the set N(r) is convex. For small values of r, N(r) is an annulus containing α in its interior (here is where we use that α is embedded). For all r it is a domain with nonpositive Euler characteristic. Let α_{r} = ∂N(r). For all except an isolated set of r, the curve α_{r} is smooth.

The Gauss-Bonnet formula says that

∫_{N(r)} K + ∫_{α_{r}} κds = 2πχ(N(r)) ≤ 0,

where K is the Gaussian curvature and κ is the geodesic curvature. Now K = 0 except at the interior points x of N(r), where the curvature is negative and concentrated so that the contribution to the first term is −mπ, for m a positive integer. The number m may jump a bounded number of times as more zeroes are included in N(r) and so is maximized by some M. Thus

∫_{α_{r}} κds ≤ mπ ≤ Mπ.

with equality in the first inequality as long as N(r) remains an annulus. Let A(r) = ||N(r)||, the area of N(r), and let L(r) denote the length of α_{r}. Then

A′(r) = L(r) and L′(r) = ∫_{α_{r}} κds ≤ mπ ≤ Mπ,

again with equality in the first inequality as long as N(r) remains an annulus. In particular this gives L(r) ≥ πr, as long as N(r) remains an annulus. Now the fact that A′′(r) ≤ Mπ together with A(0) = 0 and A′(0) ≤ ϵ, gives

A(r) ≤ Mπr^{2} + ϵr.

Choose ϵ small enough so that

A(ϵ) ≤ \frac{1}{3g − 3}.

We show that B ⊂ N(ϵ) and that this leads to the desired contradiction. Let r_{0} = ϵ/π. The first step is to notice that each component C of the boundary of B must intersect N(r_{0}). If N(r_{0}) is an annulus this follows from the fact that |α_{r_{0}}| ≥ r_{0}π = ϵ and N(r_{0}) is convex, so any curve, namely C, of length smaller than ϵ cannot be homotopic to and exterior to N(r_{0}). If χ(N(r_{0})) < 0, then C must intersect N(r_{0}) since C is homotopic to α. Since C has length at most ϵ the fact that C intersects N(r_{0}) implies that C must be contained in N(ϵ(1/π + 1/2)) ⊂ N(ϵ). Then both boundary components C of B are contained in N(ϵ), so we must have B ⊂ N(ϵ). But now

||B|| ≤ A(ϵ) ≤ \frac{1}{(3g − 3)},

which is a contradiction.

In case α is not embedded, so that α is not in the interior of N(r), we lift α to the annular cover so that it is embedded. We have the same lower bound growth estimates for the length of curves in a neighborhood of the lifted α as well as the upper bound for the area of the neighborhood. This neighborhood projects to N(r). The rest of the proof then goes through. □

The following Lemma says that the Teichmuller flow extends continuously to the compactification.

Lemma 4.6 If ϕ_{0} is not identically 0 on every component of X_{0}, then for any t, g_{t}(ϕ_{n}) → g_{t}(ϕ_{0}).

Proof. The proof is immediate by viewing the flow in the natural coordinates of quadratic differentials. □

For a given genus g, a stratum QD(σ) is not a closed subset of Q_{g}, unless σ consists of a single zero. This is because a sequence in QD(σ) may collapse a pair of lower order zeroes into a higher order zero. In addition, the degeneration of Riemann surfaces allows limits of quadratic differentials on which the topology of the surface has changed. Because a degenerated surface need not be connected, in the following definition we allow disconnected surfaces and corresponding strata.

Definition 4.7 A stratum QD(σ′) is a degeneration of a stratum QD(σ) if QD(σ′) ≠ QD(σ) and there exists a sequence ϕ_{n} ∈ QD(σ) on Riemann surfaces X_{n} ∈ R̄_{g} such that X_{n} → X_{0} in R̄_{g} and ϕ_{n} converges uniformly on compact sets to some ϕ_{∞} ∈ QD(σ′) where ϕ_{∞} is not identically zero.

For any quadratic differential ϕ which is not identically zero, denote by l(ϕ) the length of the shortest saddle connection of ϕ. Notice that l(·) is not continuous under degeneration within a fixed genus. This is because zeroes are collapsed. Thus if ϕ_{n} ∈ QD(σ) is a sequence converging to ϕ_{∞} ∈ QD(σ′) then l(ϕ_{n}) → 0 while l(ϕ_{∞}) > 0.

Now suppose a nonzero quadratic differential ϕ_{0} on a connected surface is given. For any closed interval I ⊂ [0, 2π] let

E_{I}(ϕ_{0}) = {ϕ : ∃θ_{n} → θ ∈ I, ∃t_{n} → ∞, ∃ψ_{n} : r_{ψ_{n}}g_{t_{n}}r_{θ_{n}}ϕ_{0} → ϕ}.

That is to say, E_{I} is the set of all ω limit points ϕ in the SL(2, R) orbit of ϕ_{0} where the limiting initial rotation angle lies in I. Here the limiting ϕ may belong to any stratum. Note that θ_{n} need not lie in I. Now Theorem 4.1 will follow from the next proposition.

Proposition 4.8 For any ϕ_{0}, E_{I}(ϕ_{0}) contains an exceptional ϕ.

Proof. Since I is closed, E_{I} = E_{I}(ϕ_{0}) is compact. Assume E_{I} does not contain any exceptional ϕ; we will arrive at a contradiction. Let

E_{I}^{T} = {ϕ ∈ E_{I} : ∀ϕ′ ∈ E_{I}, QD(σ′(ϕ′)) is not a degeneration of QD(σ(ϕ))}.

That is, E_{I}^{T} is the set of ϕ ∈ E_{I} which belong to a stratum that cannot be further degenerated within E_{I}. Let

l = inf_{ϕ∈E_{I}^{T}} l(ϕ).

If l = 0 choose a sequence ϕ_{n} ∈ E_{I}^{T} with lim l(ϕ_{n}) → 0. By passing to a subsequence we may assume that ϕ_{n} → ϕ_{∞} ∈ E_{I}. Since l = 0 then either the Riemann surface of ϕ_{n} has degenerated or some set of zeroes of ϕ_{n} have been collapsed to a higher order zero of ϕ_{∞}. In either case ϕ_{∞} belongs to a stratum which is a degeneration of the stratum containing ϕ_{n}, a contradiction to the definition of E_{I}^{T}.

Suppose on the other hand that l > 0. Choose t_{n} → ∞, θ_{n} → θ_{1} ∈ I such that

g_{t_{n}}r_{θ_{n}}ϕ_{0} → ϕ_{1} ∈ E_{I}^{T} .

Then l(ϕ_{1}) ≥ l. Let γ be a saddle connection of ϕ_{1} such that l(ϕ_{1}) = |γ|_{ϕ_{1}}. Let θ_{γ} be such that γ is vertical with respect to r_{θ_{γ}}ϕ_{1}. As s → ∞ the length of γ with respect to the metric of g_{s}r_{θ_{γ}}ϕ_{1} goes to 0. Thus we can choose s such that the length of γ with respect to the metric of g_{s}r_{θ_{γ}}ϕ_{1} is less than l/2. We will have a contradiction if we can show that g_{s}r_{θ_{γ}}ϕ_{1} ∈ E_{I}^{T}. Since the stratum containing ϕ_{1} cannot be degenerated within E_{I} we only need to show that g_{s}r_{θ_{γ}}ϕ_{1} ∈ E_{I}. Now by Lemma 4.6

g_{s}r_{θ_{γ}}ϕ_{1} = lim_{n→∞} g_{s}r_{θ_{γ}}g_{t_{n}}r_{θ_{n}}ϕ_{0}.

We claim that there exist sequences θ′_{n} → 0, t′_{n} → ∞, and ψ_{n} → θ such that

g_{s}r_{θ_{γ}}g_{t_{n}} = r_{ψ_{n}}g_{t′_{n}}r_{θ′_{n}}.

Canonically identify G with the unit tangent vectors to the upper half-plane with Id identified with the vertical vector at i = \sqrt{−1}. Then SL(2, R) acts by Mobius transformations. Furthermore ∞ is an attracting fixed point of the geodesic flow g_{t}. Therefore for each n, there is a unique rotation r_{−ψ_{n}} such that r_{−ψ_{n}}g_{s}r_{θ}g_{t_{n}} takes the vertical vector at i to a vector based on the imaginary axis and Then for some some small θ_{n′} and large t′_{n}, r_{−θ′_{n}}g_{−t′_{n}} takes this vector to the vertical vector at i, proving the claim. Finally,

g_{s}r_{θ_{γ}}ϕ_{1} = lim_{n→∞} r_{ψ_{n}}g_{t′_{n}}r_{θ_{n}+θ′_{n}}ϕ_{0},

that is, g_{s}r_{θ_{γ}}ϕ_{1} ∈ E_{I}, and we are done. □

Proof of Theorem 4.1. Choose a closed subinterval I′ of I contained in the interior of I. Then Proposition 4.8 says that there must be some exceptional ϕ ∈ E_{I′}. By Proposition 4.5, for a sequence t_{n} → ∞ and θ_{n} → θ_{0} ∈ I′, g_{t_{n}}r_{θ_{n}}ϕ_{0} has a metric cylinder in the homotopy class of some β_{n} and the length is uniformly bounded above by some M. Since t_{n} → ∞ and g_{t_{n}} expands horizontal lengths, the horizontal length of β_{n} in the metric of r_{θ_{n}}ϕ_{0} goes to 0.

Thus for some θ′_{n} → 0, the quadratic differential r_{θ′_{n}}r_{θ_{n}}ϕ_{0} has a metric cylinder in the class of β_{n} and since θ_{n} + θ′_{n} ∈ I, for large n, we are done. □

Proof of Theorem 4.3. Let T_{1}(S) = {(p, θ)} denote the unit tangent space to S; here p ∈ S and θ ∈ S^{1}. Let ϵ > 0. We begin by fixing a minimal uniquely ergodic direction θ ∈ S^{1}. Unique ergodicity implies that we can choose N large enough so that for all x ∈ S the leaf through x is ϵ/2-dense in S.

There are only a finite number of saddle connections of length less than or equal to 2N and so their tangent vectors lie in a finite set of directions. This means that we can choose δ sufficiently small so that if θ′ satisfies |θ − θ′| < δ, then any saddle connection in direction θ′ has length at least 2N. We may also choose δ < ϵ/2N. ¿From Theorem 4.1 we know that there is a point (x_{0}, θ_{0}) on a closed leaf L_{θ_{0}} in a metric cylinder such that |θ_{0} − θ| < δ. Now let L_{θ} be the leaf through x_{0} in direction θ extended to length N in each direction, or to a zero. Now we claim that in at least one of the two possible directions, moving distance N, the endpoint of L_{θ} and the endpoint of L_{θ_{0}} are within ϵ/2 of each other. To prove the claim, for each direction, form the simply connected domain bounded by the segments and a geodesic joining their endpoints. If both simply connected domains contained a zero in their interior, there would be a saddle connection γ of length at most 2N in direction θ′ satisfying |θ′ − θ| < δ, contradicting the choice of δ. Thus in one direction the simply connected domain is actually a metric triangle with no interior zeroes. The choice of δ < ϵ/2N says now that the endpoints are within ϵ/2 of each other, proving the claim.

Since the segment of L_{θ} is ϵ/2 dense in S, the leaf L_{θ_{0}} is ϵ dense. This can be done for every uniquely ergodic direction θ. The set of vectors determining uniquely ergodic directions is dense in the whole phase space T_{1}S. Since ϵ is arbitrary this completes the proof. □

4.2 Counting saddle connections and maximal cylinders

In this section we discuss some results on the asymptotics of the number of saddle connections and periodic orbits. No proofs will be given. As noted in the section on billiards in the unit square, the number of (parallel families of) periodic orbits of length at most L grows asymtotically as πL^{2}/2ζ(2). For a general rational billiard or flat structure, asymptotics are not known. However we do have quadratic upper and lower bounds. To be specific, given a flat structure ϕ and L > 0, let N_{1}(L) be the number of saddle connections of length at most L and N_{2}(L) the number of maximal metric cylinders of length at most L. The following theorem was proved in [39] and Theorem 4.9 There exists positive constants c_{1}, c_{2} such that

c_{2}L^{2} ≤ N_{2}(L) ≤ N_{1}(L) ≤ c_{1}L^{2}.

Different proofs of the upper estimate are given in [56] and [13]. It is possible to prove precise asymptotics for generic flat structures.

Theorem 4.10 ([13]) There exists constants c = c(σ) and s = s(σ) such that on each component of QD(σ), for μ_{0} a.e. ϕ ∈ QD(σ) one has: N_{1}(L) ∼ cL^{2} and N_{2}(L) ∼ sL^{2}.

In the next section we discuss some circumstances under which these precise asymptotics are known to hold.

5 Veech groups

5.1 Definition and examples of Veech groups

In this section we discuss the groups of affine transformations associated to flat structures. These groups give rise to subgroups in P SL(2, R). If these groups are lattices in P SL(2, R), then the flat structure has particularly nice properties.

Let ϕ be a quadratic differential on the Riemann surface X. Let Σ be a subset of X which contains the zeroes of ϕ (but may be larger). Let X′ = X \ Σ. Following Veech we define the affine group Aff^{+}(ϕ) to be the group of orientation preserving self homeomorphisms of X that map X′ to itself and are affine with respect to the natural coordinates of ϕ. This means that for each point p ∈ X′ we choose coordinates z near p so that ϕ = dz^{2} and coordinates w near f(p) so that ϕ = dw^{2}, and in these coordinates, the map f is an affine homeomorphism. We call the derivative (i.e., linear part) of that homeomorphism the derivative of f. It is well-defined independently of p, z and w up to a factor of ±I. Let a(f) = ±A be the derivative. Then by an observation of Veech [54] we have

|f^{∗}ϕ| = det(A)|ϕ|

and

∫ |ϕ| = ∫ |f^{∗}(ϕ)|,

which implies that the determinant is 1 and so A is an element of G = P SL(2, R). We define the Veech group Γ(ϕ) to be the image of Aff^{+}(ϕ) under the derivative map.

The case of most interest is when Γ is a lattice in G. Recall that a lattice Γ in G is a discrete subgroup such that G/Γ has finite volume; a lattice is nonuniform if the quotient is not compact (see, e.g., [5]).

We present several examples. The first is essentially due to Thurston [49]. Recall that a (right) Dehn twist about a simple closed curve α on a surface is a homeomorphism of the surface which fixes α and twists any curve β crossing α once to the right about α.

Take the unit square with lower left vertex at the origin and identify opposite vertical sides. Mark off 4 points on both the top and bottom of the resulting cylinder C, dividing the top and bottom into equally spaced intervals and so that the points on top lie directly above the points on the bottom. Arrange the intervals on top into two pairs and glue each pair isometrically as indicated in the figure. Do the same for the bottom.

Fig. 10

The Euclidean metric on the rectangle extends to a quadratic differential ϕ = dz^{2} on the glued surface X, which has genus 2. The 4 marked points on the top identify to a single zero of order 2; similarly for the bottom. Let Σ consist of these two points. We define an affine map f_{1} : X → X which in the coordinates (x, y) is given by

(x, y) → (x + y mod(1), y).

The map is easily seen to be well-defined on the identification space, fixes Σ and the top and bottom of C, and acts as a right Dehn twist on the core curve of C. Its derivative is

A_{1} = ( \begin{matrix}1 & 1 \\ 0 & 1 \end{matrix} ) .

To find a second affine map f_{2} notice that the gluings force all trajectories in the vertical direction to be closed as well. Indeed, an upward vertical trajectory hits the top horizontal side of the square at some point S and continues downwards from the point T of the same horizontal side which is identified with S. The surface X decomposes into vertical metric cylinders C_{1}, C_{2} of circumference 2 and height 1/4 each composed of 2 vertical strips glued together; C_{1} is formed from the rectangles 0 ≤ x ≤ 1/4 and 1/2 ≤ x ≤ 3/4 glued along tops and bottoms and similarly for C_{2}.

(Note that here circumference is the vertical dimension of the cylinders while height is the horizontal one). This allows us to introduce coordinates (0 ≤ x′ ≤ 1/4, 0 ≤ y′ ≤ 2) in C_{1} by x′ = x, y′ = y for 0 ≤ x ≤ 1/4 and x′ = 3/4 − x, y′ = 2 − y for 1/2 ≤ x ≤ 3/4 and similarly for C_{2}. The map f_{2} is a right Dehn twist in each C_{i}. It is defined by

(x′, y′) → (x′, y′ + 8x′mod(2)).

It again fixesd Σ and its derivative is

A_{2} = ( \begin{matrix}1 & 0 \\ 8 & 1 \end{matrix} ) .

The matrices A_{1}, A_{2} generate a finite index subgroup of P SL(2, Z).

Gutkin and Judge [24] showed that the Veech group is commensurable with SL(2, Z) if and only if the surface can be tiled by a single Euclidean parallelogram. Recall that two subgroups Γ_{1}, Γ_{2} of G are commensurable if there is some g ∈ G such that gΓ_{1}g^{−1} ∩ Γ_{2} has finite index in both gΓ_{1}g^{−1} and Γ_{2}.

The next set of examples are due to Veech – see [54], [55]. Part of the motivation was to construct an example of a non-arithmetic group, that is, a group, not commensurable to P SL(2, Z). These examples are the flat structures ϕ_{n} associated with billiards in right triangles with angles π/n, n ≥ 5. Veech proved the following theorem. The set Σ consists of the zeroes of ϕ_{n}.

Recall that the (p, q, r) triangle group is the index 2 subgroup of the group generated by reflections in the sides of a triangle with angles π/p, π/q, π/r in the hyperbolic plane; the triangle group is a subgroup of G, the group of isometries of the hyperbolic plane. If an angle is 0, then the vertex of the triangle is at infinity (see, e.g., [5]).

Theorem 5.1 For n odd, Γ(ϕ_{n}) is a (2, n, ∞) triangle group. For even n = 2m, Γ(ϕ_{n}) is the (m, ∞, ∞) triangle group. In either case the Riemann surface X is the nonsingular surface associated to y^{2} + x^{n} = 1.

One can check that the invariant surfaces for this triangle and the isosceles triangle with equal angles of π/n are translation isomorphic so that they determine the same Veech group.

We will not give a proof of the theorem but illustrate the main idea by an example. It is convenient to discuss Γ(ϕ_{8}) as this invariant surface was already descirbed in figure 4. The invariant surface is the regular octagon P with opposite sides identified. The surface X has genus 2 and the quadratic differential ϕ_{8} has a single zero of order 4 corresponding to the identification of all of vertices of P. The counterclockwise rotation f_{1} of order 8 about the center of P is obviously affine with respect to ϕ_{8} and is holomorphic with respect to X. It fixes the zero. Its derivative is

A_{1} = ( \begin{matrix}\sqrt{2}/2 & −\sqrt{2}/2 \\ \sqrt{2}/2 & \sqrt{2}/2 \end{matrix} ) .

To find a second affine map, rotate the octagon so that two of the opposite sides are vertical. Then we claim that every horizontal trajectory is closed and that the surface decomposes into two metric cylinders. The first cylinder C_{1} consists of horizontal segments joining the opposite vertical sides. Its boundary consists of the two horizontal segments joining opposite vertices. Next notice that because of the identifications made, each horizontal segment leaving one of the nonvertical sides is also closed; it first hits the other nonvertical side before closing. The family of these closed trajectories determine the second metric cylinder C_{2}. The boundary includes the first boundary together with an extra closed saddle connection which is the top and bottom horizontal segments of the octagon. If we normalize so that the sides of P are of length 1, then C_{1} has circumference cot π/8 = 1 + \sqrt{2} and height 1, while C_{2} has circumference 1 + cot π/8 = 2 + \sqrt{2} and height \sqrt{2}/2. Notice that the ratio of height to circumference for the first cylinder is twice that for the second one. The ratio of height to circumference is called the modulus of a metric cylinder.

Fig. 11

Then there exists an affine map which preserves each cylinder fixing the boundary pointwise and whose derivative is

A_{2} = ( \begin{matrix}1 & 2 cot π/8 \\ 0 & 1 \end{matrix} ) .

The map is the square of a right Dehn twist on C_{1} and a right Dehn twist on C_{2}. Now A_{1}, A_{2} generate a (8, ∞, ∞) triangle group which is not commensurable with P SL(2, Z) ([5]).

The following theorem combines results from [12],[57] and [59], and complements Theorem 5.1.

Theorem 5.2 The Veech groups of the flat surfaces associated with the three series of triangles with angles

(π/2n, π/n, (2n − 3)π/2n), ((2k − 1)π/4k, (2k − 1)π/4k, π/2k)

and

(kπ/2k + 1, kπ/2k + 1, π/2k + 1), n ≥ 4, k ≥ 2,

are, respectively,

(3, n, ∞), (2k, ∞, ∞) and (2k + 1, ∞, ∞)

triangle groups. In all cases the set Σ is the set of zeroes.

5.2 Veech dichotomy

We now discuss some properties of Γ(ϕ). These results are all due to Veech – [54], [55].

Lemma 5.3 Let f ∈ Aff^{+}(ϕ) such that A = a(f) ≠ ±Id, and let ζ ∈ R^{2} \ {0} such that Aζ = ±ζ. Then every leaf in directon corresponding to ζ is either a saddle connection or closed so that the surface decomposes into cylinders C_{1}, …, C_{p} in direction ζ. Moreover there exists a positive integer k such that for each i the map f^{k} preserves C_{i}, fixing the boundary, and is a power of a Dehn twist in C_{i}.

Proof. Clearly f fixes the foliation in direction ζ. For some k, that can be assumed even, f^{k} maps each separatrix to itself. Since Aζ = ±ζ, f^{k} must be the identity on each separatrix. Each separatirix is either a saddle connection or dense in an open set. In the latter case, since f^{k} is real analytic, f^{k} would be the identity in the open set, and so A^{k} = ±Id. However A is an unipotent, therefore A^{k} ≠ ±Id. Thus every separatrix is a saddle connection which means that the surface decomposes into cylinders such that f^{k} is the identity on the boundary of each cylinder. As it is linear on each cylinder, it must be a power of a Dehn twist. □

Veech also shows that the moduli of the cylinders are rationally related.

Lemma 5.4 If a(f) ≠ ±Id then f is not isotopic to the identity.

Proof. Since f is affine in the coordinates determined by ϕ, it is a Teichmuller map determined by ϕ. Teichmuller maps are extremal quasiconformal maps within their homotopy class. This means that f cannot be homotopic to the identity, since the identity is conformal. □

Corollary 5.5 Aff^{+}(ϕ) is a subgroup of the mapping class group of the Riemann surface.

Lemma 5.6 Γ(ϕ) is a discrete subgroup of G.

Proof. Suppose f_{n} ∈ Aff^{+}(ϕ) is such that a(f_{n}) → ±Id. By passing to a subsequence and using the Arzela-Ascoli theorem we can assume that f_{n} converges uniformly to a diffeomorphism f. For large m, f_{m} ∘f_{m+1}^{−1} is isotopic to the identity, which by Lemma 5.4 gives a(f_{m+1}) = a(f_{m}) and so a(f_{m}) = ±Id for large m. □

We now consider consequences of the assumption that Γ(ϕ) is a lattice. Begin again by canonically identifying G with the unit tangent vectors to the upper halfplane with Id identified with the vertical vector at i = \sqrt{−1}. Then SL(2, R) acts on itself by Mobius transformations. Recall that SL(2, R) also acts on quadratic differentials. If we associate the identity in G to ϕ, then we canonically identify the SL(2, R) orbit of ϕ with SL(2, R).

Theorem 5.7 (Veech dichotomy). Suppose Γ(ϕ) is a lattice. For any direction either the foliation in that direction is minimal or its every leaf is closed or a saddle connection. In the latter case the surface decomposes into cylinders of closed leaves and there is f ∈ Aff^{+}(ϕ) with a(f) ≠ I which preserves these cylindrers and acts as Dehn twists on them. Furthermore Γ is a nonuniform lattice.

Proof. If the foliation is not minimal there is a saddle connection in that direction. By rotating, we can assume the direction is vertical. Then g_{t}ϕ has a saddle connection which approaches 0 in length. This means that the ω limit set under the flow g_{t} of Id in G/Γ is empty. Since Γ is a lattice this can only happen if the point at infinity is a fixed point for some parabolic element of Γ. In particular, the lattice is not uniform. Such a parabolic element fixes the vertical vector ζ = [0, 1]. It remains to apply Lemma 5.3. □

A different proof of the above theorem can be found in [57].

Theorem 5.7 says that, just as in the case of the flat torus, there is a dichotomy: either the flow is minimal or every orbit is closed. Since the triangle groups are lattices, the Veech dichotomy applies, in view of Theorem 5.1, to billiards in the isosceles triangles with equal angle π/n; it applies to regular polygons as well.

In fact, the analogy with the flat torus extends to ergodicity as well.

Theorem 5.8 Suppose Γ(ϕ) is a lattice. If the foliation in a direction is minimal, then it is uniquely ergodic.

Proof. Again we can assume the direction is vertical. If the foliation is not uniquely ergodic then by Theorem 3.8 g_{t}ϕ eventually leaves every compact set of moduli space. In particular, it cannot have an ω limit point in G/Γ. Again this implies that Γ has a parabolic element fixing infinity. By Lemma 5.3 the leaves in the vertical direction are all closed, which contradicts the minimality. □

5.3 Asymptotics

Let ϕ be a quadratic differential. Recall that we defined N_{2}(ϕ, T) as the number of homotopy classes of closed geodesics of length less than T or equivalently, the number of maximal cylinders whose circumference has length at most T. Recall in Theorem 4.9 there are upper and lower quadratic bounds for the number of maximal cylinders. Veech (op. cit.) has shown the following theorem.

Theorem 5.9 Suppose Γ(ϕ) is a lattice. Then there exists a constant c = c(ϕ) such that N_{1}(T) ∼ cT^{2}.

The proof is using Eisenstein series. The constant c can be computed explicitly in the case of isosceles triangles T_{n} with angles (π/n, π/n, (n − 2)π/n). In that case

c = \frac{n(n^{2} − 1)}{48π(n − 2)|T_{n}|},

where |T_{n}| is the area of the triangle.

Gutkin and Judge [24] have given another proof of the same theorem using the mixing of the horocyclic flow. The constant c is given in their work in purely topological terms.

5.4 Covers

Constructing a flat surface M from a rational billiard polygon P it may happen that some of the cone angles are equal to 2π or equivalently, the points are not zeroes of the quadratic differential. For example, if P is a square then M is a flat torus with no singular points. From the billiard view point this means that one can continuously define the extention of the billiard trajectory through the vertex of P; this is possible if and only if the angle is of the form π/n. Adding such a point to Σ, the set of points that are required to be preserved by the affine diffeomorphisms may change the Veech group. It is particularly important to keep track of removable singular points in the study of covers of flat structures.

Definition 5.10 Let (X_{i}, ϕ_{i}), i = 1, 2 be flat structures and Σ_{i} ⊂ X_{i}, i = 1, 2 sets that contain the zeroes of X_{i}. Then (X_{1}, ϕ_{1}, Σ_{1}) covers (X_{2}, ϕ_{2}, Σ_{2}) if there exists a continuous map f : X_{1} → X_{2}, called a cover, that sends Σ_{1} to Σ_{2} and which is given, in local coordinates on the complement of these sets by parallel translation. The multiplicity of a cover is the number of preimages of a non-singular point in X_{2} (independent of the point).

In particular, a cover is a branched holomorphic cover of X_{2} by X_{1} branched over Σ_{2}.

Definition 5.11 Two covers f_{1} : X_{1} → Y and f_{2} : X_{2} → Y are called isomorphic if there exists an isomorphism of the flat structures g : X_{1} → X_{2} such that f_{1} = f_{2}g. Similarly one defines an isomorphism of covers f_{1} : Y → X_{1} and f_{2} : Y → X_{2}.

The following finitness property holds (see [57] for a proof).

Proposition 5.12 A given flat structure admits a finite number of isomorphic classes of covers with a given multiplicity. Likewise, the number of isomorphic classes of covers realizable by a given flat structure is finite.

The next construction provides examples of covers of flat structures associated with billiards in polygons. Let P_{1} and P_{2} be rational polygons, and assume that P_{2} tiles P_{1} by reflections. This means that P_{1} is partitioned into a number of isometric copies of P_{2}, each two either disjoint, or having a common vertex or a common side, and if two of these polygons have a side in common then they are symmetric with respect to this side.

For example, a right triangle with angle π/n tiles by reflections a regular n-gon – see figure 4 for n = 8. Another example is given by the above mentioned Gutkin’s almost integrable billiard polygons: if such a polygon is drawn on a, say, integer square lattice in the plane, then it is tiled by the unit square.

Let (X_{i}, ϕ_{i}) be the flat structures associated with the polygon P_{i}, i = 1, 2, and assume that P_{2} tiles P_{1}. Let Σ_{1} be the set of zeroes of ϕ_{1} (cone points with angles greater than 2π.

Lemma 5.13 There is a set Σ_{2} containing the zeroes of ϕ_{2} such that (X_{1}, ϕ_{1}, Σ_{1}) covers (X_{2}, ϕ_{2}, Σ_{2}).

Proof. Denote by G_{1} and G_{2} the groups generated by the linear parts of the reflections in the sides of the polygons P_{1} and P_{2}. Then G_{1} ⊂ G_{2}. Each copy of P_{2}, involved in the tiling of P_{1}, is identified with P_{2} by an isometry; this isometry is a composition of reflections in the sides of P_{2}. These identifications, combined, define a projection p from P_{1} to P_{2}. Given a point x ∈ P_{1}, let α(x) ∈ G_{2} be the linear part of the isometry that takes P_{2} to the tile (a copy of P_{2}) that contains x.

Consider the map g : P_{1} × G_{1} → P_{2} × G_{2} that sends (x, β) to (p(x), βα(x)). Notice that the surfaces X_{i} are quotient spaces of P_{i} × G_{i}, i = 1, 2, and the map g determines a map f : X_{1} → X_{2}. This is the desired cover. The multiplicity equals n/m where n is the number of tiles in the tiling and m = [G_{2} : G_{1}].

Some singular points of the first structure may project to removable singular points of the second one. This happens only if P_{2} has a vertex angle π/n. One then adds these points to Σ_{2}. □

Note that one does not need to add removable singular points if none of the angles of P_{2} is of the form π/k.

As an example to Lemma 5.13 consider again a regular octagon P_{1} tiled by right triangles P_{2} with angle π/8. The flat surface M_{2} for the triangle is the regular octagon whose opposite sides are identified, while the flat surface M_{1} for the regular octagon is a union of 8 octagons with some gluings of the sides. Thus M_{1} covers M_{2} with multiplicity 8.

The next result relates the Veech groups of flat structures one of which covers the other; it was obtained independently by Gutkin and Judge [23], [24] and by Vorobets [57]. Recall that two subgroups of G are commensurate if they share a common subgroup that has a finite index in each.

Theorem 5.14 If (X_{1}, ϕ_{1}, Σ_{1}) covers (X_{2}, ϕ_{2}, Σ_{2}) then Γ(ϕ_{1}) and Γ(ϕ_{2}) are commensurate. In particular, one of the groups Γ(ϕ_{1}) and Γ(ϕ_{2}) is a lattice if and only if the other is.

Proof. Consider the set of triples S = {(X, ϕ, f)} where (X, ϕ) is a flat structure and f : X_{1} → X is a cover. Let S̄ be the set of equivalence classes of such triples considered up to isomorphism of covers.

Let g ∈ Aff^{+}(ϕ_{1}) and let A = a(g) be its derivative. Then fg is a cover of the flat structure (X, A^{−1}(ϕ)) by (X_{1}, ϕ_{1}). This defines a right action of Aff^{+}(ϕ_{1}) on S, and this action descends to an action on S̄. Consider the subgroup Aff_{0}^{+}(ϕ_{1}) ⊂ Aff^{+}(ϕ_{1}) that consists of the affine transformations acting trivially on S̄. By Proposition 5.12 S̄ is finite, therefore this subgroup has finite index. Let Γ_{0} ⊂ Γ(ϕ_{1}) consist of the derivatives of the elements of Aff_{0}^{+}(ϕ_{1}); then Γ_{0} has finite index too.

Let g ∈ Aff_{0}^{+}(ϕ_{1}) and A = a(g). Then, for every (X, ϕ, f) ∈ S the flat structures (X, ϕ) and (X, A^{−1}ϕ) are isomorphich. Therefore A ∈ Γ(ϕ). Since (X_{1}, ϕ_{1}) covers (X_{2}, ϕ_{2}), it follows that Γ_{0} ⊂ Γ(ϕ_{2}). Thus Γ_{0} ⊂ Γ(ϕ_{1}) ∩ Γ(ϕ_{2}) and has finite index in Γ(ϕ_{1}). To show that Γ_{0} has finite index in Γ(ϕ_{2}) as well one applies a similar argument to the set of covers of (X_{2}, ϕ_{2}). □

Corollary 5.15 ([28]). If a rational polygon P_{2} has no angles of the form π/n and P_{2} tiles P_{1} by reflections then the respective Veech groups are commensurable.

Hubert and Schmidt [28] have shown that the Veech group associated to the isosceles triangle with angles 2π/n, (n − 2)π/2n, (n − 2)π/2n is not a lattice even though the right triangle with angle π/n tiles it with one reflection. This is due to the appearance of removable singular points that we discussed above.

6 Miscellaneous results

6.1 Stable periodic trajectories

The material in this subsection is taken from the paper [18].

Definition 6.1 A periodic billiard trajectory in a polygon Q is called stable if an arbitrary small perturbation of the polygon Q leads to a perturbation of this trajectory but not to its destruction.

For example, a 2-periodic trajectory in a square, perpendicular to a side, is not stable: a deformation of the square to a quadrilateral without parallel sides destroys this trajectory. The Fagnano 3-periodic trajectory in an acute triangle, connecting the foot points of the altitudes, is stable, and the same holds for the 6-periodic trajectory which is the double of the Fagnano one – see figure 1 again.

What follows is a criterium for the stability of periodic trajectories. Label the consecutive sides of the polygon Q by 1, 2, ..., k. Without loss of generality, assume that the periodic trajectory has an even number of links 2n (otherwise, double it). Then the trajectory is encoded by the sequence i_{1}, ..., i_{2n} of the labels of the consecutively visited sides of Q.

Lemma 6.2 The trajectory is stable if and only if the labels in the sequence i_{1}, ..., i_{2n} can be partitioned into pairs of identical labels such that the label in each pair appears once at an odd position and once at an even one.

Proof. Denote by α_{i} the angle, made by i-th side of Q with a fixed direction. Unfold the periodic trajectory to a straight line l. Then the (2n)-th copy of Q along this line, denoted by Q_{2n}, is parallel to Q and, moreover, the parallel translation t that takes Q to Q_{2n} preserves l. The composition of two reflections is a rotation through the angle, twice that between the axes of reflection. It follows that Q_{2n} is obtained from Q by a rotation through the angle

2(α_{i_{1}} − α_{i_{2}} + α_{i_{3}} − ... + α_{i_{2n−1}} + α_{i_{2n}}),

and this angle is a multiple of 2π.

Thus for the trajectory to be stable it is necessary that the variation of this angle satisfies the equation

δα_{i_{1}} − δα_{i_{2}} + δα_{i_{3}} − ... + δα_{i_{2n−1}} + δα_{i_{2n}} = 0

for every perturbation of the polygon. An arbitrary variation of Q gives rise to an arbitrary variation of the directions of its sides. Hence the above relation holds only if its terms cancel pairwise, as claimed.

Conversely, assume that the condition of the lemma on the coding of a periodic trajectory is satisfied. Using the same notation as before, one has: t(l) = l. Let Q′ be a polygon, sufficiently close to Q, whose sides are labeled the same way as in Q. Reflect Q′ consecutively in its sides according to the sequence i_{1}, ..., i_{2n}. Then Q′_{2n} is parallel to Q′. Let t′ be the respective parallel translation; this translation is close to t. Choose a point x ∈ l ∩ Q ∩ Q′ and let l′ be the line through x, invariant under t′. If Q′ is sufficiently close to Q then l′ is the unfolding of a billiard trajectory in Q′ labeled i_{1}, ..., i_{2n}, and since t′(l′) = l′ this trajectory is (2n)-periodic. □

The lemma implies the next property of irrational billiards.

Corollary 6.3 If the angles of a billiard k-gon are maximally independent over the rationals, that is, the dimension of the linear space over Q, generated by the angles, equals k − 1, then every periodic billiard trajectory in this polygon is stable.

Proof. Consider a periodic billiard trajectory, labeled i_{1}, ..., i_{2n}. Then

α_{i_{1}} − α_{i_{2}} + α_{i_{3}} − ... + α_{i_{2n−1}} + α_{i_{2n}} = πm, m ∈ Z.

Rewrite this as

(α_{i_{1}} − α_{1}) − (α_{i_{2}} − α_{1}) + (α_{i_{3}} − α_{1}) − ... + (α_{i_{2n−1}} − α_{1}) + (α_{i_{2n}} − α_{1}) = πm.

If the labels do not satisfy the condition of the above lemma, that is, do not cancel pairwise, one obtains a non-trivial relation over Q on the angles α_{2} − α_{1}, α_{3} − α_{1}, ..., α_{k} − α_{1}. It remains to notice that the linear space over Q, generated by these angles, coincides with that, generated by the angles of the billiard polygon. □

6.2 Encoding billiard trajectories. Polygonal billiards have zero entropy

The encoding of billiard trajectories by the consecutively visited sides of the billiard polygon provides a link between billiard and symbolic dynamics. Let us consider this encoding in more detail. We follow the paper [17].

Assume that the billiard k-gon Q is simply connected (most of the results hold without this assumption), and its sides are labeled 1, ..., k. The phase space of the billiard transformation T consists of the inward unit vectors whose foot points are on the boundary ∂Q and whose forward orbits never hit a vertex of Q. Given a phase point x, denote by w(x) the infinite sequence of labels 1, ..., k that encodes the forward T -orbit of x. Given an infinite word w, denote by X(w) the set of phase points x with w(x) = w. Call a subset S ⊂ X(w) a strip if all vectors in S are parallel and their foot points constitute an interval on a side of Q. An open strip is defined analogously. The billiard transformation takes a strip to a strip, preserving its width.

Start with two simple observations. First, if w(x) = w(y) then the vectors x and y are parallel. If not, the unfolded trajectories of x and y linearly diverge, so a vertex of a copy of Q will fall into the angle between them. First time this happens the reflections of the two trajectories occur at different sides of the polygon, thus w(x) ≠ w(y).

Fig. 12

Secondly, if x and y are parallel vectors in X(w) then every parallel vector, whose foot point lies between those of x and y also belongs to X(w). This is true because Q is simply connected. It follows that X(w) is the maximal strip corresponding to the word w. The trajectories of its boundary points come arbitrarily close to vertices of Q.

The next theorem from [17] shows that the encoding is one-to-one on the set of non-periodic billiard trajectories.

Theorem 6.4 If w is a (2n)-periodic word then each vector from X(w) has a 2nperiodic trajectory and X(w) is an open strip. If w is aperiodic then the set X(w) consists of at most one point.

Proof. Start with the first claim. Consider x ∈ X(w) and unfold its trajectory to the line l. One claims that the (2n)-th copy of Q along this trajectory denoted, as before, by Q_{2n}, is parallel to Q. If not, x and T^{2n}(x) have their foot points on the same side of Q but they are not parallel. According to the remark, preceding the theorem, w(x) ≠ w(T^{2n}(x)) which contradicts the periodicity of w.

Let t be the parallel translation that takes Q to Q_{2n}. One wants to show that t(l) = l. Suppose not. Then, for a sufficiently great integer k, the polygon t^{k}(Q) does not intersect l. This is a contradiction because

this polygon is Q_{2nk}. Finally, the strip X(w) is open because a periodic trajectory stays a bounded distance from the vertices of the polygon.

Now consider the second claim. Assume, to the contrary, that the set X(w) is a maximal strip of non-zero width; call this strip S. Let x ∈ S be the vector whose base point lies exactly in the middle of S.

The proof is especially simple if Q is a rational polygon, and we start with this case. Since Q is rational, the trajectory of x has a finite set of directions. It follows that this trajectory will visit some side of Q infinitely often with some fixed direction. Call these points x_{i} = T^{n_{i}}(x) and let S_{i} = T^{n_{i}}(S). All these strips have the same width, therefore they must intersect. Suppose S_{i} and S_{j} intersect. Then the maximal width strips S̄_{i} and S̄_{j}, containing S_{i} and S_{j}, respectively, are also parallel and intersect. These maximal strips cannot coincide, otherwise w would be periodic. Therefore a part of the boundary of S̄_{i} lies inside S̄_{j} (or vice versa). Since the trajectories of the boundary points of the maximal strip S̄_{i} come arbitrarily close to vertices of Q, there are vertices inside S̄_{i}, a contradiction.

Consider a general polygon Q. One cannot claim anymore that the trajectory of x visits some side of Q infinitely often with some fixed direction. This claim will be replaced by a weaker recurrence result.

Let Y be the forward limit set of x under the mapping T. Since the width of S is positive, the trajectories of the points from Y stay bounded distance from the vertices of Q. Therefore the restriction of T to Y is continuous. Note that Y is closed and bounded, hence compact.

One uses the strengthened version of the Poincaré recurrence theorem, due to H. Furstenberg [15]. This theorem implies that there exists a uniformly recurrent point y ∈ Y. This means the following: for every neighborhood U of y there is a constant C such that the return times n_{i} > 0, defined by T^{n_{i}}(y) ∈ U, satisfy the inequalities n_{i+1} − n_{i} < C.

Fix a sufficiently small ϵ > 0 and consider the unfolded maximal strip S′ of y together with its ϵ neighborhood S′_{ϵ}. Since S′ is maximal, some vertices of copies of Q, unfolded along this strip, will fall into S′_{ϵ}. Claim: they fall with uniformly bounded gaps into each of the two components of S′_{ϵ} − S′.

This claim follows from the uniform recurrency of y. More specifically, let z be the leftmost point of S′. Then there is m > 0 such that the foot point of T^{m}(z) lies within ϵ/2 of a vertex v of Q; let m be the first such number. Let U be a neighborhood of y such that for every u ∈ U the m-th iterate T^{m}(u) is ϵ/2 close to T^{m}(y); such a neighborhood exists since T is continuous at y. Let n_{i} be the return times of y to U. Then T^{m+n_{i}}(y) is ϵ/2 close to T^{m}(y), and therefore T^{m+n_{i}}(z) is ϵ/2 close to T^{m}(z). It follows that the foot points of T^{m+n_{i}}(z) lie within ϵ from the vertex v. The claim is proved.

To finish the proof of the theorem recall that y is a forward limit of x: there is a sequence n_{i} → ∞ such that x_{i} = T^{n_{i}}(x) → y. The argument, given above in the case of a rational polygon Q, shows that no two vectors x_{i} and x_{j} are parallel. Consider the intersection of the unfolded maximal strip for x_{i} with S′_{ϵ} − S′. Let α_{i} be the angle between these strips of width ϵ and, say, δ. Then α_{i} → 0 as i → ∞. The intersection is a parallelogram which, by elementary geometry, contains a rectangle of width ϵ and length l_{i} = (δ − ϵ cos α_{i})/ sin α_{i}. Since the gaps between the vertices in S′_{ϵ} − S′ are uniformly bounded and l_{i} → ∞ as i → ∞, a vertex will eventually appear inside the unfolded maximal strip for x_{i}. This is a contradiction. □

As a consequence, the closure of every non-periodic billiard orbit contains a vertex of the billiard polygon.

An important corollary of the theorem concerns the entropy of polygonal billiards.

Corollary 6.5 The metric entropy of the billiard mapping with respect to any invariant measure is zero.

Proof. Morally, the argument goes as follows: the ”past” is uniquely determined by the ”future”, therefore the entropy vanishes.

More specifically, consider the set Σ of words in 1, ..., k that encode the billiard trajectories for time from −∞ to ∞, and let S be the shift transformation of Σ. Then the encoding map conjugates the billiard transformation and S. By the theorem, the shift S has a one-sided generator, the partition into k parts according to the value of the zero symbol in a word. This means that for this partition η the σ-algebra of measurable sets is generated by ∨_{i=0}^{∞}S^{−i}(η). It is a standard result in ergodic theory that the metric entropy of a transformation with a one-sided generator vanishes (see, e.g., [10]). □

The variational principle implies that the topological entropy vanishes as well. The zero entropy result was proved in [7] and [46] for the canonical billiard mea-

sure, and in [29], [17] and [22] for topological entropy. The latter work concerns a broader class of transformations, the so-called, polygon exchanges.

The zero entropy result implies that a number of quantities, associated with a polygonal billiard, grow slower than exponentially: the number of different words of length n in Σ, the number of strips of n-periodic trajectories or periodic trajectories of length not greater than L, etc. Conjecturally, all these numbers have a polynomial growth.

6.3 Complexity of billiard trajectories in rational polygons

Consider an aperiodic billiard trajectory in a rational polygon in a direction θ. According to Theorem 1.4 its code w is an aperiodic sequence. Generically this sequence enjoys a weaker periodicity property, called quasiperiodicity, described in the next result.

Proposition 6.6 For all but countably many directions θ each finite segment of w appears in w infinitely many times.

Proof. Let i_{1}, ..., i_{n} be a segment of w, and let x be a phase point on the trajectory under consideration whose first n successive reflections occur in the sides labeled i_{1}, ..., i_{n}. There is a neighborhood U of x such that for every phase point y ∈ U the first n reflections occur in the same sides of the polygon. By Theorem 1.8, for all but countably many directions θ, the trajectory is dense on the invariant surface. It follows that x returns to U infinitely many times, and each time the segment i_{1}, ..., i_{n} reappears in w. □

A convenient way to measure the complexity of aperiodic trajectories and their codes is provided by the next definition.

Definition 6.7 The complexity function p(n) of an infinite sequence w is the number of distinct n-element segments of w.

We start with the complexity of billiard trajectories in a square; the results are due to Hedlund and Morse [26]. We slightly modify the encoding using only two symbols, say, 0 and 1, to indicate that a trajectory reflects in a horizontal or a vertical side, respectively.

Theorem 6.8 For every aperiodic trajectory one has: p(n) = n + 1.

The sequences with complexity p(n) = n + 1 are called Sturmian.

Proof. Start with the following observation. Every aperiodic trajectory is dense in the respective invariant torus. Therefore p(n) can be computed as the number of different initial segments of length n in the codes w(x) where x ranges over the phase vectors having a fixed direction.

Unfold the billiard trajectory to a line l. Partition the square grid in the plane into ”ladders”, going in the south-east – north-west direction, as shown in figure 11. The n-th symbol in the code of the trajectory is 0 or 1, according as l meets a horizontal or a vertical segment of n-th ladder.

Fig. 13

Let (e_{1}, e_{2}) be the orthonormal frame. Consider the linear projection of the plane onto the diagonal x + y = 0 whose kernel is parallel to the line l. Factorize the diagonal by the translation through e_{1} − e_{2} and identify the quotient space with the unit circle S^{1}. The vertices of the first ladder are the lattice points (a, b) with a+b = 1 or a + b = 2. Since l has an irrational slope the projections of the vertices of the first ladder partition the circle into two irrational arcs. Let T be the rotation of S^{1} through the length of an arc, that is, through the projection of e_{1}.

The number of different initial n-segments corresponding to the lines, parallel to l, equals the number of segments into which the projections of the vertices of the first n ladders partition S^{1}. Each ladder is obtained from the first one by the translation through e_{1}. It follows that p(n) equals the cardinality of the orbit T^{i}(0), i = 0, ..., n. Since T is an irrational rotation of the unit circle all points of this orbit are distinct, and p(n) = n + 1. □

There are generalizations of the above theorem to multi-dimensional cubes – see [2] and [4].

Next we consider the complexity of billiard trajectories in convex rational polygons. The following result is due to P. Hubert ([27]).

Let Q be a convex rational k-gon with the angles πm_{i}/n_{i}, i = 1, ..., k (m_{i} and n_{i} are coprime), and let N be the least common multiple of n_{i}’s. Let θ be a direction such that there are no generalized diagonals in this direction. Consider a billiard trajectory in direction θ which avoids the vertices. Let w be its coding in the alphabet {1, ..., k}, and denote by p(n) the complexity of w.

Theorem 6.9 For all sufficiently great n one has: p(n) = n(k − 2)N + 2N.

In particular, if Q is a square, one obtains: p(n) = 4(n + 1). This does not contradict the previous theorem because the encoding there was different and, actually, 4 times less precise: the pair of parallel sides were labeled identically.

Proof. Consider the invariant surface M of the billiard flow. Recall that M is the result of pairwise identifying the sides of 2N copies of Q. Thus M has the structure of a complex; denote by v, e and f the number of vertices, edges and faces. Note that e = Nk and f = 2N. The edges can be labeled by pairs (i, j), i = 1, ..., k, j = 1, ..., N so that the (i, j)-th edge corresponds to the i-th side of Q. Let E ⊂ M be the union of the edges.

The billiard trajectory under consideration can be encoded in an obvious way in the alphabet {(i, j)}; denote its code by W. The projection (i, j) → i sends W to w (we will always denote by W words in the alphabet {(i, j)} and by w their projections). In other words, the new encoding takes into account not only the side of Q in which a reflection occurs but also the angle of reflection that can take finitely many values.

Let P (n) be the complexity of W.

Claim: P (n) = n(k − 2)N + 2N for all n ≥ 1.

Arguing as in the proof of the preceding theorem, P (n) equals the number of distinct initial segments of length n in the codes W (x) where x ranges over points of E.

Next, given a word W_{n} of length n, consider the set X(W_{n}) consisting of the points x ∈ E such that the initial segment of W (x) is W_{n}. Then, arguing as in the previous section, X(W_{n}) is an interval on an edge from E.

Denote by T the first return map of the directional flow F_{θ} to E (that is, the billiard map). Consider the set of initial segments of length (n + 1) in W (x), x ∈ X(W_{n}). The word W_{n} will have different successors if and only if T^{n}(X(W_{n})) contains a vertex V of M. We say that W_{n} splits at V.

To find P (n + 1) − P (n) one needs to learn how many words of length n split at each vertex. Let V be a vertex with cone angle 2πc. Then there are c incoming saddle connections. Trace each one back n steps, and let W^{1}, ..., W^{c} be the respective n-length words. Then T^{n}(X(W^{α})) contains V for each α = 1, ..., c.

The words W^{α} are all different since their (n−1)-st letters are distinct. The latter hold because in the polygon Q there is only one trajectory that starts at a given side in a given direction and goes straight to a given vertex.

Thus all the n-length words that split at V are W^{1}, ..., W^{c}. It follows that

P (n + 1) − P (n) = ∑ c(V),

sum over all vertices. To evaluate this sum note that the Euler characteristic of M equals the sum of indices of singular points of the flow F_{θ}; the index at vertex V equals 1 − c(V). Hence

χ(M) = ∑(1 − c(V)) = v − ∑ c(V).

On the other hand, χ(M) = v − e + f, and it follows that ∑ c(V) = (k − 2)N. Therefore P (n + 1) − P (n) = (k − 2)N, and since P (1) = kN, the claim follows.

To finish the proof of the theorem it remains to show that P (n) = p(n) for all sufficiently great n. Suppose not; then there exist arbitrary long words W ≠ W′ that are the initial segments of the codes of phase points x ∈ E such that w = w′.

The billiard trajectories, corresponding to W and W′, start at the same side of Q but make different angles with it: otherwise, since w = w′, they would reflect at the same sides of Q and meet them at the same angles which would imply that W = W′. Note that the angle between the two trajectories is bounded below by a constant α depending on Q and θ.

Unfold the trajectories to straight lines. Since these lines linearly diverge, there is a constant m, depending on Q and α, such that after at most m reflections of Q along either of the unfolded trajectories a vertex of a copy of Q will fall into the angle between the lines. Therefore if the length of W and W′ is greater than m the codes w and w′ are distinct, a contradiction. □

If Q is a general k-gon let N be the least common denominator of its π-rational angles and s be the number of its distinct π-irrational angles. Then one has the next upper bound for the complexity p_{θ}(n) of the billiard trajectories in Q in a fixed initial direction θ – see [25].

Theorem 6.10 For all n one has:

p_{θ}(n) ≤ kNn(1 + \frac{n}{2})^{s}.

Using techniques, similar to the ones in the proof of Theorem 6.9, S. Troubetzkoy obtained in [50] a complexity lower bound for arbitrary polygonal billiards. Unlike the previous results, his theorem concerns billiard trajectories in all directions, so that p(n) now means the number of distinct words of length n in the coding in the alphabet {1, ..., k} of all billiard orbits in a k-gon.

Theorem 6.11 For every polygon there exists a constant c such that p(n) ≥ cn^{2} for all n ≥ 0.

A similar estimate is proved in [50] for d-dimensional polyhedra with the exponent 2 replaced by d.

6.4 Periodic trajectories in some irrational billiards

The following elementary argument shows that every rational polygon has a periodic billiard trajectory of a special kind; it was found independently by A. Stepin ([18]) and by M. Boshernitzan ([8]).

Shoot the billiard ball in the direction, perpendicular to a side of the polygon Q. By Poincaré’s recurrence theorem, for almost every initial position, the ball will return to the original side at an angle, arbitrarily close to π/2. Since the set of possible directions of the ball is discrete in a rational polygon, this angle will be equal to π/2. After the ball bounces off of the side it will backtrack the same trajectory, so the trajectory is periodic.

Surprisingly, a variation of this argument applies to some irrational polygons. This was observed by B. Cipra, R. Hanson and A. Kolan ([11]), and then generalized by E. Gutkin and S. Troubetzkoy ([25]). The next result is due to the former authors.

Theorem 6.12 Given a right triangle with π-irrational acute angles, almost every (in the sense of measure) billiard trajectory, that starts at a side of the right angle in the direction, perpendicular to this side, returns to the same side in the same direction.

Proof. The idea is to apply the construction which gave the invariant surfaces of the billiard flow in rational polygons. A significant difference with the rational case is that the invariant surface will not be compact.

Start with reflecting the triangle in the sides of the right angle to obtain a rhombus R. The study of the billiard inside the triangle reduces to that inside the rhombus. Let α be the acute angle of R.

Fig. 14

Consider the beam of horizontal trajectories, starting at the upper half of the vertical diagonal of the rhombus, and construct the invariant surface of the phase space, corresponding to this beam. This surface consists of rhombi, obtained from R by the action of the group A(R), whose sides are pasted pairwise in an appropriate manner. Since α is π-irrational, this surface is not compact. The invariant surface is partially foliated by the parallel trajectories from the horizontal beam, and this foliation has an invariant transversal measure, ”the width of a beam”.

Each rhombus involved is obtained from the original one by a rotation through the angle nα where n ∈ Z. Such a rhombus will be referred to as the n-th rhombus and denoted by R_{n}; in particular, R = R_{0}. A trajectory from the beam under consideration may leave the n-th rhombus through a side which has one of the two possible directions. Call such a side positive if the trajectory enters an (n + 1)-st rhombus and negative if it enters an (n − 1)-st one.

One wants to show that almost every trajectory returns to R_{0} (where they hit the vertical diagonal in the perpendicular direction). We will prove that for every ϵ > 0 the relative measure of the set of trajectories that do not return to R_{0} is less than ϵ.

Let ϵ be given. Since α is π-irrational, there exists n > 0 such that the vertical projection of the positive side of R_{n} is less than ϵ. This implies that the relative measure of the set of trajectories that make it to (n + 1)-st rhombi is less than ϵ.

The rest of trajectories are bound to stay in R_{0}, R_{1}, ..., R_{n}; call this set of trajectories S. The union of the rhombi from 0 through n is compact, therefore the Poincaré recurrence theorem applies to S. It follows that almost every trajectory in S is recurrent, that is, returns to R_{0}. □

In fact, the above proof gives more: for every direction in a rhombus almost every billiard trajectory in this direction returns arbitrarily close to itself and in the same direction. A relevant definition is proposed in [25].

Definition 6.13 Given a billiard polygon Q, a direction is called recurrent if almost every billiard trajectory in this direction returns arbitrarily close to itself and in the same direction. A polygon Q is called strongly recurrent if every direction is recurrent.

In particular, every rational polygon is strongly recurrent. An argument, similar to the preceding proof, gives the next result from [25], applicable, in particular, to all parallelograms.

Theorem 6.14 Let Q be a polygon whose sides have one of the two fixed directions. Then Q is strongly recurrent. In particular, the billiard orbits, perpendicular to each side of Q, are periodic with probability one.

6.5 A non-periodic trajectory that is not dense in the configuration space

An example of such a trajectory was constructed by G. Galperin in [16].

Fig. 15

Consider a centrally symmetric hexagon (so that its opposite sides are parallel and congruent) such that the bisector of the angle BAC = α is perpendicular to the side AF and the the bisector of the angle ABF = β is perpendicular to the side BC.

Since the bisector of α is perpendicular to BC, the reflection of AB in BC is the line BF. Thus the reflection of any vertical segment leading to BC is parallel to BF. Similarly, the reflection of vertical segments in CD are parallel to AC. After another reflection the segment becomes vertical again. Choose a line, perpendicular to the side AB; let X, Y, Z be the projections on this line of the points A, E, F, respectively. The set of vertical lines is identified with a horizontal segment XY, and the second iteration of the billiard transformation induces the exchange of the intervals XZ and ZY. An exchange of two intervals is equivalent to a circle rotation. For generic angles α and β each orbit of this circle rotation is dense. Therefore each vertical billiard trajectory is dense in the hexagon.

Extend the non-vertical sides of the hexagon to obtain a parallelogram. Then the two triangles, added to the hexagon, are never visited by any vertical trajectory, that starts inside the original hexagon.

Similar but more tedious considerations prove the following result by Galperin.

Theorem 6.15 There exists an acute angle α_{0} such that for almost every (in the sense of measure) α ∈ (0, α_{0}) a right triangle with the acute angle α contains a non-periodic and not everywhere dense billiard trajectory.

References

[1] P. Arnoux. Ergodicité Générique des Billards Polygonaux. Sém. Bourbaki, No 696, (1987-88).

[2] P. Arnoux, C. Mauduit, I. Shiokawa, J.-I. Tamura. Complexity of sequences defined by billiards in the cube. Bull. SMF, 122, F.1, (1994), 1-12.

[3] E. Aurell, C. Itzykson. Rational billiards and algebraic curves. J. Geom. and Phys., 5, (1988), 191-208.

[4] Yu. Baryshnikov. Complexity of trajectories in rectangular billiards. Comm. Math. Phys., 174, (1995), 43-56.

[5] A. Beardon. The geometry of discrete groups. Springer-Verlag, 1983.

[6] L. Bers. Finite dimensional Teichmuller spaces and generalizations, Bull. AMS, 5, (1981), 131-172.

[7] C. Boldrighini, M. Keane, F. Marchetti. Billiards in polygons. Ann. of Prob., 6, (1978), 532-540.

[8] M. Boshernitzan. Billiards and rational periodic directions in polygons. Amer. Math. Monthly, 99, (1992), 522-529.

[9] M. Boshernitzan, G. Galperin, T. Kruger, S. Troubetzkoy. Periodic billiard orbits are dense in rational polygons. Trans. AMS, 350 (1998), 3523-3535.

[10] I. Cornfeld, S. Fomin, Ya. Sinai. Ergodic theory. Springer-Verlag, 1982.

[11] B. Cipra, R. Hanson, A. Kolan. Periodic trajectories in right triangle billiards. Phys. Rev., E 52, (1995), 2066-2071.

[12] C. Earle, F. Gardiner. Teichmuller disks and Veech’s F-structures. Contemp. Math., 201, AMS, Providence, 1997, 165-189.

[13] A. Eskin, H.Masur. Pointwise asymptotic formulas on flat surfaces. Preprint [14] R. Fox, R. Kershner. Geodesics on a rational polyhedron. Duke Math. J., 2,

(1936), 147-150.

[15] H. Furstenberg. Recurrence in Ergodic Theory and Combinatorial Number Theory. Princeton Univ. Press, 1981.

[16] G. Galperin. Nonperiodic and not everywhere dense billiard trajectories in convex polygons and polyhedrons. Comm. Math. Phys., 91, (1983), 187-211.

[17] G. Galperin, T. Kruger, S. Troubetzkoy. Local instability of orbits in polygonal and polyhedral billiards. Comm. Math. Phys., 169, (1995), 463-473.

[18] G. Galperin, A. Stepin, Ya. Vorobets. Periodic billiard trajectories in polygons: generating mechanisms. Russ. Math. Surv., 47, No 3, (1992), 5-80.

[19] E. Gutkin. Billiards in polygons. Physica D, 19, (1986), 311-333.

[20] E. Gutkin. Billiards in polygons: survey of recent results. J. Stat. Phys., 83, (1996), 7-26.

[21] E. Gutkin. Billiards on almost integrable polyhedral surfaces. Ergod. Th. and Dyn. Syst., 4, (1984), 569-584.

[22] E. Gutkin, N. Haydn. Topological entropy of generalized polygon exchanges. Bull. AMS, 32, (1995), 50-57.

[23] E.Gutkin, C.Judge. The geometry and arithemtic of translation surfaces with applications to polygonal billiards. Math. Res. Lett., 3, (1996), 391-403.

[24] E.Gutkin, C.Judge. Affine mappings of translation surfaces: geometry and arithmetic. Preprint

[25] E. Gutkin, S. Troubetzkoy. Directional flows and strong recurrence for polygonal billiards. in Pitman Res. Notes Math. Ser., 362, 1996.

[26] G. Hedlund, M. Morse. Symbolic dynamics 2. Sturmian trajectories. Amer. J. of Math., 62, (1940), 1-42.

[27] P. Hubert. Complexité de suites définies par des billards rationnels. Bull. Soc. Math. France, 123, (1995), 257-270.

[28] P.Hubert, T.Schmidt, Veech groups and polygonal coverings. Preprint

[29] A. Katok. The growth rate for the number of singular and periodic orbits for a polygonal billiard. Comm. Math. Phys., 111, (1987), 151-160.

[30] A. Katok, B. Hasselblatt. Introduction to the modern theory of dynamical systems. Cambridge Univ. Press, 1995.

[31] A. Katok, A. Zemlyakov. Topological transitivity of billiards in polygons. Math. Notes, 18, (1975), 760-764.

[32] M. Kean. Coding problems in ergodic theory. Proc. Int. School of Math. Phys., Univ. Camerino, 1974.

[33] R. Kenyon, J. Smillie. Billiards in rational-angled triangles. Preprint

[34] S. Kerckhoff, H. Masur, J. Smillie. Ergodicity of Billiard Flows and Quadratic Differentials. Ann. of Math., 124, (1986), 293-311.

[35] M. Kontsevich, A. Zorich. Connected components of the moduli spaces of holomorphic differentials with prescribed singularities. Preprint

[36] I.Kra. On the Nielsen-Thurston-Bers type of some self-maps of Riemann surfaces. Acta. Math., 146, (1981), 231-270.

[37] H.Masur. Interval exchange transformations and measured foliations. Ann. of Math., 115, (1982), 169-200.

[38] H. Masur, Closed trajectories for quadratic differentiasl with an application to billiards. Duke Math. J., 53, (1986), 307-314.

[39] H. Masur. Lower bounds for the number of saddle connections and closed trajectories of a quadratic differential. In Holomorphic Functions and Moduli, Vol.1, D. Drasin ed., Springer-Verlag, 1988.

[40] H.Masur. The growth rate of trajectories of a quadratic differential. Erg. Th. Dyn. Syst., 10, (1990), 151-176.

[41] H. Masur. Hausdorff dimension of the set of nonergodic foliations of a quadratic differential. Duke Math. J., 66, (1992), 387-442.

[42] H. Masur, J.Smillie. Hausdorff dimension of sets of nonergodic foliations, Ann. of Math., 134, (1991), 455-543

[43] H. Masur, J. Smillie. Quadratic differential with prescribed singularities and pseudo-Anosov diffeomorphisms. Comm.. Math. Helv., 68, (1993), 289-307.

[44] J.C.Oxtoby, S.M.Ulam. Measure preserving homeomorphisms and metrical transitivity, Ann. of Math., 42, (1941), 874-920.

[45] R. Richens, M. Berry. Pseudointegrable systems in classical and quantum mechanics. Physica D, 2, (1981), 495-512.

[46] Ya. Sinai. Introduction to ergodic theory. Princeton Univ. Press, 1976.

[47] K. Strebel. Quadratic differentials. Springer-Verlag, 1984.

[48] S. Tabachnikov. Billiards. SMF Panoramas et Synthèses, 1995, No 1.

[49] W.P. Thurston. On geometry and dynamics of diffeomorphisms of surface. Bull Amer. Math. Soc., 18, (1988), 417-431.

[50] S. Troubetzkoy. Complexity lower bounds for polygonal billiards. Chaos, 8, (1998), 242-244.

[51] W.Veech. Strict ergodicity in zero dimensional dynamical systems and the Kronecker-Weyl theorem mod 2. Trans. AMS, 140, (1969), 1-34.

[52] W.Veech. Teichmuller geodesic flow. Ann. of Math., 124, (1986), 441-530.

[53] W. Veech. Moduli spaces of quadratic differentials. J. D’Analyse Math., 55, (1990), 117-171.

[54] W.Veech. Teichmuller curves in moduli space. Eisenstein series and and an application to triangular billiards. Invent. Math., 97, (1990), 117-171.

[55] W.Veech The billiard in a regular polygon, Geom.Funct.Anal. 2 (1992) 341-379 [56] Ya. Vorobets. Ergodicity of billiards in polygons. Mat. Sb., 188, (1997), No 3,

65-112.

[57] Ya. Vorobets. Plane structures and billiards in rational polygons: the Veech alternative. Russ. Math. Surv., 51, (1996), No 5, 3-42.

[58] P. Walters. An Introduction to Ergodic Theory. Springer-Verlag, 1982.

[59] C. Ward. Calculation of Fuchsian groups associated to billiards in a rational triangle. Ergod. Th. and Dyn. Syst., 18, (1998), 1019-1042.