AN ANALYTIC SEPARATION OF SERIES

OF REPRESENTATIONS FOR SL(2; ℝ)

Simon Gindikin

Abstract. For the group SL(2; ℝ) holomorphic wave fronts of the projections on different series of representations are contained in some disjoint cones. These cones are convex for holomorphic and antiholomorphic series, and this corresponds to the known fact that these projections can be extended holomorphicaly in some Stein tubes in SL(2; ℂ) [GG]. For the continuous series this cone is not convex and the projections are boundary values of 1-dimensional ¯∂-cohomology in a non-Stein tube.

The central circumstance in the harmonic analysis on pseudo Riemann symmetric spaces, including real semisimple Lie groups, is the existence of several series of representations. In the Plancherel formula the decomposition on series is the crucial step, since the decompositions on irreducible representations inside series reduces to the commutative Mellin transform for corresponding Cartan subgroups. In my opinion, the analytic nature of the decomposition on series needs to be clarified. Of course, from the algebraic point of view, different series correspond to different classes of Cartan subgroups. From the point of view of functional analysis, different series correspond to different types of spectrum but spectral difference of different discrete series is already not very instructive.

I believe that it is natural to separate series by using complex analysis. In a sense, the decomposition of L^{2} on a symmetric space X on components corresponding to different series, is a multidimensional version of the decomposition of L^{2} on the line on Hardy spaces of holomorphic functions in upper and lower half-planes. Specifically, holomorphic wave fronts of functions from different series lie in some disjoint cones. Such a cone is convex only for holomorphic discrete series. In these cases functions from the series holomorpicaly extend in some tubes in comlexified symmetric spaces X_{ℂ} with the wedge X. This phenomenon was noted in our joint paper with Gelfand [GG], and it was investigated in complete generality by Olshansky, Stanton, Hilgert, Ólafsson etc. (cf.[HO]). For other series these cones are nonconvex and corresponding functions can be characterized as boundary values of higher ¯∂-cohomology in non-Stein tubes with the wedge X. The flat version of such decompositions was considered in [G1,G2]. Let us emphasize that the connection of discrete series with complex analysis is quite customary, but this ideology assumes strong connections with complex analysis of all series, including maximally

Typeset by AMS-TEX

Key words and phrases. integral geometry, horospherical transform, series of representations, ¯∂ -cohomology, holomorphic wave front.

Partly supported by NSF-grant DMS-0070816. The author thanks the Max-Planck-Institut für Mathematik at Bonn for their hospitality.

continuous ones. Moreover, this method is hopefully strong enough to separate multiplicities when we have non multiplicity free decomposition in the Plancherel formula. Possibilities to separate multiplicities by intrinsic tools of representations theory look quite problematic. We can not yet justify this picture in general case. Since these facts are new and non trivial already for SL(2; ℝ) we decided that it makes sense to explain the picture in this simplest case. The basic tool in this paper is the integral geometry on SL(2; ℝ) developed in [G3]. Let us recall that the basic idea of this a construction is the replacement of real horospheres in the usual horospherical transform by complex horospheres without real points and the integration along real horospheres of Cauchy type integrals.

The universal separation of series in the language of complex analysis is one of demonstrations of the fundamental multidimensional phenomenon of the extension of real functions in the complex domain not only as holomorphic functions, but also as higher cohomology. Of course, the conception of hyperfunctions is the most important example, but it is not unique.

Real and complex geometries associated with SL(2; ℝ). All details of this and next sections can be found in [G3]. We will consider the group G = SL(2; ℝ) as the hyperboloid of signature (2, 2) in ℝ^{4}:

(1)  □(x) = x_{1}x_{4} − x_{2}x_{3} = 1,  x = ( \begin{matrix}x_{1} & x_{2} \\ x_{3} & x_{4} \end{matrix} ) ∈ ℝ_{x}^{4}.

Correspondingly, we will consider the complex group G_{ℂ} = SL(2; ℂ) as the complex hyperboloid □(z) = 1, z = x + iy ∈ ℂ.

Let us consider the bilinear form x · y corresponding to the quadratic form □x. Then on G_{ℂ} we have

(2)  □x − □y = 1,  x · y = 0.

On G acts the group G×G by left and right translations and this group is locally isomorphic to SO(2, 2). In our considerations 3 domains in G_{ℂ}\G play a central role. Firstly, we take 2 connected components G_{±} of the set of z = x + iy ∈ G_{ℂ} where □y > 0. Then we take the domain G_{0} where □y < 0, □x > 0. All these domains have G as their wedges (Shilov boundary) and the domains G_{±} are Stein. They are all invariant relative to G × G (but, of course, are non homogeneous). We will identify dual spaces using our bilinear form with spaces ℝ_{ξ}^{4}, ℂ_{ζ}^{4}, ζ = ξ + iη, and we will consider cones

(3)  Ξ = {□ξ = 0},  Ξ_{ℂ} = {□ζ = 0}.

Let us call by horospheres sections L(ζ, p) of the hyperboloid G_{ℂ} by the isotropic hyperplanes

ζ · z = p, ζ ∈ Ξ_{ℂ}.

We are interested here only by horospheres which have no real points (do not intersect G). Between them they are the horospheres with real ζ and complex p:

(4)  ξ · z = p,  p ∈ ℂ\ℝ.

Let Ξ̃_{0} be their set. Then we have 2 connected components Ξ̃_{±} of the set

(5)  □ξ = □η > p^{2}.

We pick up the signs in the indexes in such a way that

(6)  L(ζ, p) ⋂ G_{∓} = ∅

if L(ζ, p) ∈ Ξ̃_{±}. Moreover, the union of horospheres from Ξ̃_{±} coincides with the supplements G^{±} to the closure of G_{∓}. The pairs (ζ, p) give a homogeneous parameterization of the horospheres. If p ≠ 0, then we always can take p = 1 and parameterize such horospheres by points of Ξ_{ℂ}. We denote through Ξ_{±}, Ξ_{0} the domains in Ξ_{ℂ} corresponding to the families of horospheres Ξ̃_{±}, Ξ̃_{0}. Outside of these classes there is a set of horospheres without real points, dependent on less parameters and not participating in the results of this paper [G3].

Horospheres corresponding to boundary points of Ξ̃_{±}:

(7)  ζ · z = 1, z = ξ + iη, □ξ = □η = 1,

have the unique intersection points with G : x = ξ. On the boundary of Ξ̂_{0} lie horospheres with real (ζ = ξ, p) which intersect G on real horospheres. Along with horospheres we also consider horocycles - isotropic lines on the hyperboloid G_{ℂ}. They have the form

(8)  z = a + tζ,  □ζ = 0,  a · ζ = 0.

Let us denote this horocycle as l(ζ, a); t here is the parameter along the horocycle; a is defined up to a multiple of ζ. The union of horocycles with fixed ζ coincides with the horosphere L(ζ, 0). The horospheres L(ζ, p) with p ≠ 0 are fibering on the horocycles l(λ, a), where λ · ζ = 0, λ ≠ cζ, ζ · a = p. We will use for a dual parameterization of the horocycles their special representation as the intersection of horospheres:

(9)  l(λ, ν; p) = L(λ, 0) ∩ L(ν, p),  λ · ν = 0, p ≠ 0.

It is the representation of the horocycle l(λ, a), ν · a = p. The vector ν ∈ Ξ in (9) modulus λ has 2 choices ν′, ν′′ up to multiplicative factors. Then all horospheres containing the horocycle l(λ, ν; p) have either the form L(c_{1}λ + c_{2}ν′) or the form L(c_{1}λ + c_{2}ν′′).

The tube G_{0} is union of some horocycles with real ζ = ξ, and its closure Ḡ_{0} coincides with union of all such horocycles. Correspondingly, Ḡ_{0} is the union of the horospheres L(ξ, 0) with real ξ ∈ Ξ. The union of all horocycles l(λ, a) ⊂ G_{0} with λ ∈ Ξ has 2 connected components L^{±}(λ) - half-horospheres. Using the parameters in (9) we can define them as

L^{±}(λ) = {z ∈ L(λ, 0); ±ℑ(ν · z) > 0}.

Only the choice of sign in L^{±} depends of the choice of ν. The tube G_{0} is the disjoint union of the semi-horospheres L^{±}(λ), λ ∈ Ξ.

Integral geometry on G = SL(2; ℝ). Integral geometry on the hyperboloid G is based on the next decomposition of δ-function on G on planar waves. We consider for a function f on G the differential form

(10) κf[x] = \frac{f(x)}{\{ζ · (u − x − i0)\}^{3}} μ(u, du) ∧ [u + x, ζ, dζ, dζ],  □x = □u = 1,

for fixed x ∈ G, where

μ(u, du) = d(□u)⌋du

is the invariant 3-form on G and [a_{1}, a_{2}, a_{3}, a_{4}] denotes the determinant of the matrix with the columns a_{j} in which matrix coefficients - forms multiply by the exterior way. The difference from the classical case of the decomposition on planar waves in ℝ^{n} is that we have here an overdeterminate situation: it is a 5-form on a 6-dimensional manifold. The crucial point is that the form κf[x] is closed and, if f ∈ C_{0}^{∞}(G), then by the integration of restrictions of this form on appropriate cycles over the support of f we obtain different reconstructions of f(x) (different decompositions of δ-function on G on planar waves). By cycles herwe mean relative cycles: projections of their boundaries on G do not intersect the support of f. We consider here only functions with compact supports and then, using the invariance, it is possible to extend formulas on different functional spaces. In our constructions it is important that it is possible to take complex ζ in (9) in spite of we consider the real hyperboloid G . This possibility is crucial for the construction of horospherical transform on SL(2; ℝ). Following [G3] let us take in the plane x · λ = 0 any cycle Λ(x) which intersects each ray from 0 in one point (e.g. a sphere) and take in ℂ_{ζ}^{4} the cycle

γ(x) = \{ζ = \sqrt{□λ}x + λ, λ ∈ Λ(x)\},

where we take either the positive evaluation of the root if it is real or ia, a < 0, if it is imaginary. We have □ζ = 0 on these cycles. We integrate κf[x] on the cycle

(11)  Γ^{(1)} = {supp(f) × γ(x)}.

The form κf[x] will be regular on this cycle, since, as can be checked directly, horospheres L(ζ, ζ · x + iε), ε > 0, (x is fixed) have no real points: they will be in Ξ̃_{0} for □λ > 0 and in Ξ̃_{±} if □λ < 0. As the result,

(12)  \frac{1}{4(πi)^{3}} ∫_{Γ^{(1)}} κf[x] = f(x).

Let us remind that this inversion formula was proved in [G3] by a homotopy of the cycle Γ^{(1)} to the real cycle corresponding to the inversion formula for the projective Radon transform in ℝP^{3}.

This formula gives the inversion of horospherical transform on G. If the horosphere L(ζ, p) has no real points, let us define the (complex) horospherical transform

(13)  f̂(ζ, p) = ∫_{G} \frac{f(u)}{ζ · u − p}μ(du),  ℑp ≠ 0.

By the integration on u in the inversion formula (12) this formula can be transformed in

(14)  f(x) = \frac{1}{4(πi)^{3}} ∫_{Λ(x)} Lf̂(\sqrt{□λ}x + λ, \sqrt{□λ} + i0)[λ, dλ, dλ],

where

L = \frac{1}{p} \frac{∂}{∂p} + \frac{∂^{2}}{∂p^{2}} .

We integrate in (14) along the set of horospheres passing through the point x and take boundary values of the horospherical transform corresponding to shifts of these horospheres to horospheres without real points. Cycles γ(x) were picked so that such shifts would exist. In the formula [λ, dλ, dλ] we take the determinant of order 3 where λ is represented by some coordinates in the plane x · ξ = 0 (the integrand does not depend on the choice of coordinates).

In this way we consider the horospherical transform (13) f ↦ f̂ and its inversion (14) f = Sf̂. It is natural to decompose the horospherical transform f̂ on parts f̂_{±}, f̂_{0}, corresponding to Ξ̃_{±}, Ξ̃_{0}. It is shown in [G3] that f̂_{±}, f̂_{0} decompose on representations of correspondingly holomorphic, antiholomorphic, and continuous series. Let us decompose the cycle γ(x) on parts γ_{±}(x), γ_{0}(x) on the boundaries of Ξ̃_{±}, Ξ̃_{0}. They correspond to the dividing of the cycle Λ(x) on parts Λ_{±}(x), Λ_{0}(x); on Λ_{±}(x) we have □λ < 0 and on Λ_{0}(x) there is □λ > 0. If we will integrate in (14) on there parts Λ_{±}(x), Λ_{0}(x) we will obtain projections f_{±}, f_{0} of f on corresponding holomorphic, antiholomorphic, and continuous series: they express correspondingly through f̂_{±}, f̂_{0}. The operators of the projections S_{±}, S_{0} are the compositions of the horospherical transforms f ↦ f̂_{±}, f̂_{0} and the inverse horospherical transform (14) where the integration is taken only on the corresponding part Λ_{±}(x), Λ_{0}(x). Our aim is to investigate the analytic nature of f_{±}, f_{0} by using the integral representations of these operators following from (13), (14).

Holomorphic wave fronts of projections on series. The description of holomorphic wave fronts of projections on series is a direct corollary of the inversion formula (14). Let us remind that for a distribution F on a manifold G a covector a ∈ T_{x}^{∗}(G)\0 is not contained in the holomorphic wave front of F in the point x, if locally in a neighborhood of x it is possible to represent F as a sum of boundary values of holomorphic functions in some tubes inside of the half-space {ℑ(a · z) < 0}. For us it is sufficient to recall that, as directly follows from the definition, if F locally extends holomorphicaly in a half-space {ℑ(a · z) > 0}, then the holomorphic front can not be more than the ray generated by a. A more general fact is that if F locally extends holomorphicaly in a tube with a convex cone V, then its holomorphic wave front is contained in the dual cone V^{∗}.

For the computation of the wave fronts of f_{±}, f_{0} let us investigate the decomposition (14) in a neighborhood of a point x ∈ G. We have here the superposition of horospherical waves Lf̂(ζ, ζ · x), ζ ∈ γ(x). The cycles here depend on x but locally this dependence is not essential and we can investigate

(15)  ∫_{γ(x)} Lf̂(ζ, ζ · w)[λ, dλ, dλ]

when w is a neighborhood of x.

If ζ ∈ Ξ_{±}, then the function f̂(ζ, ζ · z) will be holomorphic in the domain on G_{ℂ} defined by conditions

(16)  G^{±}(ζ) = {z ∈ G_{ℂ}; |ζ · z|^{2} < □ξ,  ξ = ℜζ}.

Let us call these domains the horospherical cylinders: they are unions of horospheres without real points from Ξ̃_{±} with given ζ. We have

(17)  G^{±}(ζ) = G^{±}(λζ),  |λ| = 1.

Of course, this definition makes the sense only for ζ with □ξ > 0. We can suppose in the definition of horospherical cylinders that □ξ = 1 and then G_{±}(ζ) = {|ζ·w| < 1}; for such ζ only one of cylinders G^{±}(ζ) is nonempty. Their boundary intersects G on the circles {x = aξ + bη, a^{2} + b^{2} = 1}, ζ = ξ + η. Horospherical cylinders are Stein and G^{±}(ζ) give the coverings of G^{±} correspondingly. For real ζ = ξ ∈ Ξ functions f̂(ξ, ξ · z) are holomorphic in half-spaces {z ∈ G_{ℂ}; ℑ(ξ · x) > 0}. Of course, all these holomorphy properties are preserved under the action of the differential operator L.

Let us localize the picture in a neighborhood of x ∈ G on the tangent plane x · u = 1. We take u = x + v, x · v = 0 and we use v as coordinates. We will also identify T_{x}^{∗}(G) with the plane x·λ = 0. Let us remind that just such λ parameterize the points of the cycle γ(x). The described above holomorphy domains locally, on the tangent space, correspond just to half-spaces ℑ(λ · v) > 0 for all ζ ∈ γ(x). It means that the holomorphic wave front of Lf̂(ζ, ζ · w), ζ = \sqrt{□}λx + λ, reduces to cλ, c > 0. It immediately gives estimates of the wave fronts of the results of integration on γ_{±}(x).γ_{0}(x). Namely, in the cotangent plane x · λ = 0 we consider 3 cones: 2 connected components V_{±}(x) of the set □λ < 0 and the set V_{0}(x) where □λ > 0. We pick up indexes ± such that for λ ∈ V_{±}(x) the horospheres

L(\sqrt{□λ}x + λ, \sqrt{□λ} + i0) ∈ Ξ̃_{±}.

We obtain

Proposition 1. Holomorphic wave fronts of projections on series f_{±}, f_{0} of f ∈ C_{0}^{∞} are contained correspondingly in the cones V_{±}(x), V_{0}(x).

This proposition demonstrates the dramatic difference of projections on series from point of view of local complex analysis, but we will see that this difference extends to a global complex analytic picture.

Cohomological extensions of projections on discrete series. Since the cones V_{±}(x) are convex, it is natural to expect that f_{±} admit holomorphic extensions in some Stein tubes. Moreover, it was proved in [GG] that they indeed admit holomorphic extensions in G_{±}, but this result was obtained as a consequence of representations of S_{±} through characters and Plancherel coefficients for discrete series and the holomorphic extensions of explicit formulas for these characters. Here we obtain the holomorphic extensions directly from the inversion formulas.

In flat case the holomorphicity in tube domains is connected with decompositions on planar waves [G2]. Its horospherical analogue (12) does not give an extension directly, but it admits another complex analytical interpretation - as cohomological extension of a higher dimension. Let us consider the integrand in (15)

(18)  σ[f̂](w|ζ, dζ) = cLf̂(ζ, ζ · w)[w, ζ, dζ, dζ],  c = \frac{1}{4(πi)^{3}} ,

as (closed) 2-form on ζ ∈ Ξ_{±} holomorphicaly depending on parameters w ∈ G^{±}(ζ). Following the conception of Čech cohomology for smoothly parameterized Stein cover G^{±}(ζ) [G1,G2], we can connect with such closed forms ϕ(w|ζ, dζ) with parameters some 2-dimensional analytic cohomology class on G^{±}. It follows from [BEG] that these cohomology are isomorphic to the Dolbeault cohomology. A morphism in the Dolbeault cohomology can be constructed very explicitly. Let Z(w) be a smooth function on G^{±} such that

w ∈ G^{±}(Z(w)).

Then (0,2)-part of the form ϕ(w|ζ, dζ)|_{ζ=Z(w)} is ¯∂-closed and defines the Dolbeault cohomology class. Apparently, such functions exist, since sets F (w) = {ζ; w ∈ G^{±}(ζ)}, w ∈ G^{±}, are contractible.

The domains G^{±} are tubes with the edge G. It is possible to define boundary values of 2-dimensional cohomology H^{2}(G^{±}, O) on G. On the language of forms with parameters we take boundary values of ϕ on w on boundaries of horospherical cylinders G^{±}(ζ) (which all include G) and define

b_{±}ϕ(x) = ∫_{γ_{±}(x)} ϕ(x|ζ, dζ).

For general ϕ we can define the boundary values as hyperfunctions on G, but in our case ϕ = σ[f̂] we can use classical boundary values. Then projections f_{±} coincide with the boundary values of σ[f̂]:

(19)  f_{±} = b_{±}σ[f̂].

Thus we have:

Proposition 2. The formula (18) gives the canonical extensions of f_{±} as elements of H^{(2)}(G^{±}, O).

Holomorphic extensions of projections on discrete series. It is natural to expect that H^{(2)}(G^{±}) ∼ H(G_{±}). To produce the holomorphic extensions of f_{±} in G_{±} we need another trick. Namely, we will replace the cycle Γ^{(1)} in (12) by another cycle

Γ^{(2)} = ⋃_{u∈supp f} γ(u)

In this way if in Γ^{(1)} the choice of ζ was defined by x, then in G^{(2)} it is defined by u. Let us remark that similar 2 types of choices of cycles appear in the Cauchy-Fantappie formula [GH]. First of all, the form κf[x] is regular on this cycle since the horospheres L(ζ, ζ · u + i0), u ∈ G, ζ ∈ γ(u,) have no real points. For the same reason we can build a homotopy of Γ^{(1)} to Γ^{(2)} through cycles on which κ[f] is regular (by moving x to u along some paths and using the cycles γ). Such a way we can reconstruct f using Γ^{(2)} instead of Γ^{(1)}. Let us decompose the cycle Γ^{(2)} on parts Γ_{0}^{(2)}, Γ_{±}^{(2)} corresponding to parts γ_{0}(u), γ_{∓}(u) (we change the signs in indices). Let us consider integrals on these parts. We can see that sets Γ_{0}^{(1)} and Γ_{0}^{(2)} coincide and hence both give f_{0}. Correspondingly, the integrations on Γ_{±}^{(2)} give the projections on discrete series f_{±}:

(19)  f_{±}(x) = c ∫_{Γ_{±}^{(2)}} κ[f], c = \frac{1}{4(πi)^{3}} .

We can remove −i0 and remark that for each u ∈ G the integrand will be holomorphic for z ∈ G_{±} since horospheres L(ζ, ζ · u), u ∈ G, ζ ∈ γ_{∓}(u), lie in the closure of G^{∓} and they do not intersect G_{±} (they intersect G only in the point u). It directly gives holomorphic extensions of f_{±} to G_{±}.

Integrating κ[f] on ζ ∈ γ_{∓}(u) for fixed u we obtain the reproducing kernel Szegö:

(20)  S_{±}(z, u) = c ∫_{Λ_{∓}(u)} \frac{[z + u, u + iλ, dλ, dλ]}{\{(u + iλ) · (u − z)\}^{3}},  u ∈ G, z ∈ G_{±},

where Λ_{±}(u) are sheets of the two-sheet hyperboloid {λ; u · λ = 0, □λ = −1}.The kernels S_{±} have the convolutor form on the group G.

This Szegö kernel was defined and computed in [GG] using explicit formulas for characters and the Plancherel formula on SL(2; ℝ): the final expression is

S_{±}(z, u) = c{[tr(zx^{−1}) − 2]^{3} [tr(zx^{−1}) + 2]}^{−1/2}.

The Szegö kernel was investigated for other symmetric spaces of Hermitian type, but explicit formulas are known only in a few cases. I believe that it is interesting to have explicit formulas for kernels, but it would be more important to find their integral representations similar to (20). Such representations are like the classical Bochner’s representation of the Szegö kernel for tube domains, but the difference is quite substantial. We proved:

Proposition 3. The projections on discrete series f_{±} admit holomorphic extensions in the tubes G_{±} where they are represented by Szegö type integral formulas

(21)  f_{±}(z) = ∫_{G} S_{±}(z, u)f(u)μ(du), z ∈ G_{±}

with the kernels (20).

In this a way we constructed 2 integral representations of the projections on discrete series: one of them connects the projections with the horospherical transform and another one realizes the holomorphic extension.

Real horocyclic transform and cohohomological extension of f_{0}. Since the holomorphic wave front of the projection on continuous series f_{0} is contained in nonconvex cones V_{0}(u), we can not hope to extend it holomorphicaly. Instead we will extend it as 1-dimensional cohomology in a nonconvex tube G_{0}.

Firstly, the inversion formula (12) (as well as its analogue for the cycle Γ^{(2)}) gives the presentation of the projection f_{0}:

(22)  f_{0}(x) = \frac{1}{4(πi)^{3}} ∫_{G×SΞ} \frac{f(u)}{\{ξ · (u − x) − i0\}^{3}} μ(u, du) ∧ [x + u, ξ, dξ, dξ],

where SΞ is the factorization of the cone Ξ (3) relatve to the dilatation ξ ↦ rξ, r > 0: we integrate along any cycle intersecting any generating ray of this cone in one point. This formula after the integration on u gives the formula

(23)  f_{0}(x) = \frac{1}{2(πi)^{3}} ∫_{G×SΞ} Lf̂(ξ, ξ · x + i0)[x, ξ, dξ, dξ)].

We used in this computation that since ξ · ξ = 0,

\frac{[y, ξ, dξ, dξ]}{y · ξ}

is independent of y. To see it, is sufficient to replace a row in the determinant by the combination of all rows with the coefficients ξ_{i}.

The formula (22) delivers also the Szegö type kernel reconstucting f_{0}:

S_{0}(x, u) = \frac{1}{4(πi)^{3}} ∫_{SΞ} \frac{[x + u, ξ, dξ, dξ]}{\{ξ · (u − x) − i0\}^{3}} .

Of course,

S_{0}(x, e) = δ(x) − S_{+}(x, e) − S_{−}(x, e).

Let us remark that the integrand in (22) is not even. As usualy we can leave only its even part by the replacing {ξ · (u − x)}^{−3} by πδ′′(ξ · (u − x)). Correspondigly, in (23) we can write the local inversion formula by the replacing of the complex horospherical transform f̂ (13) by the real horospherical transform

Rf(ξ, p) = ∫_{G} f(u)δ(ξ · u − p)μ(du).

We can not connect with (23) an extension of f_{0} as a 2-dimensional cohomology. One of reasons is that the set of horospheras from Ξ̃_{0} passing through a point is not contractible. The way to correct it is to replace the horospherical transform by the horocyclic one. Technicaly, we will make in (22) the substitution

(24) ξ = λ + tν,  t > 0, λ, ν ∈ Ξ, x · λ = 0, λ · ν = 0, ν · x ≠ 0.

The last condition in (24) it would be possible to replace by the condition that λ and μ are not proportional: two elements of Ξ orthogonal to x ∈ G and mutualy orthogonal are proportional. Let Δ(x) be the set of such pairs ξ, η factorized on the equivalency relation (ξ, η) ∼ (rξ, sη), r > 0, s > 0. Then, after the substitution ξ = λ + tν from (24) in (22) and the integration on 0 < t < ∞, we obtain

(25)  f_{0}(x) = \frac{1}{16π} ∫_{G×Δ(x)} \frac{f(u)δ(λ · u)}{\{ν · (u − x) − i0\}^{2}} μ(du) ∧ [x, λ, ν, dν].

Apparently (25) is independent of a specific choice of the cycle Δ(x), satisfying to (24), and ν there is the function of λ.

For the deduction of (25) we used the next facts. For fixed x ∈ G, ξ · x ≠ 0, we pick up λ from the conditions λ · x = 0, λ · ξ = 0. Such λ lie on 2 rays. Then we pick up ν ∈ Ξ from the condition ν · ξ = 0. Such ν modulus ξ lie on 2 lines. Then we use that

∫_{0}^{∞} \frac{tdt}{(a + bt − i0)^{3}} = −1/2(a − i0)^{−1}(b − i0)^{−2}.

Here a = λ · u, b = ν · u. Eventualy we replace (λ · u − i0)^{−1} by its even part πδ(λ · u). Then we remark that on Δ(x) we have: dλ is proportional to ν and [x + u, λ, ν, dλ] = 0.

Let us replace in (25) the integrand by its even part: take {ν·(u−x)−i0}^{2} by {ν· (u−x)}^{2} such that we use the distribution y^{−2} (we take Cauchy principal value).The formula (25) permits to reconstruct f_{0} through the real horocyclic transform (26)

f̌(λ, ν, p) = ∫_{G} f(x)δ(λ · u)δ(ν · u − p)μ(du), λ, ν ∈ Ξ, λ · ν = 0, λ ≠ ν, p ∈ ℝ.

In this way we integrate f in (26) along the real horocycle x = a+tλ, t ∈ ℝ, ν·a = p. Then (25) means that

(27)  f_{0}(x) = \frac{1}{π} ∫_{Δ(x)} \frac{f̌(λ, ν, p)}{(p − ν · x)^{2}} [x, λ, ν, dν].

In fact we integrate in (27) along the set H(x) of real horocycles passing through x ∈ G. They parametrise by projectivization of the set of ξ ∈ Ξ orthogonal to x.The cycle Δ(x) covers H(x) with the multiplicity 16.

For the cohomological extension we will extend the integrand in (25) for any fixed λ ∈ ξ (and ν) on x to the half-horospheres L^{±}(λ). Such a way we have 1-form on Ξ with coefficients in L^{±}:

(28) τ[f](z|λ, dλ) = \frac{1}{16π} f(u)δ(λ · u)\{ν · (u − z)\}^{−2},  λ ∈ Ξ, z ∈ L^{±}(λ).

This form defines a 1-dimensional cohomology class on G_{0} and (25) realizes f_{0} as its boundary values. Since L^{±}(λ) have the codimension 1, there are some technical differences with [G1,G2] which we will consider in another place.

Proposition 4. The projection f_{0} on the continuous series admits the representation (22) which can be transform in the reconstructions of f_{0} through either the horospherical tranform f̂_{0}(23) or the real horocyclic transform f̌ (27). The last one permits the extension of f_{0} in G_{0} as 1-dimensional analytic cohomology.

References

[BEG] T.Bailey, M.Eastwood, S.Gindikin, Nonclassical descriptions of analytic cohomology, Preprint (2002).

[GG] I.Gelfand, S.Gindikin, Complex manifolds whose skeletons are real semisimple Lie groups and holomorphic discrete series, Functional Anal.Appl. 11 (1977), no. 4, 19–27.

[G1] S.Gindikin, Fourier transform and Hardy spaces of ¯∂-cohomology in tube domains, C.R. Acad.Sci.Paris 415 série I (1992), 1139–1143.

[G2] S.Gindikin, Real integral geometry and complex analysis, Integral geometry, Radon transform and complex analysis (E.Casadio Tarabusi, M.Picardello, G.Zampieri, eds.), Lecture Notes in Math., vol. 1684, Springer, 1998, pp. 70–98.

[G3] S.Gindikin, Integral geometry on SL(2; ℝ), Math. Res.Lett. 7 (2000), no. 4, 417–432.

[GH] S.Gindikin, G.Henkin, The Cauchy-Fantappie formula on projective space, Amer.Math. Soc.Transl.(2) 146 (1990), 23–32.

[HO] J.Hilgert, G.Ólafsson, Causal Smmetric Saces, Geometry and Harmonic Analysis, Academic Press, 1966.

Departm. of Math., Hill Center, Rutgers University, 110 Frelinghysen Road, Piscataway, NJ 08854-8019

   E-mail address: gindikin@math.rutgers.edu