DIFFERENTIABLE FUNCTIONS DEFINED IN

CLOSED SETS. A PROBLEM OF WHITNEY

Edward Bierstone, Pierre D. Milman and Wiesław Pawłucki

Abstract. In 1934, Whitney raised the question of how to recognize whether a function f defined on a closed subset X of ℝ^{n} is the restriction of a function of class C^{p}. A necessary and sufficient criterion was given in the case n = 1 by Whitney, using limits of finite differences, and in the case p = 1 by Glaeser (1958), using limits of secants. We introduce a necessary geometric criterion, for general n and p, involving limits of finite differences, that we conjecture is sufficient at least if X has a “tame topology”. We prove that, if X is a compact subanalytic set, then there exists q = q_{X}(p) such that the criterion of order q implies that f is C^{p}. The result gives a new approach to higher-order tangent bundles (or bundles of differentiable operators) on singular spaces.

1. Introduction

In 1934, Hassler Whitney published three pioneering articles on criteria for a

function f : X → ℝ, where X is a closed subset of ℝ^{n}, to be the restriction of a

function of class C^{p}, [W1], [W2], [W3]. (C^{p} means continuously differentiable to

order p, where p ∈ ℕ.) Whitney’s extension theorem [W1] gives a necessary and

sufficient condition for a field of polynomials ∑_{|α|≤p} f_{α}(a)(x − a)^{α}, a ∈ X, where

f_{0} = f, to be the field of Taylor polynomials of a C^{p} function. (We use multiindex

notation: α = (α_{1}, …, α_{n}) ∈ ℕ^{n}, |α| = α_{1} + ⋯ + α_{n} and x^{α} = x_{1}^{α_{1}} ⋯ x_{n}^{α_{n}}. ℕ

denotes the nonnegative integers.) In general, the functions f_{α} are, of course, not

uniquely determined by f. In Differentiable functions defined in closed sets. I [W2],

Whitney raises the deeper question of a necessary and sufficient criterion involving

only the values of f, and he answers the question in the case n = 1. Whitney

Research partially supported by the following grants: E.B. – NSERC OGP0009070, P.M. – NSERC OGP0008949 and the Killam Foundation, W.P. – KBN 5 PO3A 005 21.

2 DIFFERENTIABLE FUNCTIONS DEFINED IN CLOSED SETS

proves that, if X is a closed subset of the real line, then f extends to a C^{p} function if and only if the limiting values of all p’th divided differences [x_{0}, x_{1}, …, x_{p}]f, where the x_{i} ∈ X and x_{i} ≠ x_{j} if i ≠ j, define a continuous function on the diagonal {x_{0} = x_{1} = ⋯ = x_{p}}. ([x_{0}, x_{1}, …, x_{n}]f = p!c_{p}, where P (x) = c_{0} + c_{1}x + ⋯ + c_{p}x^{p} is the Lagrange interpolating polynomial for f at the points x_{0}, x_{1}, …, x_{p}; i.e., the unique polynomial of degree at most p such that P (x_{i}) = f(x_{i}), i = 0, …, p.)

“Differentiable functions defined in closed sets. II” never appeared, and up to now the only significant progress on Whitney’s problem following [W3] seems to have been the beautiful theorem of Georges Glaeser ([G], 1958) which solves the problem in the case p = 1 (cf. [Br]. See also Remark 2.3.) Glaeser defines a “(linearized) paratangent bundle” τ(X) using limits of secant lines. (See Section 3 below.) Suppose that f is continuous and let τ(f) denote the paratangent bundle of the graph of f. Then τ(f) can be regarded as a bundle over X, and τ(f) ⊂ τ(X)×ℝ (but τ(f) does not necessarily project onto τ(X)). Glaeser proves that f is the restriction of a C^{1} function if and only if τ(f) defines a function τ(f) : τ(X) → ℝ (i.e., τ(f) is the graph of a function τ(X) → ℝ; it will be convenient to identify a function with its graph).

In this article, we introduce a “(linearized) paratangent bundle of order p” τ^{p}(X), for any p ∈ ℕ, using limits of finitely supported distributions with values in the dual space P_{p}^{∗} of the space P_{p} = P_{p}(ℝ^{n}) of polynomial functions on ℝ^{n} of degree at most p (Section 4 below). Each fibre τ_{a}^{p}(X), a ∈ X, is a linear subspace of P_{p}^{∗}. Our construction involves a new interpretation of the remainder condition in Whitney’s extension theorem.

To every function f : X → ℝ, we associate a bundle ∇^{p}f ⊂ τ^{p}(X) × ℝ.

Conjecture. f is the restriction of a C^{p} function if and only if ∇^{p}f defines a function ∇^{p}f : τ^{p}(X) → ℝ.

Moreover, if ∇^{p}f : τ^{p}(X) → ℝ and ∇_{a}^{p}f = 0, for some a ∈ X, then there exists F ∈ C^{p}(ℝ^{n}) such that F |X = f and T_{a}^{p}F = 0, where T_{a}^{p}F denotes the Taylor polynomial of order p of F at a.

BIERSTONE, MILMAN, PAWŁUCKI 3

Necessity of the criterion ∇^{p}f : τ^{p}(X) → ℝ is not difficult; the following theorem is proved in Section 4.

Theorem 1.1. If f : X → ℝ extends to a C^{p} function, then

∇^{p}f : τ^{p}(X) → ℝ .

Moreover, if F ∈ C^{p}(ℝ^{n}) and F |X = f, then, for all a ∈ X and ξ ∈ τ_{a}^{p}(X) ⊂ P_{p}^{∗},

∇^{p}f(ξ) = ξ(T_{a}^{p}F) .

The converse direction is true if X is a C^{p} submanifold. In Section 4, we prove more precisely:

Theorem 1.2. If X ⊂ M, where M is a C^{p} submanifold of ℝ^{n} and X is the closure of its interior in M, then f : X → ℝ extends to a C^{p} function if and only if ∇^{p}f : τ^{p}(X) → ℝ.

In order to make the conjecture tractable in general, it is reasonable to restrict to closed sets X that have a “tame” geometry (“géométrie modérée”); for example, closed subanalytic sets or, more generally, closed sets that are definable in an ominimal structure (cf. [vdD]). Our main result is the following theorem (proved in Section 5).

Theorem 1.3. Let X be a compact subanalytic subset of ℝ^{n}. Then there is a function q = q_{X}(p) ≥ p from ℕ to itself such that, if f : X → ℝ, q ≥ q_{X}(p) and

∇^{q}f : τ^{q}(X) → ℝ ,

then f extends to a C^{p} function. If, moreover, a ∈ X and ∇_{a}^{q}f = 0, then f extends to a C^{p} function that is p-flat at a.

The novelty of Theorem 1.3 lies in the construction of τ^{p}(X) and ∇^{p}f. Let X be a compact subanalytic subset of ℝ^{n}. Then there is a compact real analytic manifold M such that dim M = dim X, and a real analytic mapping φ : M → ℝ^{n} such that

4 DIFFERENTIABLE FUNCTIONS DEFINED IN CLOSED SETS

φ(M) = X (by the Uniformization Theorem [BM1, Thm. 0.1]). Let g = f ∘ φ. We prove that if ∇^{p}f : τ^{p}(X) → ℝ, then:

(1) ∇^{p}g : τ^{p}(M) → ℝ (Theorem 5.2); therefore, g ∈ C^{p}(M) by Theorem 1.2. (All notions make sense for manifolds.)

(2) g is formally a composite with φ; i.e., for all a ∈ X, there exists P ∈ P_{p}(ℝ^{n}) such that g − P ∘ φ is p-flat at every point b ∈ φ^{−1}(a) (Corollary 5.3).

Theorem 1.3 is then a consequence of the following composite function theorem [BMP]: There is a function q = q_{φ}(p) such that if g ∈ C^{q}(M) is formally a composite with φ, then there exists F ∈ C^{p}(ℝ^{n}) such that g = F ∘ φ. (Moreover, if S is a finite subset of X and g is q-flat on φ^{−1}(S), then there exists F with the additional property that F is p-flat on S.)

Let C^{(∞)}(X) = ⋂_{p∈ℕ} C^{p}(X), where C^{p}(X) denotes the space of restrictions of C^{p} functions to X.

Corollary 1.4. If X ⊂ ℝ^{n} is a closed subanalytic set and f : X → ℝ, then f ∈ C^{(∞)}(X) if and only if ∇^{p}f : τ^{p}(X) → ℝ, for all p ∈ ℕ.

Of course C^{∞}(X) ⊂ C^{(∞)}(X), where C^{∞}(X) denotes the restrictions of C^{∞} functions to X, and C^{(∞)}(X) = C^{∞}(X) if X is a C^{∞} submanifold (but not in general [P2]). Among closed subanalytic sets, equality characterizes the proper subclass of sets that have a “semicoherent” (or stratified coherent) structure [BM2]; see Remarks 2.7 below. In Section 2, we use Theorem 1.3 to compare the paratangent bundle τ^{p}(X) with another natural idea of a bundle of differential operators on a singular space. Surprisingly, uniformity of a “Chevalley estimate” (which also characterizes the class of semicoherent subanalytic sets) is related to important stability properties of these bundles. (See Remarks 2.7, Theorem 2.9 and Corollary 2.10.)

The space C^{(∞)}(X) seems to be an interesting function space for a closed set X that is definable in an o-minimal structure. A definable set has a C^{p} cell decomposition for every p, but C^{∞} cell decomposition, in general, is unknown and likely untrue. Theorem 1.3 provides strong evidence for the conjecture above in the case of a definable set X. (See Final Remarks 5.5.) The loss of differentiability in Theorem 1.3 is related to the use of [BMP] via a uniformization of X.

2. Geometric and algebraic paratangent bundles

In this section, we introduce a “Zariski paratangent bundle” T^{p}(X) – a (higherorder) C^{p} analogue of the Zariski tangent bundle studied in algebraic geometry – and we use the results above to compare the paratangent bundle τ^{p}(X) with T^{p}(X). (Whitney [W4] makes such a comparison in order 1, for various notions of tangent spaces to an analytic variety.) It seems less interesting to use the Zariski paratangent bundle to provide a criterion to recognize whether a function f : X → ℝ is the restriction of a C^{p} function because T^{p}(X) is defined already in terms of the ideal of C^{p} functions vanishing on X (hence essentially in terms of the space of restrictions to X of C^{p} functions); see Remark 2.3. The interest is rather in the opposite direction – to use the conjecture or the results in Section 1, involving limits of finite differences, to get a better understanding of higher-order tangent bundles (or bundles of differential operators) on singular spaces. This section is not used in the rest of the paper, except in Remarks 4.13(1).

We use the notation of Section 1. If V ⊂ P_{p} = P_{p}(ℝ^{n}), let V^{⊥} denote the orthogonal complement of V in the dual space P_{p}^{∗}. Let X be a closed subset of ℝ^{n}. Let I^{p}(X) ⊂ C^{p}(ℝ^{n}) denote the ideal of C^{p} functions that vanish on X. Definition 2.1. The Zariski paratangent bundle of order p, T^{p}(X), is the subbundle of X × P_{p}^{∗} with fibre T_{a}^{p}(X) = (T_{a}^{p}I^{p}(X))^{⊥}, for each a ∈ X. (See Definition 3.1 below.)

The bundle T^{p}(X) is closed in X × P_{p}^{∗} because, for all h ∈ I^{p}(X), Z(h) := {(a, ξ) ∈ X × P_{p}^{∗} : ξ(T_{a}^{p}h) = 0} is closed, and T^{p}(X) = ⋂_{h∈I^{p}(X)} Z(h). Moreover, if a ∈ X, then

τ_{a}^{p}(X) ⊂ (T_{a}^{p}I^{p}(X))^{⊥} ⊂ P_{p}^{∗}

(by Definition 4.12 or Theorem 1.1); thus τ^{p}(X) ⊂ T^{p}(X). Both τ^{p}(·) and T^{p}(·) are functors on the category of closed (or locally closed) subsets of Euclidean spaces

6 DIFFERENTIABLE FUNCTIONS DEFINED IN CLOSED SETS

(with morphisms given by the restrictions of C^{p} mappings); cf. Theorem 5.2(1) below.

Consider q ≥ p. Let a ∈ ℝ^{n}. Let \underline{m}_{a}^{p+1} ⊂ P_{q} denote the subspace of polynomials of order at least p + 1 at a; i.e.,

\underline{m}_{a}^{p+1} = \{P ∈ P_{q} : (D^{α}P)(a) = 0, |α| ≤ p\}.

There is a projection P_{q} → P_{p} defined by truncating terms in x − a of order > p in the expression of any P ∈ P_{q} as a polynomial in x − a. Let P_{p}^{∗} ↪ P_{q}^{∗} denote the embedding dual to this projection. The projection induces an isomorphism P_{q}/\underline{m}_{a}^{p+1} ≅ P_{p}, and the embedding P_{p}^{∗} ↪ P_{q}^{∗} has image (\underline{m}_{a}^{p+1})^{⊥}.

Let τ^{q}(X)_{p} denote the subbundle of X × P_{p}^{∗} with fibre

τ_{a}^{q}(X)_{p} := τ_{a}^{q}(X) ∩ (\underline{m}_{a}^{p+1})^{⊥} ,

for each a ∈ X, where (\underline{m}_{a}^{p+1})^{⊥} is identified with P_{p}^{∗} via the embedding above. Of course, τ^{p}(X)_{p} = τ^{p}(X) .

Lemma 2.2. τ^{q}(X)_{p} is a closed subbundle of X × P_{p}^{∗}.

Proof. There is a continuous bundle mapping (cf. Definition 4.23) X×P_{p}^{∗} → X×P_{q}^{∗} with closed image, where, for each a ∈ X, the fibre P_{p}^{∗} over a is embedded in P_{q}^{∗} as above. Via this mapping, X × P_{p}^{∗} is a closed subbundle of X × P_{q}^{∗}. Of course, τ^{q}(X)_{p} = τ^{q}(X) ∩ (X × P_{p}^{∗}). □

If F ∈ C^{q}(ℝ^{n}), let T_{a}^{p}F denote the Taylor polynomial of order p of F at a. Let T^{q}(X)_{p} ⊂ X × P_{p}^{∗} denote the bundle with fibre

T_{a}^{q}(X)_{p} := (T_{a}^{p}I^{q}(X))^{⊥} = T_{a}^{q}(X) ∩ (\underline{m}_{a}^{p+1})^{⊥} ,

for each a ∈ X. Then T^{q}(X)_{p} is closed in X × P_{p}^{∗}, and

τ^{q}(X)_{p} ⊂ T^{q}(X)_{p} .

BIERSTONE, MILMAN, PAWŁUCKI 7

Remark 2.3. It is possible to formulate various criteria for the existence of a C^{p} extension of f : X → ℝ involving only the values of f on X. The following is essentially tautological.

Implicit function criterion. Suppose that f is continuous. Then f is the restriction of a C^{p} function if and only if, for every a ∈ X, there is a neighbourhood of (a, f(a)) in ℝ^{n} × ℝ in which the graph of f lies in a C^{p} submanifold whose tangent space at (a, f(a)) contains no vertical vector (i.e., no derivation in the vertical direction).

The result of O’Farrell and Watson [O’FW] is a closely related criterion that can be expressed in terms of the Zariski paratangent bundle T^{p}(X). Suppose that f is continuous. Let T^{p}(f) denote the Zariski paratangent bundle of order p of the graph of f. Then T^{p}(f) can be regarded as a bundle over X, and the projection ℝ^{n} × ℝ → ℝ^{n} induces a bundle mapping π : T^{p}(f) → T^{p}(X). The theorem of [O’FW] asserts essentially that f is the restriction of a C^{p} function if and only if π is bijective. “Only if” is obvious. On the other hand, if π is bijective, then, for each a ∈ X, T_{a}^{p}(f)_{1} contains no derivation in the vertical direction. It follows from Definition 2.1 that, in some neighbourhood of (a, f(a)), the graph of f lies in a submanifold whose tangent space at (a, f(a)) contains no vertical derivation. The result follows from the implicit function criterion.

Our conjecture in Section 1 implies that

τ^{p}(X) = T^{p}(X) :

We need only show that T^{p}(X) ⊂ τ^{p}(X). Suppose that P ∈ τ_{a}^{p}(X)^{⊥} ⊂ P_{p}; we have to show that P ∈ T_{a}^{p}I^{p}(X). Let f = P |X. The conjecture asserts that there exists F ∈ C^{p}(ℝ^{n}) such that F |X = f and T_{a}^{p}F = 0. Let G = P − F. Then G ∈ I^{p}(X), and T_{a}^{p}G = P − T_{a}^{p}F = P.

The following is a corollary of Theorem 1.3.

Corollary 2.4. Suppose that X is a compact subanalytic subset of ℝ^{n}. Let q = q_{X}(p) denote a function satisfying the assertion of Theorem 1.3. If s ≥ p and q ≥ q_{X}(s), then

T^{s}(X)_{p} ⊂ τ^{q}(X)_{p} .

Proof. Let a ∈ X and let P ∈ τ_{a}^{q}(X)_{p}^{⊥} ⊂ P_{q}. We have to show that P ∈ T_{a}^{p}I^{s}(X)+ \underline{m}_{a}^{p+1}. Recall that τ_{a}^{q}(X)_{p} = τ_{a}^{q}(X) ∩ (\underline{m}_{a}^{p+1})^{⊥}; thus P ∈ τ_{a}^{q}(X)^{⊥} + \underline{m}_{a}^{p+1}; i.e., there exists Q ∈ τ_{a}^{q}(X)^{⊥} ⊂ P_{q} such that T_{a}^{p}Q − P ∈ \underline{m}_{a}^{p+1}. Let f = Q|X. By Theorem 1.1, ∇^{q}f : τ^{q}(X) → ℝ and ∇_{a}^{q}f = 0 (since Q ∈ τ_{a}^{q}(X)^{⊥}). By Theorem 1.3, there exists F ∈ C^{s}(ℝ^{n}) such that F |X = f and T_{a}^{s}F = 0. Set G = Q − F. Then G ∈ I^{s}(X), and T_{a}^{p}G − P = T_{a}^{p}Q − P ∈ \underline{m}_{a}^{p+1}. □

Following the viewpoint of Corollary 1.4 above, we can also introduce

\underline{τ}^{p}(X) := ⋃_{q≥p} τ^{q}(X)_{p} ,
\underline{T}^{p}(X) := ⋃_{q≥p} T^{q}(X)_{p} .

Then \underline{τ}^{p}(X) ⊂ \underline{T}^{p}(X). Corollary 2.5. If X is a closed subanalytic subset of ℝ^{n}, then \underline{τ}^{p}(X) = \underline{T}^{p}(X). Definition 2.6. Let ℝ[[x − a]] denote the ring of formal power series in (x_{1} − a_{1}, …, x_{n} − a_{n}). If F ∈ ℝ[[x − a]], let T_{a}^{p}F (x) denote the Taylor polynomial of order p of F at a; i.e., the polynomial of degree ≤ p obtained by truncating the terms of F of order > p in x − a. Suppose a ∈ X. We define the formal local ideal F_{a}(X) of X at a as {F ∈ ℝ[[x − a]] : T_{a}^{p}F (x) = o(|x − a|^{p}), where x ∈ X, for all p ∈ ℕ}. (See [BM2, Lemma 6.1].) Remarks 2.7. For each q ≥ p, τ^{q}(X)_{p} ⊂ τ^{q+1}(X)_{p} (as in Remarks 4.13(1) below) and T^{q}(X)_{p} ⊂ T^{q+1}(X)_{p}; i.e., {τ^{q}(X)_{p}}_{q≥p} and {T^{q}(X)_{p}}_{q≥p} are increasing sequences of closed subbundles of X×P_{p}^{∗}. If {τ^{q}(X)_{p}}_{q≥p} (respectively, {T^{q}(X)_{p}}_{q≥p}) stabilizes, then \underline{τ}^{p}(X) (respectively, \underline{T}^{p}(X)) is closed. Since τ^{q}(X)_{p} ⊂ T^{q}(X)_{p}, for all q ≥ p, it follows from Corollary 2.4 that, if X is a compact subanalytic set, then, for all p ∈ ℕ, {τ^{q}(X)_{p}}_{q≥p} stabilizes if and only if {T^{q}(X)_{p}}_{q≥p} stabilizes. We say that a closed subanalytic subset X of ℝ^{n} is semicoherent if it has a locally

BIERSTONE, MILMAN, PAWŁUCKI 9

finite subanalytic stratification such that the formal local ideals F_{a}(X) are generated over each stratum by finitely many subanalytically parametrized formal power series [BM2, Definition 1.2]. In Corollary 2.10 below, we show that a compact subanalytic subset X of ℝ^{n} is semicoherent if and only if the sequence {T^{q}(X)_{p}}_{q≥p} (or the sequence {τ^{q}(X)_{p}}_{q≥p}) stabilizes, for all p ∈ ℕ.

For compact subanalytic sets X in general, however, \underline{T}^{p}(X) is not necessarily closed, and \{T^{q}(X)_{p}\}_{q≥p} does not necessarily stabilize even if \underline{T}^{p}(X) is closed. If n ≤ 4, dim X ≤ 2, or dim X ≥ n − 1, then X is semicoherent. In ℝ^{5}, consider any sequence of distinct points {a_{j}} tending to the origin along some line. By the construction of [P1], there is a compact 3-dimensional subanalytic subset X of ℝ^{5} such that X is not semicoherent, X is semicoherent outside 0, and F_{a}(X) = 0 if and only if a ∈ \{a_{j}\} (cf. [BM2, Examples 1.29]). By Corollary 2.10, \underline{T}^{p}(X)\\underline{T} _{0}^{p}(X) is closed in (X\0) × P_{p}^{∗}, for all p. For all p and j, \underline{T} _{a_{j}}^{p}(X) = P_{p}^{∗}, by Lemma 2.8, since F_{a_{j}}(X) = 0. It follows that, if 0 ∈ \{a_{j}\}, then \underline{T}^{p}(X) is closed, for all p. On the other hand, if 0 ∉ \{a_{j}\}, then F_{0}(X) ≠ 0, so there exists p such that \underline{T} _{0}^{p}(X) ≠ P_{p}^{∗}, and it follows that \underline{T}^{p}(X) is not closed.

Let X be a closed subset of ℝ^{n} and let a ∈ X. Then {T_{a}^{p}I^{q}(X)}_{q≥p} is a decreasing sequence of linear subspaces of P_{p}. Let s_{X}(a, p) denote the smallest integer s ≥ p such that T_{a}^{p}I^{q}(X) = T_{a}^{p}I^{s}(X), for all q ≥ s. Of course, s_{X}(a, p) is the smallest integer s ≥ p such that T_{a}^{s}(X)_{p} = ⋃_{q≥p} T_{a}^{q}(X)_{p}.

Lemma 2.8. Suppose that X is subanalytic. Let a ∈ X and let s ≥ p. Then s ≥ s_{X}(a, p) if and only if

T_{a}^{p}I^{s}(X) = T_{a}^{p}F_{a}(X) .

Proof. First we show that, if s ≥ s_{X}(a, p), then T_{a}^{p}I^{s}(X) ⊂ T_{a}^{p}F_{a}(X), for any closed X ⊂ ℝ^{n}. Let p_{0} = p and p_{j} = s_{X}(a, p_{j−1}), for all j ≥ 1. Let P = T_{a}^{p}f_{0}, where f_{0} ∈ I^{p_{1}}(X). We have to show that P = T_{a}^{p}F, where F ∈ F_{a}(X). By the definition of {p_{j}}, for all j ≥ 1, T_{a}^{p_{j−1}}I^{p_{j}}(X) = T_{a}^{p_{j−1}}I^{p_{j+1}}(X), so that we can inductively

10 DIFFERENTIABLE FUNCTIONS DEFINED IN CLOSED SETS

find f_{j} ∈ I^{p_{j+1}}(X), j ≥ 1, such that T_{a}^{p_{j−1}}f_{j−1} = T_{a}^{p_{j−1}}f_{j}. Let Q_{j} = T_{a}^{p_{j}}f_{j}, j ≥ 0, so that Q_{j}(x) = o(|x − a|^{p_{j}}), where x ∈ X, and Q_{j−1} = T_{a}^{p_{j−1}}Q_{j}, j ≥ 1. Take F ∈ ℝ[[x − a]] such that Q_{j} = T_{a}^{p_{j}}F, for all j. Then P = T_{a}^{p}F and, for all j, T_{a}^{p_{j}}F = Q_{j} = o(|x − a|^{p_{j}}), where x ∈ X, as required.

We can assume that X is a compact subanalytic set. Then T_{a}^{q}F_{a}(X) ⊂ T_{a}^{q}I^{q}(X), for all q, as follows: There is a compact real analytic manifold M and a real analytic mapping φ : M → ℝ^{n} such that φ(M) = X. Let F ∈ F_{a}(X). Take f ∈ C^{∞}(ℝ^{n}) such that F is the formal Taylor series of f at a. Set g = f ∘ φ. Then g is flat on φ^{−1}(a). By [BMP], for all q ∈ ℕ, there exists f_{q} ∈ C^{q}(ℝ^{n}), such that T_{a}^{q}f_{q} = 0 and g = f_{q} ∘ φ. Then f − f_{q} ∈ I^{q}(X) and T_{a}^{q}(f − f_{q}) = T_{a}^{q}f = T_{a}^{q}F.

Thus T_{a}^{p}F_{a}(X) ⊂ T_{a}^{p}I^{q}(X), for all q ≥ p. If s ≥ s_{X}(a, p), then T_{a}^{p}I^{s}(X) ⊂ T_{a}^{p}F_{a}(X); hence T_{a}^{p}I^{s}(X) = T_{a}^{p}F_{a}(X). Conversely, if T_{a}^{p}I^{s}(X) = T_{a}^{p}F_{a}(X), then, for all q ≥ s, T_{a}^{p}I^{q}(X) ⊂ T_{a}^{p}I^{s}(X) = T_{a}^{p}F_{a}(X) ⊂ T_{a}^{p}I^{q}(X), so that T_{a}^{p}I^{q}(X) = T_{a}^{p}I^{s}(X), and hence s ≥ s_{X}(a, p). □

Let X denote a compact subanalytic subset of ℝ^{n}. Let φ : M → ℝ^{n} be a real analytic mapping from a compact real analytic manifold M, such that φ(M) = X. By [BMP], for all p ∈ ℕ, there exists q ≥ p with the following property: if a ∈ X and g ∈ C^{q}(M) such that g is formally a composite with φ and g is q-flat on φ^{−1}(a), then there exists f ∈ C^{p}(ℝ^{n}) such that g = f ∘ φ and f is p-flat at a. Let q_{φ}(p) denote the least such q.

By a lemma of Chevalley (cf. [BM2, Section 6]), for all k ∈ ℕ, there exists l ∈ ℕ, l ≥ k, with the following property: for every polynomial F (x) such that F (x) = o(|x − a|^{l}), where x ∈ X, there exists G ∈ F_{a}(X) such that T_{a}^{k}F = T_{a}^{k}G. Given k, let l_{X}(a, k) denote the least such l. We call l_{X}(a, k) a Chevalley estimate.

Theorem 2.9. Let X be a compact subanalytic subset of ℝ^{n} and let φ : M → ℝ^{n} be a real analytic mapping as above. Then, for all a ∈ X and all p ∈ ℕ,

s_{X}(a, p) ≤ l_{X}(a, p) ≤ q_{φ}(s_{X}(a, p)) .

BIERSTONE, MILMAN, PAWŁUCKI 11

Proof. For the first inequality, let s = l_{X}(a, p); by Lemma 2.8, it is enough to show that if f ∈ I^{s}(X), then T_{a}^{p}f ∈ T_{a}^{p}F_{a}(X). Let P = T_{a}^{s}f. Then P (x) = o(|x − a|^{s}), where x ∈ X, so the result follows from the definition of l_{X}(a, p).

For the second inequality, let s = s_{X}(a, p) and q = q_{φ}(s). Let F denote a polynomial such that F (x) = o(|x−a|^{q}), where x ∈ X. By Lemma 2.8, it is enough to show that T_{a}^{p}F ∈ T_{a}^{p}I^{s}(X). Let g = F ∘ φ. Then g ∈ C^{∞}(M) and g is q-flat on φ^{−1}(a). By [BMP], there exists f ∈ C^{s}(ℝ^{n}) such that T_{a}^{s}f = 0 and g = f ∘ φ. Then f = F on X and T_{a}^{s}f = 0, so that T_{a}^{p}F = T_{a}^{p}(F − f) ∈ T_{a}^{p}I^{s}(X). □

Corollary 2.10. Let X be a compact subanalytic subset of ℝ^{n}.

(1) Let p ∈ ℕ. Then the increasing union ⋃_{q≥p} T^{q}(X)_{p} stabilizes if and only if there exists l ∈ ℕ such that l_{X}(a, p) ≤ l, for all a ∈ X.

(2) The increasing union ⋃_{q≥p} T^{q}(X)_{p} (or the increasing union ⋃_{q≥p} τ^{q}(X)_{p}) stabilizes, for all p ∈ ℕ, if and only if X is semicoherent.

Proof. The first assertion is immediate from Theorem 2.9. By [BM2, Theorem 1.13], X is semicoherent if and only if there exists a uniform Chevalley estimate; i.e., a function l = l_{X}(k) such that l_{X}(a, k) ≤ l_{X}(k), for all k ∈ ℕ and a ∈ X. So the second statement also follows. □

3. Glaeser’s construction

Let X be a metric space and let V be a real vector space of finite dimension r.

Definition 3.1. A bundle (of linear subspaces of V) over X is a subset E of X ×V such that, for all a ∈ X, the fibre E_{a} := {v ∈ V : (a, v) ∈ E} is a linear subspace of V.

Definition 3.2. A Glaeser operation (on bundles of linear subspaces of V over X) is an operation ρ that associates to each bundle E a bundle ρ(E) such that:

(1) \overline{E} ⊆ ρ(E);

(2) ρ is local; i.e., if E, F are bundles over X and E_{a} = F_{a} for all a ∈ U, where U ⊂ X is open, then ρ(E)_{a} = ρ(F)_{a} for all a ∈ U.

12 DIFFERENTIABLE FUNCTIONS DEFINED IN CLOSED SETS

We include a proof of the following lemma of Glaeser [G] because we use it in Sections 4 and 5.

Lemma 3.3. Let E be a bundle of linear subspaces of V over X, and let ρ be a Glaeser operation. Write ρ^{i} := ρ ∘ ⋯ ∘ ρ (i times). Then:

(1) ρ^{i} = ρ^{2r} if i ≥ 2r;

(2) Ê := ρ^{2r}(E) is a closed bundle;

(3) dim Ê_{a} is an upper-semicontinuous function of a ∈ X.

Proof. Set

d_{i}(a) = dim ρ^{i}(E)_{a} , λ_{i}(a) = inf_{δ>0} sup_{σ(a,x)<δ} d_{i}(x) ,

for all a ∈ X and i = 0, 1, …, where σ(·, ·) denotes the metric on X. Then, for all i, λ_{i} is upper-semicontinuous and d_{i} ≤ λ_{i} ≤ d_{i+1} (the latter inequality since \overline{ρ^{i}(E)} ⊂ ρ^{i+1}(E)). Let

G_{i} := int {a ∈ X : d_{i}(a) = d_{i+1}(a)}

and let Z_{i} := X\G_{i}. Then G_{i} ⊂ G_{i+1} for all i, since, for all a ∈ G_{i}, ρ^{i}(E)_{a} = ρ^{i+1}(E)_{a} and therefore ρ^{i+1}(E)_{a} = ρ^{i+2}(E)_{a} by locality (Definition 3.2, property (2)). Thus Z_{i} ⊃ Z_{i+1} for all i.

We claim that d_{i+2}(a) > i/2, for all a ∈ Z_{i}, i ∈ ℕ. First, this holds for i = 0: Otherwise, there exists a ∈ Z_{0} such that d_{2}(a) = 0. Then λ_{1}(a) = 0, hence d_{0}(x) = d_{1}(x) = 0 in a neighbourhood of a, so that a ∈ G_{0} (a contradiction). The claim is true for i = 1 because, if a ∈ Z_{1}, then a ∈ Z_{0} so that d_{3}(a) ≥ d_{2}(a) ≥ 1. Consider i ≥ 2 and suppose that d_{j+2}(a) > j/2 for all a ∈ Z_{j}, when j < i. Set Σ_{i} := \{a ∈ X : d_{i}(a) < d_{i+1}(a)\}. Then Z_{i} = \overline{Σ}_{i}, so Σ_{i} ⊂ Z_{i} ⊂ Z_{i−2}. Therefore, if a ∈ Σ_{i}, then d_{i}(a) > (i − 2)/2, so that d_{i+1}(a) > i/2. It follows that, for all a ∈ Z_{i}, d_{i+2}(a) ≥ λ_{i+1}(a) > i/2. This proves the claim, by induction.

It follows that Z_{2r} = ∅, so (1) holds. (2) follows because \overline{Ê} ⊂ ρ(Ê) ⊂ Ê, and (3) because d_{2r}(a) ≤ λ_{2r}(a) ≤ d_{2r+1}(a), a ∈ X. □

BIERSTONE, MILMAN, PAWŁUCKI 13

Example 3.4. [G]. For any bundle E ⊂ X × V, define

Ẽ := ⋃_{a∈X}{a} × Span E_{a}

where Span denotes the linear span). Set λ(E) := ˜ \overline{E}. Then λ is a Glaeser operation and λ(E) ⊂ ρ(E) for any Glaeser operation ρ. Definition 3.5. [G]. Let X be a closed subset of ℝ^{n}. Define

ptg(X) := \{(a, σu) ∈ X × ℝ^{n} : σ ∈ ℝ , u = lim_{j→∞} \frac{x_{j} − y_{j}}{|x_{j} − y_{j}|} ,

where (x_{j}), (y_{j}) ⊂ X, x_{j} ≠ y_{j}, for all j ,

and lim_{j→∞} x_{j} = a = lim_{j→∞} y_{j}} ;

τ(X) :=  ; { \atop ptg(X)} ,

where the “saturation” is with respect to the Glaeser operation λ of Example 3.4. We call τ(X) the (linearized) paratangent bundle of X. If a ∈ X, the fibre τ_{a}(X) is called the paratangent space of X at a.

Let f : X → ℝ be a continuous function. (We identify a function with its graph and therefore write τ(f) = τ(graph f).) We can consider τ(f) as a bundle over X, so that τ(f) ⊂ τ(X) × ℝ. Theorem 3.6. [G], [Br]. Let X be a closed subset of ℝ^{n} and let f : X → ℝ be a continuous function. Then there exists F ∈ C^{1}(ℝ^{n}) such that F |X = f if and only if τ(f) : τ(X) → ℝ (i.e., τ(f) is the graph of a function τ(X) → ℝ). In this case, each τ_{a}(f) : τ_{a}(X) → ℝ, a ∈ X, is the restriction to the paratangent space τ_{a}(X) of the derivative of F.

4. Higher-order paratangent spaces

The remainder term in Taylor’s theorem. Let X ⊂ U ⊂ ℝ^{n}, where U is open and X is closed in U. Let p ∈ ℕ. Let F = (F^{α})_{α∈ℕ^{n},|α|≤p}, where each F^{α} : X → ℝ. If a ∈ X, define

(T_{a}^{p}F)(x) := ∑_{|α|≤p} \frac{1}{α!}F^{α}(a)(x − a)^{α} .

Let D^{α} := ∂^{|α|}/∂x_{1}^{α_{1}} ⋯ ∂x_{n}^{α_{n}}, α ∈ ℕ^{n}. If |α| ≤ p and b ∈ X, set

(4.1)  (R_{a}^{p}F)^{α}(b) := F^{α}(b) − D^{α}(T_{a}^{p}F)(b)

= F^{α}(b) − ∑_{|β|≤p−|α|} \frac{1}{β!}F^{α+β}(a)(b − a)^{β} ,

(4.2)  δ_{α}(a, b) := \frac{(R_{a}^{p}F)^{α}(b)}{|b − a|^{p−|α|}} .

We recall Whitney’s extension theorem [W1]:

Theorem 4.3. Let F^{α} : X → ℝ, |α| ≤ p. Then there exists f ∈ C^{p}(U) such that (D^{α}f)|X = F^{α}, for all |α| ≤ p, if and only if δ_{α}(a, b) → 0 if a, b ∈ X and |α| ≤ p, as |a − b| → 0.

We say that F = (F^{α}) is a C^{p} Whitney field on X if it satisfies the conditions of Theorem 4.3.

Let P_{p} = P_{p}(ℝ^{n}) denote the real vector space of polynomial functions on ℝ^{n} of degree at most p. Let ξ ∈ P_{p}^{∗}, where P_{p}^{∗} denotes the dual of P_{p}. Set

(4.4)  ξ(F, a) := ξ(T_{a}^{p}F) = ∑_{|α|≤p} F^{α}(a)ξ_{α}(a) ,

where

(4.5)  ξ_{α}(a) := ξ (\frac{1}{α!}(x − a)^{α}) .

If η ∈ P_{p}^{∗} and b ∈ X, then

η(F, b) = ∑_{|α|≤p} F^{α}(b)η_{α}(b)

= ∑_{|α|≤p} ( ∑_{|β|≤p−|α|} \frac{1}{β!}F^{α+β}(a)(b − a)^{β} + δ_{α}(a, b)|b − a|^{p−|α|}) η_{α}(b)

= ∑_{|α|≤p} ∑_{β≤α} \frac{1}{β!}F^{α}(a)(b − a)^{β}η_{α−β}(b) + ∑_{|α|≤p} δ_{α}(a, b)|b − a|^{p−|α|}η_{α}(b)

= ∑_{|α|≤p} ∑_{β≤α} \frac{1}{β!}F^{α}(a)(b − a)^{β}η (\frac{1}{(α − β)!}(x − b)^{α−β}) + ∑_{|α|≤p} δ_{α}(a, b)|b − a|^{p−|α|}η_{α}(b)

= ∑_{|α|≤p} \frac{1}{α!}F^{α}(a)η (∑_{β≤α} \binom{α}{β}(b − a)^{β}(x − b)^{α−ρ}) + ∑_{|α|≤p} δ_{α}(a, b)|b − a|^{p−|α|}η_{α}(b)

= ∑_{|α|≤p} F^{α}(a)η (\frac{1}{α!}(x − a)^{α}) + ∑_{|α|≤p} δ_{α}(a, b)|b − a|^{p−|α|}η_{α}(b) .

(If α = (α_{1}, …, α_{n}) and β = (β_{1}, …, β_{n}), then β ≤ α means β_{i} ≤ α_{i}, i = 1, …, n.) Therefore,

(4.6)  η(F, b) = η(F, a) + ∑_{|α|≤p} δ_{α}(a, b)|b − a|^{p−|α|}η_{α}(b);

(4.7)  ξ(F, a) + η(F, b) = (ξ + η)(F, a) + ∑_{|α|≤p} δ_{α}(a, b)|b − a|^{p−|α|}η_{α}(b) .

We will use the following lemma only in the case k = 1 (but see Remarks 4.13(2) and Final Remarks 5.5).

Lemma 4.8. Let X ⊂ U ⊂ ℝ^{n}, where U is open and X is closed in U. Let (a_{ij}) = (a_{i1}, a_{i2}, …) and (ξ_{ij}) = (ξ_{i1}, ξ_{i2}, …) denote sequences in X and P_{p}^{∗}, respectively, for i = 0, 1, …, k, such that:

(1) The sequences (a_{ij}), i = 0, 1, …, k, converge to a common point a ∈ X, and ∑_{i=0}^{k} ξ_{ij} converges to ξ ∈ P_{p}^{∗}.

(2) |a_{ij} −a_{0j}|^{p−|α|}|ξ_{ij,α}(a_{ij})| ≤ c, for all i, j and |α| ≤ p (where c is a constant).

16 DIFFERENTIABLE FUNCTIONS DEFINED IN CLOSED SETS

If F = (F^{α})_{|α|≤p} is a C^{p} Whitney field on X, then

ξ(F, a) = lim_{j→∞} ∑_{i=0}^{k} ξ_{ij}(F, a_{ij}) .

Proof. For each j = 1, 2, …,

ξ(F, a) − ∑_{i=0}^{k} ξ_{ij}(F, a_{ij})

= ξ(F, a) − ∑_{i=0}^{k} ξ_{ij}(F, a) + ∑_{i=0}^{k} (ξ_{ij}(F, a) − ξ_{ij}(F, a_{0j})) + ∑_{i=0}^{k} (ξ_{ij}(F, a_{0j}) − ξ_{ij}(F, a_{ij}))

= (ξ − ∑_{i=0}^{k} ξ_{ij}) (T_{a}^{p}F) + ∑_{i=0}^{k} ξ_{ij}(T_{a}^{p}F − T_{a_{0j}}^{p}F) + ∑_{i=0}^{k} ξ_{ij}(T_{a_{0j}}^{p}F − T_{a_{ij}}^{p}F)

= (ξ − ∑_{i=0}^{k} ξ_{ij}) (T_{a}^{p}F) + ∑_{i=0}^{k} ∑_{|α|≤p} δ_{α}(a_{0j}, a)|a − a_{0j}|^{p−|α|}ξ_{ij,α}(a)

− ∑_{i=0}^{k} ∑_{|α|≤p} δ_{α}(a_{0j}, a_{ij})|a_{ij} − a_{0j}|^{p−|α|}ξ_{ij,α}(a_{ij}) .

Each of the three terms tends to 0 as j → ∞. □ The paratangent bundle of order p. We consider Glaeser operations on bundles of subspaces of P_{p}^{∗}. Let X ⊂ U ⊂ ℝ^{n}, where U is open and X is closed in U. Let E ⊂ X × P_{p}^{∗} be any bundle of linear subspaces of P_{p}^{∗} over X. Define

(4.9)  ΔE := {(a, b, ξ + η) : a, b ∈ X, ξ ∈ E_{a}, η ∈ E_{b},
|a − b|^{p−|α|}|η_{α}(b)| ≤ 1 for all |α| ≤ p} .

Let π : X^{2} × P_{p}^{∗} → X × P_{p}^{∗} denote the projection π(a, b, ξ) = (a, ξ). Define

(4.10)  E′ := π(\overline{ΔE} ∩ \{(a, a, ξ) : a ∈ X, ξ ∈ P_{p}^{∗}\}) .

Clearly, \overline{E} ⊂ E′. We define a Glaeser operation

(4.11)  ρ(E) := Ẽ′ := ⋃_{a∈X}{a} × Span E′_{a} .

(Recall Example 3.4.)

Definition 4.12. Let X ⊂ U ⊂ ℝ^{n} be as above, and set

E := {(a, λδ_{a}) : a ∈ X, λ ∈ ℝ} ,

where δ_{a} ∈ P_{p}^{∗} denotes the delta-function δ_{a}(P) := P (a), P ∈ P_{p}. Define

τ^{p}(X) := Ê ,

where Ê denotes the saturation of E with respect to the Glaeser operation (4.11) (cf. Lemma 3.3). We call τ^{p}(X) the (linearized) paratangent bundle of X of order p.

Remarks 4.13. (1) Consider q ≥ p. Recall that X × P_{p}^{∗} embeds in X × P_{q}^{∗} as a closed subbundle, where, for each a ∈ X, the fibre P_{p}^{∗} over a is identified as in Section 2 with (\underline{m}_{a}^{p+1})^{⊥} ⊂ P_{q}^{∗}. If b ∈ X and η ∈ (\underline{m}_{b}^{p+1})^{⊥}, then η_{α}(b) = 0, for all p < |α| ≤ q. It follows from the definition above that τ^{p}(X) is a closed subbundle of τ^{q}(X).

(2) The definition above involves distributions with values in P_{p}^{∗} supported at pairs of points a, b ∈ X, according to (4.9), and suffices for all results in this paper. But a more general definition of τ^{p}(X) involving distributions supported at k + 1 points (where k ≥ p) is necessary for our main conjecture in Section 1. See Final Remarks 5.5. We have stated Lemma 4.8 and Lemma 5.1 below for distributions supported at k + 1 points in order that they be available more generally.

Now let Φ ⊂ X × (P_{p}^{∗} × ℝ) be a bundle of linear subspaces of P_{p}^{∗} × ℝ over X. Define

(4.14) ΔΦ := {(a, b, ξ + η, λ + μ) : a, b ∈ X, (ξ, λ) ∈ Φ_{a} , (η, μ) ∈ Φ_{b} ,
|a − b|^{p−|α|}|η_{α}(b)| ≤ 1 for all |α| ≤ p} ;
(4.15) Φ′ := π(\overline{ΔΦ} ∩ \{(a, a, ξ, λ) : a ∈ X, ξ ∈ P_{p}^{∗} , λ ∈ ℝ\}) ,

where π : X^{2} × P_{p}^{∗} × ℝ → X × P_{p}^{∗} × ℝ is the projection π(a, b, ξ, λ) = (a, ξ, λ). As before, \overline{Φ} ⊂ Φ′.

18 DIFFERENTIABLE FUNCTIONS DEFINED IN CLOSED SETS

Definition 4.16. Define

∇^{p}f := Φ̂ ,

where

Φ := {(a, λδ_{a}, λf(a)) : a ∈ X, λ ∈ ℝ}

and Φ̂ denotes the saturation with respect to the Glaeser operation ρ(Φ) := Φ̃′.

Clearly, ∇^{p}f ⊂ τ^{p}(X)×ℝ. Theorem 1.1 is a restatement of Theorem 4.18 below.

Lemma 4.17. Let X ⊂ U ⊂ ℝ^{n}, where U is open and X is closed in U. Let f : X → ℝ. Let p ∈ ℕ and suppose there is a C^{p} Whitney field F = (F^{α})_{|α|≤p} on X such that F^{0} = f. Consider the bundles E, Φ over X and the Glaeser operations ρ of Definitions 4.12 and 4.16. Then, for each i ∈ ℕ,

ρ^{i}(Φ) : ρ^{i}(E) → ℝ ;

moreover, if a ∈ X and ξ ∈ ρ^{i}(E)_{a}, then

ρ^{i}(Φ)(ξ) = ξ(T_{a}^{p}F) = ξ(F, a) .

Proof. First consider i = 1. Any element of Φ′ can be expressed

lim_{j→∞}(a_{0j}, a_{1j}, ξ_{0j} + ξ_{1j}, λ_{0j}f(a_{0j}) + λ_{1j}f(a_{1j})) ,

where ξ_{ij} = λ_{ij}δ_{a_{ij}}, i = 0, 1, j = 1, 2, …, and (a_{ij}), (ξ_{ij}) satisfy the hypotheses of Lemma 4.8 (case k = 1). Of course , λ_{ij}f(a_{ij}) = ξ_{ij}(F, a_{ij}), for all i, j. By Lemma 4.8,

lim_{j→∞}(λ_{0j}f(a_{0j}) + λ_{1j}f(a_{1j})) = lim_{j→∞}(ξ_{0j} + ξ_{1j})(F, a) .

Therefore, Φ′ : E′ → ℝ and, for all ξ ∈ E′_{a}, a ∈ X, Φ′(ξ) = ξ(F, a) = ξ(T_{a}^{p}F). It follows that ρ(Φ) : ρ(E) → ℝ and, for all ξ ∈ ρ(E)_{a}, ρ(Φ)(ξ) = ξ(T_{a}^{p}F) = ξ(F, a).

The result then follows from Lemma 4.8, by induction on i. □

BIERSTONE, MILMAN, PAWŁUCKI 19

Theorem 4.18. Let X ⊂ U ⊂ ℝ^{n}, where U is open and X is closed in U. Let f : X → ℝ. Let p ∈ ℕ and suppose there is a C^{p} Whitney field F = (F^{α})_{|α|≤p} on X such that F^{0} = f. Then

∇^{p}f : τ^{p}(X) → ℝ ;

moreover, if a ∈ X and ξ ∈ τ_{a}^{p}(X) ⊂ P_{p}^{∗}, then

∇^{p}f(ξ) = ξ(T_{a}^{p}F) = ξ(F, a) .

This is an immediate consequence of Lemma 4.17. Remarks 4.19. Let f : X → ℝ. If ∇^{p}f : τ^{p}(X) → ℝ, then ∇^{p}f (as well as f) is necessarily continuous and, for all a ∈ X, the induced function on the fibre ∇_{a}^{p}f : τ_{a}^{p}(X) → ℝ is necessarily linear. Consider q ≥ p. Then ∇^{p}f is a closed subbundle of ∇^{q}f (cf. Remarks 4.13(1)); if ∇^{q}f : τ^{q}(X) → ℝ, then ∇^{p}f : τ^{p}(X) → ℝ and ∇^{p}f is the restriction of ∇^{q}f.

We will show that Theorem 1.2 is a consequence of Theorem 4.21 below. Lemma 4.20. Suppose that X ⊂ U ⊂ ℝ^{n}, where U is open and X is the closure of int X in U. Then τ^{p}(X) = X × P_{p}^{∗}, for all p ∈ ℕ. Proof. It is enough to show that τ_{a}^{p}(X) = P_{p}^{∗}, where a ∈ int X. If α ∈ ℕ^{n}, |α| ≤ p, and b ∈ X, define D^{α}(b) ∈ P_{p}^{∗} by

D^{α}(b)(P) := (D^{α}P)(b) , P ∈ P_{p} .

If |α| < p, then

\frac{D^{α}(b) − D^{α}(a)}{|b − a|} ⟶ ∑_{i=1}^{n} u_{i}D^{α+(i)}(a)

if b → a and \frac{b − a}{|b − a|} → u = (u_{1}, …, u_{n}), where (i) denotes the multiindex with 1

in the i’th place and 0 elsewhere. Let η = \frac{D^{α}(b)}{|b − a|} (where |α| < p). Then, for all γ ∈ ℕ^{n}, |γ| ≤ p,

|a − b|^{p−|γ|}η_{γ}(b) = |a − b|^{p−|γ|−1}D^{α} (\frac{1}{γ!}(x − b)^{γ}) (b)

= \{ \begin{matrix}0 & , γ ≠ α \\ |a − b|^{p−|α|−1} & , γ = α . \end{matrix}

20 DIFFERENTIABLE FUNCTIONS DEFINED IN CLOSED SETS

By Definition 4.12, it follows by induction on |γ| that D^{γ}(a) ∈ τ_{a}^{p}(X), for all |γ| ≤ p; i.e., τ_{a}^{p}(X) = P_{p}^{∗}. □

Theorem 4.21. Let X ⊂ U ⊂ ℝ^{n}, where U is open and X is the closure of intX in U. Let f : X → ℝ. Let p ∈ ℕ. Suppose that

∇^{p}f : τ^{p}(X) → ℝ .

Then there is a C^{p} Whitney field F = (F^{α})_{|α|≤p} on X such that F^{0} = f.

Proof. By Lemma 4.20, we have

∇^{p}f : X × P_{p}^{∗} → ℝ .

Define

F^{α}(a) := (∇^{p}f)(a, D^{α}(a)) , a ∈ X ,

for all α ∈ ℕ^{n}, |α| ≤ p. (We use the notation of the proof of Lemma 4.20.) Let c ∈ X and α ∈ ℕ^{n}, |α| ≤ p. Then

\frac{D^{α}(b) − ∑_{|β|≤p−|α|} \frac{1}{β!}(b − a)^{β}D^{α+ρ}(a)}{|b − a|^{p−|α|}} ⟶ 0

as a, b → c, where a, b ∈ X, a ≠ b. (In fact, this element of P_{p}^{∗} equals zero since,

for all P ∈ P_{p}, (D^{α}P)(b) = (T_{a}^{p−|α|}D^{α}P)(b).) If η := \frac{D^{α}(b)}{|b − a|^{p−|α|}}, then

|a − b|^{p−|γ|}η_{γ}(b) = \{ \begin{matrix}0 & , γ ≠ α \\ 1 & , γ = α . \end{matrix}

Hence

\frac{(∇^{p}f)(b, D^{α}(b)) − ∑_{|β|≤p−|α|} \frac{1}{β!}(b − a)^{β}(∇^{p}f)(a, D^{α+β}(a))}{|b − a|^{p−|α|}} ⟶ 0

as a, b → c in X, a ≠ b; in other words,

F^{α}(b) − ∑_{|β|≤p−|α|} \frac{1}{β!}F^{α+β}(a)(b − a)^{β} = o(|b − a|^{p−|α|}) ,

BIERSTONE, MILMAN, PAWŁUCKI 21

as required. □ Remark 4.22. The following is a simple generalization of Theorem 4.21: Consider 1 ≤ m ≤ n and X ⊂ U × {0} ⊂ ℝ^{m} × ℝ^{n−m} = ℝ^{n}, where U is open in ℝ^{m} and X = \overline{int X} as a subset of U. Let f : X → ℝ. Suppose that ∇^{p}f : τ^{p}(X) → ℝ. Then there is a C^{p} Whitney field F = (F^{α})_{α∈ℕ^{n},|α|≤p} on X such that F^{0} = f and F^{α} = 0 when α_{m+1} + ⋯ + α_{n} > 0. Definition 4.23. Let X, Y denote metric spaces and V, W finite-dimensional real vector spaces. Let E ⊂ X × V and F ⊂ Y × W be bundles (of linear subspaces of V and W, respectively). A morphism E → F is a continuous mapping ψ : E → F of the form ψ(a, v) = (φ(a), ψ_{1}(a, v)), where (a, v) ∈ E, such that, for all a ∈ X, ψ_{1}(a, ·) : E_{a} → F_{φ(a)} is linear. An isomorphism is a morphism with a continuous inverse (which is necessarily a morphism).

Suppose that U_{1}, U_{2} are open subsets of ℝ^{n} and that X_{1}, X_{2} are closed subsets of U_{1}, U_{2}, respectively. Let σ : U_{1} → U_{2} be a C^{p} diffeomorphism (p ∈ ℕ) such that σ(X_{1}) = X_{2}. Clearly, σ induces an isomorphism σ_{∗} : τ^{p}(X_{1}) → τ^{p}(X_{2}). (See Theorem 5.2 below.) If f_{1} : X_{1} → ℝ and f_{2} : X_{2} → ℝ are functions such that f_{1} = f_{2} ∘ σ, then σ induces an isomorphism σ^{∗} : ∇^{p}f_{1} → ∇^{p}f_{2}. These observations can be used to generalize the results above to manifolds. Theorem 1.2 is a special case of the following. Theorem 4.24. Let X ⊂ M ⊂ U ⊂ ℝ, where U is open, M is a closed C^{p} submanifold of U, and X = \overline{intX} as a subset of M. Let f : X → ℝ. If ∇^{p}f : τ^{p}(X) → ℝ, then f is the restriction of an element of C^{p}(U).

5. Composite functions

Let U, V be open subsets of ℝ^{n}, ℝ^{m} (respectively) and let φ : V → U be a C^{p} mapping. Let b ∈ V, a = φ(b). Then φ induces a linear mapping

φ_{b}^{∗} : P_{p}(ℝ^{n}) → P_{p}(ℝ^{m})
P ↦ T_{b}^{p}(P ∘ φ) ;

22 DIFFERENTIABLE FUNCTIONS DEFINED IN CLOSED SETS

i.e., if x = (x_{1}, …, x_{n}), y = (y_{1}, …, y_{m}) denote the coordinates of ℝ^{n}, ℝ^{m} (respectively) and φ = (φ_{1}, …, φ_{n}), then φ_{b}^{∗}(P) is given by substituting T_{b}^{p}φ = (T_{b}^{p}φ_{1}, …, T_{b}^{p}φ_{n}) into P (x) = (T_{a}^{p}P)(x) and truncating terms involving (y − b)^{β} where |β| > p. By duality, there is a linear mapping

φ_{∗b} : P_{p}(ℝ^{m})^{∗} → P_{p}(ℝ^{n})^{∗} ;

i.e., φ_{∗b}(η)(P) = η(φ_{b}^{∗}(P)), where η ∈ P_{p}(ℝ^{m})^{∗} and P ∈ P_{p}(ℝ^{n}).

Note that φ_{∗b}(δ_{b}) = δ_{a}. We will need the following lemma only in the case k = 1 (cf. Lemma 4.8 and Remarks 4.13(2)).

Lemma 5.1. Let X, Y be closed subsets of U, V (respectively), where U ⊂ ℝ^{n}, V ⊂ ℝ^{m} are open. Let φ : V → U be a C^{p} mapping such that φ(Y) ⊂ X. Let (b_{ij}) = (b_{i1}, b_{i2}, …) and (η_{ij}) = (η_{i1}, η_{i2}, …) denote sequences in Y and P_{p}(ℝ^{m})^{∗}, respectively, for i = 0, 1, …, k such that:

(1) The sequences (b_{ij}), i = 0, 1, …, k, converge to a common point b ∈ Y, and

(∑_{i=0}^{k} η_{ij}) converges to η ∈ P_{p}(ℝ^{m})^{∗}.

(2) |b_{ij} − b_{0j}|^{p−|β|}|η_{ij,β}(b_{ij})| ≤ c, for all i, j and β ∈ ℕ^{m}, |β| ≤ p, where c is a constant.

Set a_{ij} = φ(b_{ij}) ∈ X and ξ_{ij} = φ_{∗b_{ij}}(η_{ij}), for all i, j, and set a = φ(b), ξ = φ_{∗b}(η). Then:

(1′) (a_{ij}) converges to a ∈ X, for all i = 0, 1, …, k, and (∑_{i=0}^{k} ξ_{ij}) converges to ξ.

(2′) |a_{ij} − a_{0j}|^{p−|α|}|ξ_{ij,α}(a_{ij})| ≤ c′, for all i, j and α ∈ ℕ^{n}, |α| ≤ p, where c′ is a constant.

Proof. Obviously, each (a_{ij}) converges to a. Let P ∈ P_{p}(ℝ^{n}). Then, for each j,

ξ(P) − ∑_{i=0}^{k} ξ_{ij}(P) = η(φ_{b}^{∗}(P)) − ∑_{i=0}^{k} η_{ij}(φ_{b_{ij}}^{∗}(P))

= η(G, b) − ∑_{i=0}^{k} η_{ij}(G, b_{ij}) ,

BIERSTONE, MILMAN, PAWŁUCKI 23

where G denotes the C^{p} Whitney field on Y induced by P ∘φ. Therefore (1′) follows from Lemma 4.8.

There is a constant C such that, for all i, j and α ∈ ℕ^{n}, |α| ≤ p,

|a_{ij} − a_{0j}|^{p−|α|}|ξ_{ij,α}(a_{ij})|

≤ C|b_{ij} − b_{0j}|^{p−|α|} |η_{ij} (φ_{b_{ij}}^{∗} (\frac{1}{α!}(x − a_{ij})^{α}))|

≤ C|b_{ij} − b_{0j}|^{p−|α|} ∑_{{β∈ℕ^{m} \atop |α|≤|β|≤p}} Λ_{β}((D^{γ}φ(b_{ij}))_{1≤|γ|≤p})|η_{ij,β}(b_{ij})| ,

where each Λ_{β} is a polynomial function. Therefore, (2′) follows from (2). □

Theorem 5.2. Suppose X ⊂ U ⊂ ℝ^{n} and Y ⊂ V ⊂ ℝ^{m}, where U, V are open and X, Y are closed in U, V (respectively). Let φ : V → U be a C^{p} mapping such that φ(Y) ⊂ X. Then:

(1) φ induces a bundle morphism

φ_{∗} : τ^{p}(Y) → τ^{p}(X)

such that , if b ∈ Y and η ∈ τ_{b}^{p}(Y) ⊂ P_{p}(ℝ^{m})^{∗}, then

φ_{∗}(η) = φ_{∗b}(η) .

Moreover, let f : X → ℝ and set g = f ∘ φ : Y → ℝ. Suppose that ∇^{p}f : τ^{p}(X) → ℝ. Then:

(2) ∇^{p}g : τ^{p}(Y) → ℝ and, if b ∈ Y and η ∈ τ_{b}^{p}(Y), then

∇^{p}g(η) = ∇^{p}f(φ_{∗}η) .

(3) Let b ∈ Y and a = φ(b). Choose P ∈ P_{p}(ℝ^{n}) such that P |τ_{a}^{p}(X) : τ_{a}^{p}(X) → ℝ coincides with ∇_{a}^{p}f (where we have identified P_{p}(ℝ^{n}) with P_{p}(ℝ^{n})^{∗∗}). Then, for all η ∈ τ_{b}^{p}(Y) ⊂ P_{p}(ℝ^{m})^{∗},

∇^{p}g(η) = η(φ_{b}^{∗}(P)) .

24 DIFFERENTIABLE FUNCTIONS DEFINED IN CLOSED SETS

Proof. (1) follows from Lemma 5.1 and the definition of the paratangent bundle in the same way that Lemma 4.17 is proved above using Lemma 4.8.

Consider b_{ij} ∈ Y and η_{ij} ∈ τ_{b_{ij}}^{p}(Y) ⊂ P_{p}(ℝ^{m})^{∗}, i = 0, 1, j = 1, 2, …, satisfying the hypotheses of Lemma 5.1 (case k = 1). Let a_{ij}, ξ_{ij}, a and ξ be as in Lemma 5.1. Then, by Lemma 5.1 and Remark 4.19,

∇^{p}f(φ_{∗b}(η)) = ∇^{p}f(ξ)

= lim_{j→∞} (∇^{p}f(ξ_{0j}) + ∇^{p}f(ξ_{1j}))

= lim_{j→∞} (∇^{p}f(φ_{∗b_{0j}}(η_{0j})) + ∇^{p}f(φ_{∗b_{1j}}(η_{1j}))) .

(2) then follows in the same way that (1) is proved.

To prove (3): Let η ∈ τ_{b}^{p}(Y) and let ξ = φ_{∗b}(η) ∈ τ_{a}^{p}(X) ⊂ P_{p}(ℝ^{n})^{∗}. Then ∇^{p}f(ξ) = ξ(P), by the choice of P, so that

∇^{p}f(φ_{∗b}(η)) = ∇^{p}f(ξ)
= φ_{∗b}(η)(P)
= η(φ_{b}^{∗}(P)) ,

and the result follows from (2). □

Corollary 5.3. Suppose X ⊂ U ⊂ ℝ^{n} and V ⊂ ℝ^{m}, where U, V are open and X is closed in U. Let φ : V → U be a C^{p} mapping such that φ(V) ⊂ X. Let f : X → ℝ and set g = f ∘ φ. Suppose that ∇^{p}f : τ^{p}(X) → ℝ. Then:

(1) g ∈ C^{p}(V).

(2) g is formally a composite with φ; i.e., for all a ∈ X, there exists P ∈ P_{p}(ℝ^{n}) such that g − P ∘ φ is p-flat at every point b ∈ φ^{−1}(a).

Proof. (1) follows from Theorem 5.2 (2) and Theorem 4.21.

Let a ∈ X. Choose P ∈ P_{p}(ℝ^{n}) as in Theorem 5.2 (3). Let b ∈ φ^{−1}(a). We will show that

(5.4)  T_{b}^{p}g = T_{b}^{p}(P ∘ φ) .

BIERSTONE, MILMAN, PAWŁUCKI 25

Since T_{b}^{p}(P ∘ φ) = φ_{b}^{∗}(P), (5.4) means that, for all η ∈ τ_{b}^{p}(V) = P_{p}(ℝ^{m})^{∗},

η(T_{b}^{p}g) = η(φ_{b}^{∗}(P)) .

But η(T_{b}^{p}g) = (∇^{p}g)(η), by Theorem 4.18, so the result follows from Theorem 5.2 (3). □ Differentiable functions on closed subanalytic sets. Proof of Theorem 1.3. Let X be a compact subanalytic subset of ℝ^{n}. By [BM1, Thm. 0.1], there is a compact real analytic manifold M and a real analytic mapping φ : M → ℝ^{n} such that φ(M) = X. By [BMP], there is a function q = q_{φ}(p) from ℕ to itself such that, if g ∈ C^{q}(M) and g is formally a composite with φ, then there exists F ∈ C^{p}(ℝ^{n}) such that g = F ∘ φ; moreover, if S is a finite subset of X and g is q-flat on φ^{−1}(S), then there exists F with the additional property that F is p-flat on S.

Let f : X → ℝ. Let p ∈ ℕ and suppose that ∇^{q}f : τ^{q}(X) → ℝ, where q = q_{φ}(p). Let g = f ∘ φ. By Corollary 5.3 (generalized to a manifold V), g ∈ C^{q}(M) and g is formally a composite with φ. Therefore, f ∈ C^{p}(X). □ Remark 5.4. If X is a closed subanalytic subset of ℝ^{n}, then τ^{p}(X) is a closed subanalytic subset of ℝ^{n} × P_{p}(ℝ^{n})^{∗}. Final Remarks 5.5. (1) Let X ⊂ U ⊂ ℝ^{n}, where U is open and X is closed in U. Let f : X → ℝ. Our definitions of τ^{p}(X) and ∇^{p}f involve limits of distributions with values in P_{p}(ℝ^{n})^{∗} supported at two points. We can generalize the definitions (and all constructions in the article) by using distributions supported at k + 1 points, for any k = 1, 2, . . .. We simply modify (4.9) and (4.10) in the following way: Let E ⊂ X × P_{p}(ℝ^{n})^{∗} be any bundle of linear subspaces of P_{p}(ℝ^{n})^{∗} over X. Define

(5.6) Δ_{k+1}E := {(a_{0}, a_{1}, …, a_{k}, ξ_{0} + ξ_{1} + ⋯ + ξ_{k}) : a_{i} ∈ X , ξ_{i} ∈ E_{a_{i}} ,
|a_{i} − a_{0}|^{p−|α|}|ξ_{iα}(a_{i})| ≤ 1 , for all |α| ≤ p , i = 0, …, k} ;
(5.7) E′_{k+1} := π(\overline{Δ_{k+1}E} ∩ \{(a, a, …, a, ξ) : a ∈ X , ξ ∈ P_{p}(ℝ^{n})^{∗}) ,

26 DIFFERENTIABLE FUNCTIONS DEFINED IN CLOSED SETS

where π is the projection π(a_{0}, a_{1}, …, a_{k}, ξ) = (a_{0}, ξ). We define τ_{k+1}^{p}(X) = Ê as before, and ∇_{k+1}^{p}f ⊂ τ_{k+1}^{p}(X) × ℝ also in a similar way.

Of course τ_{k}^{p}(X) ⊂ τ_{k+1}^{p}(X) and ∇_{k}^{p}f ⊂ ∇_{k+1}^{p}f, for all k ≥ 2; in particular, if ∇_{k+1}^{p}f : τ_{k+1}^{p}(X) → ℝ, then ∇_{k}^{p}f : τ_{k}^{p}(X) → ℝ. We have used only τ^{p}(X) = τ_{2}^{p}(X) in this article because it suffices for all the results. Our main conjecture in Section 1 should be understood as requiring τ_{k+1}^{p}(X), where k ≥ p. (For example, if

X = ⋃_{i=0}^{p}{(x, y) ∈ ℝ^{2} : y = ix^{2}} ,

then k = p is necessary and sufficient.)

Questions. Does there exist r = r(X, p) ∈ ℕ such that τ_{k}^{p}(X) = τ_{r}^{p}(X) if k ≥ r? If X is subanalytic, can we take r = p + 1?

(2) It is not difficult to see that the definition of Δ_{k+1}E above is equivalent to that given by replacing the condition

|a_{i} − a_{0}|^{p−|α|}|ξ_{iα}(a_{i})| ≤ 1 , i = 0, …, k ,

by the condition

|a_{i} − a_{0}|^{p−|α|}|ξ_{iα}(a_{0})| ≤ 1 , i = 0, …, k .

(Likewise in Lemma 4.8). It is not possible, however, to define τ_{k+1}^{p}(X) using limits

ξ = lim_{j→∞} ∑_{i=0}^{k} ξ_{ij}

(in the notation of Lemma 4.8) where condition (2) of Lemma 4.8 is replaced by the symmetric condition

(5.8)  |a_{ij} − a|^{p−|α|}|ξ_{ij,α}(a_{ij})| ≤ c ,

for all i, j and |α| ≤ p.

For example, let (x_{1}, y_{1}) = (1, 1) and, for each j = 1, 2, …, define (x_{j+1}, y_{j+1})

inductively as follows: If j is odd (respectively, even), let (x_{j+1}, y_{j+1}) be the in-

tersection point of the line through (x_{j}, y_{j}) with slope 2 (respectively, −2) and the

arc y = −x^{2}, x > 0 (respectively y = x^{2}, x > 0). Let X = {0} ∪ {x_{j} : j ≥ 1} ⊂ ℝ.

Define F^{0}(0) = 0, F^{0}(x_{j}) = y_{j}, for all j, and F^{1}(a) = 0, for all a ∈ X. Then

lim_{j→∞} \frac{(R_{0}^{1}F)^{0}(x_{j})}{|x_{j} − 0|} = 0 ,

but

\frac{(R_{0}^{1}F)^{0}(x_{j})}{|x_{j} − x_{j+1}|}

does not tend to zero as j → ∞, so that F is not a Whitney field. Take a_{0j} = x_{j},

a_{1j} = x_{j+1}, ξ_{0j} = \frac{δ_{x_{j}}}{x_{j} − x_{j+1}} and ξ_{1j} = \frac{δ_{x_{j+1}}}{x_{j} − x_{j+1}} , for all j. Then the condition (2)

of Lemma 4.8 (case k = 1) is satisfied, but not the symmetric condition (5.8).

References

[BM1] E. Bierstone and P.D. Milman, Semianalytic and subanalytic sets, Inst. Hautes Études Sci. Publ. Math. 67 (1988), 5–42.

[BM2] E. Bierstone and P.D. Milman, Geometric and differential properties of subanalytic sets, Ann. of Math. 147 (1998), 731–785.

[BMP] E. Bierstone, P.D. Milman and W. Pawłucki, Composite differentiable functions, Duke Math. J. 83 (1996), 607–620.

[Br] S. Bromberg, An extension theorem in the class C^{1}, Bol. Soc. Mat. Mexicana (2) 27 (1982), no. 2, 35–44.

[vdD] L. van den Dries, Tame topology and o-minimal structures, London Math. Soc. Lecture Note Series, vol. 248, Cambridge Univ. Press, Cambridge, 1998.

[G] G. Glaeser, Études de quelques algèbres tayloriennes, J. Analyse Math. 6 (1958), 1–124. [O’FW] A.G. O’Farrell and R.O. Watson, The tangent stars of a set, and extensions of smooth

functions, J. Reine Angew. Math. 430 (1992), 109–137.

[P1] W. Pawłucki, On relations among analytic functions and geometry of subanalytic sets, Bull. Polish Acad. Sci. Math. 37 (1989), 117-125.

[P2] W. Pawłucki, Examples of functions C^{k}-extendable for each k finite, but not C^{∞}- extendable, Singularities Symposium – Łojasiewicz 70, Banach Center Publ., vol. 44, Polish Acad. Sci., Warsaw, 1998, pp. 183–187.

[W1] H. Whitney, Analytic extensions of differentiable functions defined in closed sets, Trans. Amer. Math. Soc. 36 (1934), 63–89.

[W2] H. Whitney, Differentiable functions defined in closed sets. I, Trans. Amer. Math. Soc. 36 (1934), 369–387.

[W3] H. Whitney, Functions differentiable on the boundaries of regions, Ann. of Math. 35 (1934), 482–485.

[W4] H. Whitney, Tangents to an analytic variety, Ann. of Math. 81 (1965), 496-549.

E.B. and P.M.: Department of Mathematics, University of Toronto, Toronto, Ontario, Canada M5S 3G3

E-mail address: bierston@math.toronto.edu, milman@math.toronto.edu

W.P.: Uniwersitet Jagielloński, Instytut Matematyki, ul. Reymonta 4, 30059 Kraków, Poland

E-mail address: Wieslaw.Pawlucki@im.uj.edu.pl