# GAUGED FLOER HOMOLOGY FOR HAMILTONIAN ISOTOPIES I: DEFINITION …guangbox/VHF1.pdf ·...

date post

12-Mar-2020Category

## Documents

view

2download

0

Embed Size (px)

### Transcript of GAUGED FLOER HOMOLOGY FOR HAMILTONIAN ISOTOPIES I: DEFINITION …guangbox/VHF1.pdf ·...

GAUGED FLOER HOMOLOGY FOR HAMILTONIAN ISOTOPIES I:

DEFINITION OF THE FLOER HOMOLOGY GROUPS

GUANGBO XU

Abstract. We construct the vortex Floer homology group V HF (M,µ;H) for an aspherical Hamil-

tonian G-manifold (M,ω) with moment map µ and a class of G-invariant Hamiltonian loop Ht,

following the proposal of [3]. This is a substitute for the ordinary Hamiltonian Floer homology of

the symplectic quotient of M . We achieve the transversality of the moduli space by the classical

perturbation argument instead of the virtual technique, so the homology can be defined over Z orZ2.

Contents

1. Introduction 1

2. Basic setup and outline of the construction 7

3. Asymptotic behavior of the connecting orbits 14

4. Fredholm theory 21

5. Compactness of the moduli space 28

6. Floer homology 31

Appendix A. Transversality by perturbing the almost complex structure 37

References 46

1. Introduction

1.1. Background. Floer homology, introduced by Andreas Floer (see [8], [9]), has been a great

triumph of J-holomorphic curve technique invented by Gromov [17] in many areas of mathemat-

ics. Hamiltonian Floer homology gives new invariants of symplectic manifolds and its Lagrangian

submanifolds and has been the most important approach towards the solution to the celebrated

Arnold conjecture initiated in the theory of Hamiltonian dynamics; the Lagrangian intersection

Floer homology is the basic language in defining the Fukaya category of a symplectic manifold

and stating Kontsevich’s homological mirror symmetry conjecture; several Floer-type homology

theory, including the instanton Floer homology ([7], [4]), Heegaard-Floer theory ([29]), Seiberg-

Witten Floer homology ([22]), ECH theory ([20], [21]), has become tools of understanding lower

dimensional topology.

All these different types of Floer theory, are all certain infinite dimensional Morse theory, whose

constructions essentially apply Witten’s point of view ([34]). Basically, if f : X → R is certain

Date: December 24, 2013.

1

2 GUANGBO XU

smooth functional on manifold X (which could be infinite dimensional), then with an appropriate

choice of metric on X, we can study the equation of negative gradient flow of f , of the form

x′(t) +∇f(x(t)) = 0, t ∈ (−∞,+∞). (1.1)

If some natural energy functional defined for maps from R to X is finite for a solution to the aboveequation, then x(t) will converges to a critical point of f . Assuming that all critical points of f is

nondegenerate, then usually we can define a Morse-type index (or relative indices) λf : Critf →Z. Then for a given pair of critical points a−, a+ ∈ Critf , the moduli space of solutions to thenegative gradient flow equation which are asymptotic to a± as t → ±∞, denoted by M(a−, a+),has dimension equal to λf (a−)−λf (a+), if f and the metric are perturbed generically. If λf (a−)−λf (a+) = 1, because of the translation invariance of (1.1), we expect to have only finitely many

geometrically different solutions connecting a− and a+. In many cases (which we call the oriented

case), we can also associate a sign to each such solutions.

On the other hand, we define a chain complex over Z2 (and over Z in the oriented case), spannedby critical points of f and graded by the index λf ; the boundary operator ∂ is defined by the

(signed) counting of geometrically different trajectories of solutions to (1.1) connecting two critical

points with adjacent indices. We expect a nontrivial fact that ∂ ◦ ∂ = 0. So a homology group isderived.

1.2. Hamiltonian Floer homology and the transversality issue. In Hamiltonian Floer the-

ory, we have a compact symplectic manifold (X,ω) and a time-dependent Hamiltonian Ht ∈C∞(X), t ∈ [0, 1]. We can define an action functional AH on a covering space L̃X of the con-tractible loop space of X. The space L̃X consists of pairs (x,w) where x : S1 → X is a contractibleloop and w : D→ X with w|∂D = x; the action functional is defined as

AH(x,w) = −∫Dw∗ω −

∫S1Ht(x(t))dt. (1.2)

The Hamiltonian Floer homology is formally the Morse homology of the pair(L̃X,AH

).

The critical points are pairs (x,w) where x : S1 → X satisfying x′(t) = XHt(x(t)), where XHt isthe Hamiltonian vector field associated to Ht; these loops are 1-periodic orbits of the Hamiltonian

isotopy generated by Ht. Then, choosing a smooth S1-family of ω-compatible almost complex

structures Jt on X which induces an L2-metric on the loop space of X, (1.1) is written as the Floer

equation for a map u from the infinite cylinder Θ = R× S1 to X, as∂u

∂s+ Jt

(∂u

∂t−XHt(u)

)= 0. (1.3)

Here (s, t) is the standard coordinates on Θ. This is a perturbed Cauchy-Riemann equation, so

Gromov’s theory of pseudoholomorphic curves is adopted in Floer’s theory.

There is always an issue of perturbing the equation in order to make the moduli spaces transverse,

so that the ambiguity of counting of solutions doesn’t affect the resulting homology. Floer originally

defined the Floer homology in the monotone case, which was soon extended by Hofer-Salamon

([19]) and Ono ([28]) to the semi-positive case. Finally by applying the “virtual technique”, Floer

homology is defined for general compact symplectic manifold by Fukaya-Ono ([15]) and Liu-Tian

([23]).

GAUGED FLOER HOMOLOGY 3

1.3. Hamiltonian Floer theory in gauged σ-model. In this paper, we consider a new type of

Floer homology theory proposed in [3] and motivated from Dostoglou-Salamon’s study of Atiyah-

Floer conjecture (see [6]). The main analytical object is the symplectic vortex equation, which was

also independently studied initially in [3] and by Ignasi Mundet in [25].

The symplectic vortex equation is a natural elliptic system appearing in the physics theory

“2-dimensional gauged σ-model”. Its basic setup contains the following ingredients:

(1) The target space is a triple (M,ω, µ), where (M,ω) is a symplectic manifold with a Hamil-

tonian G-action, and µ is a moment map of the action. We also choose a G-invariant,

ω-compatible almost complex structure J on M .

(2) The domain is a triple (P,Σ,Ω), where Σ is a Riemann surface, P → Σ is a smooth G-bundle, and Ω is an area form on Σ.

(3) The “fields” are pairs (A, u), where A is a smooth G-connection on P , and u : Σ→ P ×GMis a smooth section of the associated bundle.

Then we can write the system of equation on (A, u):

{∂Au = 0;

∗FA + µ(u) = 0.(1.4)

Here ∂Au is the (0, 1)-part of the covariant derivative of u with respect to A; FA is the curvature

2-form of A; ∗ is the Hodge star operator associated to the conformal metric on Σ with area formΩ; µ(u) is the composition of µ with u, which, after choosing a biinvariant metric on g, is identified

with a section of adP → Σ. This equation contains a symmetry under gauge transformations onP . Moreover, its solutions are minimizers of the Yang-Mills-Higgs functional:

YMH(A, u) := 12

(‖dAu‖2L2 + ‖FA‖

2L2 + ‖µ(u)‖

2L2

)(1.5)

which generalizes the Yang-Mills functional in gauge theory and the Dirichlet energy in harmonic

map theory.

Now, similar to Hamiltonian Floer theory, consider the following action functional on a covering

space of the space of contractible loops in M × g. Let H : M × S1 → R be an S1-family of G-invariant Hamiltonians; for any contractible loop x̃ := (x, f) : S1 → M × g with a homotopy classof extensions of x : S1 →M , represented by w : D→M , the action functional (given first in [3]) is

ÃH(x, f, w) := −∫Dw∗ω +

∫S1

(µ(x(t)) · f(t)−Ht(x(t))) dt. (1.6)

The critical loops of ÃH corresponds to periodic orbits of the induced Hamiltonian on the symplecticquotient M := µ−1(0)/G. The equation of negative gradient flows of ÃH , is just the symplecticvortex equation on the trivial bundle G × Θ, with the standard area form Ω = ds ∧ dt, and theconnection A is in temporal gauge (i.e., A has no ds component). If choosing an S1-family of

G-invariant, ω-compatible almost complex structures Jt, then the equation is written as a system

4 GUANGBO XU

of (u,Ψ) : Θ→M × g: ∂u

∂s+ Jt

(∂u

∂t+XΨ(u)− YHt

)= 0;

∂Ψ

∂s+ µ(u) = 0.

(1.7)

Solutions with finite energy are asymptotic to loops in CritÃH . Then the moduli space of suchtrajectories, especially those zero-dimensional ones, gives the definition of the boundary operator in

the Floer chain complex, and hence the Floer homology group. We call these homology theory the

vortex Floer homology. We have to use certain Novikov ring Λ, which will be defined in Section

2, as the coefficient ring, and the vortex Floer homology will be denoted by V HF (M,µ;H,J ; Λ).

The main part of this paper is devoted to the analysis about (1.7) and its moduli space, in order

to define V HF (M,µ;H,J ; Λ).

1.4. Lagrange multipliers. The action functional (1.6) seems to be already complicated, not to

mention its gradient flow equation (1.7). However, the action functional (1.6) is just a Lagrange

multiplier of the action functional (1.2). Indeed there is a much simpler situation in the case of

the Morse theory of a finite-dimensional Lagrange multiplier function, which is worth mentioning

in this introduction as a model.

Suppose X is a Riemannian manifold and µ : X → R is a smooth function, with 0 a regularvalue. Then consider a function f : X → R whose restriction to X = µ−1(0) is Morse. Thencritical points of f |X are the same as critical points of the Lagrange multiplier F : X × R → Rdefined by F (x, η) = f(x) + ηµ(x), and the Morse index as a critical point of f |X is one less thanthe index as a critical point of F . Then instead of considering the Morse-Smale-Witten complex

of f |X , we can consider that of F . In generic situation, these two chain complexes have the samehomology (with a grading shifting), and a concrete correspondence can be constructed through the

“adiabatic limit” (for details, see [33]).

Indeed, the vortex Floer homology proposed by Cieliebak-Gaio-Salamon and studied in this

paper is an infinite-dimensional and equivariant generalization of this Lagrange multiplier technique.

Therefore, the vortex Floer homology is expected to coincide with the ordinary Hamiltonian Floer

homology of the symplectic quotient (the proof of this correspondence will be treated in separate

work).

1.5. Advantage in achieving transversality. It seems that by considering the complicatd equa-

tion (1.7) and the moduli spaces we can only recover what we have known of the Hamiltonian Floer

theory of the symplectic quotient. But the trade-off is that the most crucial and sophisticated

step–transversality of the moduli space–can be achieved more easily. The advantage of lifting to

gauged σ-model is because, in many cases, M has simpler topology than M . So the issue caused by

spheres with negative Chern numbers is ruled out by topological reason. This phenomenon allows

us to achieve transversality of the moduli space by using the traditional “concrete perturbation” to

the equation. Moreover, when using virtual technique, the Floer homology group of the symplectic

quotient can only be defined over Q but here it can be defined over Z or Z2.

GAUGED FLOER HOMOLOGY 5

1.6. Computation of the Floer homology group and adiabatic limits. The ordinary Hamil-

tonian Floer homology HF (M,H) of a compact symplectic manifold can be shown to be canon-

ically independent of the Hamiltonian H and to be isomorphic to the singular homology of M .

This correspondence plays significant role in proving the Arnold conjecture. To prove this isomor-

phism, basically two methods have been used. One is to use a time-independent Morse function

as the Hamiltonian, and try to prove that when the function is very small in C2-norm, there is no

“quantum contribution” when defining the boundary operator in the Floer chain complex; this was

also Floer’s original argument. Another is via the Piunikhin-Salamon-Schwarz (PSS) construction,

introduced in [30].

For the case of the vortex Floer homology, it has been well-expected to be isomorphic to the

singular homology of the symplectic quotient M . To prove this isomorphism we can try the similar

methods as for the ordinary Hamiltonian Floer homology (which we will discuss in Section 6.3), as

well as the adiabatic limit method, which we discuss here.

Indeed, for any λ > 0, we consider a variation of (1.7)∂u

∂s+ Jt

(∂u

∂t+XΨ(u)− YHt

)= 0;

∂Ψ

∂s+ λ2µ(u) = 0.

(1.8)

This can be viewed as the symplectic vortex equation over the cylinder R × S1 with area formreplaced by λ2dsdt. The moduli space of solutions to the above equation also defines a Floer

homology group, and by continuation method we can show that this homology is (canonically)

independent of λ.

Then we would like to let λ approach to ∞. By a simple energy estimate, solutions of (1.8) will“sink” into the symplectic quotient M and become Floer trajectors of the induced pair (H, J); at

isolated points there will be energy blow up, and certain “affine vortices” will appear, which are

finite energy solutions to the symplectic vortex equation over the complex plane C. In the Gromov-Witten setting, the work of Gaio-Salamon [16] shows that (in special cases), the Hamiltonian-

Gromov-Witten invariants with low degree insertions coincide with the Gromov-Witten invariants

of the symplectic quotient, via the Kirwan map κ : H∗G(M) → H∗(M). The high degree partshall be corrected, by the contribution from the affine vortices. This leads to the definition of the

“quantum Kirwan map” (see [38], [35]).

In the case of vortex Floer homology, as long as we can carefully analyze the contribution of

affine vortices (maybe with similar restriction on M as in [16]), we could prove that V HF (M,µ;H)

is isomorphic to HF (M ;H), with appropriate changes of coefficients.

It is an interesting topic to consider the reversed limit λ→ 0, and it actually motivated the workof the author with S. Schecter [33], where they considered the nonequivariant, finite dimensional

Morse homology. In [33] it was shown that, the Morse-Smale trajectories, as λ→ 0, will converge tocertain “fast-slow” trajectories, and the counting of such trajectories defines a new chain complex,

which also computes the same homology.

1.7. Gauged Floer theory for Lagrangian intersections. In Frauenfelder’s thesis and [12], he

used the symplectic vortex equation on the strip R× [0, 1] to define the “moment Floer homology”

6 GUANGBO XU

for certain types of pairs of Lagrangians (L0, L1) in M . The Lagrangians are not G-invariant

in general, but their intersections with µ−1(0) reduce to a pair of Lagrangians (L0, L1) in the

symplectic quotient M . Then by the calculation in the Morse-Bott case, he managed to prove the

Arnold-Givental conjecture with certain topological assumption on M .

Woodward also defined a version of gauged Floer theory in [36], where he considered a pair of

Lagrangians L0, L1 in the symplectic quotient M . They lift to a pair of G-invariant Lagrangians

L0, L1 ⊂ µ−1(0) ⊂M . Then his equation for connecting orbits is the naive limit of the symplecticvortex equation on the strip R × [0, 1], by setting the area form to be zero. Since the strip iscontractible, the equation is just the J-holomorphic equation on the strip, and two solutions are

regarded equivalent if they differ by a constant gauge transformation. Then he applied this Floer

theory to the fibres of the toric moment map for any toric orbifold and showed the relation between

the nondisplacibility of toric fibres and the Hori-Vafa potential, which reproduces and extends the

results of Fukaya et. al. [13] [14].

Both of the above take advantage of the simpler topology of M than the symplectic quotient, as

we mentioned above, to avoid certain virtual technique. Further work are expected to relate the

Hamiltonian gauged Floer theory we studied here and the Lagrangian versions, for example, by

constructing the so-called “open-close map”.

1.8. Organization and conventions of this paper. In Section 2 we give the basic setup, includ-

ing the action functional, the definition of the Floer chain complex and the equation of connecting

orbits. In Section 3 we proved that each finite energy solution is asymptotic to critical loops of the

action functional. In Section 4 we study the Fredholm theory of the equation of connecting orbits

(modulo gauge transformations); we show that the linearized operator is a Fredholm operator whose

index is equal to the difference of Conley-Zehnder indices of the two ends of the connecting orbit. In

Section 5 we prove that our moduli space is compact up to breaking, if assuming the nonexistence

of nontrivial holomorphic spheres. In Section 6 we summarize the previous constructions and give

the definition of the vortex Floer homology (where we postpone the proof of transversality). We

also prove the invariance of the homology group by using continuation method. In the final Section

we give some discussions on our further work along this line.

In the appendices we provide detailed proof of several technical theorems. Most importantly,

we showed that by using concrete perturbation of the almost complex structure, we can achieve

transversality of the moduli space, which allows us to avoid the more sophisticated virtual technique.

We use Θ to denote the infinite cylinder R×S1, with the axial coordinate s and angular coordinatet. We denote Θ+ = [0,+∞)× S1 and Θ− = (−∞, 0]× S1.G is a connected compact Lie group, with Lie algebra g. Any G-bundle over Θ is trivial, and we

just consider the trivial bundle P = G×Θ. Any connection A can be written as a g-valued 1-formon Θ. We always use Φ to denote its ds component and Ψ to denote its dt component.

There is a small � > 0 such that for the �-ball g∗� ⊂ g∗ centered at the origin of g∗, U� := µ−1(g∗� )can be identified with µ−1(0)× g∗� . We denote by πµ : U� → µ−1(0) the projection on the the firstcomponent, and by πµ : U� →M the composition with the projection µ−1(0)→M .

GAUGED FLOER HOMOLOGY 7

1.9. Acknowledgments. The author would like to thank his PhD advisor Gang Tian for introduc-

ing him to this field and the support and encouragement. He would like to thank Urs Frauenfelder,

Kenji Fukaya, Chris Woodward, and Weiwei Wu for many helpful discussions and encouragement.

2. Basic setup and outline of the construction

Let (M,ω) be a symplectic manifold. We assume that it is aspherical, i.e., for any smooth map

f : S2 → M ,∫S2f∗ω = 0. This implies that for any ω-compatible almost complex structure J on

M , there is no nonconstant J-holomorphic spheres.

Let G be a connected compact Lie group which acts on M smoothly. The infinitesimal action g 3ξ 7→ Xξ ∈ Γ(TM) is an anti-homomorphism of Lie algebra. We assume the action is Hamiltonian,which means that there exists a smooth function µ : M → g∗ satisfying

µ(gx) = µ(x) ◦Ad−1g , ∀ξ ∈ g, d (µ · ξ) = ιXξω. (2.1)

Suppose we have a G-invariant, time-dependent Hamiltonian H ∈ C∞c(M × S1

)with compact

support. For each t ∈ S1, the associated Hamiltonian vector field YHt ∈ Γ(TM) is determined by

ω(YHt , ·) = dHt ∈ Ω1(M). (2.2)

The flow of YHt is a one-parameter family of diffeomorphisms

φHt : M →M,dφHt (x)

dt= YHt

(φHt (x)

)(2.3)

which we call a Hamiltonian path.

To achieve transversality, we put the following restriction on Ht. It will be used only in the

appendix.

Hypothesis 2.1. There exists a nonempty interval I ⊂ S1 such that Ht ≡ 0 for t ∈ I.

On the other hand, we need to put several assumptions to the given structures, which are still

general enough to include the most important cases (e.g., toric manifolds as symplectic quotients

of Euclidean spaces).

Hypothesis 2.2. We assume that µ : M → g∗ is proper, 0 ∈ g∗ is a regular value of µ and theG-action restricted to µ−1(0) is free.

With this hypothesis, µ−1(0) is a smooth submanifold of M and the symplectic quotient M :=

µ−1(0)/G is a symplectic manifold, which has a canonically induced symplectic form ω. Also, the

Hamiltonian function Ht descends to a time-dependent Hamiltonian

Ht : M → R (2.4)

by the G-invariance of Ht. It is easy to check that YHt is tangent to µ−1(0) and the projection

µ−1(0)→M pushes YHt forward to YHt . Then we assume

Hypothesis 2.3. The induced Hamiltonian Ht : M → R is nondegenerate in the usual sense.

Finally, in the case when M is noncompact, we need the convexity assumption (cf. [2, Definition

2.6]).

8 GUANGBO XU

Hypothesis 2.4. There exists a pair (f, J), where f : M → [0,+∞) is a G-invariant and properfunction, and J is a G-invariant, ω-compatible almost complex structure on M , such that there

exists a constant c0 > 0 with

f(x) ≥ c0 =⇒ 〈∇ξ∇f(x), ξ〉+ 〈∇Jξ∇f(x), Jξ〉 ≥ 0, df(x) · JXµ(x) ≥ 0, ∀ξ ∈ TxM. (2.5)

In this paper, to achieve transversality, we need to perturb J near µ−1(0) (see the appendix).

The above condition is only about the behavior “near infinity”, so such perturbations don’t break

the hypothesis.

2.1. Equivariant topology.

2.1.1. Equivariant spherical classes. Recall that the Borel construction of M acted by G is MG :=

EG ×G M , where EG → BG is a universal G-bundle over the classifying space BG. Then theequivariant (co)homology of M is defined to be the ordinary (co)homology of MG, denoted by

HG∗ (M) for homology and H∗G(M) for cohomology.

On the other hand, for any smooth manifold N , we denote by S2(N) to be the image of the

Hurwitz map π2(N) → H2(N ;Z), and classes in S2(N) are called spherical classes. We define theequivariant spherical homology of M to be SG2 (M) := S2(MG).

Geometrically, any generator of SG2 (M) can be represented by the following object: a smooth

principal G-bundle P → S2 and a smooth section φ : S2 → P ×GM . We denote the class of thepair (P, φ) to be [P, φ] ∈ SG2 (M).

2.1.2. Equivariant symplectic form and equivariant Chern numbers. The equivariant cohomology

of M can also be computed using the equivariant de Rham complex(Ω∗(M)G, dG

). In Ω2(M)G,

there is a distinguished closed form ω−µ, called the equivariant symplectic form, which representsan equivariant cohomology class.

We are interested in the pairing 〈[ω − µ], [P, u]〉 ∈ R. It can be computed in the following way.Choose any smooth connection A on P . Then there exists an associated closed 2-form ωA on

P ×G M , called the minimal coupling form. If we trivialize P locally over a subset U ⊂ S2,such that A = d+ α, α ∈ Ω1(U, g) with respect to this trivialization, then ωA can be written as

ωA = π∗ω − d(µ · α) ∈ Ω2(U ×M). (2.6)

Then we have

〈[ω − µ], [P, u]〉 =∫S2u∗ωA (2.7)

which is independent of the choice of A.

On the other hand, any G-invariant almost complex structure J on X makes TX an equivariant

complex vector bundle. So we have the equivariant first Chern class

cG1 := cG1 (TM) ∈ H2G(M ;Z). (2.8)

This is independent of the choice of J .

GAUGED FLOER HOMOLOGY 9

2.1.3. Kirwan maps. The cohomological Kirwan map is a map

κ : H∗(M ;R)→ H∗(M ;R). (2.9)

Here we take R-coefficients for simplicity. It is easy to check that

κ([ω − µ]) = [ω] ∈ H2(M ;R), κ(cG1 ) = c1(TM) ∈ H2(M ;R). (2.10)

We define

NG2 (M) = ker[ω − µ] ∩ kercG1 ⊂ SG2 (M), N2(M) = ker[ω] ∩ kerc1(TM) ⊂ S2(M), (2.11)

and

Γ := SG2 (M)/NG2 (M). (2.12)

2.2. The spaces of loops and equivalence classes. Let P̃ be the space of smooth contractibleparametrized loops in M × g and a general element of P̃ is denoted by

x̃ := (x, f) : S1 →M × g. (2.13)

Let P̃ be a covering space of P̃, consisting of triples x := (x, f, [w]) where x̃ = (x, f) ∈ P and[w] is an equivalence class of smooth extensions of x to the disk D. The equivalence relation isdescribed as follows. For each pair w1, w2 : D → M both bounding x : S1 → M , we have thecontinuous map

w12 := w1#(−w2) : S2 →M (2.14)

by gluing them along the boundary x. We define

w1 ∼ w2 ⇐⇒ [w12] = 0 ∈ SG2 (M). (2.15)

Denote by LG := L∞G := C∞(S1, G) the smooth free loop group of G. Then for any point

x0 ∈M , we have the homomorphism

l(x0) : π1(G)→ π1(M,x0) (2.16)

which is induced by mapping a loop t 7→ γ(t) ∈ G to a loop t 7→ γ(t)x0 ∈ M . For differentx1 ∈M and a homotopy class of paths connecting x0 and x1, we have an isomorphism π1(M,x0) 'π1(M,x1); it is easy to see that l(x0) and l(x1) are compatible with this isomorphism. This means

kerl(x0) ⊂ π1(G) is independent of x0. Then we define

LMG :={γ : S1 → G | [γ] ∈ kerl(x0) ⊂ π1(G)

}. (2.17)

Let L0G ⊂ LMG be the subgroup of contractible loops in G.It is easy to see that LMG acts on P̃ (on the right) by

P̃ × LMG → P̃((x, f), h) 7→ h∗(x, f)(t) =

(h(t)−1x(t),Ad−1h(t)(f(t)) + h(t)

−1∂th(t)).

(2.18)

Here the action on the second component can be viewed as the gauge transformation on the space

of G-connections on the trivial bundle S1×G. (For short, we denote by d log h the g-valued 1-formh−1dh, which is the pull-back by h of the left-invariant Maurer-Cartan form on G.)

10 GUANGBO XU

But LMG doesn’t act on P̃ naturally; only the subgroup L0G does: for a contractible loop

h : S1 → G, extend h arbitrarily to h : D→ G. The homotopy class of extensions is unique becauseπ2(G) = 0 for any connected compact Lie group ([1]). Then the class of (h

−1x, h∗f, [h−1w]) in P̃

is independent of the extension. It is easy to see that the covering map P̃→ P̃ is equivariant withrespect to the inclusion L0G→ LMG. Hence it induces a covering

P̃/L0G→ P̃/LMG. (2.19)

2.3. The deck transformation and the action functional. We now define an action of SG2 (M)

on P̃/L0G. Take a class A ∈ SG2 (M) represented by a pair (P, u), where P → S2 is a principalG-bundle and u : S2 → P ×GM is a section of the associated bundle.

Consider Un ' C∗ ∪ {∞} ⊂ S2 as the complement of the south pole 0 ∈ S2. Take an arbitrarytrivialization φ : P |Un → Un ×G, which induces a trivialization

φ : P ×GM |Un → Un ×M. (2.20)

Then φ ◦ u is a map from Un to M and there exists a loop h : S1 → G and x ∈M such that

limr→0

φ ◦ u(reiθ) = h(θ)x. (2.21)

Note that the homotopy class of h is independent of the trivialization φ and the choice of x.

Now, for any element (x, f, [w]) ∈ P̃, find a smooth path γ : [0, 1] → M such that γ(0) = w(0)and γ(1) = x. Then define γh : S

1 × [0, 1]→M by γ(θ, t) = h(θ)γ(t).On the other hand, view D \ {0} ' (−∞, 0]× S1. Consider the map

wh(r, θ) = h(θ)w(r, θ)

and the “connected sum”:

u#̃w := (φ ◦ u) #γh#wh : D→M (2.22)

which extends the loop

xh(θ) = h(θ)x(θ). (2.23)

Denote fh := Adhf − ∂th · h−1. Then we define the action by

A#[x, f, [w]] =[xh, fh, [u#̃w],

], ∀A ∈ SG2 (M). (2.24)

On the other hand, there exists a morphism

SG2 (M)→ kerl(x0) ⊂ π1(G) (2.25)

which sends the homotopy class of [P, u] to the homotopy class of h : S1 → G where h is the onein (2.21). Then it is easy to see the following.

Lemma 2.5. The action (2.24) is well-defined (i.e., independent of the representatives and choices)

and is the deck transformation of the covering P̃/L0G→ P̃/LMG.

GAUGED FLOER HOMOLOGY 11

Now by this lemma, we denote P :=(P̃/L0G

)/NG2 , which is again a covering of P := P̃/LMG,

with the group of deck transformations isomorphic to Γ. We will use x to denote an element in P̃

or P̃/L0G and use [x] an element in P.

We can define a 1-form B̃H on P̃ by

T(x,f)P̃ 3 (ξ, h) 7→∫S1{ω (ẋ(t) +Xf − YHt , ξ(t)) + 〈µ(x(t)), h(t)〉} dt ∈ R. (2.26)

Its pull-back to P̃ is exact and one of the primitives is the following action functional on P̃:

ÃH(x, f, [w]) := −∫Bw∗ω +

∫S1

(µ(x(t)) · f(t)−Ht(x(t))) dt. (2.27)

The zero set of the one-form B̃H consists of pairs (x, f) such that

µ(x(t)) ≡ 0, ẋ(t) +Xf(t)(x(t))− YHt(x(t)) = 0. (2.28)

The critical point set of ÃH are just the preimage of ZeroB̃H under the covering P̃→ P̃.

Lemma 2.6. ÃH is L0G-invariant and B̃H is LMG-invariant.

Proof. Take any h : S1 → G, extend it smoothly to some h : D→ G. Then we see that

(h−1w)∗ω = ω(∂x(h

−1w), ∂y(h−1w)

)dxdy = w∗ω + d

(µ(h−1w) · d log h

). (2.29)

Also, we see that

µ(h−1(t)x(t)) ·(

Ad−1h(t)f(t) + h(t)−1h′(t)

)= µ(x(t)) · f(t) +

(µ(h−1w) · d log h

)∣∣∂D . (2.30)

By Stokes’ theorem and the G-invariance of Ht, we obtain the invariance of ÃH . The LMG-invariance of B̃H follows from equivariance of the involved terms in a similar way. �

So we have the induced action functional ÃH : P̃/L0G→ R and it satisfies the following.

Lemma 2.7. For any [x] = [x, f, [w]] ∈ P̃/L0G and any A ∈ SG2 (M), we have

ÃH (A#[x]) = ÃH ([x])− 〈[ω − µ], A〉. (2.31)

Proof. Use the same notation as we define the action A#[x], we see that∫D\{0}

w∗hω =

∫Dw∗ω +

∫ 0−∞

ds

∫S1dtω (h∗∂sw, h∗X∂t log h(w))

=

∫Dw∗ω −

∫ 0−∞

∫ 10d (µ(w) · d log h) =

∫Dw∗ω −

∫S1

(µ(x(t))− µ(w(0)))d log h. (2.32)

Also ∫S1×[0,1]

γ∗hω = −∫S1

(µ(w(0))− µ(x)) d log h. (2.33)

In the same way we can calculate∫S2\{0}

(φ ◦ u)∗ ω = 〈[ω − µ], [P, u]〉 −∫S1µ(x)d log h. (2.34)

12 GUANGBO XU

So we have

ÃH (A#[x]) = −∫D

(u#̃w

)∗ω +

∫S1

{〈µ(h(t)x(t)),Adh(t)f(t)− h′(t)h(t)−1

〉−Ht(h(t)x(t))

}dt

= −〈[ω − µ], A〉 −∫Dw∗ω +

∫S1〈µ(x(t)), f(t)〉 −Ht(x(t))dt = −〈[ω − µ], A〉+ ÃH ([x]) (2.35)

�

This lemma implies that ÃH descends to a well-defined function

AH : P→ R. (2.36)

Our Floer theory will be formally a Morse theory of the pair (P,AH).Before we move on to the chain complex, we see that AH is a Lagrange multiplier function

associated to the action functional AH of the induced Hamiltonian H on the symplectic quotientM . Let P̃M be the space of contractible loops in M and let PM be pairs (x, [w]) where x ∈ P̃Mand w : D→M extends x; [w] = [w′] if (−w)#w′ is annihilated by both ω and c1(TM). Then forany (x, [w]) ∈ PM , we can pull-back the principal G-bundle µ−1(0)→ M to D. Any trivialization(or equivalently a section s) of this bundle over D induces a map ws : D→ µ−1(0) whose boundaryrestriction, denoted by x : S1 → µ−1(0), lifts x. Now, if (x, [w]) ∈ CritAH , i.e.

0 = x′(t)− YHt(x(t)) = (πµ)∗(x′(t)− YHt(x(t))

)(2.37)

there exists a smooth function, fs : S1 → g such that

x′(t) +Xfs(t)(x(t))− YHt(x(t)) = 0. (2.38)

Then this gives a map

ι : P̃M → P̃/L0G(x, [w]) 7→ [xs, fs, [ws]]

(2.39)

By the correspondence of the symplectic forms and Chern classes between upstairs and downstairs,

we have

Proposition 2.8. The class [xs, fs, [ws]] is independent of the choice of the section s and only

depends on the homotopy class of w. Moreover, it induces a map

ι :(PM ,CritAH

)→ (P,CritAH) (2.40)

2.4. The Floer chain complex. For R = Z2, Z or Q, we consider the Novikov ring of formalpower series over a base ring R to be the downward completion of R[Γ]:

ΛR := Λ↓R :=

{∑B∈Γ

λBqB

∣∣∣∣∣ ∀K > 0,# {B ∈ Γ | 〈[ω − µ], B〉 > K, λB 6= 0}

GAUGED FLOER HOMOLOGY 13

2.5. Gradient flow and symplectic vortex equation. Now we choose an S1-family of G-

invariant, ω-compatible almost complex structures J on M , we assume that

f(x) ≥ c0 =⇒ Jt(x) = J(x) (2.43)

where (f, J) is the convex structure which we assume to exist in Hypothesis 2.4. Then ω and Jt

defines a Riemannian metric on M , which induces an L2-metric g̃ on the loop space LM . On the

other hand, we fix a biinvariant metric on the Lie algebra g which induces a metric g2 on Lg; it

also identifies g with g∗ and we use this identification everywhere in this paper without mentioning

it. These choices induce a metric on P.

Then, it is easy to see that formally, the equation for the negative gradient flow line of AH isthe following equation for a pair (u,Ψ) : Θ→M × g

∂u

∂s+ Jt

(∂u

∂t+XΨ − YHt

)= 0,

∂Ψ

∂s+ µ(u) = 0.

(2.44)

The equation is invariant under the action of LG. The action is defined by

LG×Map (Θ,M × g) → Map (Θ,M × g)g∗(u,Ψ)(s, t) =

(g(t)−1u(s, t),Ad−1g(t)Ψ(s, t) + g(t)

−1∂tg(t)) (2.45)

Definition 2.9. The energy for a flow line ũ (or its Yang-Mills-Higgs functional) is defined to be

E (ũ) := E (u,Ψ) :=1

2‖du+ (XΨ − YHt)⊗ dt‖

2L2 +

1

2

∥∥∥∥∂Ψ∂s∥∥∥∥2L2

+1

2‖µ(u)‖2L2 . (2.46)

The second equation of (2.44) is actually the symplectic vortex equation on the triple (P, u,Ψdt),

where P → Θ is the trivial G-bundle, u is a section of P×GM and Ψdt corresponds to the covariantderivative d+ Ψdt. (For a detailed introduction to the symplectic vortex equation, see [3] or [25]).

This is why we name our theory the vortex Floer homology.

The connection d+ Ψdt has already been put in the temporal gauge, i.e., it has no ds compo-

nent. A more general equation on pairs (u, α), with α = Φds+ Ψdt ∈ Ω1(Θ)⊗ g is∂u

∂s+XΦ + Jt

(∂u

∂t+XΨ − YHt

)= 0,

∂Ψ

∂s− ∂Φ∂t

+ [Φ,Ψ] + µ(u) = 0.

(2.47)

We write the object (u, α) in the form of a triple ũ = (u,Φ,Ψ). The above equation is invariant

under the action by GΘ := C∞ (Θ, G), which is defined by

GΘ ×Map (Θ,M × g× g) → Map (Θ,M × g× g)

g∗

uΦΨ

(s, t) = g(s, t)

−1u(s, t)

Ad−1g(s,t)Φ(s, t) + g(s, t)−1∂sg(s, t)

Ad−1g(s,t)Ψ(s, t) + g(s, t)−1∂tg(s, t)

(2.48)

14 GUANGBO XU

A solution ũ = (u,Φ,Ψ) to (2.47) is called a generalized flow line. The energy of a generalized

flow line is

E(u,Φ,Ψ) =1

2‖du+XΦ ⊗ ds+ (XΨ − YHt)⊗ dt‖L2 +

1

2‖µ(u)‖2L2 +

1

2‖∂sΨ− ∂tΦ + [Φ,Ψ]‖2L2 .

(2.49)

It is easy to see that any smooth generalized flow line is gauge equivalent via a gauge transformation

in GΘ to a smooth flow line in temporal gauge; and the energy is gauge independent.

2.6. Moduli space and the formal definition of the vortex Floer homology. We focus on

solutions to (2.47) with finite energy. In Section 3 we will show that, any such solution is gauge

equivalent to a solution ũ = (u,Φ,Ψ) such that there exists a pair x̃± = (x±, f±) ∈ ZeroB̃H with

lims→±∞

Φ(s, t) = 0, lims→±∞

(u(s, ·),Ψ(s, ·)) = x̃±. (2.50)

Hence for any pair [x±] ∈ CritAH , we can consider solutions which “connect” them. We denote by

M ([x−], [x+]; J,H) (2.51)

the moduli space of all such solutions, modulo gauge transformation.

In the appendix we will show that, if we assume that Ht vanishes for t in a nonempty open

subset I ⊂ S1, and J a generic, “admissible” family of almost complex structures, then the spaceM ([x−], [x+]; J,H) is a smooth manifold, whose dimension is equal to the difference of the Conley-Zehnder indices of [x±]. Moreover, assuming that the manifold M cannot have any nonconstant

pseudoholomorphic spheres, we show thatM ([x−], [x+]; J,H) is compact modulo breakings. Finally,there exist coherent orientations on different moduli spaces, which is similar to the case of ordinary

Hamiltonian Floer theory (see [10]). Then, the signed counting of isolated gauged equivalence

classes of trajectories in our case has exactly the same nature as in finite-dimensional Morse-Smale-

Witten theory, which defines a boundary operator

δJ : V CF∗(M,µ;H; ΛZ)→ V CF∗−1(M,µ;H; ΛZ). (2.52)

The vortex Floer homology is then defined

V HF∗ (M,µ; J,H; ΛZ) = H (V CF∗(M,µ;H; ΛZ), δJ) . (2.53)

Moreover, for a different choice of the pair (J ′, H ′), we can use continuation principle to prove

that the chain complex (V CF∗(M,µ;H′; ΛZ), δJ ′) is chain homotopic to (V CF∗(M,µ;H; ΛZ); δJ).

There is a canonical isomorphism between the homologies, and we denote the common homology

group by V HF∗(M,µ; ΛZ). The details are given in Section 6 and the appendix.

3. Asymptotic behavior of the connecting orbits

In this section we analyze the asymptotic behavior of solutions ũ = (u,Φ,Ψ) to (2.47) which has

finite energy and for which u(Θ) has compact closure in M . We call such a solution a bounded

solution. We denote the space of bounded solutions by M̃bΘ. We can also consider the equationon the half cylinder Θ+ or Θ− and denote the spaces of bounded solutions over Θ± by M̃bΘ± .

The main theorem of this section is

GAUGED FLOER HOMOLOGY 15

Theorem 3.1. (1) Any (u,Φ,Ψ) ∈ M̃bΘ± is gauge equivalent (via a smooth gauge transforma-tion g : Θ± → G) to a solution (u′,Φ′,Ψ′) ∈ M̃bΘ± such that there exist x̃± = (x±, f±) ∈ZeroB̃H and

lims→±∞

(u′(s, ·),Ψ′(s, ·)

)= x̃±, lim

s→±∞Φ(s, ·) = 0 (3.1)

uniformly for t ∈ S1.(2) There exists a compact subset KH ⊂M such that for any (u,Φ,Ψ) ∈ M̃bΘ, we have u(Θ) ⊂

KH .

We will prove (1) for ũ ∈ M̃bΘ+ in temporal gauge, i.e., Φ ≡ 0 and the case for Θ− is the same.Then (2) follows from a maximum principle argument, given at the end of this section. The proof

is based on estimates on the energy density, which has been given by others in several different

settings (see [2], [16]). The only possibly new ingredient is that we have a nonzero Hamiltonian

here.

3.1. Covariant derivatives. The S1-family of metrics gt := ω(·, Jt·) induces a metric connection∇ on the bundle u∗TM . Moreover, we define

∇A,sξ = ∇sξ +∇ξXΦ, ∇A,tξ = ∇tξ +∇ξXΨ. (3.2)

Also, on the trivial bundle Θ× g, define the covariant derivative

∇A,sθ = ∇sθ + [Φ, θ], ∇A,tθ = ∇tθ + [Ψ, θ]. (3.3)

We denote by ∇A the direct sum connection on u∗TM × g. Note that it is compatible with respectto the natural metric on this bundle.

Define the g-valued 2-form ρt on M by

〈ρ(ξ1, ξ2), η〉t = 〈∇ξ1Xη, ξ2〉t = −〈∇ξ2Xη, ξ1〉t, ξi ∈ TM, η ∈ g. (3.4)

We list several useful identities of this covariant derivative. The reader may refer to [16] for

details.

Lemma 3.2. For ũ in temporal gauge, we have the following equalities

∇A,sXη −X∇A,sη = ∇∂suXη, ∇A,tXη −X∇A,tη = ∇∂tu+XΨXη. (3.5)

∇A,s (∂tu+XΨ)−∇A,t∂su = X∂tΨ. (3.6)

∇A,s (dµ · Jξ)− dµ · J (∇A,sξ) = ρ(∂su, ξ), ∇A,t (dµ · Jξ)− dµ · J (∇A,tξ) = ρ(∂tu+XΨ, ξ).(3.7)

On the other hand, since our equation is perturbed by a Hamiltonian H, we consider another

covariant derivative which takes H into account. Let φtH be the Hamiltonian isotopy defined by Ht

and

JHt :=(dφtH

)−1 ◦ Jt ◦ dφtH , t ∈ Rbe the 1-parameter family of almost complex structures. They are still compatible with ω and

hence defines a family of metric gHt , with induced inner product denoted by 〈·, ·〉H,t. We denotethe induced metric connection on u∗TM by ∇H .

16 GUANGBO XU

We think (u,Ψ) as a map from R× R→M × g, periodic in the second variable. Then we havea well-defined connection D on u∗TM → R× R, given by

(Dsξ) (s, t) =(∇Hs ξ

)(s, t), (Dtξ) (s, t) = ∇Ht ξ +∇Hξ XΨ −∇Hξ YHt . (3.8)

Lemma 3.3. D is compatible with the inner product 〈·, ·〉H,t on u∗TM .

Proof. By direct calculation, we see for ξ, η ∈ Γ(u∗TM),

∂s〈ξ, η〉H,t = 〈Dsξ, η〉H,t + 〈ξ,Dsη〉H,t,

∂t〈ξ, η〉H,t =dgtdt

(ξ, η) +〈∇gtt ξ +∇

gtξ XF , η

〉H,t

+〈ξ,∇gtt η +∇gtη XF

〉Ht

= −(LYHtgt

)(ξ, η) +

〈∇gtt ξ +∇

gtξ XF , η

〉H,t

+〈ξ,∇gtt η +∇gtη XF

〉Ht

= 〈Dtξ, η〉H,t + 〈ξ,Dtη〉H,t .

�

3.2. Estimate of the energy density.

Lemma 3.4. There exist positive constants c1 and c2 depending only on (X,ω, J, µ,Ht) and the

subset Ku, such that for any flow line (u, 0,Ψ) in temporal gauge, we have

∆(|∂su|2H,t

)≥ −c1|∂su|4H,t − c2. (3.9)

Proof. First we see that

1

2∆ |∂su|2H,t =

(∂2s + ∂

2t

)|∂su|2H,t = ∂s 〈Ds∂su, ∂su〉H,t + ∂t〈Dt∂su, ∂su〉H,t

= |Ds∂su|2H,t + |Dt∂su|2H,t +

〈(D2s +D

2t

)∂su, ∂su

〉H,t

.(3.10)

We denote vs = ∂su ∈ Γ(R2, u∗TM) and vt = ∂tu+XΨ − YHt ∈ Γ(R2, u∗TM). Hence vs = −Jtvt.Then we have the following computation

Dsvt −Dtvs=∇Hs (∂tu+XΨ − YHt)−∇Ht ∂su+∇H∂su (−XΨ + YHt) = X∂sΨ = −Xµ(u).

(3.11)

Dsvs +Dtvt

=Ds(−Jtvt) +Dt(Jtvs)

=−∇Hs (Jtvt) +∇Ht (Jtvs) +∇HJvs(XΨ − YHt)

=−∇Hs (Jtvt) +∇Ht (Jtvs) + [Jtvs, XΨ − YHt ] +∇HXΨ−YHt (Jtvs)

=−∇Hs (Jvt) +∇Hvt(Jtvs) + J̇tvs + [Jtvs, XΨ − YHt ]

=− (∇HvsJt)vt − Jt∇Hs vt + (∇HvtJt)vs + Jt∇

Hvtvs + J̇tvs + Jt [vs, XΨ] + [YHt , Jtvs]

=− (∇HvsJt)vt + (∇HvtJt)vs − Jt[vs, vt]− JX∂sΨ + J̇tvs + Jt[vs, XΨ] + Jt [YHt , vs] + (LYHtJt)vs

=− (∇HvsJt)vt + (∇HvtJt)vs + (LYHtJt)vs + JtXµ + J̇tvs.

(3.12)

GAUGED FLOER HOMOLOGY 17

On the other hand, for any (s, t) ∈ Σ, any ξ ∈ Tu(s,t)M , we extend ξ and vs(s, t) to be G-invariantvector fields locally. Then for the Riemann curvature tensor associated to JHt , we have

RH(vs, XΨ)ξ = ∇Hvs∇HXΨξ −∇HXΨ∇

Hvsξ −∇

H[vs,XΨ]

ξ = ∇Hvs∇Hξ XΨ −∇H∇HvsξXΨ. (3.13)

Hence

(DsDt −DtDs) ξ

= ∇Hs(∇Ht ξ +∇Hξ (XΨ − YHt)

)−∇Ht (∇Hs ξ)−∇H∇Hs ξ(XΨ − YHt)

= RH(vs, ∂tu)ξ +∇Hs ∇Hξ (XΨ − YHt)−∇H∇Hs ξ(XΨ − YHt)−(d

dt∇Hs)ξ

= RH(vs, ∂tu+XΨ)ξ −∇Hs ∇Hξ YHt +∇H∇Hs ξYHt +∇Hξ X∂sΨ −

(d

dt∇Hs)ξ

= RH(vs, vt + YHt)ξ −∇Hs ∇Hξ YHt +∇H∇Hs ξYHt −∇Hξ Xµ −

(d

dt∇Hs)ξ. (3.14)

The third equality above uses (3.13).

Then we denote

D2svs +D2t vs = Ds (Dsvs +Dtvt) + (DtDs −DsDt) vt −Dt (Dsvt −Dtvs) =: Q1 +Q2 +Q3.

By (3.12),

〈Q1, vs〉H,t =〈∇Hs

(−(∇HvsJt

)vt +

(∇HvtJt

)vs +

(LYHtJt

)vs + JtXµ + J̇tvs

), vs

〉H,t

≥ −C1(|vs|3H,t + |vs|2H,t + |vs|4H,t + |vs|2H,t

∣∣∇Hs vs∣∣H,t + |vs|H,t ∣∣∇Hs vs∣∣H,t) (3.15)for some C1 > 0. Here we used the fact that µ and dµ are uniformly bounded because u(Θ+) has

compact closure.

By (3.14), for some C2 > 0, we have

〈Q2, vs〉Ht =〈−RH(vs, vt + YH,t)vt +∇Hs ∇HvtYHt −∇

H∇Hs vt

YHt +∇HvtXµ +(d

dt∇Hs)vt, vs

〉H,t

≥ −C2(|vs|4H,t + |vs|3H,t + |vs|2H,t + |vs|H,t

∣∣∇Hs vs∣∣H,t) . (3.16)By (3.11), for some C3 > 0, we have

〈Q3, vs〉H,t= 〈DtXµ, vs〉H,t

=〈∇Ht Xµ +∇HXµ(XΨ − YHt), vs

〉H,t

=〈Xdµ·∂tu +∇HvtXµ + [Xµ, XΨ − YHt ], vs

〉=〈Xdµ·∂tu +∇HvtXµ −X[µ,Ψ], vs

〉=〈Xdµ·vt −Xdµ·XΨ +∇

HvtXµ −X[µ,Ψ], vs

〉H,t

=〈Xdµ·vt +∇HvtXµ, vs〉H,t ≥ −C3|vs|2H,t.

(3.17)

18 GUANGBO XU

Hence for some C4 > 0 and c1, c2 > 0, we have

1

2∆|vs|2H = |Dsvs|2H + |Dtvt|2H + 〈Q1 +Q2 +Q3, vs〉H ≥ |∇gts vs|2H + 〈Q1 +Q2 +Q3, vs〉H

≥ |∇gts vs|2H − C4

(4∑i=2

|vs|iH +2∑i=1

|vs|iH |∇gts vs|H

)≥ −c1|vs|4H − c2. (3.18)

�

Now we consider the other part of the energy density. We have the following calculation:

1

2∆ |µ(u)|2 = |∇A,sµ|2 + |∇A,tµ|2 + 〈∇A,s∇A,sµ+∇A,t∇A,tµ, µ〉. (3.19)

And (see [16, Lemma C.2])

∇A,s∇A,sµ(u) +∇A,t∇A,tµ(u)

=∇A,sdµ · vs +∇A,tdµ · vt = ∇A,t (dµ · Jvs)−∇A,s (dµ · Jvt)

=− 2ρ(vs, vt) + dµ · Jt (∇A,tvs −∇A,svt) + dµ(J̇tvs

)=dµ ·

(JtXµ + J̇tvs

)− 2ρ(vs, vt).

(3.20)

Since u(Θ) has compact closure, we may assume that supu(Θ) |ρ| ≤ cu. Hence there exists c3, c4 > 0such that

∆ |µ(u)|2 ≥ −c3 − c4 |vs|4 . (3.21)

3.3. Decay of energy density. To proceed, we quote the following lemma (cf. [32, Page 12]).

Lemma 3.5. Let Ω ⊂ R2 be an open subset containing the origin and e is defined over Ω. Supposeit satisfies

∆e ≥ −A−Be2,

then ∫Br(0)

e ≤ π16B

=⇒ e(0) ≤ 8πr2

∫Br(0)

e+Ar2

4. (3.22)

Apply the above lemma to the function |vs|2H,t+|µ(u)|2 and note that different norms are actuallyequivalent, we see

Proposition 3.6. If (u,Ψ) satisfies the equation (2.44) such that u(Θ) has compact closure and

E(u,Ψ)

GAUGED FLOER HOMOLOGY 19

3.4. Approaching to periodic orbits.

Proposition 3.7. Any (u, 0,Ψ) ∈ M̃bΘ+ is gauge equivalent to a solution to (2.47) (v,Φ,Ψ) onΘ+ with the following properties

(1) (v,Ψ)|{s}×S1 converges in C0 to an element of ZeroB̃H as s→ +∞;

(2) lims→+∞

Φ(s, ·) = 0 uniformly in t.

Proof. By the above proposition, we know that finite energy implies that µ(u(s, ·))→ 0 as s→ +∞.Let g∗� ⊂ g∗ be the �-open ball of the origin. Then there is an equivariant symplectic diffeomorphism

U� := µ−1(g∗� )→ µ−1(0)× g∗� . (3.23)

Then µ|U� is just the projection of the right hand side of (3.23) onto the second factor (see forexample [18]). Let πµ be the projection on to the first factor, then we define the almost complex

structure

J0,t := π∗µ

(Jt∣∣µ−1(0)

)and the vector field

Y0,Ht := π∗µ

(YHt

∣∣µ−1(0)

).

Then there exists K1 > 0 depending on (M,ω, J, µ,Ht) such that

‖J0,t(x)− Jt(x)‖ ≤ K1|µ(x)|, ‖Y0,Ht(x)− YHt(x)‖ ≤ K1|µ(x)|, ∀x ∈ U�.

We denote πµ : µ−1(g∗� ) → M the composition of πµ with the projection µ−1(0) → M . For

(u, 0,Ψ) ∈ M̃bΘ+ , we have proved that µ(u) converges to 0 uniformly as s → +∞. Hence forN := N(�) sufficiently large, u(s, ·) maps ΘN+ := [N,+∞)× S1 into U�. So on ΘN+ , we have

∂su+ J0,t (∂tu+XΨ(u)− Y0,Ht(u)) = (J0,t − Jt) (∂tu+XΨ(u)− Y0,Ht) + Jt (YHt − Y0,Ht) .(3.24)

Denoting u := πµ ◦ u : ΘN+ →M and applying (πµ)∗ to the above equality, we see on ΘN+ ,∥∥∂su+ J t (∂tu− YHt(u))∥∥ ≤ K2� (3.25)

for some constant K2. Here J t is the induced almost complex structure on M . Hence for s ≥ N ,the family of loops u(s, ·) in the quotient will be close (in C0) to some 1-periodic orbits γ : S1 →Mof YHt .

We take a lift p ∈ µ−1(0) with ππµ(p) = γ(0). We see that there exists a unique gp ∈ G suchthat

φ1Hp = gpp. (3.26)

Suppose gp = exp ξp, ξp ∈ g. It is easy to see that the loop

(x(t), f(t)) :=(exp(−tξp)φtH(p), ξp

)(3.27)

is an element of ZeroB̃H . We will construct a gauge transformation g̃ on Θ+ and show thatg̃∗(u, 0,Ψ) satisfies the condition stated in this proposition.

20 GUANGBO XU

Take a local slice of the G-action near p. That is, an embedding i : B2n−2kδ → µ−1(0) where

B2n−2kδ is the δ-ball in R2n−2k such that i(0) = p and (y, g) 7→ g(i(y)) is a diffeomorphism from

B2n−2kδ ×G onto its image.Denote u(s, t) = (v(s, t), ξ(s, t)) ∈ µ−1(0) × g∗ with respect to the decomposition (3.23). Then

for s large enough, there exists a unique g(s) ∈ G such that

g(s)v(s, 0) ∈ i(B2n−2kδ

), g(s)v(s, 0)→ p. (3.28)

Moreover, by the fact that |∂su| converges to zero, we see

lims→+∞

∣∣g(s)−1ġ(s)∣∣ = 0. (3.29)Define hs(t) ∈ G by

hs(0) = 1, hs(t)−1∂hs(t)

∂t= Ψ(s, t).

Then by the fact that lims→+∞ |∂sΨ| = 0 we see that

lims→∞

|∂s log hs(t)| = 0. (3.30)

Thus we have

d(gpp, g(s)hs(1)g(s)

−1p)

≤ d(gpp, φ

1Hg(s)v(s, 0)

)+ d

(φ1Hg(s)v(s, 0), g(s)hs(1)v(s, 0)

)+ d

(g(s)hs(1)v(s, 0), g(s)hs(1)g(s)

−1p)

= d(gpp, φ

1Hg(s)v(s, 0)

)+ d

(φ1Hv(s, 0), hs(1)v(s, 0)

)+ d (g(s)v(s, 0), p)

=: d1(s) + d2(s) + d3(s).

(3.31)

Here d is the G-invariant distance function induced by an invariant Riemannian metric. By (3.26)

and (3.28), we have d1(s) + d3(s)→ 0. By the decay of energy density, i.e.,

lims→+∞

supt|∂tu+XΨ − YHt | = 0,

we have d2(s)→ 0. Hence we have

lims→+∞

d(gpp, g(s)hs(1)g(s)

−1p)

= 0. (3.32)

Since the G-action on µ−1(0) is free, we have

lims→+∞

g(s)hs(1)g(s)−1 = gp (3.33)

Then by (3.33), there exists a continuous curve ξ(s) ∈ g defined for large s, such that

g(s)hs(1)g(s)−1 = exp ξ(s), lim

s→+∞ξ(s) = ξp. (3.34)

Then apply the gauge transformation

g̃(s, t) = hs(t)−1g(s)−1 exp(tξ(s)) (3.35)

to the pair (u, 0,Ψ), we see

(g̃∗u) (s, 0) = g̃(s, 0)−1(u(s, 0)) = g(s)u(s, 0)→ p, g̃∗ (Ψdt) = ξ(s)dt+ η(s, t)ds (3.36)

GAUGED FLOER HOMOLOGY 21

where

η = ∂s log g̃ = g̃−1∂g̃

∂s.

The fact that lims→+∞ ‖η‖ = 0 follows from (3.29) and (3.30).Hence

lims→+∞

(g̃∗u)|{s}×S1 =(exp(−tξp)φtHp, ξp

)∈ ZeroB̃H .

�

Definition 3.8. Let x̃± := (x±, f±) ∈ ZeroB̃H . We denote

M̃ (x̃−, x̃+) := M̃ (x̃−, x̃+; J,H) :={

(u,Φ,Ψ) ∈ M̃bΘ | lims→±∞(u,Φ,Ψ)|{s}×S1 = (x̃±, 0)}. (3.37)

For x± = (x̃±, [w±]) ∈ CritÃH which projects to x̃± via CritÃH → ZeroB̃H , we define

M̃ (x−, x+) := M̃ (x−, x+; J,H) :={

(u,Φ,Ψ) ∈ M̃(x̃−, x̃+) | [u#w−] = [w+]}. (3.38)

Then it is easy to deduce the following energy identity for which we omit the proof.

Proposition 3.9. Let x± ∈ CritÃH . Then for any (u,Φ,Ψ) ∈ M̃ (x−, x+), we have

E (u,Φ,Ψ) = ÃH (x−)− ÃH (x+) . (3.39)

3.5. Convexity and uniform bound on flow lines. We will show in this subsection the following

Proposition 3.10. There exists a compact subset KH ⊂ M such that for any (u,Φ,Ψ) ∈ M̃bΘ,u(Θ) ⊂ KH .

Proof. The proof is to use maximum principle as in [2, Subsection 2.5]. We claim this proposition

is true for

KH = SuppH ∪ f−1 ([0, c1])

where

c1 = max

{c0, sup|µ(x)|≤1

f(x)

}(3.40)

where c0 is the one in Hypothesis 2.4.

Suppose the statement is not true. Then there exists a solution ũ = (u,Φ,Ψ) ∈ M̃bΘ whichviolates this condition and (s0, t0) ∈ Θ such that u(s0, t0) /∈ SuppH and f(u(s0, t0)) > c1. Onthe other hand, by the previous results, we know that lim

s→±∞µ(u(s, t)) = 0 so lim

s→±∞f(u(s, t)) ≤ c1.

Hence f(u) achieves its maximum at some point of Θ. As in the proof of [2, Lemma 2.7], we see that

f(u) is subharmonic on u−1 (M \KH) and hence f(u) must be constant. However, this contradictswith the fact that lim

s→±∞f(u(s, t)) ≤ c1. �

4. Fredholm theory

In this section we investigate the infinitesimal deformation theory of solutions to our equation

(modulo gauge). For a similar treatment of a relevant situation, the reader may refer to [4].

22 GUANGBO XU

4.1. Banach manifolds, bundles, and local slices. First we fix two loops x̃± ∈ ZeroB̃H . Forany k ≥ 1, p > 2, we consider the space of W k,ploc -maps ũ := (u,Φ,Ψ) : Θ → M × g × g, such thatΦ ∈ W k,p (Θ, g) and (u,Ψ) is asymptotic to x̃± = (x±, f±) at ±∞ in W k,p-sense. Then this is aBanach manifold, denoted by

B̃k,p := B̃k,p(x̃−, x̃+). (4.1)

The tangent space at any element ũ ∈ B̃k,p is the Sobolev space

TũB̃k,p = W k,p (Θ, u∗TM ⊕ g⊕ g) . (4.2)

We denote by ẽxpt the exponential map of M×g×g, where the Riemannian metric on M is ω(·, Jt·)which is t-dependent. Then the map ξ̃ 7→ ẽxptũξ̃ is a local diffeomorphism from a neighborhood of0 ∈ TũB̃k,p and a neighborhood of ũ in B̃k,p.

Then consider a pair x± = (x±, f±, [w±]) ∈ CritÃH with x̃± = (x±, f±) ∈ ZeroB̃H . We define

B̃k,p(x−, x+) :={ũ = (u,Φ,Ψ) ∈ B̃k,p(x̃−, x̃+) | [w−#u] = [w+]

}. (4.3)

Let Gk+1,p0 be the space of Wk+1,ploc -maps g : Θ → G which is asymptotic to the identity of G at

±∞. Then this is a Banach Lie group. The gauge transformation extends to a free Gk+1,p0 -actionon B̃k,p(x̃−, x̃+) (resp. B̃k,p(x−, x+)), because the symplectic quotient M is a free quotient. Thenthis makes the quotient

Bk,p(x̃−, x̃+) := B̃k,p(x̃−, x̃+)/Gk+1,p(

resp. Bk,p(x−, x+) := B̃k,p(x−, x+)/Gk+1,p0)

(4.4)

a Banach manifold. Indeed, to see this we have to construct local slices of the Gk+1,p-action. Forany ũ ∈ B̃k,p (whose image in Bk,p is denoted by [ũ]), consider the operator

d∗0 : TũB̃k,p → W k−1,p (Θ, g)(ξ, φ, ψ) 7→ −dµ(Jtξ)− ∂sφ− [Φ, φ]− ∂tψ − [Ψ, ψ],

(4.5)

which is the formal adjoint of the infinitesimal Gk+1,p-action. Then as in gauge theory, we have anatural identification

T[ũ]Bk,p ' kerd∗0 (4.6)

(where the orthogonal complement is taken with respect to the L2-inner product) and the expo-

nential map ẽxpt induces a local diffeomorphism

kerd∗0 3 ξ̃ 7→[ẽxptũξ̃

]∈ Bk,p. (4.7)

If we have g± ∈ L0G, and x′± = g∗±x± ∈ CritÃH , then the pair (g−, g+) extends to a smoothgauge transformation on Θ which identifies B̃k,p(x−, x+) with B̃k,p(x′−, x′+). Then, with abuse ofnotation, if x± ∈ CritÃH/L0G, then we can denote by Bk,p(x−, x+) to be the common quotientspace. Finally, for two pairs [x±] ∈ CritAH , we define

Bk,p([x−], [x+]) :=⋃

y±∈CritÃH/L0G, [y±]=[x±]

Bk,p(y−, y+), (4.8)

which is a discrete union of Banach manifolds.

GAUGED FLOER HOMOLOGY 23

Over B̃k,p(x−, x+), we have the smooth Banach space bundle Ẽk−1,p(x−, x+), whose fibre over ũ isthe Sobolev space

Ẽk−1,pũ := Wk−1,p (u∗TM ⊕ g) . (4.9)

The Gk+1,p0 -action makes Ẽk−1,p an equivariant bundle, hence descends to a Banach space bundle

Ek−1,p(x−, x+)→ Bk,p(x−, x+)(

or Ek−1,p ([x−], [x+])→ Bk,p ([x−], [x+])). (4.10)

Moreover, the H-perturbed vortex equation (2.47) gives a section

F̃ : B̃k,p (x−, x+)→ Ẽk−1,p(x−, x+) (4.11)

which is Gk+1,p-equivariant. So it descends to a section

F : Bk,p ([x−], [x+])→ Ek−1,p ([x−], [x+]) .

Then we see that M̃ (x−, x+; J,H) is the intersection of F̃−1(0) with smooth objects. We define

M ([x−], [x+]; J,H) = F−1(0), (4.12)

whose elements, by the standard regularity theory about symplectic vortex equation (see [2, The-

orem 3.1]), all have smooth representatives. Therefore M ([x−], [x+]; J,H) is independent of k, p.The linearization of F̃ at ũ is

D̃ũ := dF̃ũ : (ξ, φ, ψ) 7→

(∇A,sξ + (∇ξJt) (∂tu+XΨ − YHt) + Jt (∇A,tξ −∇ξYHt) +Xφ + JXψ

∂sψ + [Φ, ψ]− ∂tφ− [Ψ, φ] + dµ(ξ)

)(4.13)

Hence the linearization of F , under the isomorphism (4.6), is the restriction of D̃ũ to (kerd∗0)⊥.

We define the augmented linearized operator

Dũ := D̃ũ ⊕ d∗0 : TũB̃k,p → Ẽk−1,pũ ⊕W

k−1,p (Θ, g) . (4.14)

It is a standard result that the Fredholm property of dF[ũ] is equivalent to that of Dũ for anyrepresentative ũ. Hence in the remaining of the section we will study the Fredholm property of the

augmented operator.

4.2. Asymptotic behavior of Dũ. Up to an L0G-action we can choose representatives x± =

(x±, f±, [w±]) such that f± are constants θ± ∈ g. Take any ũ = (u,Φ,Ψ) ∈ B̃k,p(x−, x+).For ξ̃ := (ξ, ψ, φ) ∈ TũB̃k,p(x−, x+), define J̃(ξ, ψ, φ) = (Jξ,−φ, ψ). Here ψ is the variation of Ψ

and φ is the variation of Φ. Then

Dũ

ξψφ

= ∇A,s ξψ

φ

+J̃ ∇A,tξ −∇ξYH,t∇A,tψ

∇A,tφ

+ 0 JLu Ludµ 0 0

L∗u 0 0

ξψ

φ

+q(s, t) ξψ

φ

=:

∇s∂s

∂s

+ J̃ ∇t∂t

∂t

+R(s, t) + q(s, t) ξψ

φ

. (4.15)

24 GUANGBO XU

Here q(s, t) is a linear operator such that lims→±∞

q(s, t) = 0; and

Rx̃±(t) := lims→±∞R(s, t) =

Jt∇(Xθ± − YHt) JtLu Ludµ 0 −adθ±L∗u adθ± 0

. (4.16)Here Lu : g→ u∗TM is the infinitesimal action along the image of u and L∗u is its dual.

The following result is implied by Hypothesis 2.3.

Proposition 4.1. x̃ := (x, θ) ∈ ZeroB̃H , then for all t-dependent, G-invariant, ω-compatible almostcomplex structure Jt, the self-adjoint operator

L2(S1, x∗TM ⊕ g⊕ g

)→ L2

(S1, x∗TM ⊕ g⊕ g

) ξψφ

7→J̃ ∇t∂t

∂t

+Rx̃(t) ξψ

φ

. (4.17)has zero kernel.

Proof. Suppose (ξ, ψ, φ)T is in the kernel, which means

Jt∇tξ + Jt∇ξ (Xθ − YHt) +Xφ + JtXψ = 0, (4.18)

−dφdt

+ dµ(ξ)− [θ, φ] = 0, (4.19)

dψ

dt+ dµ(Jtξ) + [θ, ψ] = 0. (4.20)

Apply dµ ◦ Jt to (4.18), we get

dµ(JtXφ) = dµ(∇tξ +∇ξ(Xθ − YHt)).

Hence for any η ∈ g,d

dt〈dµ(ξ), η〉g =

d

dtω(Xη, ξ)

=ω([YHt −Xθ, Xη], ξ) + ω(Xη,∇tξ −∇ξ(YHt −Xθ))

=ω(X[θ,η], ξ) + 〈dµ(JXφ), η〉g=〈dµ(ξ), [θ, η]〉g + 〈dµ(JXφ), η〉g=〈dµ(JXφ)− [θ, dµ(ξ)], η〉g.

(4.21)

Therefore,

d

dtdµ(ξ) = dµ(JXφ)− [θ, dµ(ξ)]. (4.22)

Then by (4.19),

dµ(JtXφ)

=d

dtdµ(ξ) + [θ, dµ(ξ)]

=d

dt

(dφ

dt+ [θ, φ]

)+

[θ,dφ

dt+ [θ, φ]

]=φ′′ + 2

[θ, φ′

]+ [θ, [θ, φ]].

(4.23)

GAUGED FLOER HOMOLOGY 25

Suppose ‖φ‖ takes its maximum at t = t0 ∈ S1. Then for t ∈ (t0 − �, t0 + �), define φ̃(t) =Ade(t−t0)θφ(t). Then the right hand side of (4.23) is equal to Ade(t0−t)θ φ̃

′′(t). Hence at t = t0,

0 ≥12

d2

dt2‖φ‖2 = 1

2

d2

dt2

∥∥∥φ̃∥∥∥2 = 〈φ̃′′, φ̃〉+ ∥∥∥φ̃′∥∥∥2g

=〈dµ(JtXφ(t0)), φ(t0)

〉+∥∥∥φ̃′(t0)∥∥∥2 = ∥∥Xφ(t0)∥∥2 + ∥∥∥φ̃′(t0)∥∥∥2 . (4.24)

Hence Xφ ≡ 0, which implies φ ≡ 0 and by (4.19), ξ is tangent to µ−1(0).Now x∗Tµ−1(0) = Et⊕Lxg, where Et = (Lxg)⊥ ∩x∗Tµ−1(0) and the orthogonal complement is

taken with respect to the Riemannian metric g = ω(·, Jt·). Then with respect to this (G-invariant)decomposition, write

ξ = ξ⊥(t) +Xη(t).

Then take the Et-component of (4.18), and use the nondegeneracy assumption on the induced

Hamiltonian Ht on the symplectic quotient, we see that ξ⊥ ≡ 0. The only equations left is ∇tXη(t) +∇Xη(t)(Xθ − YHt) +Xψ = 0,ψ′ + [θ, ψ] + dµ(JXη) = 0.

The first equation is equivalent to

η′ + [θ, η(t)] + ψ = 0.

Hence

η′′(t) + 2[θ, η′(t)

]+ [θ, [θ, η]] = dµ(JtXη).

This can be treated similarly as (4.23), using maximum principle, which shows that η ≡ 0 andhence ψ ≡ 0. �

Corollary 4.2. Under Hypothesis 2.3, for any x̃± ∈ ZeroB̃H and any ũ ∈ B̃k,p (x̃−, x̃+), theaugmented linearized operator Dũ is Fredholm for any k ≥ 1, p ≥ 2.

Corollary 4.3. There exists δ = δ(x̃±) > 0 such that, for any ũ = (u,Φ,Ψ) ∈ M̃ (x̃−, x̃+; J,H)and any ξ̃ ∈ kerDũ, there exists c > 0 such that∣∣∣ξ̃(s, t)∣∣∣ ≤ ce−δ|s|. (4.25)In particular, if (u, 0,Ψ) ∈ M̃bΘ, then |∂su| and |∂sΨ| decay exponentially.

Proof. The first part is standard, see for example [32, Lemma 2.11]. For a solution ũ = (u, 0,Ψ)

in temporal gauge, by the translation invariance of the equation (2.44), we see that ξ̃ = (∂su, βs =

0, βt = ∂sF ) ∈ kerdF̃ũ. Moreover,

−d∗0ξ̃ =− d∗0(∂su, 0, ∂sΨ) = ∂t∂sΨ + L∗u(∂su) + [Ψ, ∂sΨ]

=− ∂t(µ(u)) + dµ(Jt∂su) + [Ψ, ∂sΨ]

=− dµ(∂tu) + dµ(Jt∂su) + [Ψ, ∂sΨ]

= dµ (XΨ − YHt)− [Ψ, µ] = 0.

(4.26)

This implies that ξ̃ ∈ kerDũ. Choose a smooth gauge transformation g : Θ → G such thatg∗ũ satisfies the asymptotic condition of Proposition 3.7. Then g∗ξ̃ ∈ kerDg∗ũ, which decaysexponentially. So does ξ̃. �

26 GUANGBO XU

4.3. The Conley-Zehnder indices. In this subsection we define a grading on the set CritAH ,which is analogous to the Conley-Zehnder index in usual Hamiltonian Floer theory, and we will

call it the same name.

For the induced Hamiltonian system on the symplectic quotient M , we have the usual Conley-

Zehnder index

CZ : CritAH → Z. (4.27)

We prove the following theorem

Theorem 4.4. There exists a function

CZ : CritAH → Z (4.28)

satisfying the following properties

(1) For the embedding ι : CritAH → CritAH , we have

CZ ◦ ι = CZ; (4.29)

(2) For any B ∈ Γ and [x] ∈ CritAH we have

CZ (B# [x]) = CZ ([x])− 2cG1 (B). (4.30)

(3) For [x±] = [x±, f±, [w±]] ∈ CritAH and [ũ] ∈ Bk,p ([x−], [x+]) with [ũ#w−] = [w+], we have

ind(dF[ũ]

)= CZ ([x−])− CZ ([x+]) . (4.31)

We first review the notion of Conley-Zehnder index in Hamiltonian Floer homology. Let A :

[0, 1]→ Sp(2n) be a continuous path of symplectic matrices such that

A(0) = I2n, det (A(1)− I2n) 6= 0. (4.32)

We can associate an integer CZ(A) to A, called the Conley-Zehnder index. We list some properties

of the Conley-Zehnder index below which will be used here (see for example [31]).

(1) For any path B : [0, 1]→ Sp(2n), we have CZ(BAB−1) = CZ(A);(2) CZ is homotopy invariant;

(3) If for t > 0, A(t) has no eigenvalue on the unit circle, then CZ(A) = 0;

(4) If Ai : [0, 1]→ Sp(2ni) for n = 1, 2, then CZ(A1 ⊕A2) = CZ(A1) + CZ(A2);(5) If Φ : [0, 1]→ Sp(2n) is a loop with Φ(0) = Φ(1) = Id, then

CZ(ΦA) = CZ(A) + 2µM (Φ) (4.33)

where µM (Φ) is the Maslov index of the loop Φ.

With this algebraic notion, in the usual Hamiltonian Floer theory one can define the Conley-

Zehnder indices for nondegenerate Hamiltonian periodic orbits. In our case, the induced Hamil-

tonian Ht : M → R has the usual Conley-Zehnder index

CZ : CritAH → Z. (4.34)

Then, for each x = (x, f, [w]) ∈ CritÃH , the homotopy class of extensions [w] induces a homotopyclass of trivializations of x∗TM over S1. With respect to this class of trivialization, the operator

(4.17) is equivalent to an operator J0∂t + A(t), which defines a symplectic path. We define the

GAUGED FLOER HOMOLOGY 27

Conley-Zehnder index of x to be the Conley-Zehnder index of this symplectic path. By the second

and fifth axioms listed above, this index induces a well-defined function

CZ : CritAH → Z (4.35)

which satisfies (2) and (3) of Theorem 4.4.

Now we prove (1). For any contractible periodic orbits x : S1 → M of YHt and any extensionw : D→M of x, we can lift the pair (x,w) to a tuple x = (x, f, [w]) ∈ CritÃH .

Proposition 4.5. If ι : CritAH → CritAH is the inclusion we described in Proposition 2.8, then

CZ ◦ ι = CZ. (4.36)

Proof. Since the Conley-Zehnder index is homotopy invariant, and the space of G-invariant ω-

compatible almost complex structures is connected, we will compute the Conley-Zehnder index

using a special type of almost complex structures, and modify the Hamiltonian H.

Starting with any almost complex structure J on M and a G-connection on µ−1(0)→M , J liftsto the horizontal distribution defined by the connection. On the other hand, the biinvariant metric

on g gives an identification g ' g∗. We denote by η∗ ∈ g∗ the metric dual of η ∈ g. Recall that wehave a symplectomorphism

µ−1 (g∗� ) ' µ−1(0)× g∗� . (4.37)

For η ∈ g, we define JXη = η∗ ∈ g∗, as a vector field on µ−1(0)× g∗. Then this gives a G-invariantalmost complex structure on TM |µ−1(0), compatible with ω. Then we pullback J by the projectionµ−1(0)× g∗� → µ−1(0) and denote the pullback by J .

We also modify Ht by requiring that Ht(x, η) = Ht(x) for (x, η) ∈ µ−1(0)×g∗� . Then the modifiedHt can be continuously deformed to the original one, and it doesn’t change H hence doesn’t change

CritAH . Moreover, it is easy to check that for the modified pair (J,H),(LYHtJ

)Xη =

[LYHt , JXη

]= 0. (4.38)

Now for any (x,w) ∈ CritAH , we lift it to (x, f, [w]) ∈ CritÃH with w : D → µ−1(0) and fbeing a constant θ ∈ g. Then any symplectic trivialization of x̃∗TM → S1 induces a symplectictrivialization

φ : x∗TM ' S1 ×[R2n−2k ⊕ (g⊕ g)

](4.39)

such that φ(Xη, JXζ) = (0, η, ζ). Then we see, with respect to φ, the operator (4.17) restricted to

g4 is η1

ψ

η2

φ

7→ J̃ ddt

η1

ψ

η2

φ

+φ− [θ, η2]η2 − [θ, φ]ψ + [θ, η1]

η1 + [θ, ψ]

=:(J̃d

dt+ S

)η1

ψ

η2

φ

. (4.40)Here we used the property (4.38) and

J̃ :=

[0 −Idg⊕g

Idg⊕g 0

], S =

[0 Idg⊕g − adθ

Idg⊕g + adθ 0

](4.41)

28 GUANGBO XU

Moreover, the operator (4.17) respect the decomposition in (4.39). Hence by the fourth axiom of

Conley-Zehnder indices we listed above, we have

CZ (x, θ, [w]) = CZ(x,w) + CZ(eJ̃St

). (4.42)

As we have shown in the proof of Proposition 4.1 that for any θ the operator (4.17) is an isomor-

phism, we can deform θ to zero and compute instead CZ(eJ̃S0t) for

S0 =

[0 Idg⊕g

Idg⊕g 0

], (4.43)

thanks to the homotopy invariance property. Then we see that

eJ̃S0t =

[e−t 0

0 et

](4.44)

which has no eigenvalue on the unit circle for t > 0. Therefore by the third axiom, CZ(eJ̃S0t) =

0. �

5. Compactness of the moduli space

For a general Hamiltonian G-manifold, the failure of compactness of the moduli space of con-

necting orbits comes from two phenomenon. The first is the breaking of connecting orbits, which is

essentially the same thing happened in finite dimensional Morse-Smale-Witten theory. The second

is the blow-up of the energy density, which results in sphere bubbling. Since here we have assumed

that there exists no nontrivial holomorphic sphere in M , so we only have to consider the breakings.

5.1. Moduli space of stable connecting orbits and its topology. Let’s fix a pair [x±] ∈CritAH . Denote by M̂([x−], [x+]) := M̂([x−], [x+]; J,H) =M([x−], [x+]; J,H)/R the quotient of themoduli space by the translation in the s-direction. We denote by {ũ} the R-orbit in M̂([x−], [x+])of [ũ] ∈M([x−], [x+]; J,H) and call it a trajectory from [x−] to [x+].

Definition 5.1. A broken trajectory from [x−] to [x+] is a collection

ũ :=({ũ(α)

})α=1,...,m

:=({u(α),Φ(α),Ψ(α)

})α=1,...,m

(5.1)

where for each α,{ũ(α)

}∈ M̂ ([xα−1] , [xα]) and E(ũ(α)) 6= 0. Here {[xα]}α=0,...,m is a sequence of

critical points of AH and[x0] = [x−] , [xm] = [x+] .

We regard the domain of ũ as the disjoint union

∪mα=1Θ

and let Θ(α) ⊂ ∪mα=1Θ the α-th cylinder.

We denote by

M ([x−] , [x+]) (5.2)

the space of all broken trajectories from [x−] to [x+]. Then naturally we have inclusion

M̂ ([x−] , [x+])→M ([x−] , [x+]) . (5.3)

GAUGED FLOER HOMOLOGY 29

Definition 5.2. We say that a sequence of trajectories {ũi} = {ui,Φi,Ψi} ∈ M̂ ([x−] , [x+]) from[x−] to [x+] converges to a broken trajectory

ũ :=({ũ(α)

})α=1,...,m

if: for each i, there exists sequences of numbers s(1)i < s

(2)i < · · · < s

(m)i and gauge transformations

g(α)i ∈ GΘ such that for each α (

g(α)i

)∗ (s

(α)i

)∗(ui,Φi,Ψi) (5.4)

converges to(u(α),Φ(α),Ψ(α)

)on any compact subset of Θ and such that for any sequence of (si, ti)

with

limi→∞|si − s(α)i | =∞, ∀α

we have

limi→∞

e (ũi) (si, ti) = 0.

It is easy to see that this convergence is well-defined and independent of the choices of represen-

tatives of the trajectories. We can also extend this notion to sequences of broken connecting orbits.

We omit that for simplicity.

The main theorem of this section is

Theorem 5.3 (Compactness of the moduli space of stable connecting orbits). The space

M ([x−] , [x+]) is a compact Hausdorff space with respect to the topology defined in Definition 5.2.

Indeed the proof is routine and it has been carried out in many literature for general symplectic

vortex equations, for example [25], [2], [26], [37]. Since bubbling is ruled out, the proof is almost the

same as that for finite dimensional Morse theory, while the gauge symmetry is the only additional

ingredient.

5.2. Local compactness with uniform bounded energy density. For any compact subset

K ⊂ Θ, consider a sequence of solutions ũi := (ui,Φi,Ψi) such that the image of ui is contained inthe compact subset KH ⊂M and such that

lim supi→∞

E(ũi)

30 GUANGBO XU

5.3. Energy quantization. To prove the compactness of the moduli space, we need the following

energy quantization property.

Proposition 5.5. There exists �0 := �0(J,H) > 0, such that for any connecting orbit ũ ∈ M̃bΘ, wehave E(ũ) ≥ �0.

Proof. Suppose it is not true. Then there exists a sequence of connecting orbits, represented by

solutions in temporal gauge ṽi := (vi, 0,Ψi) ∈ M̃bΘ, such that

E(ṽi) > 0, limi→∞

E(ṽi) = 0. (5.6)

We first know that there is a compact subset KH ⊂ M such that for every i, the image vi(Θ) iscontained in KH . Then we must have

limi→∞

supΘ

(|∂svi|+ |µ(vi)|) = 0. (5.7)

Indeed, if the equality doesn’t hold, then we can find a subsequence which either bubbles off a

nonconstant holomorphic sphere at some point z ∈ Θ (if the above limit is∞), or (after a sequenceof proper translation in s-direction) converges to a solution (with positive energy) on compact

subsets (if the above limit is positive and finite). Either case contradicts the assumption. Therefore

we conclude that for any � > 0, the image of vi lies in U� := µ−1(g∗� ) for i sufficiently large.

Recall that we have projections πµ : U� → µ−1(0) and πµ : U� → M . Then for all large i andany s, πµ(vi(s, ·)) is C0-close to a periodic orbit of H in M . Since those orbits are discrete (inC0-topology, for example), we may fix one such orbit γ ∈ ZeroBH and choose a subsequence (stillindexed by i) such that

lim supi→∞

sup(s,t)∈Θ

d (πµ(vi(s, t)), γ(t)) = 0. (5.8)

Then, use a fixed Riemannian metric on M with its exponential map exp, we can write

πµ(vi(s, t)) = expγ(t)ξi(s, t)

where ξi ∈ Γ(S1, γ∗TM

). Let B�(γ

∗TM) be the �-disk of γ∗TM . Then expγ pulls back µ−1(0)→

M to a G-bundle Q→ B�(γ∗TM), together with a bundle map γ̃ : Q→ µ−1(0). We can trivializeQ by some

φ : Q→ G×B�(γ∗TM).

Now we take a lift x̃ := (x, f) ∈ ZeroB̃H of γ. Then we can write

φ(γ̃−1(x(t))

)= (g0(t), γ(t)). (5.9)

On the other hand, we write

φ(γ̃−1πµ(vi(s, t))

)= (gi(s, t), ξi(s, t)). (5.10)

Take the gauge transformation g̃i(s, t) = gi(s, t)g0(t)−1. Then write

ṽ′i := (v′i,Φ′i,Ψ′i) := g̃

∗i ṽi. (5.11)

GAUGED FLOER HOMOLOGY 31

Then by the exponential convergence of vi as s→ ±∞, we see that ∂sgi(s, t) decays exponentiallyand hence Φ′i converges to zero as s→ ±∞. On the other hand, we see that

φ(γ̃−1πµ(v

′i(s, t))

)= (g0(t), ξ̃i(s, t)). (5.12)

Therefore

ṽ′i ∈ M̃(x̃, x̃). (5.13)

But it is also easy to see that the homotopy class of ṽ′i is trivial. Because the energy of connecting

orbits only depends on its homotopy class, we see that the energy of ṽ′i, and hence the energy of ṽi

is actually equal to zero, which contradicts with the hypothesis. �

5.4. Proof of the compactness theorem. It suffices to prove, without essential loss of gener-

ality, that for any sequence [ũi] ∈ M ([x−], [x+]; J,H) represented by unbroken connecting orbits(ui,Φi,Ψi) ∈ M̃ (x−, x+), there exists a convergent subsequence. By the assumption that thereexists no nontrivial holomorphic sphere in M , we have

supi,Θ|∂sui +XΦi(ui)| < +∞. (5.14)

Then the limit (broken) connecting orbits can be constructed by induction and the energy quan-

tization property (Proposition 5.5) guarantees that the induction stops at finite time. The details

are standard and left to the reader.

6. Floer homology

In this section we use the moduli spacesM ([x−], [x+]; J,H) to define the vortex Floer homologygroup V HF∗ (M,µ;H). We also discuss further works and related problems in the last three

subsections.

By Corollary A.12, we can choose a generic S1-family of “admissible” almost complex structures

J ∈ J̃ regH which is regular with respect to H. Such an object is a smooth t-dependent family ofalmost complex structures Jt, such that for each t, Jt is G-invariant, ω-compatible, and outside a

neighborhood U of µ−1(0), Jt ≡ J; inside U , Jt preserves a distribution gCU . Being regular impliesthat for all pairs [x±] ∈ CritAH , the moduli space M ([x−], [x+]; J,H) is a smooth manifold with

dimM ([x−], [x+]; J,H) = CZ([x−])− CZ([x+]). (6.1)

Moreover, there is free R-action on M ([x−], [x+]; J,H) by time translation, whose orbit space isdenoted by M̂([x−], [x+]; J,H). Combining the compactness theorem, we have

Proposition 6.1. If J ∈ J̃ regH , then

CZ([x−])− CZ([x+]) ≤ 0 =⇒M ([x−], [x+]; J,H) = ∅; (6.2)

CZ([x−])− CZ([x+]) = 1 =⇒ #M̂ ([x−], [x+]; J,H)

32 GUANGBO XU

6.1. The gluing map and coherent orientation. The boundary operator of the Floer chain

complex is defined by the (signed) counting of M̂([x−], [x+]; J,H). If we want to define the Floerhomology over Z2, then we don’t need to orient the moduli space; otherwise, the orientation ofM̂([x−], [x+]; J,H) can be treated in the same way as the usual Hamiltonian Floer theory, sincethe augmented linearized operator Dũ (whose determinant is canonically isomorphic to the deter-

minant of the actual linearization dF[ũ]), is of the same type of Fredholm operators considered inthe abstract setting of [10]. We first give the gluing construction and then discuss the coherent

orientations of the moduli spaces.

In this subsection we construct the gluing map for broken trajectories with only one breaking.

The general case is similar. This construction is, in principle, the same as the standard construction

in various types of Morse-Floer theory (see [32] [10]), with a gauge-theoretic flavor. The gauge

symmetry makes the construction a bit more complex, since we always glue representatives, and

we want to show that the gluing map is independent of the choice of the representatives.

In this subsection, we fix the choice of the admissible family J ∈ J̃ regH and omit the dependenceof the moduli spaces on J and H.

For any pair x± ∈ CritAH , we say that a solution ũ ∈ M̃ (x−, x+) is in r-temporal gauge, if itsrestrictions to [r,+∞)× S1 and (−∞, r]× S1 are in temporal gauge, for some r > 0. Now we fix anumber r = r0 and only consider solutions in r0-temporal gauge.

Now we take three elements x, y, z ∈ CritÃH with

CZ(x)− 1 = CZ(y) = CZ(z) + 1. (6.4)

Assume y = (y, η) : S1 →M × g. We would like to construct, for a large R0 > 0, the gluing map

glue : M̂ ([x], [y])× (R0,+∞)× M̂ ([y], [z])→ M̂ ([x], [z]) . (6.5)

Now consider two trajectories [ũ±] = [u±,Φ±,Ψ±], [ũ−] ∈ M ([x], [y]), [ũ+] ∈ M ([y], [z]) withtheir representatives both in r0-temporal gauge and ũ± are asymptotic to y as s → ∓∞. Thenthere exists R1 > 0 such that

±s ≥ R1 =⇒ u∓(s, t) = expy(t) ξ∓(s, t). (6.6)

Here Θ+R1 = [R1,+∞)× S1 and Θ−R1 = (−∞,−R1]× S

1 and ξ± ∈W k,p(

Θ∓R1 , y∗TM

).

Next, we take a cut-off function ρ such that s ≥ 1 =⇒ ρ(s) = 1, s ≤ 0 =⇒ ρ(s) = 0. For eachR >> r0, denote ρR(s) = ρ(s−R). We construct the “connected sum”

uR(s, t) =

u−(s+R, t), s ≤ −R2 − 1

expy(t)

(ρR

2(−s)ξ−(s+R, t) + ρR

2(s)ξ+(s−R, t)

)s ∈

[−R2 − 1,

R2 + 1

]u+(s−R, t), s ≥ R2 + 1

(6.7)

(ΦR,ΨR) (s, t) =

(Φ−(s+R, t),Ψ−(s+R, t)) , s ≤ −R2 − 1(

0, ρR2

(−s)Ψ−(s+R, t) + ρR2

(s)Ψ+(s−R, t)), s ∈

[−R2 − 1,

R2 + 1

](Φ+(s−R, t),Ψ+(s−R, t)) . s ≥ R2 + 1

(6.8)

Now it is easy to see that, if we change the choice of representatives ũ± which are also in r0-temporal

gauge, the connected sum ũR := (uR,ΦR,ΨR) doesn’t change for s ∈ [−R2 − 1,R2 + 1] and hence

GAUGED FLOER HOMOLOGY 33

we obtain a gauge equivalent connected sum. Moreover, if we change y by y′ which represent the

same [y] ∈ CritAH , then we can obtain

ũ′− ∈ M̃(x, y′

), ũ′+ ∈ M̃

(y′, z

)(6.9)

which are also in r0-temporal gauge, and we obtain a connected sum ũ′R which is gauge equivalent

to ũR.

Now we consider the augmented linearized operator DR := DũR .

Lemma 6.2. There exists c > 0 and R0 > 0 such that for every R ≥ R0 and η̃ ∈ Ẽ2,pũR ⊕W2,p (Θ, g),

we have

‖D∗Rη̃‖W 1,p ≤ c ‖DRD∗Rη̃‖Lp . (6.10)

Proof. Same as the proof of [32, Proposition 3.9] �

Hence we can construct a right inverse

QR := D∗R (DRD

∗R)−1 : Ẽ0,pũR ⊕ L

p (Θ, g)→ TũRB̃1,p (6.11)

with

‖QR‖ ≤ c. (6.12)

Now we write QR := (QR,AR) with QR : Ẽ0,pũR → TũRB̃1,p (x, z). Then actually the image of QR

lies in the kernel of d∗0 and therefore QR is a right inverse to dF̃ũR |kerd∗0 . Because our constructionis natural with respect to gauge transformations, we see that QR induces an injection

QR : E0,p[ũR] → T[ũR]B1,p (6.13)

which is a right inverse to the linearized operator dF[ũR] and which is bounded by c. By the implicitfunction theorem, we have

Proposition 6.3. There exists R1 > 0, δ1 > 0 such that for each R ≥ R1, there exists a uniqueξ̃ ∈ ImQR = kerd∗0 ⊂ TũRB̃1,p,

∥∥∥ξ̃∥∥∥W 1,p

≤ δ1 such that

F̃(

expũR ξ̃)

= 0,∥∥∥ξ̃∥∥∥

W 1,p≤ 2c

∥∥∥F̃(ũR)∥∥∥Lp. (6.14)

Therefore, the gluing map can be defined as

glue ([ũ−], R, [ũ+]) =[expũR ξ̃

]∈ M̂ ([x], [z]; J,H) . (6.15)

On the other hand, it is easy to see that the augmented linearized operators Dũ for all con-

necting orbits ũ is of “class Σ” considered in [10]. Therefore, by the main theorem of [10], there

exists a “coherent orientation” with respect to the gluing construction. Choosing such a coher-

ent orientation, then to each zero-dimensional moduli space M̂([x], [y]; J,H), we can associate thecounting χJ([x], [y]) ∈ Z, where each trajectory [ũ] contributes to 1 (resp. -1) if the orientation of[ũ] coincides (resp. differ from) the “flow orientation” of the solution. Then we define

δJ : V CFk (M,µ;H; ΛZ) → V CFk−1 (M,µ;H; ΛZ)

[x] 7→∑

[y]∈CritAH

χJ([x], [y])[y](6.16)

34 GUANGBO XU

As in the usual Floer theory, we have

Theorem 6.4. For each choice of the coherent orientation on the moduli spaces M([x], [y]; J,H),the operator δJ in (6.16) defines a morphism of ΛZ-modules satisfying δJ ◦ δJ = 0.

This makes (V CF∗(M,µ;H; ΛZ), δJ) a chain complex of ΛZ-modules, to which will be generally

referred as the vortex Floer chain complex. Therefore the vortex Floer homology group is defined

as the ΛZ-module

V HFk (M,µ; J,H; ΛZ) :=ker (δJ : V CFk (M,µ;H; ΛZ)→ V CFk−1 (M,µ;H; ΛZ))im (δJ : V CFk+1 (M,µ;H; ΛZ)→ V CFk (M,µ;H; ΛZ))

(6.17)

6.2. The continuation map. Now we prove that the vortex Floer homology group defined above is

independent of the choice of admissible family of almost complex structures and the time-dependent

Hamiltonians, and, if we use the moduli space of (1.8) instead of (1.7) to define the Floer homol-

ogy, independent of the parameter λ > 0. So far the argument has been standard, by using the

continuation principle.

Let (Jα, Hα, λα) and(Jβ, Hβ, λβ

)be two triples where λα, λβ > 0, Hα, Hβ are G-invariant

Hamiltonians satisfy Hypothesis 2.1 and 2.3, and Jα ∈ J̃ regHα,λα , Jβ ∈ J̃ reg

Hβ ,λβ(for notations refer to

the appendix).

We choose a cut-off function ρ : R → [0, 1] with s ≤ −1 =⇒ ρ(s) = 1 and s ≥ 1 =⇒ ρ(s) = 0.Then we define

Hs,t = ρ(s)Hαt + (1− ρ(s))H

βt , λs = ρ(s)λ

α + (1− ρ(s))λβ. (6.18)

We denote this family of Hamiltonians by H . Now, as in the appendix, we consider the space of

families of admissible almost complex structures J̃(Jα, Jβ

)consisting of smooth families of almost

complex structures J = (Js,t)(s,t)∈Θ, such that for each l ≥ 1,∣∣∣e|s|Js,t − Jαt ∣∣∣Cl(Θ−×M)

GAUGED FLOER HOMOLOGY 35

One thing to check in defining the continuation map is the energy bound of solutions, which

implies the compactness of the moduli space. We define the energy to be

E(ũ) = E(u,Φ,Ψ)

=1

2

(|∂su+XΦ(u)|2L2 +

∣∣∂tu+XΨ(u)− YHs,t(u)∣∣2L2 + ∣∣λ−1s (∂sΨ− ∂tΦ + [Φ,Ψ])∣∣2L2 + |λsµ(u)|2L2)= |∂su+XΦ(u)|2L2 + |λsµ(u)|L2 (6.22)

where, the last equality holds only for ũ a solution to (6.20). Then we have

Proposition 6.5. For any solution ũ to (6.20) whose energy is finite and whose image in M has

compact closure, we have

E (ũ) = AHα([xα])−AHβ ([xβ])−∫

Θ

∂Hs,t∂s

(u)dsdt. (6.23)

Proof. We can transform the solution ũ into temporal gauge. Then the energy density is

|∂su|2 + |λsµ(u)|2 =ω(∂su, ∂tu+XΨ − YHs,t(u)

)− µ(u) · ∂Ψ

∂s

=ω(∂su, ∂tu)−∂

∂s(µ(u) ·Ψ) + ∂

∂s(Hs,t(u))−

∂Hs,t∂s

(u).

(6.24)

Then integrating over Θ, we obtain (6.23). �

Theorem 6.6. There exists a Baire subset J̃ regH ,λs(Jα, Jβ

)⊂ J̃

(Jα, Jβ

), such that for any J ∈

J̃ regH ,λs(Jα, Jβ

), the moduli space N ([xα], [xβ]; J ,H , λs) is a smooth oriented manifold with

dimN(

[xα], [xβ]; J ,H , λs)

= CZ([xα])− CZ([xβ]). (6.25)

So in particular, when CZ(xα) = CZ(xβ), N is of zero dimension. The algebraic count of N givesan integer χ

([xα], [xβ]

). Then we define the continuation map

contβα : V CF∗ (M,µ; Jα, Hα, λα; ΛZ) → V CF∗

(M,µ; Jβ, Hβ, λβ; ΛZ

)[xα] 7→

∑[xβ ]∈CritA

Hβ

χ(

[xα], [xβ])

[xβ]. (6.26)

Now we have the similar results as in ordinary Hamiltonian Floer theory.

Theorem 6.7. The map contβα is a chain map. The induced map on the vortex Floer homology

groups is independent of the choice of the homotopy H , the family J of almost complex structures,

the cut-off function ρ. In particular, contβα is a chain homotopy equivalence. If (Jγ , Hγ) is another

admissible pair and λγ > 0, then in the level of homology

contγβ ◦ contβα = cont

γα.

Proof. The proof is essentially based on the construction of various gluing maps and the compact-

ness results about N([xα], [xβ]; J ,H , λs

)when CZ([xα]) − CZ([xβ]) = 1. As in the gluing map

constructed in proving the property δ2J = 0, we need to specify a gauge to construct the approxi-

mate solutions. We can still use solutions in r-temp