The term  "vertex operator"  in mathematics refers mainly to certain operators (in a generalized sense of the term) used in physics to describe interactions of physical states at a  "vertex"  in string theory [a9] and its precursor, dual resonance theory; the term refers more specifically to the closely related operators used in mathematics as a powerful tool in many applications, notably, constructing certain representations of affine Kac–Moody algebras (cf. also Kac–Moody algebra) and other infinite-dimensional Lie algebras, addressing the problems of the  "Monstrous Moonshine"  phenomena for the Monster finite simple group, and studying soliton equations (cf. also Moonshine conjectures). The term  "vertex operator"  also refers, more abstractly, to any operator corresponding to an element of a vertex operator algebra or a related operator.

Vertex operators arose in mathematics in the following construction of the  "basic"  highest weight representation of the simplest affine Lie algebra by means of formal differential operators in infinitely many formal variables (cf. also Representation of a Lie algebra): Consider the space of polynomials in the formal variables , . The formal expression constructed in [a12], is a basic example of a vertex operator (for other examples, see Kac–Moody algebra). Here denotes the formal exponential series and is another formal variable commuting with all . For each , the coefficient of in the expansion of the vertex operator in powers of is a well-defined linear operator on , and the main point is that the operators , , () and () span a Lie algebra of operators isomorphic to the affine Lie algebra [a12]. This vertex operator had been considered by physicists [a3] for other purposes. It was interpreted [a4] as the infinitesimal Bäcklund transformation for the Korteweg–de Vries equation in soliton theory.

This work was generalized [a11] to all the basic representations of the simply-laced (equal-root-length) affine Lie algebras and their Dynkin-diagram-induced twistings. H. Garland remarked that the differential operators reminded him of the  "vertex operators"  that physicists had been using, starting in [a8], in dual resonance theory. The resemblance turned into a coincidence in the construction by I. Frenkel and V. Kac [a5] and G.B. Segal [a13] of the untwisted vertex operator realization of the basic representations of the simply-laced affine Lie algebras. This construction had been anticipated by physicists ([a10], [a2]) in the case of . The untwisted vertex operator representations allowed one to look in a new way at the finite-dimensional simple Lie algebras, viewed as subalgebras of affine Lie algebras. The case of was used in string theory in the construction of the heterotic string by D. Gross, J. Harvey, E. Martinec, and R. Rohm (cf. [a9]). The operators constructed in [a12] and [a11] are understood as examples of twisted vertex operators, and give the principally-twisted vertex operator realization of the basic representations. Untwisted and twisted vertex operators entered fundamentally into the construction of the  "moonshine module"  [a6] for the Fischer–Griess Monster group and into the discovery of its canonical structure of vertex operator algebra ([a1], [a7]). There is a great variety of interesting examples of vertex operators. The notion of vertex (operator) algebra is an abstraction of fundamental properties of vertex operators discovered by physicists and mathematicians, and provides an elegant and powerful framework for the study and application of vertex operators.

References

[a1]	R.E. Borcherds,   "Vertex algebras, Kac–Moody algebras, and the monster"  Proc. Nat. Acad. Sci. USA , 83  (1986)  pp. 3068–3071
[a2]	T. Banks,   D. Horn,   H. Neuberger,   "Bosonization of the Thirring models"  Nucl. Phys. , B108  (1976)  pp. 119
[a3]	E.F. Corrigan,   D.B. Fairlie,   "Off-shell states in dual resonance theory"  Nucl. Phys. , B91  (1975)  pp. 527–545
[a4]	E. Date,   M. Kashiwara,   T. Miwa,   "Vertex operators and functions: transformation groups for soliton equations II"  Proc. Japan Acad. Ser. A Math. Sci. , 57  (1981)  pp. 387–392
[a5]	I.B. Frenkel,   V. Kac,   "Basic representations of affine Lie algebras and dual resonance models"  Invent. Math. , 62  (1980)  pp. 23–66
[a6]	I.B. Frenkel,   J. Lepowsky,   A. Meurman,   "A natural representation of the Fischer–Griess monster with the modular function as character"  Proc. Nat. Acad. Sci. USA , 81  (1984)  pp. 3256–3260
[a7]	I.B. Frenkel,   J. Lepowsky,   A. Meurman,   "Vertex operator algebras and the monster" , Pure Appl. Math. , 134 , Acad. Press  (1988)
[a8]	S. Fubini,   G. Veneziano,   "Duality in operator formalism"  Nuovo Cimento , 67A  (1970)  pp. 29
[a9]	M.B. Green,   J.H. Schwarz,   E. Witten,   "Superstring theory" , Cambridge Univ. Press  (1987)
[a10]	M.B. Halpern,   "Quantum solitons which are fermions"  Phys. Rev. , D12  (1975)  pp. 1684–1699
[a11]	V. Kac,   D. Kazhdan,   J. Lepowsky,   R.L. Wilson,   "Realization of the basic representations of the Euclidean Lie algebras"  Adv. Math. , 42  (1981)  pp. 83–112
[a12]	J. Lepowsky,   R.L. Wilson,   "Construction of the affine Lie algebra "  Comm. Math. Phys. , 62  (1978)  pp. 43–53
[a13]	G. Segal,   "Unitary representations of some infinite-dimensional groups"  Comm. Math. Phys. , 80  (1981)  pp. 301–342