Permutation diagrams in symmetric function theory and Schubert calculus

Abstract

A fundamental invariant of a permutation is its inversion set, or diagram. Natural machinery in the representation theory of symmetric groups produces a symmetric function from any finite subset of <bold>N</bold><super>2</super>, namely its <italic>generalized Schur function</italic>. When this subset is a permutation diagram, one obtains the Stanley symmetric function of a permutation. By exploiting this connection to representation theory, we obtain results on the interaction of pattern avoidance with the theory of Stanley symmetric functions. In particular, we show that for any positive integer <italic>k</italic>, the permutations whose Stanley symmetric function has at most <italic>k</italic> (classical) Schur function terms are exactly those which avoid a finite set of patterns. The cohomology ring of a Grassmannian is a quotient of the ring of symmetric functions, and Liu has given a class of subvarieties—the diagram varieties—whose cohomology classes are conjecturally represented by the generalized Schur functions. We give a counterexample to this conjecture. On the other hand, we use a degeneration of Coskun's rank varieties to show that Liu's conjecture does give an upper bound on the classes of diagram varieties of permutation diagrams. We also show that the cohomology class of any rank variety is represented by a Stanley symmetric function, using Knutson–Lam–Speyer's work on positroid varieties.