Yesterday, John Reynolds passed away. I had the privilege of being
one of his graduate students, and much of what I know about what it
means to be a scientist is due to his example. I just want to share a
few small anecdotes about him.
When I was a new graduate student, I began working on the project
which would eventually become my thesis. When I described one of my
early approaches to this problem, he asked a series of questions
about how I would handle one sticky technical point after another,
and I showed him how the language and its semantics were carefully
set up so that this issue in question could not arise. After I
finished my explanation, he nodded thoughtfully, and told me,
"Neel, you've done some very good engineering, but engineering is
not science! Good engineers develop a sense of how to avoid
obstacles -- but a scientist's job is to face them head-on and
flatten them."
This made a really big impression on me. As academic computer
scientists, we have two great luxuries: we can choose what to
study, and we can spend as long as necessary studying it. As a
result, our ethical obligation is one step removed from solving
problems: our role is to produce understanding, so that
engineers (indeed, all of society) have new methods to solve new
classes of problems.
Once, John told me about some of his work at Argonne National
Laboratory, where he designed the COGENT
programming language, which he told me was what convinced him
that he should leave physics to do computer science. He said that
while today he could (indeed, many people could) design much better
languages, he would always regard it fondly, since it was his first
successful language design.
I asked him what made it successful, and he told me that there
was another scientist at Argonne, who used it to write some
simulation code. When John read the program listing, he realized
that he could not understand how the program worked, since it
relied domain knowledge that he lacked. So this was John's
definition of a successful language design: if you have a user who
has used it to write a program you couldn't have, your language has
succeeded, since it has helped a fellow human being solve one of
their own particular problems.
I've always liked his definition, since it manages to avoid an
obsession with nose-counting popularity metrics, while still
remembering the essentially social purpose of language design.
Another time, I asked John what he thought about object-oriented
programming. He told me he thought it was too soon to tell. I burst
out laughing, since I thought he was joking: Simula 67 is
considerably older than I am, and I thought that surely that was
sufficient time to form an opinion.
But he insisted that, no, he was not kidding! From a formal,
mathematical, perspective, he did not like object-oriented
programming very much: he had studied various formalizations of
objects in his work on Idealized Algol and on intersection types,
and found they introduced some unwanted complexities. However, so
many excellent engineers still thought that objects were a good
idea, that he was unwilling to dismiss the possibility that there
was a beautiful mathematical theory that we haven't discovered yet.
I thought this was a great demonstration of Keats's negative
capability, but what makes it really impressive was that John
married it to a profound intellectual rigour: I've never known
anyone who could produce counterexamples faster than John
could. While he tested every idea ruthlessly, he never closed
himself off from the world, giving him a rare ability to make
judgements without passing judgement.
I haven't really said anything about John's enormous scientific
achievements, because the deepest lesson I learned from him is not any
one of his theorems, but rather his fundamentally humane
vision of science, as something we do for and with our fellow
scientists and community.
Ars longa, vita brevis, but together we make a chain
through the ages --- and more links of that chain than most can claim
were forged by you, John.
However, Olle Fredriksson wrote us within one day of the draft going online with a link to his Haskell implementation. This was immensely reassuring, since (a) he implemented it so quickly, and (b) he didn't need to ask us how to implement it -- this means we didn't leave anything important out in the description!
In fact, Olle is technically the first person to implement the algorithm in the paper, since both Joshua and I have actually implemented variants of this algorithm for different languages. I'll try to write a small ML implementation to go along with Olle's Haskell code in the next few weeks. If you want to beat me to it, though, I have no objections!
Part of the reason for the silence on this blog is that it's been a
very busy spring for me. I've got three new drafts to announce, each
of which I played at least some role in:
Bidirectional typechecking, in which terms either synthesize a
type or are checked against a known type, has become popular
for its scalability (unlike Damas-Milner type inference,
bidirectional typing remains decidable even for very expressive
type systems), its error reporting, and its relative ease of
implementation. Following design principles from proof theory,
bidirectional typing can be applied to many type
constructs. The principles underlying a bidirectional approach
to polymorphism, however, are less obvious. We give a
declarative, bidirectional account of higher-rank polymorphism,
grounded in proof theory; this calculus enjoys many properties
such as η-reduction and predictability of annotations. We give
an algorithm for implementing the declarative system; our
algorithm is remarkably simple and well-behaved, despite being
both sound and complete.
In this paper, we show how to extend bidirectional typechecking
to support higher-rank polymorphism. Bidirectional typechecking is
attractive in part because it is very easy to implement, and
happily our extension remains simple -- it turns out the fanciest
data structure you need to support higher-rank polymorphism is an
ordered list!
Effective support for custom proof automation is essential
for large-scale interactive proof development. However,
existing languages for automation via tactics either (a)
provide no way to specify the behavior of tactics within the
base logic of the accompanying theorem prover, or (b) rely on
advanced type-theoretic machinery that is not easily
integrated into established theorem provers.
We present Mtac, a lightweight but powerful extension to
Coq that supports dependently-typed tactic
programming. Mtac tactics have access to all the features
of ordinary Coq programming, as well as a new set of typed
tactical primitives. We avoid the need to touch the trusted
kernel typechecker of Coq by encapsulating uses of these
new tactical primitives in a monad, and instrumenting Coq
so that it executes monadic tactics during type inference.
Since I wasn't the lead author, I see no need to be modest: this
is really, really, good. The power-to-weight ratio of
this approach to tactics is incredibly high, and I seriously think
that every dependently typed language implementor ought to consider
adding something like Mtac to their system.
Functional reactive programming (FRP) is an elegant approach to
declaratively specify reactive systems. However, the powerful
abstractions of FRP have historically made it difficult to predict
and control the resource usage of programs written in this style.
In this paper, we give a new language for higher-order reactive
programming. This language generalizes and simplifies prior type
systems for reactive programming, supporting the use of
first-class streams, such as streams of streams; first-class
functions and higher-order operations; and permits encoding many
temporal operations beyond streams, such as terminatable streams,
events, and even resumptions with first-class
schedulers. Furthermore, our language supports an efficient
implementation strategy permitting us to eagerly deallocate old
values and statically rule out spacetime leaks, a notorious source
of inefficiency in reactive programs. Furthermore, these memory
guarantees are achieved without the use of a complex substructural
type discipline.
We also show that our implementation strategy of eager
deallocation is safe, by showing the soundness of our type system
with a novel step-indexed Kripke logical relation.
You can download an almost
completely undocumented implementation to play with this
kind of FRP language. It's also good if you want to see what a
language with linear types looks like. Also, I use the type
inference algorithm described in the first draft. This doesn't
have dependent types (yet?), so no Mtac, though. :)
John retired recently, so I should probably post something in his honor. When I have time, I'll post a little something about some of John's work (with Peter O'Hearn) on how to use functor-category semantics as a guide to programmming with polymorphism. If you don't want to wait, you can look at their paper From Idealized Algol to Polymorphic Linear Lambda Calculus.
In my last post, I showed how you could implement the Int construction over resumptions, to derive a higher-order programming language from a first-order one. However, this is not yet a practical tool. The code I gave presented the language as a bunch of combinators, which rapidly get quite painful to write serious programs. Most of us prefer the lambda calculus to SKI combinators for good reason --- programs with variables and binders are much easier to read, since the code that actually does work in not obscured in a rat's nest of plumbing combinators that route data from one combinator to another.
I had hoped to rectify that in this post, by showing how to compile the linear lambda calculus into combinator terms. But when I started
to write it, I realized that I needed to write a warmup post first, which would introduce the basic ideas in a simpler setting. So I'll defer compiling linear types for a little while, and show how to embed total functional programming into OCaml.
In this post, I'll show how to turn the Geometry of Interaction construction --- a model of linear logic --- into a runnable Ocaml code. . The basic idea behind this paper is that we can view higher-order behavior, as a disciplined mode of use of a first-order language with recursion --- basically, if you fix a calling convention, then you can compile first-class functions directly and compositionally into a first-order program.
This is interesting because there are a lot of combinator libraries out there which can be seen as DSLs with a natural notion of "first-order program", but which lack a good notion of "higher-order program". (For example, most Haskell libraries based on arrows qualify.) As a programming model, this is very unsatisfying, since this restriction basically says that the author of the library can build interesting abstractions, but clients of the library can't. Wouldn't it be nice if there were a systematic way to turn any first-order language (with a bit of structure) into a higher-order one?
That's exactly what the G construction (also called the Int construction) does. It says that if you have a first-order language with sequential and parallel composition, plus feedback, then you've really got a higher-order language. Stated more formally, in categorical jargon, it says that there is a way of constructing a monoidal closed category from any symmetric monoidal category with a trace operator. (Monoidal products mean parallel composition, categories give you sequential composition, and trace is how category theorists pronounce recursion.)
Instead of giving the categorical derivation, I'll work very concretely, following the same example Abramsky gave in his classic paper Retracing Some Paths in Process Algebra.
First, let's get some Ocaml preliminaries out of the way. I'm just
defining function composition and empty and sum types, and their
operations. These should really be in the Ocaml Pervasives
environment, but they aren't. The Haskell prelude is nicer in this regard.
let($)fgx= f (g x)type('a,'b)sum= Inl of'a| Inr of'bletinlx= Inl x
letinry= Inr y
letcasefg=function Inl x-> f x | Inr y -> g y
letsumfg= case (inl $ f)(inr $ g)letswapv= case inr inl v
typezero= Void ofzeroletrecabort(Void v)= abort v
Our running example will come from dataflow and visual programming languages like Simulink. These languages have a model of programming in which you have a bunch of stateful components (adders, multipliers, accumulators), which are connected together by means of wires. This is naturally a first-order formalism, in that it really only makes sense to send data over the wires --- you can't send components over the wire. One way of modeling these kinds of stateful components is by means of resumptions.
A resumption from $I$ (read "input") to $O$ (read "output") is essentially an element of the solution to the domain equation $R = I \to (O \times R)$. You can think of a gadget of this type as something which takes an input, and then gives you an output, plus a new resumption to use on the next input. So it's a bit like a state machine, but formulated in such a way that you don't have to give an explicit state set. It's easy to turn this domain equation into a recursive ML type:
The identity resumption just echoes its input as its output.
(* val id : ('a, 'a) r *)letrecid= R(funx->(x, id))
To compose two resumptions $f$ and $g$, we just feed an input into $f$, and take $f$'s output as the input into $g$.
(* val ( >> ) : ('a, 'b) r -> ('b, 'c) r -> ('a, 'c) r *)letrec(>>)(R f)(R g)= R(funx->let(y,f')= f x inlet(z,g')= g y in(z, f' >> g'))
Note that with composition, we have a category $R$, the category of resumptions. The objects are types, a morphisms $f : I \to O$ is a resumption taking $I$ inputs and producing $O$ outputs.
This category is also "monoidal closed", with the tensor product $A \otimes C$ getting defined as the sum type $A + C$. The intuition is that if we think of $A$ and $C$ as message types, we may want to consider how to react to $A$ and $C$ messages simultaneously. In particular, if we have two machines $f : A \to B$ and $g : C \to D$, we can combine them by taking an $A+C$ message and dispatching to either $f$ or $g$ respectively.
(* val ( ** ) : ('a, 'b) r -> ('c, 'd) r -> (('a, 'c) sum, ('b, 'd) sum) r *)letrec(**)(R f)(R g)= R(function| Inl x->let(y,f')= f x in(inl y, f' **(R g))| Inr x ->let(y,g')= g x in(inr y,(R f)** g'))
We can also implement feedback --- if we have an $f : A \otimes B \to C \otimes B$, then we can construct a map $A \to C$ by feeding $f$ the $A$, and repeatedly sending the output back into the input until $f$ coughs up a $C$ rather than a $B$:
(* val trace : (('a, 'b) sum, ('c, 'b) sum) r -> ('a, 'c) r *)letrectracef= R(funa->letrecloop(R f)v=match f v with|(Inl c, f')->(c, trace f')|(Inr b, f')-> loop f' (inr b)in
loop f (inl a))
Now, we can also implement various associativity and commutativity properties. The names are not particularly great (they are just the standard names for the natural transformations associated with a tensor product in category theory), but all the real action is in the types.
(* val sym : (('a, 'b) sum, ('b, 'a) sum) r *)letrecsym= R(funv->(swap v, sym))(* val rho : (('a, zero) sum, 'a) r *)letrecrho:(('a,zero)sum,'a)r= R(function| Inl a->(a, rho)| Inr z -> abort z)(* val rho' : ('a, ('a, zero) sum) r *)letrecrho':('a,('a,zero)sum)r= R(funa->(inl a, rho'))(* val lambda : ((zero, 'a) sum, 'a) r *)letreclambda:((zero,'a)sum,'a)r= R(function| Inl z-> abort z
| Inr a->(a, lambda))(* val lambda' : ('a, (zero, 'a) sum) r *)letreclambda':('a,(zero,'a)sum)r= R(funa->(inr a, lambda'))(* val alpha : ((('a, 'b) sum, 'c) sum, ('a, ('b, 'c) sum) sum) r *)letrecalpha:((('a,'b)sum,'c)sum,('a,('b,'c)sum)sum)r= R(function| Inl (Inl a)->(inl a, alpha)| Inl (Inr b)->(inr (inl b), alpha)| Inr c ->(inr (inr c), alpha))(* val alpha' : (('a, ('b, 'c) sum) sum, (('a, 'b) sum, 'c) sum) r *)letrecalpha':(('a,('b,'c)sum)sum,(('a,'b)sum,'c)sum)r= R(function| Inl a->(inl (inl a), alpha')| Inr (Inl b)->(inl (inr b), alpha')| Inr (Inr c)->(inr c, alpha'))end
Now, we give the G construction in terms of the resumption module we've just defined. In the G construction, we build a new category $G(R)$ from the old one we've just defined. The basic idea here is to talk about bidirectional communication, and so for objects in our new category $G$ will be pairs of objects from our old one.
Objects of $G$ will be pairs $(A^+, A^-)$, where you can think of $A^+$ as a type of messages sent out, and the type $A^-$ as the type of messages that are accepted.
A morphisms of $G$, $f : (A^+,A^-) \to (B^+, B^-)$ will be resumption of type $A^+ \otimes B^- \to A^- \otimes B^+$. Note that the polarities of $A$ and $B$ are flipped in this mapping
-- we are transforming $A$ messages out to $B$ messages out, and we need to accept $B$ messages in order to transform them into $A$ messages in.
moduleG=structopenResumption
Now, let's define the type of morphisms in $G(R)$. Since we don't have type-level pairing in ML, we end up having four type parameters. We use a dummy G constructor to get the type system to produce more useful inferred types.
The identity map in $G(R)$ is just the symmetry map in $R$.
letgid= G sym
Composition is trickier. If we have a map $f : (A^+,A^-) \to (B^+, B^-)$ and $g : (B^+,B^-) \to (C^+,C^-)$, we can compose it by feeding $f$'s $B$-output into $g$'s input, and vice-versa, using the trace operator. It's best visualized with the following diagram:
Actually doing so requires applying the associativity and commutativity properties of sum types $A + B \simeq B + A$ and $(A + B) + C \simeq A + (B + C)$.
Once we have composition, we can define a tensor product on the $G(R)$ category as well, so that
$(A^+,A^-) \oplus (B^+,B^-) = (A^+ + B^+, A^- + B^-)$. Again, implementing the functorial action requires
implementing the associativity maps.
One cool thing about this category (which leads to the coolest fact about it) is that it naturally supports a dualization, which sends $(A^+,A^-)^\ast \triangleq (A^-,A^+)$. This also has an action on morphisms, which flips around the direction, so that from $f : (A^+,A^-) \to (B^+,B^-)$ we can get $f^\ast : (B^+,B^-)^\ast \to (A^+,A^-)^\ast$.
(* val dualize : ('a, 'b, 'c, 'd) map -> ('d, 'c, 'b, 'a) map *)letdualize(G (R f))=letrecdualf= R(funv->let(v',f')= f (swap v)in(swap v', dual f))in
G (dual f)
Now, here's what I consider the neatest fact about the Int construction -- it turns a traced monoidal category into a monoidal closed category. So we can define the linear exponential $(A^+,A^-) \multimap (B^+,B^-) \triangleq (A^+,A^-)^\ast \otimes (B^+,B^-)$.
The ML types for curry and uncurry somewhat
obscure the fact that they implement the isomorphism
$$\mathrm{Hom}((A^+,A^-) \otimes (B^+,B^-), (C^+,C^-)) \simeq
\mathrm{Hom}((A^+,A^-), (B^+,B^-) \multimap (C^+,C^-))$$
However, now that we have this isomorphism, it's possible to compile arbitrary lambda-terms in multiplicative linear logic down into compositions of resumptions. But that's a story for another post!
In the meantime, if this intrigues you, you can play with the Verity compiler by Dan Ghica and his students, as well as the IntML compiler by Ulrich Schöpp and Ugo Dal Lago, both of which are compilers using the GoI interpretation. Verity is aimed at distributed computing, and IntML is a programming language whose type system allows you to write only (and all) the LOGSPACE programs. As you can see, there are a lot of potential applications...
In my previous post, I mentioned limit-colimit coincidence, and how
the ability to define fixed points makes that possible. It turns out
there's a very pretty way of using parametricity to encode this. Now,
inductive and coinductive types are definable in System F as follows:
It's possible to show using parametricity that these type definitions
are actually the inductive and coinductive types we expect, whenever
F is a positive type. That is, a type constructor F
is positive whenever we have a function:
In lattice theory, you can apply the Knaster-Tarski theorem to get
a fixed point of a function f : L → L when L is a complete
lattice and f is monotone. Intuitively, you can see map
as the proof that the functor F is monotone (really, functorial)
in the lattice (really, category) of types.
It turns out that in a temporal setting, we also need to specify
that we can invoke the algebra/coalgebra operations at multiple
times, which we can specify using the modal stability connective
□A, which changes the definitions to:
Now, suppose we have a Nakano style recursive type μ̂α.F(•α), where
every occurrence of the recursive type occurs beneath a delay modality.
Then, assuming that F is positive, we can show how to embed elements of
νβ. F(•β) into elements of μα. F(•α) as follows:
embed : να. F(a) → μα. F(α)
embed pack(β, (seed, h₁)) = // seed : β, h₁ : □(β → F(•β))
let stable(f) = h₁ in // f : β → F(•β) at all times
Λα. λh₂. // h₂ : □(F(•α) → α)
let stable(g) = h₂ in // g : F(•α) → α at all times
let loop : β → α =
fix loop = λb:β. // in body, loop can only be used later
g(map [•α]
[•β]
(λd:•β. let •b' = d in •(loop b')) // has type •β → •α
(f b)) // has type F(•β)
in
loop seed
This suggests that guarded term-level recursion is enough to make certain
least and greatest fixed points collapse. I say "suggests", since proving that
these types are equivalent requires a parametric model, which I haven't actually
constructed yet. It also means that just polymorphism and the stability modality
□A are enough to construct an interesting reactive language, even
without a term-level fixed point fix x. e.
This function, minus the temporal staging, should also work for F plus
ordinary general recursion. I had just never thought about how the
coincidence plays out syntactically when you don't take recursive types as a
primitive!