On the Nature of Programming Languages

Link post

What are we do­ing when de­sign­ing a pro­gram­ming lan­guage? We de­cide whether it’s go­ing to be im­per­a­tive or declar­a­tive. We add a bunch of op­er­a­tors. We add some kind of ob­jects and visi­bil­ity rules. We de­cide to make it ei­ther strongly-typed or weakly-typed. Maybe we add gener­ics or in­her­i­tance, or maybe mul­ti­ple in­her­i­tance. And so on.

The ques­tion that in­ter­ests me is whether the na­ture of these tools is de­ter­mined by the prob­lems we want to solve, whether they are, in some way, in­her­ent to or, in other words, di­rectly in­ferrable from the na­ture of the prob­lem at hand, or whether they are rather crutches for our imag­i­na­tion, a purely psy­cholog­i­cal con­structs that help our im­perfect brains to deal with the com­plex­ity of the real world.

If you asked me ten years ago, I would prob­a­bly ar­gue for the lat­ter.

And it’s not hard to “prove” it: If two peo­ple write code to solve the same prob­lem and one makes a ter­rible spaghetti mon­ster in COBOL while the other goes for su­per-el­e­gant and highly ab­stracted solu­tion in Haskell, does it re­ally mat­ter to the com­puter? As long as the two are com­piled to the same ma­chine code, the ma­chine does not care. All the clever con­structs used, all the el­e­gance, they are there only to guide our in­tu­ition about the code.

So, you see, the de­sign of a pro­gram­ming lan­guage is de­ter­mined by hu­man psy­chol­ogy, by the quirks and foibles of the hu­man brain. If there are in­her­i­tance hi­er­ar­chies in your lan­guage it’s not be­cause things in the real world tend to ar­range in neat trees — in fact, they never do, and if you want to point out the tree of life, I have bad news for you: most or­ganisms have two par­ents and the tree of life is re­ally a DAG of life — it’s be­cause hu­man brains like to think in terms of tax­onomies. They just hap­pen to find it eas­ier to deal with the com­plex world in that par­tic­u­lar way.

But is it re­ally so?

If you asked me to­day I wouldn’t be so sure.

But it’s a ques­tion not easy to an­swer. If you wanted to re­ally tackle it you would need to take hu­man brain out of the equa­tion, so that you can find out whether the same kinds of “lan­guage con­structs” arise even with­out a brain hav­ing any­thing to do with it.

So here’s the idea: What about ge­netic al­gorithms? They don’t re­quire a pro­gram­mer. Yet we can look at them and de­ter­mine if a lan­guage fea­ture, say, en­cap­su­la­tion does emerge from the pro­cess.

Or, rather, given the rel­a­tive scarcity of data on ge­netic al­gorithms, let’s have a look at evolu­tion by nat­u­ral se­lec­tion. There’s a code (ge­netic code), just like any pro­gram­ming lan­guage it can be ex­e­cuted (thus pro­duc­ing a phe­no­type) and there’s definitely no brain in­volved. And we can ask: Is it just a ran­dom mess of in­struc­tions that hap­pens, by a blind chance, to pro­duce a vi­able phe­no­type or is there any in­ter­nal struc­ture to the code, some­thing that we would rec­og­nize as a fea­ture of a pro­gram­ming lan­guage?

And, I think, the an­swer may be yes.

Let’s have a look at the con­cept of “evolv­abil­ity”. What it says is that nat­u­ral se­lec­tion may, in some cases, may pre­fer in­di­vi­d­u­als who have no di­rect, phys­i­cal ad­van­tage, but whose progeny is more likely to adapt well to the chang­ing evolu­tion­ary pres­sures.

But what does that even mean? Well, here’s a con­crete ex­am­ple:

Imag­ine two gazel­les that are, phe­no­typ­i­cally, the same. They look ex­actly the same, they be­have the same etc. One would naively ex­pect that nei­ther of them would be preferred by nat­u­ral se­lec­tion.

But con­sider this: One of them has the length of the left rear leg en­coded in one gene and the length of the right rear leg in a differ­ent gene. The other has the length of both hind legs en­coded in a sin­gle gene. If that gene mu­tates both legs will be ei­ther longer or shorter, but they will never have differ­ent lengths.

Which of them is go­ing to fare bet­ter in the race for sur­vival?

I would bet on the sin­gle-gene one. When the en­vi­ron­ment changes and de­mands longer hind legs the evolu­tion­ary pro­cess would re­main blind, some of the offspring would have longer legs and some of them would have shorter legs, but at least there wouldn’t be the need to in­vest pre­cious re­sources in all those un­hope­ful mu­tants with left leg shorter than the right one.

What the con­cept of evolv­abil­ity says is that rep­re­sen­ta­tion mat­ters. The en­cod­ing isn’t, in the long run, se­lec­tively neu­tral.

And as pro­gram­mers we have a di­rect equiv­a­lent of the above: We call it sub­rou­tines. If the code is do­ing the same thing at two places, it’s bet­ter to cre­ate a sin­gle sub­rou­tine and call it twice than to make two copies of the code.

Yet an­other ex­am­ple: Do you think that an or­ganism in which ev­ery gene effects ev­ery part of its phe­no­type, the nose, the tail, the metabolism, the be­havi­our, is go­ing to fare bet­ter than an or­ganism where each gene is spe­cial­ized for a sin­gle con­crete task?

No, it’s not. Every fea­ture de­pend­ing on ev­ery gene means that ev­ery mu­ta­tion is go­ing to change the phe­no­type sig­nifi­cantly, in mul­ti­ple ways. It’s very likely that at least one of those changes is go­ing to be deadly.

And again, there’s a clear coun­ter­part to that in pro­gram­ming lan­guages. It’s called mod­u­lar­ity or, if you wish, en­cap­su­la­tion.

So, in the end, it seems the rep­re­sen­ta­tion, the lan­guage fea­tures mat­ter even if there’s no hu­man brain around to take ad­van­tage of them.

That is not to say that some of the lan­guage fea­tures aren’t purely brain-ori­ented. Sane vari­able nam­ing, for ex­am­ple, is likely to be such fea­ture.

But still, at least some of what we have prob­a­bly goes deeper than that and is, in fact, ob­jec­tively use­ful.

Now, to end with a lighter topic: Although I am a big fan of Stanisław Lem and there­fore I have pretty se­ri­ous doubts about whether we’ll be able to com­mu­ni­cate with the aliens if we ever meet them (hav­ing differ­ent so­ciety, differ­ent biol­ogy, differ­ent brain, differ­ent ev­ery­thing is not go­ing to make it easy) the rea­son­ing above gives us at least some hope. If both we and the aliens write com­puter pro­grams they are prob­a­bly go­ing to share at least some fea­tures (sub­rou­tines, mod­u­lar­ity). Cal­ling that com­mon­al­ity “un­der­stand­ing” may be an ex­ag­ger­a­tion but it’s still bet­ter than noth­ing.

April 22nd, 2019

by mar­tin_sustrik