Syntax vs semantics: alarm better example than thermostat

I had a post on em­piri­cally bridg­ing syn­tax and se­man­tics. It used the ex­am­ple of tem­per­a­ture, build­ing on McCarthy and Searle’s dis­pute about the be­liefs of ther­mostats.

But tem­per­a­ture wasn’t an ideal illus­tra­tion of my points, as hu­mans are not fine in their tem­per­a­ture sen­si­tivity, so I’m pre­sent­ing a bet­ter ex­am­ple here: de­tect­ing an in­truder.

In­ter­nal and ex­ter­nal variable

The ex­ter­nal vari­able is a boolean which cor­re­sponds to whether there is any hu­man in a cer­tain ini­tially empty green­house.

There are five differ­ent “agents” with in­ter­nal vari­ables :

  • An alarm on the door of the green­house, which goes off it a cir­cuit is bro­ken by the door be­ing opened; in­ter­nal vari­able .

  • A heat-de­tect­ing cam­era that starts an alarm if there is a some­thing vaguely hu­man sized and hu­man-tem­per­a­ture in­side the green­house (which is made out of sap­phires, ob­vi­ously); in­ter­nal vari­able .

  • A mo­ti­vated hu­man guard who pe­ri­od­i­cally looks into the green­house; in­ter­nal vari­able .

  • A re­source­ful hu­man with a lot of time and money, solely ded­i­cated to de­tect­ing any in­tru­sion into the green-house; in­ter­nal vari­able .

  • A su­per­in­tel­li­gent robot ver­sion of the re­source­ful hu­man; in­ter­nal vari­able .

Then all the cor­re­late well with the in a lot of cir­cum­stances. If a passerby or a naive bur­glar get into the green­house, they will trig­ger the door alarm and the heat alarm, while the guard, the re­source­ful hu­man, and the robot will all see the in­truder.

It is, how­ever, pretty easy to fool the door alarm: sim­ply go through a win­dow. Con­versely, some­one could open the door with­out en­ter­ing (or the wind or an earth­quake could do so), caus­ing the alarm to trig­ger with no-one in the green­house. So and are cor­re­lated in a rel­a­tively nar­row set of en­vi­ron­ments . And if we con­sider in­stead the vari­able “the elec­tric cir­cuit that goes through the door is un­bro­ken”, then it’s clear that and are much bet­ter cor­re­lated than and ; if there’s a se­man­tic mean­ing to , then it’s far closer to than it is to .

The heat-cam­era can also be fooled. Sim­ply spray the lense with some in­frared-opaque paint, then en­ter at your leisure. For the con­verse, maybe a en­ter­ing bear could trig­ger the alarm. It seems clear that is cor­re­lated with in a much wider set of en­vi­ron­ments, .

The hu­man guard is hard to fool in ei­ther di­rec­tion. We hu­mans are very good at figur­ing out when other hu­mans are around, so, as­sum­ing the guard is mod­er­ately at­ten­tive, trick­ing the guard in ei­ther di­rec­tion re­quires a lot of work—though it is prob­a­bly eas­ier to trig­ger a false pos­i­tive (the guard mis­tak­enly thinks that there’s a per­son in the green­house) than a false nega­tive (the guard doesn’t no­tice some­one ac­tu­ally in the green­house). Con­fus­ing or over­whelming the guard be­comes pos­si­ble for in­tel­li­gent ad­ver­saries. Still, the set of en­vi­ron­ments where is cor­re­lated to is much larger.

The re­source­ful hu­man is even harder to fool, be­cause they have all the ad­van­tages of the guard, plus any ex­tra pre­cau­tions they may have taken (such as adding alarms, cam­eras, crowds of on­look­ers, etc...). So is larger still.

Fi­nally, bring­ing in a su­per­in­tel­li­gence re­ally ex­tends the ac­cu­racy of , even against in­tel­li­gent ad­ver­saries, so is again much larger than any of the pre­vi­ous sets of en­vi­ron­ments.

Not strict in­clu­sion, not perfect correlation

The agents above are on a hi­er­ar­chy: ev­ery one of them has a much larger set of en­vi­ron­ments where is cor­re­lated with , than do any of the ones be­fore that agent.

But none of the in­clu­sions are strict. If some­one sprays the heat-sen­si­tive cam­era but then walks in through the door, the door-alarm will de­tect the in­tru­sion even as the cam­era misses it. If some­one dis­guises them­selves as a table, they might be able to fool the guard but be caught by the cam­era. The re­source­ful hu­man has their own per­son­al­ity, so there might be some ma­nipu­la­tion of them that would fall flat for the guard.

And fi­nally, even a su­per­in­tel­li­gence is com­putable, so the No Free Lunch the­o­rems im­ply that there are some, stupidly com­pli­cated, en­vi­ron­ments in which , , , and are all equal to , but is not.

Since no com­putable agent can have a perfect cor­re­la­tion with the vari­able in ques­tion, there is a sense in which no sym­bol can be perfectly grounded (this gets even more ob­vi­ous when you start slic­ing into the defi­ni­tion, and start won­der­ing about the mean­ings of “hu­man” and “a cer­tain green­house” in ).

But, de­spite the lack of perfect in­clu­sion and perfect cor­re­la­tion, there is a strong sense in which the later agents are bet­ter cor­re­lated than the ear­lier ones. As­sume that we have a sen­si­ble com­puter lan­guage to pick a com­plex­ity prior in, and up­date on the world be­ing roughly as we be­lieve it to be. Then I’d be will­ing to wa­ger that the pos­te­rior prob­a­bil­ities of the en­vi­ron­ments in which there are cor­re­la­tions, will be or­dered:

  • .