Press "Enter" to skip to content

Posts published in “Computing”

The art and science of mechanized thought.

[seventy-two] minus sixty: a long, long time ago …

centaur 0

I saw some people blogging about their 20th blogging anniversaries, so I decided to check how long my blog has been up. And .. So! I apparently missed the blog's 20th birthday, as it started in November 2001 ...

... unless I blogged it and forgot about it. And I also missed my first (recorded) web page's 25th birthday ...

... as I started my website sometime in 1996.

So no birthday post for you. I guess I'll have to wait to the blog's 25th (or web page's 30th) birthday in 2026.

-the Centaur

[fifty-three] minus twenty-five: momentum is real

centaur 0

Now, see, this is what I was talking about last time. At the Game Developer's Conference today, after a long and informative series of talks, I ended up having an impromptu roundtable about language model planning, followed by a dinner with a friend talking about the history of the conference from a behind-the-scenes perspective.

The discussions were fascinating, and there are potentials for integrating these new language model technologies with older techniques like logic programming to very good results.

But, by the time the dinner was done, I was exhausted, and crashed back at my room, trying to sleep off some of the effects of two nights of rich dinners, no sleep, and hard-core information overload.

But I still had more work to do, creating my slides for the upcoming HRI in Academia and Industry Workshop at the AAAI Spring Symposium Series, not to mention my document updates on the main social navigation benchmarking paper itself. And, of course, all of that could not get finished in one night, not if I want to get up early enough to attend what are sure to be packed talks tomorrow morning.

But my point, and I did have one, is that if I had relied on myself to blog at the end of the day "when everything was done", I wouldn't have blogged at all, because everything is NOT done. But, since I had momentum from earlier in the day, it was easy to pull up the window and put together a quick post.

This post.

So, momentum is real. Once you start doing something, it's easier to keep doing it.

-the Centaur

Pictured: Various lines and slides from today's GDC, and a nice dessert from Amber India.

[fifty-two] minus twenty-six: oh yeah, gdc

centaur 0

Oh yeah, I only offhandedly mentioned - back at the Game Developer's Conference!

Hi Mom! Oh wait, she's gone. That ... went dark fast, Francis. Well, hopefully she's watching up there and is not too mad that I'm still wasting my time on such frivolous things. But, I got my job at Google through the AI Programmers' Roundtable in ... 2005, I think it would have been, so this is not frivolous to me. And it's a great place to find out what's going on in the field ...

... and I must say, the first talk out of the gate got dense, fast!

Just how I like it.

Back into the fray!

-the Centaur

I’m scared of Homebrew’s installation procedure, but I still love Homebrew

centaur 0

This is a followup to my Making Computers Useful series, started all the way back in 2014. (Funnily enough, the 2013-era iMac featured in that series is now pretty damn useless as it has fallen out of update range, and locks up if you run Dropbox and Google Drive at the same time).

But, the overall goal here is to document some of the stuff that I need to do to make computers work for me. Typically, there’s a lot of friction in software, and it takes a good bit of work to make that all function on a new machine. Sometimes that becomes a deep dive.

This is one of those stories.

So today, while updating the Embodied AI Workshop’s website prior to the launch of the 2023 version, I wanted to run the tree command. Tree is great because it can help you understand the structure of a directory tree, like so:

I felt I needed this because the Embodied AI website is built on yarn and gatsby, and it turned a relatively simple site into 1.6 gigabytes of generated junk (which I noticed when one of my older computers started literally wheezing as it tried to mirror all those unexpected files):

As it turns out, you can get tree via Homebrew. Homebrew is a “package manager,” kind of like an “app store for the command line,” and Homebrew helps you get standard Linux tools, like tree, onto your Mac so you can take advantage of all the hidden Unix goodness in your Macintosh. 

However … I’m a bit leery of Homebrew because this is how it installs itself: 

I mean, WHAT? curl a file and run it with bash? Seriously. Now, look, I’m not saying Homebrew isn’t safe - every indication is that it is - but that this METHOD of installation is a recipe for disaster. 

Why? Well, in case you’re not in the know, what this installation instruction is suggesting is to DOWNLOAD RANDOM CODE FROM SOMEWHERE ON THE INTERNET and RUN IT ON YOUR COMPUTER WITHOUT CHECKING IT AT ALL!

Nothing can go wrong with this plan.

Now, I’m no expert, but I’m familiar enough with this stuff to know what I’m doing. SO, first I checked with a few quick searches to see [is homebrew for mac safe] and it appeared to be.

SO I downloaded the software with JUST the CURL part, like so:

curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh > homebrew-install.sh

... so I could examine it more closely.

Folks, seriously, never do this on a site you do not trust.

After I had the code, I then inspected this homebrew-install.sh file to find out whether it was safe. I didn't see any obvious malware, but when I ran it, it wanted me to TYPE MY PASSWORD.

Seriously?

Please, I’m asking you, do not hot-pipe random software straight off the internet and run it straight from the command line and give it your password when it asks. If someone intercepts the website, and gets your password, they can do anything.

(SERIOUSLY. Once I was working with a legitimate Google representative about a Google ads program and when I went to log in to Google ads to check something, a hacker injected a fake Google ads site between me and Google, and damn near got my password. Only two-factor authentication saved me, as it broke some key link in the chain.)

BUT … it is the PATTERN I’m talking about here, not the specifics. Everything I’ve seen about Homebrew says that it is safe. I’ve even used it before, on other machines. SO, after some more research, and a little more code analysis, I confirmed this password-asking was safe, and gingerly went ahead.

And it went fine. 

I had to pay thirty million bitcoin to a Russian spammer, but I wasn’t using it anyway, and I’m sure at least they got to buy a cup of coffee or something with it. :-D

Seriously. It went fine. And I love Homebrew. I just go through this every time I need to “bash” run a piece of “curl”-ed software straight off the Internet and then it asks for my password.

Still, tree worked like a charm. (Screenshots of its use were above). There are more pieces of Homebrew software I need to install, but as one test, I tried to install “banner”, a program to create oversized pieces of text, which I use in scripts to alert me that a big task is done.

But, it seems like Mac already has a version of banner, which works differently on Mac than Linux, printing VERY large ASCII banners that are vertical rather than horizontal. That’s useful, but not for my case, so I dug around for an equivalent tool.  brew install figlet is the way to go:

All great! 

It didn’t help me with my work on the Embodied AI website, as I had already moved on to fixing other problems on that website, and was only “brewing” things in the background while I did other tasks (like remote-attend the church vestry retreat).

But removing this friction will help me in the future. The next time I need to examine the tree structure of a directory, it's one command away. I can put banners in my scripts. And I can easily add new software with 'brew' the next time it comes up.

AND, as a bonus, I discovered a site which is doing something very much like what I want to do with the Making Computers Useful series, Sourabh Bajaj’s Mac OS Setup Guide, which “... covers the basics of setting up a development environment on a new Mac.” I have an internal document called “Mac OS X New System Tasks” which documents for myself the travails I go through every time I get a new Mac, and Sourabh’s guide seems like it provides a public version of what I want to do. Which is great! Less work for me. ;-D

On to the next task!

-the Centaur

P.S. As another added bonus, I composed this in Google Docs, and pasted it straight into Gutenberg, the new Wordpress block editor. It worked like a charm ... EVEN DOWN TO PASTING IN THE IMAGES! If this is a feature of Gutenberg, I will have to consider saying my favorite three words about it ... "I was wrong."

P.P.S. Don't hold your breath on that, though, I'm waiting for the other shoe to drop.

[forty-three] minus twenty-two: it gets stale

centaur 0

Recently I went to do something in Mathematica - a program I've used hundreds if not thousands of times - and found myself stumped on a simple issue related to defining functions. I've written large, complicated Mathematica notebooks, yet this thing I done hundreds of times was stymieing me.

But - yes - I'd done it hundreds of times; but not regularly in the past year or so.

My knowledge had gone stale.

Programming, it appears, is not like riding a bike.

What about other languages? I can remember LISP defun's, mostly, but would I get a C++ class definition right? I used to do that professionally, eight years ago, and have published articles on programming C++ ... but I've been writing almost exclusively Python and related scripting languages for the past 7 years.

Surprisingly, my wife and I had this happen in real life. We went to cook dinner, and surprisingly found some of the stuff in the pantry had gone stale. During the pandemic, you see, we bought ahead, since you couldn't always find things, but we consumed enough of our staples that they didn't go stale.

Not so once the rate of consumption dropped just slightly - eating out 2-3 times a week, eating out for lunch 2-3 times a week - with a slight drop in variety. Which meant the very most common staples were consumed, but some of the harder-to-find, less-frequently-used stuff went bad.

We suspect some of it may have had near-expired dates we hadn't paid attention to, but now that we're looking, we're carefully looking everywhere to make sure our staples are fresh.

Maybe, if there are skills we want to rely on, we should work to keep those skills fresh too.

Maybe we need to do more than just "sharpen the saw" (the old adage that work goes faster if you take the time to maintain your tools). Perhaps the saw needs to be pulled out once a while and honed even if you aren't sawing things regularly, or you might find that it's gone rusty while it's been stored away.

-the Centaur

Pictured: The bottom layers of detritus of the Languages Nook of the Library of Dresan, with an ancient cast-off office chair brought home from the family business by my father, over 30 years ago.

[forty-two!] minus nineteen: well, at least i have a system now

centaur 0

Recently, when digging through old posts, I was reminded that Classic Editor posts are broken in WordPress - all the paragraph breaks are gone, and the content is mashed up into one grey wall of text. Thanks, WordPress, for forcing everyone to switch to a worse editing experience AND breaking all our old content.

[hang on a second, i have to start clicking around at random places on the page to try to find the widget or control that will let me start typing again after inserting an image, because software USABILITY has been replaced by "user experience" folks from a graphic design background who have mistaken making things LOOK GOOD IF THEY HAD BEEN PRINTED for the very different ACTUALLY WORKING WELL AS A TOOL - I'm looking at you, WordPress Gutenberg, Dropbox Paper, and everything like you where you have to hover or click or click and select and hover random parts of the page to make it work. Okay, I can start typing again.]

[[ and yeah it just did it again while i was just fricking typing ]]

Ok we're back.

Ok?

Ok.

Anyhoo, I have like a thousand old posts (1371 published, according to the dashboard), but the block converter for fixing these no longer works. I wish I had discovered this problem earlier, but I just didn't expect to have to do blog archaeology when I moved to Gutenberg.

Regardless, however, I now have a system. I open the All Posts page on the WordPress dashboard, and scroll backwards in time until Classic Editor posts start showing up - nice that they provide that nudge to get us to use the new editor, isn't it. Once I find some Classic Editor posts, if you hover - AAAAARRRRRGH, don't mind me - I say, if you hover, you get the option to open with the Block Editor. FORTUNATELY, this is ACTUALLY a link and not a bizarre Javascript pseudo-button - Good WordPress, Good WordPress, have a cookie - and a right click will allow you to open this in a NEW WINDOW.

SO! I go down one entire page of results, opening them in a new window, until I've hit all the Classic Editor posts on that page. This creates a gazillion tabs, true, but then you can click on each tab in turn, and there's a simple three-click process which will activate the block editor, convert the old text, and - BAM! - update. Optionally, one more click will bring up the updated post so you can doublecheck it before closing the tab.

The process is laborious - but it's easy to get a whole page full of results at a time, and you can't easily lose your place, as you close your tabs as you go. I've gotten through 3 pages of results so far, each with 50 posts, so I've updated probably something north of 150 pages.

There are 25 more pages of posts to go, but it doesn't take more than 30 minutes, so I can do one a day for about a month and rescue all the old pages.

A lot of work ... but at least I now have a system.

-the Centaur

Pictured: The House With The Impressive Tree In The Front Yard, found in a nearby neighborhood, as photographed in Night Mode on my Android phone during a walk with my wife.

Announcing Logical Robotics

centaur 0

So, I'm proud to announce my next venture: Logical Robotics, a robot intelligence firm focused on making learning robots work better for people. My research agenda is to combine the latest advances of deep learning with the rich history of classical artificial intelligence, using human-robot interaction research and my years of experience working on products and benchmarking to help robots make a positive impact.

Recent advances in large language model planning, combined with deep learning of robotic skills, have enabled almost magical developments in explainable artificial intelligence, where it is now possible to ask robots to do things in plain language and for the robots to write their own programs to accomplish those goals, building on deep learned skills but reporting results back in plain language. But applying these technologies to real problems will require a deep understanding of both robot performance benchmarks to refine those skills and human psychological studies to evaluate how these systems benefit human users, particularly in the areas of social robotics where robots work in crowds of people.

Logical Robotics will begin accepting new clients in May, after my obligations to my previous employer have come to a close (and I have taken a break after 17 years of work at the Search Engine That Starts With a G). In the meantime, I am available to answer general questions about what we'll be doing; if you're interested, please feel free to drop me a line at via centaur at logicalrobotics.com or take a look at our website.

-the Centaur

phewww ….

centaur 0

... finally, a chance to catch a break.

It's been a difficult few weeks due to "the Kerfluffle" which I hope to blog about shortly (those on my LinkedIn have seen it already) but equally as much from a Stanford extension class I was taking on Deep Reinforcement Learning (XCS234 - speaking as an expert in this area seeking to keep my skills sharp, I can highly recommend it: I definitely learned some things, and according to the graphs, so did my programs).

Finally, that's over, and I have a moment to breathe.

And maybe start blogging again.

-the Centaur

Pictured: A mocha from Red Rock Cafe, excellent as always, and a learning curve from one of my programs from class (details suppressed since we're not supposed to share the assignments).

finally ….

centaur 0

... a computer with a fast fricking hard drive. I use a backup scheme in which one older computer has all my file mirroring services on it (Dropbox, Google Drive, etc) and then backs that data up to a local Time Machine backup. But the old iMac I had had long ago reached the point where it couldn't back up to local network storage and needed a directly connected USB drive, and eventually that, too, gave up the ghost, with Google Drive and Dropbox essentially strangling each other to death if you tried to load them simultaneously. In other news, unrelated except for the inexorable passage of time, my personal daily driver laptop had reached the point where half the keys skip and the battery life was down to roughly 1 minute.

SO! I bought a refurbished Apple Silicon MacBook Pro. Even though it is a gently used machine, way cheaper than the most recent models, this M1 Max screamer has downloaded most of Dropbox and a large chunk of Drive without breaking a sweat. Apparently, the larger, faster SSD of a 2022 MacBook Pro beats the heck out of the old spinny hard drive of a 2015 (or is it 2013?) iMac. Who knew? And it can serve as a daily driver until such time as I can afford a top of the line machine, if I even need one if Apple Silicon is as fast as they say.

Cross you fingies ...

-the Centaur

Ugh, WordPress updates edition …

centaur 0

... the block editor of Wordpress seems to be making my old non-block-editor posts turn into solid walls of text. See the post "Pascal's Wager and Purchasing Parsley":

Yeah, it's not supposed to be looking like that. Gotta track those down and fix them.

In other news, my Half-Cheetah policy is successfully training to "expected" levels of performance. Yay! I guess that means my code for the assignment is ... sorta correct? Time to clean it up and submit it.

-the Centaur

Once again, I’m running deep learning on a Macbook …

centaur 0

... and the trick to getting it working was, as usual, "working just a little bit harder than you want to". Shortly after my last post, I got REINFORCE, a classic reinforcement learning algorithm, successfully training on my local machine, with apparent learning for all three environments in the assignment (though whether my solution is able to reach the expected final level of performance or not is still an open question).

-the Centaur

… and may I say,

centaur 0

the amount of work needed to put up that one-word, one-image blogpost was entirely out of proportion to the amount of benefit involved. I have fixed site errors with fewer hoops than it took to publish something via the WordPress app, and the fix was actually uninstalling and reinstalling the app, which apparently had gotten into some kind of cruftly state in which it could no longer upload posts.

To be clear, I'm not picking on WordPress here. But I have a Ph.D in Artificial Intelligence and used to work on the front end of Google search. If I can't post a one-word, one-image post on the world's most popular blogging platform using their own easy-to-use official phone app, how are people who have not spent thirty-plus years in the industry supposed to get any work done?

This experience I just had - almost the simplest possible post not uploading after a few minutes - in another industry would be like ... like .. like picking up a hammer and nailing one nail into a piece of wood, only to find the nails popping out a minute later and flying across the room. You ask your carpenter buddy, "what gives," and they say, "Oh, that. You've got hammer voodoo going on there. Just take the hammer back to Home Depot, return it, and buy a new one. Then the nail will go in just fine."

You know what? I'm going to learn from this.

I will endeavor to make the robots less irritating when something goes wrong.

-the Centaur

P.S. AAAA! And this post didn't publish because the interface threw up an extra dialog box after I tried to publish, asking, "Are you sure?" I'm sure I didn't need you throwing up that extra dialog box AFTER I left the page so I spent time looking for it on the home page when it hadn't actually published at all. Aaaa!

It is not like riding a bike.

RIP Jeff Bezos (and/or Richard Branson)

centaur 0
rip jeff bezos

You know, Jeff Bezos isn’t likely to die when he flies July 20th. And Richard Branson isn’t likely to die when he takes off at 9am July 11th (tomorrow morning, as I write this). But the irresponsible race these fools have placed them in will eventually get somebody killed, as surely as Elon Musk’s attempt to build self-driving cars with cameras rather than lidar was doomed to (a) kill someone and (b) fail. It’s just, this time, I want to be caught on record saying I think this is hugely dangerous, rather than grumbling about it to my machine learning brethren.

Whether or not a spacecraft is ready to launch is not a matter of will; it’s a matter of natural fact. This is actually the same as many other business ventures: whether we’re deciding to create a multibillion-dollar battery factory or simply open a Starbucks, our determination to make it succeed has far less to do with its success than the realities of the market—and its physical situation. Either the market is there to support it, and the machinery will work, or it won’t.

But with normal business ventures, we’ve got a lot of intuition, and a lot of cushion. Even if you aren’t Elon Musk, you kind of instinctively know that you can’t build a battery factory before your engineering team has decided what kind of battery you need to build, and even if your factory goes bust, you can re-sell the land or the building. Even if you aren't Howard Schultz, you instinctively know it's smarter to build a Starbucks on a busy corner rather than the middle of nowhere, and even if your Starbucks goes under, it won't explode and take you out with it.

But if your rocket explodes, you can't re-sell the broken parts, and it might very well take you out with it. Our intuitions do not serve us well when building rockets or airships, because they're not simple things operating in human-scaled regions of physics, and we don't have a lot of cushion with rockets or self-driving cars, because they're machinery that can kill you, even if you've convinced yourself otherwise.

The reasons behind the likelihood of failure are manyfold here, and worth digging into in greater depth; but briefly, they include:

  • The Paradox of the Director's Foot, where a leader's authority over safety personnel - and their personal willingness to take on risk - ends up short-circuiting safety protocols and causing accidents. This actually happened to me personally when two directors in a row had a robot run over their foot at a demonstration, and my eagle-eyed manager recognized that both of them had stepped into the safety enclosure to question the demonstrating engineer, forcing the safety engineer to take over audience questions - and all three took their eyes off the robot. Shoe leather degradation then ensued, for both directors. (And for me too, as I recall).
  • The Inexpensive Magnesium Coffin, where a leader's aesthetic desire to have a feature - like Steve Job's desire for a magnesium case on the NeXT machines - led them to ignore feedback from engineers that the case would be much more expensive. Steve overrode his engineers ... and made the NeXT more expensive, just like they said it would, because wanting the case didn't make it cheaper. That extra cost led to the product's demise - that's why I call it a coffin. Elon Musk's insistence on using cameras rather than lidar on his self-driving cars is another Magnesium Coffin - an instance of ego and aesthetics overcoming engineering and common sense, which has already led to real deaths. I work in this precise area - teaching robots to navigate with lidar and vision - and vision-only navigation is just not going to work in the near term. (Deploy lidar and vision, and you can drop lidar within the decade with the ground-truth data you gather; try going vision alone, and you're adding another decade).
  • Egotistical Idiot's Relay Race (AKA Lord Thomson's Suicide by Airship). Finally, the biggest reason for failure is the egotistical idiot's relay race. I wanted to come up with some nice, catchy parable name to describe why the Challenger astronauts died, or why the USS Macon crashed, but the best example is a slightly older one, the R101 disaster, which is notable because the man who started the R101 airship program - Lord Thomson - also rushed the program so he could make a PR trip to India, with the consequence that the airship was certified for flight without completing its endurance and speed trials. As a result, on that trip to India - its first long distance flight - the R101 crashed, killing 48 of the 54 passengers - Lord Thomson included. Just to be crystal clear here, it's Richard Branson who moved up his schedule to beat Jeff Bezos' announced flight, so it's Sir Richard Branson who is most likely up for a Lord Thomson's Suicide Award.

I don't know if Richard Branson is going to die on his planned spaceflight tomorrow, and I don't know that Jeff Bezos is going to die on his planned flight on the 20th. I do know that both are in an Egotistical Idiot's Relay Race for even trying, and the fact that they're willing to go up themselves, rather than sending test pilots, safety engineers or paying customers, makes the problem worse, as they're vulnerable to the Paradox of the Director's Foot; and with all due respect to my entire dot-com tech-bro industry, I'd be willing to bet the way they're trying to go to space is an oversized Inexpensive Magnesium Coffin.

-the Centaur

P.S. On the other hand, when Space X opens for consumer flights, I'll happily step into one, as Musk and his team seem to be doing everything more or less right there, as opposed to Branson and Bezos.

P.P.S. Pictured: Allegedly, Jeff Bezos, quick Sharpie sketch with a little Photoshop post-processing.

The Embodied AI Workshop is Tomorrow, Sunday, June 20th!

centaur 0
embodied AI workshop

What happens when deep learning hits the real world? Find out at the Embodied AI Workshop this Sunday, June 20th! We’ll have 8 speakers, 3 live Q&A sessions with questions on Slack, and 10 embodied AI challenges. Our speakers will include:

  • Motivation for Embodied AI Research
    • Hyowon Gweon, Stanford
  • Embodied Navigation
    • Peter Anderson, Google
    • Aleksandra Faust, Google
  • Robotics
    • Anca Dragan, UC Berkeley
    • Chelsea Finn, Stanford / Google
    • Akshara Rai, Facebook AI Research
  • Sim-2-Real Transfer
    • Sanja Fidler, University of Toronto, NVIDIA
      Konstantinos Bousmalis, Google

You can find us if you’re signed up to #cvpr2021, through our webpage embodied-ai.org or at the livestream on YouTube.

Come check it out!

-the Centaur

He thinks he invented Java because he was in the room when someone made coffee

taidoka 0

... came up as my wife and I were discussing the "creative hangers-on form" of Stigler's Law. The original Stigler's Law, discovered by Roger Merton and popularized by Stephen Stigler, is the idea that in science, no discovery is named after its original discoverer.

In creative circles, it comes up when someone who had little or nothing to do with a creative process takes credit for it. A few of my wife's friends were like this, dropping by to visit her while she was in the middle of a creative project, describing out loud what she was doing, then claiming, "I told her to do that."

In the words of Finn from The Rise of Skywalker: "You did not!"

In computing circles, the old joke referred to the Java programming language. I've heard several variants, but the distilled version is "He thinks he invented Java because he was in the room when someone made coffee."  Apparently this is a good description of how Java itself was named, down to at least one person  claiming they came up with the name Java and others disputing that, even suggesting that they opposed it, claiming instead that someone else in the room was responsible - while that person in turn rejected the idea, noting only that there was some coffee in the room from Peet's.

Regardless, I dispute Howard Aiken's saying "Don't worry about people stealing your ideas. If your ideas are any good, you'll have to ram them down people's throats." Nah. Once you've forced an idea down someone's throat, they won't just swallow it, they'll claim it was in their stomach all along.

-the Centaur

The Embodied AI Workshop at CVPR 2021

centaur 0
embodied AI workshop

Hail, fellow adventurers: to prove I do something more than just draw and write, I'd like to send out a reminder of the Second Embodied AI Workshop at the CVPR 2021 computer vision conference. In the last ten years, artificial intelligence has made great advances in recognizing objects, understanding the basics of speech and language, and recommending things to people. But interacting with the real world presents harder problems: noisy sensors, unreliable actuators, incomplete models of our robots, building good simulators, learning over sequences of decisions, transferring what we've learned in simulation to real robots, or learning on the robots themselves.

interactive vs social navigation

The Embodied AI Workshop brings together many researchers and organizations interested in these problems, and also hosts nine challenges which test point, object, interactive and social navigation, as well as object manipulation, vision, language, auditory perception, mapping, and more. These challenges enable researchers to test their approaches on standardized benchmarks, so the community can more easily compare what we're doing. I'm most involved as an advisor to the Stanford / Google iGibson Interactive / Social Navigation Challenge, which forces robots to maneuver around people and clutter to solve navigation problems. You can read more about the iGibson Challenge at their website or on the Google AI Blog.

the iGibson social navigation environment

Most importantly, the Embodied AI Workshop has a call for papers, with a deadline of TODAY.

Call for Papers

We invite high-quality 2-page extended abstracts in relevant areas, such as:

  •  Simulation Environments
  •  Visual Navigation
  •  Rearrangement
  •  Embodied Question Answering
  •  Simulation-to-Real Transfer
  •  Embodied Vision & Language

Accepted papers will be presented as posters. These papers will be made publicly available in a non-archival format, allowing future submission to archival journals or conferences.

The submission deadline is May 14th (Anywhere on Earth). Papers should be no longer than 2 pages (excluding references) and styled in the CVPR format. Paper submissions are now open.

I assume anyone submitting to this already has their paper well underway, but this is your reminder to git'r done.

-the Centaur

A Bayesian Account of Miracles

centaur 0
bayes headshot

Christianity is a tall ask for many skeptically-minded people, especially if you come from the South, where a lot of folks express Christianity in terms of having a close personal relationship with a person claimed to be invisible, intangible and yet omnipresent, despite having been dead for 2000 years.

On the other hand, I grew up with a fair number of Christians who seem to have no skeptical bones at all, even at the slightest and most explainable of miracles, like my relative who went on a pilgrimage to the Virgin Mary apparitions at Conyers and came back "with their silver rosary having turned to gold."

Or, perhaps - not to be a Doubting Thomas - it was always of a yellowish hue.

Being a Christian isn't just a belief, it's a commitment. Being a Christian is hard, and we're not supposed to throw up stumbling blocks for other believers. So, when I encounter stories like these, which don't sound credible to me and which I don't need to support my faith, I often find myself biting my tongue.

But despite these stories not sounding credible, I do nevertheless admit that they're technically possible. In the words of one comedian, "The Virgin Mary has got the budget for it," and in a world where every observed particle event contains irreducible randomness, God has left Himself the room He needs.

But there's a long tradition in skeptical thought to discount rare events like alleged miracles, rooted in  Enlightenment philosopher David Hume's essay "Of Miracles". I almost wrote "scientific thought", but this idea is not at all scientific - it's actually an injection of one of philosophy's worst sins into science.

Philosophy! Who needs it? Well, as Ayn Rand once said: everyone. Philosophy asks the basic questions What is there? (ontology), How do we know it? (epistemology), and What should we do? (ethics). The best philosophy illuminates possibilities for thought and persuasively argues for action.

But philosophy, carving its way through the space of possible ideas, must necessarily operate through arguments, principally verbal arguments which can never conclusively convince. To get traction, we must move beyond argument to repeatable reasoning - mathematics - backed up by real-world evidence.

And that's precisely what was happening right as Hume was working on his essay "Of Miracles" in the 1740's: the laws of probability and chance were being worked out by Hume's contemporaries, some of whom he corresponded with, but he couldn't wait - or couldn't be bothered to learn - their real findings.

I'm not trying to be rude to Hume here, but making a specific point: Hume wrote about evidence, and people claim his arguments are based in rationality - but Hume's arguments are only qualitative, and the quantitative mathematics of probability being developed don't support his idea.

But they can reproduce his idea, and the ideas of the credible believer, in a much sounder framework.

In all fairness, it's best not to be too harsh with Hume, who wrote "Of Miracles" almost twenty years before Reverend Thomas Bayes' "An Essay toward solving a Problem in the Doctrine of Chances," the work which gave us Bayes' Theorem, which became the foundation of modern probability theory.

If the ground is wet, how likely is it that it rained? Intuitively, this depends on how likely it is that the rain would wet the ground, and how likely it is to rain in the first place, discounted by the chance the ground would be wet on its own, say from a sprinkler system.

In Greenville, South Carolina, it rains a lot, wetting the ground, which stays wet because it's humid, and sprinklers don't run all the time, so a wet lawn is a good sign of rain. Ask that question in Death Valley, with rare rain, dry air - and you're watering a lawn? Seriously? - and that calculus changes considerably.

Bayes' Theorem formalizes this intuition. It tells us the probability of an event given the evidence is determined by the likelihood of the evidence given the event, times the probability of the event, divided by the probability of the evidence happening all by its lonesome.

Since Bayes's time, probabilistic reasoning has been considerably refined. In the book Probability Theory: The Logic of Science, E. T. Jaynes, a twentieth-century physicist, shows probabilistic reasoning can explain cognitive "errors," political controversies, skeptical disbelief and credulous believers.

Jaynes's key idea is that for things like commonsense reasoning, political beliefs, and even interpreting miracles, we aren't combining evidence we've collected ourselves in a neat Bayesian framework: we're combining claims provided to us by others - and must now rate the trustworthiness of the claimer.

In our rosary case, the claimer drove down to Georgia to hear a woman speak at a farmhouse. I don't mean to throw up a stumbling block to something that's building up someone else's faith, but when the Bible speaks of a sign not being given to this generation, I feel like its speaking to us today.

But, whether you see the witness as credible or not, Jaynes points out we also weigh alternative explanations. This doesn't affect judging whether a wet lawn means we should bring an umbrella, but when judging a silver rosary turning to gold, there are so many alternatives: lies, delusions, mistakes.

Jaynes shows, with simple math, that when we're judging a claim of a rare event with many alternative explanations, our trust in the claimer that dominates the change in our probabilistic beliefs. If we trust the claimer, we're likely to believe the claim; if we distrust the claimer, we're likely to mistrust the claim.

What's worse, there's a feedback loop between the trust and belief: if we trust someone, and they claim something we come to believe is likely, our trust in them is reinforced; if we distrust someone, and they claim something we come to believe is not likely, our distrust of them is reinforced too.

It shouldn't take a scientist or a mathematician to realize that this pattern is a pathology. Regardless of what we choose to believe, the actual true state of the world is a matter of natural fact. It did or did not rain, regardless of whether the ground is wet; the rosary did or did not change, whether it looks gold.

Ideally, whether you believe in the claimer - your opinions about people - shouldn't affect what you believe about reality - the facts about the world. But of course, it does. This is the real problem with rare events, much less miracles: they're resistant to experiment, which is our normal way out of this dilemma.

Many skeptics argue we should completely exclude the possibility of the supernatural. That's not science, it's just atheism in a trench coat trying to sell you a bad idea. What is scientific, in the words of Newton, is excluding from our scientific hypotheses any causes not necessary or sufficient to explain phenomena.

A one-time event, such as my alleged phone call to my insurance agent today to talk about a policy for my new car, is strictly speaking not a subject for scientific explanation. To analyze the event, it must be in a class of phenomena open to experiments, such as cell phone calls made by me, or some such.

Otherwise, it's just a data point. An anecdote, an outlier. If you disbelieve me - if you check my cell phone records and argue it didn't happen - scientifically, that means nothing. Maybe I used someone else's phone because mine was out of charge. Maybe I misremembered a report of a very real event.

Your beliefs don't matter. I'll still get my insurance card in a couple of weeks.

So-called "supernatural" events, such as the alleged rosary transmutation, fall into this category. You can't experiment on them to resolve your personal bias, so you have to fall back on your trust for the claimer. But that trust is, in a sense, a personal judgment, not a scientific one.

Don't get me wrong: it's perfectly legitimate to exclude "supernatural" events from your scientific theories - I do, for example. We have to: following Newton, for science to work, we must first provide as few causes as possible, with as many far-reaching effects as possible, until experiment says otherwise.

But excluding rare events from our scientific view of the world forecloses the ability of observation to revise our theories. And excluding supernatural events from our broader view of the world is not a requirement of science, but a personal choice - a deliberate choice not to believe.

That may be right. That may be wrong. What happens, happens, and doesn't happen any other way. Whether that includes the possibility of rare events is a matter of natural fact, not personal choice; whether that includes the possibility of miracles is something you have to take on faith.

-the Centaur

Pictured: Allegedly, Thomas Bayes, though many have little faith in the claimants who say this is him.

The Soul is the Form of the Body

centaur 0
aquinas headshot

If you've ever gone to a funeral, watched a televangelist, or been buttonholed by a street preacher, you've probably heard Christianity is all about saving one's immortal soul - by believing in Jesus, accepting the Bible's true teaching on a social taboo, or going to the preacher's church of choice.

(Only the first of these actually works, by the way).

But what the heck is a soul? Most religious people seem convinced that we've got one, some ineffable spiritual thing that isn't destroyed when you die but lives on in the afterlife.  Many scientifically minded people have trouble believing in spirits and want to wash their hands of this whole soul idea.

Strangely enough, modern Christian theology doesn't rely too much on the idea of the soul. God exists, of course, and Jesus died for our sins, sending the Holy Spirit to aid us; as for what to do with that information, theology focuses less on what we are and more on what we should believe and do.

If you really dig into it, Christian theology gets almost existential, focusing on us as living beings, present here on the Earth, making decisions and taking consequences. Surprisingly, when we die, our souls don't go to heaven: instead, you're just dead, waiting for the Resurrection and the Final Judgement.

(About that, be not afraid: Jesus, Prince of Peace, is the Judge at the Final Judgment).

This model of Christianity doesn't exclude the idea of the soul, but it isn't really needed: When we die, our decision making stops, defining our relationship to God, which is why it's important to get it right in this life; when it's time for the Resurrection, God has the knowledge and budget to put us back together.

That's right: according to the standard interpretation of the Bible as recorded in the Nicene creed, we're waiting in joyful hope for a bodily resurrection, not souls transported to a purely spiritual Heaven. So if there's no need for a soul in this picture, is there any room for it? What is the idea of the soul good for?

Well, quite a lot, as it turns out.

The theology I'm describing should be familiar to many Episcopals, but it's more properly Catholic, and more specifically, "Thomistic", teachings based on the writings of Saint Thomas Aquinas, a thirteenth-century friar who was recognized - both now and then - as one of the greatest Christian philosophers.

Aquinas was a brilliant man who attempted to reconcile Aristotle's philosophy with Church doctrine. The synthesis he produced was penetratingly brilliant, surprisingly deep, and, at least in part, is documented in books which are packed in boxes in my garage. So, at best, I'm going to riff on Thomas here.

Ultimately, that's for the best. Aquinas's writings predate the scientific revolution, using a scholastic style of argument which by its nature cannot be conclusive, and built on a foundation of topics about the world and human will which have been superseded by scientific findings on physics and psychology.

But the early date of Aquinas's writings affects his theology as well. For example (riffing as best I can without the reference book I want), Aquinas was convinced that the rational human soul necessarily had to be immaterial because it could represent abstract ideas, which are not physical objects.

But now we're good at representing abstract ideas in physical objects. In fact, the history of the past century and a half of mathematics, logic, computation and AI can be viewed as abstracting human thought processes and making them reliable enough to implement in physical machines.

Look, guys - I am not, for one minute, going to get cocky about how much we've actually cracked of the human intellect, much less the soul. Some areas, like cognitive skills acquisition, we've done quite well at; others, like consciousness, are yielding to insights; others, like emotion, are dauntingly intractable.

But it's no longer a logical necessity to posit an intangible basis for the soul, even if practically it turns out to be true. But digging even deeper into Aquinas's notion of a rational soul helps us understand what it is - and why the decisions we make in this life are so important, and even the importance of grace.

The idea of a "form" in Thomistic philosophy doesn't mean shape: riffing again, it means function. The form of a hammer is not its head and handle, but that it can hammer. This is very similar to the modern notion of functionalism in artificial intelligence - the idea that minds are defined by their computations.

Aquinas believed human beings were distinguished from animals by their rational souls, which were a combination of intellect and will. "Intellect" in this context might be described in artificial intelligence terms as supporting a generative knowledge level: the ability to represent essentially arbitrary concepts.

Will, in contrast, is selecting an ideal model of yourself and attempting to guide your actions to follow it. This is a more sophisticated form of decision making than typically used in artificial intelligence; one might describe it as a reinforcement learning agent guided by a self-generated normative model.

What this means, in practice, is that the idea of believing in Jesus and choosing to follow Him isn't simply a good idea: it corresponds directly to the basic functions of the rational soul - intellect, forming an idea of Jesus as a (divinely) good role model, and attempting to follow in His footsteps in our choice of actions.

But the idea of the rational soul being the form of the body isn't just its instantaneous function at one point in time. God exists out of time - and all our thoughts and choices throughout our lives are visible to Him. Our souls are the sum of all of these - making the soul the form of the body over our entire lives.

This means the history of our choices live in God's memory, whether it's helping someone across the street, failing to forgive an irritating relative, going to confession, or taking communion. Even sacraments like baptism that supposedly "leave an indelible spiritual character on the soul" fit in this model.

This model puts the following Jesus, trying to do good and avoid evil, and partaking in sacraments in perspective. God knows what we sincerely believe in our hearts, whether we live up to it or not, and is willing to cut us slack through the mechanisms of worship and grace that add to our permanent record.

Whether souls have a spiritual nature or not - whether they come from the Guf, are joined to our bodies in life, and hang out in Hades after death awaiting reunion at the Resurrection, or whether they simply don't - their character is affected by what we believe, what we do, and how we worship here and now.

And that's why it's important to follow Jesus on this Earth, no matter what happens in the afterlife.

-the Centaur

Day 057

centaur 0
Turing Drawing Alan Turing, rendered over my own roughs using several layers of tracing paper. I started with the below rough, in which I tried to pay careful attention to the layout of the face - note the use of the 'third eye' for spacing and curved contour lines - and the relationship of the body, the shoulders and so on. Turing Rough 1 I then corrected that into the following drawing, trying to correct the position and angles of the eyes and mouth - since I knew from previous drawings that I tended to straighten things that were angled, I looked for those flaws and attempted to correct them. (Still screwed up the hair and some proportions). Turing Rough 2 This was close enough for me to get started on the rendering. In the end, I like how it came out, even though I flattened the curves of the hair and slightly squeezed the face and pointed the eyes slightly wrong, as you can see if you compare it to the following image from this New Yorker article: Turing Photo -the Centaur

Free Will and the Halting Problem

centaur 0
turing headshot

Lent is when Christians choose to give things up or to take things on to reflect upon the death of Jesus. For Lent, I took on this self-referential series about Lent, arguing Christianity is following Jesus, and that following role models are better than following rules because all sets of rules are ultimately incompete.

But how can we choose to follow Jesus? To many Christians, the answer is simple: "free will." At one Passion play (where I played Jesus, thanks to my long hair), the author put it this way: "You are always choose, because no-one can take your will away. You know that, don't you?"

Christians are highly attached to the idea of free will. However, I know a fair number of atheists and agnostics who seem attached to the idea of free will being a myth. I always find this bit of pseudoscence a bit surprising coming from scientifically minded folk, so it's worth asking the question.

Do we have free will, or not?

Well, it depends on what kind of free will we're talking about. Philosopher Daniel Dennett argues at book length that there are many definitions of "free will", only some varieties of which are worth having. I'm not going to use Dennett's breakdown of free will; I'll use mine, based on discussions with people who care.

The first kind of "free will" is undetermined will: the idea that "I", as consciousness or spirit, can make things happen, outside the control of physical law. Well, fine, if you want to believe that: the science of quantum mechanics allows that, since all observable events have unresolvable randomness.

But the science of quantum mechanics also suggests we could never prove that idea scientifically. To see why, look at entanglement: particles that are observed here are connected to particles over there. Say, if momentum is conserved, and two particles fly apart, if one goes left, the other must go right.

But each observed event is random. You can't predict one from the other; you can only extract it from the record by observing both particles and comparing the results. So if your soul is directing your body's choices, we could only tell by recording all the particles of your body and soul and comparing them.

Good luck with that.

The second kind of "free will" is instantaneous will: the idea that "I", at any instant of time, could have chosen to do something differently. It's unlikely we have this kind of free will. First, according to Einstein, simultaneity has no meaning for physically separated events - like the two hemispheres of your brain.

But, more importantly, the idea of an instant is just that - an idea. Humans are extended over time and space; the brain is fourteen hundred cubic centimeters of goo, making decisions over timescales ranging from a millisecond (a neuron fires) to a second and a half (something novel enters consciousness.)

But, even if you accept that we are physically and temporally extended beings, you may still cling to - or reject - an idea of free will: sovereign will, the idea that our decisions, while happening in our brains and bodies, are nevertheless our own. The evidence is fairly good that we have this kind of free will.

Our brains are physically isolated by our skulls and the blood-brain barrier. While we have reflexes, human decision making happens in the neocortex, which is largely decoupled from direct external responses. Even techniques like persuasion and hypnosis at best have weak, indirect effects.

But breaking our decision-making process down this way sometimes drives people away. It makes religious people cling to the hope of undetermined will; it makes scientific people erroneously think that we don't have free will at all, because our actions are not "ours", but are made by physical processes.

But arguing that "because my decisions are made by physical processes, therefore my decisions are not actually mine" requires the delicate dance of identifying yourself with those processes before the comma, then rejecting them afterwards. Either those decision making processes are part of you, or they are not.

If they're not, please go join the religious folks over in the circle marked "undetermined will."

If they are, then arguing that your decisions are not yours because they're made by ... um, the decision making part of you ... is a muddle of contradictions: a mix of equivocation (changing the meaning of terms) and a category error (mistaking your decision making as something separate from yourself).

But people committed to the non-existence of free will sometimes double down, claiming that even if we accept those decision making processes as part of us, our decisions are somehow not "ours" or not "free" because the outcome of our decision making process is still determined by physical laws.

To someone working on Markov decision processes - decision machines - this seems barely coherent.

The foundation of this idea is sometimes called Laplace's demon - the idea that a creature with perfect knowledge of all physical laws and particles and forces would be able to predict the entire history of the universe - and your decisions, so therefore, they're not your decisions, just the outcome of laws.

Too bad this is impossible. Not practically impossible - literally, mathematically impossible.

To see why, we need to understand the Halting Problem - the seemingly simple question of whether we can build a program to tell if any given computer program will halt given any particular input. As basic as this question sounds, Alan Turing proved in the 1930's that this is mathematically impossible.

The reason is simple: if you could build an analysis program which could solve this problem, you could feed itself to itself - wrapped in a loop that went forever if the original analysis program halts, and halts if it ran forever. No matter what answer it produces, it leads to a contradiction. The program won't work.

This idea seems abstract, but its implications are deep. It applies to not just computer programs, but to a broad class of physical systems in a broad class of universes. And it has corollaries, the most important being: you cannot predict what any arbitrary given algorithm will do without letting the algorithm do it.

If you could, you could use it to predict whether a program would halt, and therefore, you could solve the Halting Problem. That's why Laplace's Demon, as nice a thought experiment as it is, is slain by Turing's Machine. To predict what you would actually do, part of the demon would have to be identical to you.

Nothing else in the universe - nothing else in a broad class of universes - can predict your decisions. Your decisions are made in your own head, not anyone else's, and even though they may be determined by physical processes, the physical processes that determine them are you. Only you can do you.

So, you have sovereign will. Use it wisely.

-the Centaur

Pictured: Alan Turing, of course.