Press "Enter" to skip to content
Featured

Embodied AI and Unsolved Problems in Social Robot Navigation

centaur 0

Hey folks! One more sticky post reminding y'all that coming up next week is the Fifth Annual Embodied AI Workshop, Tuesday, June 18th, at CVPR, the Computer Vision and Pattern Recognition Conference. I am the lead organizer, though you'll probably only hear me yapping for any extended length of time if you show up in the first ten minutes - I give the intro talk at 8:50am.

Next is the Workshop on Unsolved Problems in Social Robot Navigation, held at the Robotics, Science and Systems Conference. Our paper deadline is coming up June 7th, and the workshop itself will be held July 19th at RSS in the Netherlands. I'm an organizer for this one, but I'll only be able to attend virtually due to my manager (me) telling me I'm already going to enough conferences this year, which I am.

After that .... Dragon Con!

-the Centaur

Pictured: Again, from the archives, until I fix the website backend.

[twenty twenty-four day one six eight]: what ISN’T embodied AI?

centaur 0
two hangry cats

The Embodied AI Workshop is coming up this Tuesday, starting at 8:50am, and I am busy procrastinating on my presentation(s) by trying to finish all the OTHER things which need to be done prior to the workshop.

One of the questions my talk raises is what ISN'T embodied AI. And the simplest way I can describe it is that if you don't have to interact with an environment, it isn't embodied.

Figuring out that the golden object on the left and the void on the right is a tremendously complex problem, solved by techniques like CNNs and their variants Inception and ResNet.

But it's a static problem. Recognizing things in the image doesn't change things in the image. But in the real world, you cannot observe things without affecting them.

This is a fundamental principle that goes all the way down to quantum mechanics. Functionally, we can ignore it for certain problems, but we can never make it go away.

So, classical non-interactive learning is an abstraction. If you have a function which goes from image to cat, and the cat can't whap you back for getting up in its bidnes, it isn't embodied.

-the Centaur

Pictured: Gabby, God rest his fuzzy little soul, and Loki, his grumpier cousin.

[twenty twenty-four post one six six]: what is embodied AI?

centaur 0
big red stop button for a robot, i think from bosch

So, as I've said, Embodied AI is just around the corner. But what is this workshop about? Embodied AI, of course! It says so on the tin.

But the key thing that makes "embodied AI" different from computer vision is that you must interact with an environment; the key thing that makes "embodied AI" different from robotics is that technically it doesn't need to be a real physical environment, as long as the environment is dynamic and there are consequences for actions.

SO, we will have speakers talking about embodied navigation, manipulation, and vision; generative AI to create environments for embodied agents; augmented reality; humanoid robots; and more.

Okay, now I really am going to crash because I have to fly tomorrow.

Onward!

-the Centaur

Pictured: An e-stop (emergency stop) button from a robot. Looks a little jury-rigged there, Chester.

[twenty twenty-four post one six five]: embodied ai is almost here

centaur 0

Ok, the image is from ICRA, but I am still traveling, and have not fixed the problem on the website backend. BUT, Embodied AI is this coming Tuesday, so please drop in if you are at CVPR!

More later, I had several long days at the customer site and I am going to go crash now.

-the Centaur

[twenty twenty-four post one six four]: it’s not every bite, but their sum

centaur 0

So I had a really good set of meals in Vancouver over the last few days - one at old favorite Gotham Steakhouse, one at my consulting client's office where they ordered Persian from Shishlik Grill in for lunch, and one vegan meal at my other old favorite, the Lebanese restaurant Nuba in Gastown.

And it struck me, as a foodie, that while these meals were good, their sum was better than their individual bites. In particular, the hummus-tabbouleh-falafel-pita combo at Nuba was solid all around - definitely good but not the best I've ever had - but the sum of all of them into a meal was extraordinarily satisfying.

This is true even in the case where the food itself is extraordinary. One of my favorite meals is the blackened salmon quesadilla at Aqui's - it's off-menu, so you have to know that you can order it, and how - and while that quesadilla is one of the best food items ever, it's the whole plate - the mango salad, the tropical tea, and the special combination of salsas and pickled jalapenos that I add to it - that takes it over the top.

I mean, in one sense, I knew that - I knew a great meal wasn't just one great dish - but walking out of Nuba today, with a really great, really satisfying Lebanese meal in me - really struck that home.

Blogging every day.

-the Centaur

Pictured: Three great meals (or drinks) from the archives: a cauliflower steak (which they also have, in another form, at Nuba), a Page One or Cafe Salzburg from Cafe Intermezzo, and the blackened salmon quesadilla at Aqui's (mango salad, with my custom four-flavor salsa combo on the side).

[twenty twenty-four post one six three]: paranoia will ensure ya

centaur 0

Okay, I was flying Tuesday, so I'm just going to pretend this was an abbreviated post, something something busy busy something something flying to Vancouver something something robot consulting.

At least I didn't try to fly on an expired passport ... this time. Strange how paranoid a mistake can make you! Like how I missed a flight - two days in a row - trying to leave London, ~30 years ago, the first time due to my mistake, the second due to a train stoppage, so I now try to go to airports ~2 hours early ... and missing my flight to Comic-Con due to traffic made me paranoid enough to leave ~3 hours early in LA's rush hour traffic so I'd have time to make it through any unexpected snafus with my international flight.

But that paranoia got me there safely and on time ... this time.

-the Centaur

Pictured: Me, at some event in 2015 ... wait, I owned this scarf in 2015???

[twenty twenty-four post one six two]: behindiness

centaur 0

Super far behind, because we're in "the stretch" leading up to Embodied AI Five - which also happens to be the week of a site visit at one of my consulting clients. So, this past Monday, I met with them online, took care of some Neurodiversiverse stuff, met friends for dinner, then started packing to fly.

And, while I did draw, I forgot to blog. Mucha-girl disapproves.

Still, blogging every day, even if I have to backfill.

-the Centaur

Pictured: Detail of Alphonse Mucha's poster for Princess Hyacinth, incorporating, when you look more closely, a disturbingly strong right arm on the princess there - in my mind, probably symbolizing both her father, the blacksmith, and probably echoing Mucha's pro-Slavic symbolic interest in the goddess Slavia.

[twenty twenty-four post one six one]: one more thing

centaur 0

So one thing here to remind myself to blog about it in more details - I attended a panel at the Nebulas on "Moving Beyond Milford" which was very useful. Milford, for those not steeped in writerly inside baseball, refers to the Milford Writer's Workshop, or, more generally, a critiquing model in which a group gets together, shares stories to read in advance, and everyone critiques each other.

The key element of Milford is that each person in the group gets their turn to critique - say, four minutes - during which nobody else can speak - not the other authors, not other group members, with the exception of a facilitator who can keep things on track. And most people seem to agree that the gag rule is critical to Milford in that it helps authors to learn to take criticism - and shuts up "that guy" so he doesn't dominate the critique.

But it can cause people to pile on, or for criticism to be repetitive, or even misplaced. So people were recommending different approaches - online, threaded critique, or structured critique where you had to start off with what resonated with you in the story before critiquing it, or encouraging facilitated discussion so everyone doesn't pile on with dittos.

I had "office hours" at the Nebulas, during which I advised other authors on problems - not because I'm some super experienced author or anything, but simply because I'm an editor and publisher, which means authors who are arguably equally or more experienced than me thought they might benefit from talking to an editor and publisher about specific problems. Which we did, with a couple of people.

And, when I did, I took the advice of the "Moving Beyond Milford" panel: I reformulated my critique into a five-point breakdown:

  • What did I think the story was about? Reiterating what the story is about ensured that I "got" what the author was trying to do, in an attempt to head off at the pass any misunderstandings.
  • What did I like about the story? Identifying what resonated with you about the story helps the author understand what's working about the story which they probably shouldn't change.
  • What areas of improvement did I see? This is something that can crush newbie authors - or experienced authors hit with impostor's syndrome - so it's important to formulate this in terms of suggestions.
  • What features or turns of phrase stuck with me? These might be small things, but I think highlighting key sentences, elements of description, or ideas are important to remind authors they can be effective.
  • What areas could potentially use copyediting? If there are typos, grammatical errors, or other opportunities for low-level textual improvement, highlight them here.

But even though that's what I used when I analyzed the story, that's not how I presented the above material to the author in our meetings. What I did instead was use the following script (after the meet-and-greet):

  • Here's what I think I read. I started off by briefly reiterating what I thought the story was about, so the author knew I had read their piece (and we could clear up any misconceptions).
  • What do you want help with? I then asked the author to explain what areas they needed help in. If anything about that wasn't clear, I asked them to explain in their own words the problem.
  • Let's brainstorm solutions to your problem. Before digging into my notes, we discussed their problem in greater depth and used story structure ideas to start looking for solutions.
  • Let's discuss where my notes intersect with your concerns. Then, we dug into where my notes intersected their problems, focusing on the parts where they needed help.
  • Once we have ideas about a solution, then share the other notes. Where the other notes were still relevant, I shared them, trying to build suggestions about how to make the story stronger.

Overall, I wanted to not dive in with how I thought the story could be better, but to improve the author's experience working with the story first, then focus on how my thoughts about the story could help them.

I think this is a better approach than tackling the story proper as an entity divorced from its author. Once a story is done, we can talk about the text as an entity independent of its author, but BEFORE the story is done, it's a work in process being worked on by a real human being, asking for help.

When critiquing, put helping the author first, and worry about your personal pet peeves some other time.

-the Centaur

Pictured: Some writing advice from me, from back in the day, while blog image uploading is down.

[twenty twenty-four day one six oh]: zero inbox

centaur 0

Okay, the Nebulas are over, and I should blog about that, and I'm a day or two ahead on drawing, which I should post when I get the website backend fixed, and I'm a day behind on blogging, so I should get caught up on all that.

But I just spent about two hours pursuing, and achieving, zero inbox across all three of my major email accounts, so I am VERY tired, and I am going to crash shortly.

Zero inbox is the discipline of clearing ALL the messages from your inbox - either by handling them, or categorizing them into folders for further action. This means what comes into the inbox in the future can be more quickly dealt with (or more easily unsubscribed from).

Now, I have a LOT of email in the folders I filed - probably hundreds of messages. But I had at least twenty thousand messages built up across all three accounts, most of which were spam, promotions, social media notifications, forum posts, or other notifications which were functionally worthless.

Now, even though there are hundreds of messages to process ... they're just in the hundreds.

And that feels way more doable.

Okay I go crash now.

-the Centaur

Pictured: A blast from the past in the Atlanta Airport (while blog images are still down).

[twenty twenty-four day one five nine]: fight the molasses

centaur 0

So! I made it. I'm at the hotel where the Nebulas will be happening, and by the time you read this (according to my records this should be post 159, and according to this day-of-the-year calendar that should be Friday, but as I type it's a hair before midnight Wednesday night) I should have already done the first of my events.

But I also have a day job, and since I was "drawing, on average, every day" and reading Neal Asher's Shadow of the Scorpion on the flight out, I took advantage of my oops-you-checked-in-in-advance-but-we-didn't-reserve-your-actual-room drink voucher and got an Old Fashioned at the soon-to-close hotel bar.

(Apologies to the hotel bar staff: I came in 30 minutes before close, just as they were cleaning up and switching over to sidework, but me plopping myself down at the bar apparently opened the floodgates, for something like half a dozen people then showed up right after I did).

At the bar, I cracked open Visual Studio Code, ChatGPT and Stack Overflow in an attempt to find a more parsimonious dataset representation for one of my clients. I'd built a horribly data-inefficient version of a machine learning dataset for them on the principle "get the fucker running so we can see whether it works" but the fucker worked, so we need now to make it at least marginally more efficient as we now turn our attention to "let's see whether this fucker can fucking scale up."

It looks like a data representation called HDF5 is worth a first shot (not that it's the best or the only, but it has C++ and Python bindings and appears simple to integrate into both our custom data set writer and into our custom PyTorch data loader). So, I did a little digging via Bing/Google to verify the best way to install HDF5 for Python (h5py for Conda, in this case) and set down to try out ChatGPT's recommended test case.

But ... the installation locked up.

Restarted the install. No dice. Then I thought it was the janky hotel Wi-Fi. Switched to my own personal hotspot. No dice. Tried a bunch of StackOverflow recommendations to fix the problem. No dice. Fifteen, then thirty, then forty-five minutes stretched by, as I tried to get a simple darn package to load.

This is, as I've said before, the problem of "molasses" in computer programming: the gummy gook which makes it impossible to do simple tasks. Another colleague called it "the novice penalty, and it's real": people who work in a domain all the time learn the tricks to make it work, but novices don't know these tricks, and struggle to do things that "experts" think are easy because they've forgotten they are difficult.

I almost gave up. But molasses needs to be fought. As I often say, oftentimes, you need to work a little bit harder than you think you need to, and when you do, you'll find that you're greatly rewarded by a breakthrough. Molasses can gum us up, but if we push through, we may find that it becomes smooth sailing.

In this case, the solution was actually to use ChatGPT's suggestion for installing the HDF5 package: 'pip install h5py', rather than ' conda install anaconda:h5py'. The benefit of doing it the 'conda' way is that the installation is in a 'Python environment' that corrals the installed software so it doesn't break anything else; but, for whatever reason, my conda environment was having trouble with that, so pip - which installs the program globally on the computer, across 'environments' - was the way to go,

From there I was able to start making progress on my dataset loader problem, and have a clear direction for the project to take tomorrow. Had I accepted the slowdown imposed by the molasses, I would have returned to this problem tomorrow with no real clue of the next steps to take, other than remembering that I had tried a bunch of stuff, got exhausted, and decided to start fresh in the morning.

Sometimes that's the right thing to do, of course. But if we can push through the molasses through to the other side, we often will be doubly rewarded: not only will we solve the immediate problem we were facing, but also will have a solid foundation to move forward on our next task.

So don't let the molasses bog you down. Push on through, and leave it behind if you can.

-the Centaur

Pictured: One from the archives - some Mathematica analysis of a problem - while the blog images are down.

[twenty twenty-four day one five eight]: precognitively at the nebulas

centaur 0

In case the travel process goes kazoo, I'm nabbing tomorrow's post in today - looks like on June 6th I will be at the meet and greet space Thursday at 2:30pm (Meet-and-Greet Table E Anthony Francis Pasadena – Fountain Foyer), no matter what the sticky post says.

I will have books, and will sign them; hope to see you there.

Zoom zoom.

-the Centaur

Pictured: a gif way back from an old Dragon Con.

[twenty twenty-four day one five seven]: on my way to the nebulas

centaur 0

Packing today (yesterday), flying tomorrow (today), so no real post for you. Yesterday I got up to drawing one five six and took in pushups and a walk plus Spectral Iron editing, so I'm caught up for yesterday, not so much for today, so I'll need to squeeze in some drawing time. Pictured: me at Dragon Con um ... 2019 ... which is a stand-in for some Nebula-specific post which I can't do until I update the blog backend.

Blogging every day, though. Keeping a regular habit helps.

-the Centaur

[twenty twenty-four day one five six]: ugh and not so ugh

centaur 0

So the broken door lock was indeed not a problem either power tool girl or I could have easily handled - it took the locksmith almost forty five minutes to lever, chisel and snip the latch assembly out of the door, using quite a bit of specialized equipment -air pumps and such - to try to create space before finally giving up and applying judicious elbow grease, a wrench and a hammer. When he finally got it out, pieces abruptly tumbled out in a tiny little rain of already-broken parts from deep within the latch assembly.

But the repair itself was cheap, and the same guy offered a great rate to re-key our other locks as well. So we now have easy access to my office again, and a plan for fixing some of the dead old locks around this rambling home. One ugh problem may just have made another ugh problem go away - which suggests that when you're facing a lot of problems, you should just dig in and try to fix them, one by one, until hopefully all those problems go away, leaving you with new problems for a new day.

-the Centaur

Pictured: Downtown Greenville's Falls Park, which is a beautiful place for a walk. Since, in the intervening hour since my last post, I haven't fixed the backend of the blog. Another ugh ...

Blogging every day: Today's exercise was thirty push ups, and probably a walk later, maybe or maybe not at Falls Park. Yesterday's exercise, which I didn't blog, was sixty pushups and an excursion in the attic. Yesterday's read was Neal Asher's PRADOR MOON, which I quite liked; today will be packing for the Nebulas. Yesterday's editing was Dakota Frost #4, SPECTRAL IRON; probably also that today. Drawing is up to one five five, so I need to work in a drawing today, ideally two so I don't have to worry about it while traveling.

The Nebulas, Embodied AI, and Unsolved Social Navigation

centaur 0

Hey folks, another "sticky post" for my next three events: The Nebula Conference, the Embodied AI Workshop, and the Workshop on Unsolved Problems in Social Robot Navigation!

Coming up is the Nebula Conference, the Science Fiction and Fantasy Writers of America's conference, where again I'll be talking about artificial intelligence and signing books!

  • Thursday, June 6th: 2:30pm: Anthony Francis Meet-and-Greet Table E Pasadena
  • Saturday, June 8th: 10:30am: Why AI Needs Humans Pasadena

There will also be office hours but I think you have to sign up in advance for those, and we have a scheduling snafu regarding those anyway.

After that is the Embodied AI Workshop on June 18th, hosted at CVPR, the Computer Vision and Pattern Recognition Conference, where I am the lead organizer, though you'll probably only hear me yapping for any extended length of time if you show up in the first ten minutes - I give the intro talk at 8:50am.

Finally is the Workshop on Unsolved Problems in Social Robot Navigation, held at the Robotics, Science and Systems Conference. Our paper deadline is coming up June 7th, and the workshop itself will be held July 19th at RSS in the Netherlands. I'm an organizer for this one, but I'll only be able to attend virtually due to my manager (me) telling me I'm already going to enough conferences this year, which I am.

After that .... Dragon Con!

-the Centaur

Pictured: More from the archives, until I fix the website backend.

[twenty twenty-four day one five five]: late post for you

centaur 0
Snow in South Carolina

Last night was date night, but we also had to climb up into the attic to see what's up with this "leaky" roof. But it's bone dry up there, despite the recent rains, and even though it looks like there might be some damage, it doesn't seem like it can be causing what we're seeing with the drywall damage in the ceiling.

SO: The good news is, we likely don't need to get the whole roof replaced.

BUT: The bad news is, we don't know what's going on, and now need to seek new causes.

It's really easy to catastrophize: we were worried that we'd find a nest of mold up there and need to replace the whole roof. That isn't the problem, so that bullet is dodged. But now we've got a deeper mystery: we have what looks like water damage in an area that is - apparently - dry, with no explanation. And that's the thing about science: one thing can look like another, and causes can be hidden - so you need to take out the time to collect the observations and do the experiments and carefully check your work.

Especially if it can cost you a whole roof.

-the Centaur

Pictured: One from the archives - snow, likely from two years ago, as that frost killed most of that vine.

[twenty twenty-four day one five three]: con carolinas day two

centaur 0

Back at Con Carolinas for day two (but once again images from the archives while my blog is getting updated in the background).

Today I was on a lively panel about the "Trials and Tribulations of AI" and if there's anything I could take away from that, it would be that "modern AIs do not check their work, so if you use them, you have to."

There's a whole debate on whether they're "really intelligent" and you probably can bet where I come down on that - or maybe you can't; here goes:

  • Yes, modern AI's are "artificial intelligence" - they literally are what that phrase was invented to describe.
  • No, modern AI's are not artificial general intelligence (AGI) - yet - and I can point you to a raft of papers describing either the limitations of these systems or what is needed for a full AGI.
  • Yes, they're doing things we would normally describe as intelligent, but ...
  • No, they're doing "thinking on a rocket sled", facing backward, tossing words on the track in a reverse of the Wallace and Gromit track-laying meme, unable to check or correct their own work.

These systems "hallucinate", just like humans are mistaken and make things up, but do so in ways alien to human thought, so if we use them in areas we can't check their work, we must do so with extreme caution.

And then there's the whole RAFT of ethics issues which I will get to another day.

Next up: "Neurodivergence and Writing" at 6:30pm, and "Is THAT even POSSIBLE" at 9:30pm!

Onward!

-the Centaur

Pictured: NOT Con Carolinas - I think this was Cafe Intermezzo.

Journaling: Today's Event: Con Carolinas. Today's Exercise, 30 pushups, planning a walk later today. Today's Drawing: finished one five three yesterday, will tackle one five four after I tackle my fix-my-roof thing.

[twenty twenty-four day one five two]: con carolinas first day after-action report

centaur 0

This is the day after day one five two, but, whatevz, I had to deal with a minor emergency yesterday, so you have to deal with a late post. Regardless, I was at Con Carolinas, at the "Hooked" panel, which went well, and if there's anything I could take away from that panel, it would be the following:

  • Your hook for your story isn't just your first line, but it encompasses everything from your genre, your prior output as a writer, the cover, the title, the subtitle or blurb, the first line, the first paragraph, the first page, the first chapter.
  • All of those can attract your reader's attention; to engage their interest, you need to raise a story question which needs answering. This can be the surface problem, the deeper story-worthy question, an exciting incident, the voice of the main character, an intriguing setup, or a mystery ... that makes people want more.
  • Most of the panelists felt that you should leave out of your hook any excess description or backstory that does not help build that story question. Those issues can be raised later, once the story is moving; only when your readers are desperate to have questions answered should you spend time answering them.

I'm sure I could say more, but I'm not, because I have a leak in a roof to deal with. BUT, since I am not going to be able to post new images for a bit, I'm going to change my strategy for my "Blogging Every Day" posts with a little Livejournal-style annotation! Lo:

Today's event? Con Carolinas, where I saw a lot of old friends and was on the "Hooked" panel. Today's exercise? Just thirty pushups and a relatively brief walk. Today's drawing? More Goldman studies: by my count, I am up to day one five three, which means I'm caught up (as this blogpost is one day behind).

That's it! Here's hoping I have enough bits left to post.

-the Centaur

Pictured: From the archives, the red editor's pen, over a redacted manuscript. Full disclosure: my normal editing pen is blue, as I am partially colorblind - while I can see red, it doesn't stand out for me the way blue does. There is no such thing for me as "fire engine red" unless I'm wearing Enchroma glasses (which do not give you true color vision, by the way, but they certainly can make certain colors stand out more). I was probably using the red pen in this case either because the blue one blew up, or I need two kinds of notes.

Con Carolinas, the Nebulas, and the Embodied AI Workshop!

centaur 0
The Centaur and His Books

Hey folks! I am appearing at several conventions in the next few weeks, so I'm creating a "sticky post" to let y'all know about my schedule - in case the problems I'm having with my blog software get worse, at least this will be up here to let y'all know I'll be talking about AI, robots and writing in the next few weeks!

First up is Con Carolinas, the Carolinas' longest running science fiction convention, where I will be on four panels and an author signing, talking about book openings, artificial intelligence, neurodivergence, and what's possible and what's not in science and science fiction!

Friday, May 31st:

Saturday, June 1:

Sunday, June 2:

Next up is the Nebula Conference, the Science Fiction and Fantasy Writers of America's conference, where again I'll be talking about artificial intelligence and signing books!

  • Thursday, June 6th: 2:30pm: Anthony Francis Meet-and-Greet Table E Pasadena
  • Saturday, June 8th: 10:30am: Why AI Needs Humans Pasadena

There will also be office hours but I think you have to sign up in advance for those, and we have a scheduling snafu regarding those anyway.

Finally is the Embodied AI Workshop on June 18th, hosted at CVPR, the Computer Vision and Pattern Recognition Conference, where I am the lead organizer, though you'll probably only hear me yapping for any extended length of time if you show up in the first ten minutes - I give the intro talk at 8:50am.

Hope to see y'all there!

-Anthony