Press "Enter" to skip to content

Posts tagged as “Teaching Robots to Learn”

Announcing the 5th Annual Embodied AI Workshop

centaur 0

Thank goodness! At last, I'm happy to announce the Fifth Annual Embodied AI Workshop, held this year in Seattle as part of CVPR 2024! This workshop brings together vision researchers and roboticists to explore how having a body affects the problems you need to solve with your mind.

This year's workshop theme is "Open-World Embodied AI" - embodied AI when you cannot fully specify the tasks or their targets at the start of your problem. We have three subthemes:

  • Embodied Mobile Manipulation: Going beyond our traditional manipulation and navigation challenges, this topic focuses on moving objects through space at the same time as moving yourself.
  • Generative AI for Embodied AI: Building datasets for embodied AI is challenging, but we've made a lot of progress using "synthetic" data to expand these datasets.
  • Language Model Planning: Lastly but not leastly, a topic near and dear to my heart: using large language models as a core technology for planning with robotic systems.

The workshop will have six speakers and presentations from six challenges, and perhaps a sponsor or two. Please come join us at CVPR, though we also plan to support hybrid attendance.

Presumably, the workshop location will look something like the above, so we hope to see you there!

-the Centaur

Pictured: the banner for EAI#5, partially done with generative AI guided by my colleague Claudia Perez D'Arpino and Photoshoppery done by me. Also, last year's workshop entrance.

Adios, 2023

centaur 0

Wow, what a year. I'd love to say it had its pros and cons, but the stark reality of it is that the one bad thing - getting laid off, not just years before I wanted to retire, but one day after we successfully showed our new project was working, thus throwing me years off course in my research - overshadows all the good stuff. As I was describing it to my wife, it's like falling down into a well and finding some shiny rocks down there. They might be nice rocks - heck, they might even be gold, and worth a fortune - but you've still got to cope with falling into the well, and figure out how to climb back out again, before taking advantage of the good stuff.

And, I have to admit, there was a lot of good stuff. We submitted a great paper on social robot navigation and held a great workshop on embodied AI, which was renewed for next year. Thinking Ink Press published a new book, was chosen for the Innovative Voices program and launched a successful Kickstarter. We proposed the Neurodiversiverse anthology, announced it at Dragon Con, and have almost a hundred submissions as of the close of the year. That inspired me to write two new stories, "Blessing of the Prism" and "Shadows of Titanium Rain", which I think are some of my best work. I even started a robotics consulting business and got a few clients, which is helping to reduce the uncertainty.

But 2023 was the worst year for me for a while. There have been bad ones recently - in 2016 we elected a wannabe dictator and many of my friends and family seemed to lose their minds; in 2019 my mother died; and in 2020 I had the double whammy of the pandemic with the most stressful period of my work life. But, like 2023, each of those years had ups with the downs: in 2016, my current research thread started; in 2019, we proved that our research ideas were working (for all the good it did us); and in 2020, we moved back to my hometown into what we hope is our forever home.

And yet, with the exception of the loss of my mother, none of those seemed quite as life changing as getting laid off. Even for Mom, I was somewhat prepared: my father had unexpectedly lost one of his siblings early, and our extended family had developed a kind of shared knowledge of how to cope with loss. I had already lost my father and grandmother, and knew that Mom, while healthy, was in her mid-80s, and could pass at any time; so I was spending as much time as practical with her. I spoke to her the day she died. And so, after she was gone, I started down a road that I had been preparing for mentally for a long, long time.

But I wasn't in the mindset that Google would kill off half its robotics program just in AI's hour of triumph. We were even working on a projects directly related to Google's new large language model focus. It made no sense, and left this strange kind of void, creating a severance I didn't expect for another decade.

Despite all of what happened this year, I keep coming back to one thing:

Was it worth it if I wrote those two new stories?


So, farewell, you crazy year you: thanks for all you gave me. My wife even said "Supposedly what you do on New Year's Eve is what you'll do for the rest of the year," and today we worked on our businesses, worked on writing and art, met friends old and new, and even moved furniture (which, metaphorically, is her new business venture). So's here's to more writing, more art, more friends, and more business in 2024!

-the Centaur

P.S. I see that I kept up "Blogging Every Day" in 2023 for 91 days, almost a quarter of the year; my earlier attempt at "Drawing Every Day" in 2021 lasted 103 days, a little over a quarter of the year. Let's see if we can break both those records in 2024, now that I have far more free time (and flexible time) on my hands!


centaur 0

Still at the Conference on Robot Learning. LOTS of robot dogs were about, lots of diffusion model and transformer work, and lots of language model planning. More later, gotta crash.

-the Centaur

[thirty-nine] minus twenty-one: what a team effort

centaur 0

Wow. We're done with the paper. And what a team effort! So many people came together on this one - research, infra, operations, human-robot interaction folks, the whole nine yards. It's amazing to me how interdisciplinary robotics is becoming. A few years ago 7 authors on a paper was unusual. But out of the last 5 papers I helped submit, the two shortest papers had 8 authors, and all the others were 15 or more.

And it's not citation inflation. True, this most recent paper had a smaller set of authors actively working on the draft, collating contributions from a larger group running the experiments ... but the previous paper had more than 25 authors, all of whom materially contributed content directly to the draft.

What a wonderful time to be alive.

And to recover from food poisoning.

-the Centaur

Pictured: this afternoon's draft of the paper, just prior to a video conference to hammer out some details.

The Embodied AI Workshop at CVPR 2021

centaur 0
embodied AI workshop

Hail, fellow adventurers: to prove I do something more than just draw and write, I'd like to send out a reminder of the Second Embodied AI Workshop at the CVPR 2021 computer vision conference. In the last ten years, artificial intelligence has made great advances in recognizing objects, understanding the basics of speech and language, and recommending things to people. But interacting with the real world presents harder problems: noisy sensors, unreliable actuators, incomplete models of our robots, building good simulators, learning over sequences of decisions, transferring what we've learned in simulation to real robots, or learning on the robots themselves.

interactive vs social navigation

The Embodied AI Workshop brings together many researchers and organizations interested in these problems, and also hosts nine challenges which test point, object, interactive and social navigation, as well as object manipulation, vision, language, auditory perception, mapping, and more. These challenges enable researchers to test their approaches on standardized benchmarks, so the community can more easily compare what we're doing. I'm most involved as an advisor to the Stanford / Google iGibson Interactive / Social Navigation Challenge, which forces robots to maneuver around people and clutter to solve navigation problems. You can read more about the iGibson Challenge at their website or on the Google AI Blog.

the iGibson social navigation environment

Most importantly, the Embodied AI Workshop has a call for papers, with a deadline of TODAY.

Call for Papers

We invite high-quality 2-page extended abstracts in relevant areas, such as:

  •  Simulation Environments
  •  Visual Navigation
  •  Rearrangement
  •  Embodied Question Answering
  •  Simulation-to-Real Transfer
  •  Embodied Vision & Language

Accepted papers will be presented as posters. These papers will be made publicly available in a non-archival format, allowing future submission to archival journals or conferences.

The submission deadline is May 14th (Anywhere on Earth). Papers should be no longer than 2 pages (excluding references) and styled in the CVPR format. Paper submissions are now open.

I assume anyone submitting to this already has their paper well underway, but this is your reminder to git'r done.

-the Centaur