Good Friday Vigil

Good Friday Vigil at Saint Stephen’s in-the-Field. We dress down the church and set up a bare wood cross and labyrinth, and encourage people to sign up to stay and pray so we have coverage all night.

I am a night owl, so I signed up for 1 a.m. through 2 a.m. So why am I here with a cough at 2:45 a.m. when I have an early-for-me meeting tomorrow? Someone changed my slot without telling me, to 2 a.m. through 4 a.m.

So I had the double pleasure of waiting fifteen minutes in the cold for the shift change (while I confirmed, via Google Docs history, that I was not misremembering my time), finding out that the person inside was still only partially through their two hour shift, going home to crash, and coming back to wait in the colder cold again while the previous person ran over. (The irony of the sleeping apostles is not lost on me).

This has been my least effective Lent in recent memory. I went to Ash Wednesday service to get ashes, only to get quizzed about it by my favorite server at one of my favorite restaurants, who then to my dismay turned into an insulting, manipulative proselytizer. I have had a surprising share of similar bad reactions with people leaving me more rattled about how I treat and react to people (even though I was never the aggressor) than focused on God or reading the Bible. Visiting the sick has not worked as my friend who is hurt the most is too touch and go for visitors. And giving up alcohol for Lent proved more of an inconvenience than a prompt for reflection.

And yet, like going to church on Sunday, or volunteering for the church Vestry, or reading the Bible, the Vigil is serving its function: to draw my attention back to God.

May God’s peace, which passes all understanding, be with you always.

-the Centaur

Learning to Drive … by Learning Where You Can Drive

I often say “I teach robots to learn,” but what does that mean, exactly? Well, now that one of the projects that I’ve worked on has been announced – and I mean, not just on arXiv, the public access scientific repository where all the hottest reinforcement learning papers are shared, but actually, accepted into the ICRA 2018 conference – I  can tell you all about it!

When I’m not roaming the corridors hammering infrastructure bugs, I’m trying to teach robots to roam those corridors – a problem we call robot navigation. Our team’s latest idea combines “traditional planning,” where the robot tries to navigate based on an explicit model of its surroundings, with “reinforcement learning,” where the robot learns from feedback on its performance.

For those not in the know, “traditional” robotic planners use structures like graphs to plan routes, much in the same way that a GPS uses a roadmap. One of the more popular methods for long-range planning are probabilistic roadmaps, which build a long-range graph by picking random points and attempting to connect them by a simpler “local planner” that knows how to navigate shorter distances. It’s a little like how you learn to drive in your neighborhood – starting from landmarks you know, you navigate to nearby points, gradually building up a map in your head of what connects to what.

But for that to work, you have to know how to drive, and that’s where the local planner comes in. Building a local planner is simple in theory – you can write one for a toy world in a few dozen lines of code – but difficult in practice, and making one that works on a real robot is quite the challenge. These software systems are called “navigation stacks” and can contain dozens of components – and in my experience they’re hard to get working and even when you do, they’re often brittle, requiring many engineer-months to transfer to new domains or even just to new buildings.

People are much more flexible, learning from their mistakes, and the science of making robots learn from their mistakes is reinforcement learning, in which an agent learns a policy for choosing actions by simply trying them, favoring actions that lead to success and suppressing ones that lead to failure. Our team built a deep reinforcement learning approach to local planning, using a state-of-the art algorithm called DDPG (Deep Deterministic Policy Gradients) pioneered by DeepMind to learn a navigation system that could successfully travel several meters in office-like environments.

But there’s a further wrinkle: the so-called “reality gap“. By necessity, the local planner used by a probablistic roadmap is simulated – attempting to connect points on a map. That simulated local planner isn’t identical to the real-world navigation stack running on the robot, so sometimes the robot thinks it can go somewhere on a map which it can’t navigate safely in the real world. This can have disastrous consequences – causing robots to tumble down stairs, or, worse, when people follow their GPSes too closely without looking where they’re going, causing cars to tumble off the end of a bridge.

Our approach, PRM-RL, directly combats the reality gap by combining probabilistic roadmaps with deep reinforcement learning. By necessity, reinforcement learning navigation systems are trained in simulation and tested in the real world. PRM-RL uses a deep reinforcement learning system as both the probabilistic roadmap’s local planner and the robot’s navigation system. Because links are added to the roadmap only if the reinforcement learning local controller can traverse them, the agent has a better chance of attempting to execute its plans in the real world.

In simulation, our agent could traverse hundreds of meters using the PRM-RL approach, doing much better than a “straight-line” local planner which was our default alternative. While I didn’t happen to have in my back pocket a hundred-meter-wide building instrumented with a mocap rig for our experiments, we were able to test a real robot on a smaller rig and showed that it worked well (no pictures, but you can see the map and the actual trajectories below; while the robot’s behavior wasn’t as good as we hoped, we debugged that to a networking issue that was adding a delay to commands sent to the robot, and not in our code itself; we’ll fix this in a subsequent round).

This work includes both our group working on office robot navigation – including Alexandra Faust, Oscar Ramirez, Marek Fiser, Kenneth Oslund, me, and James Davidson – and Alexandra’s collaborator Lydia Tapia, with whom she worked on the aerial navigation also reported in the paper.  Until the ICRA version comes out, you can find the preliminary version on arXiv:

https://arxiv.org/abs/1710.03937
PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-based Planning

We present PRM-RL, a hierarchical method for long-range navigation task completion that combines sampling-based path planning with reinforcement learning (RL) agents. The RL agents learn short-range, point-to-point navigation policies that capture robot dynamics and task constraints without knowledge of the large-scale topology, while the sampling-based planners provide an approximate map of the space of possible configurations of the robot from which collision-free trajectories feasible for the RL agents can be identified. The same RL agents are used to control the robot under the direction of the planning, enabling long-range navigation. We use the Probabilistic Roadmaps (PRMs) for the sampling-based planner. The RL agents are constructed using feature-based and deep neural net policies in continuous state and action spaces. We evaluate PRM-RL on two navigation tasks with non-trivial robot dynamics: end-to-end differential drive indoor navigation in office environments, and aerial cargo delivery in urban environments with load displacement constraints. These evaluations included both simulated environments and on-robot tests. Our results show improvement in navigation task completion over both RL agents on their own and traditional sampling-based planners. In the indoor navigation task, PRM-RL successfully completes up to 215 meters long trajectories under noisy sensor conditions, and the aerial cargo delivery completes flights over 1000 meters without violating the task constraints in an environment 63 million times larger than used in training.

 

So, when I say “I teach robots to learn” … that’s what I do.

-the Centaur

Random Sketchworks

Got inspired by all the art at Dragon Con, particularly Comfort and Adam’s Guide to Self Publishing Comics which reminded me of my interview questions about my long stalled comic f@nu fiku … Time to break out the sketchbook again …

-the Centaur

Dragon Con Schedule, 2017 Edition!

20160903_114711.jpg

Hail, fellow adventurers! I’ll be back at Dragon Con again this year, with a great set of panels! Sometimes that includes dropping in on the Writing Track, but the ones we have officially scheduled so far are:

  • Androids & Automatons
    Friday 1pm – Sheraton / Augusta
    This presentation will cover the history of artificial life from the ancient Greeks to modern automata artists. Techniques for creating your own automata will also be shared.
  • Practical Time Travel for the Storyteller
    Saturday 4pm – Sheraton / Athens
    This panel discusses the real science behind time travel, as well as how these scientific theories can place both challenging & rewarding demands on the stories we tell. Time dilation, the grandfather paradox, & others will be explained & these theories discussed.
  • The Magic and Technology of Building Alternate Worlds
    Saturday 5:30pm – Sheraton / Augusta
    Be it alchemical spells or industrial revolution, many alternate history and Steampunk worlds feature magic, fantastic technological marvels, or even both. But each choice shapes the worlds these authors build. This roundtable discuss worldbuilding and the balance between fantasy and science.
  • Author Reading: Anthony Francis
    Sunday 2:30pm – Hyatt / University
    SF & Urban Fantasy author of Frost Moon, Blood Rock and Liquid Fire reads from his work, including selections from Jeremiah Willstone and the Clockwork Time Machine, from future Dakota Frost novels, and maybe even some Cinnamon Frost stories!

Also, I was scheduled to do a SAM Talk, but it was inadvertently booked over my author reading, and I pretty much have to prioritize my own author reading over a SAM Talk even if there might be more people at the other room. So if you attend my author reading, you may also get to hear what was intended to be my SAM Talk, “Risk Getting Worse”.

Screenshot 2017-08-31 04.07.05.png

Hope to see you all there – from my end of the table, it kind of looks like this:

20160903_215541.jpg

Here’s crossing fingers that we get the double booking all worked out!

-the Centaur

The Climb up the Hill

IMG_20170704_154722.jpg

Taking on a challenge like writing a novel can seem daunting. A good novel can range from 60,000 words for a young adult novel or a romance up to 360,000 word for a fantasy novel, with a typical length closer to 90,000 to 120,000 words. For perspective, a paragraph in a five-paragraph essay can be 100 words, so a 100,000 word novel like my first novel, FROST MOON, is like a thousand-paragraph essay. To someone who had trouble getting those 500 words down, that’s incredibly daunting.

IMG_20170704_152921.jpg

Challenges like National Novel Writing Month can, paradoxically, make it easier. 50,000 words in a month seems daunting, but that’s only half a full-length novel, and even more so, it’s not 50,000 words of a finished novel: it’s 50,000 words of unpolished first draft. You can let yourself write drek you’re not proud of if it gets words on the page. If you’re the kind of person daunted by the thought of writing a whole novel, or paralyzed by perfectionism, National Novel Writing Month offers an easier path up the hill.

IMG_20170704_153310.jpg

Still, it’s a long hill. And it can be daunting, no doubt. Especially if you tend to get behind, like I do, or if you tend to get trapped polishing your words, as I often do. You sometimes need tips and techniques to help yourself get past the stumbling blocks.

IMG_20170704_153247.jpg

Here are a few of the ones that have worked for me in the past.

  • Commit to Writing Every Day. This is really hard to do, so you need to make sure you do it. You’ll fail, of course, but if you’re constantly writing every day, then missing a day won’t hurt you. If you’re not writing every day, however, soon, you’ll be a week behind, with no way to come back.
  • Track Your Progress. If you don’t track your progress, you can find yourself far off the path. I use a spreadsheet which tells me how far ahead (or, more often, behind) I am and how many words I need to write each day to stay on track for the end of the month. Other people use the National Novel Writing Month site to track this.
  • Put Writing First. Turn off your Internet until you’ve written. Put off that computer game. Defer going to see that movie until the end of the month. For me, since I read and write over lunch, dinner, and coffee, I have to shut off my internet and defer reading the next scientific paper in my stack until I’ve gotten my word count for the day (with the exception of when my food actually is hot on the plate; I’ll read then as it’s hard to write and eat at the same time).
  • Don’t Edit – Just Write. Editing can come later. It’s not adding words to your document. Get your first draft down, then edit it, or you won’t get your 50,000 words in the month, and if you can’t get through 50,000 words without getting bogged down in edits, you’ll never get your 100,000 word novel done.
  • Don’t Delete – Use Strikethrough. Don’t cut words during Nano, that’s just shooting yourself in the foot. The point is to get through your whole story. If you don’t like something you wrote, strike it out, put it in italics, whatever, just mark it for future revision and write what you want instead as the next paragraph. Trust me, your inner editor can turn on your new idea just as easily as it can turn on your old – so get them both down and move on. I use a special style in Word called “Summary” for this purpose – italics surrounded by dotted lines.
  • Don’t Research – Use Angle Brackets. My writing group uses <angle brackets> to indicate something that must be filled in later. Other people use the copyediting term TK – to come (sic). The point is, use something unique and searchable. Turn off your Internet, resist the desire to chase links in Wikipedia or TV Tropes, and write <TK: for that thing you wanted to fill in here but couldn’t think of at the time> so you can come back to it later.
  • ALTERNATELY. In addition to angle brackets, I use the word “ALTERNATELY” when I realize a scene’s gone the wrong way. Rather than rewriting or creating new connective material, I say ALTERNATELY, or perhaps “ALTERNATELY: Dakota realizes something is wrong” and then pick up where I left off. On the next pass, this is easily fixable.
  • Remember, at some point, you may hate it. Sooner or later, every writer falls out of love with their manuscript. That’s OK. “If you’re going through hell, keep going.” The point is not to cater to your emotions, but to get through your emotions to the end of your project. Let the hate flow through you! And move on to the next bit. Sooner or later, every writer falls in love with their manuscript – and the sooner you write more words, the sooner you’ll get there.

Your mileage may vary, of course, but these tips helped me.

IMG_20170704_153428.jpg

Writing 50,000 words of rough draft is not writing a novel. You’ve got a lot more to go – between 10,000 and 310,000 words depending on whether you’re aiming at Goosebumps or George R. R. Martin. But if you can get 50,000 words under your belt, you’ll have the pleasure of looking back and realizing you can accomplish quite a climb.

IMG_20170704_153501.jpg

20 Goes at Nano

Camp Nano 2017-07-29-a.png

Above you see a big pile of all the words I’ve written in National Novel Writing Month and related challenges, laid out horizontally by day of month and laid down vertically by the challenge in which I wrote them, creating an interesting strata effect, like words deposited by a geological process. This month marks my 20th attempt at Nano, 18 of which were successful:

Deliverance 2002 Nanowrimo WINNER
Frost Moon 2007 Nanowrimo WINNER
Blood Rock 2008 Nanowrimo WINNER
Liquid Fire 2009 Nanowrimo WINNER
Clockwork 2010 Nanowrimo WINNER
Clockwork 2010 December Nano FAILED
Hex Code 2011 Nanowrimo WINNER
Clockwork 2012 Script Frenzy WINNER
Spectral Iron 2012 Nanowrimo WINNER
Marooned 2013 Nanowrimo WINNER
Spectral Iron 2014 Camp Nanowrimo WINNER
Spectral Iron 2014 August Nano FAILED
Phantom Silver 2014 Nanowrimo WINNER
Spectral Iron 2015 Camp Nanowrimo WINNER
Hex Code 2015 Nanowrimo WINNER
Phantom Silver 2016 Camp Nanowrimo WINNER
Phantom Silver 2016 Camp Nanowrimo WINNER
Spiritual Gold 2016 Nanowrimo WINNER
Spiritual Gold 2017 Camp Nanowrimo WINNER
Spiritual Gold 2017 Camp Nanowrimo WINNER

As I’ve noted before, the two in which I failed were “off months” where I tried to tackle Nanowrimo on my own. For me, it’s much harder without the external benefit of the contest, and on the two times I tried it I bombed out after a few days. You can see that in this graph, which shows the number of words I’m ahead or behind at each part of the month:

Camp Nano 2017-07-29-c.png

This graph means the most to me, because I was involved in the creation of it, and so intuitively understand it; if I see my monthly progress (the darkest line above) below the dotted line of the average, I know to worry; if I see it below my worst track for any part of the month, I know to really get cracking. Looks like the farthest behind I ever got (and succeeded) was 20,00 words behind, on LIQUID FIRE in 2009, and in PHANTOM SILVER in 2016.

But for people not intimately involved in laying down those tracks, the average amount ahead / behind per day is perhaps more useful:

Camp Nano 2017-07-29-d.png

This shows that a successful Nanowrimo participant can be very far ahead, or very far behind, and still win in the month. Do what works for you! There’s a lot of wiggle room in there.

But if you’re more interested in brass tacks, here’s the maximum and average amount I wrote in each day:

Camp Nano 2017-07-29-b.png

This shows that typically at the start of Nano I’m writing a little bit less than the needed word count per day, and at the end of Nano I’m writing a little bit more – but that the maximum I have to do each day is radically more than that – once almost 10,000 words (and that was a hell of a push, I can tell you – that was PHANTOM SILVER in July of 2016, and I was down to the wire, writing 7000 words in the last day – and finding the Camp Nano counter was 2000 words off of Microsoft Word’s count, so I had to generate 2,000 more words in the last couple of hours).

I will probably dig a bit more into SPIRITUAL over the last two days of the 30 day challenge (I know July has an extra day, but I can use the break). I’m not quite done – the manuscript is at 171,330 words, but maybe 20,000 to 30,000 words of that are in-manuscript notes that need to be turned into text, and then I have a lot I want to cut. During Nano, if I change my mind about how a scene is going, I don’t cut it and rewrite it, because that defeats the purpose of generating words; I write the word ALTERNATELY on its own line and rewrite the scene. After Nano, all that needs to get edited, merged and/or cut.

Often, I find that I’m not satisfied with the first rough draft text I produce in Nano. There are amazing gems in there, but also drek. But at the same time, I find that I am almost always very satisfied at having a text that flows through all the scenes I wanted to write. The idea of a scene in your head is just that – an idea. It’s not real until you write it. If you don’t write it, you can’t improve it – you’ll either long for it to be written, or you’ll elaborate on your idea of it in your head endlessly, or, worst of all, get caught up in the smug satisfaction of your own unfinished work, admiring the creation of something awesome that doesn’t actually exist.

But once you write it, you can see whether the idea works or not. You can decide to keep it, or refine it, or discard it. Even better, it springboards you – into new alternates for the same scene, or new ideas for what happens next, or new insights into your character, their plot, and the themes of your story.

Don’t just dream your story – write it down. Only by writing dreams down can you turn them into reality.

And Nanowrimo is a great place to get started with that. The 50,000 word challenge may seem impossible. It may not even seem like the kind of thing you want to do. No one is making you, after all: you don’t have to. But if your head is filling with ideas and you can’t get them out, why not take on an impossible seeming challenge to write 50,000 words of them down.

Believe me, it’s possible.

Camp Nano 2017-07-29-e.png

-the Centaur

Viiictory^18

Camp-2017-Winner-Twitter-Header.jpg

So, for the eighteenth time, I have won a National Novel Writing Month challenge … this time, the 50,000 word challenge for Camp Nano of July 2017!

Camp Nano 2017-07-28-a.png

The cafe I’m in is about to close, but I’m proud to say I (a) finished the 50,000 words a few days early so I can relax this weekend and (b) solved some problems in my manuscript, making it easier for me to reach that final finish line for Dakota Frost #6, SPIRITUAL GOLD!

Camp Nano 2017-07-28-b.png

More tomorrow when I have more time to reflect on getting this much closer to the end …

Onward!

-the Centaur

Mining that SPIRITUAL GOLD

Well, we’re getting ahead of the curve at last on SPIRITUAL GOLD … two days ahead.

Camp Nano 2017-07-24-a.png

My writing retreat this weekend has paid off. I spent some time hanging out with the Treehouse Writers at the Linde Lane Tea Room in Dixon, California, then holed up in a hotel in downtown Davis, hanging out in bookstores and coffeehouses in an attempt to make some progress on SPIRITUAL GOLD. The actual day of the drive was a wash, but after that, I managed to get more than two days worth of words done in each day, and almost that today.

Camp Nano 2017-07-24-b.png

Now at last I’m ahead of the curve, and if I can stay there for a few more days, I’ll win Camp Nano. More importantly, however, I’ve marched forward in the manuscript so I’m around Chapter 37 out of roughly 50, with much of the text of the remainder partially written and merely needing some ironing out. With luck, I’ll finish SPIRITUAL GOLD at the end of the month, and shortly thereafter, and then can begin editing Dakota Frosts #4-#6 together as one big trilogy.

Onward!

-the Centaur