Press "Enter" to skip to content

Posts tagged as “Development”

Rescuing Google Drive?

centaur 0

IMG_20121029_235958.jpg

Ok, the above is a rescue cat, but the point remains. In an earlier post I understandably got a bit miffed when moving a folder within Google Drive - an operation I've done before, many times - mysteriously deleted over a hundred and fifty files. I was able to rescue them, but I felt like I couldn't trust Google Drive - a feeling confirmed when the very next time I used it to collect some quick notes, the application crashed.

But I love the workflow of Google Drive - the home page of Google Drive can show you, very very quickly, either your hierarchy of folders, your recently accessed files, or a search of all your files, and once you've found a file it appears far quicker than most normal applications like Microsoft Word, Microsoft Excel, or Photoshop. Word, Excel and Photoshop kick Google Drive's ass on specialized uses, but many documents don't need that, and Google Drive is a great alternative.

But what about files disappearing? A non starter. However, there are ways around that problem.

Google Drive of course has the ability to export files. You can even export an entire directory in this fashion. If you really want to get serious, you can use Google Takeout, a data migration tool by Google which enables you to export all your Google Drive data, part of Google's Data Liberation Front.

But all those rely on one time manual operations. I want something that works automatically, so for my money it's the Google Drive API that really comes to the rescue. That enables developers to create applications like cloudHQ, which syncs between Google Drive, Dropbox and several other services. I've tried out cloudHQ experimentally and it works on a single folder.

Next I'm going to try it on a larger scale, though it will require a little re-sorting of how I've got Dropbox and Google Drive working. Most likely, I'm going to need to either uninstall Google Drive from my primary computer and sync all its files into Dropbox by CloudHQ, or else manually unsyc certain folders so I don't get double-storage on this machine.

Regardless, there is a silver lining. Now let's see if it's also a silver bullet.

-the Centaur

Pictured: Me holding Loki, our outdoor rescue cat. He's large marge, let me tell you.

The End for Google Drive

centaur 0

Screen Shot 2013-03-03 at 1.13.54 PM.png

Recently I was doing some task and needed to track down some information. I couldn't find the document I wanted at first in my Google Drive, but once I did, I realized I had several documents, all on the same topic, so I did with Google Drive the same thing I'd done before on Google Drive: I went to the Google Drive folder and reorganized the files.

Big mistake.

Quickly red "x's" started appearing in my folders. More and more "unsyncable" files started showing up in the Google Drive status list. And then a status message popped up: "The files you have deleted are now in Google Drive's Trash."

Uh-oh.

Understand: I had deleted no files or folders. I simply moved them around - and I've done this before. A lot. On Google Drive, not just Dropbox. But something apparently happened in the sync, and Google Drive thought I'd deleted the folders.

So it trashed all those files.

Understand, Google Drive "documents" on your hard drive aren't "documents"; they're little text files with pointers to a location in Google Drive, like this (where UNREADABLE_IDENTIFIER is a string of alphanumeric gobbledegook):

{"url": "https://docs.google.com/document/d/UNREADABLE_IDENTIFIER/edit", "resource_id": "document:UNREADABLE_IDENTIFIER"}

This pathetic little bit of nonsense is all I would have had left of a 200 word start to an essay - if I hadn't acted quickly. I started to look online, and found this alarming bit of information:

https://support.google.com/drive/bin/answer.py?hl=en&answer=2375102

Declutter your Google Drive by removing unwanted and outdated files, folders, and Google Docs from your Google Drive. Anything that you own and remove from Google Drive will be in the trash until you permanently delete or restore them.

Moving Google Docs files out of your Google Drive folder will cause their counterpart files on the web to be moved to the trash. If you then purge the trash, those files will become permanently inaccessible. Because the Docs files in your Drive folder are essentially links to files that exist online, moving these files back into your Drive folder after purging the trash online will not restore the files, as their online counterparts will have been deleted.

OMG! The contents of my documents may be lost forever if I purge the trash. But it gets worse...

http://support.google.com/drive/bin/answer.py?hl=en&answer=2494934

If something in Google Drive is moved to the trash, you'll see a warning and you may lose access to it at any time. Read one of the following sections to learn how to restore it to your Google Drive from the trash. When you restore something, it'll be recovered in Google Drive on the web, to the Google Drive folder on your computer, and to your mobile devices.

If the item is in a folder, you’ll need to restore the entire folder to recover any individual items inside of it.

So I quickly returned to Google Drive. Everything you see above with a little red X was gone, all those files and 150 more. I hunted down the Trash (which was harder than you might think, as there was some persistent search in my Google Drive window that was removing the Trash folder from my view) and restored EVERYTHING that I had never deleted in the first place.

Now, this shouldn't have been a surprise. I always knew this could happen, ever since I gladly installed Google Drive on on my Mac in the hope that it would data liberate the Google Documents I had, only to find in my horror that Google Drive wasn't a syncing system, like Dropbox, but a cloud system, which is useless.

In case anyone misses the point: If you use Google Drive to store documents and also have the Google Drive client stored on a machine, Google Drive can get tricked into thinking you've deleted files, at which point it will move them to the Trash, at which point, unlike things you've deliberately trashed, it can delete them at any time - and you'll never get them back.

After some thought, I'm calling a hard stop on all use of Google Documents, except those I'm using to collaborate with others, where the collaboration features of the Google Doc outweigh the potential of risk. I can always save those files to a hard backup of a Word document or an Excel spreadsheet.

But I work for a living as a writer. And I can't work with a system that can arbitrarily trash hundreds of files and thousands upon thousands of words of documents with no hope of recovery just because I moved a folder … correctly.

Like Ecto, I have to rethink my use of these online tools - rethink them in a way that ensures that for every significant thing that I use in some convenient online system, I have a saved copy in an archivable backup.

More updates as I develop a new system.

-the Centaur

Approaching 33, Seen from 44

centaur 0

33-to-44.png

I operate with a long range planning horizon – I have lists of what I want to do in a day, a week, a month, a year, five years, and even my life. Not all my goals are fulfilled, of course, but I believe in the philosophy “People overestimate what they can do in a year, but underestimate what they can do in a decade.”

Recently, I’ve had that proven to me.

I’m an enormous packrat, and keep a huge variety of old papers and materials. Some people deal with clutter by adopting the philosophy “if you haven’t touched it in six months, throw it away.” Clearly, these people don’t write for a living.

So, in an old notebook, uncovered on one of my periodic archaeological expeditions in my library, I found an essay – a diary entry, really – written just before my 33rd birthday, entitled “Approaching 33” – and I find its perspective fascinating, especially when you compare what I was worried about then with where I am now.

“Approaching 33” was written on the fifth of November, 2011. That’s about five years after I split with my ex-fiancee, but a year before I met my future wife. It’s about a year after I finished my nearly decade-long slog to get my PhD, but ten years before when I got a job that truly used my degree. It’s about seven months after I reluctantly quit the dot-com I helped found to care for my dying father, but only about six months after my Dad actually died. And it’s about 2 months after 9/11, and about a month after disagreements over 9/11 caused huge rifts among my friends.

In that context, this is what I wrote on the fifth of November, 2011:

Approaching 33, your life seems seriously off-track. Your chances of following up on the PhD program are minimal – you will not get a good faculty job. And you are starting too late to tackle software development; you are behind the curve. Nor are you on track for being a writer.

The PhD program was a complete mistake. You wasted ten years of your life on a PhD and on your ex-fiancee. What a loser.

Now you approach middle fucking age – 38 – and are not on the career track, are not on the runway. You are stalled, lacking the crucial management, leadership and discipline skills you need to truly succeed.

Waste not time with useful affirmations – first understand the problem, set goals, fix things and move on. It is possible, only if you face clearly the challenges which are ahead of you.

You need to pick and embrace a career and a secondary vocation – your main path and your entertainment – in order to advance at either.

Without focus, you will not achieve. Or perhaps you are FULL OF SHIT.

Think Nixon. He had major successes before 33, but major defeats and did not run for office until your age. You can take the positive elements of his example – learn how to manage now, learn discipline now, learn leadership now, by whatever means are morally acceptable.

Then get a move on your career – it is possible. Do what you gotta do and move on with your life!

It appears I was bitter.

Apparently I couldn’t emotionally imagine I could succeed, but recognized, intellectually, that if I focused on what was wrong, and worked at it, then maybe, just maybe, I could fix it. And in the eleven years that have past … I mostly have.

Eleven years ago, I was enormously bitter, and regretted getting my PhD. It took five years, but that PhD and my work at my search-engine dot-com helped land me a great job, and after five more years of work I ended up at a job within that job that used every facet of my degree, from artificial intelligence to information retrieval to robotics to even computer graphics. My career took a serious left turn, but I never gave up trying, and eventually, I succeeded as a direct result of trying.

Eleven years ago, I felt enormously alone, having wasted a lot of time on a one-sided relationship that should have ended naturally after its first year, and having wasted many years after that either alone or hanging on to other relationships that were doomed not to work. But I never stopped looking, and hoping, and it took another couple of years before I found my best friend, and later married her.

Eleven years ago, I felt enormously unsure of my abilities as a software developer. At the dot-com I willingly stepped back from a software lead role when I was asked to deliver on an impossible schedule, a decision that was proved right almost immediately, and later took a quarter’s leave to finish my PhD, a decision that took ten years to prove itself. But even though both of those decisions were right, they started a downward spiral of self-confidence, as we sought out and brought in faster, more experienced developers to take over when I stepped back. While my predictions about the schedule were right, my colleagues nevertheless got more done, more quickly, ultimately culling out almost all of the code I wrote for the company. After a while, I felt I was contributing no more and, at the same time, needed to care for my dying father, so I left. But my father died shortly thereafter, six months before we expected. I found myself unable not to work, thinking it irresponsible even though I had savings, so I found a job at a software company whose technical lead was an old friend that who had been the fastest programmer I’d ever worked with in college, and now who had a decade of experience programming in industry – which is far more rigorous than programming in academia. On top of that, I was still recuperating from an RSI scare I’d had four years earlier, when I’d barely been able to write for six months, much less type. So I wrote those bitter words above when I was quite uncertain about whether I’d be able to cut it as a software developer.

Eleven years later — well, I still wish I could code faster. I’m surrounded by both younger and older programmers who are faster and snappier than I am, and I frequently feel like the dumbest person in the room. But I’ve worked hard to improve, and on top of that, slowly, I’ve come to recognize that I have indeed learned a few things – usually, the hard way, when I let someone talk me out of what I’m sure I know, and am later proved right – and have indeed picked up a few skills – synthetic and organizational skills, subtle and hard to measure, which aren’t needed for a small chunk of code but which are vital as projects grow larger in size and design docs and GANTT charts are needed to keep everything on track. I’d still love to code faster, to get up to speed faster, to be able to juggle more projects at once. But I’m learning, and I’ve launched things as a result of what I’ve learned.

But the most important thing is that I’ve been writing. A year after I wrote that note, I gave National Novel Writing Month a try for the first time. I spent years trying to perfect my craft after that, ultimately finding a writing group focused just on writing and not on critique. Five years later, I gave National Novel Writing Month another try, and wrote FROST MOON, which went on to both win some minor awards and to peak high on a few minor bestseller lists. Five years after that, I’ve finished four novels, have starts to four more, and am still writing.

I have picked my vocation and avocation – I’m a computer programmer, and a writer. I actually think of it as having two jobs, a day job and a night job. At one point I thought I was going to transition to writing full time, and I still plan to, but then my job at work became tremendously exciting. Ten years from now, I hope to be a full time writer (and I already have my next “second job” picked out) but I’m in no rush to leave my current position; I’m going to see where it takes me. I learned that long ago when I had a chance to knuckle down and finish my PhD, or join an unrelated but exciting side project to build a robot pet. The choice to work on the emotion model for that pet indirectly landed me a job at two different search engines, even though it was the skills I learned in my PhD that I was ultimately hired for. The choice to keep working on that emotion model directly led to my current dream job, which is one of the few jobs in the world that required the combined skills of my PhD and side project. Now I’m going to do the same thing: follow the excitement.

Who knows where it will lead? Maybe it will help me develop the leadership skills that I complained about in “Approaching 33.” Maybe it will help me re-awaken my research interests and lead to that faculty job I wanted in “Approaching 33.” Maybe it will just help me build a nest egg so when I finally switch to writing full time, I can pursue it with gusto. Or maybe, just maybe, it’s helping me learn things I can’t even yet imagine how I’ll be using … when I turn 55.

After I sign off this blogpost, I’m going to write “Passing 44.” Most of that’s going to be private, but I can anticipate it. I’ll complain about problems I want to fix with my writing – I want it to be more clear, more compelling, more accessible. I’ll complain about problems I want to fix at work – I want to work faster, to ramp up more quickly, and to juggle more projects well while learning when to say no. And I’ll complain about martial arts and athletics – I want to ramp up working out, to return to running, and to resume my quest for a black belt. And there are more things I want to achieve – wanting to be a better husband, friend, pet owner, person – a lot of which I’m going to keep private until I write “Passing 44, seen from 55.”

I’m going to set bigger goals for the next ten years. Some of them might not come to pass, of course. I bet a year from now, I’ll have only seen the barest movement along some of those axes. But ten years from now … the sky’s the limit.

-the Centaur

Pictured: Me at 33 on the left, me at 44 on the right, over a backdrop shot at my home at 44, including a piece of art by my wife entitled "Petrified Coral".

Ecto, Strike Two

centaur 0

Ecto just ate a HUGE post. Second time this has happened.

Time for a new blogging client?

-the Centaur

A Really Good Question

centaur 0

layout.png

Recently I was driving to work and thinking about an essay by a statistician on “dropping the stick.” The metaphor was about a game of pick-up hockey, where an inattentive player would be asked to “drop the stick” and skate for a while until they got their head in the game. In the statistical context, this became the action of stopping people who were asking for help with a specific statistical task and asking what problem they wanted to solve, because often solving the actual problem may be actually very different from fixing their technical issue and may require completely different approaches. That gets annoying sometimes when you ask a question to a mailing list and someone asks you what you're trying to solve rather than addressing the issue you've raised, but it's a good reflex to have: first ask, "What's the problem?"

Then I realized something even more important about projects that succeeded or failed in my life – successes at radical off the wall projects like the emotional robot pet project or the cell phone robots with personalities project or the 3d object visualization project, and failures at seemingly simpler problems like a tweak to a planner at Carnegie Mellon or a test domain for my thesis project or the failed search improvement I worked on during my third year at the Search Engine that Starts with a G. One of the things I noticed about the successes is that before I got started I did a hard core intensive research effort to understand the problem space before I tackled the problem proper, then I chose a method of approach, and then I planned out a solution. Paraphrasing Eisenhower, even though the plan often had to change once we started execution, the planning was indispensable. The day-to-day immersion in the problem that you need for planning provides the mental context you need to make the right decisions as the situation inevitably changes.

In failed projects, I found one or more things – the hard core research or the planning – wasn’t present, but that wasn’t all that was missing. In the failure cases, I often didn’t know what a solution would look like. I recently saw this from the outside when I conducted a job interview, and found that the interviewee clearly didn't understand what would constitute an answer to my question. He had knowledge, and he was trying, but his suggested moves were only analogically correct - they sounded like elements of a solution, but didn't connect to the actual features of the problem. Thinking back, a case that leapt to mind from my own experience was a project all the way back in grade school, where I we had an urban planning exercise to create an ideal city. My job was to create the map of the city, and I took the problem very literally, starting with a topographical map of the city's center, river and hills. Now, it's true that the geography of a city is important - for an ideal city, you'd want a source of water, easy transport, a relatively flat area for many buildings, and at least one high point for scenic vistas. But there was one big problem with my city plan: there were no buildings, neighborhoods, or districts on it! No buildings or people! It was just the land!

Ok, so I was in grade school, and this was one of my first projects, so perhaps I could be excused for not knowing what I was doing. But the educators who set up this project knew what they were doing, and they brought on board an actual city planner to talk to us about our project. When he saw my maps, he pointed out this wasn't a city plan and sat down with all of us to brainstorm what we'd actually want in a city - neighborhoods, power plants, a city center, museums, libraries, hospitals, food distribution and industrial regions. At the time, I was saddened that my hard work was abandoned, and now in hindsight I'm saddened that the city planner didn't take a minute or two to talk about how geography affects cities before beginning his brainstorming exercise. But what struck me most about this in hindsight is that I really didn't know what constituted an answer to the problem.

suddenclarity.png  

So, I asked myself, “What counts as a solution to this problem?” – and that, I realized, is a very good question.

-the Centaur

Pictured: an overhead shot of a diorama of the control room of the ENIAC computer as seen at the Computer History Museum, and of course our friend Clarence having his sudden moment of clarity.

Viiiictory Seven Times

centaur 0

spectralironwin.png

For the seventh time, I've won the National Novel Writing Month "contest", completing 50,000 words of a new novel in just 30 days. Actually, it took me just 29 days. Woohoo!

Winner-180x180.jpg

This year's entry, SPECTRAL IRON, is the fourth book in the Dakota Frost series, my urban fantasy series featuring the best magical tattooist in the Southeast (and she's not afraid to tell you that herself). SPECTRAL IRON was a bit of a detour from the work I was doing to edit LIQUID FIRE, the third entry in the series, but I'm glad I did: SPECTRAL IRON taught me a lot about what makes a book coherent and I can use that to edit LIQUID FIRE.

So what is SPECTRAL IRON about? Originally, I was thinking the story was about a villain that murders ghosts, but now it's looking like the villain is a ghost who's a murderer. Maybe. There are some very interesting plot complications developing. Let me see if I can pull out an excerpt that doesn't give much away. Well, maybe it spoils a minor surprise, but it doesn't give away the plot. This is the kind of thing they'd put in a movie trailer. Regardless ... SPOILERS:

Now, all that was left was to walk down a hundred more yards of train tracks in the dark.

The dolly had left us, but the spotlight had not. The mobile klieg operator wheeled it forward, slowly, tracking me, Ron and Sunny as we walked down the pathetic, waterlogged track. The further we went, the more layers of mystery were stripped off, one by one, by the light.

By the end, we no longer stood in a chasm of night. We merely stood in a dilapidated warehouse loading bay, a long, low brick-walled chamber, weathered with graffiti, with chained-up wooden doors atop its loading dock and beer bottles in the puddles between its train tracks.

“Nothing here,” the Lady Nyissa said. “Nothing obvious, at any rate.”

I stopped before the back wall of the loading dock. It stretched up before us, a mottled wall of brick thirty feet wide and fifty feet high, with a notch cut out of its bottom right by the platform and another cut out the top by a door. Rusted zig-zag metal stairs led up to it.

“Well,” I said, putting my foot on the train-brake at the end of the tracks, staring down at the pathetic mud puddle rippling before us between the end of the tracks and the wall. “It looks like The Exposers have found another Al Capone’s vault.”

Oh, me and my dumb mouth.

From the water erupted a foul spray of black—topped by a bone white mask.

So, there's a few thousand more words of brain dump to go, and then it's back to editing LIQUID FIRE, revising THE CLOCKWORK TIME MACHINE, and working on the DOORWAYS TO EXTRA TIME anthology, oh, and revising my own story for the anthology, "The Doorway to Extra Time" ... aaaa! But at least I have this year's Nano victory to console me:

Screen shot 2012-11-29 at 10.13.37 PM.png

Regardless, now that Nanowrimo and 24 Hour Comics Day and the Google Holiday Toy Collection are all behind me, I'm looking forward to getting back to my other projects, including all my writing, the Dakota Frost blog, and, heck, I dunno, my wife, friends and cats. Onward and upwards!

-the Centaur

Derailed

centaur 0

spectral-iron-day16-progress-1.png

Software launches. Anthology editing. I am now officially behind. Time to get back to Nano.

Fortunately I have the next nine days off, starting with tomorrow!

This is why I plan Nano carefully ahead ... this always happens, so you need to plan to have a buffer ... not just getting ahead early, but a place and time to catch up later for if and when you fall behind.

-the Centaur

Just add a dimension

centaur 0

Screen shot 2012-11-13 at 12.19.30 PM.png

At long last, the project I've been working on at the Search Engine That Starts With A G is live. The Google Shopping 3D experience has been launched to the world. From the blogpost:

Explore products in 360-degree detail on Google Shopping Having trouble imagining what a toy actually looks like from the online picture? Now, when searching for a subset of toys on Google Shopping, you can see 360-degree photos of the products. These interactive images bring the in-store feeling of holding and touching a product to your online browsing. Look for the “3D” swivel icon on the product image to see a toy in 360-degree view, on HTML5 enabled browsers. We’ve also put together a Holiday Toy Collection featuring this enhanced imagery—explore the collection on this site. 360-degree imagery is coming for other types of products soon.

Depending on how you count, we've been working on this project for six months, a year, a year and a half, or two years. We've been in launch crunch proper for six months or so, but planning for the launch began a year ago after the launch of the Galaxy Nexus in glorious 3D WebGL (and yes, it did take us the whole year to get this far, and it was really tight down near the end).

The technology demonstrations that led to that launch and made this launch possible began six months before that, and the actual team that was working on them started just over two years ago - and, honestly, it feels like we've been in crunches and sprints for the entire time. Christmas 2012 seemed both far away and far too soon a year ago. It was barely possible.

But we made it.

I don't share much about the innards of The Search Engine That Starts With A G, especially on a project like this, so I'm going to draw to a close with this thought: I work on a wonderful team filled with fantastic people, geniuses and innovators and hard workers all, and each and every one of them were really critical to making this possible (and I mean that. We had NO slack).

I'd be proud to go into [software] battle with you wonderful guys and gals, any time, any where.

-the Centaur

Pictured: the 3D (well, really 360-degree spinner) of the Lego Jabba palace. Article title shamelessly stolen from Asimov. Final quote thieved from Patton.

Postscript: You know, I said "geniuses and innovators and hard workers all" but it occurred to me afterward that most of what these geniuses achieved is not at all obvious. The greatest things we did in this project are completely invisible; you would only notice them if we had failed. Despite seeming to be very simple - a few links, a 3D icon, a rotating swivel - this project actually was the most technically rigorous one I've ever worked on, including both my PhD and the search engine startup I worked on. So when I said these guys are geniuses, I really mean that - they delivered perfection so great it becomes almost invisible.

Now that’s what I’m talking about…

centaur 0

spectraliron.png

Yesterday was nearly a wash - worn out after three long consecutive work days pushing software in preparation for a release, and then out late on date night with my wife - dinner at Aqui's (yum) and movie Tron 3 (AKA Wreck it Ralph, you're not fooling me, Disney). Totally. Worth. It., of course, but still ... less than 300 words done for the day.

sandiyums.png

But today? Up early to take my wife to the airport, had breakfast at Crepevine, and got almost triple that before even 9:30AM!

spectral-iron-day10-progress-1.png

And now, I'm in for my second writing session, before even 10AM. This is what makes Nano work.

Excelsior!

-the Centaur

Pictured: Crepevine as seen from the upper window of Cafe Romanza, my wife at Aqui's, and progress.

UPDATE: Writing Session 2 done, I am now officially caught up for the day:

spectral-iron-day10-progress-2.png

And, despite the last week's slippages, I'm still ahead overall for Nano:

spectral-iron-day10-progress-3.png

Plus there are at least one and maybe two or three more writing sessions today.

Hyperion!

You have got to be kidding me

centaur 0

Screen shot 2012-11-09 at 1.50.38 AM.png

You have got to be kidding me.

I noticed a little extra space on my previous post at the top of a quote I pulled out of SPECTRAL IRON. I wanted to cut it out, so I went to Ecto, my blog client, and switched to its HTML mode. This is what I found embedded in my document as a result of the cut and paste - three hundred and thirty five lines of hidden goop, which looks like it came from Microsoft Word:

  <p><!--[if gte mso 9]><xml>

<o:DocumentProperties>

<o:Revision>0</o:Revision>

<o:TotalTime>0</o:TotalTime>

<o:Pages>1</o:Pages>

<o:Words>172</o:Words>

<o:Characters>779</o:Characters>

<o:Company>Mythologix Press</o:Company>

.... hundreds of lines deleted ...

mso-style-parent:"";

mso-padding-alt:0in 5.4pt 0in 5.4pt;

mso-para-margin:0in;

mso-para-margin-bottom:.0001pt;

mso-pagination:widow-orphan;

font-size:10.0pt;

font-family:"Boldface PS","serif";}

</style>

<![endif]-->

<!--StartFragment--></p>


Charming! Feel like dieting much, Word? Axually, it looks like this may be part of a strategy to ensure formatted cut and paste works in Word and other programs, probably just interacting badly with Ecto.

S'ok, Word. We love you anyway, just the way you are.

-the Centaur

P.S. Pro tip: Option-Command-V pastes unformatted in Ecto.\