Hey, I'm Matt, a Software Engineer currently based in San Francisco. I work for Caring.com primarily in Ruby on Rails, React, and Ember
For a Rails side project recently I finally had the need for a background task handler. Now traditionally I’d jump at Sidekiq, and for good reason. It’s sexy, relatively simple, and comes with a ton of niceties like automatic exponential back-off retries. For this project however, at least at this stage, Sidekiq is overkill. We just need to run some ActionMailers out of the render loop, so we’d like to avoid booting up another server if possible.
So, in comes SuckerPunch, a gem built to address exactly this issue. SuckerPunch uses Celluloid to operate a barebones background task handler inside the server process, avoiding the significant memory overhead of running two Rails instances. Now while its worth noting thanks to the Global Interpreter Lock present in MRI, CPU intensive jobs are still liable to slow down your requests, however the GIL does not apply to external I/O, and so the bulk of what makes a mailer slow (negotiating SMTP) doesn’t block the server.
The topic that’s got me - and it seems a great deal of others - hot under the collar lately is that of the ‘self driving’ car. As Google and others further the technology necessary for a world free from steering wheels, one can’t help but wonder just what that world will be like. Many have dabbled in this topic but I hope to raise some points here I’ve not myself heard.
Time for some serious business. Buzzfeed level serious. Today we’re talking about something a Brit holds dear and oft claims Americans do not. Today we’re talking about… chocolate. One of the many things friends and family repeatadly warn you of before flying out is invariably how shit the chocolate in America is. “They put weird stuff in it! Anti melting agents!” they say. Well of course, they’re right. And you would to if you wanted to eat chocolate as opposed to brown peanut soup when you opened your Snickers wrapper. But that’s not the crux of the chocolate issue. No there is in fact an entirely different offender that no one warns you of…
This is how the weekend began.
Its been a few days since the 48 hour coding grind that was the Launch Hackathon, and I think I’ve caught up on enough sleep to attempt a cohesive paragraph or two. Launch prided itself on being the largest hackathon ever, with 1,500 applicants. They didn’t all show, but it was certainly huge.
Up until now I’ve found it hard to articulate exactly what it is that makes Silicon Valley such a different experience to the rest of the world, but after a few weeks of living it, I think I’ve got it now. Let me take you through some of the mad things than happen here that are just considered normal to the locals.
Firstly, the sheer density of tech workers here noticeably affects the culture of the area, particularly in marketing. Driving into SF on the freeway, you can expect to see billboard after billboard advertising tech conferences, services, and products, right alongside ads for Miller light beer. Here, tech companies have brands so respected that wearing a facebook or twitter or dropbox T-shirt around town is nothing, its just… normal.
[Advanced warning, this is very much a Mum post, don’t expect any philosophical insights!] Several trips to IKEA , and several Amazon orders later, I can finally say that the five of us are looking pretty settled in at 126 Coleridge. Naturally, the house hasn’t quite acquired much in the way of a soul yet, but it sure has a lot of cheap Swedish furniture. Here’s my rather, shall we say, minimalist room:
As I woke up this morning, horrifically hungover from yet another ‘work hard, play harder’ Friday night, to a team of Asian men disassembling all the furniture in our bedroom, I couldn’t help but reflect on what an incredibly brilliant and bizarre few weeks its been. This situation at least is somewhat explainable; Vic, the guy that runs this hacker house, is being evicted because unsurprisingly a 3 bedroom flat is not well designed for 15 people to stay in! We found this out while flat hunting… when we saw the flat we were currently sat in on craigslist.
It’s been a while since I blogged but the topic of this post has been on my mind since we landed. I’ve been noticing millions of subtle differences in the design of almost everything in the US compared to Britain. You might think that with no language barrier the experience couldn’t be that jarring, but I’d argue the many differences aside from the language serve to be just as much a culture shock as struggling to ask for directions when abroad.
I find it particularly interesting because it makes you appreciate the design for things you’d never even considered were thought through by a person somewhere, often because they clearly haven’t been here! Things like, the displays at platform level on the tube tell you not only when the next train to X is arriving, but also the train after that. What use is that!? When would that ever be helpful? Oh good, my train to work which is just pulling in now is here, but if I wait 20 minutes, I can arrive 20 minutes later! Or perhaps, buying a $5 footlong from Subway only to be rung up at $5.49 because tax is added at the till almost everywhere.
The first commute, yesterday, went less than smooth, though I only really have myself to blame. Californian public transport, though surprisingly ubiquitous, is very confusing! Unlike the UK where even if we do have many private companies running the rail, we still essentially have one rail system. In Cali there are multiple fairly disjoint train networks, each with their own nuances and designs. The CalTrain’s nuance is that every train stops at seemingly random stations, and has its own unique number. And to avoid dragging on, that your honour, is why I got off an hours walk from work instead of 10 minutes, and my boss had to come collect me in his Tesla Model S (every cloud has its silver lining!)
Our first full day in SF started, for almost everyone in my 8 person dorm room, at about 6am. Bodies seemed to come to a consensus that despite all feeling absolutely exhausted, sleep was unnecessary. As such we were all waking up through the night, until most gave up as the sun rose. I don’t remember the jet lag being this way around the last time I was in America, but since we had to be on 2nd Street by 9am, there were no complaints here.
I write this first post 3 hours into my 11 hour flight from Heathrow to San Francisco. Several of you asked me to write up my ramblings over the course of this… expedition, for lack of a better word, so do please bear with me as the nature of this blog evolves and hopefully settles down in time. I’m sure it’ll be a lot chirpier by Monday!
A moment ago, having killed the first few hours watching Iron Man 3, I cracked into “Practical Object Oriented Design in Ruby”, when a came across the line (written in the context of beginning a new project) “you will never know less than you know right now”. Truth be told I didn’t get much further than that paragraph; it has been very difficult to focus with my mind spinning as it is. But I couldn’t help but notice how that rather poetic line seemed oddly relevant to my current situation. New beginnings, filled with so much uncertainty, but fortunately things can only get clearer from here. Family and friends insist this will be the best time of my life, and I’ve no doubt they’re right. Nevertheless these first few days will be a culture shock that’s difficult to stomach.
As SNGTRKR is still in its infancy, we find ourselves tweaking the image sizes our models have on a semi-regular basis. Fortunately Paperclip makes that a doddle; just a one line change to the model definition and away we go.
Now all thats left to address is the missing copies of this new image size in production. We have well over 10,000 records with large images associated with them in the database, so the naïve approach of reprocessing every model isn’t going to suffice here. In theory though we still have a weapon in our arsenal; Paperclip provides the
paperclip:refresh:missing_stylesrake task, which will look for a YAML file in your project that stores the previously generated styles.
Unfortunately, that’s as far as that file goes. It does not store per record granularity, so if that rake should bail at some point through your 10,000+ records, well, start again! Haha, screw that. This actual happened to us, for reasons I am still not 100% sure on, but I can only assume it was an S3 connection issue or similar. To solve it, I wrote my own method to do roughly the same as what Paperclip does, but in a slightly more intelligent way; we check whether a potentially missing image exists for each record of a given ActiveRecord model (this is a Rails-centric solution) by actually trying to open the URL the image would be at.
After experiencing some problems configuring Paperclip to behave the way we wanted with S3, I decided to move SNGTRKR to the Carrierwave gem. I had feared this would be a horrible breaking change, but in reality it wasn’t too bad at all. Nevertheless there were some undocumented or poorly documented differences that I thought I’d cover here.
The vast majority of what you need to know is covered in the Carrierwave documentation, specifically the migrating from paperclip section. In summary, you include a mixin that changes the URL syntax in your Carrierwave uploader to be equivalent to Paperclip’s, allowing you to copy paste your Paperclip path into Carrierwave without problem.
The one bit of advice that was not clear to be till delving into the Carrierwave docs was this: the first argument to the
mount_uploaderargument dictates the column name in which the filename of your upload is expected to be held.
At SNGTRKR we needed something to make fuzzy searching in two models and across multiple fields easy. In other words we needed an indexer. This was my first time setting up any indexing engine, and my research led me to Apache Solr. Part of the reason for choosing Solr over any other solution was its tight Rails integration thanks to the fantastic Sunspot gem. Not only does Sunspot have a super easy API to connect your Rails app to your Solr service, but it also has a ready packaged Solr engine that is easily spun up and down with a rake task.
We use Capistrano for deployment, so making sure we could get Sunspot to automate itself during deployment was critical. To that end, here’s what I needed to add to my
deploy.rb. This is an amalgamation of two or three scripts I’ve seen online, none of them had quite them same setup as me, so I thought it would be worth adding one to the pool.
I have decided, somewhat under the influence of 37signals brilliant book, Rework that its time I start documenting all the even vaguely clever webby things I get up to day to day. This blog is where that documentation will happen.
At this stage, I have no idea whether this will be of use to anyone else in the world, but I do know one thing; almost everything I’ve learned in web development in the last two years has been from reading the blogs of people more experienced than I. And knowing that, I figure its about time I at least opened up to the possibility I could for once be the giant on whom’s shoulders another budding web developer stands.
So read on, I hope you find it helpful, even just a little.
subscribe via RSS