Archive for June, 2010

I May Be A Complete Failure

June 27th, 2010

I have spent the better part of my intellectually conscious life trying to algorithmically “beat the market.” (To be precise, I started around age seventeen; I am now twenty-five.) According to some very well-tested academic theories regarding markets, I am pursuing a fool’s-dream. I have continued to labor this long under the assumption that there is a pretty obvious selection problem when it comes to publishing findings that contradict financial orthodoxy: if you were to find a method that earned out-sized returns, I don’t believe academic prestige trumps monetary gains. It is my perception that people who end up as professors of finance are typically people who had the desire to study markets in order to profit from them, but who never found their holy grail. If they had found something spectacular, I don’t think the incentive to publish is very high. (There are exceptions, but nothing ground-breaking.)

After eight years, I have nothing concrete to show for my efforts. As a consequence of shifting needs, I have learned a lot of computer science (e.g. compiler design, algorithms, and some unnecessarily high number of languages). Obviously, this skill set is valuable, but I have no successful projects to use as credentials. Every few months, I find myself excited over the preliminary results of my increasingly sophisticated simulations, only to be disappointed a few short weeks later to find that I was simply wrong. This has happened so many times that I no longer grow excited when I see positive results — I’ve grown into a hardened, semi-depersonalized skeptic.

My latest iteration of development appears exceptionally promising, but I expect it to bear no fruits. I learn each time, and my understanding of markets (and complex systems in general) is approaching some level of refinement, but I have no way of estimating when I might cross the line into profitability; worse I may be approaching this level asymptotically, with my limitations acting as a ceiling just below my goal. I feel like a modern day Tantalus.

I recognize that “beating the market” algorithmically may be either impossible or simply out of my reach, but I soldier on because I still find it fascinating. I believe that, had I switched course years ago, writing off the project as foolish, I would have probably, or at least possibly, been wealthy by other means by now. (Every time I started pursuing a product development project, I found myself shifting back towards market soon after the initial new project euphoria had faded.) If I could offer advise to my younger self, prior to perusing this path, I’d probably say don’t make the attempt. I have neither ethical nor moral objections to profiting by speculation. I merely believe I could have acquired the satisfaction that comes from achievement a long time ago, instead of bearing the frustration that accompanies not achieving something in spite of my best efforts. Nonetheless, I will not stop trying. I’m not blind to the possibility that I am a smart fool, but I want this more than anything else. I’m not sure where I would draw the line, where I would finally say giving up is the proper thing to do. I hope I never have to make that decision. I hope success finally obliterates the need for that decision.

I’m not sure why I wrote this. To some degree, it might be a warning notice to those who are considering following this path. As I said, I find markets fascinating, but most people (my earlier self included) enter the fray believing it to be a sure and short path to riches; it’s not. There are far less risky paths to wealth, especially for entrepreneurial programmers.

Sending Binary Data with the Juno Framework

June 24th, 2010

Juno is a small Python framework similar to the Ruby Sinatra Framework. I needed a very simple way to look at a large set of png files in a specific way across a number of computers, so I opted to use Juno. It took me about twenty minutes to figure out how to send binary data dynamically. (That is, how to send image data via decorated function, not through Juno’s static directory convention.)

The following works, although it might not be the Juno developers preferred method.

def your_request_handler(web):
    content_type('image/png')
    append(string_of_binary_data)

It’s obviously simple but for me, it wasn’t obvious.

Apple Owns Your Harddrive

June 12th, 2010

My first generation MacBook Air has been performing progressively worse over the past year. I had assumed it was just a consequence of software cruft, so I thought nothing of it. Then, last week, git commits started taking several minutes for small Django apps. It was time to fix the problem. Not wanting to be bothered, I did a simple OSX reinstall. (Although, since the Air has no optical drive, it was a bit bothersome; for some reason, the Air refused to communicate with my Desktop.)

Upon resinstall, I was quickly disappointed. The problem was not corrected. That annoying little beachball kept on spinning for practically every task. This suggested harddrive, so I took a look in the logs. S.M.A.R.T. was indicating imminent hard disk failure. (I really have no clue why no glaring alert was generated; I assume S.M.A.R.T. indicated the same thing a year ago — when I was under warranty — but I never checked the logs. Unfortunately, by reinstalling OSX, I lost access to historical logs.) After perusing a few articles online suggesting that replacing a first gen Air drive is very unpleasant, I just decided to bite the bullet and go to the Apple Store.

I went without an appointment, only because I was ignorant of their reservation policy. (Kudos to Apple on reservations; it’s a smart idea.) I was forced to wait around at the mall for an hour until a slot opened up. After learning the mall had no bookstore — a fact that made the snob in me feel angry — I turned to skimming HackerNews. (HackerNews is to geeks what Facebook is to non-geeks — somewhere to go when your bored and want stimulation.) Coincidentally, a blog post about hard drive replacements at Apple happened to be one of the front page stories for the day. This was fortunate, because it warned me that Apple would not allow me to keep my hard disk before having to deal with the genius bar “genius”; it spread my anger out over several quiet minutes instead of one loud minute. Thank you for making me appear like a decent person, HackerNews.

So basically, after deleting all my files and writing a quick python script to fill the hard drive with random numbers, I finally got to talk with my genius. (When you have a failing drive, filling it with random garbage is a painful operation to watch.) I must admit, Apple hires or trains employees very well. He was calm, funny, and generally charismatic. After I told him I want to keep my drive, he clearly explained Apple’s policy: they refurbish the drives and use “government standard encryption” to prevent data theft. By “government standard,” I assume they mean they overwrite the drive about a bajillion times past the point (one overwrite) where data could reasonably be expected to be recovered. I assume his answer is considered satisfactory for most users, but I was unhappy. I’m not paranoid enough to think my single-pass overwrite was going to make me vulnerable, but it’s still my drive. Furthermore, for drives that have not been overwritten, the eventual buyer is not the point of vulnerability — the many hands the drive passes through before being fixed are the dangerous ones. If the user requests their drive, not giving it to them is a bad policy.

Later on I realized that there is something more troubling about this policy. Apple, a consumer product company, is profiting on the failure of their own products. That’s pretty uncool. The replacement cost on my invoice was listed at $139.06 with an additional $85.00 in labor (HARDWARE REPAIR-LEVEL 1). By comparison, a new Toshiba MK8009GAH 80GB internal 1.8″ ATA-100 4200 rpm drive costs $114.85. Considering the cost of the very similar Toshiba drive, I don’t believe they were replacing my bad drive with a refurbished drive. (If they were, that would be especially ugly.) I assume they sell the refurbished drives as part of a refurbished MacBook Air, not as an individual component. Regardless, if the cost of a new drive plus shipping and less the profit from selling my drive refurbished is less than $139.06, Apple is earning money off product failure (i.e. shipping < refurbished profit). I’m angry.

Update:

A commenter on HackerNews asked why I didn’t just go to a non-apple service center. The truthful answer is because I am lazy. Nonetheless, it appears like the policy is set by apple and is universal amongst non-apple service centers. From MacService:

The old drive (working or not) is returned to Apple. If you wish to keep your original drive, Apple charges a significant core charge.

P.S. I’d be curious to see some hardware failure rate comparisons on Apple products. It’s my opinion that they produce bad hardware, but no one cares because most Apple consumers buy the latest version of whatever product Job’s slings out.