Dear Perforce:
Fuck you.
Fuck you, you miserable, untrustworthy, misleading, overpriced bastard. I hope your office goes up in flames along with all your off-site backups. I pray that some open source product that actually works is embraced by all the major companies and drives you out of business. I hope that no other company is duped by your salespeople into thinking you have something even remotely close in quality to the ancient and craptastic product known as CVS. Never before have I experienced so much pain in the most simplistic of version control tasks as I have since starting to work at a company that made the mistake of considering you.
I am in total agreement with Linus that CVS is evil and that anyone who chooses to use it knowing the alternatives is both “stupid and ugly”, but I would switch from Perforce to CVS in a heartbeat. I would bow down an kiss CVS’s authors feet if I could avoid ever using your self flagellation “tool” again.
I am currently the only one working in my codebase:
Pitchforks and vile thoughts,
-masukomi
[Update: Please note that I'm not just bitching about Perforce. No, I don't like it, but I've done something about it. I went and wrote a tool to sync perforce with various Distributed Version Control Systems, but then I came to the conclusion that Git was just too damn cool to bother with the other ones, and switched to using Git with it's git-p4 tool to sync to Satan, er, perforce. If you're interested in that it can be found in the contrib directory of the git source. Just copy it to somewhere in your path and you're good to go (requires Python).]
[Update 2] I’ve responded to the many comments about this that were left here, and on Reddit Programming in a new post.
Code Underwriters Lloyds of London is able to do what they do thanks to the concept of underwriters. The simplistic version is that a risk is spread amongst a group of underwriters. If nothing goes wrong they get a cut of the profits relative the the percentage of the risk they took on. If things go wrong they take pay for whatever portion of the risk they agreed to take on. It occurred to me that being a software developer is in many ways like being an underwriter, except, in most cases, without the big payoff. You see, each developer on a project tends to take responsibility for a portion of the code-base. We talk about spreading this responsibility around by increasing the bust count but the fact of the matter is that this rarely ever happens. Most code-bases tend to be divided into sections each with a bus count of one or two. When something goes wrong the developer(s) responsible for the responsible piece of code are generally expected to do whatever it takes to make it work now. In some cases this means being woken at 3AM, hauling your butt into work, and remaining there until the bug is squashed. Just like an underwriter at Lloyds, the more risk you take responsibility for the more screwed you’ll be when things eventually go wrong. Unlike a Lloyds underwriter we don’t get a cut of the profits relative to the amount of responsibility we take on. It is, obviously, quite debatable just what portion of the profits would constitute a “fair” portion of the profits, but in general we just get paid our salary. The first employees of a company frequently do get the payout, but most developers aren’t them. I’ve also heard tale of magical things called “bonuses” but it seems rare that developers actually get them or that they’re enough to buy you more than one box under the Christmas tree if you do see them. So, for the most part, we can treat these things as non-existent, which brings us back to our salary. Which is, in many ways, like insurance. While you don’t get a huge portion of the profits, you do get a guarantee of sorts from your employer that you’ll continue to get paid well even if things don’t work out. Which makes your employer a fellow underwriter in the development of your software. There are three things that need to be underwritten in any given software project: The effort that will be required to fix things when they break, the money that will be required to finance the fixing of broken things, the money that will be required to finance the creation of potentially broken things. There’s also the money required to finance the project(s) needed to compensate for the projects that didn’t bring in more money than they cost to create and/or maintain but that’s a bit too recursive for this example. From a financial standpoint, I think it’s understandable why developers don’t get a larger cut of the profits. While the worst case scenario for us is rather unpleasant, it’s also very short lived. But, we still get paid quite well during all those months and years when the project wasn’t bringing in anything notable, and even when it tanks. Unfortunately developers are human, and humans are emotional creatures. We tend to not pay attention to the thing that are present in our every day lives, like regular paychecks, and focus on the atypical large events. Like having a fear of flying. If you’re afraid of flying then your brain is not paying fair attention to the boring everyday fact that millions of people are whizzing about the skies without any problems. It’s just that when a plane goes down it does so in such an atypically spectacular way, and that’s what we focus on. So I’d like to propose a couple things for my fellow code underwriters to help mitigate our risk, and thus minimize the atypical large events and bring our perceptions more in line with the small portion of the risk that we’re actually taking on.
Defensive Programming 101
For any given programmer the following statement should always be treated as truth: My code sucks, but your code sucks more.
Good version control habits and test coverage will get you out of most jams related to your own code but we rarely write apps that are comprised of just our code. There are almost always libraries from other people code that you’ll include to save yourself from having to re-invent the wheel. Obviously you don’t want to start writing unit tests for code from other projects (you’d never finish) but there are some basic steps you can take to minimize your chances of failure.
The fact is that no matter how good your code is you just can’t trust anything you didn’t write. Furthermore, you can’t trust what people will do with their code.
You can’t trust other people’s code because it rarely has good test coverage or documentation so you can’t be sure if it is broken, or if it works the way you think it does. And unless it’s a huge open source project you can generally assume that not many people have looked at it either.
You can’t trust what they will do with their code because it’s their code not your code. Whoever “they” are they’ve got their own ideas about where it’s going next and how to get it there, and they may not jive with how you want to use it.
Your project must never rely on the successful build of another project, because it may depend on the successful build of yet another project, or even worse, end up in a sea of circular dependencies.
The results of another project’s “current” build should similarly not be trusted. It’s probably not tested well, the implementation could change completely tomorrow, and it may just be buggy. Whenever possible only use release builds. If you can’t use a release build at least try and use one that other people have had success with for a little while.
Even if you take all the steps possible to defend against other people’s bad code there’s still the fact that humans are incredibly fallible.
You can not assume that the correct libraries will be where they’re “supposed to be” or that the version that is there is 100% compatible with the version you coded against. Even if it is you can’t assume that someone won’t go and “upgrade” it to a version that isn’t compatible. Always bring your own libraries and keep them local to your app. Disk space is cheap and unless you’re on an embedded device it’s better to bring along copies of the libraries you coded against. It doesn’t matter if the ones you bring along are completely outdated relative to the latest version; what’s important is that you can guarantee that the version you coded against works.
Your project must never rely on another project or its libraries. Period. You can not assume that the other project will be on every box your project is installed on or that the version on those boxes be in a state that is either working or compatible with yours. Take whatever you need from their project and copy it into yours. If you need libraries that it generates or contains, copy them. If you must use source code copy it but be very careful. You don’t want to end up forking it and having to maintain your own version. Instead, copy it in but do whatever it takes to make it blazingly obvious that the code should not be edited unless absolutely necessary. When you do copy over make sure to copy the smallest amount possible. If you can, build it into a separate library file that you can update when the need arises.
You also can’t assume that someone isn’t going to screw up a config file, and that includes you. Just like libraries, bring your own. Maybe you’re working on a webapp which has some global configuration file that’s shared by other webapps on the same box. Don’t trust it. Yes use it so that you’ll be in sync with the other apps but since you can’t guarantee that some sysadmin won’t accidentally delete or change some piece of it that’s crucial to your app bring along a fallback with some bare bones settings that should work under any circumstance you can prepare for. This is exactly why your operating system has a “safe mode”. The operating system developers know you can’t trust the users not to accidentally screw something up or install an app that screws something up. Have a fallback.
If your app is configured differently for different deployments (maybe you’ve got a dev box, a QA box, and a production box, each with their own different database settings) then have a separate config file for each deployment. Don’t have a single config file with different sections that get commented or uncommented based on which deployment it is. You simply can’t guarantee that you, or your coworkers, won’t forget to comment or uncomment the right bits when deploying.
It’s a little odd for me. For reasons beyond the grasp of my conscious mind I can see myself staying at Akamai for a long time, and in my experience this is a very atypical thing for a programmer. We tend to be corporate nomads, moving from company to company every couple years, looking for new projects to captivate our minds. Until Akamai the only programmers I’d encountered who stayed in one place for very long were like people going through the motions of work and life. They continued where they were because it paid the bills and they couldn’t think of something better to do. But, while Akamai has plenty of new faces like mine, I keep encountering people who’ve been here for four years, seven years, since the beginning, and they’re happy. They’re not just working here, they honestly seem happy to be working here. And in many ways it really seems like a “corporate family” , something I previously thought was a pipe dream of “upper management” types.
What’s even more intriguing about this is that day to day life in Akamai isn’t tangibly different from most other software companies of our size. We’re no Google with game rooms, washing machines, or any other home away from home type things. We’re bursting at the seams with cubes and the fanciest “perk” is bagels on Wednesday mornings. I sit by the little kitchenish area of our floor and the “water-cooler” conversations are the background sounds of my day. They’re smiles and greetings, impromptu problem solving, and hearing what others are working on, talk about new cars, and evening classes, but rarely ever frustrations.
What is is that Akamai is doing that maintains these smiles?
For me, a little bit of it is luck. We’ve a good team, with really diverse (and good) personalities, a fun challenge, and a good manager, and that’s a great place to be.
99 lines of code on the wall. 99 lines of code. You look around, refactor it down… 98 lines of code on the wall.
98 lines of code on the wall. 98 lines of code. You look around, refactor it down… 97 lines of code on the wall.
Or, alternately
function singVerse(numLines){
if (numLines > 0){ document.write(“” + numLines + ” lines of code on the wall.\n”); document.write(“” + numLines + ” lines of code.\n”); document.write(“You look around, refactor it down…\n”); numLines -= 1; document.write(“” + numLines + ” lines of code on the wall.\n”); singVerse(numLines); } else if (numLines == 0) { document.write(“Totally bug free code on the wall\n”);
} else {
document.write(“Need more tests for the code on the wall.\n”); }
}
singVerse(99);
I keep thinking back to a short comment at Bar Camp Manchester in the Unit Testing talk. When asked if anyone had written an app with 100% code coverage the guy beside me raised his hand. Now I’ve been advocating for a while now that it’s essentially wasted time to bother testing your getters and setters, and when I mentioned something to this effect he said, “How do you know you haven’t made a typo in a variable name?”
…
My mind just kind of spun for a second on that one for a second. I guess I’ve been in Java land for so long that the concept of having such a thing exist for more than a few seconds just doesn’t enter my brain. I mean first off, the IDE will bitch at you, because it can’t compile it, and even if you’re not using an IDE the compiler will bitch at you. I just responded that “I use a compiled language so that’s never something I have to worry about.”
Now, as many of you know I’m still a huge fan of languges like Perl and Ruby so I work in them fairly often. And I still say that it’s not worth your time writing tests for your getters and setters even in them because unlike Java these languages don’t have IDEs that happily spit out a getter and setter for every variable in your class regardless of if you need it or not. As a result, you only create the ones you need. And if you need them, then they’re going to be used, and if they’re used, you can pretty much guarantee that you’ll see the results of any typos show up really quickly when you test the methods that actually do something noteworthy, because they’ll use the getters and setters (otherwise why did you write them?) and either blow up when the compiler hits them or not work as expected, which is why you’re writing a test.
I’ve been trying to get my head around what exactly Hadoop is, how one would use it, and how I can take advantage of it. Well it turns out that Hadoop is a lot more than just an open source Java implementation of MapReduce. It’s also an entire supporting infrastructure for replicating data and intelligently distributing your MapReduces across your cluster.
Knowing that I’m not the only one wishing there was a single document with all the Hadoop and MapReduce basics, I wrote one. Or, more accurately, I summarized the info on various pages into one. Enjoy, and let me know if it was useful to you.
P.S. The really short version is that if you have large data sets/files that need mining, and lots of computers (or a willingness to run your data through Amazon’s EC2 and S3) Hadoop is worth checking out. I’ll try installing it this weekend and write up my notes for you.
How Borders made me into a regular customer (or How Barnes & Nobles Screwed up and lost a loyal customer.)
Let me set the stage. Up until roughly one year ago I was a Barnes & Noble shopper. The only reason I’d go into a Borders was if there didn’t happen to be a B&N around. And then, while I was checking out at a Borders one day the cashier said “Do you have a Borders Rewards card?”
“No” I said, intending to blow it off like all the other useless “customer loyalty” cards that get pushed on you until she said something along the lines of “All I’d need is your e-mail address and you’ll get discounts and coupons.”
It was the just asking for an e-mail address that did it. It free. It’s fast, so I won’t annoy the people behind me in line, and worst case scenario I could always just filter their e-mails into the trash. So I signed up. Now, every week they send me an e-mail with a coupon (usually 20 – 25% off), some discount, and announcements of large new releases. You can get the e-mail monthly or only when you’ve accrued Borders Bucks if you want. If I forget to bring in the coupon they usually have one behind the counter that they’ll scan for me when I present my Borders Rewards card. And, for every $150 I spend they send me $5 in “Borders Bucks”, essentially a $5 off coupon. Considering that programming books are always $40+ and I read a lot of sci-fi / fantasy I actually get them. Plus, since I know that the prices are essentially the same at Borders and B&N but shopping at Borders will get me coupons my shopping has totally reversed. Now I only shop at B&N if there isn’t a Borders around. Oh, and in case I’m really screwed for a good book store Borders happens to own Waldenbooks and my Borders Rewards card works there too.
And that happened about a month ago. I forget what I was looking for but my local Borders didn’t have it so I went over to B&N and when I went to check out they asked me if I had a B&N Member Card. “No” I said, thinking maybe I’d have an excuse to start shopping at B&N again, “It’s only $25 ,” he said “and you get ….”
Twenty Five Dollars! Twenty Five Dollars! Now… let me think. Borders = Free & gives me discounts. On the other hand I could give B&N $25 and get the same discounts and… well… nothing. I’d just be giving the $25. Needless to say I didn’t take the offer. But there was another side effect that I don’t think B&N thought about. Now I don’t want to shop at B&N because I know that B&N is trying to screw it’s customers out of $25 for something Borders was happy to give me for free.
Borders wants my business and is willing to give me discounts, and an occasional $5 in exchange for my patronage. Barnes and Noble wants to charge me for the privilege of getting discounts. Given those two options which would you choose? Now I don’t shop at Borders just because of the discounts, now I shop at Borders because they know a good deal more about TreatingCustomer Right than B&N does.
If you don’t have a Borders Rewards card I highly recommend you go get onenow. It’s quick, easy, free, and the e-mails aren’t obnoxious.
P.S. I’m not opposed to paid membership / discount programs. There are some businesses where they work well. Costco is a great example. I even did it in an old online business. We charged $10 for membership and I made damn sure that the average customer would get way more than $10 in savings. But, in the case of my old co. and Costco there wasn’t another company out there offering the exact same discounts, or better, for free. To try and charge people an annual fee for something they can get from an equally placed competitor is bullshit and shows that you don’t really care about your customers, just getting into their wallets.
[Update]: A related post How Borders Lost My Sale.
There’s an interesting educational series called Connections which follows the circuitous paths of history and fills your brain with interesting facts. But, while I do enjoy watching it, it’s premise drives me nuts because of how amazingly tenuous some of the “connections” are. So, just to make sure I wasn’t an idiot, I took notes through a couple of episodes. Here’s what I learned in the episode called New Harmony. I may misspell some of the places and people and be slightly off in my understanding of any of this but please bear with me. At least I learned stuff. Singapore is the cleanest modern city thanks in no small part to it’s ginormous incinerator which is controlled by microchips which were discovered when Shockley found that if you add impurities to crystals you get extra electrons which led to the creation of the transistor without which we wouldn’t have microchips. Now Schockley made this discovery with a crystal called Germanium (remember this for later) which was discovered by a German named Vinkler who also invented industrial air filters, the descendents of which clean the air in Singapore’s incinerator. But, Vinkler while a great mineralogist was really in the business of selling cobalt which is used to make blue dye which was in demand to make fake Ming vases because the real ones were too damn expensive. Ming vases are white porcelain with cobalt blue pictures on them. Cobalt because it’s one of the few things that can withstand the heat required to fire porcelain. Now the Chinese discovered cobalt when they went to Istanbul which was at the end of the Silk Road. In Istanbul they used cobalt on tiles in mosaics for anything really important like the incredibly famous mosque called The Blue Mosque of Istanbul (Isopia?) where they hired the greatest tile artists to make it beautiful no expense spared (thus all the blue tile) and cover up the fact that the Christians had actually built it and copy a lot of great Christian mosaic work that displayed the glory of Byzantium along the way. Now these Christians were actually an underground sect (literally at times) around that point in time and weren’t given legitimacy until The Donation of Constantine when Constantine gave the west to the Pope and thus legitimized him. But, about a thousand years later the document on which The Donation of Constantine was written was exposed as a fake (probably by the Vatican) by a guy named Vinkler who surprisingly found himself not dead thanks to a local Godfather to whom a lot of people owed favors including a Portuguese guy named Martens who was the head of the Oceanic Exploration Agency which was roughly what you’d get if you combined NASA with the Defense Department. And they were the only ones who knew how to “run up the latitudes” thanks to a nifty invention that only they had called a Quadrant. This meant they could intentionally leave sight of land and not worry about never being able to find home again. Well, one Portuguese accidentally went a bit too far west and hit Brazil which the Portuguese colonized with Jews who wanted to get away from persecution (again) but eventually The Inquisition made it to Brazil and many of those Jews headed to Holland where they became diamond merchants. One of the diamond merchants had a son who was really into optics and made some really nice lenses which got the interest of the English Royal Society who came over to check them out and see what else was going on in Holland. They came across a guy with a 250x microscope who’d discovered and documented that there were micro-organisms everywhere! Word of this got round to Switzerland where Byron, Shelly, and Bryce (a child at the time) where having dinner and ended up discussing them. One thing led to another and someone mentioned that a scientist had recently galvanized pasteur and brought it back to life. This got Shelly’s wife in a tizzy and she went and wrote Frankenstein. Except Frankenstein wasn’t just an attack on mucking about where we had no business. It was also a subtle attack on the technology of the industrial revolution which she felt was ruining people’s lives thanks in no small part to the beliefs of her socialist father. Her father happened to be friends with a guy who made a workers paradise in his factory in New Lennik Scotland which was way too revolutionary for its time and left him depressed that he’d never be able to get anywhere with the idea. Until a colony came up for sale in Indiana which he bought with the financial backing of a guy named William Maclure who was quite inspired by the factory and was all into the idea of creating a socialist commune which they called New Harmony Indiana. Maclure happened to be a geologist who made the first geological map of the fledgling United States and found some Germanium deposits which happened to be the same mineral that was used in the first transistors that led to microchips and Singapore being what it is today. To summarize: microchip from transistor made with germanium discovered by a German who sold cobalt for fake Ming vases. Ming era potters found cobalt in Istanbul which has a great mosque which was a cover up for Christian works which became legitimate by a The Donation of Constantine which a thousand years later was exposed as a fake by a guy who wasn’t killed thanks to a Godfather who also helped out the head of the Portuguese Oceanic Exploration Agency which had great navigational tech and three years later “discovered” Brazil where their Jews happily ran away to until the Inquisition caught up with them, at which point many of them moved to Holland and one of them had a child who made great lenses which caught the interest of the English Royal Society that found a dude with a microscope who told them about micro-organisms which they in turn told everybody about. Word got around to Mary Shelly who wrote a book that most people don’t realize is an attack on industrialization thanks to her socialist father who had a friend who started a commune in Indiana with the financial backing of a geologist who found some Germanium deposits in the U.S. and Germanium is what was used in the first transistor. While the story is cool, some of those connections are Damn tenuous.
I was speaking with someone at BarCamp and they suggested that your unit test should never be longer than the thing it’s testing. His thinking was that if your test is longer than what it’s testing then your tests will eventually need a test framework to make sure they’re correct. This was of course one of his excuses for not writing tests at all. And, superficially at least, there is some validity to his argument, but it’s based on a flawed premise. It’s based on the premise that the code you write in your test suite is complex enough that it too could be brittle, and if you ever find this happening it’s a sure sign that you’re doing in wrong. Let me explain: Unit Tests should be simple… really simple. Each test routine should basically have these parts:
Instantiating an object is not a complex operation. It’s usually something along the lines of
Bar myBar = new Bar();
You don’t have to worry about that breaking, normally because constructors don’t tend to do much. If your constructor does have any complexity happening under the covers then you’ve got a unit test for it somewhere else and thus you don’t have to worry about it. Creating a mock object not only avoids potential complexity but utilizes a well tested library so you don’t need to test that either. Yeah it may take a few lines but it isn’t complex or brittle and it is guaranteed to work exactly as you tell it to. Assertions are just
assertEquals(“camels”, myBar.favoriteAnimal()); // or maybe… assertTrue(myBar.isSimplistic());
First off, the assertions themselves are methods from a unit testing suite, you can rest assured that they not only work correctly but have full unit test coverage. So you don’t have to worry about them. Second, they’re brain dead simple. There’s no logic, so there’s nothing to break. The only thing here with the potential to blow up is the methods you’re testing and that’s why you wrote the test in the first place. Your unit test should have absolutely no business logic in it. If it does then you probably need to refactor your code. So, lets look at a simple concrete example of why the length of your method is irrelevant to the length of your test. Most of you are familiar with QuickSort. It’s a simple and fast method to sort a list of things. You can probably write a QuickSort algorithm in under 15 lines of code. So what happens if you pass your quicksort algorithm a list of twenty shuffled items. You’d want to test that it correctly sorted all twenty of them wouldn’t you? Considering that this is a test you know exactly what the input is and exactly what the output should be, but there’s only one way to really test it. You have to check every slot of the returned list and make sure it’s got the correct item in it. So that’s at least one line to set up the list you’re going to pass to your quicksort method, one line to pass it in, and twenty lines to test the results. Something like this:
SortingUtilities sortUtils = new SortUtils(); String[] unsortedMen = new String[]{“tom”, “dick”, “Harry”……and 17 more}; String[] sortedMen = sortUtils.myQuickSortMethod(unsortedMen); assertEquals(“dick”, sortedMen[0]); assertEquals(“Harry”, sortedMen[1]); // case insensitivity testing too assertEquals(“tom”, sortedMen[2]); // and so on for another 16 lines
Grand total of 23 lines plus 2 more to start and end the test method itself, plus you’d probably want to break that list of unsortedMen down into multiple lines so it would be readable. All that just to test a fifteen line method. For good measure you might want to add a line to test that the size of the array that came out is the same as the size of the array that went in, just to make sure that nothing was accidentally duplicated in there. But nothing, nothing, in that list even remotely needs to be tested itself because it’s all brain dead simple. There’s nothing complex. There’s no business logic, there’s nothing you have to worry about. The fact that it happens to have more lines than the method it’s testing is totally irrelevant. The only excuse I’ll hear for not wanting to do this is that you’re too damn lazy. At least that’s honest. But claiming that the length of your test suite has any relevance to the length of the method under test is just plain ignorant. You’re a programmer. Use your brain. Don’t make poorly thought out, half-assed excuses not to do the right thing. If you’re going to avoid doing the right thing at least be honest about why you’re doing it. Laziness sucks, but at least it’s honest, and since people don’t generally like thinking of themselves as lazy it has the potential to kick your ass into gear. And, because it always comes up, the excuse that writing all those lines of test code “slows you down” or “takes too long” is bullshit because we all know that the vast majority of time spent on any project is in debugging and maintenance and when you’ve got good tests that time is dramatically reduced. You end up saving far more time than you spent writing your tests, and your end result is just more bug free and stable. If typing out twenty lines of brain dead assertions takes you more than a minute and a half you need to get off your butt and take a typing course.