Wednesday, May 30, 2012

jPOS and the next generation

Having worked in the POS and payments market for many years I've evaluated, worked with and recommended the jPOS toolset. On it's own it provides a codec-like API for encoding and decoding ISO8583 messages. The API are generic enough and support all of the important variations and it's user extensible when it does not.

The best part, however, is that they also have an EE-ish version that provides an almost complete solution for implementing your merchant gateway, acquiring processor or issuing processor. Don't get me wrong, you still need to be an expert to implement one of these systems but the jPOS stack is going to give you a java-leg up.

In truth however, @apr is probably the best part of the offering. He has a keen eye for where he sees the project going and what features need to be added as it expands. He also knows what features should be left to the user to implement. And he knows how to communicate the system design as well as educate the user in how and where to implement the user's code without effecting any sort of upgrade path.

I'm so fond of this system I which I had a language agnostic version that might be more like a meta programming or code generated version so that I could plug-in the target language ad-hoc. And if that was too much to ask, then I'd want a python version so I could operate in the same space and maybe get a little more productivity.

Tuesday, May 29, 2012

Will the real social network please standup.

My wife and I watched "Social Network"  the other night. I cannot say that I had a new found appreciation for Facebook after watching it. In fact the complete opposite.

If there is any truth at all to the movie then we, the rank -n- file investor, should have known in advance that Facebook stock was going to smell like a cow pie. I did not invest in FB for exactly that reason and I get a sense that my decision was well founded.

The one thing that the movie portrayed was that the Zuck was fast and loose. He had an itch and he scratched it. The IPO was no different when (1) his team hacked the button (2) got married on the same day (3) wore a hoodie to the even showing that he still has not grown up.

I do not know what the future holds for FB. It could end up being the Microsoft of it's generation. But while The Zuck scored 1600 on his SATs I don't think that qualifies him to run one of the newly richest companies on the planet. And I certainly do not trust him or his company with my personal information.

Monday, May 28, 2012

Python PEP-405 - virtualenv - like

PEP-405 is recommendation to include some virtualenv-like like functionality into the python stdlib. I suppose this idea might actually fly if python were driven from a single PYTHON_HOME or PYTHON_PATH env variable and for the most part it seems that PEP-405 suggests that potential.
It should be noted that this PEP was also endorsed by Ian Bikling the inventor of the proper virtualenv. --PEP-405

There is some discussion about backward compatibility but it is sort of vague and very mystical in a hand waving sort of way. One reason it might actually work well is that one application like the current virtualenv toolkit does not have to carry around all of the version info needed to work in each python version.

But let's be clear. PEP-405 is not virtualenv. It is virtualenv-like. It is also approved for deployment in version python 3.3 and I do not see anything about back porting.

Virtualenv is a killer feature. If they miss the mark and abandon all that came before I hope that someone picks up the slack.

You're Fired!

There was a time when Donald Trump's tagline really meant something. It meant that you were incompetent or you royally screwed up. It was a phrase that no one ever wanted to hear. And it was a Scarlet Letter that you carried from one job to the next.

However, in today's politically correct and litigious workplace while you might be fired for doing something wrong you'll never know about it and anyone checking your references will never get anything other than "yes, I can confirm that Mr Bucker worked for us from <start date> to <end date>". If they said anything other than that they would be opening themselves up to a defamation lawsuit and those get ugly fast.

In countries like Sweden they have all sorts of laws to protect workers. Firing an employee in Sweden "for cause" is possible but it very hard. First there is the amount of documentation that is required. And even after that there is a period of required severance which can amount to a year or more (as I understand it).

There are a lot of reasons why employer and employee separate and only a small fraction are related to abject failure. I wish there were a survey on this. But the truth is you'll never really know why unless someone tells you. And even then you'll never know if that was the real truth or the truth that the someone was comfortable with. The only thing you'll ever know is whether you resigned before the ink dried on the termination letter.

Saturday, May 26, 2012

Changes to this website

Just a quick note about some changes I've made to this website.

(1) I decided to change the name and sub-title. This side generally projects my experience and thoughts and while the title was initially acceptable and very web 2.0 it lacked a genuine description of the information I might impart.

(2) I removed the Box control in the sidebar and replaced it with a shared DropBox folder. I liked the Box control, however, it failed me when a colleague tried to download my docs. So that was changed. Hopefully DropBox's site is stronger than the Box tool.


One lost secret of Agile teams

I recently wrote about Agile Anti Patterns and as the sun sets tonight I started thinking specifically about Agile teams. Historically there have been times when individual contributors have excelled and then times when teams functioned better. But the point I want to make to management...
While there are times when teams function better than individuals the current research by Agile researchers who have a need and desire for Agile principles to be proven do not seem to take into consideration

For example:
(10 years exp * 10 members) is not equal to (2 years exp * 10 members)

but more interestingly:
(10 years exp * 20 members) is still not equal to (2 years exp * 10 members)

The one thing that most Agile project managers do not convey is that there is no substitute for individual maturity as a component of working on a team and there is also no substitute for experience when solving problems in the same problem space.

Consider this... 100 of the smartest college grads and maybe even PHDs from MIT are not likely to get a rocket like the Dragon to the ISS on the first try. While there may be some of these resources on the team they are not the driving force. I'm certainly not writing a $100 check for a grad student's senior project let alone $10B to go into space.

Is it possible to be agile and an experienced Perl developer?

In the modern Perl book the author writes:

A Perl novice might multiply a list of numbers by three by writing:
my @tripled;
my $count = @numbers;
for (my $i = 0; $i < $count; $i++)
$tripled[$i] = $numbers[$i] * 3;

A Perl adept might write:
my @tripled;
for my $num (@numbers)

push @tripled, $num * 3;

An experienced Perl hacker might write:
my @tripled = map { $_ * 3 } @numbers;

As I look at these three snippets of code they all make perfect sense to me and while I very infrequently use the map function it's not unknown to me. The reason I do not use it very often is for the maintainer's sake. If I'm going to challenge myself to remember what or how I implemented something then why should I put that burden on the next person.

So the challenge is this. With an Agile team you can expect that the members are going to have varying experience. Will it help or harm the team of the expert programmers write expert level code or if the rank and file try to write expert level code? Will the team be so distracted with output based on experience that the team is generally distracted from the real mission?

What I'm suggesting is that there is a balance.

Friday, May 25, 2012

perlbrew and mojolicious

I'm not a fan of the guys over at mojo but it's probably the better of the Perl micro-webframeworks out there. So I was curious if mojolicious was going to work with perlbrew.

The first thing I did was install perlbrew. There are several ways to do it. I decided upon the first option:
curl -Lk | bash

What I do not like about the above command is that the code is assumed to be good and safe. It would have been a little more helpful of the code were downloaded from CPAN.

Once the module was installed. I was directed to add a line to my .bash_profile and then restart my terminal session. Easy enough.

NOTE: I did not recall what the base version was so I edited the .bash_profile file again and commented out the line that I was instructed to include. Then I opened a new terminal session and executed the command:
perl -v

My default/host perl version was 5.12.3. And I wanted to install perl 5.16.0 the latest and current version of perl:
perlbrew install 5.16.0

Easy enough! At this point there was a message on the console that suggested a tail command that I could use to monitor the build. That was easy too. In the end it took about an hour or so and I had a working Perl 5.16.0. (feel the perlbrew doc for the interesting commands)

As a last step I wanted to see what was going to happen when I installed mojolicious, could it be installed in userspace, and which version was it doing to use. So I installed mojo:
curl -L | perl - Mojolicious

I omitted the 'sudo' that the mojo guys recommended and it installed fine. But now the proof needed to be in the pudding. I created a file:
use Mojolicious::Lite;
get '/' => {text => 'Hello World! ' . $] };

Notice that I added the $] to the message. This is going to append the Perl version number to the end of the hello world string. The good news is that when I ran the application:

and launched my browser, I received a message that told me I was using Perl version 5.16.0. Perlbrew was a success and so was Mojo.

Another killer app for Perl

I've written about perldoc and CPAN as being Perl's killer apps. I've also written about Ruby's RVM and Python's Virtualenv. Now I get to write about Perl's perlbrew.

I've been tweeting(@rbucker) with a couple of techies today s a result of a comment that one of the made. Something to the effect that virtualenv was going to be made a core python app. Suggesting that it was going to be rolled into the distro.

If you've been around a while and you have a little intuition... it should be going off at this very moment. I'm not going to go into the high level discussion that I had with these guys nor am I going to go into the micro details. What I will say, in summary, is that this is a very bad idea and as a result virtualenv should become very unstable as a result.

Which got me thinking about Ruby and Perl. On the one hand I know that Ruby has RVM but is there something for Perl? Yep! As I write this article I have installed perlbrew and I'm installing Perl 5.16.0 at this very moment.

I do not know anything about perlbrew at this point other than it seems to be installing Perl properly and in userspace where I want it. If all goes well and I have the required prerequisites all should be well in the next little while. I really like Perl and Python. The idea of dumping Python feel like jumping the shark. Perl-6 and Python-3 feel unnatural at the moment. I'm just hopeful that virtualenv and perlbrew can keep my world glued together until the rest shakes out.

Killer new feature

I'm trying to locate an eBook called "Effective Perl Programming (2nd ed)". It's a fairly popular book with pretty good recommendations. But what has totally pissed me off is that the one site that I thought was legit, redirected me to one of two commercial sites that either wanted my personal information or wanted me to download their downloader. Can you say malware or privacy?

Boy it would be really nice of the search engines could filter these sites out of my results... and since Chrome is my browser if nefarious links could be removed or highlighted.

And while I'm at it I want some way to reject SMS messages from people who are not in my address book.

Coda2 - comments

Panic recently released Coda2 and Diet-Coda. I really like many or all of the new features and I'm looking forward to getting a complete handle on it as a tool... I've used other versions in the past.

My only complaints are that they (1) do not currently support HG/BitBucket and (2) no split vertically (3) no native support for remote tmux (4) no Perl resource book(s) (5) no markdown preview or mmd support.

One of my biggest complaints of the previous version of Coda was that if the connection to the remote server was iffy then saving a changed file could be a challenge. The last thing I ever want to do is save the file locally, wait for the connection to resume, and then forced to copy the file to the remote server. For this challenge we might be better off with the equivalent of a STASH that is/was replicated to the server and then back to the local computer.

Thursday, May 24, 2012

TDD is putting the cart before the horse

Can you imagine the Budweiser Clydesdales trying to push their beer wagon? Personally I do not see it happening any time soon. So as I think about TDD (test driven development) that's exactly what I envision.

When programmers develop code there are a number of things that they should be doing... not the least of which is functional tests and regression testing. Now whether these tests are actually written after the function is written or not can be debated to some extent. What cannot be debated is just how much TDD is implemented before actual implementation of the target application, module or function.

I recently heard a story about a CTO who demanded that the entire suite of TDD test cases be implemented before the first line of code was written for the target application. This was clearly an unrealistic expectation and probably just abuse of power.

PS: consider dependency injection if you can. It makes testing easier without having to backdoor any sort of state or dependency.

Agile Anti Patterns

When you've been around the park as many times as I have you can get a little ambivalent about the direction you take whether it's clockwise or counter-clockwise. And while I reject the formal nature of Agile, Scrum, KanBan, ScrumBan there is some value and there are also some traps.

So today I'm starting my anti-pattern list and I hope you'll contribute.

(*) You never read the Agile Manifesto.

(*) You never read the Principles behind the Agile Manifesto.

I only know two of the names on the list of 17 original signatories. Andrew Hunt and Dave Thomas. These guys are fantastic programmers and now publishers. If you assume that they have 60 years experience between them, 30 each, that might be 510 years experience in just those 17; and so there is some wisdom in the original documentation.
Each of us is on a journey through life, but few, if any of us are on the exact same journey. --Rabbi Norman Lipson

(*) The wisdom of the group or team. The implementation of Agile has become team centric in team rooms instead of offices. All you have to do is read Herding Cats and study ergonomics + human peripheral vision  to know why this is a bad idea. What is the point of a team room when (a) members put on noise canceling headphones to drown out the sounds of their peers or (b) when people try to get your attention by waving their hands wildly in order to catch your attention. Team rooms have their place but not exception or the rule. The team is meant to support the individual as the individual is meant to support the team.

I interviewed at Microsoft in the late 1980's. In those days everyone had a window office to themselves (look at the shape of the campus buildings) and everyone had a door. When the door was close the rule was "honor the closed door". In early 2000 I had a neighbor who recently relocated from Seattle. The new rule was "don't close your door".

(*) Complete Agile adoption. I'm not sure I even know what that means. Project managers who are dedicated to their craft the way that programmers or other skilled individuals are will always endeavor to learn more and expand their personal and professional growth. Agile has already grown from the principles and the manifesto into something akin to 7-habits. Except that Agile is no longer a 12 step program.

(*) Gaming the system. I'm certain a social psychologist could make their career on this topic. Anyone who says it does not happen in the workplace is either lying, naive, or a fool. We all strive to get more for and with  less.

Your anti-patterns are welcome!

Wednesday, May 23, 2012

Fork bomb - Ruby

I'm reading a new book titled Working with Unix Processes which is published by, you guessed it, I cannot tell if this book is golden or beta but that nugget aside it's actually an easy book to read. The author is trying to show how connected Ruby is to Unix processes and for the most part he meets the mark.

What troubled me is not the tight coupling of Ruby to Unix but it was the notion of a fork bomb and that Ruby uses about 500MB (half a gig) of main memory before the application actually does anything. This suggests that with just a fews forks and average memory amounts one could bomb the target system.

Which has me asking the same question of Python, C, java and go.

PS: forking is probably not the right thing to do. daemontools and tools like it are a better approach for demonizing your apps.

Tuesday, May 22, 2012

Malware in the Apple AppStore?

My 2 year old daughter has decided that she is going to handle the DVDs in this family.  She has already figured out how to open and close the DVD player and she knows most of her favorite DVDs on sight. The bad news is that DVDs are designed to self destruct at the hands of toddlers. That's just the way it is.

So it is under the umbrella of fair use I've been trying to rip my DVDs and then upload them to my Tivo for viewing... she had not figured out that remote yet.

I went to the Apple AppStore and started looking for DVD rippers and there are quite a lot to choose from. Unfortunately there appears to be a clause in the developer agreement that does not always get posted on the AppStore description. The AppStore version is probably not going to rip encrypted DVDs.

Many of the vendors have a general workaround for this. They let you buy their software directly through their website. This is a nice option to have except the websites I've been to are the same websites I would never purchase software from in the first place.

And here is where the malware starts.

I do not know these guys but if you look closely at many of the screenshots on the AppStore and their websites ... (1) the software has remarkably similar layouts and features (2) for some reason the publishers are Asian like Li and Chen (3) some of the publisher websites are defunct (4) one publisher generated a 404 from dropbox (5) one publisher, easymac, directed me to softpedia as it's highest search results.

All of this makes be very uneasy. I know that the functionality of of all of these apps are the same because they pull from the same toolset and APIs. But the fact that the layout and icons are almost identical is just plain scary. All in all I'm starting to think that the AppStore sandbox requirement is going to be a good thing.

Monday, May 21, 2012

Lead Generation - the analog enemy

Many years ago I worked for Premier Global. They were in the messaging business. Everything from fax, to voice, to email. That included incoming and outgoing messages. I remember one fine day when a coworker was recognized by executive management for tweaking their robot dialer to wait a few seconds after the remote side picke up the phone. The assumption being that the receiver would say hello and if they were interrupted by the dialer that they would most likely get a hangup. Anyway, they were right. The conversion rate was much higher now.

I've hit this topic several times this year... and now that I'm all the do not call lists I'm still getting calls. The one list that really pissed me off this weekend was the Broward County Police and Benevolent Society. The agent happened to drop the dirty little secret that there is more than one company working for the Police and Fire. This disgusts me on many levels.

Anyway, after reading this morning's jobs wanted lists I saw that there is yet another lead generation company using robo-dialers looking for programmers. So the question is... with all of these lead generation companies attempting to get our attention at almost the same "best" time of day. What would it take for these robo-dialers to completely saturate the analog phone system? Figure that any one of the Amazon servers could handle 5,000 - 10,000 calls at once depending on the networking. A complete DDOS could be scripted in a few hundred lines of code.

Sunday, May 20, 2012

Apple OSX FileVault - a false sense of security

The guys at Apple have done a pretty poor job describing the inner workings of FileVault and FileVault2. I'm sure that part of that is for security through obscurity. And the rest might be FUD like federal agency back doors. Recently there was a well publicized DEBUG flag that put the user's password in the clear on the disk.

All of that aside. Back in 2011,  when I was working onsite in Sweden, my laptop decided to stop working. The problem was quite serious and at some point in the diagnostic I was whacking it against a coffee table. On the upside; my client had a contract with Dell who provided many hundreds of identical mini laptops running the latest Ubuntu with a totally encrypted hard drive. The user had to enter a password for the encryption and then a userid and password to win access to the OS. What was interesting about this model was that unless someone entered the HDD password there was no way to get to the data. I'm fairly certain that the password contained some shared info between the CPU, the user and Dell; in some way.

The way that Apple is doing things with FileVault2 is almost similar, however, it's never clear whether the user's userid and password are sufficient to get past the encryption. In which case not everything is encrypted. This is especially bad if the users' name and picture are displayed in the login screen. A bad guy prone to violence or severe measures can now match a user to his or her hardware definitively.

In many ways the original FileVault is better. (1) because only the encrypted data is encrypted. DUH! and therefore the CPU cost to decrypt can be reduced for things like audio and video files. (2) It takes the actual users' credentials in order to access the data which has the downside that the OS could always be compromised.

I think I like the Dell approach. It's overt and so there is mo mistaking that the system is encrypted. The Apple version leaves too many questions for the legitimate user.

Wednesday, May 16, 2012

New Book idea - Startup Accounting

Commenting on articles has been a bit of a hobby for me lately. In this latest installment I take on the startup entrepreneur. In some circles they can and cannot spell. But how are they able to afford things?

For one; they hire cheap labor. They may be programmers themselves. They hire college programmers trading compensation for internships with ping pong tables or stock options. They practice silent HR discrimination trying to determine who has a family and who has kids. Or trying to determine who can work nights and weekends. Many, however, will not outsource because they are actually trying to protect some unrealistic intellectual property, however, some companies have opened development offices in the remote reaches of the world like Dublin Ireland.

And that's about. it.

PS: the bit about the HR department was recently reported on the national news.

Correcting some misconceptions

I just read an article from MickeyMcKay where he espouses proper grammar for entrepreneurs. While I have no idea if he's correct or not I should mention:
In deference to mickeymckay just a few years ago it was reported that bad grammar and spelling [was] a sign of a good executive. -- @rbucker

Another author was writing about not being able to find Ruby on Rails programmers in Atlanta; and that he was forced to troll the local user groups in the hopes of finding candidates.

A few years ago I was having a similar problem recruiting RoR programmers in Birmingham Alabama. The root cause was that there were simply not enough RoR programmers at all. And so if we wanted to hire them away from their projects we had to raise our already high salary expectations.

As for the corrections:

I do not believe that grammar is a key indicator of anything except someone that might be educated or at least remembers their education. On the other hand it might also be the mark of someone with a good spelling and grammar checker. Either way it's cancelled out and generally meaningless.

Qualified programmers of all types are in demand; they are very selective about their work, compensation, environment, contribution, and so on. The real reasons however; (1) most new programmers have huge egos that need to be satisfied (2) most experienced programmers cost too much; (3) no matter their situation almost all of them are waiting for a huge payday that has not come yet by the latest fly-by-night contrived social app of the day.

Monday, May 14, 2012

The business of hiring programmers

As I sit here on the eve of an interview and the programming test which I always dread taking and I find myself reading page after page of java interview questions. One blogger pointed out a number of professional test sites... but it all caused me to come back to the same place.

Isn't it more important to hire someone who is more generally skilled and knowledgeable than someone who is singularly focused on java or some particular language? And how would you interview that person?

Apple OSX mindshare

Apple's mindshare is as much their hardware as it is their software.

Just a few minutes ago I was reconfiguring my in favor of GMail in the browser and Sparrow or MailPlane on the desktop. This was just part of a whim but it's probably a good thing that many application developers are so narrowly divided.

I also spent a little time on my BETA version of Apple's Messages. A replacement for iChat. Some of the preferences were clearly BETA and a lot was leftover from iChat. But after all that I can see that I'm still going to need Skype. Sure iChat and Messages support voice and video chat. I also have FaceTime for my desktop too. But in the end those apps are OSX only.

So long as MicroSoft is putting MS Word on the OSX desktop; Apple should be putting their apps on Windows.

You have the next great disruptive idea; now what?

I have one client who wants to build and host everything in their domain. They do not have the killer disruptive app or anything but they have a good idea for a service business that is going to keep them in twinkles for a good long time.

But if you had the next great idea and you wanted to focus strictly on the components that are considered core to the business and not the periphery. For example; instead of a social component you'd connect to Facebook. Instead of an emailer you'd use mailgun. And for logging maybe something like loggly. Of course there is mongodb, puppet and chef services and so on...

When I look at these services the costs seem on the high side per server. And when you combine the costs; they are in the stratosphere. For example; I have a client where I run 4 asterisk servers and 4 admin consoles. I wanted to try newrelic but when I looked at the pricing it was simply too much. The servers were costing 2K/mo for all 8 of them and newrelic was going to cost more than double.

Recently the first client I mentioned; has started to change course. There are some things they are now willing to outsource and something that absolutely need to be internally supported. It's a tough formula but the line is somewhere between profit and overall cost.

Back to my original question. If I outsourced and managed everything except my core. What is it going to cost? Is it going to be reliable enough? Will it scale?  How do I get to the next step?

But what will it cost?

Saturday, May 12, 2012

Apple OSX Junk Mail

There is a lot to like about Apple's Mail application that comes with OSX and there are a few things I don't like.

(1) the integration with other tools like iCal is similar to what MicroSoft did years ago with it's mail APIs. It is brutally painful when the Mail application is not configured or when all of the mail accounts have been disabled. What's worse is that the act of emailing an alert from iCal actually launches the Mail application instead of using the APIs behind the scenes.

(2) I would have liked it if the junk mail between my GMail account and the Mail application were synchronized. It really bugs me that I have two folders. I do not know who does junk mail better but I have to imagine that it is Google only because they actually have the entire population of GMail users rating emails for them. While the Apple Mail user base has to wait for the next release or update in order to get the new spam rules.

(3) The one killer feature I like about GMail and MailPlane is that very little or absolutely nothing is actually stored on my local computer. That means that all of the email remains on Google's servers. This is great especially with my MacBook Air which has very limited space. The last thing I want to do is download 10 years of emails from 10 or 15 accounts. Also this reduces the time it takes to bring a new computer online.

Mailplane has released a new application called replies. It's similar to MailPlane and Sparrow in many ways but it seems to download my entire email cache from GMail. Sparrow appears to download everything too. (I need to check their files to be sure) I only wish Mailplane looked a little more polished the way that sparrow does.

Friday, May 11, 2012

Freelance licensing of 3rd party software

In a related difference of opinion...

As a freelance programmer code comes from 3 places. (1) I write it from scratch, (2) I use some OpenSource code with compatible licensing, (3) I use some of my own libraries that are not strictly licensed one way or the other.

Whatever code I write is usually going to be "work for hire" and so it will become the property of my client. He or she will have to decide what that means to the general public as in OpenSource or if it's considered intellectual property.

On the other hand, if I included 3rd party licensed software then it's pretty simple. Everyone has to adhere to the terms of whatever license(s) are written. Just about everything is going to work here. Even the GPL is manageable to a large extent.

The real challenge is what about my code library? My intended license is generally "non-exclusive use", however, my client wanted to be able to decide whether this code could be used or not. I suppose this sort of position could be reasonable but what I object to is that we are separately negotiating pricing; where my pricing is based on doing as little work as possible by stitching together as much 3rd party code as possible.

The bottom line.... if you want exclusive rights to some code then it needs to be written for you and you have to pay for the labor. So while it is possible to scope some projects of a certain size. It becomes impossible as the assignment grows.

Should Freelance Programmers Offer Warranties?

I'm currently in the contract phase of negotiations with a potential new client. Every time I read the contract I see something new that I did not pick up on the previous time. It's only two pages and yet I find myself skimming instead of reading. (and in case you're interested, he reads my blog)

What is challenging for me right now are the number of vectors of risk against the bottom line. It's generally understood that everyone is going to disagree on the cost of a project, specially when it comes to cost per hour. And even if you cost the job instead of the labor; both parties are going to try to estimate. Of course there is the underlying incentive to work hard, shave time, and so on.

And then I saw the clause referring to Warranties.

Up until this point in my freelance career I had never offered a warranty. When the client took possession of the code they were obliged to test it and release me lock stock and barel... upon payment. But if there is an uncompensated warranty period it creates a potential burden beyond the development period and into other projects that might be ongoing.

Warranties are like an insurance policy offered by the manufacturer. There is a embedded cost in every toaster that pays for the claims. Some of that money goes to major claims and others for self insured. But what is the right amount for freelance software?

Wednesday, May 9, 2012

iPad mini? What iPad maxi or Mac-si

I'm never sure where these rumor guys get their info but one thing that I always thought "we" needed was a desktop version of an iOS device. This would make certain knowledge worker environments better and safer for the companies. But then, just yesterday, my wife asked me for a Mac or a hand-me-down Mac for our 2 year old. She seems to have mastered her iPhone3 (another hand-me-down in airplane mode with all personal information stripped)

But as I contemplate strapping a Mac Mini to the underside of her ikea play table with a monitor fixed permanently to the same table and the wires strapped down and the childproof power strip. Maybe I really need to get her a refurbished iPad? Maybe what I really need is a iPad maxi?

The idea of having to constantly repair my child's computer the way I have to work on my parents computers scares me. It's hard enough doing my day job and responding to my clients, my bosses, and now my 2yr old. 

Do Not Call

A weeks or months ago I received a call from a robot dialer and I was proud of myself when I discovered that all I had to do was say "do not call" and the robot quietly went away never to be heard from again.

Well, this week has been hell. At least twice a day I get a call from a robot dialer reporting to be from "cardholder services" offering me protection and after the sales pitch it offers me a "do not call" option.

No matter how many times I've selected the DNC option they continue to call.

One time I opted to speek to a representative. When I started to talk about the DNC he promptly hung up. I was hoping that was going to be the end of it. But no. Clearly he was on a commission and I was taking up his time. Next time I plan to get the card company that they are selling for.

Here's the thing. Unless they are getting my number from public records they have no reason to call me. Maybe once, but certainly not more than that. If they are getting my information from the credit card companies or banks that I associate with... chances are that they sold my info or are getting a cut and therefore there's probably a loophole in the T&C or Privacy terms. And since I'm not a lawyer I'll never know for certain.

An then there is possibly the mos disturbing trend. I just received a letter from American Express. My account information has been compromised. It's out there somewhere. But what is bothering me... (1) the information is out there (2) my credit protection/notification is substandard (3) government is not really protecting me but is protecting the credit card companies (4) the credit card companies probably gave my information to these guys in the first place. It's not like they are not making enough from us/me already.

The credit card companies have no incentive to actually do anything meaningful. You can fire them but then your credit scores are damaged.

I am or was on both the national and Florida state DNC lists... but I signed up again. Let's see if that has any effect.

PS: some important links: (National Do Not Call and Florida State Do Not Call)

Java: everything should be public

If not everything then at least all of the methods and classes.

I wish I new the history of this decision and more importantly what is keeping this artifact of the language in place. I suppose from a historical perspective it has not really caused any trouble. The language designers had some ideas that were rooted in commercial software and commercial software libraries. I'm remembering various commercial JDBC drivers, crypto drivers, X.25 drivers, MQ drivers. But in the modern development environment black box development is no longer the norm; so it might be time to change with the times.

Looking at Ruby, Perl, Python, even Groovy. They are all dynamic languages. They are all compiled or processed at runtime and so there is no benefit to private or protected objects. The code is there for the reading if you are so inclined. Java and C++ are compiled languages. Java does have some capability for runtime meta programming. But while historically developers purchased libraries to supplement the core JDK, they are now using Maven repositories like Ruby's Gems, Python's PyPi, and Perl's CPAN.

private and protected are now more for vanity than any "protection" that the Java's creators had envisioned.

Apache OpenOffice 3.4.0 released - Seriously?

There are a number of office suites out there and for just about every platform. The front runner(s) for the PC is MicroSoft Office; for the Mac it's Microsoft Office and iWork; for Linux it's [Apache] OpenOffice.

I have used OO since before it was Sun's OpenOffice. Even when Sun held the property there was a commercial and a community edition. (I paid for the commercial version once) I stopped following OO about the same time as I purchased my first Mac. Support was limited and the UI was consistant with the PC and Linux version but not my newly adopted Mac UI.

One of the things that makes OO possible is it's use of Java and that the JVM is available on so many platforms. But while the JVM has been getting better and faster it is still slower than native applications in the same space. For one thing the office suite functionality is ever increasing and in order to be a meme the developers have to add code and so it gets bigger and clunkier.

And unless you've been living under a rock for the last twenty to thirty years; the people who know say that we only use a small fraction of it's features. To say that we seem to be collecting or hoarding features for the on day that we might actually need it is distasteful to say the least.

Back in the day when MicroSoft Word and WordPerfect were taking the PC industry by storm... and IBM's typewriter business was slowly dissolving... IBM acquired a suite of DOS based integrated office tools that included a simple database, spreadsheet, and word processor. One nice thing was they could be integrated so that things like mailing lists and mail-merge were possible. The idea was simple. Take the 50-75% of the features that most people really use and implement them well. (see IBM writing assistant - Google Search

So as Apache releases a new version of OpenOffice I'm thinking back to the days when having a distraction free, simple, and true word processor meant something.

Beware of mobile payments

With the likes of PayAnywhere and Square are making moves in the mobile payment space one should always remain vigilant when handing your credit cards to anyone.

To start. PayAnywhere and Square; while they are a Point of Sale(POS) application implemented on a mobile device they are really a mobile merchant payment device or mPOS. The distinction is going to be important because for the time being these devices are riding the coat tails of the in-app and cardholder facing payment in order to get marketshare.

Cardholder facing payment services and apps require that the cardholder install an app on their mobile device. The vehicle for installing the software is typically a 3rd party like the Apple AppStore which acts as a vetting process for the app vendor.

Merchant facing apps, while it's a good idea that the apps are installed from a 3rd party like the appstore, it's not required.  A merchant can, in fact, develop their own application, download a development version of the application to a mobile device, and you'd never know the difference. They could be skimming your credit cards in plain sight.

With an mPOS application, like most traditional devices, you are the mercy of the merchant that they are trustworthy, however, unlike traditional POS devices where there is typically a professional service organization supporting the device. Most mobile devices are self maintained or maintained by amateurs.

The point I'm getting to here... mPOS devices and payments are not any more or less secure than traditional POS systems. Make sure you trust the merchant or the clerk with your card before you hand it over.

PS: Square does offer an interesting alternative. It's s suite for the cardholder and the merchant that lets the cardholder initiate the payment from the cardholder facing device then is loosely integrated with the merchant facing device.

Monday, May 7, 2012

The History of my Payments Experience

During a phone screen this weekend I was asked to describe all of my payments experience in a 2-3 page cover letter. I quickly wrote an outline and started filling in the blanks and submitted my first draft. This morning I printed the first draft which was now 7 pages. I have since cleaned up the spelling and much of the grammar. It's not meant to be a memoir and some descriptions are subjectively technical; and I've left out details that professionals should already know. Anyway here it is.


The following text represents the many payment systems I designed, implemented, supported, updated, managed, and contributed to in some way. It should be needless to say that I have worked on other projects in other vertical markets and other languages. I trust you will see the value that I bring to the business as well as the technology. One final note. These are my personal accomplishments. Sometimes I was part of a team and sometimes I worked alone it just depended on scheduling, resources, SME, etc.

In 1993 I started working, as a contractor, for NaBanco (acquired by First Data) as a contractor. I designed and implemented a TSR, written in assembler, for their FoxPro/DOS hospitality application. The TSR was designed to connect to each of the property's Zon terminals and download it's transactions. It would then post the transactions in the FoxPro database. Later the FoxPro app would send all of the aggregated data to the NaBanco's host via the TSR. One last thing that the TSR would do (in the days prior to the popular internet) was a trivial email service for HQ to communicate with the properties.

After this project was finished my manager recommended me to the HR department. I interviewed with and was hired to design and develop the ValueLink platform. This was a closed loop stored value system. The First client, BlockBuster Video, needed a working platform ASAP. Once the hardware was selected I went about defining the toolset. Having evaluated Informix, which was currently running on NaBanco's debit system, I decided on Oracle with PRO*C and a RAD GUI development tool from Computer Associates.

There were a number of tough challenges in designing this system. At the time I did not have any experience on Sun hardware and while I had worked on databases for years I did not know much about SQL other than the evaluation I had just performed. Additionally I had to learn multi-threading, multiplexing transactions over X.25, and everything that comes with OLTP production support. And while I had experience with the Zon terminal there was still a lot more to learn.

The next challenge was the helpdesk. I implemented the first desktop app with a toolset from CA (Computer Associates). The app lacked performance based on the PCs at BlockBuster's offices in Ft Lauderdale. I used a 2400 baud dial up modem to connect the two locations. Shortly after the project went live I hired a VB programmer to rewrite the application, however, since the application was also going to be used internally we were going to have a lot more users connected than I wanted. So I implemented a REST/SOAP-like server using Java and Java WebServer from Sun. It worked brilliantly and was later used by the IVR subsystem.

Finally, I was introduced to Perl. I used Perl to implement two major systems. The first was the card account creation in order to generate plastics and send them to manufacturing and I also used Perl for generating product performance reports (TPS reports).

In the end I was able to implement a fast, flexible, and reliable system that now transacts over 700 TPS every single day(with plenty of headroom) and hosts thousands of merchants and over 500M accounts.

This platform's most notable accounts include: BlockBuster Video, Walmart, Starbucks, and the USPS.
WildCard Systems was a client of First Data, however, during the early stages of their discovery it was decided that First Data was not going to be able to deliver. Mostly because they were going in a different direction. Since many of the people who were engaged in the conversation were friends it was easy for me move over.

At WildCard I was tasked with designing a different type of open loop stored value card system. I had implemented the first multi-wallet system that was to be used by insurance companies in order to pay or deliver money to the insured. While WildCard eventually circled back to HSA, FSA and eligibility applications they moved away from direct insurance applications.

The authorization system was implemented in two parts. The first part was a java based front end system that would connect to the association, reformat the transaction (the process of message normalization), adapt to network impedance, and then execute the particular transaction request against a set of T-SQL stored procedures and complex data configuration with rules. This front-end system was eventually certified to work with: Visa, Amex,MasterCard, Discover, First Data Resources. The overall platform replaced Visa's LAC platform.

Early on it was discovered that the state of the art PC was not going to keep up with our needs so I implemented a rudimentary replication engine in java. This application would sync 4 master-master database servers in different data-centers over a dedicated WAN connection. Eventually others in the department as well as Microsoft tried their hand at replication.

I designed a template language that could emit html, pdf, txt, and csv files. This was written in Perl and was intended to limit the roundtrips to the DB. As a domain specific language it was non-trivial to produce reports and the demand was greater than the staff could produce. Eventually all of the data had to be replicated to a farm of 5 database servers in order to produce the reports.

One of the newer projects I worked on was "WebDog". This internal-use webapp performed a number of functions supporting the operations staff. (1) it was a production migration management system, where developers wanting to submit code for production would write a ticket that had to be approved and the app managed the workflow. (2) it monitored all of the SQL Server databases. (3) It monitored all of the front end processors. (4) the most important thing it monitored was the approval ratio. When the ratio was out of spec we knew there was a release problem. (5) lastly it was responsible for deciding which SQL Server was the current master.

This killer app was conceived on a beach in Nantucket; modeled after Star Trek, deployed on FreeBSD, used MySQL, written in Perl, receiving requests via apache and mod_perl, and templated responses with Mason.

Notable clients included: AAA, Bank of America as well as the Visa Buxx brand.

After leaving WildCard I decided to work on a side project. One of the last discussions we had at WildCard had to do with TPS rates. The existing system was only working at about 25-TPS at 100% CPU Utilization (8 CPU with 16GB RAM). I posited that (1) there was a problem with our SAN. It has been reported that period SAN drives suffered from brown-outs. (2) there many examples based in truth bashing MicroSoft and SQL Server. Oracle was so much more performant. (3) T-SQL was a pig, all of the code was essentially doing hash lookups O(1) using a relational search O(lg(n)).

So I submitted two papers to SleepyCat, the makers of Berkeley DB. The papers represented payment system designs based on BDB and BDB-XML. I received two honorable mentions. I also implemented one of my designs using Java and BDB. I was able to get 1500TPS on a single core, single spindle drive.

**sidebar** by this time in my professional development I had discovered erlang. The notion that if a language like erlang can offer 9-sigma, if implemented correctly, in a phone switch environment then how different could that be in payments. 9-sigma would be a great platform/language to implement payment.

What attracted me to eDiets was a similarity to a side project I was working on, however, one of the projects I implemented for the company was a prototype erlang merchant gateway. This allowed their internal payment system to connect to different acquirer systems. The first prototype was implemented in erlang and later it was replaced with a java implementation as an ATG plug-in. The team was excited about the erlang potential, however, management steared the company toward more java.

I joined MetaVentures to support their existing CRM platform for Verifone magstripe devices. The Perl application communicated device configuration and transaction details to/from the Verifone devices. Since I had payment knowledge I was tasked to design and implement a complete end to end payment system. This included; POS, HSM, merchant gateway, and PCI compliance. The HSM and merchant gateway were implemented in erlang. The POS is a mix of languages including Perl, C, SQL and bash.

While the erlang systems were interesting to construct it was uneventful. Certifying with multiple acquirers was as simple as changing the message templates. They have been running without interruption since they were installed. There are necessary enhancements, however, none of the current team members really want to spend any time on erlang. (to be continued later)

The POS was interesting in that it needed to support a kiosk mode browser in javascript which used websockets to communicate with local webservice daemons that were connected to barcode scanners, scales, customer facing displays, pin pads, and a magstripe reader.

The gateway was certified with RBS, Global, and First Data. And is PCI compliant.
Insight Card Systems implemented a Ruby/Rails platform for account and card management. At specific times of the day it would perform account balance updates to a service provider and the service provider would send transaction details back. The system suffered from a number of problems including reporting performance and reporting accuracy. Even though I was the director of development I was optimizing the SQL and training the programmers on new ways to get more performance out of their platform, and making production operations decisions. Furthermore I implemented proper release process in order to reduce downtime and improve release quality.

As the director I had a number of other roles and assignments. I needed to hire more staff and bring development in-house. (currently outsourced). I also had to redesign a system that had 5-10x capacity with the same hardware that was currently at 100% capacity. And I had to address client expectations and customization.
I started Florida Freelance because of the economic times we live in. I had a couple of contracts that I knew I could work on. The first was a VOIP arbitrage system that generated about 1M minutes a day in call volume. This was an integrated Asterisk switch and a connected dashboard. I was tasked with redesigning the system because the original system was dropping calls, losing calls, performing badly, and could not handle the volume they needed. While this project is not a payment system is does demonstrate my ability to scale.

My second client, a company in Stockholm Sweden; hired me based on my experience. They wanted me to contribute to their existing platform and help them design new applications in areas I had detailed experience. Their platform is implemented in erlang, however, I built several interfaces in java and C as part of another plan to unify their message passing and logging. I also performed a complete PCI audit of their HQ and operations centers in Stockholm.

**sidebar** One of the interesting features of erlang is hot-code replacement. The erlang core allows developers to replace modules on the fly without interruption. However, while many erlang programmers think this is a cool feature it is actually a detriment to payment systems. Hot-plugging code causes transactions in flight to become unreproducable due to the version mismatch of sub-modules through the transaction. From an operations POV, if you are going to switch master/slave or HA configurations in order to release new versions... then you might as well restart the app. This way you are assured that the app will restart.

A recent client in Portland Oregon, asked me to perform a number of projects. The first was a one-day design and overall roadmap for their future issuing platform and to see whether I was compatible with the CEO. A few months later they asked me to perform a due diligence on a potential payment vendor's platform. And finally to design a custom issuing system for them in the EU. This was to include to EMV for chip. Shortly after beginning this part of the project I was tasked to design the same for China Union Pay.

Another client in Atlanta Georgia; has decided to rewrite their erlang gateway and HSM. While the system has been running this entire time it still suffers from the inability to enhance the application. Initially they wanted to implement the new platform in C but I convinced them that Python/tornadoweb/redis was a good choice. They recently certified with WorldPay on the first attempt. The entire project took less than a month.

There was a brief moment when I was having second thoughts about Python. The team was made up of Perl programmers, however, their tech lead was not grocking it and wanted a chance to contribute and python was going to be a lot easier for him to learn and easier still for the others to adopt.

So that's about everything payments. I look forward to fielding any questions you might have.

monolithic code repo does not mean monolithic executables

The title of this article just about says it all.

(1) There is something to be said for having multiple github repositories. This is mostly beneficial when you are a project owner and really need or want to split the code into usable verticals.

(2) but when you an a company with limited resources or SCCS challenged resources then you might consider a monolithic approach.

(3) The monolithic approach is not to be feared. The notion of namespaces is still maintained. Any lib can include any lib without having to build sub-modules and it gives global visibility to all of the source and not some module off the side.

(4) From time to time, however, devs think that a monolithic code tree means a monolithic application. This could not be farther from the truth. The release process of a monolithic application is much longer than a narrow vertical.

**The monolithic code repo does not mean a single homogenous language or environment either. Just everything in one place.

Domain Specific Languages - Why?

I'm not a fan of Domain Specific Languages (DSL) specially ones that are close enough to the real thing. For example; hibernate for java is a very good ORM layer but at some point you have to decide whether to implement stored procedures or us their DSL. The challenge is the optimizing of HQL requires hibernate in order to execute whereas if you use stored procedures you can use the bare CLI (command line interface) for your database. The later is so much easier!

Recently I've been looking at Chef and Puppet. There was a time when the key differentiator was that Ruby implemented a DSL based on ruby and Chef was all ruby. This makes Chef more attractive to me... specially since Puppet changed their direction even though they might not admit it.

Sunday, May 6, 2012

Between DropBox and Google Drive

It's too easy to compare these products features and functions. They are, in fact, fairly similar. It's also easy to find reasons to bash one over the other for things like the terms of service.

I'll admit I jumped on the Google Drive as soon as it arrived. In fact I'm running both. I'm still concerned about the quality of the google sync, the lack of documentation, and iPhone app. and let's not forget google's recent recent fickleness toward it's other properties.

In this stage of the game I have a taste of desktop unification that I really like it. The challenge is getting all those rough edges sanded down in a favorable way.

Google clearly has a bias toward android so will we get a premium iPhone app? Don't know that there is incentive. If not then DropBox needs to acquire some apps.

Chef and Puppet have their place - but not everywhere

I have been following Chef and Puppet for a while and I have yet to get a working installation. Chef is probably a better choice for a number of reasons but puppet seems to have the corporate mindshare.

The target systems I'm deploying are credit card systems and the one thing that is uber important is that the transactions be 100% predictable and reproducible. So the notion of hot-plugging the erlang way is not acceptable.

Therefore one key feature is going to be click-triggered updates so that the (a) the entire system is not brought down at once (b) humans are monitoring when the upgrades are taking place (c) and if there is a problem it can be rolled back.

One very serious drawback of Chef and Puppet is their use of ruby. Ruby is not installed by default on all platforms and as such, even with RVM, is a challenge to manage on it's own.

Finally, while there is at least one dead tree volume out there for puppet, they produce commercial and community editions. This is another set of hurdles because they offer prepackaged recipes for the commercial version and the community recipes are not reliable.

Saturday, May 5, 2012

Job Search - The Modern Way and the New Way

I hate the job search. Have I ever said that before? I'll say it again, I hate the job search.

Looking at my job search bookmark I have 29 sites that I visit from time to time and of that about 15 I check daily... whether I'm in the market or not I like to know what's going on and what technologies people are hiring for, (in many ways it's a game).

But when it comes to actually applying for a position there are so many things that go wrong.

As a candidate you have three or four things on your person that you use to apply for a job. (a) cover letter template, (b) short resume, (c) long resume, (d) references. Your expectation is that you want to upload these 4 documents, at once, in response to any posting of interest.

But no. That's not the way it works in so many cases that I'll never understand. For example JobVite and Taleo have several pages of personal and demographic information that they capture internally, then they re-prompt you for similar information for the client... and there is undoubtedly a username and password requirement.
When it comes to a job search site I have no loyalty at all. I am, however, biased toward sites with less friction. I constantly kill JobVite and Taleo applications. Their application process by no way filters the right employees. HR people should read up on game theory.

This is dumb as a bag of pet rocks! I submitted my resume. I'll wait patiently for a response. Sometimes you get a response and sometimes you do not. It's the same whether you use Taleo or send a direct email. Hiring is a long and hard process for everyone. Hiring is also #10 or lower on the manager's to do list on any given day. No amount of personal information or opt-in/out or pre-screen forms is going to improve that.

A new twist to the process is the "apply with LinkedIn" button. This would be ok, except, LinkedIn is the professional FaceBook. And as such 3rd party applications/websites that use this button are given access to your LinkedIn account. It could expose your connections to other types of communication when really you intent was to let the 3rd party access to your online resume.

If it were up to me:

The Employer would (a) create an account (b) create a job posting (c) receive daily summary emails (d) allow me to response with canned templates, (e) reporting.

The Candidate would (a) search, (b) apply with an email address, 2 attachments of any kind (c) receive emails from the employer.

If you fail to identify the best candidate because the candidate self filters ... you still "lose".

Friday, May 4, 2012

Linux and *BSD need a lot of polish

Linux and *BSD need a lot of polish if they ever hope to assault the desktop. Chrome OS is supposed to be based on Linux but that's the internals that people never see. The actual desktop is rendered from the Chrome browser and while this is akin to a dumb terminal it is still kludgy because it's all running on top of X. And while X is powerful... it's simply not modern enough to compete. (The same can be said for the various desktop managers). Frankly they all suck. The windows desktop is better than than an of the X/wm derivatives... even though Windows 7 seems to have chunky controls.

Job Search - The Golf Club Way

There was a time when membership seemed to mean something. At least the members thought so. Back in the time of Mad Men member candidates were subjectively excluded rather than objectively included. Strangely, however, there are two groups of people to which this generalization applies. The young and the old; but for very different reasons.

The old do not like change because change means doing something different. What worked yesterday is going to work today and tomorrow. They know that because it worked yesterday and the day before that. In many ways it's hard to argue that, however, I remember when the "new math" came out... and when Canada converted to the metric system. In the end life was better too. (clearly there was risk)

The young have a different view of the world. They look at the future and they see that everything looks like an uphill challenge. Salaries, responsibility, expenses, experience and so on. But when you add competition most younger talent will behave the same as the old. (think survivor)

So why the long prelude? I read a job posting for a python programmer and there were a number of things that caught my eye and which suggest that the author was young and/or new to professional software development.

Overall the job description was short. It used tokens like participate, and discuss to give the candidate a sense of inclusion. Then there was the three bullet points that said the same thing "write code".

Under the qualifications section they describe a number of pythonic tools that they use; but the one thing that caught my attention was "adheres to PEP-8". I apologize to all of the real python programmers out there... but NEVER put "adhere's to PEP-8" in a job description. If PEP-8 was really that good of an idea it would be cooked into the language instead of a lint-like tool; and even then it only catches a subset of the concerns.

The point I'm trying to make... and doing it poorly. If you are going to write a job description then find ways to include people rather than exclude them. As part of the selection committee you want to have as wide swath of people to choose from and you do not want candidates to self filter from the process... like they do in country clubs.

Thursday, May 3, 2012

In defense of dynamic languages

There are a good many truths and there are a better set of likelihoods. Given the current state of dynamic languages today they are less performance than static and functional languages, however, it is also true that dynamic languages are more productive than static and functional languages. (I am not talking about savants)
Don't optimize your code at the first stage. First make it right, then (if necessary) make it fast (while keeping it right). --erlang programming rules

It is likely that regardless of the size of your project, the size or makeup of your team, or the breakthrough that you think the project represents... that your project is going to have average results at best. The Google's, FaceBooks and Twitters of the world are extreme edge cases. As proof, look at the iPhone app store. There are over 600,000 apps and only a very small fraction of those apps have the following that Angry Birds does.

So before you go off in a corner reinventing the wheel in your favorite language consider this. WHat is going to be your return on investment? I cannot blame you for learning a new language or tool that would enhance your marketability or even just for hobby sake. But if your intent is to make some money and maybe a little independence they you really need to consider your ROI. And if you're making money then rewriting your killer app in whatever killer fast programming language is available (and popular) will make make plenty of sense.

This is why I'm hot on python and python's django, tornadoweb, flask; perl and perl's mojolicious; ruby and ruby's sinatra and rails; redis, sqlite, zeromq.

PS: While I'm not a fan of erlang, partly because of what it represents, I really like it's Programming Rules and Conventions(PRC). By comparison python's PEP-8 is amateurish. The PRC starts off with ideas like the one quoted above and giving you ideas on how best to approach the problem. This is like python's PEP-20 but again it's like signing your name with a crayon instead of a fountain pen.

another bad day for open source

One of the hallmarks of a good open source project is just how complicated it is to install, configure and maintain. Happily gitlab and the ...