Thursday, July 31, 2014

GoLang Message Queues

Approximately 4 years ago I designed and build a payment gateway in Python. The APIs were exposed to the POS devices as REST calls and the outbound request was either a standard TLS socket or HTTPS to the acquiring processor. The REST components were implemented using a standard event driven async webserver and the worker side was implemented as a collection of multiple worker processes each processing one connection and one transaction at a time. In the middle of the two was a transaction broker that handled the queueing as well as impedance correction between the two state machines.

The transaction was parsed, converted into a ctx (context) and stored in a redis hash as quickly as possible. A UUID was assigned to the transaction and was the only part of the transaction (the key) that was passed from function to function; and if that function needed some data then it pulled it from redis directly. Finally the transaction ctx was passed around using ZeroMQ.

Over the last few weeks I have been evaluating go chan and things have been going nicely. Channels are a nice model especially when considering the flow based programming model. The challenge, however, is that the channels are not network enabled and if you want them to work over a network then you either need a library or you need to implement them yourself.

The bigger challenge is that golang networking is where the data structure to be passed needs to be marshaled into some payload that is easily transmitted and on the other end reconstituted. And so now you get into the protbuf, msgpack, json, bson, xml argument. In a client/server pair the message needs to be processed 4 times. In a client/server/gateway system the payload is processed 6 times, and in a client/broker/server/gateway system the payload is processed 8 times.

When calculating Big-O values for some algorithm most computer scientists compute the number of comparisons based on some trivial "n". As in integers or strings or some arbitrary structure; and while "n" is the number of compares of the object when you determine the actual number of comparisons based on the number of simple objects (ints) things devolve quickly. When "n" is an object then the number of comparisons are the same regardless of the type. But when determining the actual amount of work that the string version will perform it is some multiple of "n" based on the average length of the string or average size of the struct.

Lessons learned (a) process the object as little as possible. (b) move the object as little as possible. (c) transport a proxy for the object instead.

As for GoLang there are a number of MQ options:

  • ZeroMQ/crossroads - implemented in C++ and although there are several integrations it's not as solid as the python version
  • libchan - not sure what the state of the project. It seems to be a lynchpin for docker but the code has not been updated for a while and the documentation and test cases puts a burden on the dev. The travis instance is not running. Poorly documented. "like channels" is is wrong.
  • mbxchan - a waste of time. The project is not an idiomatic workspace. the installation instructions never refer to GOPATH or GOROOT. They never specify the pwd so that you know where things are relative to GOPATH etc.
  • nats - just released. Can't say much yet. The documentation is weak and there is a case where they carried a ruby install script forward. One good point is that the docker version is close to the current rev except they do not include the dockerfile in the source. While they have a github account and both the gnatsd and nats projects are present their homepage nats.io has a separate download page. This is not very idiomatic since go get and go install are the preferred methods.
  • nsq - the docker images are old. The Dockerfile is non-existent. Shame on me but I should probably know more about this because a good docker instance is needed and of great benefit.
  • nanomsg - one of the things I liked about nanomsg is that it supports multiple topologies. Fan-Out, fan-In, etc...
  • netchan - this is the old deprecate netchan library from the golang authors. It had not been replaced yet. What's nice about this model is that it really works.  The actual payload is abstracted away from the user and the user code is strictly limited to the channels and configuring the export/import. The downside of this model is that only one worker can exist on the same port on the same machine. Fan-Out routing requires multiple ports or hosts.
There are several choices... none of which would compete with the python/ZMQ version but I think I should have removed ZMQ and gone bare metal redis.

UPDATE: que-go seems to be contender. My only hesitation is that it comes from a ruby design and while that is a positive feature if I were migrating apps... it's not necessarily a good idea for a greenfield in which case I'd prefer to try gnatsd from Apcera.

Wednesday, July 30, 2014

Brokered Message Queue Math

It should be obvious that a MQ implemented with a broker is going to experience at least a 50% reduction in TPS throughput because a single broker doubles the number of transactions.  1x from the client to the broker and 1x from the broker to the worker. 50% is the optimum so long as the overhead of the [virtual] network or broker persistence etc...

There is some nuance here and there where the two topologies perform more of the same work but in the end the broker handles the transaction callstack a little better and at some point becomes a BUS that can implement some slightly more complex message routing... see flow based programming.

Monday, July 28, 2014

Gaming Disney Vacations

Getting the best price from the Disney Vacation Company is a game and one that is designed for you to lose but I wonder how many people never even try because it's so expensive and difficult.

Just because Disney posts a price that does not mean it's the best or final lowest price. Partly you have to be in the right place at the right time and you need to have be lucky enough to get one of whatever budget they allocated. Much the way the airlines set prices there is a portion of the rooms or tickets that seem to be allocated for discount rates and everyone else pays full price or possibly some other rate.

In 2014 DVC offered a Discovery ticket that was about $39/day for a 3 or 4 day ticket. The ticket was only on sale from January to June and expired in June 2014.

Doing some ticket price searching I found some Orlando businesses who were selling one-day tickets at $39/day but I have no idea if they are real or what other limits the tickets had. There was a 3 day ticket but you were limited to the same park for all three days.

There is a magic your way ticket but that's not much of a discount; not even when linked to a resort or a package.

There is a AAA option too but the resident savings only amounted to an additional $5/day.

Resort pricing, is similarly, all over the place.

Disney needs to find a better way to sell their unique product. Some people just want to buy their tickets and get on line at the park. Having to work hard to get the right price when the price might not even be available make Disney unattainable. Money is the object!

Sunday, July 27, 2014

Danger Google Search - Serious OpenSource Weakness

Warning - a bit of a scatterbrain today

I needed to install a fresh Windows install in a VMware virtual machine along with a number of developer tools like Visual Studio. I also needed to install the latest version of Firefox and Chrome. Sadly there is a very serious threat to privacy trending out there and so long as the barrier is lower to modify the source and the cost to advertise is lower than the value... this will continue.

Google, Yahoo and Bing searches for Chrome, Firefox (UPDATE: Opera) browsers all returned Ad or otherwise sponsored links to 3rd parties who are repackaging these applications in their own installer with, in some cases, malware. One key difference is that these installers also ask for root access; unlike the default install of Firefox and Chrome which install in userspace and not root.

I also tried DuckDuckGo and while the results are formatted differently and the Google ranking in the results were higher they also provided links to 3rd parties although some are more reputable than the others.

One strongest features of all three browsers is that they install and run in userspace. If the installer asks for your password then there is something very wrong going on.

This alone makes a strong case for factory bootstraping of key tools and brands.

Back in the day Microsoft defended itself and it's business practices by making it difficult to install 3rd party browsers like Mozilla. Today everyone loses. An Apple genius recently recommended that I disable FileVault; not because it was bad but because failure created product perception problems. This sort of malware, misuse and misappropriation of free/open software is going to hurt everyone.

**Sandboxed apps is one solution and may or may not be a good remedy but thanks for trying. On the other hand there is something to be said for the ChromeOS approach too.

Monday, July 21, 2014

Apple Genius - MacBook Air

Here is the short version -

(a) MacBook Air 11" is dead.
(b) It is the second time it happened.
(c) after multiple attempts I managed to repair it so I could get some basic work done.
(d) MacBook Air 11" is dead again.

(i) I made my appointment with the Mac Genius
(ii) I arrived early but they were running 30 minutes behind as usual
(iii) after some basic attempts to perform some diagnostics the Genius gave up
(iv) they took my MBA in for repair. If the store replaced the SDD they would charge me $400 and if they sent it to the Depot for repair it would cost me $310.

That's when the Genius offered some advice. (1) while filevault2 is better than the previous version... don't use it. (2) the drive is already secure without filevault implying that OSX was going to magically respect permissions if the SDD was removed and mounted as an external drive on another machine. He clearly does not understand OSX internals and how privilege escalation works. This does not even qualify as a hacker response just common sense.

Wednesday, July 16, 2014

SaaS or On Premise?

[UPDATE] here is an interesting link to an article on "tools of the trade".  What makes it interesting is the number of services that any business might have to decide to build, buy, or host. The pricing is very interesting.

After yet another MacBook Air hardware failure I'm looking at Cloud-9 and Nitrous as my primary IDE. I'm also considering Google's many cloud offerings too.

So the question is going to be... what is the ROI and what is the risk or exposure of going full Cloud?

Hardware - while the cost of the ChromeBox hardware ranges from $170USD to $1500USD it's clear that the more expensive hardware comes with more features (touch, 4G, more memory and disk).

Software - much of the software echo system is the same. There is free, not so free, and commercial software. With the online software most of the good stuff is by subscription. The disturbing part of this is that we tend to hoard software for that split second when it might be called into service. And then it remains dormant for a good long while. Exceptions include google drive and related apps. And if I step into Cloud9 or Nitrous I'll probably have a friend for life.

But what else to I need? I recently used a backup monitoring application... I signed up for the free trial. Watched it backup my google drive. Watched the monitor. And then forgot all about it. It lay dormant for the rest of the trial... and then I cancelled.

Although the internet is practically everywhere... being offline could be a good thing. The idea that my laptop might not work everywhere means that I can return to being in the moment instead of having a disaster recovery plan for every footstep.

** I wonder if Google has a version of ChromeOS that could run as bootcamp?

** take the $1200USD that I paid for my MackBook. Add the time lost when it crashed. Subtract the cost of software and other services. Subtract the time saved by being able to have spare hardware around the house or being able to go to the corner store.

Tuesday, July 15, 2014

Lesson Confirmed

My MacBook Air crashed again last night and there isn't even a shred of hope that it will recover. The one thing I will benefit from is that all my code was committed to fossil-scm, Github or BitBucket. All of my docs are on Google's Apps and everything thing else is in iCloud. So once I get the computer repaired I'll be back in business in no time.

While I have been a green-trunk fanboy I am no longer. I happen to be lucky that everything was committed before the crash and it is better to be lucky than good. At least in this case. This means that there has to be a better way to write code that every change is committed immediately. This is also a major jolt for Nitrous.IO and C9... if they are more reliable than my desktop. Clearly when things go bad there... they go really bad.

Monday, July 14, 2014

Resetting your password with a paperclip

I remember resetting my IBM PC password by inserting a paperclip through the vent on the front of the box and shorting two pins while turning the machine on. I'm trying to remember why... (a) there was a keylock on the back of the box preventing it from being opened. (b) the password was stored in the battery backed memory which was part of the BIOS IPL (initial program load).

Sunday, July 13, 2014

Too many micro services

Micro services are all the rage. What makes this interesting is the swing from monolithic applications. This same pendulum swings in the operating system realm too; consider the microkernel.

What will make the future of micro service is interesting is a complete set of micro services may end up looking something very similar to J2EE. In which case you might just implement in the j2ee framework and skip all the bugs and infrastructure development costs.

On the other hand there are many good reasons for avoiding java in the first place.

Friday, July 11, 2014

Cross compiling go programs

It's amazing when your makefile cross compiles your go program the very first time. This is particularly nice when implementing continuous integration.

However there is one big failure that one tends to forget. Continuous integration like Travis and drone only compile on Linux machines. Which means if you have test cases that exercise your Windows programs then you need a different strategy.

Strike one for simple cross compilers.

Sunday, July 6, 2014

Core Competency and Outsourcing PaaS

Over the last several years there have been a number of NEW special purpose PaaS and SaaS services; everything from monitoring, reporting, alerting, databases, containers and so on. If a company/developer does not have a particular core competency then picking that up from one of these providers would seem valuable but at what cost and what would the cost be for a complete outsourced solution?

Monitoring - newrelic($149 per server per month), logly ($49 per month)
Email - Mailgun ($0.00050 per email after 10K)
Pager - pagerduty ($19 per month per user)
Authentication - stormpath ($19 per month per app)
Storage - Dropbox ($9.99 per month per user) Amazon ($0.03 per GB per month)  Rackspace ($0.10 per GB per month) Google ($0.02 per GB per month)
DVCS - github ($8 per month per 5 users) bitbucket ($10 per month per users)
CI/CD - Travis ($149 per month), drone ($25 per month), cloudbees ($60 per month++)
RDBMS - (I'm sure there is one but I did not look hard enough)
NoSQL-  MongoHQ ($18 per GB per month) MongoLab ($89 per month)
VPS - google, ec2, rackspace etc (varies from $10 to over $350 per node per month)
Domain reg. godaddy, moniker, network solutions, google, gandi ... varies from $7 to a lot depending on the TLD and the provider... not to mention all that upselling)
MQ - IronMQ ($29 per month), Amazon ($0.50 per million messages), Google (looks like it's included)
Bare Metal - Peer1 ($150 per CPU per month)
Business - Google ($5 per user per month), Yahoo ($1 per user per month; email only)
Bug Tracking - Bugzilla (Free but needs hosting), Github (free), Github (free)
Wiki - Github (free), BitBucket (free)
SPA hosting - Github (free), BitBucket (free)
VOIP/SIP - asterisk, freeswitch, bluebox and so many others
Remote Desktop - copilot, teamviewer, RDP, VNC (some free and some not)
Web IDE - Nitrous ($19 per user per month), Cloud-9 ($19 per user per month)

One thing that is immediately obvious is that the a la cart method is very expensive. Using the Google or Amazon strategy means implementing your application in a vendor lock-in way but it also means that your costs are lower, you'll scale easier, and your operational readiness (six sigma) will be much higher from the start. One downside is that CI/CD is a lot harder on the Google and Amazon app platforms.

another bad day for open source

One of the hallmarks of a good open source project is just how complicated it is to install, configure and maintain. Happily gitlab and the ...