Skip to main content

Posts

Showing posts from July, 2014

GoLang Message Queues

Approximately 4 years ago I designed and build a payment gateway in Python. The APIs were exposed to the POS devices as REST calls and the outbound request was either a standard TLS socket or HTTPS to the acquiring processor. The REST components were implemented using a standard event driven async webserver and the worker side was implemented as a collection of multiple worker processes each processing one connection and one transaction at a time. In the middle of the two was a transaction broker that handled the queueing as well as impedance correction between the two state machines.

The transaction was parsed, converted into a ctx (context) and stored in a redis hash as quickly as possible. A UUID was assigned to the transaction and was the only part of the transaction (the key) that was passed from function to function; and if that function needed some data then it pulled it from redis directly. Finally the transaction ctx was passed around using ZeroMQ.

Over the last few weeks I h…

Brokered Message Queue Math

It should be obvious that a MQ implemented with a broker is going to experience at least a 50% reduction in TPS throughput because a single broker doubles the number of transactions.  1x from the client to the broker and 1x from the broker to the worker. 50% is the optimum so long as the overhead of the [virtual] network or broker persistence etc...

There is some nuance here and there where the two topologies perform more of the same work but in the end the broker handles the transaction callstack a little better and at some point becomes a BUS that can implement some slightly more complex message routing... see flow based programming.

Gaming Disney Vacations

Getting the best price from the Disney Vacation Company is a game and one that is designed for you to lose but I wonder how many people never even try because it's so expensive and difficult.

Just because Disney posts a price that does not mean it's the best or final lowest price. Partly you have to be in the right place at the right time and you need to have be lucky enough to get one of whatever budget they allocated. Much the way the airlines set prices there is a portion of the rooms or tickets that seem to be allocated for discount rates and everyone else pays full price or possibly some other rate.

In 2014 DVC offered a Discovery ticket that was about $39/day for a 3 or 4 day ticket. The ticket was only on sale from January to June and expired in June 2014.

Doing some ticket price searching I found some Orlando businesses who were selling one-day tickets at $39/day but I have no idea if they are real or what other limits the tickets had. There was a 3 day ticket but you …

Danger Google Search - Serious OpenSource Weakness

Warning - a bit of a scatterbrain today

I needed to install a fresh Windows install in a VMware virtual machine along with a number of developer tools like Visual Studio. I also needed to install the latest version of Firefox and Chrome. Sadly there is a very serious threat to privacy trending out there and so long as the barrier is lower to modify the source and the cost to advertise is lower than the value... this will continue.

Google, Yahoo and Bing searches for Chrome, Firefox (UPDATE: Opera) browsers all returned Ad or otherwise sponsored links to 3rd parties who are repackaging these applications in their own installer with, in some cases, malware. One key difference is that these installers also ask for root access; unlike the default install of Firefox and Chrome which install in userspace and not root.

I also tried DuckDuckGo and while the results are formatted differently and the Google ranking in the results were higher they also provided links to 3rd parties although some…

Apple Genius - MacBook Air

Here is the short version -

(a) MacBook Air 11" is dead.
(b) It is the second time it happened.
(c) after multiple attempts I managed to repair it so I could get some basic work done.
(d) MacBook Air 11" is dead again.

(i) I made my appointment with the Mac Genius
(ii) I arrived early but they were running 30 minutes behind as usual
(iii) after some basic attempts to perform some diagnostics the Genius gave up
(iv) they took my MBA in for repair. If the store replaced the SDD they would charge me $400 and if they sent it to the Depot for repair it would cost me $310.

That's when the Genius offered some advice. (1) while filevault2 is better than the previous version... don't use it. (2) the drive is already secure without filevault implying that OSX was going to magically respect permissions if the SDD was removed and mounted as an external drive on another machine. He clearly does not understand OSX internals and how privilege escalation works. This does not even qu…

SaaS or On Premise?

[UPDATE] here is an interesting link to an article on "tools of the trade".  What makes it interesting is the number of services that any business might have to decide to build, buy, or host. The pricing is very interesting.

After yet another MacBook Air hardware failure I'm looking at Cloud-9 and Nitrous as my primary IDE. I'm also considering Google's many cloud offerings too.

So the question is going to be... what is the ROI and what is the risk or exposure of going full Cloud?

Hardware - while the cost of the ChromeBox hardware ranges from $170USD to $1500USD it's clear that the more expensive hardware comes with more features (touch, 4G, more memory and disk).

Software - much of the software echo system is the same. There is free, not so free, and commercial software. With the online software most of the good stuff is by subscription. The disturbing part of this is that we tend to hoard software for that split second when it might be called into service.…

Lesson Confirmed

My MacBook Air crashed again last night and there isn't even a shred of hope that it will recover. The one thing I will benefit from is that all my code was committed to fossil-scm, Github or BitBucket. All of my docs are on Google's Apps and everything thing else is in iCloud. So once I get the computer repaired I'll be back in business in no time.

While I have been a green-trunk fanboy I am no longer. I happen to be lucky that everything was committed before the crash and it is better to be lucky than good. At least in this case. This means that there has to be a better way to write code that every change is committed immediately. This is also a major jolt for Nitrous.IO and C9... if they are more reliable than my desktop. Clearly when things go bad there... they go really bad.

Resetting your password with a paperclip

I remember resetting my IBM PC password by inserting a paperclip through the vent on the front of the box and shorting two pins while turning the machine on. I'm trying to remember why... (a) there was a keylock on the back of the box preventing it from being opened. (b) the password was stored in the battery backed memory which was part of the BIOS IPL (initial program load).

Too many micro services

Micro services are all the rage. What makes this interesting is the swing from monolithic applications. This same pendulum swings in the operating system realm too; consider the microkernel.
What will make the future of micro service is interesting is a complete set of micro services may end up looking something very similar to J2EE. In which case you might just implement in the j2ee framework and skip all the bugs and infrastructure development costs.
On the other hand there are many good reasons for avoiding java in the first place.

Cross compiling go programs

It's amazing when your makefile cross compiles your go program the very first time. This is particularly nice when implementing continuous integration.
However there is one big failure that one tends to forget. Continuous integration like Travis and drone only compile on Linux machines. Which means if you have test cases that exercise your Windows programs then you need a different strategy.
Strike one for simple cross compilers.

Core Competency and Outsourcing PaaS

Over the last several years there have been a number of NEW special purpose PaaS and SaaS services; everything from monitoring, reporting, alerting, databases, containers and so on. If a company/developer does not have a particular core competency then picking that up from one of these providers would seem valuable but at what cost and what would the cost be for a complete outsourced solution?

Monitoring - newrelic($149 per server per month), logly ($49 per month)
Email - Mailgun ($0.00050 per email after 10K)
Pager - pagerduty ($19 per month per user)
Authentication - stormpath ($19 per month per app)
Storage - Dropbox ($9.99 per month per user) Amazon ($0.03 per GB per month)  Rackspace ($0.10 per GB per month) Google ($0.02 per GB per month)
DVCS - github ($8 per month per 5 users) bitbucket ($10 per month per users)
CI/CD - Travis ($149 per month), drone ($25 per month), cloudbees ($60 per month++)
RDBMS - (I'm sure there is one but I did not look hard enough)
NoSQL-  MongoHQ ($18 per …