Skip to main content

Posts

Showing posts from December, 2014

Lots of DVCS angst

Recent articles covering GHTorrent,github and AWS keys made me cringe. As much as I like bitbucket, github, launchpad and others I'm scared that a quick slip could give away the keys to the kingdom. Even if it's an accident.

I think whatever the circumstance your code have to be In-house and private. 
Fossil Fossil+docker. https://github.com/kassanmoor/fossildocker dockerfile https://www.sqlite.org/debug1/info/a7fc0c5f6e822bb3ad497b43231c6c0d0f70403f
Git Gogs  http://gogs.io
Fossil is great because backups are as simple as copying a single SQLite file. It also includes a wiki, issues manager, CLI and web GUI. The binary is both client and server; and available for major operating systems. 
Gogs is git with a web wrapper. However Git has an advantage with many proper client apps. Tower, github, tortoisegit, sourceit, and many more. 
My first choice is fossil as it feels the most sensible. 
Link https://jordan-wright.github.io/blog/2014/12/30/why-deleting-sensitive-information-from…

Interesting Paul Graham Quotes

Paul graham is not my favorite and YCombinator even less so. But he has written and made comments that have caught my attention.
We knew that everyone else was writing their software in C++ or Perl. But we also knew that that didn't mean anything. If you chose technology that way, you'd be running Windows. This is actually very telling. Another article I read about computer chess programs and how they are designed to maximize the number of future moves... this applies to selecting the right languages and frameworks.
When you choose technology, you have to ignore what other people are doing, and consider only what will work the best. This goes without saying. If you import or incorporate too much 3rd party code you get their opinions and cruft and their design concerns and not your own.
For one thing, it was obvious that rapid development would be important in this market. This is too general and obvious. It's important to all businesses.

References:
http://www.paulgraham.co…

from exaggeration to resumes

I've told you a million times to stop exaggerating I do not remember who said it or when I first heard it but there it is.
You drive on a parkway and park on a driveway another quote that fits. But the ultimate example was an acquaintance who posted a picture of a computer kit and with a statement to the effect "my friend built a computer". In actuality she had assembled the computer not built it.

As for resumes it's important to be clear what you actually designed, implemented, coded, tested etc... A resume full of exaggeration will be discarded by the most junior reviewer.

PS: I also wonder about directors and producers.

Kindle Paperwhite Trouble

I created a collection on my iPad's Kindle application.  I also populated the collection with 11 books and periodicals. But wouldn't you know it... it would not sync with my paperwhite.

The first problem was that I had set the parental controls at some point and... [a] I had meant to prevent anyone from stealing my kindle and downloading everything they wanted so I protected the store. [b] I had also protected the cloud because that's where all the personal content is backed up. [c] I had forgotten the passcode. CRAP!

[a] that's easy enough. The parental controls are granular enough to separate the store from the cloud. But since I had included [b] the cloud I was not able to navigate to the cloud view even though all the help and links hinted that I was supposed to be able to. But of course it was locked.

So locked meant I was going to get misleading text from Amazon and that the help was going to be no help at all. The giveaway should have been [i] the fact that the …

iPhone disable AutoStart iTunes and iPhoto

I like iTunes and iPhoto. I think I'd prefer that Google did a better job and they may in the future but then Apple might figure out that my MacBook Air does not and will not have enough storage and my wife's 1TB of storage is no longer enough. Using external drive is simply not efficient and it's hard for the novice or uninterested.

Even though I'm a tech I have no interest in splitting my photo albums and using external USB drives to hold my pictures. It's just unsatisfying. But when you consider the iCloud option ... it's just incomplete and very expensive. So that's a non-starter too.

Frankly once I've created an iPhoto album it would be fine to store the original images in a fraction of their original resolution and then move on. I'd rather Picasa did the whole facial recognition photo/time-lapse thing instead.

So my current challenge is that whenever I plug my USB into my computer in order to charge my iPhone or iPad both iTunes and iPhoto pop…

How not to conduct an interview

I came across this interview question site and it drove me to fits. Anyone who would use this site, it's questions, develop or use interview questions like it is just stupid and lazy. The site represents 4 years of college and two years of ACM competitions.  At the time they were fun but they are meaningless in the context of the work I have performed over 35 years.

In my professional programming career I've only had to implement or model against a standard deviation once. And never once had I specifically answered an interview question that was remotely related to any higher level math. Let's face it... these questions are all derivatives from university studies and while that was 25 years ago I wouldn't remember any of these the day after graduation.

The frustration is that while I was in college we bumped chests and high-fived with the idea that university was teaching us to research and think critically. The specifics of maths, physical sciences or humanities were …

boot2docker in fishshell

I have been an on and off again user of fish shell. There are a number of reasons to be wary of a new shell let along one that you downloaded from the net. But if you are like me then from time to time it's ok to take a risk. And while there are some open source projects that promote their security and actually aren't this one is just plain easy going.

One of the things that caused me to go back to bash was poor integration with the tools that I used that were tightly bound to bash or zsh. For a time I used GVM (golang version manager). It's scripts are complicated, deep, and managed the environment assuming it was bash. So when I started to get into my Go programming this became a serious issue. Since then I no longer care about GVM and I'd rather have a better shell experience. bash is just fine but fish can be fun.

So while I'm also getting deep into docker and boot2docker there was a challenge. boot2docker needed to set the environment so that the host'd do…

ngrok in production and other uses

I recently came across ngrok and while the demo that I watched was recommending it development environment I think it might have a few more use cases.

The example that was given: You have implemented some application or feature that you want to share but the user is remote, outside the firewall, or asking the user to update his hosts file make the process painful.

Enter ngrok.

Register with ngrok. Then download their client application. Launch your application in localhost mode. Then configure and launch the ngrok client. The ngrok client will open a long poll connection to the ngrok server. You give the user a target URL to enter into his browser... and voila.

The user's browser connects to the ngrok server, which forwards the message request to the ngrok client, which then opens a connection to the configured client application. The experience is complete.

What makes this model desirable is several fold [a] all applications would be configured as localhost support the just-one e…

Asterisk Reading List

This post is meant to be a list of references for deploying a feature complete Asterisk system that can compete with SwitchVox and provide a proper call dashboard with screen pop capability.

Reloading an asterisk server - hoping to reload without dropping the the calls in progress.

Asterisk CLI

FreePBX CLI - reloading with dropping calls

Docker Asterisk Configuration, more code

FreePBX download

There is very little code written in or for Go that includes interacting with an Asterisk PBX. Any interaction will require CLI simulation, REST type calls or possibly something else.

NodeJS frameworks

In summary I'm not a fan of the nodejs platform or the web frameworks that have evolved from it. on the other hand I might not have a choice if someone else is throwing the switch so I guess I should know a little something about it/them.

I just watched several videos covering meteor, sails, deployd, and I have written a few express and hapi apps; this article is a good summary. Total is another candidate although new and not much of a following. When it comes down to it anything beyond yellow world required a lot of work and when you get to the edge cases it's even more complicated.

The only silver lining is that meteor offers a comprehensive book and video collection called discover meteor. The next step is going to be trying to identify how all of this might integrate into a docker or rocket strategy such that real work can be accomplished at scale.

PS: I ignored a large number of frameworks.  In fact any framework that claimed to be the NG or next generation of framework w…

Rest APIs in Go

There are a number of 3rd party libs like gorilla and rip.  They offer a regex experience in the REST encapsulation of the handler. But like the other ins I've been critical of this too can be performed in your code. The single strongest justification is that the filters are not difficult to add and you own the flexibility where you want it. If you want to add a POST() or GET() middleware handler you can. Simple as that.

UPDATE - bone is a very nice little multiplexer. It basically adds Post() to the handle interface. While gorilla has a mux too, bone is considerably smaller. It's so small that to create a package instead of a gist is a shame.

ectd 2.0 RC1

I like that the CoreOS team has release etcd rc1 and container format and I like it even more that they included a rocket version too. I cannot wait for rocket to be installed by default next to Docker. I wonder if a bridge would be useful or hazardous?

something to try in erlang

I have been working on a flow based programming implementation that would be implemented in go. Part of what would make it successful is that all of the nodes would exist in one package... even main. In this way all nodes are global.

While I have yet to figure out if there are practical limits to the number of public objects in a package I'm also wondering whether or not modules are necessary in erlang. As the major downfall of hot plugging is that the version is by module only and if you have to update multiple modules the transaction repro is sacrificed. But if the application were implemented in a single module as I suggested the FBP application then would hot plugging be more reliable?

... on xen

A few years ago I was introduced to the erlangonxen project. After the initial impression they went back to the drawing board and produced a new and impressive demo. The second demo showed tens of thousands of application instances being spun up and torn down.

In the interim I introduced myself to docker and was able to produce better numbers. EOX was deploying a new environment every 300ms and I was able to deploy a docker container in under 40ms.

As I started to dig into EOX I determined that there was no real OS underneath it and that there were severe limits to the filesystem. Something like 500 files. I still think there are some limits to the design but they may not be as important as I first thought.  First of all it is significant that the application can be launched on demand and it can be readonly preventing any number of potential security problems. One can also address the multi-tenant issue by just building the front-end on demand per tenant.

Now add elixir on xen and Mir…

Testing MicroServices in Go

Testing MicroServices in Go is no different that testing micro services in any language. Go does not offer any additional challenges. The hardest part is simply the testing. MicroServices are no different than the systems in a micro kernel. And as such they are hard to "prove". Once you get past the unit testing you can use the same test framework for integration testing (which is the closest thing to test the service as you can get). 
The stdlib for testing is sufficient. Some of the 3rd party libs make some parts easier and in some cases they can coexist. But for all the effort sticking to the stdlib will avoid any conflict as it works just fine. There are bigger challenges depending on whether you've decided to mock the test attributes or if you're going to perform actual transactions.
Docker, gitreceive, bash are easily integrated so that you do not need the complexity of Travis, Drone, TeamCity, Jenkins or any of those other CIs. The hardest part is the radiato…

Command line parsing in Go

There are a number of CLI libraries out there. Of course there is the stdlib, called flag, and then there are plenty of 3rd party versions. Cobra is an interesting manifestation because they have taken the controller approach as demonstrated in this article. The claim that the author makes is at least in two parts. [1] that the CLI behaves in a way consistent with the go tools. [2] supports nested commands.

The challenge for the CLI is that there are so many opinions as to what is easy or best. The go tool and the docker CLI look a lot alike. And while the go authors have indicated that it is idiomatic to do as much as you can with the stdlib Cobra could be an exception.

I decided to review the code in order to get a handle on the author and the implementation. If the author simply wrapped the flag package I would probably ignore the project and implement my own variation on the controller. The code seems nicely documented, however, one commonly argues that better function and attribu…

Vendoring and dependencies

I'm not so sure that vendoring is the way to go. APIs are a contract between the parties and should be immutable once they achieve production status. The APIs should be layered away from the underlying processes or business. 
While not everyone believes in this contract style you could or should impose that layer yourself. In this way your application will not experience a total collapse if there is a 3rd party change. 
Vendoring can be expensive during the build step. It's expensive on the VCS system. Requires considerable effort to update. And in the end it'll come down to your testing anyway.

Response to generics in golang

For the time being it seems that the golang authors are set against generics. Phew! And while there have not been any new arguments only squeaky wheels... If the golang authors ever implement generics I'll slip back to erlang. 
"Yes it's annoying to rewrite boilerplate"
If you believe that statement as the argument for generics then turn in you diploma and clean out your desk. First of all how much code could that be? Second, that's what you get paid for? Added complexity and time in the compiler for what? Just go watch the intro ocaml video.

Choosing the right host for your docker containers

First of all there are a lot of choices, however, they come in two varieties. The first is and probably the most common is the general purpose Linux distribution. So long as you have a modern kernel with the requisite kernel modules Docker is likely to work. (FreeBSD's compatibility mode is an API layer and not a kernel virtualization). The second type of host is the special purpose or dedicated host.

To put it plainly the general purpose Linux distro is best utilized as development environment so long as all of the other components are present depending on your production environment. ie if you have a dependency on etcd or geard etc...

On the other hand the dedicated distros work equally as well as a production environment and development environment (without the desktop). Here is a list of the hosts.
CoreOS - supports both Docker and RocketProjectAtomic (Fedora, CentOS, RedHat flavors)Ubuntu 'Snappy Core' - I'm not sure of that's a project name or the brand.Boot2d…

"A Formula for the Number of Days in Each Month"

Here is the function in go:
package main import ( "fmt" "math" ) // ref: http://cmcenroe.me/2014/12/05/days-in-month-formula.html func f(x float64) float64 { return 28 + math.Mod((x+math.Floor(x/8)), 2) + math.Mod(2, x) + 2*math.Floor(1/x) } func main() { fmt.Println("Hello, playground") for i := 1; i < 13; i++ { fmt.Println(i, f(float64(i))) } }

The code is here if you want to run it now.

Is there a plan for #Plan9

What I know is that I don't know? For the last while I have noticed that the Golang team has been building a version of Go for both ARM and plan9. While the ARM version makes perfect sense as there is a group that believes in Go on android. But Go on plan9?
Without making any value statements about plan9 it has not had a release since 2002. I posted on twitter and I emailed Rob Pike; and the response did not seem very enthusiastic. Rob directed me to the mailing lists and twitter indicated it was homegrown.
My take away from some comments made in response to a Tannenbaum letter in 1992 is that Plan9 might be a perfect OS to absorb into your stack if you have the cash to staff. It might even stave off the need to head into container land (I need more evidence to prove this but I am thinking about Ford's decision to take on QNX instead of Windows for their next generation)

First look at CoreOS Rocket

I just completed an internal blog post as part of a Docker/CoreOS introduction and at the same time I decided to read the available Rocket documentation. While I have not executed the examples one thing that caught my eye was that [a] the target application was a fully baked statically linked Go app. [b] while the manifest file indicated it was a Linux container it said nothing about the actual OS.

This is actually an interesting idea since [a] most containers are just running a user app or some single purpose service. [b] that service probably does not need anything more than just itself. So why on earth would I need all the cruft associated with selecting an OS and version.

When I consider the workflow it seems that it's missing some tools that the CoreOS team seems to have deferred to Docker and/or the host. It's not even clear whether or not the base CoreOS is base enough to build the Rocket container instances.  And as such when are they going to establish that Trust?

Docker vs Rocket

The Docker - Rocket discussion is about to get into full swing but in fact the conversation started a long time ago and without getting into a debate on the subjective qualities of each project let's discuss the facts.

Docker is currently the leader in the Linux container market. With their acquisition of the fig project (now called machine), the implementation of the libcontainer (departing from the true LCX container), swarm for clustering containers. While the registry is open source Docker offers several SaaS solutions from free to enterprise.

CoreOS started as a bare metal OS that was meant to be immutable, much like ChromeOS, consume Docker containers as it's distributable unit of work. The CoreOS team developed etcd and fleetd to support Docker clusters. Multiple projects like Deis and Kubernetes were implemented and took advantage of the structure of CoreOS. Frankly when you have a heavy host OS & containers then you have many multiples of systems that need maintai…

the golang hotspot

Google's Go has a great many killer features.  One in particular was called out by Derek Collison, founder of Apcera, at a box conference this year. He said something to the effect that Go is excellent because deploying an application is as simple as copying the executable [all the dependencies are baked in].

There is nothing untrue about the statement, however, when you're deploying webapps where there many hundreds or thousands of static artifacts like javascript, css, html and template versions of the above then packaging all of these artifacts using packages like go-binddata is unusable.

There's not much for me to say in order to justify this position as you'll have to experience it for yourself but converting these files into source code causes an explosion in file size.