Skip to main content

Posts

Showing posts from August, 2014

parallel execution - flowstaller

The orchestration tool I'm working on got a shot today.  Go has some interesting properties. For example there is something called a Zero value than can be applied to structures as well as first class attributes.
an empty string is the zero value for a string
a 0 (zero) is the zero value for an integer
False is a zero value for a boolean  A zero value for a struct is when all of the member attributes have their zero value.

When decoding or Unmarshaling it's impossible to know whether the string you decoded yielded an object instance or a zero value.
type SomeStructType struct {
    MyValue string
} In order to Unmarshal something into this structure I need a wrapper structure:
type SomeStructWrapper struct {
    Something SomeStructType `json:"SomeStructType"`} Unmarshaling a string like: { "SomeStructType":{"MyValue":"Hello World"}} Only works when you're Unmarshaling into a wrapper instance. s := &SomeStructWrapper{}
err := json.Unmarsh…

Silence

WARNING - this is potentially a long story and could, in fact, make a good TED talk.

I started programming, for entertainment, in 1982 as a Junior or Senior in High School. I got my first paid gig writing some software for my father's consulting business. About 10-12 years later I was making changes to the OS/2 kernel and GUI internals.

Shortly after that I started work for a payments processor; and while my work leading up to payments was technical and challenging this new job was 9-5 and 24x7x365; and along with a paycheck came a beeper. (if you were lucky you got the full alpha-text pager and not just the numeric; some even had two-way text capability; later came the first BlackBerrys).

When these pagers went off you had to find a pay-phone and call in. I recall being less than 4 miles from the office when my pager went off and pulling over to a gas station... and sitting on the phone for 45 minutes talking the operations guy off the ledge.

The reason I'm writing this is becaus…

Docker - Data-Only Containers

I recently deployed shykesdevbox setup and while he did not provide a docker build or docker run command I was able to get things running:

docker run -it -v /media/state/shared/:/var/shared/ rbucker/devbox /bin/bash Sadly, in my case, I mounted the volume from my host system; but in the dockervolumes article there is a recommendation that "we" use data-only containers. On the surface that makes plenty of sense until I read this comment: Volumes persist until no containers use them This really creates a potential problem if you hope that you data is going to survive. In my case I might want to have a redis and/or postgres microservice. If I used a data-only container and either redis or pg crashed then the data-only container and it's data will be gone. The only way that this might be resolved is if the data-only container then used a host volume. That way (a) you get the benefit of linked volumes; which might someday be networked (b) some sort of persistence.

VMware announced docker support

VMware is joining a long line of commercial brands that have decided to get into the Docker LOB(line of business). It's not at all curious but to be expected as there are a number of other brands that are already there.
The most curious brand is Microsoft. On the one hand it makes perfect sense as I believe Azure is supporting some elements of linux but is still a Windows platform. But as VMware enters what exactly do they have in mind? Docker requires a linux kernel and VMware, while it uses Linux for some of it's offerings, is not dedicated to the container. It is interesting to consider VMware's vCloud offering. It's a complete dashboard for managing the virtual hardware and it's not a stretch to extend the metaphor to include containers. (I've installed all manner of OS on a vCloud system (constructed orchestration services around vCloud) and containers will not take much to implement). VMware will have to choose the right UX and workflows so that the vApp …

Flow Based Programming in Go

Go is not the ideal language for implementing a Flow Based Programming environment but it works.

One challenge was that the network graph of nodes started off as statically compiled and linked during the POC stage.  Later I was able to leverage the init() function in order to link the types in what I called the registration process. In the package's init function I would create an instance of a wrapper type that would make it easy to decode the JSON config. This meant that when the dynamic network was applied and the mini-microservices were started that the messages would flow as expected.

**I have not gotten to the point where I know how multiple instances of a particular node is handled... in sort of a fan-out model. But I'm sure it's in the book.

**one other challenge is that while the goroutines and channels are well suited... they are limited to the current process. The networked version of the channel (netchan) has been deprecated and no suitable replacement has been na…

Docker is punk rock

... as in he urban dictionary definition. Docker is experiencing a velocity of attention that I have not seen since the peak of the dot-com days. There has been no single technology that has receive a disproportionate mindshare when the scope is as narrow as the Linux Kernel version 3.10+. Specially with the number of Windows servers in production and the number of entrenched virtualization brands.
The container solution would seem to be a good one. It's particularly interesting when you consider microservices and even mini-microservices when you embed flow-based programming services inside a microservice.
If you have an interest in flow based programming please drop me a note. I've constructed several POCs in go that use channels as the static type pipe and while the Morrison model is what I hope to achieve I'm just trying to get something into the wild. (I also constructed one in NodeJS but once I got into deep promises I gave it up.

DSL in review

UPDATE:  My initial notes have been erased by the ether. It seems that my iOS blogger app ate my homework. So here is the summary.

Last week I was asked the question "why is flowstaller implemented as a DSL or DSL-like instead of implementing the tasks as first class functions" in the language of choice? At the time I was exhausted and not able to give put together a strong argument, however, after reading an article today I found the voice I was looking for.

The article I read made the argument that the author had built a DSL and refined it until it was no longer capable of completing the task for which it was designed. I also read Stack Overflow question/answer... the question was whether or not a physics researcher should implement the research questions in a DSL or in a first class language?

While the first problem is a challenge it's not impossible. Usually it means one of two things.  (a) the DSL was not thought out well enough in advance to account for the major c…

Rex is not Rexx more like Chef.

http://www.rexify.org/

It’s obviously a DSL implemented in perl with bits that work in OSX, Windows and most heavily developed in the *Nix world. Since perl is a dynamic language like python and ruby; Rex will suffer the same runtime dependency warts that Chef, Puppet, Ansible and SaltStack do or will.
When will flowstaller achieves it's tipping point?

Smartphone reduction

Making the decision to remove the apps that serve to distract rather than enhance has been a success. The only apps I have on my phone are the ones that are meant to interrupt whatever I'm doing... phone, SMS, email.  Everything else has been moved to my iPad. The apps on my iPad are the ones where I might want to do some casual reading or searching such as twitter, Facebook, LinkedIn, Google+ and a few others. And if there is a link that needs some concentrated reading I always forward the links to my email or cloud bookmarks for reading or research later when I can dedicate focused attention.

What a great decision! I feel better already!

Re-factoring my smart tools

UPDATE: The question I have... which queue do I put twitter? fast or medium? Clearly there is nothing in any tweet that is going to change my day's priorities but it could definitely derail it.
Looking at the number of apps that are running on my iPhone I realize that there's a problem. It's not that there are too many applications or that I don't use them regularly. I simply came to the realization that I needed a priority queue. 
My iPhone was to be used for communications that needed to be real time or near instantaneous. This would be text messages phone calls emails.
Since my iPad mini is slightly bulkier than my iPhone and rarely take them with me everywhere I go. It made more sense that this be my medium queue. Apps like Facebook, LinkedIn, Google plus, and many others.
Finally my slow queue. This would be my personal laptop or my employers laptop. On my laptop I will install many of the applications that have native apps. For the rest I simply have bookmarks on my …

Chef, Puppet, Ansible and SaltStack

I've been writing batch installer for years. Different batch sh, bash, zsh, bat, ant, nant; and dynamic languages like perl, python and ruby. But at the end of the day they are all the same. It's some engine that runs some code that produces some expected outcome. And if you're lucky and the tool provides something that looks like a DSL such that the scripts are easy and fast to construct then you've hit the jackpot.

All of this comes to mind as I'm building my own installer DSL. No matter how I parse my work product it looks exactly like the others. It's just a script being executed by an engine. The script can be embedded into a "solo" executable, pulled from a repository, pushed to an agent, tunneled through SSH and a REST API. The only thing that makes my tool different is that (a) it's almost 100% cross platform  [Linux, Darwin, Windows] (b) everything that is needed is statically linked (c) automatic self-update (d) scheduling and (e) orches…

DevOps is not a silver bullet

"You keep using that word. I do not think it means what you think it means." --Inigo Montoya (Princess Bride) I just read a headline on LinkedIn.  "Why DevOps is Key to Software Success". My initial impression from the title and many like it is that all of the programmers and operations staff need to be cross trained and merge into this new classification leaving the uni-skilled programmer or operator in the dust.

Point in fact that Wikipedia defines DevOps as a method and not a role. On the one hand I'm relieved that I was wrong but disheartened by a recent conversation where a manager referred to a DevOps as a person in a role. What I realize is that I have been a practicing the DevOps method for over 30 years. In that time every program that I put into production needed the design, care and feeding stipulated by the DevOps method.

DevOps is nothing new. It just has a new name.

http benchmarks all very interesting but who really needs it

These benchmarks are very interesting to me. When I'm designing systems I'm always thinking about huge numbers and scale. But the reality is... who needs it? How many projects really need 800k connections? And while the article is nice and well written I'm going to bookmark it and probably forget about it until I clean my bookmarks. It would be great to get to that scale but I think the number of businesses that get there are in the extreme minority. But thanks for posting anyway... right now time to market and reliability are more important.