Tuesday, October 15, 2019

another bad day for open source

One of the hallmarks of a good open source project is just how complicated it is to install, configure and maintain. Happily gitlab and the git experience has been exactly one of those. There is nothing like a failure to authenticate an SSL certificate in the middle of a basic upgrade.

"we" have always said that open source puts the ownership and cost on the company. That commercial software was an insurance policy with distributed ownership and support.

Saturday, October 12, 2019

k3os is not a general purpose OS

if all you are building is a lightsout k8s/k3s cluster then maybe k3os is a good choice. However if you are trying to choose one OS to rule them all then k3os is not it. That the website boasts that "Doesn’t require a package manager" you have to ask... "for what?" Don't get me wrong... if you're trying to make the OS immutable like CoreOS then maybe this is not the right way.

It's been a long day.... include a package manager or kill the project and make k3s easier and more reliable to install.

open source -- you suck

I'm having a bad day but in the calm of the moment some things are clear to me.
open source sucks
I'm just a minion programmer and I've always railed again using Microsoft tools. My reasons are many. But while I have had great success with open source there was a time when "they" did quality work. People used to take pride in their work and professional reputation meant something. Now people just throw whatever spaghetti code they can against the wall and hope it sticks. (with few exceptions)
The flutter team sucks and it's a suckier product. Too ambitious and too many volunteers trying to impress the hiring team.
So I understand when you use open source tools and there is a problem and you want some help. On the one side you either have to share your code and IP or you have to hire contractors and get NDA, legal and mgmt involved. This sort of thing can be project suicide.

On the other hand try going with Microsoft. Management feels the warm embrace of the support contract but then as a tech getting past the frontline is yet another barrier to entry. In almost every way this is a complex pain in the ass. There was a time when, at IBM or Microsoft, one could escalate. There was a sales person or a manager or a director that you knew and that you could leverage for better help. But it took patience.

Open source has none of that.

I have a partial answer for that but it's for another day.

Friday, October 11, 2019

flutter: what agile got wrong

Let's get some facts straight. Google does not apply resources in the quantities required to design, build, integrate, deploy and market so that they prop up the open source market. There is real profit motive in play and to suggest otherwise is disingenuous. Flutter is an attempt by Google to convert iPhone first to Android first by way of Android too.
the sum of the parts is not the sum but the average --Richard Bucker

Let's be real for a minute. If you have the budget then you do not care about Flutter. You'll have two teams or maybe three and you'll develop your products in or out of parity. If you do not have the budget you'll get the best team you can assemble based on what sales and marketing has on hand. If you have an iPhone sales team then that's what you're going to develop for because that's how you perceive the market.

The problem with Flutter as a tool and platform is that it's not stable. They seem to be relying on free resources to provide support. The documentation is poor and their howto videos are marketing infomercials for managers. It's not until you actually try to bolt the iPhone and Android applications together that the pain starts... if not already as in my case.


So what did agile get wrong. One thing for sure. Releasing often or too often for one. Having a customer that is so huge that you cannot hear the feedback (when the customer is the population at large). And it is possible to release too early. As evidenced by the support team suggesting users use the unstable channels to solve problems.
this was accomplished before the internet and before Google. -- Richard Bucker
In the 1980s I was hired to start work on a new payment system that was to become the anchor of the giftcard industry. At the time I was a DOS system and dbase programmer with some OS/2 internals and some OS/2 application development experience. I had already achieved my 10,00 hours so I was a rock star when it came to absorbing new tech. Within 3 months I built the giftcard system on Sun Solaris, Oracle SQL, Java, designed REST before it was REST, integrated the system into the company's mainframe operations workflow, implemented an integrated IVR, and was the only after hours support person.

The one component I keep trying to forget is the help desk software that I wrote using some tools from CA (computer associates). I'm sure it was fine for some tasks but as a general purpose multi-platform application environment it sucked. And I was too bold not to take the bullet early. The hard lesson I learned is that the sum of the parts is not the sum but the average.  Keep in mind that all of this was done without the internet and at the time Unix system admins were hoarding online and dead tree manuals.

So here's the thing... I head to learn by experimenting with the tools at hand. I had to wait for the mail in order to get the next release and that could be weeks or months and sometimes years. But when I/we received a 1.0 release we knew what it was not production ready but effectively an early release. When 1.1 arrived we knew there was something there to work with. (see the Java language history). The next place was the bookstore. It was the next best place to get good information. I had hundreds of manuals that filled in the gap. At the time the software release cycle was slow enough that there was a book business behind it. (see pragmatic programmer and O'Reilly). And when all else failed there was the library or computer science department at any college.
building on top of crap is a waste of time even if there is time to market. The cost over time will be higher than just getting it right. --Richard Bucker
Now the internet is at warp speed and developers are dumping as much crap on the internet as they possibly can as fast as they can... is it marketing? Is it just to be relevant? Over the years I've brushed against peers who want to use the latest tool or toy until I've experienced it first hand. Early on it was about taking the lead by being the lead. Then it was about not having to support junk I did not believe in and not wanting to replace it with yet another piece of junk... it just misdirects the conversation to something that is nonsense.
tcl and expect are old enough not to have a logo
I'd previously written an orchestration system for vCloud. It served a particular purpose and was designed in functional silos. Now I'm building some orchestration tools for vSphere that solves a different problem. Now I need to deploy and configure different linux guests. All the cool people are using Chef, Puppet, Ansible, Teraform. I've experienced some of those but they are so complicated and worse for non-programmers that might need to support or deploy it. So this new tool is build on bash and expect. Silly me it just works and it exposes just enough for operations staff to use and with a little personal growth to extend.

So now the real rant is over and while I do not regret saying THIS IS NOT NORMAL on a flutter support site... I regret accepting management's decision to use flutter. It's just not normal.

Monday, October 7, 2019

ultimate vs essential

I worked for a company that thought they had the ultimate employees and the ultimate product and the ultimate management... but once you pealed back the layers it was the essential elements that won the day. and while I look thru google domains for another vanity domain I see the ultimate.company is available and essential.company is not.

As I think about it again there is a meaningful distinction between essential and ultimate.

Ugh, docker-machine is abandonware

To comment; I'm disappointed and pissed that this is my reality. Docker-machine is not a rock star but it works. It does not have many providers it does not support many OS. So now what? I suppose I can make the argument that if docker-machine is meant to be the turbo pascal of tools then maybe I should just skip it. Ugh.


  • download the OVA file (As of Oct 2019 it's 3.0)
  • create a VM guest with the OVA; there are few params... one for disk and none for RAM. when creating the machine hostnames make them unique
  • the OVA, by default, is not enough storage so resize it or use the ISO
  • change the password (root/changeme)
  • 'ifconfig' to get the IP address
  • check sshd status: 'systemctl status sshd'
  • add the new machine to docker-machine: 'docker-machine create --driver none -url=tcp://192.168.86.147:2376 photon3' or use the generic `docker-machine create --driver generic --generic-ip-address=192.168.86.147 --generic-ssh-user=root photon3` (generic is better)
That was some basic system config... now comes k3s


  • iptables -A INPUT -p tcp --dport 6443 -j ACCEPT
  • gotta remember to save the iptables changes: iptables-save >/etc/systemd/scripts/ip4save
  • curl -sfL https://get.k3s.io | sh -
  • cat /var/lib/rancher/k3s/server/node-token
  • curl -sfL https://get.k3s.io | K3S_URL=https://192.168.86.185:6443 K3S_TOKEN=token_goes_here_for_agent sh -
  • kubectl get nodes
  • kind the master: kubectl cluster-info | grep 'Kubernetes master' | awk '/http/ {print $NF}'
One strange thing is that the OVA file really limits the amount or RAM and the disk is small too. There is a belief that we need many VMs with limited resources each. Well this is just not how it's supposed to be put together.

I had to expand the disk with these instructions.

As I'm writing this I've shutdown the machine and doubled the disk and ram.

check photon updates: tdnf updateinfo info
photon update tdnf update -y


Other tools:
  • tdnf install -y awk
  • add an existing kubernetes cluster to gitlab (doc)

Compares to k3s, docker swarm has as much cruft.
  • tdnf install -y git
  • get the worker token from the leader: docker swarm join-token worker
  • check the docker service: systemctl status docker
  • start docker: systemctl start docker
  • restart docker: systemctl restart docker
  • join the swarm: docker swarm join --token <token_goes_here> 192.168.86.28:2377
  • check the swarm inventory: docker node ls
  • add labels if necessary: docker node update --label-add type=queue worker1
I've said this about k3s before. It's complicated. The docker swarm setup did not need any iptable changes. It used most of the stuff that was already there. The swarm deploy and container deploy is pretty simple. It's still just simple.

Orchestration through simple tools

When I hit my 10,000 hours I realized that the basis for my success as a programmer was my shear laziness. It's that laziness that became my mantra long before the agile manifesto. I'm only putting together a list now so it will need some refinement.
  • Know when to stop. There are so many good reasons to stop. My favorite is that the longer you delay the more likely it is that the customer is going to ask for something else. In reality a 30 minute delay to color a report header for a one-time report that could be done in seconds by hand; especially when time is critical.
  • Tools that are stable, terse and simple are the most productive even if you have to create a DSL of your own. This way the minutia is separated from the work. So much easier to debug and easy to be fast.
So I get severely peeved when developers design tools like git. It's just tool damn expensive from multiple points of view.

another bad day for open source

One of the hallmarks of a good open source project is just how complicated it is to install, configure and maintain. Happily gitlab and the ...