Skip to main content


Showing posts from September, 2011

Agile is still dead and has been since 1991

[updated 2011.09.30] yet another response to Agile is good.
When you have so much of you career invested in something like Agile, XP etc... it can be hard to see the forest for the trees. I had a consulting job in The Haag many years ago. IBM was the incumbent contractor at the customer site (a bank) but after 5 years on the job they had not written a single line of functioning code. In the office there were two teams of software people... both behind closed doors. The first team was the Data team and the second team was Functional. They rarely spoke and they never shared information. I was there for a week, introduced the client to OO and we had a functioning prototype. Smart people do smart things, You cannot make an underachiever exceptional by using Agile. Either they get "it" or they don't.
I just commented on a blog. I'm sure there is some validity to his post beyond observing that Agile Scrum is broken. It certainly is not what it was originally intended but for…

Still managed a Mojo upgrade

Even in my half inebriated state I managed to upgrade my Mojolicious development installation from version 1.98 to 1.99. I cannot wait to restart my latest perl project... with only 2 parts written so far.
$ sudo -s 'cpanm Mojolicious'
CPAN/cpanminus might be the second killer app. (gems and peaks simply do not measure up)

What will be the effect of internet memory?

I thought I would have a deep philosophical viewpoint but after walking the dog and considering the obvious it's not that difficult. The challenge is that the generations that follow me already have a more open viewpoint of the online social experience. They are no less opinionated, fearful, or cliquish than my generation but their memories are much shorter.

I'm not sure if that is because my-gen stores it's knowledge base in local or near "cache" and the next-gen uses google.

So what is going to happen you go to take that all important career making interview and the interviewer pulls out a stack of blog posts or pictures that demonstrate questionable moral character or outright disdain for everything lawful and good.

Will it be easy enough to say "I've changed my opinion since then" or "I was just acting out childhood angst" or my new favorite... "I was publishing without the filter of a professional editor"? We are being judged a…

Message Queues and nothing but Message Queues

[Update 2011.09.21] The ink had hardly dried on this post when I decided to quickly evaluate gearman. It's immature and direction internally seems dizzy. So pass on this one too.

I'm reading up on MQs again while I'm waiting for a conference call. I do not want to disrupt my code before the demo (rule #1 of demo-club). So I'm making yet another list of all of the MQs out there.

Amazon SQS
Google Queue/Task
Oracle Advanced MQ
IBM Websphere MQ
JBoss Messaging
Sun Open Message Queue
Apache Qpid

There are several ways to compare these MQs. Core source language, client language support, implementation detail, inspiration or initial design, current activity, licensing, cost, deploy OS, performance/TPS, use-cases, dependencies, deep dependencies, mindshare.

So here is that list again.

ActiveMQ - apache and meant for Java JMS although there are libs for other languages it really depends on the actual message payload.

pydoc - the anti-killer appp

Some time ago I wrote about perldoc being perl's killer app. I still think that's true. Recently I've been going through my python code and attempting to add useful comments that could be picked up by some auto documenting mechanism. Clearly they exist... is full of doc and I'm certain it's rendered.

Included with python, since 2.1, is pydoc. At first glance it appears to be an analog to perldoc; and for the most part it isn't.

pydoc will generate text or html - perldoc will convert the poddoc to html, latex, man, text
pydoc has a graphic UI embedded - but it requires TK (and x-windows)
pydoc has an embedded http server - could leak important data if the pydoc is internal use only.
perldoc generates static files - pydoc generates dynamic files
pydoc recurses the class dependency tree - if the stack is deep the document is long (there is a lesson in this)

In a way, it's nice that pydoc scans your code looking for your comments in your classes etc. The fo…

mojolicious – first app (part 2)

I was going to call it a night but I just wanted to make a little more progress on this first app. I'd like to think I'm tired and cranky because I'm not seeing it. Mojolicious generated the following as my first app:
#!/usr/bin/env perl
use Mojolicious::Lite;

# Documentation browser under "/perldoc" (this plugin requires Perl 5.10)
plugin 'PODRenderer';

get '/welcome' => sub {
my $self = shift;


@@ index.html.ep
% layout 'default';
% title 'Welcome';
Welcome to Mojolicious!

@@ layouts/default.html.ep
<!doctype html><html>
<head><title><%= title %></title></head>
<body><%= content %></body>
That's everything. It's their version of hello world. The app started right away, however, it was my fault.  I pointed my browser to and the page that popped up was some sort of 404 error p…

mojolicious - first app (part 1)

[Update] I should have mentioned that the elapsed time was only about 15 min; maybe less.

I'm a perl programmer from way back and while it has been 2 years since I've written any perl of consequence I've decided that my client's application was going to be implemented in perl... just for the fun of it. (I've been writing a lot of python+tornadoweb+redis+mongodb and I'm very comfortable in this space. Mojolicious is a half step outside my comfort zone because I really do not want to rewrite this app in python if I don't have to. I'm a big believer in DRY)

I'm certain that everything is installed. I used the same installation for documentation's sake recently:
# need to untar and then relocate the files to the path

# MOJOLICIOUS - (as root)
curl -L | perl - App:cpanminus
curl -L | perl - Mojolicious
curl -L | perl - Redis
curl -L | perl - ZeroMQ

Benchmarks - Hello World

If you've ever worked for a credit card company around the winter holiday season then you appreciate, as I do, benchmarking your application; early and often. Usually, however, this sort of benchmarking is meant to identify slow parts of the system as well as potential bugs that might have mad it through development. In this case testing usually centers around a very small set of transactions. Transaction types with the highest volume.

The "hello world" benchmark, on the other hand, is the complete opposite. It tells you some interesting things about your application but only the smallest bit.

Consider the hello_world that the different web frameworks are now benchmarking against. Even the slowest webservers are functioning under 100ms and most of that time is socket negotiation. For example, in tornadoweb, only executes about 35 lines of code in order to get the desired response. Since TW is single threaded all of the speed and performance is pushed to the python implemen…

Everything is in the cloud - it's all damage control now

There was a time when I was really concerned about what information was "out there". Then I tried to control that information. And finally I realized it was hopeless. Now it's just a matter of trying to keep it clean and ongoing damage control. Not that I've done anything wrong or embarrassing in my life...

The fact remains that the likes of google, yahoo, "social media", advertisers, and so are... while they might really want my demographic information, spending habits, income, neighbors, surfing habits, they will accept my anonymous information just the same. And I give them plenty of both.

The way it works... you visit a website. The site owner has contracted with someone like google for some sort of service. Whether it's advertising, sales or just some general tracking information it's not important. That service drops a cookie on your browser. Later, you visit another site. That site contracts with the same service provider... and looks for a coo…

How does [mongoDB] concurrency work

This is a short post to call attention to this [mongoDB concurrency]. I do not doubt, for a second, that concurrency is difficult. It's probably very hard. Part of which depends on the overall architecture, resultset size, record version collision, replication, sharding, and conflict resolution; among others.

I do not have a particular concern other than it is an important problem and that it's going to effect write performance and potentially some reads. (This is very similar to the GIL problem in Python). There is a ticket on their system that references this issue and plans to resolve the issue by pushing the lock to the collection instead of the DB. While this is more granular I'm not sure it resolves the real issue with a workable solution. (And I do not have a formal recommendation)

It's not clear to me whether this problem references a lock across all shards or just the one shard. Therefore, could the problem be alleviated by adding shards? (doubt that)

I can say t…

Perl RestMQ

I'm starting a Perl version of the python tool restmq. Restmq is really cool but my only complaint is that it uses cyclone; the tornadoweb server lookalike written for twisted.

Sure, I'd be justified porting it to tornadoweb but after spending time with mojolicious it seems like a better fit and more fun.

Contributors are welcome:

Tweet... What? Huh?

I received a 'being followed' notice from twitter today. Since this does Not happen very often and it's usually a spammer I check them all out. It turns out that this guy is a CEO for a social media company with 15k followers and he follows 15k people.

As my investigation continued I read his tweets for September 9. There must have been over 100 of them and they appeared to be legit. Which got me to thinking. How does he read tweets from 15k people; how does he keep that many conversations straight in his head; why; and what about his day job? If I were on the board of his company I'd be pissed!

And so that got me to thinking. What about the tweeter or blogger ithat tweets about pizza for dinner. Where is the marketing or social capital in that? It feels like smoke and mirrors.

[update 2011.09.11] and then there all the stupid bots that reply to everything. On topic or not.

Quickie : password security

I cannot take credit for the recommendation that we start using words for passwords. The argument I recall reading suggested that a 40-character password made from 4 or 5 dictionary words was a) easy to remember and b) harder to crack.

At first my intuition had me thinking that this guy was nuts. But then I did some rudimentary math. First I assumed that in a strong password or the traditional sense there were a few valid chars:
a-z A-Z 0-9 and !@#$%^&*() ,./<>?;':"[]{}\|= <space>
The problem with this is that even the strongest websites limit the user to alphanumeric plus a few simple special characters. So let's say we have 72 valid chars. That means that the math for a 7 character password looks like:
72^7 = 1.00E13
72^8 = 7.22E14
72^9 = 5.19E16
72^10 = 3.74E18
I'm hoping I have the math right but there is some huge wiggle room. For instance there is a list of invalid words and sequences, dictionary attacks, and then l33t speak. So this will reduce the n…

Agile Management vs Herding Cats

I recently blogged about agile, building teams and herding cats. It's a topic that's always close at hand. I've had several job interviews over the last 4 months and agile always comes up.

Agile used to mean the tight feedback loop between the client and the producer (the producer can be a programmer). Today it has come to mean so much more.

I asked an agile coach, friend of mine, about the cats and agile thing. His response was that you cannot heard programmers like cats. And that you had to have strong leadership and define clear goals.

So I went back to my Agile Project Management book and started thumbing through. I found a passage where the author said that teams were meant to be self organizing.

All of the contradiction is making my head spin.

Agile Suggests:

there are no managers
that the teams might be made up of cats, but so what, they will self organize
priorities are set by project managers who manage projects and not people
producers don't have to make value decisio…

Incremental improvement for iPhone

When you consider the cost to develop, manufacture and deliver a single iPhone, iPod or iPad you have to realize that there is no incentive to deploy anything more than one killer feature, a few minor visual improvements, and as many bug and security patches at the QA team can deliver.

The reason for this is because; A) they make a bulk of their cash on upgrades and first time purchases. B) why upgrade the calendar or addressbook when there is an app for that? With the sale of the app they make a margin that they would never see if they acquired a better version or if they seriously upgraded all of the apps.

This is not an easy strategy to deal with. The platform is easy to develop for, however, the apple apps have so many advantages over potential replacements. For example; address book cannot be deleted and is integrated into the phone.

The same is true for all of apple's products. I figure I need 5000USD a year to keep up with my wants. Not my needs.

libev and concurrency

What is libev?

A full-featured and high-performance event loop that is loosely modelled after libevent

Web Servers?

tornado, mojo, lwp, lwpng

What can go Wrong?

blocking on unrelated resources

What to do to correct?

que, req, resp


everyone likes a good graph, but hello world sucks, real world too hard to mock, what is a transaction

"Herding cats", building teams, "getting things done"

I'm in the process of refactoring my home office. I've reduced two 7' by 4' shelves, which were filled to capacity with programming language references, OS references, networking references, framework references and now the only topic that remains are my management reading library. And even those have to go.

I plan to give away as many of these books as I can. The rest I will recycle or give to a local university library. If your interested then drop me an email with your street address (sorry US only unless you want to pay postage).

The first book in question is "herding cats: a primer for programmers who lead programmers". It's an interesting book, however, to summarize... there are all sorts of programmers. You need to identity them and their types and then manage to their type. Do not try to treat them all the same.

I have worked for a number of startups and established companies. I have worked in small and large teams. I have also been involved in a mix…

Mongolab Surprise

I'm still digging into all things NoSQL and I have tried mongohq in the past but now I wanted to try mongolab. I was not expecting anything too super fantastic as I created my user account, my database and then my first collection. But I was.

I had a csv file on my Mac. I had previously installed mongo on my computer... I clicked on the import/export link on the mongolab website and they gave me a list of commands that I could copy/paste to the command line. After I inserted my username and password I executed it. It worked first time. Very nice.

Viewing and editing documents is a snap but subject for some self exploration.

Right now my only criticism is that it's expensive and I cannot determine the value proposition solely based on the webapp. I could install a mongodb server for a small fraction of cost and then clustering and ha become something I can measure.

time time python time time

I wonder how many bugs there are out there because people use "time" incorrectly. I think I discovered a design flaw, implementation detail or just a really crazy thing to watch out for that I have no idea where to go with it except here.

I've written this code a hundred times:
date = time.strftime('%m/%d/%Y')
time = time.strftime('%H:%M:%S')
This code behaves all the time except!!! At midnight of whatever timezone you are in. Consider that if the strftime() function that is computing/formatting the date took place yesterday and the strftime() for the time variable took place tomorrow. You would get seriously affected results.

While you would think that you could do this:
now = time.time()
date = now.strftime('%m/%d/%Y')
time = now.strftime('%H:%M:%S')
It feels right and it might work in some languages. But not here. When now is assigned time.time(), it is of type float. And therefore you cannot apply the function strftime() to a float. You need an…

Static vs Dynamic Languages

I'm not certain who started the discussion and if it ever really mattered. I started to read this article at TechRepublic and I just could not get into it. I suppose that in the halls of academia there is need to have these sorts of confrontations but in the pragmatic world does it really make sense?

For instance, I'm a programmer of some widget application. Within the application I have code that represents explicit and implicit contracts between objects, classes etc. And in the layer surrounding my widget I have some sort of abstraction or API layer. This layer defines a similar contract(s) between the widget (instance) and the widget's users. Some of these contracts are defined in DOC and others in code and behavior. And when you get really nutty with inheritance static languages start to behave like dynamic ones.

So what is the point of the post? The conversation is really about all of the other features and functions. The fact that we're dwelling on "Static vs …