Skip to main content


Showing posts from August, 2011

perl web frameworks : part 2

After I finished writing part 1 of this article I thought I was going to call this subject dead as mojo was not behaving as the install documentation suggested. And if you cannot get the simplest hello world application to execute properly then all hope of doing something interesting is lost.

Alas, I head from mojo's author and all seems well. Sort of. First of all he said that I should read the perldoc. (and I like perldoc, oh yes I do). I read the doc and it does correct the statement make in the install guide. I appreciate that, however, I was not given the sense that he feels that the install doc is wrong. So we have a net-zero points here. And then we also skirted the issue of the use of the term 'daemon' as a parameter to launch the app as described in the perldoc. I don't think this issue will be resolved either. I have, however, created a github ticket with the hope that these will be corrected. Finally, I'm not sure that I care what the param name is or if …

perl web frameworks : part 1

[update 2011.08.30] as pointed out these are not micro frameworks.

I'm not sure where to begin and I'm not certain where this is going to end. I have an outline and some ideas but nothing is concrete. I'm not even sure what my conclusion is or how I'm going to get there.

What I do know is that I'm a fan of python and perl, among other languages both compiled, interpreted, functional, byte-coded and JIT. I know that I like small and micro frameworks that don't get in the way of getting things done. And I know while I like extensive and mature libraries I do not like deep dependencies. And while like terse language semantics and idioms I tend to prefer just enough to be descriptive and functional (the other kind of functional). As I reflect on that description I visualize my ideal senario as the sweet spot on the bell curve.


This past weekend I read an article where the author talked about micro web frameworks. Sadly the comparison was limited to a few python…

Get your Mojo on

This is just a quick post because I cannot contain myself. For the last few weeks I've been hacking on tornadoweb and while I like it there is plenty of room for improvement. Minutes ago I watched a slide presentation showing off several micro web frameworks implemented in python. Shortly after that I watched a mojocast.
In a job interview with Riak's Justin Sheehy I was asked about warts in python. My single complaint is the indenting. perl has none of that and it's simply not as write-once as people suggest!
That's when I happened up mojolicious. I have been following catalyst for a number of years but I've come to hate the amount of dependencies and that sometimes they will not install. Anyway, I'm anxious to try mojo out and see what goodness it brings.

If you think about it... python seems to be in chaos. You can expect to see versions 2.5, 2.6, 2.7 and 3.x as the latest version/package depending on the distribution you are using. While perl seems more static…


For the reader who is not familiar; the title of this article reads: NoSQL not equal NoDBA. And what I mean by it is that while the traditional function of the DBA is different in the NoSQL environment; one still needs a subject matter expert (SME) on the payroll in order to keep the "engine" running smoothly. NoSQL is just another specialty.

Many years ago I was caught-up in SleepyCat's BDB libraries. They worked, they were fast, and as they promised; you could forgo a DBA. I developed a few proof of concept applications using BDB and they worked great. They included speed, big data, ACID and everything they promised. Luckily for me, at the time, the projects never ran long enough for a disaster to occur. I know now that, at the time, I did not know enough about BDB to recover from even a moderate system failure.

Today we are inundated with NoSQL alternatives. Riak, MongoDB, Redis, Cassandra, Volt, Orient; just to name a few. To my knowledge, none of them actually state t…

Is pypy ready for production?

[UPDATE 2011.08.15] I've been considering my own response to the question. I don't think pypy is ready but more importantly, taking a page from the erlang book, I'm still working on getting the code correct. And by the time it's correct either pypy or some other speedup will be available (that does not require changes on my part)

PYPY seems to be getting very good results not to mention performance benchmarks. The numbers are good enough not to ignore; specially with the results at quora. But is it worth taking a risk in production even though PYPY says they have alpha code?

Slicing a list into a list of variable length sub-lists

[Update 2011-08-15] Please see my update at the end of the document.

This was not much of a programming challenge but it was a lot of fun and it only took a few minutes to decide how to implement it. The background for this task was something that, overall, I have not determined how I'm going to address it. Confused?

I'm reviewing a payment system's transaction specification. The messages are not the standard ISO 8583 messages that many vendors use, but they more like a 'C' struct in that it's just a collection of concatenated strings. The individual string, or columns, are in a predictable format but they are fixed length but different from each other. Consider this:
Field 1 : N : 6bytes
Field 2 : A/N : 4bytes
Field 3 : N : 12bytes
... and so on ...
There is a useful example for "Slicing a list into a list of variable length sub-lists", however, if you look at the examples you will see that the "step" is set to a f…

report: How and Why We Switched from Erlang to Python

How and Why We Switched from Erlang to Python at Mixpanel Engineering
Please read this article. The most important idea is here:
No one on our team is an Erlang expert, and we have had trouble debugging downtime and performance problems. So, we decided to rewrite it in Python, the de-facto language at Mixpanel.

Clojure and Scala... again

[Update 2011-08-09: I am returning from a failure to install scala and clojure. I was able to install OpenJDK from the Ubuntu packages, however, I wanted to build ant from source. And that does not appear to be possible. Ant depends on JUnit and JUnit depends on Ant. This makes it impossible to install everything from source. What's worse is that the build instructions are so much worse than I remember. There was a time when I would install all my Java tools except the JDK and my commercial JDBC drivers by hand.]

I just watched a video presentation from one of the Twitter geeks and he was going on and on about the JVM. What make it interesting is that Twitter is moving from Ruby to the JVM, in fact everything is now written in Java or Scala; and there are some research projects in Clojure.

I was looking over my resume recently as I was reviewing my Java experience for a potential position. And with about 10 practical years experience from the first version of Java (1.02) until a few…

Revisit LUA

Some months ago I wanted to look at LUA so I could investigate it and see where the benefits were and so on. I had resolved to leave it alone. Nothing interesting to see here. But then, recently, redis deployed a version with scripting which was implemented in LUA. This compounded with an unrelated post where an author suggested, and I posit the only killer feature or feature of interest, is that it's worthwhile to learn functional programming for it's own sake. (and nothing more).

So it with a grin that I say that I'm going to kick back and wait for the redis team to flesh out their version and I'll go learn it. I'm sure it's nothing complicated... I just like the design implications.

OLTP benchmarking is hard to do

I've built a number of successful OLTP systems used in the creditcard/prepaid card market place. One of these systems performs at around 12M transactions a day and the other around 900K. The first system has a lot more headroom. The CPU, disk and network are barely breathing. The latter, on the other hand, struggles and over the last few years I have found myself up late thinking about it.

The 12M system is running on Sun hardware running an Oracle database backend. The application was written in C using Oracle's embedded SQL. This one application runs multiple instances on the same box as the DB and the entire hardware/software stack is duplicated per client. This application connects directly to the internal network where OLTP transactions are routed from the company's internal POS devices for the closed network cards and from the credit card associations for the open network cards. This application also provides APIs that are called by the other applications for services…

LevelDB - a key/value database from Google

LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.
LevelDB was release a short time ago and they were quick to provide some benchmarks along with the first release which they say is version 1.1. Since I'm falling in lovepassionate strong like with Redis more every day... why not see if LevelDB was on the level.

Well,, I'm sure it is, however, there are a great number of CONS for adopting LevelDB:

Like ZeroMQ, LevelDB is a library and not a server. Although that could be an exercise left to the reader... as Riak has adapted to this engine.
It's not thread safe so that is an exercise for the reader
The most astonishing element of their description is that they support 3 actions: get, set, delete. While that is all you really need to get real work done the left is up to the reader.
Since it's a library and written in 'C', unlike ZeroMQ, there are no client modules. It's C++ all the way.


All-time low for open source

Please follow my train of thought:

java is the now new COBOL
I used to like java when it was first released 1.02 was probably the best release albeit not speedy
There are so many libraries out there that overlap and intersect that it's like looking at JCL through a kaleidoscope and I won't even hint at J2EE
There are many alternatives that cover many different languages, python, perl, ruby, go, erlang, haskel, scala, clojure and so on
Some of them are clearly better than others... some are just plain stupid
I recently endeavored to design a system using python, TornadoWeb, Redis, and ZeroMQ. (simple, easy, fun and productive) The framework is about 3000 LOC and the dependencies are shallow and easy to expand. An alternative to TornadoWeb might be cyclone but it's harder to build.
my next potential project needs to built in java including enough packages to be considered J2EE-light. Everything from Spring to Camel. There has to be a better way.
I thought to recommend SkyNet
It depen…

What is the attraction to Function Programming?

In computer science, functional programming is a programming paradigm that treats computation as the evaluation of mathematical functions and avoids state and mutable data. It emphasizes the application of functions, in contrast to the imperative programming style, which emphasizes changes in state. -Wikipedia
I've been a programmer for 25+ years and back in the earliest days a lot of PC-based code was implemented in x86 assembler. Later on 'C' became the de facto standard yet we still counted CPU instructions and IO operations.

As the web evolved and programming languages have come and gone many of the oldies are still in play. In fact many of the benefits of functional programming languages are limited or obscured by the non-functional OS and systems they rely on. Both scala and clojure rely on Java's JVM and they also depend extensively on standard old java libs, which are not functional, instead of replacing everything with native versions as one would expect.