Skip to main content


Showing posts from December, 2011

A new locale

I cannot say that I completely understand how this works. I cannot even say that I understand even the basics. In this case, however, I managed to let my intuition guide me and I seem to have gotten to the finish line.

The challenge ahead of me is that I'm getting a new warning message from mercurial when I attempt to pull the latest changesets from my repository:
-bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
I have 3 machines. My desktop, a development system and a mercurial repository. Let's call them desk, dev, and repo; respectively.

My .profile on my desktop had a few interesting statements:
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
And the values were brought forward to my dev machine. Assuming that term has a way to pull these values from nested terminal sessions. Then when I issue mercurial commands on the dev machine I see the warnings I mentioned earlier.

I logged into all three machines and I executed the command: locale. The results on my de…

Mongrel2 = mongrel + python

mongrel was originally written for the ruby folk by a programmer named Zed. I cannot speak to how useful it is or was but I remember reading the name a few times and that was about that.

Recently while I was reading up on ZeroMQ I ran into a reference to a framework called Mongrel2. Mongrel 2 was implemented using Python and ZeroMQ. It was also written by Zed.

It uses the single process/thread eventd model to implement the client-side web server. Requests are taken in from inside an IOloop and those messages are forwarded to and through ZeroMQ as quickly as possible. The programmer need only implement a route table and the "handler" (a ZeroMQ worker).

What makes it interesting is that it implements a 1:Many (client to worker ratio). And; it is implemented in a brokerless structure. In a brokered structure a single incoming transaction can yield 4x transactions on the same box and between 6x and 8x if shifting the transaction off of the current cluster. Also, they did not implem…

My New Python Project Setup

[update 2012-01-18] postgres has been updated to 9.1.2; the latest version as of today.

[update 2012-01-17] feel free to ignore my comments about Lua. While Lua might sit in an interesting place between Python and Java in an embedded/scripting place. The fact that lunatic-python does not compile and lupa depends on LuaJIT2 which is compatible with Lua 5.1 and the current Lua version 5.2 was recently released... and the comment from the LuaJIT team about adoption was a little snarky. I gotta think about something else.

[update 2011-12-29] I forgot to add twitter's bootstrap CSS/JS. I'll cover that in a future post when I also discuss modern-package-template

It's pretty simple to set things up. There are some prerequisites and some basic install packages that need root access but the intent is to get the config in userspace as soon as possible. This article covers VM slices at Rackspace using Ubuntu 11.10.

First: Install and update:

allocate the OS
select the OS and wait for it to…

Dude where's my computer?

I received a call early this morning. A former manager of mine was calling because his computer was stolen. This was not going to be that big of a deal because he regularly backed everything up and had an administrator password. Well almost. If this was just a crime of opportunity and they hardware was to be sold right away then everything is ok. If not, then he could be in some trouble and he'll have to spend some time monitoring his TRW and the like.

Whether or not you currently have valuable data on your PC or not ... at some point prior to a theft or loss you might find yourself regretting that decision. So here are some preventative measures.

completely encrypt your harddrive with a password
backup your data offsite and with encryption
use a screen saver with a non-trivial password
install a service like lojack for laptops
... and a little fun

Here is my explanation:

By encrypting your harddrive you are essentially guaranteeing that if the thief has to power the computer that he/she …

Modern Application Development

There was a time when I was very proud that I could go a year without rebooting my Linux or BSD machines; specially when it was my main development machine. Over the last 6 years of my OSX tenure I have had very similar success. The main difference is that I have been using VMware or remote virtual servers a lot more for development.

Last night I was forced to reboot my OSX machine. It was an unpleasant experience because it felt like I was being coerced into the reboot. I know the feeling all too well from my Windows days when I might have to reboot my Win machine several times a day.

Last night's experience was triggered by a large swap file. This was caused because several applications had allocated between 500M and 1.5G of memory.

Chrome of OSX
Safari (latest)
PyCharm (3 directories open but it's java)
Little Snitch (what? There shouldn't be that much data)
CrashPlan (also Java; since uninstalled)
Kernel Task (expected)
VMWare (expected)

I brought up htop in the VMware slice whe…

Response to "Seven Databases in Seven Weeks"

For this "7 in 7" book I just glanced at the motives for selecting the DBs that the author did. What caught my attention was the TOC. While the title of the book suggests that this is going to be a reference to modern databases and the NoSQL movement it included Postgres. What's curious here is that a) PSQL is not a modern database and it's not a NoSQL database either. b) While it is a modern implementation none of the modern features are mentioned.

And then there is a huge gap where BDB, BerkeleyDB, should be. While BDB is sometimes considered a NoSQL database it does not implement the CAP theorem which is consistently attached to NoSQL DBs. What makes BDB interesting, and which would seem to be the subliminal rationale for the many query dialects of the NoSQL DBs is an essay that Mike Olsen wrote where he justified BDB's APIs and the absence of a formal query language. [programmers know their the data better than any query optimizer] and then there was [the extr…

Response to "Seven Languages in Seven Weeks"

Bruce Tate is a good writer and recently he published a book titled: "Seven Languages in Seven Weeks". I do a lot of career development so I completely agree with the premise, however, the first place I get lost is the selection of languages:


Initially there is nothing wrong with this selection. Tate tells the reader that the choices were made my asking his readers. And at first glance this might makes sense (blame the reader), however, it's more dubious than that.

As I rebuked a recent blog for it's survey results pertaining to Agile because the sample group were in relative social circles to the author. I believe the same can be said here. About the only thing in Tate's favor, however, is that the words "practical" or "pragmatic" were omitted. Had they been present then I believe that the language selection might have echoed github's language survey.

In hindsight I should have read the TOC before I purc…

webapp application stack

I'm trying not to be a "bitter betty" over the time spent on Mojolicious so I've been refreshing my stack in order to prepare for future endeavors. In the meantime the application I've been building for a client required that the GUI be split into an API and a GUI rather than just an integrated GUI. At this point I have completed the API development and it's as extensible and scalable as I could hope for. Now it's time for the GUI stack.

There are several missions here. a) python; b) tornadoweb; c) keep in mind the person making changes to this app will not likely be a pythonista (thank goodness). That's it. Now I need to pick the rest of the tools. If there is some interest on your part then do some googling. They are easy to find.

PyCharm IDE - not all of the future developers need or want to understand everything from the command line.
virtualenv - handle different versions of python.
easy_install and pip - needed for installing the different dependenc…

"Mojolicious" has lost it's Mojo

I've recently encountered my second issue with Mojolicious; since the Mojo team had recently added some core developers and reinstated their ticket system via GitHub I decided to open a ticket.

The first issue I presented to the team was their use of "daemon" as a command line param to launch your webapp, however, it unexpectedly ran the app in the foreground. My understanding of "daemon" and the behavior of any number of apps is that "-d" or daemon-mode meant background execution.

The issue I am currently working on is the new 2.37 version. It's the first build from the new core team and that was a good thing, however, it would not build on my system (OpenBSD 5.0). So I reported it. The response that I received directed me to the CPAN team who verified that it worked.

Less obvious was the notion that I needed to check my dependencies. So I suggested exactly that... to which I received a reply that a) we were off topic and b) that there are no depend…

"Rails Is Not For Beginners"

[UPDATE] I don't know anything about this book and the fact that it's Ruby is not interesting but the quotes and snippets are: here

Oscar posted "Rails Is Not For Beginners". His general assertion is that Rails is complex and has a lot of code making it difficult for beginners... and that Sinatra is a better tool because it's only a fraction of the code. (about 100:1)

I responded to the original post with this comment:
"not for beginners" is mostly true but meaningless. The noobs have adopted it instead of VB. However, I think that it's a little more complicated than that and is much a psychological mystery as trying to understand the stock market.

There is something to be said about "full stack awareness" and sinatra's LOC makes that easier. Sinatra also allows you to get some real work done. My intuition tells me that sinatra does not do as much fancy META magic under the covers as Rails does; which is more of a property of the language …

RestMQ routing table for PerlMQ

[Update 2011-12-11] After several months of preparation to build this app I'm regretfully abandoning the effort. a) the mojo guys; b) since I've decided to concentrate on Python... RestMQ will have to do because there is no sense in rewriting the code for TornadoWeb. Good luck to us all.

As the project continues this article is going to convert the routing table from the RestMQ app to the PerlMQ version. The RestMQ snippet for the routing table associates the URL to a particular event handler in a list form. The Mojolicious version does not. In the Mojo version the URL and the handler's signature are declared at once.

Here is the RestMQ routing table:

(r"/", IndexHandler),
(r"/q/(.*)", RestQueueHandler),
(r"/c/(.*)", CometQueueHandler),
(r"/p/(.*)", PolicyQueueHandler),
(r"/j/(.*)", JobQueueInfoHandler),
(r"/stats/(.*)", StatusHandler),
(r"/queue", QueueHandler),
(r"/control/(.*)", QueueControlHandler),

Reverse Engineering RestMQ

RestMQ is no big secret. The source is available online and there is some doc too. The project I'm referring to was written in python using the Cyclone framework which depends on Twisted. I like both of those tools but they feel wrong for a project like this. It seems to me that the dependencies are too deep.

That's when I requested and was granted an account from the CPAN team. My intention is to build a version of RestMQ in perl using the Mojolicious framework. I originally thought to call it 'RestMQ-pl' and then I thought of 'PerlMQ' and I'm also lingering on 'MojoMQ'. The last one makes sense if I create a separate core API than can be integrated into the Mojo base code. (when I learn it)

The RestMQ code and doc is on par with just about every other open source project out there but there are some limits. One thing that is missing for me is enough code/doc that I can use as a requirements doc in order to implement the code in any language or any p…

Languages and more languages

[Update 2011-12-06] Actually there is still a lot missing from this list. The tier-1/2 dependency list is pretty high. Things like Spread, ZeroMQ, OpenSSL, JDBC, jPOS, Mnesia, Mojolicious, Twisted, Celery, ActiveRecord, Hibernate, iBatis, Riak, Redis, MongoDB, Cassandra ... and it goes deeper than that too.

As a followup article to "projects and more projects"; this article is going to cover all of the programming languages, tools and frameworks I've used over the years as they relate to the projects I've worked on. This is going to be the hardest document to write because in some cases the dependencies were too deep for me to know for sure. (thank you WebLogic) but I'll give it a shot in that maybe it'll be interesting in the end.

I'm not sure if this is everything but it's pretty darn close.

Chaos in Framework development

I think I am observing....
the more general purpose a framework or language becomes the more likely that the things implemented on top of it also become more general purpose ... recursively.
Which has me asking the question:
So at what point do we end up back at the original source?
Think about this. Sun's, now Oracle's, JVM executes on just about everything out there. The JIT is producing runtimes that are nearly as good as C. So given all of the extra heat that the JVM generates why not just write good C code in the first place?

And that has me thinking about something else...
Forget whether or not Moore's law will be true forever but assuming that it is. At what point do we stop caring?
For about the first 4 to 5 years of the PC Moore's law was really important. We were aching for CPUs that would allow us to do exponentially more work. We wanted more memory etc... MS DOS applications and MS Windows was around the corner. And we just waited for the hardware to catch up.

Now …

I want a private twitter clone for my business

While I'm not sure I know what the difference between short emails and tweets or their effect on productivity. I am wondering about the movement to ban emails from the workplace.

It's not that I mind that my conversations are in the cloud it's that I want to choose which conversations are in the cloud. Also, a recent article was posted where a CEO or two banned email from their company(s).

If I had to list a number of requirements for a private twitter:

mobile client
mobile web
desktop app
browser app
backchannel for internal employes or inner circle
server storage of conversations
temporary accounts for external resources with filtered output
up/down voting
tagging (both twitter and external tagging)
tag a selection of messages
live filtering like tweetdek in one column and my conversation(s) in another
export to PDF or some other format
slightly larger than 140 bytes

But then what about GMail or some other Mail client? If I could filter my messages by size so that the …

Detecting Mobile Browsers

I have a webapp that I'm about to normalize into separate model and view apps. It's not an uncommon approach in order to perform different rendering based on the browser type in particular mobile. SO the first step in the process of information gathering is browser detection. Like everyone before me they did the google search and found the many tools out there. Clearly this is going to be a moving target over time.

I ran the generator code and ended up with this mess:
# Ported by Matt Sullivan
import re
from django.http import HttpResponseRedirect

reg_b = re.compile(r"android.+mobile|avantgo|bada\\/|blackberry|blazer|comp
al|elaine|fennec|hiptop|iemobile|ip(hone|od)|iris|kindle|lge |maemo|midp|mm
p|opera m(ob|in)i|palm( os)?|phone|p(ixi|re)\\/|plucker|pocket|psp|symbian|
treo|up\\.(browser|link)|vodafone|wap|windows (ce|phone)|xda|xiino", re.I|r
reg_v = re.compile(r"1207|6310|6590|3gso|4thp|50[1…

Response to "2011 IT Project Success Rates Survey Results"

In a recent post 2011 IT Project Success Rates Survey Results the author posted some results from a survey and the contents had been discussed in a Dr Dobbs article which clearly provided some credibility. However the results and the conclusion almost had me fooled.

In summary when comparing iterative, agile, lean, ad-hoc and traditional project teams... Agile scored equal or better than the other project team types.

I expected this result because I was directed to the article by someone I trust and highly recommend as an Agile professional. But when I got to the bottom of the article I realized that it was not an authentic research report.

From the information given I conclude that the survey and it's conclusion were biased. a) because the respondents were contacted via methods relative to the author:
This survey was performed during the last two weeks of October and there was 178 respondents.  The survey was announced in my October 2011 DDJ column, my DDJ blog, on the Ambysoft annou…