Monday, August 27, 2012

When is enough, enough?

I have a daily standup meeting as part of my Agile Project Management process. But when I look at my calendar... on my desktop and my phone I see the time blocked off. Normally this would not be a real difficulty except that these 130-odd events look more like crumbs than slices of bread. Real and possibly important meetings are no longer visible as these events are in the way. The best way to handle this is to have a separate calendar for the daily events.

This might seem obvious but it's actually a microcosm of other issues. As I have worked for companies or projects with a 24x7 mentality or requirement it's too easy to be inundated with nonsensical or non-priority alerts of which services like mailgun and loggly are examples. These systems and your own systems may generate gobs of messages but if they are not consumed in context they will only serve to disrupt and distract rather than alert.

So when you're building an alert or log messages... make sure that the ERROR or CRIT messages are exactly that.  Do not wake me up for a warning or info or debug. Only when it really matters.

Sunday, August 26, 2012

When Art Meets Social

The Art of Unix Programming talks about how applications or systems are made from much smaller components and they are stitched together. Kevlin Henney did a presentation where he talked about the discovery of the pipe (|) character in unix systems. The key idea is construction or assembly from smaller parts and it's also dangerous. In the context of my local system the unix way is awesome. The stitching of apps together to make systems is useful, helpful, easy, and pragmatic.

The new challenge is that many of the new startups want to do the exact same thing. For example; wants to integrate with your google drive account.  LinkedIn and WordPress want to connect to your Box account to share files. wants to connect to your DropBox account in order to upload your eBooks. EverNote connects to a number of different services. Instapaper and Readability; as well as a number of RSS readers.... It's just crazy the number of overlapping systems that if they were within the same systems or owned by the same company I would not think twice but since they span companies it looks and feels like a trojan horse.
Let's not forget the countless numbers of apps that want a facebook or twitter ID before you can login. (read Spotify)

The thing that I realize most is that because I'm connecting all of these 3rd party systems I an on the hook for understanding their individual and collective security issues. For example PragProg mentions that they will upload your titles to DropBox and they go out of their way to say that they are going to be good netizens but I have no way of confirming that this is the case. They wanted and have my uid/pwd and frankly they could do what they want... if they were evil.

So what am I saying? The "unix way" is a pragmatic way to get things done, however, it's not without risk when you connect disparate entities. So tread lightly and carry a big stick.

Saturday, August 25, 2012

Linux Distros are Becoming More Like Windows...

It's hard to admit but it's happening and you can see it in the likes of Ubuntu 12.04 desktop (and earlier releases too) as well as Fedora 17. The first indicator was the need for advanced 3D video drivers in order to support the new desktop goodies. Sound like Aero/Glass yet?

This morning while I was updating a client's Ubuntu 12.04 server was that there seemed to be a request to convert /etc/resolv.conf to /etc/resolveconf. This might seem like a subtle change but when you read the manpage for resolvconf you realize that your server is going to be running a few more daemons that you previously didn't need. (I do not know enough about resolvconf and I'm certain the author would not be wasting our time but it feels wrong that we need a daemon where the traditional methods worked fine)

I watched a talk recently where the speaker pointed me to this doc. The interesting point is that the original Unix distro was only 10K lines of code. This speaks volumes considering the millions of LOC that are in the current distros. You did not get a GUI in 10K but it was a functioning OS. My complaint is that the dependency on current hardware and daemonizing everything that moves feels like a move away from the Linux roots.

PS: This will likely widen the splinter between the *nix and the BSD groups.

Friday, August 24, 2012

DataGrids - a note to the future me

Do not use built-in controls like DataGrids unless you are NEVER going to change or customize or integrate them with the likes of BootStrap or Boilerplate. In the end you'll find that it's going to take a lot more work to convert the code. It's just the way it is. (anyone remember the Win32 API wars?)

Tuesday, August 21, 2012

The Real Three Questions

(1) what do Microsoft, Apple, and Java (in the form of Eclipse, NetBeans and IntelliJ) hope to gain by having such complicated IDEs with so many dependent artifacts?

(2) if all apps were moved to the web, irrespective of the "cloud" attributes will ever be free of the desktop app tools?

(3) most *nix desktop apps appear juvenile or even retro compared to modern Windows and OSX apps; and even some java apps. Will they ever look modern and can they do it without all the cruft required by Microsoft and Apple?

The answers seem obvious to me, however, I'd prefer to hear your response before I give you mine. I thought about giving a hint but that would be too easy.

Wednesday, August 8, 2012

First year salary at a new job

The next time you find yourself having trouble negotiating your new salary and you cannot get the employer to up the ante 50 to 10% then ask yourself if there is a middle-man like a recruiter or agent in between.

For as many horror stories as there are out there; there are also great examples of recruiters that really know their business, their clients business, the marketplace, cost of living, relocation costs and plenty of other supporting details. The problem is that all of these recruiters or headhunters are getting about the same compensation.  And that compensation is based on your first year's wages.

One such company wanted to offer a lower base salary with a prorated annual bonus. With significant increases in the second year and beyond. Another company had a higher base salary but had a lower bonus and only paid half of the health insurance premium.

But this is what you can take to the bank. Before the company starts the interviewing and hiring process someone in management determines what they need in terms of headcount and what that is going to cost for at least the first 5 years. This is done so that costs can be projected and capitalized over the projects you'd be working on. It's also the difference between getting a contractor for a few months and hiring a fulltime resource.

Now when calculating this number it includes the cost of acquisition. Whether that is private advertising or through a recruiter or headhunter. It's in there. On top of that, the company is also going to want to lowball you; but that's not really that important when you consider.

(i) if you can help it avoid the recruiters that simply cast wide nets.  I get at least 10 to 15 emails and phone calls a week by some recruiter somewhere that wants to present me for some project. These guys are just fisherman. They do not have any real contact with the client. That function is performed by a different person within the company. (think about the last time you bought a car; there was the guy who was there when you walked in and kicked the first tire and then there was the guy you negotiated the deal with).

(ii) some recruiters will actually prep you for interview. Give you hints as to what is important. What talking points are key. I recently had an interview with a company that had a major crash. Some people might think of this as 100% bad but I saw this as 100% opportunity.  I wanted my recruiter to communicate to the client that I wanted some slack time in my schedule to prevent these sorts of things from happening again in the future. If the message actually got to the client it should make a considerable difference.

(iii) So if you can help it; try to get the job on your own and from there you can negotiate a stronger salary. It's going to feel dirty but once you are passed the salary and benefits discussion and you have the compensation you want you will be a happier person and ready to work on the task at hand. You certainly should not be thinking about the next interview.

And a side note for employers who use recruiters. Very well. If you do not have the time or inclination to interview and hire people then consider contracting that function too. But at least it'll be fixed cost. In the end, however, if you fail to compensate candidates for their experience and contribution you are just going to undermine your own mission.

Second and third thoughts about "the cloud"

I've been running iStatMenus and Little Snitch for quite a while. When I first started using these tools I wanted to know what my computer and the installed software was doing.  That included everything from Apple as well as the commercial and open source software I had installed. In a way Little Snitch was a dual authentication for outbound connections and iStat gave me useful indication of the resources currently in play.

Now that I have installed Mountain Lion and Little Snitch 3 (beta) I'm starting to watch my system more closely again. The first thing that caught my attention is that Google Chrome, Google Drive and DropBox are chatty. Yes, I have a few extensions installed on my browser like Google Voice,, Mail Notify, GTalk, Google Calendar, Google Task. What the hell are they doing?
If silence were golden then network silence should be platinum.

When I quit Chrome, Drive and DropBox there are still examples of apps that are talking to the network. But Why?  I just wrote an article this morning where I talk about installing non-commercial applications in userspace or adminspace and the possible side effects. So what are the lessons learned here?

(a) if you're using a cloud application then you are going to consume bandwidth and probably more than you want. So while that MacBook Air would ideal in the cloud the battery will not likely last, network costs because the apps do not know when you are on battery or using mobile tethering minutes.

(b) we, as consumers, have no idea what is actually being passed back and forth. It is not unreasonable to consider that Google could be using some sort of bittorrent-like system to seed future chome downloads and that's why I see such high usage.

(c) while my wifi bandwidth is still much higher than the ISP bandwidth it still seems that I'm saturated someplace. Even copying files from one system to the next on the same subnet seems constrained.

(d) What does not make any sense at all is the number of servers that Chrome connects to. It's not like it's one or two but is seems that there might be several dozen. What the heck are they doing? Some of it might be the different apps, but why? (that was rhetorical).

Sun's vision that the network is the computer is coming into vogue and it's starting to cost us in privacy and our wallets.

The web of trust is an evil illusion

I did not like how this article was taking shape so I'm starting over. I have a very serious two-part question for everyone.

(Q) Do you install adhoc(non commercial) binary files on your computer?
(Q) Do you install adhoc(non commercial) binary files on your computer with administrator privileges?

In the OpenSource world (no the world that Richard Stallman visits) not all source code is treated the same. For example there are some projects that are source code only (no make files), there are others with source and make files but no docs, and there are others that are so complicated or big to install that you have to install the binary (X11 is a great example); and others still have DEEP dependencies that are not automated.

It's also important to note that not all operating systems are treated the same. OSX provides Xcode virtually free of charge. The *nix systems have free and commercial toolsets. Visual Studio for Windows, on the other hand, is not free (there is an express version that might be free)

It is probably fair to say that Microsoft's sandbox is more of a petri dish for binary only malware. However, it is the users responsibility to steer clear. It's also on the tool vendors to make sure that tools are installed in userspace alone. Using duplicates or diskspace as a reason for installing as admin or root no longer exists.

As the saying goes... "Fortune Cookie: Man who put gum in jockstrap wake up with sticky dicky".

Thursday, August 2, 2012

High Frequency Trading Lacks Real Discipline

Or let me word it this way. Programmers who build and support high frequency trading systems lack any discipline and quite possibly impulse control. In my estimation they are no different than day traders who are desperate to make a buck a fraction at a time. Market Makers are no different and cut from the same cloth.

I recently interviewed with 4 different Quant and Market Maker companies in CT, NY, and NJ. I took personality tests, IQ tests, math acuity tests, programming tests, multiple phone interviews, and day long onsite interviews. As for the 15 onsite face to face interviews only 2 brought written questions and only 1 actually read my resume in advance of the interview. And all of them asked fermi questions, save one.

There are an amazing number of other similarities:

(a) The number of PhDs
(b) The number of transactions
(c) The same general approach to HFT
(d) The same general org chart and functional diversification
(e) The same approach to development, testing, and deployment
(f) The same lack of production discipline
... and the list grows

For all of their brain power KCG failed to detect the problem in the first few minutes. Common sense would dictate that secondary and tertiary systems should have been monitoring transaction volume based on past performance and future expectation as part of any release so that if these systems detected something that was out of variance it should have alerted operations staff within seconds of the event and not hours. Furthermore, the system should have been able to rollback to the previous day's configuration in order to resume normal operations. At the very least the system should have been terminated immediately. This is what I would consider block and tackle operational infrastructure. They should have taken lessons learned from the space program. Instead, my interviewers concentrated on completely non-relavent and subjective indicators.

Asking a perspective employer in NYC about their production discipline cost me an offer. Meanwhile the next day KCG crashed and burned.

PS: To the programmer who posted on reddit a number of months ago who justified his 500K salary. I hope you have made enough money to retire. You and your ilk are irresponsible and dangerous to our way of life and you should not be allowed to practice as a programmer under any circumstances.

As for the managers, directors, partners etc at the quants and market maker firms. You might be smart too but you do not know everything.... anymore.


Apple Batteries

Between my iPhone that just simply does not last 8hours while in NYC and my 1.4Ghz MacBook Air that does not last 6hours I'm just frustrated to no end. Part of the problem in NYC is that there are so many WiFi base stations in proximity at any given time that my phone is either trying to talk to or google maps is using in order to sense it's location that I can see how and why the batteries are so bad. You'd think with the population of NYC and the concentration of iPhones that it would not be an issue for me.

As for the MBA I have no explanation. I was on a plane with wifi and bluetooth turned off. I was writing some code in a small console window and watching some video. I did not have the brightness turned all the way up either. Problematic about Apple is that they are extremely secretive about things other companies might be proactive about... like telling me my batteries need to be replaced... before the warranties run out.

I'll say it again, as much as I like my Apple hardware I really want to try the new Vizio hardware with Linux and BSD as the Host OS. And I also want to give Android a shot too.

Wednesday, August 1, 2012

The next big thing in programming languages?

I've been troubled with the thought of what's next. The Python3 and Perl6 teams have been busy trying to sell themselves and the next best thing. Java has been accelerating, Go is a viable choice, even C has been getting into the act. There are also a number of JVM based languages like Scala, Clojure, Racket, IO and a few others. And then there are languages like erlang, haskell, and even prolog is making a comeback. The challenge is that each of these languages serves a niche or some overlapping niches but none serve it's master.

I'm thinking that the time has come for serious language translation.  We need a language definition that Donald Knuth would consider self documenting and yet robust and speedy enough for the most demanding tasks and terse enough for the laziest of our profession.

We need the Esperanto of programming languages. Or we need a way to convert whole programs from one language to another much the way that google translate works but for programming languages.

another bad day for open source

One of the hallmarks of a good open source project is just how complicated it is to install, configure and maintain. Happily gitlab and the ...