Skip to main content

Introduction to AppScale and Google App Engine - Part 1

I'm not certain how many parts this is going to take and what the outline looks like. It's off the cuff but I hope you find this interesting nonetheless.

Lately there has been a lot of interest in micro services in the form of Docker. While I am a Docker fan and categorizing it as a micro service container started me thinking about monolithic and micro kernel operating systems; and that also forked a conversation about J2EE and google app engine; with a little flow based programming sprinkled in for good measure.

From the history perspective micro-based systems are faster but harder to orchestrate and debug. This is the one prevailing reasons why MicroSoft's monolithic Windows kernel defeated IBM's microkernel (but that war might have also been over before it started for many other reasons not related to the kernel). Testing a micro service itself is not difficult it's the integration testing within the system and testing the orchestration system/bus itself.

I recently implemented a flow based programming system in go. It was very simple to design and code the nodes. It was straightforward to write test cases. It was even trivial to wire the nodes together in order to define some sort of flow. And graphing the nodes into some sort of visualization was also pretty simple. The system was very data driven so generating queries or refactoring the nodes and reuse became favorable. The only downside is/was that there was no debugging.  GoLang does not currently have a debugger and while the application took advantage of lots of goroutines (one goroutine per node); debugging by writing log messages was not reliable due to timing and intermingling of messages from the concurrent goroutines.

I'm not suggesting that micro services in the many new incarnations are good or bad but I would say that I'm in the business of adding business value for my employer. And in such a position there is a balance between cool tech, future proof, technical debt, and getting shit done.

A few killer reasons for going after app engine are [1] it supports some modern languages including golang and just around the corner dart [2] If the code is implemented correctly then scaling can be a simple matter of increasing the instance count [3] simple APIs for storage, query, cron, MQ.

** I have been administering close to 15 machines for friends over the past 20 years. The only thing I need to do is upgrade the operating system about once a year and apply patches once or twice a month. The challenge is having to remember what I did last time and what the risk is. Worse still is not being 100% certain what the recovery from a failure might be. I'm even thinking about backup/recovery of the user data.  It's just a big mess. If I had implemented these apps in app engine things could be a lot more carefree.

I've decided to restart my app engine experience with app scale and  going through the getting started exercise using virtualbox on my laptop following these getting started docs. (For OSX)

  • install homebrew
  • install ssh-copy-id (not available by default)
    • brew install ssh-copy-id
  • install virtualbox
  • install vagrant
  • register the image with vagrant
  • configure and install your virtual terminal
    • I had to perform the vagrant up command twice
  • deploy appscale on VM
    • the appscale up command required ssh-copy-id
** this outline is now incomplete because I could not get past this step.
There are a few more steps from here but I'm stopping for now.  Appscale is throwing an exception which is going back to vagrant which [a] has installed version 12.04LTS and is complaining about 14.04LTS [b] the guest is aborting. So I have to restart assuming that I missed a prerequisite when I installed everything else... so to the beginning.

<timeout/>

After trying the appscale install on virtualbox from the beginning I have had no luck. I get the same error(s) which suggest that there is an IP address mismatch between the appscale private network and the hosted network. As well as the hosted tools.

** running this framework means that eventually I'll be running on GCE. By deploying on GCE I will not be bothered with the host or guest OS versions. This is a big deal. It's the one reason I really like Docker+CoreOS. And even though I have indicated that micro-services are a challenge to debug it's not impossible. So, for the moment, there is no winner.  

In part 2; I will try the installation on Google Compute Engine (that's GCE and not GAE), however, this will be a little self defeating if I cannot get it to work on a plain vagrant install.

I'm just guessing; but in part 3 I might try to install appscale on my own VM install. And wondering if it'll install on CoreOS as-is.




Comments

Popular posts from this blog

Entry level cost for CoreOS+Tectonic

CoreOS and Tectonic start their pricing at 10 servers. Managed CoreOS starts at $1000 per month for those first 10 servers and Tectonic is $5000 for the same 10 servers. Annualized that is $85K or at least one employee depending on your market. As a single employee company I'd rather hire the employee. Specially since I only have 3 servers.

The pricing is biased toward the largest servers with the largest capacities; my dual core 32GB i5 IntelNuc can never be mistaken for a 96-CPU dual or quad core DELL

If CoreOS does not figure out a different barrier of entry they are going to follow the Borland path to obscurity.

UPDATE 2017-10-30: With gratitude the CoreOS team has provided updated information on their pricing, however, I stand by my conclusion that the effective cost is lower when you deploy monster machines. The cost per node of my 1 CPU Intel NUC is the same as a 96 CPU server when you get beyond 10 nodes. I'll also reiterate that while my pricing notes are not currently…

eGalax touch on default Ubuntu 14.04.2 LTS

I have not had success with the touch drivers as yet.  The touch works and evtest also seems to report events, however, I have noticed that the button click is not working and no matter what I do xinput refuses to configure the buttons correctly.  When I downgraded to ubuntu 10.04 LTS everything sort of worked... there must have been something in the kermel as 10.04 was in the 2.6 kernel and 4.04 is in the 3.x branch.

One thing ... all of the documentation pointed to the wrong website or one in Taiwanese. I was finally able to locate the drivers again: http://www.eeti.com.tw/drivers_Linux.html (it would have been nice if they provided the install instructions in text rather than PDF)
Please open the document "EETI_eGTouch_Programming_Guide" under the Guide directory, and follow the Guidline to install driver.
download the appropriate versionunzip the fileread the programming manual And from that I'm distilling to the following: execute the setup.sh answer all of the questio…

Prometheus vs Bosun

In conclusion... while Bosun(B) is still not the ideal monitoring system neither is Prometheus(P).

TL;DR;

I am running Bosun in a Docker container hosted on CoreOS. Fleet service/unit files keep it running. However in once case I have experienced at least one severe crash as a result of a disk full condition. That it is implemented as part golang, java and python is an annoyance. The MIT license is about the only good thing.

I am trying to integrate Prometheus into my pipeline but losing steam fast. The Prometheus design seems to desire that you integrate your own cache inside your application and then allow the server to scrape the data, however, if the interval between scrapes is shorter than the longest transient session of your application then you need a gateway. A place to shuttle your data that will be a little more persistent.

(1) storing the data in my application might get me started more quickly
(2) getting the server to pull the data might be more secure
(3) using a push g…