Tuesday, October 16, 2018

dynamic sizing and reporting services

Most developers and system designers underestimate the complexity of reporting systems. Sure the first few reports are easy and sometimes downright trivial but that does not last long as you become the product of your success.



First of all there is challenge with concurrent reporting... just how many reports can you run at once? That's partly a network bandwidth thing and a memory thing and a computational thing. But it depends where the reports are being executed, local or remote, and whether the DB is sized properly too.

Then there is the catalog of reports and the largest with the most computational stress and the variable idle time between reports. And then there is the reporting schedule and the manual demands; all of which have queuing theory challenges.

Of course then there's report delivery and it's challenges, archiving, replication, consistency between reports, formatting, and security. And it gets worse.

My current challenges right now is that my static reporting server has about 8GB ram which is fine for day to day reports. Running at AWS it costs about $61 a month. A very similar machine at DigitalOcean costs $40.

However the monthend financials report requires 32GB ran and at AWS it costs $320 and $160 at DigitalOcean. The thing is 32GB would be sitting idle most of the month. It would be great to increase the concurrency so the scheduler could spawn execution engines instead of trying to be monolithic. And there is the DB server... it too can limit all of the above.

No comments:

Post a Comment

static site generators

Static site generators are pretty cool. Granted they are an oversimplification of the java/json/xml/xslt site manifestation from the olden d...