KageSenshi's Logs

Abyres Data Visualization, powered by Apache Superset is now on COPR

2021-02-07T12:04:00.004+08:00

In Abyres, we use Apache Superset for many of our data visualization needs. Traditionally, we utilize Superset primarily from what distributed by Hortonworks (now Cloudera)As we are investing more into Superset, and that we are increasingly seeing Superset requirements in situation where Hadoop is not really required, we decided to further our commitment in Superset by creating our own distribution package of it.

So, we created Abyres Enterprise Data Visualization, .. a Superset distribution that have been packaged and tested for use with CentOS, RHEL and Fedora. As we are well versed in the RPM packaging world, we ensured this package to integrate seamlessly with traditional management tools available for CentOS/RHEL/Fedora.

Alongside with the distribution, we also providing commercial support & consulting for the tooling for enterprises who wishes to leverage the capabilities of Superset in their organization.

If you want to try out `ae_datavisualization`, we have made it available in COPR for CentOS Stream, Fedora 33 and EPEL-8.

To install::

# dnf copr enable izhar/ae_datavisualization

# dnf install ae_datavisualization

To configure, the config file is in `/etc/ae_datavisualization/superset_config.py`. You'll need 2 postgresql databases (`ae_datavisualization`, `ae_datavisualization_cache`) and a rabbitmq server running.

To initialize database and admin user::

# ae_datavisualization db upgrade

# ae_datavisualization init

# ae_datavisualization fab create-admin

To start service::

# systemctl start ae_datavisualization-web

# systemctl start ae_datavisualization-scheduler

# systemctl start ae_datavisualization-worker

If nothing went wrong, you should be able to access Superset at port 8999. If you wish to change the listen port, you can edit at `/etc/sysconfig/ae_datavisualization`.

Interested to know more?.. Contact us at contactus@abyres.net / +603 2724 2535

Redash RPM in COPR

2017-04-10T00:26:00.001+08:00

Just a quick announcement post.

I've added a redash build in COPR.

To install:

dnf copr enable izhar/redash
dnf install redash

Configure your database at /etc/redash.cfg

Service is managed by supervisord, so start supervisord to start the service.

service redis start
service supervisord start

(I probably should figure out how to write systemd service)

Load redash on http://localhost:5000

Announcing PlatoCDP, a Plone distribution for enterprises.

2014-08-07T10:31:00.003+08:00

Announcing the first release of PlatoCDP Plone distribution. A Plone distribution with the goal of attempting to build a more supportable, enterprise-ready, document management and content management solution for government and enterprises. The ideal aim is to develop improvements on the distribution so that it will be a viable alternative to Sharepoint.
So what is it exactly?

An enterprise Plone distribution - that is, an integrated product which includes Plone, selected Plone add-ons, 3rd party softwares and integration/automation improvements to deliver an enterprise ready product for customers.
Is it a rebrand of Plone? - It is an enterprise distribution product. PlatoCDP to Plone is like RHEL to Fedora. It is not a mere rebrand. PlatoCDP includes core Plone, additional components from the collective and its own improvements. We engage on packaging, certification and development activities to make the whole stack supportable for enterprises. These components of the distribution will be supported by KOSLAB through our support subscription.

Major features highlights of PlatoCDP 4.0b1

Out of the box clustered Plone setup - preconfigured for high availability and scaling.
Out of the box support of collective.documentviewer. The whole setup is fully functional with very minimal effort.
Commonly used add-ons such as

eea.facetednavigation
solgema.fullcalendar
plonetruegallery
contentwellportlets
collective.unslider
and more (dependency list)

Some future roadmap in mind

Development of cluster management console to aid system administrators in managing, scaling and monitoring PlatoCDP deployments, plus aiding with filing bugs and support requests.
Fix WSAPI4Plone and include it into distribution, and provide an API for 3rd party integration.
Marketplace for 3rd party add-on/themes
Video conversion integration (possibly from Plumi transcoder)
Mobile client
Folder sync client
Cloud offering

Upstream commitment - Having been a Fedora Ambassador for over 7 years now, I believe in the philosophy of innovating on upstream rather than downstream. Therefore, wherever practical, PlatoCDP improvements shall be developed primarily on the Plone collective.

Check it out at http://www.koslab.org

Lessons learned on how not to run an Open Source company

2014-08-07T10:18:00.000+08:00

Back when I co-founded Inigo, one of my primary motivation was to create an Open Source company which:

Provide quality Open Source solution for Malaysian market (in this case, KM & CMS solution)
Work closely with upstream on improving the products supported by the company (in this case, Plone)
Actively develop local talents and human resource on Open Source skills, following pure Open Source philosophy
Actively create new innovation for the market related to our core solutions
Scale the company up that it become a prominent player on its core market

Unfortunately, over the years, none of the goals were reached. Due to our business model approach, ended up as a company which primarily focus on developing client-specific add-ons on top of Plone. The projects were fun, and challenging, however, due to a poor support and pricing model, the business unable to scale and we ended up in a hamster wheel of non-stop trying to catch up to pay the bills. Mistakes done includes:

Poor business/pricing model which penalize efficiency (the more efficient you become, the less you earn), made things very unsustainable.
Hiring is difficult due to the business model creates no sustainable revenue for investing on human capital development or hiring an experienced developer. Plone is not exactly something junior developer can pick up quickly. So the bus factor of Inigo is very poor, with only me as the developer.
The support model which we adopted also was self-destructive. It is an all-encompassing support package which does not limit what code will be supported - therefore, I have to be familiar with whatever codebase the client use on their system - including custom code which was developed by other vendors - fun at first - but a too high of a bar to handle after a while - human brain have its limits.
Continuously building downstream customizations removes the benefit of being part of Open Source community. Almost none of the code can be collaborated with other contributors because it is too specific to a single client.
Growing number of non-upstream custom code, of which are too client-specific that we could not immediately reuse, (if we managed to, the pricing/support model does not reward code reusability), inability to hire for help and/or build local capacity to help things out (due to I have to focus on delivering projects to pay the bills instead of building a team) - slowly take a toll on my mental health.

Being somewhat alexithemic, I did not notice the signs that I have been stressed out for over 2 years - something which apparently many people around me, especially those in Hack In The Box community have noticed way earlier than me - until I finally broke my brain circuitry and ended up with anxiety disorder (I'm still on xanax therapy until this date), and almost falling to a deadly state of depression. It took me 4 months struggling to figure out what was wrong with the brain function, nausea and constant chest pain made things harder to manage, only after finally medically diagnosed with anxiety disorder I decided to stop whatever I was doing, and take a break from programming.

Having said that, over the past 5 months, I have been working as a solution architect in a local Open Source system integrator company. The position I'm at gain me access to observe the process utilized by 2 of the major companies which adopt the philosophy of 100% open source and high commitment to upstream - Red Hat and Hortonworks. Picked up quite a bit from them, especially on the business side of enterprise open source, and learned how they create a sustainable support model around 100% open source software.

From the newly acquired knowledge, it created a new motivation to attempt again in bringing Plone to this region, through a creation of a Plone distribution - KOSLAB PlatoCDP.

More information, on the next blog post.

Reg for ZCA addicts - Part 1: Adapter

2014-03-22T08:52:00.001+08:00

Recently I have been exploring Reg, a library inspired by zope.component written by Faassen, and I found its simplification of ZCA to be pretty interesting. Being really loving ZCA for a while now, I end up trying to figure out how the ZCA patterns fits if I were to use Reg in my future projects. So here are the list of items I discovered so far:

Adapters
This is the most common pattern I use with ZCA, primarily to simplify writing logic by standardizing the interface of objects I work with into a common set of behavior/functions:

In ZCA , adapters are done this way:

from zope import interface
from zope import component

class IDog(interface.Interface):
    def bark():
        pass

class Dog(object):
    interface.implements(IDog)

    def bark(self):
        return 'Woof'

class IDuck(interface.Interface):
    def quack():
        pass

class Duck(object):
    interface.implements(IDuck)

    def quack(self):
        return 'Quack'

class IAnimalSoundAdapter(interface.Interface):
    def make_sound():
        pass

class DogSoundAdapter(grok.Adapter):
    interface.implements(IAnimalAdapter)
    interface.adapts(IDog) 

    def __init__(self, dog):
        self._dog = dog

    def make_sound(self):
        return self._dog.bark()

class DuckSoundAdapter(grok.Adapter):
    interface.implements(IAnimalSoundAdapter)
    interface.adapts(IDuck)

    def __init__(self, duck):
        self._duck = duck

    def make_sound(self):
        return self._duck.quack()

gsm = component.getGlobalSiteManager()
gsm.registerAdapter(DogSoundAdapter, (IDog,), IAnimalSoundAdapter)
gsm.registerAdapter(DuckSoundAdapter, (IDuck,), IAnimalSoundAdapter)

dog = Dog()
duck = Duck()

dog.bark() # Woof
duck.quack() # Quack

IAnimalSoundAdapter(dog).make_sound() # Woof
IAnimalSoundAdapter(duck).make_sound() # Quack

And that is quite a code, which utilizes classes, and large external interfaces libraries and component libraries which might can make many pythonistas cringe.

In Reg, things are simplified. Similar functionality is done not through interfaces and complex object/class markers, but through simple functions. Reg adapter interfaces are simply functions:

import reg
import reg.implicit


class Dog(object):
    def bark(self):
        return 'Woof'

class Duck(object):
    def quack(self):
        return 'Quack'

@reg.generic
def make_sound(obj):
    raise NotImplementedError

def dog_sound(dog):
    return dog.bark()

def duck_sound(duck):
    return duck.quack()

registry = reg.Registry()
reg.implicit.initialize(registry)

registry.register(make_sound, [Dog], dog_sound)
registry.register(make_sound, [Duck], duck_sound)

dog = Dog()
duck = Duck()

make_sound(dog) # Woof
make_sound(duck) # Quack

Now, ZCA users might shout by now, "I want class based adapters!" (I know I did). This is how it can be implemented in Reg:

import reg
import reg.implicit

class Dog(object):
    def bark(self):
        return 'Woof'

class Duck(object):
    def quack(self):
        return 'Quack'

@reg.generic
def animal_sound_adapter(obj):
    raise NotImplementedError

class DogSoundAdapter(object):

    def __init__(self, dog):
        self._dog = dog

    def make_sound(self):
        return self._dog.bark()

class DuckSoundAdapter(object):

    def __init__(self, duck):
        self._duck = duck

    def make_sound(self):
        return self._duck.quack() 

registry = reg.Registry()
reg.implicit.initialize(registry)

registry.register(animal_sound_adapter, [Dog], DogSoundAdapter)
registry.register(animal_sound_adapter, [Duck], DuckSoundAdapter)

dog = Dog()
duck = Duck()

animal_sound_adapter(dog).make_sound() # Woof
animal_sound_adapter(duck).make_sound() # Quack

Thats it for now, next one I will probably share about multiadapters on Reg.