Acceldata Open Source Software Center
Acceldata helps customers build great data products, but first we built our own data platform which is being used by many Enterprises. Building and packaging the different components is a hard problem and there is no uniform way to package and distribute components outside few commercial offerings. These repositories represent our view of a proven distribution built using open source components only with built in Manageability and Observability.
Projects
libkapxy
Reads Kafka Producer Name/ID and Topics it's producing to from the network packets arriving to the Kafka Broker Port.
wizard
Multi-purpose execution engine with support for templates.
gurl
Minimal cURL like statically compiled CLI tool with Kerberos support.
admon
Monitors a list of Docker containers running in the localhost and sends an email in-case of a container in the list is not running.
htadmin
CRUD server to manage the htpasswd file. Used to add/remove basic auth users for nginx.
goutils
Collection of various helper functions in Go.
libkapxy
libkpxy is an extensible protocol specific network packet parser.
- Captures and decodes the network packets for juicy information.
- Currently supports parsing Kafka producer (protocol) network packets.
- Extensible to add support for more protocols.
wizard
wizard is Acceldata’s automation engine. JSON is used as DSL for the engine. It helps to automate ad-hoc tasks. The users will define a set of actions in JSON format.
These actions will be executed in the same order as the JSON file. Each action will be executed sequentially and incase of any errors the action will interrupted and break out the execution.
The user can define the action to ignore the error and continue the execution. Wizard supports (Go) templating out of the box. Templates are useful to dynamically generate output files like configurations for applications.
gurl
gurl is a curl replacement development in Go with Kerberos support. This is developed because, the default curl binary provided via the Alpine packages repository doesn't support kerberos authentication.
The users can pass the kerberos keytab path and the kerberos principal to make kerberos authenticated HTTP requests to any server. gurl has support for custom user-agent and HTTP basic authentication.
NOTE: It supports just the GET method currently
gurl can be statically compiled with GCO disabled to run in almost any OS distribution.
admon
admon monitors the locally running docker containers and the basic system resources and send email alerts as per the configuration.
Users can configure admon to monitor a set of containers running in the local machine, admon can monitor containers running on all the docker networks or a specific one. When a user defined container is not running it can send an email alert.
admon can also monitor basic system resources such as CPU, Memory and Disk utilization of the local machine and trigger an email alert based on a user defined threshold value.
htadmin
htadmin provides a simple API interface to manage the .htpasswd file.
- The .htpasswd file can be configured to restrict access with HTTP basic authentication for Nginx web servers. Refer: https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-http-basic-authentication/
- The credentials are automatically loaded, no need to reload the Nginx web server.
- The .htpasswd file will be generated in the same directory as the htadmin executable is running.
goutils
goutils is a set of various helper utilities or functionalities that can be imported.
- shellutils - helper functions related to executing shell / bash commands.
- netutils - helper functions related to network specific tasks.
- libsysd - wrapper functions on top of go-systemd module. Provides helper functions related to Linux systemd tasks.
Open Source Data Platform Project
The Open Source Data Platform project aims to provide organizations to build data products using a community supported open source data platform.