Welcome to ps_herald’s 1.4.2 documentation¶

ps herald :Version: 1.4.2¶

The ps.herald package provides three tools usable to monitor the behaviour of distributed applications

Herald’s functionality is based on the Config module, defined in the Within that documentation you’ll be enabled to use the ps.basic.Config mechanisms (especially the logging mechanism) within your application.

The Config module(among other things) enhances/uses the standard python logging package - especially the ability to log messages to a stream-socket.

This package/the herald package adds mechanisms to:

put these messages into a central database (bridge)

route/bridge these messages to additional destinations (bridge)

display in a web-server (herald)

take actions on special messages (neelix)

take actions on lost system heartbeat (neelix)

The tasks of the tools implemented within that package will be explained with the following picture .

If a program is running - let that program be named job or service (within the picture those programs are named job_[a,b,x,1,2] in column 4 (Production environment Runtime) ) it emits logging-messages (via the ps.basic.Config module).

Those logging messages are :

received by the ps_bridge daemon and inserted into a local sqlite database e.g. herald_db_d.

Those messages/events can be displayed/searched via herald.

Those messages/events can be analyzed and further events could be triggered by neelix.

Note

The ps_bridge module allows for “duplication” of event streams - meaning it is possible to insert logging messages into the local database AND to additionaly send that message to another bridge (possibly residing on another machine/port). It is possible to route those message through different ssh tunnels - so that on the different levels of trust - the messages of the behind system could be analyzed. This is shown in the following picture.

ps_herald¶

ps_herald is a flask-based (http://flask.pocoo.org/) web micro-framework, enabling us to display logging messages of the participating systems/services.

It uses sqlachemy to access the sqlite database.

ps_herald listens for html-requests (normally issued by an user-interface e.g. firefox/safari/chrome …) and returns corresponding log-messages.

The following picture shows a screenshot for the usage of herald within google-chrome …

Herald displays the messages of the “correlated” services in a table-format.

From left to right you see:

date and time of insertion of the logging message into the database

logging level

service-name , machine-name and user_defined system-name

Application-specific logging fields

package name and version, function name, line number

logging message

stack-trace

ps_bridge¶

The ps_bridge executable:

listens on a stage specific network socket
receives logging messages
- puts them into a local sqlite database
- eventually (bridge-mode) forwards the message to another socket endpoint.

ps_neelix¶

The ps_neelix executable reads it’s config file where services and ‘reactions’ are defined.

Next it consults it’s stage specific sqlite database and checks if ‘reactions’ are needed e.g. email a responsible admin, that the heartbeat of a service has been lost.

Install the package within a private virtualenv¶

To install the herald-package and it’s services, we first create a new directory e.g. $HOME/nodename/ps.herald_$BUILD_DATE, copy the following template to that directory and than execute the skript.

#The Content of $HOME/.pip/pip.conf is
#
#[global]
#index_url = https://vl-pypi.haufe-ep.de/ps/${DEV_STAGE}/+simple/
#
# and so points to the staging devpi server

/bin/rm -fR ovenv
mv venv ovenv

virtualenv venv
source venv/bin/activate


pip install ps.basic
pip install ps.herald

Maybe we establish a link in $HOME/nodename so that, in called scripts, we could use that link.

ln -s ps.herald ps.herald_$BUILD_DATE

Integrating restart and invocation of neelix/herald/bridge with a local crontab¶

Beneath an example how herald,ssh-tunnel, bridge and neelix currently are (re)started on sulu

SHELL=/bin/bash
MAILTO="thomas.setz@haufe-lexware.com"

# The ssh tunnel for the bridge bringing the data on port 9017 of the eu_cloud machine to the loacal bridge
01,17,24,36,45,51,10 * * * 0-6     ssh -f -oExitOnForwardFailure=yes -R 9017:localhost:9024 eu_cloud -N > /home/hcn/ssh_tunnel.log  2>&1
4 */2 * * 0-7                      export DEV_STAGE=PRODUCTION; cd /home/hcn/HCN_DataTransfer/u14_20170801; source venv/bin/activate && ps_neelix                      > neelix.log 2>&1
04,16,36,44,51,01  * * * 0-7       export DEV_STAGE=PRODUCTION; cd /home/hcn/HCN_DataTransfer/u14_20170801; source venv/bin/activate && ps.herald                      > herald.log 2>&1
05,15,21,34,46,50,02  * * * 0-7    export DEV_STAGE=PRODUCTION; cd /home/hcn/HCN_DataTransfer/u14_20170801; source venv/bin/activate && ps_bridge -s 1309600 -r 1024   > bridge.log 2>&1

herald package¶

Submodules¶

herald.angular_api module¶

ps.herald.angular_api.hello()[source]¶

ps.herald.angular_api.hello_to_ps_basic_logger()[source]¶

ps.herald.angular_api.index()[source]¶

ps.herald.angular_api.internal_error(exception)[source]¶

ps.herald.angular_api.list()[source]¶

ps.herald.angular_api.options()[source]¶

herald.bare_html_api module¶

ps.herald.bare_html_api.hello()[source]¶

ps.herald.bare_html_api.hello_to_ps_basic_logger()[source]¶

ps.herald.bare_html_api.index()[source]¶

ps.herald.bare_html_api.isset(param1, param2)[source]¶

ps.herald.bare_html_api.rm_logs()[source]¶

ps.herald.bare_html_api.set_search_params(request)[source]¶

herald.database module¶

ps.herald.database.close_engine(e=None)[source]¶

ps.herald.database.get_engine()[source]¶: current_app.config[‘DATABASE’] is already mapped to the ps.basic.Config.herald_sqlite_filename

ps.herald.database.get_session()[source]¶

ps.herald.database.init_app(app)[source]¶

ps.herald.database.init_db()[source]¶

herald.model module¶

The models.

class ps.herald.model.HeartBeat(**kwargs)[source]¶

Bases: sqlalchemy.ext.declarative.api.Base

Store the HeartBeat of the different systems.

id¶

newest_heartbeat¶

system_id¶

class ps.herald.model.Log(**kwargs)[source]¶

Bases: sqlalchemy.ext.declarative.api.Base

The logging message.

The model currently is build around the data structure,: the standard python logging mechanism uses to send logging Events across the class logging.StreamHandler().

api_version¶

args¶

as_dict()[source]¶: Return row as dict.

asctime¶

created¶

exc_info¶

exc_text¶

filename¶

funcname¶

id¶

levelname¶

levelno¶

lineno¶

message¶

module¶

msecs¶

name¶

package_version¶

pathname¶

process¶

processname¶

produkt_id¶

relativecreated¶

stack_info¶

sub_sub_system_id¶

sub_system_id¶

summary¶

system_id¶

thread¶

threadname¶

user_spec_1¶

user_spec_2¶

herald.ps_bridge module¶

ps.herald.ps_bridge.client_connected_handler(client_reader, client_writer)[source]¶: Start a new asyncio.Task to handle this specific client connection

ps.herald.ps_bridge.handle_client(client_reader, client_writer)[source]¶

Runs for each client connected.

The first 4 bytes of the message define the total length of the message. Hence in the first step, the first four bytes are read and then the rest of the message. This “rest” of the message is depickled and stored in the database. If the “bridge-mode” is enabled, the “byte-buffer” (first 4 bytes plus the rest of the message is also sent to another socket - the bridge socket.

ps.herald.ps_bridge.print_to_tunnel(data_p)[source]¶: Running in bridge-mode we route the incoming message to the outgoing socket.

herald.ps_herald module¶

herald.ps_neelix module¶

ps.herald.ps_neelix.check()[source]¶

[summary]

Go through the systemid’s found in the database:

check if somebody needs notification
update the hearbeat of the systemid

ps.herald.ps_neelix.notify(system_id_p, starting_at_p, till_p)[source]¶

ps.herald.ps_neelix.react(system_id_p, cause_p, value_p, row_p)[source]¶

system_id_p Name of the system for which we react
cause_p Name of the cause e.g. PATTERN or AGE
value_p Value of the cause
row_p table-row of the logging table correlated to the event

Subpackages¶

herald.graph_ql package¶

Submodules¶

herald.graph_ql.mutation module¶

class ps.herald.graph_ql.mutation.AddLog(*args, **kwargs)[source]¶

Bases: graphene.types.mutation.Mutation

class Arguments[source]¶

Bases: object

input = <ps.herald.graph_ql.typedefs.AddGLogFields object>¶

log = <graphene.types.field.Field object>¶

static mutate(self, info, input)[source]¶

class ps.herald.graph_ql.mutation.Mutation(*args, **kwargs)[source]¶

Bases: graphene.types.objecttype.ObjectType

addLog = <graphene.types.field.Field object>¶

herald.graph_ql.query module¶

class ps.herald.graph_ql.query.Query(*args, **kwargs)[source]¶

Bases: graphene.types.objecttype.ObjectType

get_by_system_id = <graphene.types.structures.List object>¶

get_logs = <graphene.types.structures.List object>¶

get_search_options = <graphene.types.scalars.String object>¶

goodbye = <graphene.types.scalars.String object>¶

hello = <graphene.types.scalars.String object>¶

static resolve_get_by_system_id(parent, info, **args)[source]¶

static resolve_get_logs(parent, info, search=None, first=None, skip=None, **args)[source]¶

static resolve_get_search_options(parent, info, **args)[source]¶

resolve_goodbye(info)[source]¶

resolve_hello(info, name)[source]¶

herald.graph_ql.schema module¶

herald.graph_ql.typedefs module¶

class ps.herald.graph_ql.typedefs.AddGLogFields(*args, **kwargs)[source]¶: Bases: graphene.types.inputobjecttype.InputObjectType, ps.herald.graph_ql.typedefs.GLogFields

class ps.herald.graph_ql.typedefs.GHearBeatFields[source]¶

Bases: object

id = <graphene.types.scalars.Int object>¶

newest_hearbeat = <graphene.types.scalars.String object>¶

system_id = <graphene.types.scalars.String object>¶

class ps.herald.graph_ql.typedefs.GHeartBeat(*args, **kwargs)[source]¶

Bases: graphene_sqlalchemy.types.SQLAlchemyObjectType

connection = None¶

class ps.herald.graph_ql.typedefs.GLog(*args, **kwargs)[source]¶

Bases: graphene_sqlalchemy.types.SQLAlchemyObjectType

connection = None¶

class ps.herald.graph_ql.typedefs.GLogFields[source]¶

Bases: object

api_version = <graphene.types.scalars.String object>¶

args = <graphene.types.scalars.String object>¶

asctime = <graphene.types.scalars.String object>¶

created = <graphene.types.scalars.String object>¶

exc_info = <graphene.types.scalars.String object>¶

exc_text = <graphene.types.scalars.String object>¶

filename = <graphene.types.scalars.String object>¶

funcname = <graphene.types.scalars.String object>¶

id = <graphene.types.scalars.Int object>¶

levelname = <graphene.types.scalars.String object>¶

levelno = <graphene.types.scalars.String object>¶

lineno = <graphene.types.scalars.String object>¶

message = <graphene.types.scalars.String object>¶

module = <graphene.types.scalars.String object>¶

msecs = <graphene.types.scalars.String object>¶

name = <graphene.types.scalars.String object>¶

package_version = <graphene.types.scalars.String object>¶

pathname = <graphene.types.scalars.String object>¶

process = <graphene.types.scalars.String object>¶

processname = <graphene.types.scalars.String object>¶

produkt_id = <graphene.types.scalars.String object>¶

relativecreated = <graphene.types.scalars.String object>¶

stack_info = <graphene.types.scalars.String object>¶

sub_sub_system_id = <graphene.types.scalars.String object>¶

sub_system_id = <graphene.types.scalars.String object>¶

summary = <graphene.types.scalars.String object>¶

system_id = <graphene.types.scalars.String object>¶

thread = <graphene.types.scalars.String object>¶

threadname = <graphene.types.scalars.String object>¶

user_spec_1 = <graphene.types.scalars.String object>¶

user_spec_2 = <graphene.types.scalars.String object>¶

Module contents¶

ps.herald.create_app(name, have_config_file=False, test_config=None)[source]¶

Welcome to ps_herald’s 1.4.2 documentation¶

ps herald :Version: 1.4.2¶

ps_herald¶

ps_bridge¶

ps_neelix¶

Install the package within a private virtualenv¶

Integrating restart and invocation of neelix/herald/bridge with a local crontab¶

herald package¶

Submodules¶

herald.angular_api module¶

herald.bare_html_api module¶

herald.database module¶

herald.model module¶

herald.ps_bridge module¶

herald.ps_herald module¶

herald.ps_neelix module¶

Subpackages¶

herald.graph_ql package¶

Submodules¶

herald.graph_ql.mutation module¶

herald.graph_ql.query module¶

herald.graph_ql.schema module¶

herald.graph_ql.typedefs module¶

Module contents¶

Module contents¶

Indices and tables¶