by Mark Ward, Technology Correspondent, BBC News website Like many subjects, information security comes with its own terminology and the jargon can be opaque to outsiders. Below is the list of terms in the Glossary that helps shed light on the murky world of cyber crime. DCL: I thought it might be interesting to compare it [...]

 Reakt – The ruleCore CEP language 

A White Paper from ruleCore

DRAFT

 

Introduction

The Rulecore CEP Language – Reakt! – When you need to react immediately.

Reakt is a language for complex event processing, or CEP for short. It is designed to provide a powerful and productive tool for developers allowing them to quickly create solutions for detection oriented CEP scenarios.

 

Reakt is not a traditional programming language. Reakt is a situation description language. By using a declarative approach it allows the developer to concentrate on

what kind of situations to detect and not how

The design goal of Reakt is to provide a way for organisations to react immediately to critical situations. It achieves this goal this by finding related events from  streams of inbound events. What a critical situation is and how the events are related are, defined by the user using Reakt. Reakt allows the user to specify conditions and temporal relationships which must hold in order for the events to be considered a noteworthy situation. When the situation is detected by Reakt, an user defined real-time report is created which provides actionable information about the situation. Allowing the enterprise to react immediately.

 

This short white paper gives an informal description of the language and explains the different language elements using examples instead of formal definitions.

Overview

Rulecore Reakt is a small domain specific complex event processing language designed for detecting situations by observing streams of events. A situation is a formal description of a combination, a pattern, of events as they happen over time along with optional restrictions among the events. A simple example of situations that you might want to detect could be

"A vehicle has visited zone A more than 10 times during the last hour"

or

"Event A comes before event B, unless event C has been seen"

Main Reakt language features:

 

• Continuous Evaluation – Designed to operate on live streams of events as they happen.
• Declarative – Rules and other items are specified using a high-level declarative approach.
• Modular – Rules are defined using modular and re-usable building blocks.
• Temporal – Handling of time is an integral part of Reakt. Timing restrictions and other temporal conditions can be stated using a number of declarations.
• Location Aware – Reakt includes built in support to understand location information and other geographical objects such as zones. 
• Implementation Independent – The elements of Reakt are defined using standard XML and are implementation independent. Different implementations can be created in order to provide specialized execution environments.
• Automatic Life Cycle Management – Rule instances are created and deleted automatically in response to events. Rule instances are analyzed and deleted if their situation is determined to be undetectable.
• Dynamic Context Management – Each rule instance maintains its own virtual view into the incoming event stream. The view provides a window into the incoming event stream in order to create a dynamic context for rule evaluation.
• Separation of Concerns – Reakt uses reusable blocks for;
  • Selection of events from the inbound stream
  • Rule life cycle management
  • Situation definition
  • Rule trigger reporting

 

The design goals of Reakt is to provide a language which allows the user to describe complex situations consisting of multiple events with non trivial temporal and geospatial relationships. The language is declarative. Lacking any procedural elements provides a base for formal verification and automated optimization and analysis of the language.

A design assumption made early in the language design was that is is likely that machines will use Reakt to create and modify rules more frequently than humans. The rate at which new rules need to be created and old ones modified is assumed to be high. Thus the language was designed to be easily understandable by machines and at the same time being reasonably human readable for system developers.

The current version of React uses XML for its syntax. XML was chosen to take advantage as much as possible of existing knowledge among the developers, known standards and existing tools for generating and parsing. It would also be possible to create alternative syntaxes for React. For example to create a more compact human friendly representation for business users.

The Reakt language was designed to support execution environments which fits into an architectural style called Event Driven Architecture (EDA). In order be a natural building block in an EDA, the ruleCore Reakt language and its supporting execution environment, the ruleCore CEP Server, was designed to be completely event driven. This purely event driven approach allows all systems in a distributed messaging architecture to integrate with ruleCore using simple events as their only communication mechanism.

 

The System Information Model

The System Information Model, or SIM for short, is both the name of the information model used by ruleCore CEP Server where all information about different items (rules, situations etc.) can be found and the name of the run-time data structure modelled after the familiar DOM model, commonly used to represent information originating from XML. The SIM is part of the ruleCore CEP Server implementation and not the language definition. But as all the language elements declared are executed against the SIM it is described here briefly. Different implementations can implement varying information models.

The SIM is used for various purposes, for example:

• Listing available definitions – All definitions of items such as rules, views, zones, entities, situations and actions can be found in the SIM. Definitions are created using the systems events, for example AddRuleDef, AddSituationDef and AddActionDef.
• Listing user defined events – Before an event can be sent into ruleCore CEP Server, it must be defined using the AddEventDef system event. The SIM contains all user defined events
• List of rule instances – When a rule instance is created, it’s state and internal data structures can be monitored by querying the SIM. The ruleCore CEP Server provides a query mechanism using XPath. But other implementations might query the SIM using other methods.
• Monitoring situation progress – As a situation develops, its progress can be monitored by querying the SIM.
• Examine view contents – Each rule instance is executed in an unique context of events called the event stream view. The view contains a window into the inbound stream of events and contains only semantically related events.
• Referencing items – When an item such as a rule needs to reference other items it is done using references through the SIM.
• Component Specific Data – Each plug-in component that extends the engine creates an entry in the SIM and may add any component specific information.

 

The SIM consists internally of dynamic data structures which are updated as needed when the internal state of the ruleCore CEP Server changes in response to inbound events. These data structures have an external representation which is the user’s view of the SIM.

The external representation of the SIM can be treated as a DOM document which can be queried using XPath and transformed using XSLT. XPath queries can be executed against the SIM using the SysInfoQuery system event.

If is also useful to note that the ruleCore CEP Server uses events for all tasks. Special pre-defined system events are used to create rules and other items in ruleCore CEP Server in addition to monitor the run-time state of the rule evaluation.

 

Structure of a Rule

The main language element in Reakt is the rule. Reakt uses a class of rules called reactive rules to provide its situation detection capabilities. Reactive rules should not to be confused with production rules or other types of rules which must be explicitly evaluated. A rule in Reakt is never explicitly evaluated, its evaluation must be automatically managed by the execution environment in response to inbound events.

A rule consists of a number of elements, the main ones being:

• Initialization directives - Specifies which events can lead to the creation of a rule instance
• Termination directives – Specifies what causes a rule instance to be deleted.
• The rule level – Specifies on what level in the rule hierarchy instances should be created.
• Trigger Directives – Controls how and when to report Rule triggering.
  
• The View – A reference to a view, providing execution context for the rule.
• The Situation – A reference to a situation to track and detect situations.
• The Action – A reference to an action, which will generate an event when
  • the situation is detected, or
  • if the situation is determined to be undetectable.

General Rule Attributes

Each rule has a name and a attribute which can be used limit the number of rule instances that can be created of a particular rule. A common usage of the limit is to create a singleton rule, of which only one single instance can exist.

Rules at every level can publish their trigger events externally so that they are visible outside of Reakt.

All rule declarations specifies a level. Every rule instance is created on a specific level in a hierarchy of rules. Level zero is the level on which inbound events arrive. Thus the lowest level for a rule is 1 (one). A rule will only see events from a level directly below it.

For example, three levels of rules where the rule at level one sees the inbound events:

<Rule name="New Location" limit="1000" level="1"> … </Rule>

<Rule name="Zone Entry" limit="1000" level="2"> … </Rule>
<Rule name="Zone Exit" limit="1000" level="2"> … </Rule>

<Rule name="Zone Visit" limit="1000" level="3"> … </Rule>

 

Rule Life Cycle Management

Rule instances are automatically created and deleted based on directives in the rule declaration section.

The <Initialize> declaration controls when a new rule instance can be created. All rule instances are created in response to events. The <Initialize> section lists those events which can lead to new rule instances being created.

Create a rule instance when an event of type GPSPosition arrives.


    <Initialize>
        <Assert>
            <Event>
                <base:XPath>sim:EventDef[@eventType="GPSPosition"]</base:XPath>
            </Event>
        </Assert>
    </Initialize>

The <Key> declaration is the unique key for each rule instance. The key consists of one or multiple properties of the init events.

    <Key>
        <Property name="Vehicle"/>
        <Property name="Zone"/>
    </Key>

The key declaration allows for easy detection of situations per something

 

For example, the above key declaration creates rule instances which are per vehicle and zone. So there would be rule instances created for each unique combination of vehicle and zone.

The <Terminate> section determines when a rule instance is deleted. The <Terminate> section consists of a number of directives which allows for creation of flexible termination strategies. A rule could for example be terminated when it has triggered three times, when it’s view is empty, on a specific date and time or when a particular event arrives.

 
    <Terminate>
        <Triggered count="3"/> 

        <ViewEmpty name="ZoneEntry"/>

        <Time>2010-01-31T00:00:00Z</Time>
        <Assert>
            <Event>
                <base:XPath>sim:EventDef[@eventType="Location"]</base:XPath>
            </Event>
        </Assert>
    </Terminate>

 

Rule Triggering

A rule may trigger when one of two things happen; the situation is detected

 

or it is automatically determined that the situation is undetectable in the future. Depending on the terminate declarations a rule can fire one or multiple times.

   

    <Trigger>
        <SituationDetected/>
        <UndetectableSituation/>
    </Trigger>

 

When a rule triggers, its action part is executed. The action part creates an event and thus the visible result when a rule triggers is a new event, created according to your specification.

 

Trigger Reporting

Related to rule triggering is the reporting of the rule triggering. The triggering of the rule and reporting that triggering is separated in two stages. Thus it is possible to postpone the report of a rule triggering. For example, it is possible to report rule triggering, or multiple triggerings, each minute or when the rule has triggered five times or whichever comes first. It is also possible to limit the trigger reports so that only a specified number of reports are created. The most common case is to create rules whose triggering is reported only once.

   <Report>        

        <Every trigCount="10" timeframe="00:10:00">
        <Max count="234"/>
        <AtTermination/>
    </Report>

 

References

The rule also contains a reference to the rule’s view, situation and action. All references in Reakt are specified using XPath and executed against the SIM.

The view reference contains a reference to a view declaration. The view provides the context in which the rule is executed

 

    <Views>
        <ViewRef name="ZoneEntry">
            <base:XPath>sim:ViewDef[@name='ZoneEntry']</base:XPath>
        </ViewRef>
    </Views>

 

The situation reference contains a reference to the situation which is evaluated in the context of the view.

    <Situations>
        <SituationRef name="ZoneEntry">
            <base:XPath>sim:SituationDef[@name="ZoneEntry"]</base:XPath>
        </SituationRef>
    </Situations>

The action reference contains a reference to the action part which generates a new event when the rule triggers.

   

    <Actions>
        <SituationDetected situationName="ZoneEntry">
            <ActionRef name="ZoneEntry" eventVisibility="external">
                <base:XPath>sim:ActionDef[@name="ZoneEntry"]</base:XPath>
            </ActionRef>
        </SituationDetected>
    </Actions>

 

Complete Rule Example

Here is a complete example of a rule.

<Rule name="ZoneEntry" limit="1000" level="1">

    <!– Create rule instance per Vehicle and Zone –>

    <Key>
        <Property name="Vehicle"/>
        <Property name="Zone"/>
    </Key>

    <!– Create a rule instance then the Location event arrives,

         if one does not exists per the key above –>

    <Initialize>
        <Assert>
            <Event>
                <XPath>EventDef[@eventType="Location"]</XPath>
            </Event>
        </Assert>
    </Initialize>

    <!– Delete rule instance when rule has triggered 300 times, the view is

         empty of at the specified time or when the Reset event arrives –>

    <Terminate>
        <Triggered count="300"/> 

        <ViewEmpty name="ZoneEntry"/>

        <Time>2010-01-31T00:00:00Z</Time>

        <Assert>
            <Event>
                <XPath>EventDef[@eventType="Reset"]</XPath>
            </Event>
        </Assert>
    </Terminate>

    <!– Report rule triggerings every 10th triggering of every tenth minute

         or at rule termination, report a maximum of 234 triggerings –>

    <Report>        

        <Every trigCount="10" timeframe="00:10:00">
        <Max count="234"/>
        <AtTermination/>
    </Report>

    <!– Trigger rule when situation is detected –>

    <Trigger>
        <SituationDetected/>
    </Trigger>
 

    <Views>
        <ViewRef name="ZoneEntry">
            <XPath>ViewDef[@name='ZoneEntry']</XPath>
        </ViewRef>
    </Views>

    <Situations>
        <SituationRef name="ZoneEntry">
            <XPath>SituationDef[@name="ZoneEntry"]</XPath>
        </SituationRef>
    </Situations>
    

    <Actions>
        <SituationDetected situationName="ZoneEntry">
            <ActionRef name="ZoneEntry" eventVisibility="external">
                <XPath>ActionDef[@name="ZoneEntry"]</XPath>
            </ActionRef>
        </SituationDetected>
    </Actions>

</Rule>

 

The View

The rule declaration contains a reference to a view. The view provides the context in which the rule is evaluated. The view keeps related events together and selects only interesting events from the inbound stream of events.

Each rule instance contains its own private view instance. The view contains a collection of events. The view is kept updated based the stream of inbound events. The view provides a dynamically changing execution environment for the rule as each rule instance is executed in the context of its view.

The view consists of a number of declarations, each specifying a property of an event that must be true in order for the event to be present in the view.

All these declarations are evaluated to determine which events are present in the view of each rule instance.

Example of properties available:

 

• The type view property is used to include events into the view only if they are of the specified type.
• The maxage view property defines the maximum age for each event in the view. You could for example use the age property to let each rule instance see only events from the past 10 minutes.
• The maxcount property defines the maximum number of events in the view. You could for example use this property to limit the view to contain only the last 100 received events.
• The match property defines which event properties must match in order for them to be included in the same view. You could for example require that the property CustomerID matches on all events in the view, thereby creating a view which contains events about the same customer.
• The assert property defines an XPath expression that must evaluate to true for all instances of a specified event type. The assert property could for example be used to compare a certain element in the event body with a constant.

<ViewDef name="ZoneEntry"> 

  <Properties>

 

    <!– All events in the view must be of types InsideZone or OutsideZone –>

    <Type>

      <Event>

        <XPath>EventDef[@eventType="InsideZone"]</XPath>

      </Event>

      <Event>

        <XPath>EventDef[@eventType="OutsideZone"]</XPath>

      </Event>

    </Type>

    <!– All events in the view must be from the same vehicle –>

    <Match name="vehicle">

      <Value>

        <Event>

          <XPath>EventDef[@eventType="OutsideZone"]</XPath>

        </Event>

        <Property name="Vehicle"/>

      </Value>

      <Value>

        <Event>

          <XPath>EventDef[@eventType="InsideZone"]</XPath>

        </Event>

        <Property name="Vehicle"/>

      </Value>

    </Match>

    <!– All events in the view must be from the same zone –>

    <Match name="zone">

      <Value>

        <Event>

          <XPath>EventDef[@eventType="InsideZone"]</XPath>

        </Event>

        <Property name="Zone"/>

      </Value>

      <Value>

        <Event>

          <XPath>EventDef[@eventType="OutsideZone"]</XPath>

        </Event>

        <Property name="Zone"/>

      </Value>

    </Match>

    <!– Keep only the last 100 events in the view –>

    <MaxCount>100</MaxCount>

  </Properties>

</ViewDef>

 

Situation

A situation describes an interesting combination of multiple events as they occur over time.

The situation detection capability of Reakt have the following features.

1. Declarative – Complex situations are described using a high level declarative approach, avoiding complex procedural solutions using traditional programming languages.
2. Temporal support – Support for describing temporal relationships between events. Timing restrictions such as deadlines for detection can easily be added.
3. Extensible – The situation detector is designed to be extended with new task specific detector nodes.
4. Monitoring – The detector is designed to be monitored and thus the progress of situation detection can easily be tracked.

 

As a situation describes an interesting combination of multiple events as they occur over time, it must start somewhere. A situation starts developing when a specific initiator event occurs. This is the first event in what could possibly be a fully developed situation consisting of multiple events. This also means that a situation has an exact point in time where is starts developing, namely the timestamp of the initiator event.

The situation develops over time and can either be detected at some point in time or automatically determined to be undetectable. A situation can develop over long periods of time and consist of a large number of contributing events. The ruleCore CEP Server provides a persistent rule state allowing rule instances and their situations to survive restarts and unexpected crashes.

The progress of a situation, as it develops, can be followed by submitting a query event to the ruleCore CEP Server about the state of a particular rule’s situation detector. The degree of completion of a situation can be monitored to provide an early warning about a situation which could possibly be detected soon.

For example, detect when a vehicle has been outside a zone for at least five seconds:

<SituationDef name="OutsideZone5s">

  <Detector>

    <After timeframe="00:00:05">

      <Not>

        <EventPickup>

          <XPath>Views/View[@default='true']/Events/Event[@eventType="InsideZone"]<XPath>

        </EventPickup>

      </Not>

    </After>

  </Detector>

</SituationDef>

The above situation example is detected when the innermost event pickup does not pick up any events from the view five seconds after the rule instance was created. If InsideZone events have arrived into the view during those five seconds the situation is not detected.

A situation is defined using a tree with detector nodes. The <Detector> node is the root of the situation detector. Each node can be either true or false depending on its semantics.

One of the central detector nodes is the <EventPickup> node which selects events from the view. It selects events by executing a XPath query against the view. If the query returns any events from the view, the <EventPickup> node becomes true.

The detector is evaluated each time it is possible that changes in the view or temporal conditions in detector nodes can affect the state of the detector.

 

Action

Actions are executed as the last part of a rule’s evaluation in response to a detected situation, or optionally when it is determined that a partially detected situation can’t be detected at any point in the future.

An action is used to create a new event. The new event is called the reaction event and is used to notify external systems about a detected situation. Different events can be created for a detected situation and for undetectable situations. 

The outbound event created by the action is created using a XSLT stylesheet. The stylesheet is executed in the context of the rule and can access all the events that contributed to the detection of the situation. The stylesheet can also access the events present in the view at the time the action is evaluated.

 

<ActionDef name="OutsideZone">   <Event>     <EventDef>       <XPath>EventDef[@eventType="OutsideZone" and @eventClass="user"]</XPath>     </EventDef>     <Body>       <XsltBuilder>         <Stylesheet><![CDATA[           <xsl:stylesheet>             <xsl:template match="child::*">               <base:EventBody>                 <xsl:for-each select="user:Views/user:View[@default='true']/user:Properties">                   <user:VehicleId><xsl:value-of select="descendant::user:MatchedValue[@name='vehicle']"/></user:VehicleId>                   <user:ZoneId><xsl:value-of select="descendant::user:MatchedZone/child::text()"/></user:ZoneId>                 </xsl:for-each>               </base:EventBody>             </xsl:template>           </xsl:stylesheet>]]>
        </Stylesheet>       </XsltBuilder>     </Body>   </Event> </ActionDef>

Reakt is a trademark of Rulecore KB of Sweden.

 

by Declan McCullagh cnet news The Department of Homeland Security’s top cybersecurity official told CNET on Wednesday that the department may eventually extend its Einstein technology, which is designed to detect and prevent electronic attacks, to networks operated by the private sector. The technology was created for federal networks. Greg Schaffer, assistant [...]

In a new development for me, I recently learned that one of the criteria for a “RETE-based rules-engine” to actually be classified as “RETE” is that the software must perform both forward and backward chaining. A well respected rules professional just informed me: If [the rules-engine] is just forward chaining it’s not RETE because the [...]

In Orwellian Event Processing the discussion moved away from my original intent, which was primarily to discuss the vendor-state-of-denial regarding the prior art for processing complex events, and gravitated toward a discussion on the “inefficiencies” of rule-based systems.  I was surprised learn that there are professionals who believe that there is no basis in fact [...]

There were some customer discussions recently on the conceptual relationships between the CEP and BPM and SOA “software stacks”. This coincided with the announcement of the OMG-backed Event Processing Consortium being set up (alongside the merger of the SOA and BPM consortia), events which themselves can be interpreted as that event processing has some special role to play alongside BPM and SOA.

[Disclosure: note TIBCO is not a member of either the OMG's EP or BPMSOA consortia - they seem to be focused on end-user rather than vendor participation - but is an OMG member and participant in standards development. We currently see the EPTS as the main advocacy group across vendors, academics, analysts and end-users, but will be monitoring the progress of the EP Consortium].

So here are a few simplistic patterns on how CEP (event processing) relates to BPM (processes) and SOA (services)…

CEP Pattern Example Diagram 1a. Standalone CEP

This is a bit of a misnomer - you don’t identify event patterns without some intent to use them, and at the very least such a use would be in a BAM type role displaying interesting correlations for some business person - who of course is engaged in some kind of business process, which may or may not be managed by BPM… Monitor a production process to provide an “additional view” or dashboard for the process control manager. 1b. CEP enriching BPM processes and/or  SOA services.

This is the conventional view of CEP, detecting the complex events that are of interest to, and useful in, appropriate processes and services.

Complex events in these cases can be as straightforward as deducing that a deliverable has been completed, or some process truly initiated. Typically the CEP system is transforming source events into business events, for onward use in (the) business processes. Identifying exception events in a business that need to handed to a workflow or case management system for resolution. 2a. CEP monitoring processes and services

This is where the sources of events are the managed processes and services themselves. This process and service monitoring is used to detect exceptions, disparities across systems, and system performance…

Note that effectively this pattern is a combination of patterns 1a and 1b above. Detect when response times exceed some metrics and suggest corrective actions such as reallocating resources. 2b. CEP-based decisions for processes and services

This is where I need to make intelligent decisions for the process and service layers, using the CEP layer as a monitoring, shared decision management component.

Note that effectively this is a slight extension of pattern 2a above. A BPMN “rule activity” sends a decision request to the CEP engine to get a valid decision for a process decision point; the CEP engine monitors the decisions made. 3a. Dynamic process and service control

This is where the events from processes and services, and external services, are combined to select which processes and services are relevant to use for the current context.

In effect, the CEP engine becomes the controlling agent for the business processes and service engine, handling for example dynamic process selection.

Note that this pattern is a further evolution of patterns 2a and 2b. In a complex business process for ever-changing fulfillment problems, CEP-based rules determine which sub-processes are valid based on incoming information. 3b. Embedded processes and services within CEP

The final evolution of the above is when you argue that the functions of the BPM and SOA stacks can be subsumed into the CEP layer.

In reality this is usually only a partial subsumption, as otherwise the centralization of services into just 1 layer could be perceived as contrary to the very idea of SOA! So this covers things like event-based policy implementations being embedded as CEP rules rather than as external services, but alongside some external services such as an operational database. Indeed, one could argue that in this case the CEP event processing agents are themselves really part of the SOA layer, not the other way round! A service gateway controlling access to existing services, but embedding decisions, service policies, and business rules.

So, did I miss anything?

Related posts:

1. Is CEP a Service or a Process?
2. Cue “Process Intelligence”?
3. 2nd Generation SOA = EDA + CEP?

One of the challenges for organisations investigating event processing technologies is “how do we know what is important”? Obviously everyone will have different views on this. Nonetheless, as TIBCO has the most experience in CEP across multiple domains and the largest customer base for a CEP technology tool, it seems to make sense to present a sample evaluation guide that shows the typical considerations and features needed.

First, a few comments. TIBCO deals with enterprises who can have sophisticated requirements around their complex event processing needs. TIBCO customers generally use CEP for operational intelligence applications covering anything from business activity and event monitoring to automated intelligent business processes. Hence TIBCO solutions handle high throughput (not just latency) and payload size (business messages can get very large), whilst providing the fault tolerance and resilience capabilities that 24×7 organizations require. But every class of CEP application has different requirements: for example an algorithmic trading solution might prioritise ultra-low-latency over other features like failover-support and logic and payload complexity. Ergo, for every use case the “example scoring factor” provided below will need to be adjusted to match the feature’s importance to the use case.

Feature Detail Reason Example Scoring
Factor 1. Event Channels A. Specific channel types required Example: JMS, MQ-Series, HTTP… 10 B. Custom channels Example: custom event type for SCADA 3 2. Modelling A. Event models Event metadata, hierarchy, inheritance 10 B. Event object / concept models Object history, hierarchy, inheritance 10 C. State and status of complex events State model and flow 8 3. Processing Elements and Expressions A. EventConditionAction rules Basic event filters, aggregations 10 B. Inference rules Intelligent event processing 6 C. Continuous queries Event streaming aggregations, statistics 6 D. Temporal regular expressions / logic Concise pattern recognition 6 E. Specialist analytics and statistical functions Capabilities for pattern discovery, possibly in conjunction with developer interface (see 7) 6 F. Other context support How to handle context across time, such as GVs etc 6 G. Extensibility How to add custom functions / algorithms / analytics? 6 4. Event augmentation A. Database access JDBC read/update etc 4 B. SOA access Invocation of existing services 6 C. BPM / workflow access Invocation of manual steps 3 5. Stateful processing and management A. Event storage in-process In-memory support 8 B. Event storage out-of-process Low latency cache and/or high latency backing store 8 6. Platform capabilities A. Scalability via distributed processing agents For large processing loads (logic and/or event size and/or rate) 8 B. Scalability via distributed event cache For supporting event patterns over processes, resilience 8 C. Failover / resilience features Hot standby and event handling if agent fails 8 D. Hot deploy / event-based updates Hot deploy of new rules, queries or patterns, or dynamic update through events 8 E. Operations and Management UI and/or reports to monitor performance and/or adapt capabilities 8 F. Performance supporting features Components to minimise latency like preprocessor, algorithms etc 10 G. Deployment environment support Desired or preferred hardware/OS/cloud platforms 8 7. Development Environment A. Standardised GUI tool Example: Eclipse-based 8 B. Verification tools Logic verification such as diagram generation and reports 6 C. Validation tools Unit and bulk test capabilities 6 D. Debugging Local / remote debugging capabilities 6 E. Team features SCCS / project repository / team development 4 8. Business interface A. Business Event / Rule / Decision Management Management (editors, impact analysis) for business-controlled event processing aspects 6 B. User Workflow / control Control and security for business updates to event processing 6 C. Event monitoring dashboard Display of events, and complex events such as KPIs 6 9. Project considerations A. Track record Numbers of Customers / users in same domain 6 B. Availability of SIs and development partners Available assistance if needed 2

One final checklist or scorecard item cannot be added to the list above, but is at least as important as the technology factors. This is the client-vendor relationship, and I was reminded of this when another CEP vendor tweeted recently about how they had been gaining business partly  due to another vendor’s “aggressiveness” in business dealings. So, just like everywhere, caveat emptor…. and maybe add a higher score to 9.A. above!

As usual, comments, feedback and suggestions welcome.

Related posts:

1. Blackboards for Complex Event Processing
2. What is Complex Event Processing? (3/5)
3. What is Complex Event Processing? (1/5)

by Dan Costa, PCmag.com “If Apple’s target is Google’s Android, why is the company going after HTC? Because HTC brought a knife to a gun fight.” Apple’s claim that HTC’s Android phones violate at least 20 of its patents seems, at a glance, like simple corporate maneuvering. Android is, arguably, the iPhone OS’s strongest competitor, [...]

When I first read The Power of Events: An Introduction to Complex Event Processing in Distributed Enterprise Systems by David Luckham I took away three high level ideas: Events are important in business. Events can be processed in a hierarchical way. Rapide is a modeling tool developed at Stanford that can be used to model complex systems. These three [...]

Two events last month showed indicators of a convergence between the analytics world and CEP world.

Firstly Louis Bajuk-Yorgan from TIBCO Spotfire attended the Predictive Analytics World conference in San Francisco. He reported that:

Three key themes showed up multiple times throughout the talks-the growing importance of text mining, the application of net lift modeling to determine the real results of a marketing campaign (ignoring those people who would have bought anyway), and (most interesting to me) the importance of operationalizing predictive analytics.

In his opening keynote speech, Eric Siegel (the conference chair) saw the most important innovation in the field of Predictive Analytics focused on applying predictive analytics to operational decisions (as opposed to more established application areas such as customer churn & product recommendations). In a later talk, James Taylor of Decision Management Solutions (and co-author of the great book “Smart (Enough) Systems”), echoed Eric’s emphasis on operational results, encapsulated in the phrase “Action support, not just decision support.” James advised building an analytic platform that focused on the end game: the need to operationalize analytic decisions.

This is great validation for us, since operationalizing analytics is at the heart of TIBCO’s vision for its combined platform with Spotfire and S+ (as shown in products like Operations Analytics).

Then a week later Andreas Gerst from the TIBCO BusinessEvents team presented at cepconf in Munich, Germany. Andreas presented on CEP and Data Mining, and in particular how both these complement each other for advanced operational intelligence around customer management. Andreas used TIBCO BusinessEvents and TIBCO Spotfire Miner as his example technologies, mentioning techniques like PMML for moving from analytics to real-time event processing technologies.

Related posts:

1. Business Analytics versus CEP
2. Event Operations via Predictive Analytics
3. TIBCO Spotfire - more analytics than BI

Well, after a long period of working on a number of operational projects, I’m going to take a break from writing code and actually do some blogging again. For some this is good news (and I quote from a private note): I’m thankful you’ve started blogging again. For a while there I was afraid [...]

Recently we completed the installation and training of an open source Bayesian classifier to replace a rule-based approach to manage forum spam.  In a nutshell, we found the rule-based approach was highly prone to both false positives and false negatives; however, a statistical approach using a Bayesian approach has turned out to be far superior. [...]

No doubt everyone has read the story where the Lower Merion Township School District remotely enabled webcams on laptops to, in their words, “locate missing laptops”. What on Earth were they thinking? Imagine your daughter has her laptop set up in her room because she did her homework last night. She goes [...]

In a previous post we commented on IBM’s Conceptual Model for Event Processing Systems, but failed to observe that IBM’s rule engine group, Ilog, was noticably absent from its contributor list. So perhaps it was both fortuitous and timely that Ilog’s Daniel Selman recently added another “viewpoint” on the position of event processing (aka the rules viewpoint).

Daniel’s take on the “primary users (sic) of rules technology” are the (use cases for) automating decisions, event responses, processes and inferences. I think these might be better classified by renaming then as decision processing, event processing, and business process processing. But inferencing, which Daniel notes as being a technology to support Artificial Intelligence, is not so much a (user or) use case, but a means of providing knowledge-based reasoning to support any of decisions, processes, etc - “AI” is not (or should not be) a means unto itself.

Standard Event Processing Design Pattern

Indeed, one could probably argue that:

• all business processes are driven by events of some kind, and involve decisions and (re)actions;
• all processes, decisions and event processing are context (i.e. state) driven; however, some processes and algorithms are used in a subordinate fashion to a stateful process (consider a typical decision service that is subordinate to its application server and database layers);
• inferencing can be used to enhance any part of the event-handling process as a form of declarative rule control.

In TIBCO’s experience, an event driven rule engine (like TIBCO BusinessEvents) can be used to provide dynamic business processes, event-driven decisions, and real-time control mechanisms - and often all 3 - exploiting and building on the fundamentals of complex event processing.

So, on the utility of rule engines, we totally agree with Daniel!

Related posts:

1. Events, Rules, Processes and Decisions…
2. BRForum07 - the CEP angle (2) - Intelligent Processes
3. Decisions, Decisions, and the Knights that Say Ni!

Further affirmation that low latency execution is not the only motivator for decisions to deploy the Apama CEP platform is found in a recent quote from Yann L'Huillier, the CTO of Turquoise. Turquoise is the London-based alternative trading venue that... Chris Martins