Internet of Everything and Hybrid Transactional Analytical Processing

Franco Petrucci, Founder and CTO, Decisyon Inc, Franco Petrucci is Chief Technology Officer at Decisyon. He founded Decisyon in 2005 and served as CEO until October 2012. After graduation, Franco fo...  More >>

Data is the organization's lifeblood. Transactional and analytical systems differ in the ability to manipulate data, at an individual record level in the former and at an aggregated level in the latter: this distinction must be overcome. This is because modern systems for the Internet of Everything require a different approach, where the combination of transactional and analytical capabilities allows the connection of people, processes, data and things. New system architectures must be able to provide systems that are both analytical and transactional at the same time.

The Hybrid Transactional Analytical Processes (HTAP) concept, created by analysts at Gartner, mainly focus on the back-end: systems such as SAP Hana, for example, are only able to provide back-end HTAP architecture. Databases that can provide support for both transactional and analytical applications at the same time, using In-memory capabilities are able to manage both transactional data of ERP systems and analytical activities, typical of a Data warehouse or of an analytics tool.

Gartner analysts predicted in their reports that “Data warehouses are evolving and are gradually moving from passive, offline and reporting after-the-fact systems , disconnected from the business processes, to structures that are intimately embedded in the business processes used for Real-Time Decision-Making and collaboration and interact with the operating systems”

The approach is right although the issue is that vendors are tackling only half of the problem by focusing solely on the back end. Instead, the new paradigm (contextuality of transactional and analytical functions) needed for the IoE paradigm requires a dual level: also the front end must follow the same approach. No longer two separate systems for transactions and analytic activities but a single front-end system.

This is necessary to operate the "horizontal paradigm" that is essential in connecting people, processes, data and things and to shift from the silo approach to the new approach required to support IoE systems. The new hybrid transactional/analytical architectures for both back-end and front-end can support this new paradigm.

Vendors are trying to approach this problem with the same architecture that they have worked with for the last 30 years but did not work. Front-end systems must follow the same logic that SAP Hana followed for the back-end: analytical and transactional at the same time, contextually.

This vertical rather than horizontal technological approach has meant that recently, in large organizations, a spiral of disillusion has hit the ERP world and the Enterprise systems in general.

The promise of integrating business processes and the connected organizations has never become a reality. In any case, these “Organizational Islands”, though internally connected, have remained essentially disconnected from the other company processes and therefore loosely integrated with the rest of the organization.

This “vertical” paradigm has permeated modern organizations for a long time and has meant that, instead of obtaining connected entities, it has not been possible to go beyond entities consisting of many different parts, which are not able to integrate and coordinate effectively with each other. Companies, though investing significant resources in large IT projects, have never been able to obtain the expected and desired benefits and, at the same time, have burdened the organization with huge system maintenance costs that sometimes add inefficiency to the processes rather than eliminating it.

This problem does not only concern the ERP world but is more general, affecting the entire application landscape of modern organizations.

This is mainly due to the fact that the current generation of systems, built with this vertical approach is not easily connectable (integrated), intelligent or collaborative or social.

Furthermore, the architecture of these systems is not able to provide the agility and flexibility required to be able to properly connect the various business processes and to easily change the methods with which these systems support the business processes.

These systems have been built over the years with a logic that does not aim at change, meaning they were made "to last" and not made "to change". In addition, they were conceived and designed as Stand Alone tools, with vertical silo logic, i.e. designed to be used alone and not to be able to dynamically connect and interact with other systems in the organization or in the ecosystem the organization belongs to. This lack of horizontal features (cross-process) means that the architecture of the existing Enterprise systems is not able to support the dynamics that the Internet of Everything requires. It is unable to effectively connect people, data, processes and things in a horizontal manner.

The reason for this is due to the way in which, over the last 30 years, organizations have created this vertical rather than horizontal application landscape.

Medium or large organizations often started with the implementation of a MRP, then becoming ERP, to which other systems were joined over the years, such as a production management system, a Supply Chain or Warehouse Management tool, then adding a business process management (BPM) and Business Intelligence tool.

The obvious aim was to support and automate more business processes, though always with a silo logic, “stacking” one application level on another to create what could be called the “Stackholm Syndrome”: a stack of applications able to support individual company processes but too rigid to easily allow changes to the methods with which the processes are supported when they inevitably evolve. In addition, there is no ability to dynamically connect people, data, processes, and things.

These functions, added over the years, have generated several application layers that are comparable to levels of different geological eras: static, inflexible, difficult to change, with an extremely rigid capacity to access the (transactional) data managed.

The overall result is that a user who is in any “Action Point”, i.e. when a decision must be made, he/she cannot easily nor contextually (simultaneously) access the data required, as such data is scattered across N application levels, is not able to access a contextual intelligence (online with the same process, when necessary) that could be generated, perhaps in predictive or prescriptive mode, and that could allow him/her to make the best decision possible, and finally does not have the ability to connect (cooperate) contextually, for example while performing a transaction, with other people who could contribute to making a better decision. All these discontinuities in business processes generate inefficiencies (negatively affecting times and costs) and can lead to wrong decisions, as in this way the activities are carried out much more slowly or even incorrectly, precisely because of wrong decisions.

The situation changes significantly if, alongside the hybrid transactional/analytical management of the back-end, such as that of the SAP Hana type, similar management is applied also to the front-end.

The current generation of business systems will never be able to support the “Smart and Connected Organization”. Therefore it is necessary to rethink the means in which current applications are developed, and support the way people work, and the business processes integrate and change. There is a need for a new paradigm that changes the foundation with which the current systems are developed and connected: it is necessary to move from the current "level" (vertical) paradigm to a new "dimension" (horizontal) paradigm. This is because the organizational world is not flat but is a multidimensional hyperspace where the user must be able to move between the various dimensions that make up its working ecosystem and not be immobilized between static and extremely rigid application layers. Transactional and analytical functions available upon the end user's demand are provided by a new generation of hybrid front-end systems: analytical and transactional at the same time.