Everything Oracle

Introduction

Your organization has just selected OBIEE as its business intelligence platform. So what developer skill sets do you need to install, configure, maintain, and support it? The skill sets that you’ll need are best delineated by looking at three distinct aspects of OBIEE functionality:

Installation and Configuration,

Metadata Mapping, and

End User Administration and Support.

If yours is going to be a substantial OBIEE implementation, with a large number of users working in disparate business areas, and if you are sourcing data from a geographically dispersed collection of heterogeneous data sources, then you’ll need many developers to support each of these functional areas.

Installation and Configuration

Documentation Volume

An OBIEE administrator will need a very good understanding of the OBIEE architecture. Simply discovering where, in the 2,800 odd pages of the OBIEE documentation, the relevant information is to be found, and then assimilating this information into a clear conceptual understanding of the product will take at least a few months of sustained effort.

Product Complexity

OBIEE has a very complex, SOA-based architecture. This would not matter if the complexities were opaque to OBIEE administrators – if they were hidden behind a master administration tool that provided a single point of maintenance. Unfortunately, this is not the case. A typical clustered configuration, one that provides good performance and a high level of resilience, will contain dozens of processes, configuration tools, and configuration files, many of them residing on different servers. And scores of different configuration parameters will need to be correctly set – and more importantly synchronized – across the multiple configuration files. Get anything wrong and OBIEE is likely to “throw a tantrum” and display an unhelpful error message. To debug OBIEE it’s necessary to systematically test the various components working upwards from the data sources at one end to the presentation services at the other in order to pinpoint the source of the problem. While consistency checking is build into some of the configuration tools, what is sadly lacking is a global, network-wide consistency check of the entire OBIEE installation, a consistency check that would state what processes and network connections were down when they should be up, and what sets of parameters were mutually inconsistent with one another.

Product Instability

Added to the not inconsiderable problem of a “difficult to manage product implementation” is the fact that OBIEE is far from stable during reconfiguration: the same action repeated, in what appears to be the same context, can on different occasions lead to very different results. An OBIEE administrator needs sufficient product exposure to determine whether OBIEE misbehaviour is due to a configuration problem that has a definite fix, or is due to a “glitch in the OBIEE matrix”, which can only be addressed by bringing one process down and then back up again, or, on occasion, by rebooting the entire set of OBIEE servers.

Minimizing Scheduled Downtime

If yours is a global operation with business users accessing a central OBIEE platform through SSL-based web connections on a 24 hour basis, then the question of minimizing scheduled downtime will become important (a weakness of the current OBIEE design is that, despite its SOA-based architecture, all its components are not hot-swappable/hot-pluggable at present). An OBIEE administrator will need to devise a strategy to minimize the downtime, maximize the likelihood that the reconfigured system will work correctly, and be sure that there is a plan B that allows a previous good configuration to be quickly reinstated should the reconfigured software fail – for example, by version controlling all the relevant configuration files, testing modified versions on a platform with a near-identical configuration to the production platform, using batch jobs to automatically reconfigure the production servers during downtime, running a set of batch tests to validate the correct functioning of the new configuration from end-to-end, and, finally, re-seeding the local server caches to ensure that KPIs will be met when the production system is brought online again.

Other Technical Specialists

An OBIEE administrator will need to work with others who are familiar with the technical aspects of your IT infrastructure. This will include hardware and network specialists; it will include individuals familiar with your organization’s authentication procedures, such as LDAP and SSO specialists; and it will include individuals familiar with your PKI infrastructure, such as those responsible for generating public/private key pairs, and obtaining certification from external authorities.

Risk Management

You will, of course, need at least two OBIEE administrators, one to provide cover for the other during holidays, periods of illness, or should one of them end his days “under the proverbial bus”. The work load will be very varied. At times, there will be little work to do, and your OBIEE administrators can be deployed on other tasks; at other times, there will be far too much work to do as your administrators try to “shepherd” OBIEE from one stable configuration to another.

Metadata Mapping

As is typical of ad hoc query tools, data is presented to business users as though it were a large collection of columns existing in a single database table. In reality, the source data is likely to exist in disparate databases licensed from disparate software vendors – the legacy of numerous corporate acquisitions. And these databases are likely to be geographical dispersed, on a regional, or even on a, global basis.

The role of the metadata mapper is to map the physical layer, representing the data sources OBIEE sees as it looks out across the network, onto the presentation layer, the semantic model that end users and client applications see when they send queries to the clustered BI Server instances.

A metadata mapper will need to understand (1) the workings of the GUI Administration tool that is used to define the mapping, and (2) the data sources that are being mapped. As far as these data sources are concerned, a metadata mapper will need input from business analysts and physical database designers who, respectively, have a clear understanding of two different aspects of the business data: its semantics and its physical implementation.

Data Semantics

As far as semantics is concerned, the key issue is this, “Is data with similar names at different locations comparable? Does it mean the same thing? Can it be meaningfully merged for the purpose of generating business intelligence reports?” Get this wrong and OBIEE reports can end up comparing “apples with oranges”. In the case of new acquisitions, particularly foreign acquisitions, the merging of business data models into a single consolidated model can take considerable time and effort.

The business analysts who have knowledge of the individual data sources will probably be geographically dispersed, so either your chief business analyst will have to travel to the relevant locations, or business analysts from these locations will have to be seconded to your OBIEE headquarters. Once your chief business analyst understands the business data semantics he can pass this information onto the metadata mappers who will carry out the actual data transformation.

Physical Implementation

But understanding the semantics is only half the problem. OBIEE will collapse in a heap unless the amount of data retrieved per query from each data source is of modest proportions. The key to good OBIEE performance is the use of “summary tables”, and if these are not available then you’ll need to ensure that all the “heavy-duty” calculations are performed efficiently by the source databases. Most modern databases will have some version of query rewrite, in which external queries are directed automatically to a summary table or to an underlying fact table in a manner that is opaque to the client application (Oracle has long had a very elegant solution to this problem using query rewrite supported by materialized views and materialized view logs).

While OBIEE can, in a superficial sense, be seen as a real-time, ETL application – a sort of “just-in-time ETL”, an “ETL-lite” – in that, as in data warehousing ETL, access to heterogeneous data sources is involved, in practice the skill sets required by the different categories of ETL developers are quite different. Data warehousing ETL takes vast quantities of data from OLTP databases at infrequent intervals and summarizes it at its leisure before forwarding the summaries onto production datamarts. OBIEE ETL takes small quantities of data that have already been aggregated using summary tables, and it does so in real time. In the case of data warehousing ETL, the key issues are to minimize the interruption to OLTP sources – by quickly “grabbling hold” of large quantities of raw data, using, say, transportable tablespaces – and to minimize the interruption to production datamarts and the business intelligence applications that they feed – by updating small data sets in datamart tables as fast as possible. The amount of time it takes a data warehouse to “digest its latest meal” is rarely of critical importance. OBIEE ETL, on the other hand, is all about retrieving small quantities of data from disparate sources and integrating it together quickly to meet client application KPIs. Bringing in large quantities of data from heterogeneous data sources is simply not an option for OBIEE. Designers of OBIEE ETL need to focus on partitioning queries into sub-queries so that each data source can return its “own part of the puzzle”, and do so very quickly. If any “heavy-duty” computation is required because summary tables do not exist in a particular database, then this computation must be performed efficiently at source.

The data sources available to OBIEE might include relational databases, such as Oracle, Teradata, or SQL Server; applications, such as SAP; legacy data sources, such as IMS; together with a plethora of flat files in a variety of formats. No one individual is likely to be familiar with all these data sources, so your chief physical database designer will need to liaise with his counterparts in subsidiary companies to understand how the relevant data needed to satisfy OBIEE queries is stored physically. He can then pass the relevant information onto your metadata mappers.

Networking Issues

Once the data has been understood and its physical locations have been identified, the next issue is how to get it to your OBIEE platform. You’ll need to ensure the security of the data while it is in transit; you’ll need to ensure that the appropriate WAN network bandwidth is available given the data volumes that you expect; and you’ll need to assess the reliability of the network – reliability is of particular importance in the case of “federated” or “heterogeneous” queries, since if the line to any one data source is down, then the entire OBIEE application will grind to a halt.

Corporate Politics

All this analysis assumes that the relevant summary tables exist in the relevant databases, but if they don’t then additional work will be required to create them. This requirement will doubtless involve the usual mix of interdepartmental/inter-company politics: if you want IT managers in corporate subsidiaries, especially overseas subsidiaries, to carry out remedial work on their databases to ensure that your OBIEE queries perform efficiently, then you’ll need substantial “corporate clout”, to say nothing of the vexed issue of agreeing which part of the business will pay for the modifications.

OBIEE does provide an alternative to manually creating summary tables at source: summary tables are defined within OBIEE and are then created and populated in remote databases by batch jobs run by the OBIEE Job Manager. This approach should be avoided if at all possible, as it involves moving the problem of data aggregation from the source database, where it belongs, to OBIEE where it most certainly does not. And you can be sure that the IT managers responsible for managing the remote databases will look none too kindly on the prospect of your developers having the write access needed to create physical structures within their databases.

End User Administration and Support

OBIEE presents a number of different faces to the business world. There are the core BI Answers and BI Dashboards components, where business users create ad hoc queries; there is the BI Delivers component, where reports are scheduled to be delivered following certain triggering events to disparate users using disparate delivery mechanisms; there is the BI Office Add-in, which allows queries to be downloaded to MS Excel and PowerPoint for those business users who feel more comfortable using these products; there is BI Publisher which can connect to BI Answers, to the BI Server, or to various databases directly, and which makes of use of templates downloaded to MS Word as part of its somewhat convoluted report production process; and for those of your business users who are on the move there are Briefing Books and Disconnected Analytics. And if you decide than OBIEE should not exist in grand isolation, but needs to be interfaced to other IT infrastructure components, then there are the SOAP/XML/XSD skills needed to use the Web Services API that connects to the BI Presentation Services instances, the ODBC skills needed to connect non-Oracle data formatting tools to the BI Server instances, and the JavaScript API that allows external applications to control the BI Scheduler.

Even when it comes to the more user friendly of these end user products, you’ll need “business savvy” developers who can provide:

Administration, and

End User Training and Support.

Firstly, most of these end user OBIEE products require some element of administration, and you’ll need IT developers to fulfil this role.

Secondly, and more importantly, if your business users are not trained to fully exploit the functionality that OBIEE provides then your investment in the product will have been partially wasted. End user training should not be seen as a poor relation to the more technically demanding OBIEE developer training. Users who understand how to manipulate business data and display it in various ways by means of pivot tables and charts will develop a good understanding of that data, and from that understanding will come better business decisions. The Guided Analytics elements provided by dashboards and by iBots provide an excellent means of offloading many routine checks from managers and executives to OBIEE, which will proactively “hunt” for anomalies that require user intervention. These features can also be used to disseminate good business practice in a non intrusive manner.

The number of developers you’ll need to manage these end user products will depend on the size of your user base and on the diversity of the business areas that are supported by OBIEE. Clearly, apart from a technical understanding of the end user products that is needed for administration purposes, you’ll need developers with a good business understanding and with the capability to provide training to end user groups.

There will, in addition, be a need to liaise with HR to identify those users who need to be trained, and especially those “superusers” who will carry out routine training and ongoing support within their own departments.

Above all, getting the most out of OBIEE will require a re-engineering of your business and an up-skilling of your user base, none of which will happen without an unstinting commitment from your organization’s executive board.

Summary

If you intend to install a non-clustered version of OBIEE for evaluation purposes, then by way of developer support you might just get away with “one man and his dog”. However, if you intend to roll out OBIEE in a large organization and make effective use of all the functionality that “lurks within the beast” then expect to engage the services of dozens of individuals possessed of a wide variety of skill sets. OBIEE probably has a great deal to offer your organization, but to exploit its full potential will require a great deal of hard work.

Everything Oracle

Home

Everything Oracle

Developer	Skill	Sets

Who needs to do what, when